Sep 17 23:10:46.160665 ip-10-0-132-188 systemd[1]: Starting Kubernetes Kubelet...
Sep 17 23:10:49.460560 ip-10-0-132-188 kubenswrapper[2562]: Flag --container-runtime-endpoint has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Sep 17 23:10:49.460560 ip-10-0-132-188 kubenswrapper[2562]: Flag --minimum-container-ttl-duration has been deprecated, Use --eviction-hard or --eviction-soft instead. Will be removed in a future version.
Sep 17 23:10:49.460560 ip-10-0-132-188 kubenswrapper[2562]: Flag --volume-plugin-dir has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Sep 17 23:10:49.460560 ip-10-0-132-188 kubenswrapper[2562]: Flag --pod-infra-container-image has been deprecated, will be removed in 1.35. Image garbage collector will get sandbox image information from CRI.
Sep 17 23:10:49.460560 ip-10-0-132-188 kubenswrapper[2562]: Flag --system-reserved has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Sep 17 23:10:49.461345 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:10:49.460635    2562 server.go:213] "--pod-infra-container-image will not be pruned by the image garbage collector in kubelet and should also be set in the remote runtime"
Sep 17 23:10:49.464420 ip-10-0-132-188 kubenswrapper[2562]: W0917 23:10:49.464401    2562 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Sep 17 23:10:49.464420 ip-10-0-132-188 kubenswrapper[2562]: W0917 23:10:49.464419    2562 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Sep 17 23:10:49.464420 ip-10-0-132-188 kubenswrapper[2562]: W0917 23:10:49.464423    2562 feature_gate.go:328] unrecognized feature gate: OVNObservability
Sep 17 23:10:49.464514 ip-10-0-132-188 kubenswrapper[2562]: W0917 23:10:49.464427    2562 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Sep 17 23:10:49.464514 ip-10-0-132-188 kubenswrapper[2562]: W0917 23:10:49.464430    2562 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Sep 17 23:10:49.464514 ip-10-0-132-188 kubenswrapper[2562]: W0917 23:10:49.464433    2562 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Sep 17 23:10:49.464514 ip-10-0-132-188 kubenswrapper[2562]: W0917 23:10:49.464436    2562 feature_gate.go:328] unrecognized feature gate: Example
Sep 17 23:10:49.464514 ip-10-0-132-188 kubenswrapper[2562]: W0917 23:10:49.464438    2562 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Sep 17 23:10:49.464514 ip-10-0-132-188 kubenswrapper[2562]: W0917 23:10:49.464442    2562 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Sep 17 23:10:49.464514 ip-10-0-132-188 kubenswrapper[2562]: W0917 23:10:49.464444    2562 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Sep 17 23:10:49.464514 ip-10-0-132-188 kubenswrapper[2562]: W0917 23:10:49.464447    2562 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Sep 17 23:10:49.464514 ip-10-0-132-188 kubenswrapper[2562]: W0917 23:10:49.464451    2562 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Sep 17 23:10:49.464514 ip-10-0-132-188 kubenswrapper[2562]: W0917 23:10:49.464453    2562 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Sep 17 23:10:49.464514 ip-10-0-132-188 kubenswrapper[2562]: W0917 23:10:49.464456    2562 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Sep 17 23:10:49.464514 ip-10-0-132-188 kubenswrapper[2562]: W0917 23:10:49.464459    2562 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Sep 17 23:10:49.464514 ip-10-0-132-188 kubenswrapper[2562]: W0917 23:10:49.464462    2562 feature_gate.go:328] unrecognized feature gate: PinnedImages
Sep 17 23:10:49.464514 ip-10-0-132-188 kubenswrapper[2562]: W0917 23:10:49.464465    2562 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Sep 17 23:10:49.464514 ip-10-0-132-188 kubenswrapper[2562]: W0917 23:10:49.464471    2562 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Sep 17 23:10:49.464514 ip-10-0-132-188 kubenswrapper[2562]: W0917 23:10:49.464473    2562 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Sep 17 23:10:49.464514 ip-10-0-132-188 kubenswrapper[2562]: W0917 23:10:49.464476    2562 feature_gate.go:328] unrecognized feature gate: DualReplica
Sep 17 23:10:49.464514 ip-10-0-132-188 kubenswrapper[2562]: W0917 23:10:49.464480    2562 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Sep 17 23:10:49.464514 ip-10-0-132-188 kubenswrapper[2562]: W0917 23:10:49.464484    2562 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Sep 17 23:10:49.464514 ip-10-0-132-188 kubenswrapper[2562]: W0917 23:10:49.464487    2562 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Sep 17 23:10:49.464986 ip-10-0-132-188 kubenswrapper[2562]: W0917 23:10:49.464491    2562 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Sep 17 23:10:49.464986 ip-10-0-132-188 kubenswrapper[2562]: W0917 23:10:49.464494    2562 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Sep 17 23:10:49.464986 ip-10-0-132-188 kubenswrapper[2562]: W0917 23:10:49.464496    2562 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Sep 17 23:10:49.464986 ip-10-0-132-188 kubenswrapper[2562]: W0917 23:10:49.464499    2562 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Sep 17 23:10:49.464986 ip-10-0-132-188 kubenswrapper[2562]: W0917 23:10:49.464502    2562 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Sep 17 23:10:49.464986 ip-10-0-132-188 kubenswrapper[2562]: W0917 23:10:49.464505    2562 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Sep 17 23:10:49.464986 ip-10-0-132-188 kubenswrapper[2562]: W0917 23:10:49.464507    2562 feature_gate.go:328] unrecognized feature gate: EventTTL
Sep 17 23:10:49.464986 ip-10-0-132-188 kubenswrapper[2562]: W0917 23:10:49.464510    2562 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Sep 17 23:10:49.464986 ip-10-0-132-188 kubenswrapper[2562]: W0917 23:10:49.464513    2562 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Sep 17 23:10:49.464986 ip-10-0-132-188 kubenswrapper[2562]: W0917 23:10:49.464515    2562 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Sep 17 23:10:49.464986 ip-10-0-132-188 kubenswrapper[2562]: W0917 23:10:49.464518    2562 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Sep 17 23:10:49.464986 ip-10-0-132-188 kubenswrapper[2562]: W0917 23:10:49.464521    2562 feature_gate.go:328] unrecognized feature gate: SignatureStores
Sep 17 23:10:49.464986 ip-10-0-132-188 kubenswrapper[2562]: W0917 23:10:49.464524    2562 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Sep 17 23:10:49.464986 ip-10-0-132-188 kubenswrapper[2562]: W0917 23:10:49.464527    2562 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Sep 17 23:10:49.464986 ip-10-0-132-188 kubenswrapper[2562]: W0917 23:10:49.464531    2562 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Sep 17 23:10:49.464986 ip-10-0-132-188 kubenswrapper[2562]: W0917 23:10:49.464534    2562 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Sep 17 23:10:49.464986 ip-10-0-132-188 kubenswrapper[2562]: W0917 23:10:49.464536    2562 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Sep 17 23:10:49.464986 ip-10-0-132-188 kubenswrapper[2562]: W0917 23:10:49.464539    2562 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Sep 17 23:10:49.464986 ip-10-0-132-188 kubenswrapper[2562]: W0917 23:10:49.464542    2562 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Sep 17 23:10:49.464986 ip-10-0-132-188 kubenswrapper[2562]: W0917 23:10:49.464545    2562 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Sep 17 23:10:49.465503 ip-10-0-132-188 kubenswrapper[2562]: W0917 23:10:49.464547    2562 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Sep 17 23:10:49.465503 ip-10-0-132-188 kubenswrapper[2562]: W0917 23:10:49.464549    2562 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Sep 17 23:10:49.465503 ip-10-0-132-188 kubenswrapper[2562]: W0917 23:10:49.464552    2562 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Sep 17 23:10:49.465503 ip-10-0-132-188 kubenswrapper[2562]: W0917 23:10:49.464554    2562 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Sep 17 23:10:49.465503 ip-10-0-132-188 kubenswrapper[2562]: W0917 23:10:49.464557    2562 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Sep 17 23:10:49.465503 ip-10-0-132-188 kubenswrapper[2562]: W0917 23:10:49.464559    2562 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Sep 17 23:10:49.465503 ip-10-0-132-188 kubenswrapper[2562]: W0917 23:10:49.464562    2562 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Sep 17 23:10:49.465503 ip-10-0-132-188 kubenswrapper[2562]: W0917 23:10:49.464565    2562 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Sep 17 23:10:49.465503 ip-10-0-132-188 kubenswrapper[2562]: W0917 23:10:49.464567    2562 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Sep 17 23:10:49.465503 ip-10-0-132-188 kubenswrapper[2562]: W0917 23:10:49.464570    2562 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Sep 17 23:10:49.465503 ip-10-0-132-188 kubenswrapper[2562]: W0917 23:10:49.464572    2562 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Sep 17 23:10:49.465503 ip-10-0-132-188 kubenswrapper[2562]: W0917 23:10:49.464575    2562 feature_gate.go:328] unrecognized feature gate: Example2
Sep 17 23:10:49.465503 ip-10-0-132-188 kubenswrapper[2562]: W0917 23:10:49.464578    2562 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Sep 17 23:10:49.465503 ip-10-0-132-188 kubenswrapper[2562]: W0917 23:10:49.464581    2562 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Sep 17 23:10:49.465503 ip-10-0-132-188 kubenswrapper[2562]: W0917 23:10:49.464584    2562 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Sep 17 23:10:49.465503 ip-10-0-132-188 kubenswrapper[2562]: W0917 23:10:49.464587    2562 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Sep 17 23:10:49.465503 ip-10-0-132-188 kubenswrapper[2562]: W0917 23:10:49.464589    2562 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Sep 17 23:10:49.465503 ip-10-0-132-188 kubenswrapper[2562]: W0917 23:10:49.464592    2562 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Sep 17 23:10:49.465503 ip-10-0-132-188 kubenswrapper[2562]: W0917 23:10:49.464596    2562 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Sep 17 23:10:49.465966 ip-10-0-132-188 kubenswrapper[2562]: W0917 23:10:49.464600    2562 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Sep 17 23:10:49.465966 ip-10-0-132-188 kubenswrapper[2562]: W0917 23:10:49.464603    2562 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Sep 17 23:10:49.465966 ip-10-0-132-188 kubenswrapper[2562]: W0917 23:10:49.464607    2562 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Sep 17 23:10:49.465966 ip-10-0-132-188 kubenswrapper[2562]: W0917 23:10:49.464609    2562 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Sep 17 23:10:49.465966 ip-10-0-132-188 kubenswrapper[2562]: W0917 23:10:49.464612    2562 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Sep 17 23:10:49.465966 ip-10-0-132-188 kubenswrapper[2562]: W0917 23:10:49.464615    2562 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Sep 17 23:10:49.465966 ip-10-0-132-188 kubenswrapper[2562]: W0917 23:10:49.464617    2562 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Sep 17 23:10:49.465966 ip-10-0-132-188 kubenswrapper[2562]: W0917 23:10:49.464620    2562 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Sep 17 23:10:49.465966 ip-10-0-132-188 kubenswrapper[2562]: W0917 23:10:49.464623    2562 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Sep 17 23:10:49.465966 ip-10-0-132-188 kubenswrapper[2562]: W0917 23:10:49.464625    2562 feature_gate.go:328] unrecognized feature gate: NewOLM
Sep 17 23:10:49.465966 ip-10-0-132-188 kubenswrapper[2562]: W0917 23:10:49.464628    2562 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Sep 17 23:10:49.465966 ip-10-0-132-188 kubenswrapper[2562]: W0917 23:10:49.464630    2562 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Sep 17 23:10:49.465966 ip-10-0-132-188 kubenswrapper[2562]: W0917 23:10:49.464633    2562 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Sep 17 23:10:49.465966 ip-10-0-132-188 kubenswrapper[2562]: W0917 23:10:49.464635    2562 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Sep 17 23:10:49.465966 ip-10-0-132-188 kubenswrapper[2562]: W0917 23:10:49.464638    2562 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Sep 17 23:10:49.465966 ip-10-0-132-188 kubenswrapper[2562]: W0917 23:10:49.464640    2562 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Sep 17 23:10:49.465966 ip-10-0-132-188 kubenswrapper[2562]: W0917 23:10:49.464644    2562 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Sep 17 23:10:49.465966 ip-10-0-132-188 kubenswrapper[2562]: W0917 23:10:49.464646    2562 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Sep 17 23:10:49.465966 ip-10-0-132-188 kubenswrapper[2562]: W0917 23:10:49.464649    2562 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Sep 17 23:10:49.465966 ip-10-0-132-188 kubenswrapper[2562]: W0917 23:10:49.464651    2562 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Sep 17 23:10:49.466455 ip-10-0-132-188 kubenswrapper[2562]: W0917 23:10:49.464654    2562 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Sep 17 23:10:49.466455 ip-10-0-132-188 kubenswrapper[2562]: W0917 23:10:49.464657    2562 feature_gate.go:328] unrecognized feature gate: OSStreams
Sep 17 23:10:49.466455 ip-10-0-132-188 kubenswrapper[2562]: W0917 23:10:49.464659    2562 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Sep 17 23:10:49.466455 ip-10-0-132-188 kubenswrapper[2562]: W0917 23:10:49.464662    2562 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Sep 17 23:10:49.466455 ip-10-0-132-188 kubenswrapper[2562]: W0917 23:10:49.464664    2562 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Sep 17 23:10:49.466455 ip-10-0-132-188 kubenswrapper[2562]: W0917 23:10:49.464669    2562 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Sep 17 23:10:49.466455 ip-10-0-132-188 kubenswrapper[2562]: W0917 23:10:49.464673    2562 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Sep 17 23:10:49.466455 ip-10-0-132-188 kubenswrapper[2562]: W0917 23:10:49.464676    2562 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Sep 17 23:10:49.466455 ip-10-0-132-188 kubenswrapper[2562]: W0917 23:10:49.464679    2562 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Sep 17 23:10:49.466455 ip-10-0-132-188 kubenswrapper[2562]: W0917 23:10:49.464682    2562 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Sep 17 23:10:49.466455 ip-10-0-132-188 kubenswrapper[2562]: W0917 23:10:49.464685    2562 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Sep 17 23:10:49.466455 ip-10-0-132-188 kubenswrapper[2562]: W0917 23:10:49.464687    2562 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Sep 17 23:10:49.466455 ip-10-0-132-188 kubenswrapper[2562]: W0917 23:10:49.464690    2562 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Sep 17 23:10:49.466455 ip-10-0-132-188 kubenswrapper[2562]: W0917 23:10:49.464692    2562 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Sep 17 23:10:49.466455 ip-10-0-132-188 kubenswrapper[2562]: W0917 23:10:49.464695    2562 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Sep 17 23:10:49.466455 ip-10-0-132-188 kubenswrapper[2562]: W0917 23:10:49.464697    2562 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Sep 17 23:10:49.466455 ip-10-0-132-188 kubenswrapper[2562]: W0917 23:10:49.464700    2562 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Sep 17 23:10:49.466455 ip-10-0-132-188 kubenswrapper[2562]: W0917 23:10:49.465457    2562 feature_gate.go:328] unrecognized feature gate: Example
Sep 17 23:10:49.466455 ip-10-0-132-188 kubenswrapper[2562]: W0917 23:10:49.465465    2562 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Sep 17 23:10:49.466924 ip-10-0-132-188 kubenswrapper[2562]: W0917 23:10:49.465468    2562 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Sep 17 23:10:49.466924 ip-10-0-132-188 kubenswrapper[2562]: W0917 23:10:49.465472    2562 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Sep 17 23:10:49.466924 ip-10-0-132-188 kubenswrapper[2562]: W0917 23:10:49.465475    2562 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Sep 17 23:10:49.466924 ip-10-0-132-188 kubenswrapper[2562]: W0917 23:10:49.465478    2562 feature_gate.go:328] unrecognized feature gate: PinnedImages
Sep 17 23:10:49.466924 ip-10-0-132-188 kubenswrapper[2562]: W0917 23:10:49.465481    2562 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Sep 17 23:10:49.466924 ip-10-0-132-188 kubenswrapper[2562]: W0917 23:10:49.465483    2562 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Sep 17 23:10:49.466924 ip-10-0-132-188 kubenswrapper[2562]: W0917 23:10:49.465486    2562 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Sep 17 23:10:49.466924 ip-10-0-132-188 kubenswrapper[2562]: W0917 23:10:49.465490    2562 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Sep 17 23:10:49.466924 ip-10-0-132-188 kubenswrapper[2562]: W0917 23:10:49.465493    2562 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Sep 17 23:10:49.466924 ip-10-0-132-188 kubenswrapper[2562]: W0917 23:10:49.465495    2562 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Sep 17 23:10:49.466924 ip-10-0-132-188 kubenswrapper[2562]: W0917 23:10:49.465498    2562 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Sep 17 23:10:49.466924 ip-10-0-132-188 kubenswrapper[2562]: W0917 23:10:49.465500    2562 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Sep 17 23:10:49.466924 ip-10-0-132-188 kubenswrapper[2562]: W0917 23:10:49.465503    2562 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Sep 17 23:10:49.466924 ip-10-0-132-188 kubenswrapper[2562]: W0917 23:10:49.465506    2562 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Sep 17 23:10:49.466924 ip-10-0-132-188 kubenswrapper[2562]: W0917 23:10:49.465509    2562 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Sep 17 23:10:49.466924 ip-10-0-132-188 kubenswrapper[2562]: W0917 23:10:49.465512    2562 feature_gate.go:328] unrecognized feature gate: SignatureStores
Sep 17 23:10:49.466924 ip-10-0-132-188 kubenswrapper[2562]: W0917 23:10:49.465515    2562 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Sep 17 23:10:49.466924 ip-10-0-132-188 kubenswrapper[2562]: W0917 23:10:49.465518    2562 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Sep 17 23:10:49.466924 ip-10-0-132-188 kubenswrapper[2562]: W0917 23:10:49.465521    2562 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Sep 17 23:10:49.467422 ip-10-0-132-188 kubenswrapper[2562]: W0917 23:10:49.465524    2562 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Sep 17 23:10:49.467422 ip-10-0-132-188 kubenswrapper[2562]: W0917 23:10:49.465526    2562 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Sep 17 23:10:49.467422 ip-10-0-132-188 kubenswrapper[2562]: W0917 23:10:49.465529    2562 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Sep 17 23:10:49.467422 ip-10-0-132-188 kubenswrapper[2562]: W0917 23:10:49.465532    2562 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Sep 17 23:10:49.467422 ip-10-0-132-188 kubenswrapper[2562]: W0917 23:10:49.465534    2562 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Sep 17 23:10:49.467422 ip-10-0-132-188 kubenswrapper[2562]: W0917 23:10:49.465537    2562 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Sep 17 23:10:49.467422 ip-10-0-132-188 kubenswrapper[2562]: W0917 23:10:49.465540    2562 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Sep 17 23:10:49.467422 ip-10-0-132-188 kubenswrapper[2562]: W0917 23:10:49.465542    2562 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Sep 17 23:10:49.467422 ip-10-0-132-188 kubenswrapper[2562]: W0917 23:10:49.465545    2562 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Sep 17 23:10:49.467422 ip-10-0-132-188 kubenswrapper[2562]: W0917 23:10:49.465548    2562 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Sep 17 23:10:49.467422 ip-10-0-132-188 kubenswrapper[2562]: W0917 23:10:49.465551    2562 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Sep 17 23:10:49.467422 ip-10-0-132-188 kubenswrapper[2562]: W0917 23:10:49.465555    2562 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Sep 17 23:10:49.467422 ip-10-0-132-188 kubenswrapper[2562]: W0917 23:10:49.465559    2562 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Sep 17 23:10:49.467422 ip-10-0-132-188 kubenswrapper[2562]: W0917 23:10:49.465561    2562 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Sep 17 23:10:49.467422 ip-10-0-132-188 kubenswrapper[2562]: W0917 23:10:49.465564    2562 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Sep 17 23:10:49.467422 ip-10-0-132-188 kubenswrapper[2562]: W0917 23:10:49.465569    2562 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Sep 17 23:10:49.467422 ip-10-0-132-188 kubenswrapper[2562]: W0917 23:10:49.465573    2562 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Sep 17 23:10:49.467422 ip-10-0-132-188 kubenswrapper[2562]: W0917 23:10:49.465576    2562 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Sep 17 23:10:49.467422 ip-10-0-132-188 kubenswrapper[2562]: W0917 23:10:49.465579    2562 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Sep 17 23:10:49.467937 ip-10-0-132-188 kubenswrapper[2562]: W0917 23:10:49.465582    2562 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Sep 17 23:10:49.467937 ip-10-0-132-188 kubenswrapper[2562]: W0917 23:10:49.465584    2562 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Sep 17 23:10:49.467937 ip-10-0-132-188 kubenswrapper[2562]: W0917 23:10:49.465587    2562 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Sep 17 23:10:49.467937 ip-10-0-132-188 kubenswrapper[2562]: W0917 23:10:49.465590    2562 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Sep 17 23:10:49.467937 ip-10-0-132-188 kubenswrapper[2562]: W0917 23:10:49.465593    2562 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Sep 17 23:10:49.467937 ip-10-0-132-188 kubenswrapper[2562]: W0917 23:10:49.465595    2562 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Sep 17 23:10:49.467937 ip-10-0-132-188 kubenswrapper[2562]: W0917 23:10:49.465599    2562 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Sep 17 23:10:49.467937 ip-10-0-132-188 kubenswrapper[2562]: W0917 23:10:49.465602    2562 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Sep 17 23:10:49.467937 ip-10-0-132-188 kubenswrapper[2562]: W0917 23:10:49.465606    2562 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Sep 17 23:10:49.467937 ip-10-0-132-188 kubenswrapper[2562]: W0917 23:10:49.465609    2562 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Sep 17 23:10:49.467937 ip-10-0-132-188 kubenswrapper[2562]: W0917 23:10:49.465612    2562 feature_gate.go:328] unrecognized feature gate: OSStreams
Sep 17 23:10:49.467937 ip-10-0-132-188 kubenswrapper[2562]: W0917 23:10:49.465615    2562 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Sep 17 23:10:49.467937 ip-10-0-132-188 kubenswrapper[2562]: W0917 23:10:49.465620    2562 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Sep 17 23:10:49.467937 ip-10-0-132-188 kubenswrapper[2562]: W0917 23:10:49.465623    2562 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Sep 17 23:10:49.467937 ip-10-0-132-188 kubenswrapper[2562]: W0917 23:10:49.465626    2562 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Sep 17 23:10:49.467937 ip-10-0-132-188 kubenswrapper[2562]: W0917 23:10:49.465629    2562 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Sep 17 23:10:49.467937 ip-10-0-132-188 kubenswrapper[2562]: W0917 23:10:49.465632    2562 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Sep 17 23:10:49.467937 ip-10-0-132-188 kubenswrapper[2562]: W0917 23:10:49.465634    2562 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Sep 17 23:10:49.467937 ip-10-0-132-188 kubenswrapper[2562]: W0917 23:10:49.465637    2562 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Sep 17 23:10:49.468425 ip-10-0-132-188 kubenswrapper[2562]: W0917 23:10:49.465639    2562 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Sep 17 23:10:49.468425 ip-10-0-132-188 kubenswrapper[2562]: W0917 23:10:49.465642    2562 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Sep 17 23:10:49.468425 ip-10-0-132-188 kubenswrapper[2562]: W0917 23:10:49.465645    2562 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Sep 17 23:10:49.468425 ip-10-0-132-188 kubenswrapper[2562]: W0917 23:10:49.465647    2562 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Sep 17 23:10:49.468425 ip-10-0-132-188 kubenswrapper[2562]: W0917 23:10:49.465650    2562 feature_gate.go:328] unrecognized feature gate: OVNObservability
Sep 17 23:10:49.468425 ip-10-0-132-188 kubenswrapper[2562]: W0917 23:10:49.465653    2562 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Sep 17 23:10:49.468425 ip-10-0-132-188 kubenswrapper[2562]: W0917 23:10:49.465655    2562 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Sep 17 23:10:49.468425 ip-10-0-132-188 kubenswrapper[2562]: W0917 23:10:49.465658    2562 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Sep 17 23:10:49.468425 ip-10-0-132-188 kubenswrapper[2562]: W0917 23:10:49.465661    2562 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Sep 17 23:10:49.468425 ip-10-0-132-188 kubenswrapper[2562]: W0917 23:10:49.465664    2562 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Sep 17 23:10:49.468425 ip-10-0-132-188 kubenswrapper[2562]: W0917 23:10:49.465667    2562 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Sep 17 23:10:49.468425 ip-10-0-132-188 kubenswrapper[2562]: W0917 23:10:49.465669    2562 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Sep 17 23:10:49.468425 ip-10-0-132-188 kubenswrapper[2562]: W0917 23:10:49.465672    2562 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Sep 17 23:10:49.468425 ip-10-0-132-188 kubenswrapper[2562]: W0917 23:10:49.465674    2562 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Sep 17 23:10:49.468425 ip-10-0-132-188 kubenswrapper[2562]: W0917 23:10:49.465677    2562 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Sep 17 23:10:49.468425 ip-10-0-132-188 kubenswrapper[2562]: W0917 23:10:49.465679    2562 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Sep 17 23:10:49.468425 ip-10-0-132-188 kubenswrapper[2562]: W0917 23:10:49.465682    2562 feature_gate.go:328] unrecognized feature gate: Example2
Sep 17 23:10:49.468425 ip-10-0-132-188 kubenswrapper[2562]: W0917 23:10:49.465684    2562 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Sep 17 23:10:49.468425 ip-10-0-132-188 kubenswrapper[2562]: W0917 23:10:49.465688    2562 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Sep 17 23:10:49.468425 ip-10-0-132-188 kubenswrapper[2562]: W0917 23:10:49.465690    2562 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Sep 17 23:10:49.468941 ip-10-0-132-188 kubenswrapper[2562]: W0917 23:10:49.465693    2562 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Sep 17 23:10:49.468941 ip-10-0-132-188 kubenswrapper[2562]: W0917 23:10:49.465696    2562 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Sep 17 23:10:49.468941 ip-10-0-132-188 kubenswrapper[2562]: W0917 23:10:49.465698    2562 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Sep 17 23:10:49.468941 ip-10-0-132-188 kubenswrapper[2562]: W0917 23:10:49.465701    2562 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Sep 17 23:10:49.468941 ip-10-0-132-188 kubenswrapper[2562]: W0917 23:10:49.465703    2562 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Sep 17 23:10:49.468941 ip-10-0-132-188 kubenswrapper[2562]: W0917 23:10:49.465707    2562 feature_gate.go:328] unrecognized feature gate: EventTTL
Sep 17 23:10:49.468941 ip-10-0-132-188 kubenswrapper[2562]: W0917 23:10:49.465710    2562 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Sep 17 23:10:49.468941 ip-10-0-132-188 kubenswrapper[2562]: W0917 23:10:49.465712    2562 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Sep 17 23:10:49.468941 ip-10-0-132-188 kubenswrapper[2562]: W0917 23:10:49.465715    2562 feature_gate.go:328] unrecognized feature gate: NewOLM
Sep 17 23:10:49.468941 ip-10-0-132-188 kubenswrapper[2562]: W0917 23:10:49.465718    2562 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Sep 17 23:10:49.468941 ip-10-0-132-188 kubenswrapper[2562]: W0917 23:10:49.465720    2562 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Sep 17 23:10:49.468941 ip-10-0-132-188 kubenswrapper[2562]: W0917 23:10:49.465723    2562 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Sep 17 23:10:49.468941 ip-10-0-132-188 kubenswrapper[2562]: W0917 23:10:49.465725    2562 feature_gate.go:328] unrecognized feature gate: DualReplica
Sep 17 23:10:49.468941 ip-10-0-132-188 kubenswrapper[2562]: W0917 23:10:49.465728    2562 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Sep 17 23:10:49.468941 ip-10-0-132-188 kubenswrapper[2562]: W0917 23:10:49.465730    2562 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Sep 17 23:10:49.468941 ip-10-0-132-188 kubenswrapper[2562]: W0917 23:10:49.465734    2562 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Sep 17 23:10:49.468941 ip-10-0-132-188 kubenswrapper[2562]: W0917 23:10:49.465736    2562 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Sep 17 23:10:49.468941 ip-10-0-132-188 kubenswrapper[2562]: W0917 23:10:49.465738    2562 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Sep 17 23:10:49.468941 ip-10-0-132-188 kubenswrapper[2562]: W0917 23:10:49.465741    2562 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Sep 17 23:10:49.468941 ip-10-0-132-188 kubenswrapper[2562]: W0917 23:10:49.465743    2562 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Sep 17 23:10:49.469451 ip-10-0-132-188 kubenswrapper[2562]: W0917 23:10:49.466153    2562 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Sep 17 23:10:49.469451 ip-10-0-132-188 kubenswrapper[2562]: W0917 23:10:49.466158    2562 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Sep 17 23:10:49.469451 ip-10-0-132-188 kubenswrapper[2562]: W0917 23:10:49.466180    2562 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Sep 17 23:10:49.469451 ip-10-0-132-188 kubenswrapper[2562]: W0917 23:10:49.466184    2562 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Sep 17 23:10:49.469451 ip-10-0-132-188 kubenswrapper[2562]: W0917 23:10:49.466186    2562 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Sep 17 23:10:49.469451 ip-10-0-132-188 kubenswrapper[2562]: W0917 23:10:49.466189    2562 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Sep 17 23:10:49.469451 ip-10-0-132-188 kubenswrapper[2562]: W0917 23:10:49.466192    2562 feature_gate.go:328] unrecognized feature gate: OSStreams
Sep 17 23:10:49.469451 ip-10-0-132-188 kubenswrapper[2562]: W0917 23:10:49.466195    2562 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Sep 17 23:10:49.469451 ip-10-0-132-188 kubenswrapper[2562]: W0917 23:10:49.466197    2562 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Sep 17 23:10:49.469451 ip-10-0-132-188 kubenswrapper[2562]: W0917 23:10:49.466200    2562 feature_gate.go:328] unrecognized feature gate: NewOLM
Sep 17 23:10:49.469451 ip-10-0-132-188 kubenswrapper[2562]: W0917 23:10:49.466202    2562 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Sep 17 23:10:49.469451 ip-10-0-132-188 kubenswrapper[2562]: W0917 23:10:49.466205    2562 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Sep 17 23:10:49.469451 ip-10-0-132-188 kubenswrapper[2562]: W0917 23:10:49.466208    2562 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Sep 17 23:10:49.469451 ip-10-0-132-188 kubenswrapper[2562]: W0917 23:10:49.466211    2562 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Sep 17 23:10:49.469451 ip-10-0-132-188 kubenswrapper[2562]: W0917 23:10:49.466214    2562 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Sep 17 23:10:49.469451 ip-10-0-132-188 kubenswrapper[2562]: W0917 23:10:49.466217    2562 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Sep 17 23:10:49.469451 ip-10-0-132-188 kubenswrapper[2562]: W0917 23:10:49.466219    2562 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Sep 17 23:10:49.469451 ip-10-0-132-188 kubenswrapper[2562]: W0917 23:10:49.466222    2562 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Sep 17 23:10:49.469451 ip-10-0-132-188 kubenswrapper[2562]: W0917 23:10:49.466225    2562 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Sep 17 23:10:49.469451 ip-10-0-132-188 kubenswrapper[2562]: W0917 23:10:49.466228    2562 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Sep 17 23:10:49.469962 ip-10-0-132-188 kubenswrapper[2562]: W0917 23:10:49.466231    2562 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Sep 17 23:10:49.469962 ip-10-0-132-188 kubenswrapper[2562]: W0917 23:10:49.466235    2562 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Sep 17 23:10:49.469962 ip-10-0-132-188 kubenswrapper[2562]: W0917 23:10:49.466238    2562 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Sep 17 23:10:49.469962 ip-10-0-132-188 kubenswrapper[2562]: W0917 23:10:49.466241    2562 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Sep 17 23:10:49.469962 ip-10-0-132-188 kubenswrapper[2562]: W0917 23:10:49.466243    2562 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Sep 17 23:10:49.469962 ip-10-0-132-188 kubenswrapper[2562]: W0917 23:10:49.466246    2562 feature_gate.go:328] unrecognized feature gate: OVNObservability
Sep 17 23:10:49.469962 ip-10-0-132-188 kubenswrapper[2562]: W0917 23:10:49.466249    2562 feature_gate.go:328] unrecognized feature gate: Example2
Sep 17 23:10:49.469962 ip-10-0-132-188 kubenswrapper[2562]: W0917 23:10:49.466251    2562 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Sep 17 23:10:49.469962 ip-10-0-132-188 kubenswrapper[2562]: W0917 23:10:49.466254    2562 feature_gate.go:328] unrecognized feature gate: Example
Sep 17 23:10:49.469962 ip-10-0-132-188 kubenswrapper[2562]: W0917 23:10:49.466257    2562 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Sep 17 23:10:49.469962 ip-10-0-132-188 kubenswrapper[2562]: W0917 23:10:49.466259    2562 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Sep 17 23:10:49.469962 ip-10-0-132-188 kubenswrapper[2562]: W0917 23:10:49.466261    2562 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Sep 17 23:10:49.469962 ip-10-0-132-188 kubenswrapper[2562]: W0917 23:10:49.466264    2562 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Sep 17 23:10:49.469962 ip-10-0-132-188 kubenswrapper[2562]: W0917 23:10:49.466266    2562 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Sep 17 23:10:49.469962 ip-10-0-132-188 kubenswrapper[2562]: W0917 23:10:49.466269    2562 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Sep 17 23:10:49.469962 ip-10-0-132-188 kubenswrapper[2562]: W0917 23:10:49.466272    2562 feature_gate.go:328] unrecognized feature gate: DualReplica
Sep 17 23:10:49.469962 ip-10-0-132-188 kubenswrapper[2562]: W0917 23:10:49.466275    2562 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Sep 17 23:10:49.469962 ip-10-0-132-188 kubenswrapper[2562]: W0917 23:10:49.466277    2562 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Sep 17 23:10:49.469962 ip-10-0-132-188 kubenswrapper[2562]: W0917 23:10:49.466279    2562 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Sep 17 23:10:49.469962 ip-10-0-132-188 kubenswrapper[2562]: W0917 23:10:49.466282    2562 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Sep 17 23:10:49.470467 ip-10-0-132-188 kubenswrapper[2562]: W0917 23:10:49.466285    2562 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Sep 17 23:10:49.470467 ip-10-0-132-188 kubenswrapper[2562]: W0917 23:10:49.466287    2562 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Sep 17 23:10:49.470467 ip-10-0-132-188 kubenswrapper[2562]: W0917 23:10:49.466290    2562 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Sep 17 23:10:49.470467 ip-10-0-132-188 kubenswrapper[2562]: W0917 23:10:49.466292    2562 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Sep 17 23:10:49.470467 ip-10-0-132-188 kubenswrapper[2562]: W0917 23:10:49.466295    2562 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Sep 17 23:10:49.470467 ip-10-0-132-188 kubenswrapper[2562]: W0917 23:10:49.466298    2562 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Sep 17 23:10:49.470467 ip-10-0-132-188 kubenswrapper[2562]: W0917 23:10:49.466300    2562 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Sep 17 23:10:49.470467 ip-10-0-132-188 kubenswrapper[2562]: W0917 23:10:49.466303    2562 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Sep 17 23:10:49.470467 ip-10-0-132-188 kubenswrapper[2562]: W0917 23:10:49.466305    2562 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Sep 17 23:10:49.470467 ip-10-0-132-188 kubenswrapper[2562]: W0917 23:10:49.466308    2562 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Sep 17 23:10:49.470467 ip-10-0-132-188 kubenswrapper[2562]: W0917 23:10:49.466311    2562 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Sep 17 23:10:49.470467 ip-10-0-132-188 kubenswrapper[2562]: W0917 23:10:49.466313    2562 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Sep 17 23:10:49.470467 ip-10-0-132-188 kubenswrapper[2562]: W0917 23:10:49.466317    2562 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Sep 17 23:10:49.470467 ip-10-0-132-188 kubenswrapper[2562]: W0917 23:10:49.466321    2562 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Sep 17 23:10:49.470467 ip-10-0-132-188 kubenswrapper[2562]: W0917 23:10:49.466323    2562 feature_gate.go:328] unrecognized feature gate: EventTTL
Sep 17 23:10:49.470467 ip-10-0-132-188 kubenswrapper[2562]: W0917 23:10:49.466326    2562 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Sep 17 23:10:49.470467 ip-10-0-132-188 kubenswrapper[2562]: W0917 23:10:49.466329    2562 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Sep 17 23:10:49.470467 ip-10-0-132-188 kubenswrapper[2562]: W0917 23:10:49.466331    2562 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Sep 17 23:10:49.470467 ip-10-0-132-188 kubenswrapper[2562]: W0917 23:10:49.466334    2562 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Sep 17 23:10:49.470938 ip-10-0-132-188 kubenswrapper[2562]: W0917 23:10:49.466337    2562 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Sep 17 23:10:49.470938 ip-10-0-132-188 kubenswrapper[2562]: W0917 23:10:49.466339    2562 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Sep 17 23:10:49.470938 ip-10-0-132-188 kubenswrapper[2562]: W0917 23:10:49.466342    2562 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Sep 17 23:10:49.470938 ip-10-0-132-188 kubenswrapper[2562]: W0917 23:10:49.466345    2562 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Sep 17 23:10:49.470938 ip-10-0-132-188 kubenswrapper[2562]: W0917 23:10:49.466348    2562 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Sep 17 23:10:49.470938 ip-10-0-132-188 kubenswrapper[2562]: W0917 23:10:49.466350    2562 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Sep 17 23:10:49.470938 ip-10-0-132-188 kubenswrapper[2562]: W0917 23:10:49.466353    2562 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Sep 17 23:10:49.470938 ip-10-0-132-188 kubenswrapper[2562]: W0917 23:10:49.466355    2562 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Sep 17 23:10:49.470938 ip-10-0-132-188 kubenswrapper[2562]: W0917 23:10:49.466358    2562 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Sep 17 23:10:49.470938 ip-10-0-132-188 kubenswrapper[2562]: W0917 23:10:49.466361    2562 feature_gate.go:328] unrecognized feature gate: PinnedImages
Sep 17 23:10:49.470938 ip-10-0-132-188 kubenswrapper[2562]: W0917 23:10:49.466363    2562 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Sep 17 23:10:49.470938 ip-10-0-132-188 kubenswrapper[2562]: W0917 23:10:49.466366    2562 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Sep 17 23:10:49.470938 ip-10-0-132-188 kubenswrapper[2562]: W0917 23:10:49.466369    2562 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Sep 17 23:10:49.470938 ip-10-0-132-188 kubenswrapper[2562]: W0917 23:10:49.466372    2562 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Sep 17 23:10:49.470938 ip-10-0-132-188 kubenswrapper[2562]: W0917 23:10:49.466374    2562 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Sep 17 23:10:49.470938 ip-10-0-132-188 kubenswrapper[2562]: W0917 23:10:49.466377    2562 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Sep 17 23:10:49.470938 ip-10-0-132-188 kubenswrapper[2562]: W0917 23:10:49.466379    2562 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Sep 17 23:10:49.470938 ip-10-0-132-188 kubenswrapper[2562]: W0917 23:10:49.466383    2562 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Sep 17 23:10:49.470938 ip-10-0-132-188 kubenswrapper[2562]: W0917 23:10:49.466387    2562 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Sep 17 23:10:49.471452 ip-10-0-132-188 kubenswrapper[2562]: W0917 23:10:49.466390    2562 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Sep 17 23:10:49.471452 ip-10-0-132-188 kubenswrapper[2562]: W0917 23:10:49.466393    2562 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Sep 17 23:10:49.471452 ip-10-0-132-188 kubenswrapper[2562]: W0917 23:10:49.466396    2562 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Sep 17 23:10:49.471452 ip-10-0-132-188 kubenswrapper[2562]: W0917 23:10:49.466403    2562 feature_gate.go:328] unrecognized feature gate: SignatureStores
Sep 17 23:10:49.471452 ip-10-0-132-188 kubenswrapper[2562]: W0917 23:10:49.466406    2562 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Sep 17 23:10:49.471452 ip-10-0-132-188 kubenswrapper[2562]: W0917 23:10:49.466409    2562 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Sep 17 23:10:49.471452 ip-10-0-132-188 kubenswrapper[2562]: W0917 23:10:49.466412    2562 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Sep 17 23:10:49.471452 ip-10-0-132-188 kubenswrapper[2562]: W0917 23:10:49.466414    2562 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Sep 17 23:10:49.471452 ip-10-0-132-188 kubenswrapper[2562]: W0917 23:10:49.466417    2562 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Sep 17 23:10:49.471452 ip-10-0-132-188 kubenswrapper[2562]: W0917 23:10:49.466420    2562 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Sep 17 23:10:49.471452 ip-10-0-132-188 kubenswrapper[2562]: W0917 23:10:49.466423    2562 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Sep 17 23:10:49.471452 ip-10-0-132-188 kubenswrapper[2562]: W0917 23:10:49.466425    2562 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Sep 17 23:10:49.471452 ip-10-0-132-188 kubenswrapper[2562]: W0917 23:10:49.466428    2562 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Sep 17 23:10:49.471452 ip-10-0-132-188 kubenswrapper[2562]: W0917 23:10:49.466430    2562 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Sep 17 23:10:49.471452 ip-10-0-132-188 kubenswrapper[2562]: W0917 23:10:49.466433    2562 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Sep 17 23:10:49.471452 ip-10-0-132-188 kubenswrapper[2562]: W0917 23:10:49.466435    2562 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Sep 17 23:10:49.471452 ip-10-0-132-188 kubenswrapper[2562]: W0917 23:10:49.466438    2562 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Sep 17 23:10:49.471452 ip-10-0-132-188 kubenswrapper[2562]: W0917 23:10:49.466440    2562 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Sep 17 23:10:49.471452 ip-10-0-132-188 kubenswrapper[2562]: W0917 23:10:49.466442    2562 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Sep 17 23:10:49.471452 ip-10-0-132-188 kubenswrapper[2562]: W0917 23:10:49.466445    2562 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Sep 17 23:10:49.471937 ip-10-0-132-188 kubenswrapper[2562]: W0917 23:10:49.466447    2562 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Sep 17 23:10:49.471937 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:10:49.466525    2562 flags.go:64] FLAG: --address="0.0.0.0"
Sep 17 23:10:49.471937 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:10:49.466569    2562 flags.go:64] FLAG: --allowed-unsafe-sysctls="[]"
Sep 17 23:10:49.471937 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:10:49.466577    2562 flags.go:64] FLAG: --anonymous-auth="true"
Sep 17 23:10:49.471937 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:10:49.466582    2562 flags.go:64] FLAG: --application-metrics-count-limit="100"
Sep 17 23:10:49.471937 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:10:49.466586    2562 flags.go:64] FLAG: --authentication-token-webhook="false"
Sep 17 23:10:49.471937 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:10:49.466590    2562 flags.go:64] FLAG: --authentication-token-webhook-cache-ttl="2m0s"
Sep 17 23:10:49.471937 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:10:49.466594    2562 flags.go:64] FLAG: --authorization-mode="AlwaysAllow"
Sep 17 23:10:49.471937 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:10:49.466599    2562 flags.go:64] FLAG: --authorization-webhook-cache-authorized-ttl="5m0s"
Sep 17 23:10:49.471937 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:10:49.466602    2562 flags.go:64] FLAG: --authorization-webhook-cache-unauthorized-ttl="30s"
Sep 17 23:10:49.471937 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:10:49.466605    2562 flags.go:64] FLAG: --boot-id-file="/proc/sys/kernel/random/boot_id"
Sep 17 23:10:49.471937 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:10:49.466609    2562 flags.go:64] FLAG: --bootstrap-kubeconfig="/etc/kubernetes/kubeconfig"
Sep 17 23:10:49.471937 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:10:49.466613    2562 flags.go:64] FLAG: --cert-dir="/var/lib/kubelet/pki"
Sep 17 23:10:49.471937 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:10:49.466616    2562 flags.go:64] FLAG: --cgroup-driver="cgroupfs"
Sep 17 23:10:49.471937 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:10:49.466619    2562 flags.go:64] FLAG: --cgroup-root=""
Sep 17 23:10:49.471937 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:10:49.466625    2562 flags.go:64] FLAG: --cgroups-per-qos="true"
Sep 17 23:10:49.471937 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:10:49.466628    2562 flags.go:64] FLAG: --client-ca-file=""
Sep 17 23:10:49.471937 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:10:49.466631    2562 flags.go:64] FLAG: --cloud-provider="external"
Sep 17 23:10:49.471937 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:10:49.466634    2562 flags.go:64] FLAG: --cluster-dns="[]"
Sep 17 23:10:49.471937 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:10:49.466639    2562 flags.go:64] FLAG: --cluster-domain=""
Sep 17 23:10:49.471937 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:10:49.466642    2562 flags.go:64] FLAG: --config="/etc/kubernetes/kubelet.conf"
Sep 17 23:10:49.471937 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:10:49.466645    2562 flags.go:64] FLAG: --config-dir="/etc/openshift/kubelet.conf.d"
Sep 17 23:10:49.471937 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:10:49.466648    2562 flags.go:64] FLAG: --container-hints="/etc/cadvisor/container_hints.json"
Sep 17 23:10:49.471937 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:10:49.466651    2562 flags.go:64] FLAG: --container-log-max-files="5"
Sep 17 23:10:49.472528 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:10:49.466655    2562 flags.go:64] FLAG: --container-log-max-size="10Mi"
Sep 17 23:10:49.472528 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:10:49.466659    2562 flags.go:64] FLAG: --container-runtime-endpoint="/var/run/crio/crio.sock"
Sep 17 23:10:49.472528 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:10:49.466662    2562 flags.go:64] FLAG: --containerd="/run/containerd/containerd.sock"
Sep 17 23:10:49.472528 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:10:49.466666    2562 flags.go:64] FLAG: --containerd-namespace="k8s.io"
Sep 17 23:10:49.472528 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:10:49.466669    2562 flags.go:64] FLAG: --contention-profiling="false"
Sep 17 23:10:49.472528 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:10:49.466679    2562 flags.go:64] FLAG: --cpu-cfs-quota="true"
Sep 17 23:10:49.472528 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:10:49.466683    2562 flags.go:64] FLAG: --cpu-cfs-quota-period="100ms"
Sep 17 23:10:49.472528 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:10:49.466686    2562 flags.go:64] FLAG: --cpu-manager-policy="none"
Sep 17 23:10:49.472528 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:10:49.466689    2562 flags.go:64] FLAG: --cpu-manager-policy-options=""
Sep 17 23:10:49.472528 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:10:49.466693    2562 flags.go:64] FLAG: --cpu-manager-reconcile-period="10s"
Sep 17 23:10:49.472528 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:10:49.466697    2562 flags.go:64] FLAG: --enable-controller-attach-detach="true"
Sep 17 23:10:49.472528 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:10:49.466700    2562 flags.go:64] FLAG: --enable-debugging-handlers="true"
Sep 17 23:10:49.472528 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:10:49.466704    2562 flags.go:64] FLAG: --enable-load-reader="false"
Sep 17 23:10:49.472528 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:10:49.466707    2562 flags.go:64] FLAG: --enable-server="true"
Sep 17 23:10:49.472528 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:10:49.466710    2562 flags.go:64] FLAG: --enforce-node-allocatable="[pods]"
Sep 17 23:10:49.472528 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:10:49.466715    2562 flags.go:64] FLAG: --event-burst="100"
Sep 17 23:10:49.472528 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:10:49.466718    2562 flags.go:64] FLAG: --event-qps="50"
Sep 17 23:10:49.472528 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:10:49.466722    2562 flags.go:64] FLAG: --event-storage-age-limit="default=0"
Sep 17 23:10:49.472528 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:10:49.466725    2562 flags.go:64] FLAG: --event-storage-event-limit="default=0"
Sep 17 23:10:49.472528 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:10:49.466728    2562 flags.go:64] FLAG: --eviction-hard=""
Sep 17 23:10:49.472528 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:10:49.466731    2562 flags.go:64] FLAG: --eviction-max-pod-grace-period="0"
Sep 17 23:10:49.472528 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:10:49.466734    2562 flags.go:64] FLAG: --eviction-minimum-reclaim=""
Sep 17 23:10:49.472528 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:10:49.466738    2562 flags.go:64] FLAG: --eviction-pressure-transition-period="5m0s"
Sep 17 23:10:49.472528 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:10:49.466743    2562 flags.go:64] FLAG: --eviction-soft=""
Sep 17 23:10:49.472528 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:10:49.466745    2562 flags.go:64] FLAG: --eviction-soft-grace-period=""
Sep 17 23:10:49.473180 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:10:49.466748    2562 flags.go:64] FLAG: --exit-on-lock-contention="false"
Sep 17 23:10:49.473180 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:10:49.466751    2562 flags.go:64] FLAG: --experimental-allocatable-ignore-eviction="false"
Sep 17 23:10:49.473180 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:10:49.466754    2562 flags.go:64] FLAG: --experimental-mounter-path=""
Sep 17 23:10:49.473180 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:10:49.466757    2562 flags.go:64] FLAG: --fail-cgroupv1="false"
Sep 17 23:10:49.473180 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:10:49.466760    2562 flags.go:64] FLAG: --fail-swap-on="true"
Sep 17 23:10:49.473180 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:10:49.466763    2562 flags.go:64] FLAG: --feature-gates=""
Sep 17 23:10:49.473180 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:10:49.466767    2562 flags.go:64] FLAG: --file-check-frequency="20s"
Sep 17 23:10:49.473180 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:10:49.466770    2562 flags.go:64] FLAG: --global-housekeeping-interval="1m0s"
Sep 17 23:10:49.473180 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:10:49.466773    2562 flags.go:64] FLAG: --hairpin-mode="promiscuous-bridge"
Sep 17 23:10:49.473180 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:10:49.466776    2562 flags.go:64] FLAG: --healthz-bind-address="127.0.0.1"
Sep 17 23:10:49.473180 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:10:49.466779    2562 flags.go:64] FLAG: --healthz-port="10248"
Sep 17 23:10:49.473180 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:10:49.466782    2562 flags.go:64] FLAG: --help="false"
Sep 17 23:10:49.473180 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:10:49.466785    2562 flags.go:64] FLAG: --hostname-override="ip-10-0-132-188.ec2.internal"
Sep 17 23:10:49.473180 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:10:49.466788    2562 flags.go:64] FLAG: --housekeeping-interval="10s"
Sep 17 23:10:49.473180 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:10:49.466791    2562 flags.go:64] FLAG: --http-check-frequency="20s"
Sep 17 23:10:49.473180 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:10:49.466794    2562 flags.go:64] FLAG: --image-credential-provider-bin-dir="/usr/libexec/kubelet-image-credential-provider-plugins"
Sep 17 23:10:49.473180 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:10:49.466797    2562 flags.go:64] FLAG: --image-credential-provider-config="/etc/kubernetes/credential-providers/ecr-credential-provider.yaml"
Sep 17 23:10:49.473180 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:10:49.466801    2562 flags.go:64] FLAG: --image-gc-high-threshold="85"
Sep 17 23:10:49.473180 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:10:49.466804    2562 flags.go:64] FLAG: --image-gc-low-threshold="80"
Sep 17 23:10:49.473180 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:10:49.466807    2562 flags.go:64] FLAG: --image-service-endpoint=""
Sep 17 23:10:49.473180 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:10:49.466811    2562 flags.go:64] FLAG: --kernel-memcg-notification="false"
Sep 17 23:10:49.473180 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:10:49.466814    2562 flags.go:64] FLAG: --kube-api-burst="100"
Sep 17 23:10:49.473180 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:10:49.466817    2562 flags.go:64] FLAG: --kube-api-content-type="application/vnd.kubernetes.protobuf"
Sep 17 23:10:49.473759 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:10:49.466820    2562 flags.go:64] FLAG: --kube-api-qps="50"
Sep 17 23:10:49.473759 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:10:49.466823    2562 flags.go:64] FLAG: --kube-reserved=""
Sep 17 23:10:49.473759 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:10:49.466826    2562 flags.go:64] FLAG: --kube-reserved-cgroup=""
Sep 17 23:10:49.473759 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:10:49.466829    2562 flags.go:64] FLAG: --kubeconfig="/var/lib/kubelet/kubeconfig"
Sep 17 23:10:49.473759 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:10:49.466832    2562 flags.go:64] FLAG: --kubelet-cgroups=""
Sep 17 23:10:49.473759 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:10:49.466835    2562 flags.go:64] FLAG: --local-storage-capacity-isolation="true"
Sep 17 23:10:49.473759 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:10:49.466838    2562 flags.go:64] FLAG: --lock-file=""
Sep 17 23:10:49.473759 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:10:49.466842    2562 flags.go:64] FLAG: --log-cadvisor-usage="false"
Sep 17 23:10:49.473759 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:10:49.466845    2562 flags.go:64] FLAG: --log-flush-frequency="5s"
Sep 17 23:10:49.473759 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:10:49.466848    2562 flags.go:64] FLAG: --log-json-info-buffer-size="0"
Sep 17 23:10:49.473759 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:10:49.466854    2562 flags.go:64] FLAG: --log-json-split-stream="false"
Sep 17 23:10:49.473759 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:10:49.466857    2562 flags.go:64] FLAG: --log-text-info-buffer-size="0"
Sep 17 23:10:49.473759 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:10:49.466860    2562 flags.go:64] FLAG: --log-text-split-stream="false"
Sep 17 23:10:49.473759 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:10:49.466862    2562 flags.go:64] FLAG: --logging-format="text"
Sep 17 23:10:49.473759 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:10:49.466865    2562 flags.go:64] FLAG: --machine-id-file="/etc/machine-id,/var/lib/dbus/machine-id"
Sep 17 23:10:49.473759 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:10:49.466869    2562 flags.go:64] FLAG: --make-iptables-util-chains="true"
Sep 17 23:10:49.473759 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:10:49.466872    2562 flags.go:64] FLAG: --manifest-url=""
Sep 17 23:10:49.473759 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:10:49.466874    2562 flags.go:64] FLAG: --manifest-url-header=""
Sep 17 23:10:49.473759 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:10:49.466879    2562 flags.go:64] FLAG: --max-housekeeping-interval="15s"
Sep 17 23:10:49.473759 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:10:49.466882    2562 flags.go:64] FLAG: --max-open-files="1000000"
Sep 17 23:10:49.473759 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:10:49.466886    2562 flags.go:64] FLAG: --max-pods="110"
Sep 17 23:10:49.473759 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:10:49.466889    2562 flags.go:64] FLAG: --maximum-dead-containers="-1"
Sep 17 23:10:49.473759 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:10:49.466892    2562 flags.go:64] FLAG: --maximum-dead-containers-per-container="1"
Sep 17 23:10:49.473759 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:10:49.466896    2562 flags.go:64] FLAG: --memory-manager-policy="None"
Sep 17 23:10:49.473759 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:10:49.466898    2562 flags.go:64] FLAG: --minimum-container-ttl-duration="6m0s"
Sep 17 23:10:49.474398 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:10:49.466901    2562 flags.go:64] FLAG: --minimum-image-ttl-duration="2m0s"
Sep 17 23:10:49.474398 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:10:49.466904    2562 flags.go:64] FLAG: --node-ip="0.0.0.0"
Sep 17 23:10:49.474398 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:10:49.466908    2562 flags.go:64] FLAG: --node-labels="node-role.kubernetes.io/worker=,node.openshift.io/os_id=rhel"
Sep 17 23:10:49.474398 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:10:49.466916    2562 flags.go:64] FLAG: --node-status-max-images="50"
Sep 17 23:10:49.474398 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:10:49.466919    2562 flags.go:64] FLAG: --node-status-update-frequency="10s"
Sep 17 23:10:49.474398 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:10:49.466923    2562 flags.go:64] FLAG: --oom-score-adj="-999"
Sep 17 23:10:49.474398 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:10:49.466926    2562 flags.go:64] FLAG: --pod-cidr=""
Sep 17 23:10:49.474398 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:10:49.466929    2562 flags.go:64] FLAG: --pod-infra-container-image="quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:a6abe77af61c5f2e3985a83e7d43538aefc38b1b0444127da14ee0e8cf8f76fc"
Sep 17 23:10:49.474398 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:10:49.466935    2562 flags.go:64] FLAG: --pod-manifest-path=""
Sep 17 23:10:49.474398 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:10:49.466938    2562 flags.go:64] FLAG: --pod-max-pids="-1"
Sep 17 23:10:49.474398 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:10:49.466941    2562 flags.go:64] FLAG: --pods-per-core="0"
Sep 17 23:10:49.474398 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:10:49.466944    2562 flags.go:64] FLAG: --port="10250"
Sep 17 23:10:49.474398 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:10:49.466947    2562 flags.go:64] FLAG: --protect-kernel-defaults="false"
Sep 17 23:10:49.474398 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:10:49.466950    2562 flags.go:64] FLAG: --provider-id="aws:///us-east-1a/i-006afd8bd56ed5d9f"
Sep 17 23:10:49.474398 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:10:49.466954    2562 flags.go:64] FLAG: --qos-reserved=""
Sep 17 23:10:49.474398 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:10:49.466959    2562 flags.go:64] FLAG: --read-only-port="10255"
Sep 17 23:10:49.474398 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:10:49.466962    2562 flags.go:64] FLAG: --register-node="true"
Sep 17 23:10:49.474398 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:10:49.466965    2562 flags.go:64] FLAG: --register-with-taints=""
Sep 17 23:10:49.474398 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:10:49.466968    2562 flags.go:64] FLAG: --registry-burst="10"
Sep 17 23:10:49.474398 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:10:49.466971    2562 flags.go:64] FLAG: --registry-qps="5"
Sep 17 23:10:49.474398 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:10:49.466974    2562 flags.go:64] FLAG: --reserved-cpus=""
Sep 17 23:10:49.474398 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:10:49.466977    2562 flags.go:64] FLAG: --reserved-memory=""
Sep 17 23:10:49.474398 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:10:49.466982    2562 flags.go:64] FLAG: --resolv-conf="/etc/resolv.conf"
Sep 17 23:10:49.474398 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:10:49.466985    2562 flags.go:64] FLAG: --root-dir="/var/lib/kubelet"
Sep 17 23:10:49.474398 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:10:49.466988    2562 flags.go:64] FLAG: --rotate-certificates="false"
Sep 17 23:10:49.475299 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:10:49.466991    2562 flags.go:64] FLAG: --rotate-server-certificates="false"
Sep 17 23:10:49.475299 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:10:49.466994    2562 flags.go:64] FLAG: --runonce="false"
Sep 17 23:10:49.475299 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:10:49.466997    2562 flags.go:64] FLAG: --runtime-cgroups="/system.slice/crio.service"
Sep 17 23:10:49.475299 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:10:49.467000    2562 flags.go:64] FLAG: --runtime-request-timeout="2m0s"
Sep 17 23:10:49.475299 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:10:49.467003    2562 flags.go:64] FLAG: --seccomp-default="false"
Sep 17 23:10:49.475299 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:10:49.467005    2562 flags.go:64] FLAG: --serialize-image-pulls="true"
Sep 17 23:10:49.475299 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:10:49.467008    2562 flags.go:64] FLAG: --storage-driver-buffer-duration="1m0s"
Sep 17 23:10:49.475299 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:10:49.467012    2562 flags.go:64] FLAG: --storage-driver-db="cadvisor"
Sep 17 23:10:49.475299 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:10:49.467016    2562 flags.go:64] FLAG: --storage-driver-host="localhost:8086"
Sep 17 23:10:49.475299 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:10:49.467019    2562 flags.go:64] FLAG: --storage-driver-password="root"
Sep 17 23:10:49.475299 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:10:49.467022    2562 flags.go:64] FLAG: --storage-driver-secure="false"
Sep 17 23:10:49.475299 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:10:49.467025    2562 flags.go:64] FLAG: --storage-driver-table="stats"
Sep 17 23:10:49.475299 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:10:49.467029    2562 flags.go:64] FLAG: --storage-driver-user="root"
Sep 17 23:10:49.475299 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:10:49.467032    2562 flags.go:64] FLAG: --streaming-connection-idle-timeout="4h0m0s"
Sep 17 23:10:49.475299 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:10:49.467035    2562 flags.go:64] FLAG: --sync-frequency="1m0s"
Sep 17 23:10:49.475299 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:10:49.467038    2562 flags.go:64] FLAG: --system-cgroups=""
Sep 17 23:10:49.475299 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:10:49.467042    2562 flags.go:64] FLAG: --system-reserved="cpu=500m,ephemeral-storage=1Gi,memory=1Gi"
Sep 17 23:10:49.475299 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:10:49.467047    2562 flags.go:64] FLAG: --system-reserved-cgroup=""
Sep 17 23:10:49.475299 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:10:49.467050    2562 flags.go:64] FLAG: --tls-cert-file=""
Sep 17 23:10:49.475299 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:10:49.467053    2562 flags.go:64] FLAG: --tls-cipher-suites="[]"
Sep 17 23:10:49.475299 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:10:49.467056    2562 flags.go:64] FLAG: --tls-min-version=""
Sep 17 23:10:49.475299 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:10:49.467059    2562 flags.go:64] FLAG: --tls-private-key-file=""
Sep 17 23:10:49.475299 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:10:49.467063    2562 flags.go:64] FLAG: --topology-manager-policy="none"
Sep 17 23:10:49.475299 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:10:49.467066    2562 flags.go:64] FLAG: --topology-manager-policy-options=""
Sep 17 23:10:49.475299 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:10:49.467069    2562 flags.go:64] FLAG: --topology-manager-scope="container"
Sep 17 23:10:49.476218 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:10:49.467072    2562 flags.go:64] FLAG: --v="2"
Sep 17 23:10:49.476218 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:10:49.467076    2562 flags.go:64] FLAG: --version="false"
Sep 17 23:10:49.476218 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:10:49.467081    2562 flags.go:64] FLAG: --vmodule=""
Sep 17 23:10:49.476218 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:10:49.467085    2562 flags.go:64] FLAG: --volume-plugin-dir="/etc/kubernetes/kubelet-plugins/volume/exec"
Sep 17 23:10:49.476218 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:10:49.467088    2562 flags.go:64] FLAG: --volume-stats-agg-period="1m0s"
Sep 17 23:10:49.476218 ip-10-0-132-188 kubenswrapper[2562]: W0917 23:10:49.467230    2562 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Sep 17 23:10:49.476218 ip-10-0-132-188 kubenswrapper[2562]: W0917 23:10:49.467235    2562 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Sep 17 23:10:49.476218 ip-10-0-132-188 kubenswrapper[2562]: W0917 23:10:49.467238    2562 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Sep 17 23:10:49.476218 ip-10-0-132-188 kubenswrapper[2562]: W0917 23:10:49.467242    2562 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Sep 17 23:10:49.476218 ip-10-0-132-188 kubenswrapper[2562]: W0917 23:10:49.467245    2562 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Sep 17 23:10:49.476218 ip-10-0-132-188 kubenswrapper[2562]: W0917 23:10:49.467248    2562 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Sep 17 23:10:49.476218 ip-10-0-132-188 kubenswrapper[2562]: W0917 23:10:49.467250    2562 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Sep 17 23:10:49.476218 ip-10-0-132-188 kubenswrapper[2562]: W0917 23:10:49.467253    2562 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Sep 17 23:10:49.476218 ip-10-0-132-188 kubenswrapper[2562]: W0917 23:10:49.467255    2562 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Sep 17 23:10:49.476218 ip-10-0-132-188 kubenswrapper[2562]: W0917 23:10:49.467258    2562 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Sep 17 23:10:49.476218 ip-10-0-132-188 kubenswrapper[2562]: W0917 23:10:49.467265    2562 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Sep 17 23:10:49.476218 ip-10-0-132-188 kubenswrapper[2562]: W0917 23:10:49.467268    2562 feature_gate.go:328] unrecognized feature gate: OVNObservability
Sep 17 23:10:49.476218 ip-10-0-132-188 kubenswrapper[2562]: W0917 23:10:49.467271    2562 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Sep 17 23:10:49.476218 ip-10-0-132-188 kubenswrapper[2562]: W0917 23:10:49.467273    2562 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Sep 17 23:10:49.476218 ip-10-0-132-188 kubenswrapper[2562]: W0917 23:10:49.467277    2562 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Sep 17 23:10:49.476218 ip-10-0-132-188 kubenswrapper[2562]: W0917 23:10:49.467279    2562 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Sep 17 23:10:49.476786 ip-10-0-132-188 kubenswrapper[2562]: W0917 23:10:49.467282    2562 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Sep 17 23:10:49.476786 ip-10-0-132-188 kubenswrapper[2562]: W0917 23:10:49.467286    2562 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Sep 17 23:10:49.476786 ip-10-0-132-188 kubenswrapper[2562]: W0917 23:10:49.467289    2562 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Sep 17 23:10:49.476786 ip-10-0-132-188 kubenswrapper[2562]: W0917 23:10:49.467291    2562 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Sep 17 23:10:49.476786 ip-10-0-132-188 kubenswrapper[2562]: W0917 23:10:49.467294    2562 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Sep 17 23:10:49.476786 ip-10-0-132-188 kubenswrapper[2562]: W0917 23:10:49.467297    2562 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Sep 17 23:10:49.476786 ip-10-0-132-188 kubenswrapper[2562]: W0917 23:10:49.467299    2562 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Sep 17 23:10:49.476786 ip-10-0-132-188 kubenswrapper[2562]: W0917 23:10:49.467302    2562 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Sep 17 23:10:49.476786 ip-10-0-132-188 kubenswrapper[2562]: W0917 23:10:49.467306    2562 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Sep 17 23:10:49.476786 ip-10-0-132-188 kubenswrapper[2562]: W0917 23:10:49.467308    2562 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Sep 17 23:10:49.476786 ip-10-0-132-188 kubenswrapper[2562]: W0917 23:10:49.467311    2562 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Sep 17 23:10:49.476786 ip-10-0-132-188 kubenswrapper[2562]: W0917 23:10:49.467314    2562 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Sep 17 23:10:49.476786 ip-10-0-132-188 kubenswrapper[2562]: W0917 23:10:49.467316    2562 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Sep 17 23:10:49.476786 ip-10-0-132-188 kubenswrapper[2562]: W0917 23:10:49.467319    2562 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Sep 17 23:10:49.476786 ip-10-0-132-188 kubenswrapper[2562]: W0917 23:10:49.467322    2562 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Sep 17 23:10:49.476786 ip-10-0-132-188 kubenswrapper[2562]: W0917 23:10:49.467325    2562 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Sep 17 23:10:49.476786 ip-10-0-132-188 kubenswrapper[2562]: W0917 23:10:49.467328    2562 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Sep 17 23:10:49.476786 ip-10-0-132-188 kubenswrapper[2562]: W0917 23:10:49.467331    2562 feature_gate.go:328] unrecognized feature gate: Example2
Sep 17 23:10:49.476786 ip-10-0-132-188 kubenswrapper[2562]: W0917 23:10:49.467333    2562 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Sep 17 23:10:49.477268 ip-10-0-132-188 kubenswrapper[2562]: W0917 23:10:49.467338    2562 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Sep 17 23:10:49.477268 ip-10-0-132-188 kubenswrapper[2562]: W0917 23:10:49.467341    2562 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Sep 17 23:10:49.477268 ip-10-0-132-188 kubenswrapper[2562]: W0917 23:10:49.467345    2562 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Sep 17 23:10:49.477268 ip-10-0-132-188 kubenswrapper[2562]: W0917 23:10:49.467347    2562 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Sep 17 23:10:49.477268 ip-10-0-132-188 kubenswrapper[2562]: W0917 23:10:49.467350    2562 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Sep 17 23:10:49.477268 ip-10-0-132-188 kubenswrapper[2562]: W0917 23:10:49.467353    2562 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Sep 17 23:10:49.477268 ip-10-0-132-188 kubenswrapper[2562]: W0917 23:10:49.467356    2562 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Sep 17 23:10:49.477268 ip-10-0-132-188 kubenswrapper[2562]: W0917 23:10:49.467360    2562 feature_gate.go:328] unrecognized feature gate: PinnedImages
Sep 17 23:10:49.477268 ip-10-0-132-188 kubenswrapper[2562]: W0917 23:10:49.467363    2562 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Sep 17 23:10:49.477268 ip-10-0-132-188 kubenswrapper[2562]: W0917 23:10:49.467366    2562 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Sep 17 23:10:49.477268 ip-10-0-132-188 kubenswrapper[2562]: W0917 23:10:49.467369    2562 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Sep 17 23:10:49.477268 ip-10-0-132-188 kubenswrapper[2562]: W0917 23:10:49.467371    2562 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Sep 17 23:10:49.477268 ip-10-0-132-188 kubenswrapper[2562]: W0917 23:10:49.467374    2562 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Sep 17 23:10:49.477268 ip-10-0-132-188 kubenswrapper[2562]: W0917 23:10:49.467376    2562 feature_gate.go:328] unrecognized feature gate: SignatureStores
Sep 17 23:10:49.477268 ip-10-0-132-188 kubenswrapper[2562]: W0917 23:10:49.467380    2562 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Sep 17 23:10:49.477268 ip-10-0-132-188 kubenswrapper[2562]: W0917 23:10:49.467384    2562 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Sep 17 23:10:49.477268 ip-10-0-132-188 kubenswrapper[2562]: W0917 23:10:49.467387    2562 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Sep 17 23:10:49.477268 ip-10-0-132-188 kubenswrapper[2562]: W0917 23:10:49.467390    2562 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Sep 17 23:10:49.477268 ip-10-0-132-188 kubenswrapper[2562]: W0917 23:10:49.467392    2562 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Sep 17 23:10:49.477742 ip-10-0-132-188 kubenswrapper[2562]: W0917 23:10:49.467395    2562 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Sep 17 23:10:49.477742 ip-10-0-132-188 kubenswrapper[2562]: W0917 23:10:49.467397    2562 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Sep 17 23:10:49.477742 ip-10-0-132-188 kubenswrapper[2562]: W0917 23:10:49.467401    2562 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Sep 17 23:10:49.477742 ip-10-0-132-188 kubenswrapper[2562]: W0917 23:10:49.467404    2562 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Sep 17 23:10:49.477742 ip-10-0-132-188 kubenswrapper[2562]: W0917 23:10:49.467407    2562 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Sep 17 23:10:49.477742 ip-10-0-132-188 kubenswrapper[2562]: W0917 23:10:49.467410    2562 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Sep 17 23:10:49.477742 ip-10-0-132-188 kubenswrapper[2562]: W0917 23:10:49.467412    2562 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Sep 17 23:10:49.477742 ip-10-0-132-188 kubenswrapper[2562]: W0917 23:10:49.467415    2562 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Sep 17 23:10:49.477742 ip-10-0-132-188 kubenswrapper[2562]: W0917 23:10:49.467418    2562 feature_gate.go:328] unrecognized feature gate: Example
Sep 17 23:10:49.477742 ip-10-0-132-188 kubenswrapper[2562]: W0917 23:10:49.467420    2562 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Sep 17 23:10:49.477742 ip-10-0-132-188 kubenswrapper[2562]: W0917 23:10:49.467423    2562 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Sep 17 23:10:49.477742 ip-10-0-132-188 kubenswrapper[2562]: W0917 23:10:49.467426    2562 feature_gate.go:328] unrecognized feature gate: OSStreams
Sep 17 23:10:49.477742 ip-10-0-132-188 kubenswrapper[2562]: W0917 23:10:49.467429    2562 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Sep 17 23:10:49.477742 ip-10-0-132-188 kubenswrapper[2562]: W0917 23:10:49.467432    2562 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Sep 17 23:10:49.477742 ip-10-0-132-188 kubenswrapper[2562]: W0917 23:10:49.467435    2562 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Sep 17 23:10:49.477742 ip-10-0-132-188 kubenswrapper[2562]: W0917 23:10:49.467437    2562 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Sep 17 23:10:49.477742 ip-10-0-132-188 kubenswrapper[2562]: W0917 23:10:49.467440    2562 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Sep 17 23:10:49.477742 ip-10-0-132-188 kubenswrapper[2562]: W0917 23:10:49.467443    2562 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Sep 17 23:10:49.477742 ip-10-0-132-188 kubenswrapper[2562]: W0917 23:10:49.467445    2562 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Sep 17 23:10:49.477742 ip-10-0-132-188 kubenswrapper[2562]: W0917 23:10:49.467448    2562 feature_gate.go:328] unrecognized feature gate: NewOLM
Sep 17 23:10:49.478254 ip-10-0-132-188 kubenswrapper[2562]: W0917 23:10:49.467452    2562 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Sep 17 23:10:49.478254 ip-10-0-132-188 kubenswrapper[2562]: W0917 23:10:49.467455    2562 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Sep 17 23:10:49.478254 ip-10-0-132-188 kubenswrapper[2562]: W0917 23:10:49.467457    2562 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Sep 17 23:10:49.478254 ip-10-0-132-188 kubenswrapper[2562]: W0917 23:10:49.467460    2562 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Sep 17 23:10:49.478254 ip-10-0-132-188 kubenswrapper[2562]: W0917 23:10:49.467462    2562 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Sep 17 23:10:49.478254 ip-10-0-132-188 kubenswrapper[2562]: W0917 23:10:49.467465    2562 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Sep 17 23:10:49.478254 ip-10-0-132-188 kubenswrapper[2562]: W0917 23:10:49.467467    2562 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Sep 17 23:10:49.478254 ip-10-0-132-188 kubenswrapper[2562]: W0917 23:10:49.467470    2562 feature_gate.go:328] unrecognized feature gate: EventTTL
Sep 17 23:10:49.478254 ip-10-0-132-188 kubenswrapper[2562]: W0917 23:10:49.467473    2562 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Sep 17 23:10:49.478254 ip-10-0-132-188 kubenswrapper[2562]: W0917 23:10:49.467476    2562 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Sep 17 23:10:49.478254 ip-10-0-132-188 kubenswrapper[2562]: W0917 23:10:49.467478    2562 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Sep 17 23:10:49.478254 ip-10-0-132-188 kubenswrapper[2562]: W0917 23:10:49.467481    2562 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Sep 17 23:10:49.478254 ip-10-0-132-188 kubenswrapper[2562]: W0917 23:10:49.467484    2562 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Sep 17 23:10:49.478254 ip-10-0-132-188 kubenswrapper[2562]: W0917 23:10:49.467486    2562 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Sep 17 23:10:49.478254 ip-10-0-132-188 kubenswrapper[2562]: W0917 23:10:49.467490    2562 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Sep 17 23:10:49.478254 ip-10-0-132-188 kubenswrapper[2562]: W0917 23:10:49.467493    2562 feature_gate.go:328] unrecognized feature gate: DualReplica
Sep 17 23:10:49.478254 ip-10-0-132-188 kubenswrapper[2562]: W0917 23:10:49.467496    2562 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Sep 17 23:10:49.478254 ip-10-0-132-188 kubenswrapper[2562]: W0917 23:10:49.467498    2562 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Sep 17 23:10:49.478254 ip-10-0-132-188 kubenswrapper[2562]: W0917 23:10:49.467502    2562 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Sep 17 23:10:49.478742 ip-10-0-132-188 kubenswrapper[2562]: W0917 23:10:49.467506    2562 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Sep 17 23:10:49.478742 ip-10-0-132-188 kubenswrapper[2562]: W0917 23:10:49.467509    2562 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Sep 17 23:10:49.478742 ip-10-0-132-188 kubenswrapper[2562]: W0917 23:10:49.467512    2562 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Sep 17 23:10:49.478742 ip-10-0-132-188 kubenswrapper[2562]: W0917 23:10:49.467515    2562 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Sep 17 23:10:49.478742 ip-10-0-132-188 kubenswrapper[2562]: W0917 23:10:49.467518    2562 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Sep 17 23:10:49.478742 ip-10-0-132-188 kubenswrapper[2562]: W0917 23:10:49.467521    2562 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Sep 17 23:10:49.478742 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:10:49.467526    2562 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]}
Sep 17 23:10:49.478742 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:10:49.476315    2562 server.go:529] "Kubelet version" kubeletVersion="v1.34.9"
Sep 17 23:10:49.478742 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:10:49.476333    2562 server.go:531] "Golang settings" GOGC="" GOMAXPROCS="" GOTRACEBACK=""
Sep 17 23:10:49.478742 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:10:49.476355    2562 watchdog_linux.go:95] "Systemd watchdog is not enabled"
Sep 17 23:10:49.478742 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:10:49.476365    2562 watchdog_linux.go:137] "Systemd watchdog is not enabled or the interval is invalid, so health checking will not be started."
Sep 17 23:10:49.478742 ip-10-0-132-188 kubenswrapper[2562]: W0917 23:10:49.476428    2562 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Sep 17 23:10:49.478742 ip-10-0-132-188 kubenswrapper[2562]: W0917 23:10:49.476433    2562 feature_gate.go:328] unrecognized feature gate: Example2
Sep 17 23:10:49.478742 ip-10-0-132-188 kubenswrapper[2562]: W0917 23:10:49.476437    2562 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Sep 17 23:10:49.478742 ip-10-0-132-188 kubenswrapper[2562]: W0917 23:10:49.476440    2562 feature_gate.go:328] unrecognized feature gate: DualReplica
Sep 17 23:10:49.479109 ip-10-0-132-188 kubenswrapper[2562]: W0917 23:10:49.476444    2562 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Sep 17 23:10:49.479109 ip-10-0-132-188 kubenswrapper[2562]: W0917 23:10:49.476447    2562 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Sep 17 23:10:49.479109 ip-10-0-132-188 kubenswrapper[2562]: W0917 23:10:49.476450    2562 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Sep 17 23:10:49.479109 ip-10-0-132-188 kubenswrapper[2562]: W0917 23:10:49.476453    2562 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Sep 17 23:10:49.479109 ip-10-0-132-188 kubenswrapper[2562]: W0917 23:10:49.476455    2562 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Sep 17 23:10:49.479109 ip-10-0-132-188 kubenswrapper[2562]: W0917 23:10:49.476458    2562 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Sep 17 23:10:49.479109 ip-10-0-132-188 kubenswrapper[2562]: W0917 23:10:49.476461    2562 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Sep 17 23:10:49.479109 ip-10-0-132-188 kubenswrapper[2562]: W0917 23:10:49.476463    2562 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Sep 17 23:10:49.479109 ip-10-0-132-188 kubenswrapper[2562]: W0917 23:10:49.476466    2562 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Sep 17 23:10:49.479109 ip-10-0-132-188 kubenswrapper[2562]: W0917 23:10:49.476469    2562 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Sep 17 23:10:49.479109 ip-10-0-132-188 kubenswrapper[2562]: W0917 23:10:49.476471    2562 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Sep 17 23:10:49.479109 ip-10-0-132-188 kubenswrapper[2562]: W0917 23:10:49.476475    2562 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Sep 17 23:10:49.479109 ip-10-0-132-188 kubenswrapper[2562]: W0917 23:10:49.476477    2562 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Sep 17 23:10:49.479109 ip-10-0-132-188 kubenswrapper[2562]: W0917 23:10:49.476481    2562 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Sep 17 23:10:49.479109 ip-10-0-132-188 kubenswrapper[2562]: W0917 23:10:49.476484    2562 feature_gate.go:328] unrecognized feature gate: SignatureStores
Sep 17 23:10:49.479109 ip-10-0-132-188 kubenswrapper[2562]: W0917 23:10:49.476486    2562 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Sep 17 23:10:49.479109 ip-10-0-132-188 kubenswrapper[2562]: W0917 23:10:49.476489    2562 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Sep 17 23:10:49.479109 ip-10-0-132-188 kubenswrapper[2562]: W0917 23:10:49.476491    2562 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Sep 17 23:10:49.479109 ip-10-0-132-188 kubenswrapper[2562]: W0917 23:10:49.476494    2562 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Sep 17 23:10:49.479768 ip-10-0-132-188 kubenswrapper[2562]: W0917 23:10:49.476497    2562 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Sep 17 23:10:49.479768 ip-10-0-132-188 kubenswrapper[2562]: W0917 23:10:49.476499    2562 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Sep 17 23:10:49.479768 ip-10-0-132-188 kubenswrapper[2562]: W0917 23:10:49.476513    2562 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Sep 17 23:10:49.479768 ip-10-0-132-188 kubenswrapper[2562]: W0917 23:10:49.476516    2562 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Sep 17 23:10:49.479768 ip-10-0-132-188 kubenswrapper[2562]: W0917 23:10:49.476519    2562 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Sep 17 23:10:49.479768 ip-10-0-132-188 kubenswrapper[2562]: W0917 23:10:49.476522    2562 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Sep 17 23:10:49.479768 ip-10-0-132-188 kubenswrapper[2562]: W0917 23:10:49.476525    2562 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Sep 17 23:10:49.479768 ip-10-0-132-188 kubenswrapper[2562]: W0917 23:10:49.476528    2562 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Sep 17 23:10:49.479768 ip-10-0-132-188 kubenswrapper[2562]: W0917 23:10:49.476531    2562 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Sep 17 23:10:49.479768 ip-10-0-132-188 kubenswrapper[2562]: W0917 23:10:49.476534    2562 feature_gate.go:328] unrecognized feature gate: OSStreams
Sep 17 23:10:49.479768 ip-10-0-132-188 kubenswrapper[2562]: W0917 23:10:49.476537    2562 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Sep 17 23:10:49.479768 ip-10-0-132-188 kubenswrapper[2562]: W0917 23:10:49.476539    2562 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Sep 17 23:10:49.479768 ip-10-0-132-188 kubenswrapper[2562]: W0917 23:10:49.476542    2562 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Sep 17 23:10:49.479768 ip-10-0-132-188 kubenswrapper[2562]: W0917 23:10:49.476545    2562 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Sep 17 23:10:49.479768 ip-10-0-132-188 kubenswrapper[2562]: W0917 23:10:49.476547    2562 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Sep 17 23:10:49.479768 ip-10-0-132-188 kubenswrapper[2562]: W0917 23:10:49.476550    2562 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Sep 17 23:10:49.479768 ip-10-0-132-188 kubenswrapper[2562]: W0917 23:10:49.476553    2562 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Sep 17 23:10:49.479768 ip-10-0-132-188 kubenswrapper[2562]: W0917 23:10:49.476555    2562 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Sep 17 23:10:49.479768 ip-10-0-132-188 kubenswrapper[2562]: W0917 23:10:49.476558    2562 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Sep 17 23:10:49.479768 ip-10-0-132-188 kubenswrapper[2562]: W0917 23:10:49.476560    2562 feature_gate.go:328] unrecognized feature gate: Example
Sep 17 23:10:49.480271 ip-10-0-132-188 kubenswrapper[2562]: W0917 23:10:49.476563    2562 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Sep 17 23:10:49.480271 ip-10-0-132-188 kubenswrapper[2562]: W0917 23:10:49.476565    2562 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Sep 17 23:10:49.480271 ip-10-0-132-188 kubenswrapper[2562]: W0917 23:10:49.476569    2562 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Sep 17 23:10:49.480271 ip-10-0-132-188 kubenswrapper[2562]: W0917 23:10:49.476571    2562 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Sep 17 23:10:49.480271 ip-10-0-132-188 kubenswrapper[2562]: W0917 23:10:49.476574    2562 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Sep 17 23:10:49.480271 ip-10-0-132-188 kubenswrapper[2562]: W0917 23:10:49.476576    2562 feature_gate.go:328] unrecognized feature gate: EventTTL
Sep 17 23:10:49.480271 ip-10-0-132-188 kubenswrapper[2562]: W0917 23:10:49.476579    2562 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Sep 17 23:10:49.480271 ip-10-0-132-188 kubenswrapper[2562]: W0917 23:10:49.476582    2562 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Sep 17 23:10:49.480271 ip-10-0-132-188 kubenswrapper[2562]: W0917 23:10:49.476585    2562 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Sep 17 23:10:49.480271 ip-10-0-132-188 kubenswrapper[2562]: W0917 23:10:49.476587    2562 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Sep 17 23:10:49.480271 ip-10-0-132-188 kubenswrapper[2562]: W0917 23:10:49.476591    2562 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Sep 17 23:10:49.480271 ip-10-0-132-188 kubenswrapper[2562]: W0917 23:10:49.476595    2562 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Sep 17 23:10:49.480271 ip-10-0-132-188 kubenswrapper[2562]: W0917 23:10:49.476599    2562 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Sep 17 23:10:49.480271 ip-10-0-132-188 kubenswrapper[2562]: W0917 23:10:49.476602    2562 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Sep 17 23:10:49.480271 ip-10-0-132-188 kubenswrapper[2562]: W0917 23:10:49.476604    2562 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Sep 17 23:10:49.480271 ip-10-0-132-188 kubenswrapper[2562]: W0917 23:10:49.476607    2562 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Sep 17 23:10:49.480271 ip-10-0-132-188 kubenswrapper[2562]: W0917 23:10:49.476609    2562 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Sep 17 23:10:49.480271 ip-10-0-132-188 kubenswrapper[2562]: W0917 23:10:49.476612    2562 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Sep 17 23:10:49.480271 ip-10-0-132-188 kubenswrapper[2562]: W0917 23:10:49.476614    2562 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Sep 17 23:10:49.480271 ip-10-0-132-188 kubenswrapper[2562]: W0917 23:10:49.476617    2562 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Sep 17 23:10:49.481089 ip-10-0-132-188 kubenswrapper[2562]: W0917 23:10:49.476620    2562 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Sep 17 23:10:49.481089 ip-10-0-132-188 kubenswrapper[2562]: W0917 23:10:49.476623    2562 feature_gate.go:328] unrecognized feature gate: NewOLM
Sep 17 23:10:49.481089 ip-10-0-132-188 kubenswrapper[2562]: W0917 23:10:49.476628    2562 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Sep 17 23:10:49.481089 ip-10-0-132-188 kubenswrapper[2562]: W0917 23:10:49.476631    2562 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Sep 17 23:10:49.481089 ip-10-0-132-188 kubenswrapper[2562]: W0917 23:10:49.476634    2562 feature_gate.go:328] unrecognized feature gate: PinnedImages
Sep 17 23:10:49.481089 ip-10-0-132-188 kubenswrapper[2562]: W0917 23:10:49.476637    2562 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Sep 17 23:10:49.481089 ip-10-0-132-188 kubenswrapper[2562]: W0917 23:10:49.476641    2562 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Sep 17 23:10:49.481089 ip-10-0-132-188 kubenswrapper[2562]: W0917 23:10:49.476643    2562 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Sep 17 23:10:49.481089 ip-10-0-132-188 kubenswrapper[2562]: W0917 23:10:49.476646    2562 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Sep 17 23:10:49.481089 ip-10-0-132-188 kubenswrapper[2562]: W0917 23:10:49.476649    2562 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Sep 17 23:10:49.481089 ip-10-0-132-188 kubenswrapper[2562]: W0917 23:10:49.476652    2562 feature_gate.go:328] unrecognized feature gate: OVNObservability
Sep 17 23:10:49.481089 ip-10-0-132-188 kubenswrapper[2562]: W0917 23:10:49.476655    2562 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Sep 17 23:10:49.481089 ip-10-0-132-188 kubenswrapper[2562]: W0917 23:10:49.476658    2562 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Sep 17 23:10:49.481089 ip-10-0-132-188 kubenswrapper[2562]: W0917 23:10:49.476661    2562 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Sep 17 23:10:49.481089 ip-10-0-132-188 kubenswrapper[2562]: W0917 23:10:49.476664    2562 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Sep 17 23:10:49.481089 ip-10-0-132-188 kubenswrapper[2562]: W0917 23:10:49.476666    2562 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Sep 17 23:10:49.481089 ip-10-0-132-188 kubenswrapper[2562]: W0917 23:10:49.476669    2562 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Sep 17 23:10:49.481089 ip-10-0-132-188 kubenswrapper[2562]: W0917 23:10:49.476672    2562 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Sep 17 23:10:49.481089 ip-10-0-132-188 kubenswrapper[2562]: W0917 23:10:49.476675    2562 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Sep 17 23:10:49.481089 ip-10-0-132-188 kubenswrapper[2562]: W0917 23:10:49.476678    2562 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Sep 17 23:10:49.481807 ip-10-0-132-188 kubenswrapper[2562]: W0917 23:10:49.476681    2562 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Sep 17 23:10:49.481807 ip-10-0-132-188 kubenswrapper[2562]: W0917 23:10:49.476683    2562 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Sep 17 23:10:49.481807 ip-10-0-132-188 kubenswrapper[2562]: W0917 23:10:49.476686    2562 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Sep 17 23:10:49.481807 ip-10-0-132-188 kubenswrapper[2562]: W0917 23:10:49.476688    2562 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Sep 17 23:10:49.481807 ip-10-0-132-188 kubenswrapper[2562]: W0917 23:10:49.476691    2562 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Sep 17 23:10:49.481807 ip-10-0-132-188 kubenswrapper[2562]: W0917 23:10:49.476693    2562 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Sep 17 23:10:49.481807 ip-10-0-132-188 kubenswrapper[2562]: W0917 23:10:49.476696    2562 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Sep 17 23:10:49.481807 ip-10-0-132-188 kubenswrapper[2562]: W0917 23:10:49.476699    2562 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Sep 17 23:10:49.481807 ip-10-0-132-188 kubenswrapper[2562]: W0917 23:10:49.476701    2562 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Sep 17 23:10:49.481807 ip-10-0-132-188 kubenswrapper[2562]: W0917 23:10:49.476704    2562 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Sep 17 23:10:49.481807 ip-10-0-132-188 kubenswrapper[2562]: W0917 23:10:49.476706    2562 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Sep 17 23:10:49.481807 ip-10-0-132-188 kubenswrapper[2562]: W0917 23:10:49.476709    2562 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Sep 17 23:10:49.481807 ip-10-0-132-188 kubenswrapper[2562]: W0917 23:10:49.476713    2562 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Sep 17 23:10:49.481807 ip-10-0-132-188 kubenswrapper[2562]: W0917 23:10:49.476716    2562 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Sep 17 23:10:49.481807 ip-10-0-132-188 kubenswrapper[2562]: W0917 23:10:49.476719    2562 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Sep 17 23:10:49.481807 ip-10-0-132-188 kubenswrapper[2562]: W0917 23:10:49.476722    2562 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Sep 17 23:10:49.482242 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:10:49.476727    2562 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]}
Sep 17 23:10:49.482242 ip-10-0-132-188 kubenswrapper[2562]: W0917 23:10:49.476857    2562 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Sep 17 23:10:49.482242 ip-10-0-132-188 kubenswrapper[2562]: W0917 23:10:49.476863    2562 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Sep 17 23:10:49.482242 ip-10-0-132-188 kubenswrapper[2562]: W0917 23:10:49.476866    2562 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Sep 17 23:10:49.482242 ip-10-0-132-188 kubenswrapper[2562]: W0917 23:10:49.476868    2562 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Sep 17 23:10:49.482242 ip-10-0-132-188 kubenswrapper[2562]: W0917 23:10:49.476871    2562 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Sep 17 23:10:49.482242 ip-10-0-132-188 kubenswrapper[2562]: W0917 23:10:49.476874    2562 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Sep 17 23:10:49.482242 ip-10-0-132-188 kubenswrapper[2562]: W0917 23:10:49.476877    2562 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Sep 17 23:10:49.482242 ip-10-0-132-188 kubenswrapper[2562]: W0917 23:10:49.476879    2562 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Sep 17 23:10:49.482242 ip-10-0-132-188 kubenswrapper[2562]: W0917 23:10:49.476882    2562 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Sep 17 23:10:49.482242 ip-10-0-132-188 kubenswrapper[2562]: W0917 23:10:49.476885    2562 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Sep 17 23:10:49.482242 ip-10-0-132-188 kubenswrapper[2562]: W0917 23:10:49.476887    2562 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Sep 17 23:10:49.482242 ip-10-0-132-188 kubenswrapper[2562]: W0917 23:10:49.476890    2562 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Sep 17 23:10:49.482242 ip-10-0-132-188 kubenswrapper[2562]: W0917 23:10:49.476894    2562 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Sep 17 23:10:49.482242 ip-10-0-132-188 kubenswrapper[2562]: W0917 23:10:49.476897    2562 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Sep 17 23:10:49.482749 ip-10-0-132-188 kubenswrapper[2562]: W0917 23:10:49.476900    2562 feature_gate.go:328] unrecognized feature gate: Example
Sep 17 23:10:49.482749 ip-10-0-132-188 kubenswrapper[2562]: W0917 23:10:49.476903    2562 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Sep 17 23:10:49.482749 ip-10-0-132-188 kubenswrapper[2562]: W0917 23:10:49.476906    2562 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Sep 17 23:10:49.482749 ip-10-0-132-188 kubenswrapper[2562]: W0917 23:10:49.476909    2562 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Sep 17 23:10:49.482749 ip-10-0-132-188 kubenswrapper[2562]: W0917 23:10:49.476911    2562 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Sep 17 23:10:49.482749 ip-10-0-132-188 kubenswrapper[2562]: W0917 23:10:49.476914    2562 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Sep 17 23:10:49.482749 ip-10-0-132-188 kubenswrapper[2562]: W0917 23:10:49.476916    2562 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Sep 17 23:10:49.482749 ip-10-0-132-188 kubenswrapper[2562]: W0917 23:10:49.476919    2562 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Sep 17 23:10:49.482749 ip-10-0-132-188 kubenswrapper[2562]: W0917 23:10:49.476922    2562 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Sep 17 23:10:49.482749 ip-10-0-132-188 kubenswrapper[2562]: W0917 23:10:49.476924    2562 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Sep 17 23:10:49.482749 ip-10-0-132-188 kubenswrapper[2562]: W0917 23:10:49.476927    2562 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Sep 17 23:10:49.482749 ip-10-0-132-188 kubenswrapper[2562]: W0917 23:10:49.476929    2562 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Sep 17 23:10:49.482749 ip-10-0-132-188 kubenswrapper[2562]: W0917 23:10:49.476932    2562 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Sep 17 23:10:49.482749 ip-10-0-132-188 kubenswrapper[2562]: W0917 23:10:49.476935    2562 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Sep 17 23:10:49.482749 ip-10-0-132-188 kubenswrapper[2562]: W0917 23:10:49.476938    2562 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Sep 17 23:10:49.482749 ip-10-0-132-188 kubenswrapper[2562]: W0917 23:10:49.476940    2562 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Sep 17 23:10:49.482749 ip-10-0-132-188 kubenswrapper[2562]: W0917 23:10:49.476943    2562 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Sep 17 23:10:49.482749 ip-10-0-132-188 kubenswrapper[2562]: W0917 23:10:49.476946    2562 feature_gate.go:328] unrecognized feature gate: Example2
Sep 17 23:10:49.482749 ip-10-0-132-188 kubenswrapper[2562]: W0917 23:10:49.476948    2562 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Sep 17 23:10:49.482749 ip-10-0-132-188 kubenswrapper[2562]: W0917 23:10:49.476951    2562 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Sep 17 23:10:49.483309 ip-10-0-132-188 kubenswrapper[2562]: W0917 23:10:49.476953    2562 feature_gate.go:328] unrecognized feature gate: NewOLM
Sep 17 23:10:49.483309 ip-10-0-132-188 kubenswrapper[2562]: W0917 23:10:49.476956    2562 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Sep 17 23:10:49.483309 ip-10-0-132-188 kubenswrapper[2562]: W0917 23:10:49.476959    2562 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Sep 17 23:10:49.483309 ip-10-0-132-188 kubenswrapper[2562]: W0917 23:10:49.476961    2562 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Sep 17 23:10:49.483309 ip-10-0-132-188 kubenswrapper[2562]: W0917 23:10:49.476963    2562 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Sep 17 23:10:49.483309 ip-10-0-132-188 kubenswrapper[2562]: W0917 23:10:49.476966    2562 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Sep 17 23:10:49.483309 ip-10-0-132-188 kubenswrapper[2562]: W0917 23:10:49.476968    2562 feature_gate.go:328] unrecognized feature gate: DualReplica
Sep 17 23:10:49.483309 ip-10-0-132-188 kubenswrapper[2562]: W0917 23:10:49.476971    2562 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Sep 17 23:10:49.483309 ip-10-0-132-188 kubenswrapper[2562]: W0917 23:10:49.476974    2562 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Sep 17 23:10:49.483309 ip-10-0-132-188 kubenswrapper[2562]: W0917 23:10:49.476976    2562 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Sep 17 23:10:49.483309 ip-10-0-132-188 kubenswrapper[2562]: W0917 23:10:49.476978    2562 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Sep 17 23:10:49.483309 ip-10-0-132-188 kubenswrapper[2562]: W0917 23:10:49.476982    2562 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Sep 17 23:10:49.483309 ip-10-0-132-188 kubenswrapper[2562]: W0917 23:10:49.476985    2562 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Sep 17 23:10:49.483309 ip-10-0-132-188 kubenswrapper[2562]: W0917 23:10:49.476988    2562 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Sep 17 23:10:49.483309 ip-10-0-132-188 kubenswrapper[2562]: W0917 23:10:49.476990    2562 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Sep 17 23:10:49.483309 ip-10-0-132-188 kubenswrapper[2562]: W0917 23:10:49.476993    2562 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Sep 17 23:10:49.483309 ip-10-0-132-188 kubenswrapper[2562]: W0917 23:10:49.476996    2562 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Sep 17 23:10:49.483309 ip-10-0-132-188 kubenswrapper[2562]: W0917 23:10:49.476998    2562 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Sep 17 23:10:49.483309 ip-10-0-132-188 kubenswrapper[2562]: W0917 23:10:49.477001    2562 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Sep 17 23:10:49.483824 ip-10-0-132-188 kubenswrapper[2562]: W0917 23:10:49.477004    2562 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Sep 17 23:10:49.483824 ip-10-0-132-188 kubenswrapper[2562]: W0917 23:10:49.477006    2562 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Sep 17 23:10:49.483824 ip-10-0-132-188 kubenswrapper[2562]: W0917 23:10:49.477009    2562 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Sep 17 23:10:49.483824 ip-10-0-132-188 kubenswrapper[2562]: W0917 23:10:49.477012    2562 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Sep 17 23:10:49.483824 ip-10-0-132-188 kubenswrapper[2562]: W0917 23:10:49.477014    2562 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Sep 17 23:10:49.483824 ip-10-0-132-188 kubenswrapper[2562]: W0917 23:10:49.477017    2562 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Sep 17 23:10:49.483824 ip-10-0-132-188 kubenswrapper[2562]: W0917 23:10:49.477020    2562 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Sep 17 23:10:49.483824 ip-10-0-132-188 kubenswrapper[2562]: W0917 23:10:49.477024    2562 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Sep 17 23:10:49.483824 ip-10-0-132-188 kubenswrapper[2562]: W0917 23:10:49.477027    2562 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Sep 17 23:10:49.483824 ip-10-0-132-188 kubenswrapper[2562]: W0917 23:10:49.477030    2562 feature_gate.go:328] unrecognized feature gate: OSStreams
Sep 17 23:10:49.483824 ip-10-0-132-188 kubenswrapper[2562]: W0917 23:10:49.477032    2562 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Sep 17 23:10:49.483824 ip-10-0-132-188 kubenswrapper[2562]: W0917 23:10:49.477035    2562 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Sep 17 23:10:49.483824 ip-10-0-132-188 kubenswrapper[2562]: W0917 23:10:49.477037    2562 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Sep 17 23:10:49.483824 ip-10-0-132-188 kubenswrapper[2562]: W0917 23:10:49.477040    2562 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Sep 17 23:10:49.483824 ip-10-0-132-188 kubenswrapper[2562]: W0917 23:10:49.477042    2562 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Sep 17 23:10:49.483824 ip-10-0-132-188 kubenswrapper[2562]: W0917 23:10:49.477045    2562 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Sep 17 23:10:49.483824 ip-10-0-132-188 kubenswrapper[2562]: W0917 23:10:49.477047    2562 feature_gate.go:328] unrecognized feature gate: SignatureStores
Sep 17 23:10:49.483824 ip-10-0-132-188 kubenswrapper[2562]: W0917 23:10:49.477050    2562 feature_gate.go:328] unrecognized feature gate: EventTTL
Sep 17 23:10:49.483824 ip-10-0-132-188 kubenswrapper[2562]: W0917 23:10:49.477052    2562 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Sep 17 23:10:49.483824 ip-10-0-132-188 kubenswrapper[2562]: W0917 23:10:49.477055    2562 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Sep 17 23:10:49.484422 ip-10-0-132-188 kubenswrapper[2562]: W0917 23:10:49.477057    2562 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Sep 17 23:10:49.484422 ip-10-0-132-188 kubenswrapper[2562]: W0917 23:10:49.477060    2562 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Sep 17 23:10:49.484422 ip-10-0-132-188 kubenswrapper[2562]: W0917 23:10:49.477062    2562 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Sep 17 23:10:49.484422 ip-10-0-132-188 kubenswrapper[2562]: W0917 23:10:49.477065    2562 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Sep 17 23:10:49.484422 ip-10-0-132-188 kubenswrapper[2562]: W0917 23:10:49.477069    2562 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Sep 17 23:10:49.484422 ip-10-0-132-188 kubenswrapper[2562]: W0917 23:10:49.477073    2562 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Sep 17 23:10:49.484422 ip-10-0-132-188 kubenswrapper[2562]: W0917 23:10:49.477076    2562 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Sep 17 23:10:49.484422 ip-10-0-132-188 kubenswrapper[2562]: W0917 23:10:49.477080    2562 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Sep 17 23:10:49.484422 ip-10-0-132-188 kubenswrapper[2562]: W0917 23:10:49.477085    2562 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Sep 17 23:10:49.484422 ip-10-0-132-188 kubenswrapper[2562]: W0917 23:10:49.477088    2562 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Sep 17 23:10:49.484422 ip-10-0-132-188 kubenswrapper[2562]: W0917 23:10:49.477091    2562 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Sep 17 23:10:49.484422 ip-10-0-132-188 kubenswrapper[2562]: W0917 23:10:49.477094    2562 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Sep 17 23:10:49.484422 ip-10-0-132-188 kubenswrapper[2562]: W0917 23:10:49.477097    2562 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Sep 17 23:10:49.484422 ip-10-0-132-188 kubenswrapper[2562]: W0917 23:10:49.477100    2562 feature_gate.go:328] unrecognized feature gate: PinnedImages
Sep 17 23:10:49.484422 ip-10-0-132-188 kubenswrapper[2562]: W0917 23:10:49.477103    2562 feature_gate.go:328] unrecognized feature gate: OVNObservability
Sep 17 23:10:49.484422 ip-10-0-132-188 kubenswrapper[2562]: W0917 23:10:49.477106    2562 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Sep 17 23:10:49.484422 ip-10-0-132-188 kubenswrapper[2562]: W0917 23:10:49.477109    2562 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Sep 17 23:10:49.484422 ip-10-0-132-188 kubenswrapper[2562]: W0917 23:10:49.477112    2562 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Sep 17 23:10:49.484422 ip-10-0-132-188 kubenswrapper[2562]: W0917 23:10:49.477115    2562 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Sep 17 23:10:49.485124 ip-10-0-132-188 kubenswrapper[2562]: W0917 23:10:49.477118    2562 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Sep 17 23:10:49.485124 ip-10-0-132-188 kubenswrapper[2562]: W0917 23:10:49.477121    2562 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Sep 17 23:10:49.485124 ip-10-0-132-188 kubenswrapper[2562]: W0917 23:10:49.477123    2562 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Sep 17 23:10:49.485124 ip-10-0-132-188 kubenswrapper[2562]: W0917 23:10:49.477128    2562 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Sep 17 23:10:49.485124 ip-10-0-132-188 kubenswrapper[2562]: W0917 23:10:49.477131    2562 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Sep 17 23:10:49.485124 ip-10-0-132-188 kubenswrapper[2562]: W0917 23:10:49.477134    2562 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Sep 17 23:10:49.485124 ip-10-0-132-188 kubenswrapper[2562]: W0917 23:10:49.477137    2562 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Sep 17 23:10:49.485124 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:10:49.477142    2562 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]}
Sep 17 23:10:49.485124 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:10:49.477285    2562 server.go:962] "Client rotation is on, will bootstrap in background"
Sep 17 23:10:49.485124 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:10:49.479414    2562 bootstrap.go:101] "Use the bootstrap credentials to request a cert, and set kubeconfig to point to the certificate dir"
Sep 17 23:10:49.485124 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:10:49.480307    2562 server.go:1019] "Starting client certificate rotation"
Sep 17 23:10:49.485124 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:10:49.480431    2562 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kube-apiserver-client-kubelet"
Sep 17 23:10:49.485124 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:10:49.480486    2562 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kube-apiserver-client-kubelet"
Sep 17 23:10:49.485509 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:10:49.482678    2562 dynamic_cafile_content.go:123] "Loaded a new CA Bundle and Verifier" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt"
Sep 17 23:10:49.485509 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:10:49.483882    2562 dynamic_cafile_content.go:161] "Starting controller" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt"
Sep 17 23:10:49.487605 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:10:49.487588    2562 log.go:25] "Validated CRI v1 runtime API"
Sep 17 23:10:49.489528 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:10:49.489514    2562 log.go:25] "Validated CRI v1 image API"
Sep 17 23:10:49.490329 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:10:49.490313    2562 server.go:1429] "Using cgroup driver setting received from the CRI runtime" cgroupDriver="systemd"
Sep 17 23:10:49.492983 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:10:49.492963    2562 fs.go:135] Filesystem UUIDs: map[7B77-95E7:/dev/nvme0n1p2 8ff051db-397c-41a3-b272-5a65729e0648:/dev/nvme0n1p4 df020e48-ca5a-4a3d-a384-808a0565bca0:/dev/nvme0n1p3]
Sep 17 23:10:49.493064 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:10:49.492982    2562 fs.go:136] Filesystem partitions: map[/dev/nvme0n1p3:{mountpoint:/boot major:259 minor:3 fsType:ext4 blockSize:0} /dev/nvme0n1p4:{mountpoint:/var major:259 minor:4 fsType:xfs blockSize:0} /dev/shm:{mountpoint:/dev/shm major:0 minor:22 fsType:tmpfs blockSize:0} /run:{mountpoint:/run major:0 minor:24 fsType:tmpfs blockSize:0} /tmp:{mountpoint:/tmp major:0 minor:31 fsType:tmpfs blockSize:0} composefs_0-33:{mountpoint:/ major:0 minor:33 fsType:overlay blockSize:0}]
Sep 17 23:10:49.497214 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:10:49.497075    2562 manager.go:216] Machine: {Timestamp:2026-09-17 23:10:49.49659867 +0000 UTC m=+1.898495429 CPUVendorID:GenuineIntel NumCores:8 NumPhysicalCores:4 NumSockets:1 CpuFrequency:3158336 MemoryCapacity:33164476416 SwapCapacity:0 MemoryByType:map[] NVMInfo:{MemoryModeCapacity:0 AppDirectModeCapacity:0 AvgPowerBudget:0} HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] MachineID:ec2bf0fce8296885680acd0394e69924 SystemUUID:ec2bf0fc-e829-6885-680a-cd0394e69924 BootID:f8173e9f-3aa9-4797-a994-47a60e592878 Filesystems:[{Device:/dev/shm DeviceMajor:0 DeviceMinor:22 Capacity:16582238208 Type:vfs Inodes:4048398 HasInodes:true} {Device:/run DeviceMajor:0 DeviceMinor:24 Capacity:6632898560 Type:vfs Inodes:819200 HasInodes:true} {Device:/dev/nvme0n1p4 DeviceMajor:259 DeviceMinor:4 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:composefs_0-33 DeviceMajor:0 DeviceMinor:33 Capacity:6111232 Type:vfs Inodes:18446744073709551615 HasInodes:true} {Device:/tmp DeviceMajor:0 DeviceMinor:31 Capacity:16582238208 Type:vfs Inodes:1048576 HasInodes:true} {Device:/dev/nvme0n1p3 DeviceMajor:259 DeviceMinor:3 Capacity:366869504 Type:vfs Inodes:98304 HasInodes:true}] DiskMap:map[259:0:{Name:nvme0n1 Major:259 Minor:0 Size:128849018880 Scheduler:none}] NetworkDevices:[{Name:br-ex MacAddress:02:ff:ce:5f:77:81 Speed:0 Mtu:9001} {Name:ens5 MacAddress:02:ff:ce:5f:77:81 Speed:0 Mtu:9001} {Name:ovs-system MacAddress:ee:e8:d7:80:b7:15 Speed:0 Mtu:1500}] Topology:[{Id:0 Memory:33164476416 HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] Cores:[{Id:0 Threads:[0 4] Caches:[{Id:0 Size:32768 Type:Data Level:1} {Id:0 Size:32768 Type:Instruction Level:1} {Id:0 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:1 Threads:[1 5] Caches:[{Id:1 Size:32768 Type:Data Level:1} {Id:1 Size:32768 Type:Instruction Level:1} {Id:1 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:2 Threads:[2 6] Caches:[{Id:2 Size:32768 Type:Data Level:1} {Id:2 Size:32768 Type:Instruction Level:1} {Id:2 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:3 Threads:[3 7] Caches:[{Id:3 Size:32768 Type:Data Level:1} {Id:3 Size:32768 Type:Instruction Level:1} {Id:3 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:}] Caches:[{Id:0 Size:37486592 Type:Unified Level:3}] Distances:[10]}] CloudProvider:Unknown InstanceType:Unknown InstanceID:None}
Sep 17 23:10:49.497214 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:10:49.497197    2562 manager_no_libpfm.go:29] cAdvisor is build without cgo and/or libpfm support. Perf event counters are not available.
Sep 17 23:10:49.497372 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:10:49.497293    2562 manager.go:232] Version: {KernelVersion:5.14.0-570.141.1.el9_6.x86_64 ContainerOsVersion:Red Hat Enterprise Linux CoreOS 9.6.20260914-0 (Plow) DockerVersion: DockerAPIVersion: CadvisorVersion: CadvisorRevision:}
Sep 17 23:10:49.498723 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:10:49.498661    2562 container_manager_linux.go:273] "Container manager verified user specified cgroup-root exists" cgroupRoot=[]
Sep 17 23:10:49.498875 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:10:49.498727    2562 container_manager_linux.go:278] "Creating Container Manager object based on Node Config" nodeConfig={"NodeName":"ip-10-0-132-188.ec2.internal","RuntimeCgroupsName":"/system.slice/crio.service","SystemCgroupsName":"/system.slice","KubeletCgroupsName":"","KubeletOOMScoreAdj":-999,"ContainerRuntime":"","CgroupsPerQOS":true,"CgroupRoot":"/","CgroupDriver":"systemd","KubeletRootDir":"/var/lib/kubelet","ProtectKernelDefaults":true,"KubeReservedCgroupName":"","SystemReservedCgroupName":"","ReservedSystemCPUs":{},"EnforceNodeAllocatable":{"pods":{}},"KubeReserved":null,"SystemReserved":{"cpu":"500m","ephemeral-storage":"1Gi","memory":"1Gi"},"HardEvictionThresholds":[{"Signal":"nodefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null},{"Signal":"imagefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.15},"GracePeriod":0,"MinReclaim":null},{"Signal":"imagefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null},{"Signal":"memory.available","Operator":"LessThan","Value":{"Quantity":"100Mi","Percentage":0},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.1},"GracePeriod":0,"MinReclaim":null}],"QOSReserved":{},"CPUManagerPolicy":"none","CPUManagerPolicyOptions":null,"TopologyManagerScope":"container","CPUManagerReconcilePeriod":10000000000,"MemoryManagerPolicy":"None","MemoryManagerReservedMemory":null,"PodPidsLimit":4096,"EnforceCPULimits":true,"CPUCFSQuotaPeriod":100000000,"TopologyManagerPolicy":"none","TopologyManagerPolicyOptions":null,"CgroupVersion":2}
Sep 17 23:10:49.499943 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:10:49.499932    2562 topology_manager.go:138] "Creating topology manager with none policy"
Sep 17 23:10:49.499985 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:10:49.499945    2562 container_manager_linux.go:309] "Creating device plugin manager"
Sep 17 23:10:49.499985 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:10:49.499959    2562 manager.go:141] "Creating Device Plugin manager" path="/var/lib/kubelet/device-plugins/kubelet.sock"
Sep 17 23:10:49.499985 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:10:49.499972    2562 server.go:74] "Creating device plugin registration server" version="v1beta1" socket="/var/lib/kubelet/device-plugins/kubelet.sock"
Sep 17 23:10:49.500067 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:10:49.500052    2562 container_manager_linux.go:318] "Creating Dynamic Resource Allocation (DRA) manager"
Sep 17 23:10:49.503824 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:10:49.503810    2562 state_mem.go:36] "Initialized new in-memory state store"
Sep 17 23:10:49.503963 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:10:49.503953    2562 server.go:1240] "Using root directory" path="/var/lib/kubelet"
Sep 17 23:10:49.504012 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:10:49.503988    2562 kubelet.go:486] "Attempting to sync node with API server"
Sep 17 23:10:49.504012 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:10:49.504005    2562 kubelet.go:387] "Adding static pod path" path="/etc/kubernetes/manifests"
Sep 17 23:10:49.504085 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:10:49.504020    2562 file.go:69] "Watching path" path="/etc/kubernetes/manifests"
Sep 17 23:10:49.504085 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:10:49.504030    2562 kubelet.go:398] "Adding apiserver pod source"
Sep 17 23:10:49.504085 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:10:49.504040    2562 apiserver.go:42] "Waiting for node sync before watching apiserver pods"
Sep 17 23:10:49.504451 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:10:49.504438    2562 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking"
Sep 17 23:10:49.504497 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:10:49.504458    2562 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking"
Sep 17 23:10:49.515091 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:10:49.506446    2562 reflector.go:439] "Caches populated" logger="kubernetes.io/kube-apiserver-client-kubelet" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162"
Sep 17 23:10:49.516610 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:10:49.516556    2562 kuberuntime_manager.go:301] "Container runtime initialized" containerRuntime="cri-o" version="1.34.13-6.rhaos4.21.git6253162.el9" apiVersion="v1"
Sep 17 23:10:49.518585 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:10:49.518566    2562 kubelet.go:960] "Not starting ClusterTrustBundle informer because we are in static kubelet mode or the ClusterTrustBundleProjection featuregate is disabled"
Sep 17 23:10:49.518682 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:10:49.518610    2562 kubelet.go:984] "Not starting PodCertificateRequest manager because we are in static kubelet mode or the PodCertificateProjection feature gate is disabled"
Sep 17 23:10:49.520096 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:10:49.520083    2562 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/portworx-volume"
Sep 17 23:10:49.520162 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:10:49.520100    2562 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/empty-dir"
Sep 17 23:10:49.520162 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:10:49.520107    2562 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/git-repo"
Sep 17 23:10:49.520162 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:10:49.520112    2562 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/host-path"
Sep 17 23:10:49.520162 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:10:49.520121    2562 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/nfs"
Sep 17 23:10:49.520162 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:10:49.520127    2562 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/secret"
Sep 17 23:10:49.520162 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:10:49.520144    2562 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/iscsi"
Sep 17 23:10:49.520162 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:10:49.520149    2562 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/downward-api"
Sep 17 23:10:49.520162 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:10:49.520156    2562 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/fc"
Sep 17 23:10:49.520162 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:10:49.520161    2562 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/configmap"
Sep 17 23:10:49.520437 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:10:49.520189    2562 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/projected"
Sep 17 23:10:49.520437 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:10:49.520196    2562 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/local-volume"
Sep 17 23:10:49.520437 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:10:49.520223    2562 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/csi"
Sep 17 23:10:49.520437 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:10:49.520228    2562 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/image"
Sep 17 23:10:49.521366 ip-10-0-132-188 kubenswrapper[2562]: E0917 23:10:49.521335    2562 reflector.go:205] "Failed to watch" err="failed to list *v1.Node: nodes \"ip-10-0-132-188.ec2.internal\" is forbidden: User \"system:anonymous\" cannot list resource \"nodes\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Node"
Sep 17 23:10:49.521429 ip-10-0-132-188 kubenswrapper[2562]: E0917 23:10:49.521335    2562 reflector.go:205] "Failed to watch" err="failed to list *v1.Service: services is forbidden: User \"system:anonymous\" cannot list resource \"services\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Service"
Sep 17 23:10:49.524053 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:10:49.524039    2562 server.go:1268] "Started kubelet"
Sep 17 23:10:49.524138 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:10:49.524121    2562 csi_plugin.go:990] Failed to contact API server when waiting for CSINode publishing: csinodes.storage.k8s.io "ip-10-0-132-188.ec2.internal" is forbidden: User "system:anonymous" cannot get resource "csinodes" in API group "storage.k8s.io" at the cluster scope
Sep 17 23:10:49.524138 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:10:49.524122    2562 ratelimit.go:56] "Setting rate limiting for endpoint" service="podresources" qps=100 burstTokens=10
Sep 17 23:10:49.524256 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:10:49.524180    2562 server_v1.go:49] "podresources" method="list" useActivePods=true
Sep 17 23:10:49.524256 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:10:49.524181    2562 server.go:180] "Starting to listen" address="0.0.0.0" port=10250
Sep 17 23:10:49.525006 ip-10-0-132-188 systemd[1]: Started Kubernetes Kubelet.
Sep 17 23:10:49.525799 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:10:49.525780    2562 server.go:310] "Adding debug handlers to kubelet server"
Sep 17 23:10:49.527773 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:10:49.527753    2562 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-8pfgf"
Sep 17 23:10:49.527955 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:10:49.527939    2562 server.go:249] "Starting to serve the podresources API" endpoint="unix:/var/lib/kubelet/pod-resources/kubelet.sock"
Sep 17 23:10:49.528899 ip-10-0-132-188 kubenswrapper[2562]: E0917 23:10:49.528883    2562 kubelet.go:1638] "Image garbage collection failed once. Stats initialization may not have completed yet" err="failed to get imageFs info: unable to find data in memory cache"
Sep 17 23:10:49.532554 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:10:49.532533    2562 fs_resource_analyzer.go:67] "Starting FS ResourceAnalyzer"
Sep 17 23:10:49.532554 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:10:49.532545    2562 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kubelet-serving"
Sep 17 23:10:49.532755 ip-10-0-132-188 kubenswrapper[2562]: E0917 23:10:49.532700    2562 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-132-188.ec2.internal\" not found"
Sep 17 23:10:49.533268 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:10:49.533243    2562 desired_state_of_world_populator.go:146] "Desired state populator starts to run"
Sep 17 23:10:49.533268 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:10:49.533245    2562 volume_manager.go:311] "The desired_state_of_world populator starts"
Sep 17 23:10:49.533268 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:10:49.533278    2562 volume_manager.go:313] "Starting Kubelet Volume Manager"
Sep 17 23:10:49.533467 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:10:49.533376    2562 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-8pfgf"
Sep 17 23:10:49.533467 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:10:49.533422    2562 reconstruct.go:97] "Volume reconstruction finished"
Sep 17 23:10:49.533467 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:10:49.533430    2562 reconciler.go:29] "Reconciler: start to sync state"
Sep 17 23:10:49.533621 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:10:49.533480    2562 factory.go:55] Registering systemd factory
Sep 17 23:10:49.533621 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:10:49.533500    2562 factory.go:223] Registration of the systemd container factory successfully
Sep 17 23:10:49.533764 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:10:49.533744    2562 factory.go:153] Registering CRI-O factory
Sep 17 23:10:49.533764 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:10:49.533764    2562 factory.go:223] Registration of the crio container factory successfully
Sep 17 23:10:49.533901 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:10:49.533823    2562 factory.go:221] Registration of the containerd container factory failed: unable to create containerd client: containerd: cannot unix dial containerd api service: dial unix /run/containerd/containerd.sock: connect: no such file or directory
Sep 17 23:10:49.533901 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:10:49.533843    2562 factory.go:103] Registering Raw factory
Sep 17 23:10:49.533901 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:10:49.533859    2562 manager.go:1196] Started watching for new ooms in manager
Sep 17 23:10:49.535229 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:10:49.534896    2562 manager.go:349] Starting recovery of all containers
Sep 17 23:10:49.535434 ip-10-0-132-188 kubenswrapper[2562]: E0917 23:10:49.535409    2562 reflector.go:205] "Failed to watch" err="failed to list *v1.CSIDriver: csidrivers.storage.k8s.io is forbidden: User \"system:anonymous\" cannot list resource \"csidrivers\" in API group \"storage.k8s.io\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.CSIDriver"
Sep 17 23:10:49.535641 ip-10-0-132-188 kubenswrapper[2562]: E0917 23:10:49.535531    2562 controller.go:145] "Failed to ensure lease exists, will retry" err="leases.coordination.k8s.io \"ip-10-0-132-188.ec2.internal\" is forbidden: User \"system:anonymous\" cannot get resource \"leases\" in API group \"coordination.k8s.io\" in the namespace \"kube-node-lease\"" interval="200ms"
Sep 17 23:10:49.543267 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:10:49.543216    2562 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv6"
Sep 17 23:10:49.559787 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:10:49.559562    2562 manager.go:354] Recovery completed
Sep 17 23:10:49.565783 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:10:49.565758    2562 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Sep 17 23:10:49.571337 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:10:49.571316    2562 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-132-188.ec2.internal" event="NodeHasSufficientMemory"
Sep 17 23:10:49.571432 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:10:49.571348    2562 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-132-188.ec2.internal" event="NodeHasNoDiskPressure"
Sep 17 23:10:49.571432 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:10:49.571359    2562 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-132-188.ec2.internal" event="NodeHasSufficientPID"
Sep 17 23:10:49.571934 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:10:49.571920    2562 cpu_manager.go:222] "Starting CPU manager" policy="none"
Sep 17 23:10:49.571934 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:10:49.571932    2562 cpu_manager.go:223] "Reconciling" reconcilePeriod="10s"
Sep 17 23:10:49.571934 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:10:49.571935    2562 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Sep 17 23:10:49.572121 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:10:49.571948    2562 state_mem.go:36] "Initialized new in-memory state store"
Sep 17 23:10:49.572688 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:10:49.572663    2562 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-132-188.ec2.internal" event="NodeHasSufficientMemory"
Sep 17 23:10:49.572752 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:10:49.572693    2562 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-132-188.ec2.internal" event="NodeHasNoDiskPressure"
Sep 17 23:10:49.572752 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:10:49.572707    2562 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-132-188.ec2.internal" event="NodeHasSufficientPID"
Sep 17 23:10:49.573998 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:10:49.573986    2562 policy_none.go:49] "None policy: Start"
Sep 17 23:10:49.574043 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:10:49.574003    2562 memory_manager.go:187] "Starting memorymanager" policy="None"
Sep 17 23:10:49.574043 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:10:49.574013    2562 state_mem.go:36] "Initializing new in-memory state store" logger="Memory Manager state checkpoint"
Sep 17 23:10:49.576008 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:10:49.575995    2562 policy_none.go:47] "Start"
Sep 17 23:10:49.610873 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:10:49.610848    2562 manager.go:337] "Starting Device Plugin manager"
Sep 17 23:10:49.611013 ip-10-0-132-188 kubenswrapper[2562]: E0917 23:10:49.610901    2562 manager.go:513] "Failed to read data from checkpoint" err="checkpoint is not found" checkpoint="kubelet_internal_checkpoint"
Sep 17 23:10:49.611013 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:10:49.610916    2562 server.go:87] "Starting device plugin registration server"
Sep 17 23:10:49.611255 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:10:49.611237    2562 eviction_manager.go:189] "Eviction manager: starting control loop"
Sep 17 23:10:49.611901 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:10:49.611266    2562 container_log_manager.go:146] "Initializing container log rotate workers" workers=1 monitorPeriod="10s"
Sep 17 23:10:49.611901 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:10:49.611471    2562 plugin_watcher.go:51] "Plugin Watcher Start" path="/var/lib/kubelet/plugins_registry"
Sep 17 23:10:49.611901 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:10:49.611551    2562 plugin_manager.go:116] "The desired_state_of_world populator (plugin watcher) starts"
Sep 17 23:10:49.611901 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:10:49.611559    2562 plugin_manager.go:118] "Starting Kubelet Plugin Manager"
Sep 17 23:10:49.612113 ip-10-0-132-188 kubenswrapper[2562]: E0917 23:10:49.612055    2562 eviction_manager.go:267] "eviction manager: failed to check if we have separate container filesystem. Ignoring." err="non-existent label \"crio-containers\""
Sep 17 23:10:49.612113 ip-10-0-132-188 kubenswrapper[2562]: E0917 23:10:49.612107    2562 eviction_manager.go:292] "Eviction manager: failed to get summary stats" err="failed to get node info: node \"ip-10-0-132-188.ec2.internal\" not found"
Sep 17 23:10:49.660355 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:10:49.660326    2562 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv4"
Sep 17 23:10:49.662499 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:10:49.660378    2562 status_manager.go:244] "Starting to sync pod status with apiserver"
Sep 17 23:10:49.662499 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:10:49.660405    2562 kubelet.go:2448] "Starting kubelet main sync loop"
Sep 17 23:10:49.662499 ip-10-0-132-188 kubenswrapper[2562]: E0917 23:10:49.660443    2562 kubelet.go:2472] "Skipping pod synchronization" err="PLEG is not healthy: pleg has yet to be successful"
Sep 17 23:10:49.663110 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:10:49.663080    2562 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160"
Sep 17 23:10:49.711447 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:10:49.711364    2562 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Sep 17 23:10:49.713612 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:10:49.713586    2562 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-132-188.ec2.internal" event="NodeHasSufficientMemory"
Sep 17 23:10:49.713759 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:10:49.713622    2562 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-132-188.ec2.internal" event="NodeHasNoDiskPressure"
Sep 17 23:10:49.713759 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:10:49.713633    2562 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-132-188.ec2.internal" event="NodeHasSufficientPID"
Sep 17 23:10:49.713759 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:10:49.713659    2562 kubelet_node_status.go:78] "Attempting to register node" node="ip-10-0-132-188.ec2.internal"
Sep 17 23:10:49.722741 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:10:49.722718    2562 kubelet_node_status.go:81] "Successfully registered node" node="ip-10-0-132-188.ec2.internal"
Sep 17 23:10:49.722832 ip-10-0-132-188 kubenswrapper[2562]: E0917 23:10:49.722746    2562 kubelet_node_status.go:536] "Error updating node status, will retry" err="error getting node \"ip-10-0-132-188.ec2.internal\": node \"ip-10-0-132-188.ec2.internal\" not found"
Sep 17 23:10:49.738441 ip-10-0-132-188 kubenswrapper[2562]: E0917 23:10:49.738408    2562 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-132-188.ec2.internal\" not found"
Sep 17 23:10:49.760788 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:10:49.760757    2562 kubelet.go:2535] "SyncLoop ADD" source="file" pods=["openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-132-188.ec2.internal","kube-system/kube-apiserver-proxy-ip-10-0-132-188.ec2.internal"]
Sep 17 23:10:49.760922 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:10:49.760832    2562 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Sep 17 23:10:49.761913 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:10:49.761894    2562 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-132-188.ec2.internal" event="NodeHasSufficientMemory"
Sep 17 23:10:49.762005 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:10:49.761932    2562 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-132-188.ec2.internal" event="NodeHasNoDiskPressure"
Sep 17 23:10:49.762005 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:10:49.761944    2562 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-132-188.ec2.internal" event="NodeHasSufficientPID"
Sep 17 23:10:49.764203 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:10:49.764189    2562 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Sep 17 23:10:49.764339 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:10:49.764315    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-132-188.ec2.internal"
Sep 17 23:10:49.764395 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:10:49.764353    2562 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Sep 17 23:10:49.765142 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:10:49.765117    2562 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-132-188.ec2.internal" event="NodeHasSufficientMemory"
Sep 17 23:10:49.765240 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:10:49.765146    2562 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-132-188.ec2.internal" event="NodeHasNoDiskPressure"
Sep 17 23:10:49.765240 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:10:49.765159    2562 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-132-188.ec2.internal" event="NodeHasSufficientPID"
Sep 17 23:10:49.765240 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:10:49.765121    2562 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-132-188.ec2.internal" event="NodeHasSufficientMemory"
Sep 17 23:10:49.765240 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:10:49.765215    2562 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-132-188.ec2.internal" event="NodeHasNoDiskPressure"
Sep 17 23:10:49.765240 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:10:49.765225    2562 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-132-188.ec2.internal" event="NodeHasSufficientPID"
Sep 17 23:10:49.767415 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:10:49.767399    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-132-188.ec2.internal"
Sep 17 23:10:49.767503 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:10:49.767425    2562 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Sep 17 23:10:49.768136 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:10:49.768119    2562 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-132-188.ec2.internal" event="NodeHasSufficientMemory"
Sep 17 23:10:49.768244 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:10:49.768153    2562 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-132-188.ec2.internal" event="NodeHasNoDiskPressure"
Sep 17 23:10:49.768244 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:10:49.768186    2562 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-132-188.ec2.internal" event="NodeHasSufficientPID"
Sep 17 23:10:49.793115 ip-10-0-132-188 kubenswrapper[2562]: E0917 23:10:49.793088    2562 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-132-188.ec2.internal\" not found" node="ip-10-0-132-188.ec2.internal"
Sep 17 23:10:49.797750 ip-10-0-132-188 kubenswrapper[2562]: E0917 23:10:49.797727    2562 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-132-188.ec2.internal\" not found" node="ip-10-0-132-188.ec2.internal"
Sep 17 23:10:49.835462 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:10:49.835429    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/3724b678d11f0286466e70c69db04952-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-132-188.ec2.internal\" (UID: \"3724b678d11f0286466e70c69db04952\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-132-188.ec2.internal"
Sep 17 23:10:49.835462 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:10:49.835461    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/380b7a798346051e115745fb0a3f8e3c-config\") pod \"kube-apiserver-proxy-ip-10-0-132-188.ec2.internal\" (UID: \"380b7a798346051e115745fb0a3f8e3c\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-132-188.ec2.internal"
Sep 17 23:10:49.835661 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:10:49.835478    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/3724b678d11f0286466e70c69db04952-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-132-188.ec2.internal\" (UID: \"3724b678d11f0286466e70c69db04952\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-132-188.ec2.internal"
Sep 17 23:10:49.839486 ip-10-0-132-188 kubenswrapper[2562]: E0917 23:10:49.839463    2562 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-132-188.ec2.internal\" not found"
Sep 17 23:10:49.936265 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:10:49.936227    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/3724b678d11f0286466e70c69db04952-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-132-188.ec2.internal\" (UID: \"3724b678d11f0286466e70c69db04952\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-132-188.ec2.internal"
Sep 17 23:10:49.936265 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:10:49.936263    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/3724b678d11f0286466e70c69db04952-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-132-188.ec2.internal\" (UID: \"3724b678d11f0286466e70c69db04952\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-132-188.ec2.internal"
Sep 17 23:10:49.936474 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:10:49.936279    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/380b7a798346051e115745fb0a3f8e3c-config\") pod \"kube-apiserver-proxy-ip-10-0-132-188.ec2.internal\" (UID: \"380b7a798346051e115745fb0a3f8e3c\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-132-188.ec2.internal"
Sep 17 23:10:49.936474 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:10:49.936309    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/host-path/380b7a798346051e115745fb0a3f8e3c-config\") pod \"kube-apiserver-proxy-ip-10-0-132-188.ec2.internal\" (UID: \"380b7a798346051e115745fb0a3f8e3c\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-132-188.ec2.internal"
Sep 17 23:10:49.936474 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:10:49.936327    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/3724b678d11f0286466e70c69db04952-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-132-188.ec2.internal\" (UID: \"3724b678d11f0286466e70c69db04952\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-132-188.ec2.internal"
Sep 17 23:10:49.936474 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:10:49.936328    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/3724b678d11f0286466e70c69db04952-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-132-188.ec2.internal\" (UID: \"3724b678d11f0286466e70c69db04952\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-132-188.ec2.internal"
Sep 17 23:10:49.940331 ip-10-0-132-188 kubenswrapper[2562]: E0917 23:10:49.940309    2562 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-132-188.ec2.internal\" not found"
Sep 17 23:10:50.041274 ip-10-0-132-188 kubenswrapper[2562]: E0917 23:10:50.041200    2562 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-132-188.ec2.internal\" not found"
Sep 17 23:10:50.097397 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:10:50.097361    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-132-188.ec2.internal"
Sep 17 23:10:50.101004 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:10:50.100984    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-132-188.ec2.internal"
Sep 17 23:10:50.141512 ip-10-0-132-188 kubenswrapper[2562]: E0917 23:10:50.141472    2562 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-132-188.ec2.internal\" not found"
Sep 17 23:10:50.242003 ip-10-0-132-188 kubenswrapper[2562]: E0917 23:10:50.241957    2562 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-132-188.ec2.internal\" not found"
Sep 17 23:10:50.342602 ip-10-0-132-188 kubenswrapper[2562]: E0917 23:10:50.342520    2562 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-132-188.ec2.internal\" not found"
Sep 17 23:10:50.407056 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:10:50.407026    2562 reflector.go:439] "Caches populated" type="*v1.Node" reflector="k8s.io/client-go/informers/factory.go:160"
Sep 17 23:10:50.433384 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:10:50.433350    2562 kubelet.go:3248] "Creating a mirror pod for static pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-132-188.ec2.internal"
Sep 17 23:10:50.445406 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:10:50.445383    2562 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]"
Sep 17 23:10:50.446891 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:10:50.446877    2562 kubelet.go:3248] "Creating a mirror pod for static pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-132-188.ec2.internal"
Sep 17 23:10:50.455426 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:10:50.455404    2562 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]"
Sep 17 23:10:50.480564 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:10:50.480536    2562 transport.go:147] "Certificate rotation detected, shutting down client connections to start using new credentials"
Sep 17 23:10:50.480977 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:10:50.480677    2562 reflector.go:571] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.RuntimeClass" err="very short watch: k8s.io/client-go/informers/factory.go:160: Unexpected watch close - watch lasted less than a second and no items received"
Sep 17 23:10:50.480977 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:10:50.480716    2562 reflector.go:571] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Node" err="very short watch: k8s.io/client-go/informers/factory.go:160: Unexpected watch close - watch lasted less than a second and no items received"
Sep 17 23:10:50.504869 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:10:50.504838    2562 apiserver.go:52] "Watching apiserver"
Sep 17 23:10:50.519157 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:10:50.519125    2562 reflector.go:439] "Caches populated" type="*v1.Pod" reflector="pkg/kubelet/config/apiserver.go:66"
Sep 17 23:10:50.519508 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:10:50.519486    2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kube-system/konnectivity-agent-fwwc5","openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-2cj88","openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-132-188.ec2.internal","openshift-multus/multus-s9wtj","openshift-network-operator/iptables-alerter-bvb88","openshift-ovn-kubernetes/ovnkube-node-6xqwg","kube-system/global-pull-secret-syncer-jt864","kube-system/kube-apiserver-proxy-ip-10-0-132-188.ec2.internal","openshift-cluster-node-tuning-operator/tuned-g9rzj","openshift-dns/node-resolver-fzpjd","openshift-image-registry/node-ca-vkllk","openshift-multus/multus-additional-cni-plugins-srxwj","openshift-multus/network-metrics-daemon-nbccd","openshift-network-diagnostics/network-check-target-t9gqw"]
Sep 17 23:10:50.521947 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:10:50.521930    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-jt864"
Sep 17 23:10:50.522010 ip-10-0-132-188 kubenswrapper[2562]: E0917 23:10:50.521995    2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-jt864" podUID="79765acc-1a30-4a47-add5-aea2bf5f9efb"
Sep 17 23:10:50.523915 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:10:50.523894    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-nbccd"
Sep 17 23:10:50.524027 ip-10-0-132-188 kubenswrapper[2562]: E0917 23:10:50.523955    2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-nbccd" podUID="4453ba96-a67f-46af-b61b-9cd8f39b6d36"
Sep 17 23:10:50.526010 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:10:50.525990    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-t9gqw"
Sep 17 23:10:50.526109 ip-10-0-132-188 kubenswrapper[2562]: E0917 23:10:50.526067    2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-t9gqw" podUID="440bce33-9c27-46bd-9d1c-0eebcc14fbc3"
Sep 17 23:10:50.529433 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:10:50.529416    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-s9wtj"
Sep 17 23:10:50.531775 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:10:50.531754    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"openshift-service-ca.crt\""
Sep 17 23:10:50.531868 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:10:50.531782    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"cni-copy-resources\""
Sep 17 23:10:50.531868 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:10:50.531782    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"multus-daemon-config\""
Sep 17 23:10:50.532048 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:10:50.532031    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"kube-root-ca.crt\""
Sep 17 23:10:50.532117 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:10:50.532070    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"default-dockercfg-f58g8\""
Sep 17 23:10:50.532237 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:10:50.532226    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-bvb88"
Sep 17 23:10:50.532845 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:10:50.532830    2562 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kubelet-serving"
Sep 17 23:10:50.534441 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:10:50.534425    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-6xqwg"
Sep 17 23:10:50.534813 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:10:50.534791    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"openshift-service-ca.crt\""
Sep 17 23:10:50.534921 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:10:50.534885    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-script\""
Sep 17 23:10:50.535102 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:10:50.535087    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-dockercfg-nxstj\""
Sep 17 23:10:50.535847 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:10:50.535832    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"kube-root-ca.crt\""
Sep 17 23:10:50.536720 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:10:50.536701    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-g9rzj"
Sep 17 23:10:50.538594 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:10:50.538573    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"kube-root-ca.crt\""
Sep 17 23:10:50.538660 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:10:50.538626    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-script-lib\""
Sep 17 23:10:50.538869 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:10:50.538855    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-node-metrics-cert\""
Sep 17 23:10:50.538869 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:10:50.538864    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-config\""
Sep 17 23:10:50.538979 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:10:50.538938    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-fwwc5"
Sep 17 23:10:50.539292 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:10:50.539276    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-nzkzp\" (UniqueName: \"kubernetes.io/projected/914194c1-d316-4f82-9beb-af7827f4a570-kube-api-access-nzkzp\") pod \"multus-s9wtj\" (UID: \"914194c1-d316-4f82-9beb-af7827f4a570\") " pod="openshift-multus/multus-s9wtj"
Sep 17 23:10:50.539372 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:10:50.539298    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/cae8ed58-1a8a-40af-a329-a3850797cc58-host-kubelet\") pod \"ovnkube-node-6xqwg\" (UID: \"cae8ed58-1a8a-40af-a329-a3850797cc58\") " pod="openshift-ovn-kubernetes/ovnkube-node-6xqwg"
Sep 17 23:10:50.539372 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:10:50.539316    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/cae8ed58-1a8a-40af-a329-a3850797cc58-host-slash\") pod \"ovnkube-node-6xqwg\" (UID: \"cae8ed58-1a8a-40af-a329-a3850797cc58\") " pod="openshift-ovn-kubernetes/ovnkube-node-6xqwg"
Sep 17 23:10:50.539372 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:10:50.539320    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-kubernetes-node-dockercfg-zc246\""
Sep 17 23:10:50.539372 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:10:50.539338    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/cae8ed58-1a8a-40af-a329-a3850797cc58-host-cni-bin\") pod \"ovnkube-node-6xqwg\" (UID: \"cae8ed58-1a8a-40af-a329-a3850797cc58\") " pod="openshift-ovn-kubernetes/ovnkube-node-6xqwg"
Sep 17 23:10:50.539372 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:10:50.539368    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/cae8ed58-1a8a-40af-a329-a3850797cc58-ovn-node-metrics-cert\") pod \"ovnkube-node-6xqwg\" (UID: \"cae8ed58-1a8a-40af-a329-a3850797cc58\") " pod="openshift-ovn-kubernetes/ovnkube-node-6xqwg"
Sep 17 23:10:50.539627 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:10:50.539399    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-42cvm\" (UniqueName: \"kubernetes.io/projected/440bce33-9c27-46bd-9d1c-0eebcc14fbc3-kube-api-access-42cvm\") pod \"network-check-target-t9gqw\" (UID: \"440bce33-9c27-46bd-9d1c-0eebcc14fbc3\") " pod="openshift-network-diagnostics/network-check-target-t9gqw"
Sep 17 23:10:50.539627 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:10:50.539423    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/914194c1-d316-4f82-9beb-af7827f4a570-host-run-multus-certs\") pod \"multus-s9wtj\" (UID: \"914194c1-d316-4f82-9beb-af7827f4a570\") " pod="openshift-multus/multus-s9wtj"
Sep 17 23:10:50.539627 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:10:50.539443    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/cae8ed58-1a8a-40af-a329-a3850797cc58-run-systemd\") pod \"ovnkube-node-6xqwg\" (UID: \"cae8ed58-1a8a-40af-a329-a3850797cc58\") " pod="openshift-ovn-kubernetes/ovnkube-node-6xqwg"
Sep 17 23:10:50.539627 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:10:50.539479    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/cae8ed58-1a8a-40af-a329-a3850797cc58-var-lib-openvswitch\") pod \"ovnkube-node-6xqwg\" (UID: \"cae8ed58-1a8a-40af-a329-a3850797cc58\") " pod="openshift-ovn-kubernetes/ovnkube-node-6xqwg"
Sep 17 23:10:50.539627 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:10:50.539510    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/cae8ed58-1a8a-40af-a329-a3850797cc58-run-openvswitch\") pod \"ovnkube-node-6xqwg\" (UID: \"cae8ed58-1a8a-40af-a329-a3850797cc58\") " pod="openshift-ovn-kubernetes/ovnkube-node-6xqwg"
Sep 17 23:10:50.539627 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:10:50.539522    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"env-overrides\""
Sep 17 23:10:50.539627 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:10:50.539541    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/cae8ed58-1a8a-40af-a329-a3850797cc58-host-cni-netd\") pod \"ovnkube-node-6xqwg\" (UID: \"cae8ed58-1a8a-40af-a329-a3850797cc58\") " pod="openshift-ovn-kubernetes/ovnkube-node-6xqwg"
Sep 17 23:10:50.539627 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:10:50.539558    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-g67q7\" (UniqueName: \"kubernetes.io/projected/cae8ed58-1a8a-40af-a329-a3850797cc58-kube-api-access-g67q7\") pod \"ovnkube-node-6xqwg\" (UID: \"cae8ed58-1a8a-40af-a329-a3850797cc58\") " pod="openshift-ovn-kubernetes/ovnkube-node-6xqwg"
Sep 17 23:10:50.539627 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:10:50.539605    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/4453ba96-a67f-46af-b61b-9cd8f39b6d36-metrics-certs\") pod \"network-metrics-daemon-nbccd\" (UID: \"4453ba96-a67f-46af-b61b-9cd8f39b6d36\") " pod="openshift-multus/network-metrics-daemon-nbccd"
Sep 17 23:10:50.540047 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:10:50.539639    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/914194c1-d316-4f82-9beb-af7827f4a570-multus-conf-dir\") pod \"multus-s9wtj\" (UID: \"914194c1-d316-4f82-9beb-af7827f4a570\") " pod="openshift-multus/multus-s9wtj"
Sep 17 23:10:50.540047 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:10:50.539668    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/914194c1-d316-4f82-9beb-af7827f4a570-etc-kubernetes\") pod \"multus-s9wtj\" (UID: \"914194c1-d316-4f82-9beb-af7827f4a570\") " pod="openshift-multus/multus-s9wtj"
Sep 17 23:10:50.540047 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:10:50.539702    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/cae8ed58-1a8a-40af-a329-a3850797cc58-host-run-ovn-kubernetes\") pod \"ovnkube-node-6xqwg\" (UID: \"cae8ed58-1a8a-40af-a329-a3850797cc58\") " pod="openshift-ovn-kubernetes/ovnkube-node-6xqwg"
Sep 17 23:10:50.540047 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:10:50.539743    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/cae8ed58-1a8a-40af-a329-a3850797cc58-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-6xqwg\" (UID: \"cae8ed58-1a8a-40af-a329-a3850797cc58\") " pod="openshift-ovn-kubernetes/ovnkube-node-6xqwg"
Sep 17 23:10:50.540047 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:10:50.539767    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/79765acc-1a30-4a47-add5-aea2bf5f9efb-original-pull-secret\") pod \"global-pull-secret-syncer-jt864\" (UID: \"79765acc-1a30-4a47-add5-aea2bf5f9efb\") " pod="kube-system/global-pull-secret-syncer-jt864"
Sep 17 23:10:50.540047 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:10:50.539782    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/914194c1-d316-4f82-9beb-af7827f4a570-system-cni-dir\") pod \"multus-s9wtj\" (UID: \"914194c1-d316-4f82-9beb-af7827f4a570\") " pod="openshift-multus/multus-s9wtj"
Sep 17 23:10:50.540047 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:10:50.539815    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/914194c1-d316-4f82-9beb-af7827f4a570-cnibin\") pod \"multus-s9wtj\" (UID: \"914194c1-d316-4f82-9beb-af7827f4a570\") " pod="openshift-multus/multus-s9wtj"
Sep 17 23:10:50.540047 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:10:50.539838    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/9fd2f397-d1b2-4790-914c-4e9b860b827a-host-slash\") pod \"iptables-alerter-bvb88\" (UID: \"9fd2f397-d1b2-4790-914c-4e9b860b827a\") " pod="openshift-network-operator/iptables-alerter-bvb88"
Sep 17 23:10:50.540047 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:10:50.539878    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qnkg2\" (UniqueName: \"kubernetes.io/projected/9fd2f397-d1b2-4790-914c-4e9b860b827a-kube-api-access-qnkg2\") pod \"iptables-alerter-bvb88\" (UID: \"9fd2f397-d1b2-4790-914c-4e9b860b827a\") " pod="openshift-network-operator/iptables-alerter-bvb88"
Sep 17 23:10:50.540047 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:10:50.539898    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"openshift-service-ca.crt\""
Sep 17 23:10:50.540047 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:10:50.539902    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/cae8ed58-1a8a-40af-a329-a3850797cc58-systemd-units\") pod \"ovnkube-node-6xqwg\" (UID: \"cae8ed58-1a8a-40af-a329-a3850797cc58\") " pod="openshift-ovn-kubernetes/ovnkube-node-6xqwg"
Sep 17 23:10:50.540047 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:10:50.539904    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"openshift-service-ca.crt\""
Sep 17 23:10:50.540047 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:10:50.539878    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"tuned-dockercfg-xhl8z\""
Sep 17 23:10:50.540047 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:10:50.539926    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/cae8ed58-1a8a-40af-a329-a3850797cc58-etc-openvswitch\") pod \"ovnkube-node-6xqwg\" (UID: \"cae8ed58-1a8a-40af-a329-a3850797cc58\") " pod="openshift-ovn-kubernetes/ovnkube-node-6xqwg"
Sep 17 23:10:50.540047 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:10:50.539999    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/cae8ed58-1a8a-40af-a329-a3850797cc58-log-socket\") pod \"ovnkube-node-6xqwg\" (UID: \"cae8ed58-1a8a-40af-a329-a3850797cc58\") " pod="openshift-ovn-kubernetes/ovnkube-node-6xqwg"
Sep 17 23:10:50.540047 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:10:50.540016    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/914194c1-d316-4f82-9beb-af7827f4a570-cni-binary-copy\") pod \"multus-s9wtj\" (UID: \"914194c1-d316-4f82-9beb-af7827f4a570\") " pod="openshift-multus/multus-s9wtj"
Sep 17 23:10:50.540047 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:10:50.540032    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/914194c1-d316-4f82-9beb-af7827f4a570-hostroot\") pod \"multus-s9wtj\" (UID: \"914194c1-d316-4f82-9beb-af7827f4a570\") " pod="openshift-multus/multus-s9wtj"
Sep 17 23:10:50.540047 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:10:50.540015    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"kube-root-ca.crt\""
Sep 17 23:10:50.540751 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:10:50.540077    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/cae8ed58-1a8a-40af-a329-a3850797cc58-run-ovn\") pod \"ovnkube-node-6xqwg\" (UID: \"cae8ed58-1a8a-40af-a329-a3850797cc58\") " pod="openshift-ovn-kubernetes/ovnkube-node-6xqwg"
Sep 17 23:10:50.540751 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:10:50.540104    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/cae8ed58-1a8a-40af-a329-a3850797cc58-env-overrides\") pod \"ovnkube-node-6xqwg\" (UID: \"cae8ed58-1a8a-40af-a329-a3850797cc58\") " pod="openshift-ovn-kubernetes/ovnkube-node-6xqwg"
Sep 17 23:10:50.540751 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:10:50.540131    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/914194c1-d316-4f82-9beb-af7827f4a570-multus-daemon-config\") pod \"multus-s9wtj\" (UID: \"914194c1-d316-4f82-9beb-af7827f4a570\") " pod="openshift-multus/multus-s9wtj"
Sep 17 23:10:50.540751 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:10:50.540148    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/cae8ed58-1a8a-40af-a329-a3850797cc58-host-run-netns\") pod \"ovnkube-node-6xqwg\" (UID: \"cae8ed58-1a8a-40af-a329-a3850797cc58\") " pod="openshift-ovn-kubernetes/ovnkube-node-6xqwg"
Sep 17 23:10:50.540751 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:10:50.540162    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/cae8ed58-1a8a-40af-a329-a3850797cc58-ovnkube-config\") pod \"ovnkube-node-6xqwg\" (UID: \"cae8ed58-1a8a-40af-a329-a3850797cc58\") " pod="openshift-ovn-kubernetes/ovnkube-node-6xqwg"
Sep 17 23:10:50.540751 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:10:50.540208    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/79765acc-1a30-4a47-add5-aea2bf5f9efb-dbus\") pod \"global-pull-secret-syncer-jt864\" (UID: \"79765acc-1a30-4a47-add5-aea2bf5f9efb\") " pod="kube-system/global-pull-secret-syncer-jt864"
Sep 17 23:10:50.540751 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:10:50.540227    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/914194c1-d316-4f82-9beb-af7827f4a570-host-run-k8s-cni-cncf-io\") pod \"multus-s9wtj\" (UID: \"914194c1-d316-4f82-9beb-af7827f4a570\") " pod="openshift-multus/multus-s9wtj"
Sep 17 23:10:50.540751 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:10:50.540244    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/914194c1-d316-4f82-9beb-af7827f4a570-host-run-netns\") pod \"multus-s9wtj\" (UID: \"914194c1-d316-4f82-9beb-af7827f4a570\") " pod="openshift-multus/multus-s9wtj"
Sep 17 23:10:50.540751 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:10:50.540265    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/914194c1-d316-4f82-9beb-af7827f4a570-host-var-lib-cni-bin\") pod \"multus-s9wtj\" (UID: \"914194c1-d316-4f82-9beb-af7827f4a570\") " pod="openshift-multus/multus-s9wtj"
Sep 17 23:10:50.540751 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:10:50.540292    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/9fd2f397-d1b2-4790-914c-4e9b860b827a-iptables-alerter-script\") pod \"iptables-alerter-bvb88\" (UID: \"9fd2f397-d1b2-4790-914c-4e9b860b827a\") " pod="openshift-network-operator/iptables-alerter-bvb88"
Sep 17 23:10:50.540751 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:10:50.540327    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/cae8ed58-1a8a-40af-a329-a3850797cc58-ovnkube-script-lib\") pod \"ovnkube-node-6xqwg\" (UID: \"cae8ed58-1a8a-40af-a329-a3850797cc58\") " pod="openshift-ovn-kubernetes/ovnkube-node-6xqwg"
Sep 17 23:10:50.540751 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:10:50.540352    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/79765acc-1a30-4a47-add5-aea2bf5f9efb-kubelet-config\") pod \"global-pull-secret-syncer-jt864\" (UID: \"79765acc-1a30-4a47-add5-aea2bf5f9efb\") " pod="kube-system/global-pull-secret-syncer-jt864"
Sep 17 23:10:50.540751 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:10:50.540375    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/914194c1-d316-4f82-9beb-af7827f4a570-multus-cni-dir\") pod \"multus-s9wtj\" (UID: \"914194c1-d316-4f82-9beb-af7827f4a570\") " pod="openshift-multus/multus-s9wtj"
Sep 17 23:10:50.540751 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:10:50.540398    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/914194c1-d316-4f82-9beb-af7827f4a570-host-var-lib-cni-multus\") pod \"multus-s9wtj\" (UID: \"914194c1-d316-4f82-9beb-af7827f4a570\") " pod="openshift-multus/multus-s9wtj"
Sep 17 23:10:50.540751 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:10:50.540420    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/cae8ed58-1a8a-40af-a329-a3850797cc58-node-log\") pod \"ovnkube-node-6xqwg\" (UID: \"cae8ed58-1a8a-40af-a329-a3850797cc58\") " pod="openshift-ovn-kubernetes/ovnkube-node-6xqwg"
Sep 17 23:10:50.540751 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:10:50.540462    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jrh7f\" (UniqueName: \"kubernetes.io/projected/4453ba96-a67f-46af-b61b-9cd8f39b6d36-kube-api-access-jrh7f\") pod \"network-metrics-daemon-nbccd\" (UID: \"4453ba96-a67f-46af-b61b-9cd8f39b6d36\") " pod="openshift-multus/network-metrics-daemon-nbccd"
Sep 17 23:10:50.541221 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:10:50.540488    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/914194c1-d316-4f82-9beb-af7827f4a570-multus-socket-dir-parent\") pod \"multus-s9wtj\" (UID: \"914194c1-d316-4f82-9beb-af7827f4a570\") " pod="openshift-multus/multus-s9wtj"
Sep 17 23:10:50.541221 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:10:50.540511    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/914194c1-d316-4f82-9beb-af7827f4a570-host-var-lib-kubelet\") pod \"multus-s9wtj\" (UID: \"914194c1-d316-4f82-9beb-af7827f4a570\") " pod="openshift-multus/multus-s9wtj"
Sep 17 23:10:50.541221 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:10:50.541104    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-2cj88"
Sep 17 23:10:50.541526 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:10:50.541513    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"default-dockercfg-sj2vg\""
Sep 17 23:10:50.541736 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:10:50.541722    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"konnectivity-agent\""
Sep 17 23:10:50.541838 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:10:50.541825    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kube-system\"/\"konnectivity-ca-bundle\""
Sep 17 23:10:50.543084 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:10:50.543069    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"openshift-service-ca.crt\""
Sep 17 23:10:50.543493 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:10:50.543457    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-sa-dockercfg-jsf4s\""
Sep 17 23:10:50.543493 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:10:50.543471    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-metrics-serving-cert\""
Sep 17 23:10:50.543626 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:10:50.543518    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"kube-root-ca.crt\""
Sep 17 23:10:50.543905 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:10:50.543891    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-fzpjd"
Sep 17 23:10:50.546107 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:10:50.546088    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"openshift-service-ca.crt\""
Sep 17 23:10:50.546202 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:10:50.546089    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"node-resolver-dockercfg-ktmvr\""
Sep 17 23:10:50.546202 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:10:50.546089    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"kube-root-ca.crt\""
Sep 17 23:10:50.546281 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:10:50.546183    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-vkllk"
Sep 17 23:10:50.548545 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:10:50.548528    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-srxwj"
Sep 17 23:10:50.549162 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:10:50.549141    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"kube-root-ca.crt\""
Sep 17 23:10:50.549287 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:10:50.549260    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"openshift-service-ca.crt\""
Sep 17 23:10:50.549366 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:10:50.549339    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"image-registry-certificates\""
Sep 17 23:10:50.549451 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:10:50.549434    2562 reflector.go:439] "Caches populated" logger="kubernetes.io/kubelet-serving" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162"
Sep 17 23:10:50.549508 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:10:50.549452    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"node-ca-dockercfg-n4jtd\""
Sep 17 23:10:50.551226 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:10:50.551211    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"default-cni-sysctl-allowlist\""
Sep 17 23:10:50.551283 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:10:50.551226    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"whereabouts-flatfile-config\""
Sep 17 23:10:50.551414 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:10:50.551399    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"multus-ancillary-tools-dockercfg-t9t5p\""
Sep 17 23:10:50.558946 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:10:50.558902    2562 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kube-apiserver-client-kubelet" expiration="2028-09-16 23:05:49 +0000 UTC" deadline="2028-03-24 19:53:31.719243262 +0000 UTC"
Sep 17 23:10:50.559005 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:10:50.558948    2562 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kube-apiserver-client-kubelet" sleep="13292h42m41.16029942s"
Sep 17 23:10:50.568458 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:10:50.568438    2562 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kubelet-serving" csr="csr-79pjd"
Sep 17 23:10:50.576423 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:10:50.576402    2562 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kubelet-serving" csr="csr-79pjd"
Sep 17 23:10:50.622219 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:10:50.622192    2562 reflector.go:439] "Caches populated" type="*v1.CSIDriver" reflector="k8s.io/client-go/informers/factory.go:160"
Sep 17 23:10:50.633890 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:10:50.633863    2562 desired_state_of_world_populator.go:154] "Finished populating initial desired state of world"
Sep 17 23:10:50.641197 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:10:50.641153    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/cae8ed58-1a8a-40af-a329-a3850797cc58-host-slash\") pod \"ovnkube-node-6xqwg\" (UID: \"cae8ed58-1a8a-40af-a329-a3850797cc58\") " pod="openshift-ovn-kubernetes/ovnkube-node-6xqwg"
Sep 17 23:10:50.641346 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:10:50.641200    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/cae8ed58-1a8a-40af-a329-a3850797cc58-host-cni-bin\") pod \"ovnkube-node-6xqwg\" (UID: \"cae8ed58-1a8a-40af-a329-a3850797cc58\") " pod="openshift-ovn-kubernetes/ovnkube-node-6xqwg"
Sep 17 23:10:50.641346 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:10:50.641220    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-42cvm\" (UniqueName: \"kubernetes.io/projected/440bce33-9c27-46bd-9d1c-0eebcc14fbc3-kube-api-access-42cvm\") pod \"network-check-target-t9gqw\" (UID: \"440bce33-9c27-46bd-9d1c-0eebcc14fbc3\") " pod="openshift-network-diagnostics/network-check-target-t9gqw"
Sep 17 23:10:50.641346 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:10:50.641243    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/be8438b1-0b71-4af1-9575-1f005b65a607-etc-selinux\") pod \"aws-ebs-csi-driver-node-2cj88\" (UID: \"be8438b1-0b71-4af1-9575-1f005b65a607\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-2cj88"
Sep 17 23:10:50.641346 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:10:50.641270    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/cae8ed58-1a8a-40af-a329-a3850797cc58-run-openvswitch\") pod \"ovnkube-node-6xqwg\" (UID: \"cae8ed58-1a8a-40af-a329-a3850797cc58\") " pod="openshift-ovn-kubernetes/ovnkube-node-6xqwg"
Sep 17 23:10:50.641346 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:10:50.641278    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/cae8ed58-1a8a-40af-a329-a3850797cc58-host-cni-bin\") pod \"ovnkube-node-6xqwg\" (UID: \"cae8ed58-1a8a-40af-a329-a3850797cc58\") " pod="openshift-ovn-kubernetes/ovnkube-node-6xqwg"
Sep 17 23:10:50.641346 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:10:50.641292    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/914194c1-d316-4f82-9beb-af7827f4a570-multus-conf-dir\") pod \"multus-s9wtj\" (UID: \"914194c1-d316-4f82-9beb-af7827f4a570\") " pod="openshift-multus/multus-s9wtj"
Sep 17 23:10:50.641346 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:10:50.641282    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/cae8ed58-1a8a-40af-a329-a3850797cc58-host-slash\") pod \"ovnkube-node-6xqwg\" (UID: \"cae8ed58-1a8a-40af-a329-a3850797cc58\") " pod="openshift-ovn-kubernetes/ovnkube-node-6xqwg"
Sep 17 23:10:50.641346 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:10:50.641316    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/d1311a9e-f175-47e1-b690-2c4164ad344b-etc-sysconfig\") pod \"tuned-g9rzj\" (UID: \"d1311a9e-f175-47e1-b690-2c4164ad344b\") " pod="openshift-cluster-node-tuning-operator/tuned-g9rzj"
Sep 17 23:10:50.641346 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:10:50.641336    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/cae8ed58-1a8a-40af-a329-a3850797cc58-run-openvswitch\") pod \"ovnkube-node-6xqwg\" (UID: \"cae8ed58-1a8a-40af-a329-a3850797cc58\") " pod="openshift-ovn-kubernetes/ovnkube-node-6xqwg"
Sep 17 23:10:50.641346 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:10:50.641349    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/914194c1-d316-4f82-9beb-af7827f4a570-multus-conf-dir\") pod \"multus-s9wtj\" (UID: \"914194c1-d316-4f82-9beb-af7827f4a570\") " pod="openshift-multus/multus-s9wtj"
Sep 17 23:10:50.641791 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:10:50.641340    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/d1311a9e-f175-47e1-b690-2c4164ad344b-etc-kubernetes\") pod \"tuned-g9rzj\" (UID: \"d1311a9e-f175-47e1-b690-2c4164ad344b\") " pod="openshift-cluster-node-tuning-operator/tuned-g9rzj"
Sep 17 23:10:50.641791 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:10:50.641394    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/d1311a9e-f175-47e1-b690-2c4164ad344b-etc-sysctl-d\") pod \"tuned-g9rzj\" (UID: \"d1311a9e-f175-47e1-b690-2c4164ad344b\") " pod="openshift-cluster-node-tuning-operator/tuned-g9rzj"
Sep 17 23:10:50.641791 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:10:50.641420    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/d1311a9e-f175-47e1-b690-2c4164ad344b-etc-tuned\") pod \"tuned-g9rzj\" (UID: \"d1311a9e-f175-47e1-b690-2c4164ad344b\") " pod="openshift-cluster-node-tuning-operator/tuned-g9rzj"
Sep 17 23:10:50.641791 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:10:50.641441    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vmqq9\" (UniqueName: \"kubernetes.io/projected/d1311a9e-f175-47e1-b690-2c4164ad344b-kube-api-access-vmqq9\") pod \"tuned-g9rzj\" (UID: \"d1311a9e-f175-47e1-b690-2c4164ad344b\") " pod="openshift-cluster-node-tuning-operator/tuned-g9rzj"
Sep 17 23:10:50.641791 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:10:50.641468    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/cae8ed58-1a8a-40af-a329-a3850797cc58-host-run-ovn-kubernetes\") pod \"ovnkube-node-6xqwg\" (UID: \"cae8ed58-1a8a-40af-a329-a3850797cc58\") " pod="openshift-ovn-kubernetes/ovnkube-node-6xqwg"
Sep 17 23:10:50.641791 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:10:50.641490    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/914194c1-d316-4f82-9beb-af7827f4a570-cnibin\") pod \"multus-s9wtj\" (UID: \"914194c1-d316-4f82-9beb-af7827f4a570\") " pod="openshift-multus/multus-s9wtj"
Sep 17 23:10:50.641791 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:10:50.641514    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/9fd2f397-d1b2-4790-914c-4e9b860b827a-host-slash\") pod \"iptables-alerter-bvb88\" (UID: \"9fd2f397-d1b2-4790-914c-4e9b860b827a\") " pod="openshift-network-operator/iptables-alerter-bvb88"
Sep 17 23:10:50.641791 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:10:50.641522    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/cae8ed58-1a8a-40af-a329-a3850797cc58-host-run-ovn-kubernetes\") pod \"ovnkube-node-6xqwg\" (UID: \"cae8ed58-1a8a-40af-a329-a3850797cc58\") " pod="openshift-ovn-kubernetes/ovnkube-node-6xqwg"
Sep 17 23:10:50.641791 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:10:50.641536    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/d1311a9e-f175-47e1-b690-2c4164ad344b-etc-sysctl-conf\") pod \"tuned-g9rzj\" (UID: \"d1311a9e-f175-47e1-b690-2c4164ad344b\") " pod="openshift-cluster-node-tuning-operator/tuned-g9rzj"
Sep 17 23:10:50.641791 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:10:50.641554    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/33db9205-3be0-47dc-ac16-e8f623e43afb-host\") pod \"node-ca-vkllk\" (UID: \"33db9205-3be0-47dc-ac16-e8f623e43afb\") " pod="openshift-image-registry/node-ca-vkllk"
Sep 17 23:10:50.641791 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:10:50.641571    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/914194c1-d316-4f82-9beb-af7827f4a570-cnibin\") pod \"multus-s9wtj\" (UID: \"914194c1-d316-4f82-9beb-af7827f4a570\") " pod="openshift-multus/multus-s9wtj"
Sep 17 23:10:50.641791 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:10:50.641574    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/cae8ed58-1a8a-40af-a329-a3850797cc58-etc-openvswitch\") pod \"ovnkube-node-6xqwg\" (UID: \"cae8ed58-1a8a-40af-a329-a3850797cc58\") " pod="openshift-ovn-kubernetes/ovnkube-node-6xqwg"
Sep 17 23:10:50.641791 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:10:50.641605    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/914194c1-d316-4f82-9beb-af7827f4a570-hostroot\") pod \"multus-s9wtj\" (UID: \"914194c1-d316-4f82-9beb-af7827f4a570\") " pod="openshift-multus/multus-s9wtj"
Sep 17 23:10:50.641791 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:10:50.641618    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/9fd2f397-d1b2-4790-914c-4e9b860b827a-host-slash\") pod \"iptables-alerter-bvb88\" (UID: \"9fd2f397-d1b2-4790-914c-4e9b860b827a\") " pod="openshift-network-operator/iptables-alerter-bvb88"
Sep 17 23:10:50.641791 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:10:50.641632    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/215c7cbb-4cd1-47be-9a44-26c1801b3fcb-cnibin\") pod \"multus-additional-cni-plugins-srxwj\" (UID: \"215c7cbb-4cd1-47be-9a44-26c1801b3fcb\") " pod="openshift-multus/multus-additional-cni-plugins-srxwj"
Sep 17 23:10:50.641791 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:10:50.641630    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/cae8ed58-1a8a-40af-a329-a3850797cc58-etc-openvswitch\") pod \"ovnkube-node-6xqwg\" (UID: \"cae8ed58-1a8a-40af-a329-a3850797cc58\") " pod="openshift-ovn-kubernetes/ovnkube-node-6xqwg"
Sep 17 23:10:50.641791 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:10:50.641662    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/914194c1-d316-4f82-9beb-af7827f4a570-hostroot\") pod \"multus-s9wtj\" (UID: \"914194c1-d316-4f82-9beb-af7827f4a570\") " pod="openshift-multus/multus-s9wtj"
Sep 17 23:10:50.642628 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:10:50.641660    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xvqw9\" (UniqueName: \"kubernetes.io/projected/215c7cbb-4cd1-47be-9a44-26c1801b3fcb-kube-api-access-xvqw9\") pod \"multus-additional-cni-plugins-srxwj\" (UID: \"215c7cbb-4cd1-47be-9a44-26c1801b3fcb\") " pod="openshift-multus/multus-additional-cni-plugins-srxwj"
Sep 17 23:10:50.642628 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:10:50.641700    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/cae8ed58-1a8a-40af-a329-a3850797cc58-run-ovn\") pod \"ovnkube-node-6xqwg\" (UID: \"cae8ed58-1a8a-40af-a329-a3850797cc58\") " pod="openshift-ovn-kubernetes/ovnkube-node-6xqwg"
Sep 17 23:10:50.642628 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:10:50.641717    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/914194c1-d316-4f82-9beb-af7827f4a570-multus-daemon-config\") pod \"multus-s9wtj\" (UID: \"914194c1-d316-4f82-9beb-af7827f4a570\") " pod="openshift-multus/multus-s9wtj"
Sep 17 23:10:50.642628 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:10:50.641734    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-h8psz\" (UniqueName: \"kubernetes.io/projected/1d92dc9b-45b6-4912-bf69-f0961e86e8ed-kube-api-access-h8psz\") pod \"node-resolver-fzpjd\" (UID: \"1d92dc9b-45b6-4912-bf69-f0961e86e8ed\") " pod="openshift-dns/node-resolver-fzpjd"
Sep 17 23:10:50.642628 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:10:50.641770    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/79765acc-1a30-4a47-add5-aea2bf5f9efb-dbus\") pod \"global-pull-secret-syncer-jt864\" (UID: \"79765acc-1a30-4a47-add5-aea2bf5f9efb\") " pod="kube-system/global-pull-secret-syncer-jt864"
Sep 17 23:10:50.642628 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:10:50.641798    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/914194c1-d316-4f82-9beb-af7827f4a570-host-run-netns\") pod \"multus-s9wtj\" (UID: \"914194c1-d316-4f82-9beb-af7827f4a570\") " pod="openshift-multus/multus-s9wtj"
Sep 17 23:10:50.642628 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:10:50.641822    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/914194c1-d316-4f82-9beb-af7827f4a570-host-var-lib-cni-bin\") pod \"multus-s9wtj\" (UID: \"914194c1-d316-4f82-9beb-af7827f4a570\") " pod="openshift-multus/multus-s9wtj"
Sep 17 23:10:50.642628 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:10:50.641861    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/cae8ed58-1a8a-40af-a329-a3850797cc58-ovnkube-script-lib\") pod \"ovnkube-node-6xqwg\" (UID: \"cae8ed58-1a8a-40af-a329-a3850797cc58\") " pod="openshift-ovn-kubernetes/ovnkube-node-6xqwg"
Sep 17 23:10:50.642628 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:10:50.641883    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/914194c1-d316-4f82-9beb-af7827f4a570-host-var-lib-cni-bin\") pod \"multus-s9wtj\" (UID: \"914194c1-d316-4f82-9beb-af7827f4a570\") " pod="openshift-multus/multus-s9wtj"
Sep 17 23:10:50.642628 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:10:50.641893    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/914194c1-d316-4f82-9beb-af7827f4a570-multus-cni-dir\") pod \"multus-s9wtj\" (UID: \"914194c1-d316-4f82-9beb-af7827f4a570\") " pod="openshift-multus/multus-s9wtj"
Sep 17 23:10:50.642628 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:10:50.641867    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/914194c1-d316-4f82-9beb-af7827f4a570-host-run-netns\") pod \"multus-s9wtj\" (UID: \"914194c1-d316-4f82-9beb-af7827f4a570\") " pod="openshift-multus/multus-s9wtj"
Sep 17 23:10:50.642628 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:10:50.641914    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/914194c1-d316-4f82-9beb-af7827f4a570-host-var-lib-cni-multus\") pod \"multus-s9wtj\" (UID: \"914194c1-d316-4f82-9beb-af7827f4a570\") " pod="openshift-multus/multus-s9wtj"
Sep 17 23:10:50.642628 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:10:50.641937    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/cae8ed58-1a8a-40af-a329-a3850797cc58-node-log\") pod \"ovnkube-node-6xqwg\" (UID: \"cae8ed58-1a8a-40af-a329-a3850797cc58\") " pod="openshift-ovn-kubernetes/ovnkube-node-6xqwg"
Sep 17 23:10:50.642628 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:10:50.641962    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jrh7f\" (UniqueName: \"kubernetes.io/projected/4453ba96-a67f-46af-b61b-9cd8f39b6d36-kube-api-access-jrh7f\") pod \"network-metrics-daemon-nbccd\" (UID: \"4453ba96-a67f-46af-b61b-9cd8f39b6d36\") " pod="openshift-multus/network-metrics-daemon-nbccd"
Sep 17 23:10:50.642628 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:10:50.641983    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/79765acc-1a30-4a47-add5-aea2bf5f9efb-dbus\") pod \"global-pull-secret-syncer-jt864\" (UID: \"79765acc-1a30-4a47-add5-aea2bf5f9efb\") " pod="kube-system/global-pull-secret-syncer-jt864"
Sep 17 23:10:50.642628 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:10:50.641987    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/914194c1-d316-4f82-9beb-af7827f4a570-multus-socket-dir-parent\") pod \"multus-s9wtj\" (UID: \"914194c1-d316-4f82-9beb-af7827f4a570\") " pod="openshift-multus/multus-s9wtj"
Sep 17 23:10:50.642628 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:10:50.641863    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/cae8ed58-1a8a-40af-a329-a3850797cc58-run-ovn\") pod \"ovnkube-node-6xqwg\" (UID: \"cae8ed58-1a8a-40af-a329-a3850797cc58\") " pod="openshift-ovn-kubernetes/ovnkube-node-6xqwg"
Sep 17 23:10:50.642628 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:10:50.641983    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/914194c1-d316-4f82-9beb-af7827f4a570-host-var-lib-cni-multus\") pod \"multus-s9wtj\" (UID: \"914194c1-d316-4f82-9beb-af7827f4a570\") " pod="openshift-multus/multus-s9wtj"
Sep 17 23:10:50.643461 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:10:50.642023    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/cae8ed58-1a8a-40af-a329-a3850797cc58-node-log\") pod \"ovnkube-node-6xqwg\" (UID: \"cae8ed58-1a8a-40af-a329-a3850797cc58\") " pod="openshift-ovn-kubernetes/ovnkube-node-6xqwg"
Sep 17 23:10:50.643461 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:10:50.642040    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/914194c1-d316-4f82-9beb-af7827f4a570-host-var-lib-kubelet\") pod \"multus-s9wtj\" (UID: \"914194c1-d316-4f82-9beb-af7827f4a570\") " pod="openshift-multus/multus-s9wtj"
Sep 17 23:10:50.643461 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:10:50.642072    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/914194c1-d316-4f82-9beb-af7827f4a570-multus-socket-dir-parent\") pod \"multus-s9wtj\" (UID: \"914194c1-d316-4f82-9beb-af7827f4a570\") " pod="openshift-multus/multus-s9wtj"
Sep 17 23:10:50.643461 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:10:50.642104    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/d1311a9e-f175-47e1-b690-2c4164ad344b-lib-modules\") pod \"tuned-g9rzj\" (UID: \"d1311a9e-f175-47e1-b690-2c4164ad344b\") " pod="openshift-cluster-node-tuning-operator/tuned-g9rzj"
Sep 17 23:10:50.643461 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:10:50.642130    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/215c7cbb-4cd1-47be-9a44-26c1801b3fcb-system-cni-dir\") pod \"multus-additional-cni-plugins-srxwj\" (UID: \"215c7cbb-4cd1-47be-9a44-26c1801b3fcb\") " pod="openshift-multus/multus-additional-cni-plugins-srxwj"
Sep 17 23:10:50.643461 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:10:50.642155    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/914194c1-d316-4f82-9beb-af7827f4a570-host-var-lib-kubelet\") pod \"multus-s9wtj\" (UID: \"914194c1-d316-4f82-9beb-af7827f4a570\") " pod="openshift-multus/multus-s9wtj"
Sep 17 23:10:50.643461 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:10:50.642199    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-t9nhg\" (UniqueName: \"kubernetes.io/projected/be8438b1-0b71-4af1-9575-1f005b65a607-kube-api-access-t9nhg\") pod \"aws-ebs-csi-driver-node-2cj88\" (UID: \"be8438b1-0b71-4af1-9575-1f005b65a607\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-2cj88"
Sep 17 23:10:50.643461 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:10:50.642276    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/914194c1-d316-4f82-9beb-af7827f4a570-multus-cni-dir\") pod \"multus-s9wtj\" (UID: \"914194c1-d316-4f82-9beb-af7827f4a570\") " pod="openshift-multus/multus-s9wtj"
Sep 17 23:10:50.643461 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:10:50.642373    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/914194c1-d316-4f82-9beb-af7827f4a570-multus-daemon-config\") pod \"multus-s9wtj\" (UID: \"914194c1-d316-4f82-9beb-af7827f4a570\") " pod="openshift-multus/multus-s9wtj"
Sep 17 23:10:50.643461 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:10:50.642530    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/cae8ed58-1a8a-40af-a329-a3850797cc58-ovnkube-script-lib\") pod \"ovnkube-node-6xqwg\" (UID: \"cae8ed58-1a8a-40af-a329-a3850797cc58\") " pod="openshift-ovn-kubernetes/ovnkube-node-6xqwg"
Sep 17 23:10:50.643461 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:10:50.642585    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/cae8ed58-1a8a-40af-a329-a3850797cc58-ovn-node-metrics-cert\") pod \"ovnkube-node-6xqwg\" (UID: \"cae8ed58-1a8a-40af-a329-a3850797cc58\") " pod="openshift-ovn-kubernetes/ovnkube-node-6xqwg"
Sep 17 23:10:50.643461 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:10:50.642606    2562 swap_util.go:74] "error creating dir to test if tmpfs noswap is enabled. Assuming not supported" mount path="" error="stat /var/lib/kubelet/plugins/kubernetes.io/empty-dir: no such file or directory"
Sep 17 23:10:50.643461 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:10:50.642624    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/914194c1-d316-4f82-9beb-af7827f4a570-host-run-multus-certs\") pod \"multus-s9wtj\" (UID: \"914194c1-d316-4f82-9beb-af7827f4a570\") " pod="openshift-multus/multus-s9wtj"
Sep 17 23:10:50.643461 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:10:50.642653    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/215c7cbb-4cd1-47be-9a44-26c1801b3fcb-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-srxwj\" (UID: \"215c7cbb-4cd1-47be-9a44-26c1801b3fcb\") " pod="openshift-multus/multus-additional-cni-plugins-srxwj"
Sep 17 23:10:50.643461 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:10:50.642687    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/1d92dc9b-45b6-4912-bf69-f0961e86e8ed-hosts-file\") pod \"node-resolver-fzpjd\" (UID: \"1d92dc9b-45b6-4912-bf69-f0961e86e8ed\") " pod="openshift-dns/node-resolver-fzpjd"
Sep 17 23:10:50.643461 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:10:50.642713    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/cae8ed58-1a8a-40af-a329-a3850797cc58-run-systemd\") pod \"ovnkube-node-6xqwg\" (UID: \"cae8ed58-1a8a-40af-a329-a3850797cc58\") " pod="openshift-ovn-kubernetes/ovnkube-node-6xqwg"
Sep 17 23:10:50.643461 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:10:50.642728    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/914194c1-d316-4f82-9beb-af7827f4a570-host-run-multus-certs\") pod \"multus-s9wtj\" (UID: \"914194c1-d316-4f82-9beb-af7827f4a570\") " pod="openshift-multus/multus-s9wtj"
Sep 17 23:10:50.644424 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:10:50.642736    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/cae8ed58-1a8a-40af-a329-a3850797cc58-var-lib-openvswitch\") pod \"ovnkube-node-6xqwg\" (UID: \"cae8ed58-1a8a-40af-a329-a3850797cc58\") " pod="openshift-ovn-kubernetes/ovnkube-node-6xqwg"
Sep 17 23:10:50.644424 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:10:50.642760    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/cae8ed58-1a8a-40af-a329-a3850797cc58-host-cni-netd\") pod \"ovnkube-node-6xqwg\" (UID: \"cae8ed58-1a8a-40af-a329-a3850797cc58\") " pod="openshift-ovn-kubernetes/ovnkube-node-6xqwg"
Sep 17 23:10:50.644424 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:10:50.642768    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/cae8ed58-1a8a-40af-a329-a3850797cc58-run-systemd\") pod \"ovnkube-node-6xqwg\" (UID: \"cae8ed58-1a8a-40af-a329-a3850797cc58\") " pod="openshift-ovn-kubernetes/ovnkube-node-6xqwg"
Sep 17 23:10:50.644424 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:10:50.642786    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/cae8ed58-1a8a-40af-a329-a3850797cc58-var-lib-openvswitch\") pod \"ovnkube-node-6xqwg\" (UID: \"cae8ed58-1a8a-40af-a329-a3850797cc58\") " pod="openshift-ovn-kubernetes/ovnkube-node-6xqwg"
Sep 17 23:10:50.644424 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:10:50.642782    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-g67q7\" (UniqueName: \"kubernetes.io/projected/cae8ed58-1a8a-40af-a329-a3850797cc58-kube-api-access-g67q7\") pod \"ovnkube-node-6xqwg\" (UID: \"cae8ed58-1a8a-40af-a329-a3850797cc58\") " pod="openshift-ovn-kubernetes/ovnkube-node-6xqwg"
Sep 17 23:10:50.644424 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:10:50.642822    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/4453ba96-a67f-46af-b61b-9cd8f39b6d36-metrics-certs\") pod \"network-metrics-daemon-nbccd\" (UID: \"4453ba96-a67f-46af-b61b-9cd8f39b6d36\") " pod="openshift-multus/network-metrics-daemon-nbccd"
Sep 17 23:10:50.644424 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:10:50.642842    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/cae8ed58-1a8a-40af-a329-a3850797cc58-host-cni-netd\") pod \"ovnkube-node-6xqwg\" (UID: \"cae8ed58-1a8a-40af-a329-a3850797cc58\") " pod="openshift-ovn-kubernetes/ovnkube-node-6xqwg"
Sep 17 23:10:50.644424 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:10:50.642846    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/914194c1-d316-4f82-9beb-af7827f4a570-etc-kubernetes\") pod \"multus-s9wtj\" (UID: \"914194c1-d316-4f82-9beb-af7827f4a570\") " pod="openshift-multus/multus-s9wtj"
Sep 17 23:10:50.644424 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:10:50.642872    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/d1311a9e-f175-47e1-b690-2c4164ad344b-sys\") pod \"tuned-g9rzj\" (UID: \"d1311a9e-f175-47e1-b690-2c4164ad344b\") " pod="openshift-cluster-node-tuning-operator/tuned-g9rzj"
Sep 17 23:10:50.644424 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:10:50.642890    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/914194c1-d316-4f82-9beb-af7827f4a570-etc-kubernetes\") pod \"multus-s9wtj\" (UID: \"914194c1-d316-4f82-9beb-af7827f4a570\") " pod="openshift-multus/multus-s9wtj"
Sep 17 23:10:50.644424 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:10:50.642898    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/d1311a9e-f175-47e1-b690-2c4164ad344b-etc-modprobe-d\") pod \"tuned-g9rzj\" (UID: \"d1311a9e-f175-47e1-b690-2c4164ad344b\") " pod="openshift-cluster-node-tuning-operator/tuned-g9rzj"
Sep 17 23:10:50.644424 ip-10-0-132-188 kubenswrapper[2562]: E0917 23:10:50.642921    2562 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 17 23:10:50.644424 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:10:50.642946    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/cae8ed58-1a8a-40af-a329-a3850797cc58-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-6xqwg\" (UID: \"cae8ed58-1a8a-40af-a329-a3850797cc58\") " pod="openshift-ovn-kubernetes/ovnkube-node-6xqwg"
Sep 17 23:10:50.644424 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:10:50.643025    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/79765acc-1a30-4a47-add5-aea2bf5f9efb-original-pull-secret\") pod \"global-pull-secret-syncer-jt864\" (UID: \"79765acc-1a30-4a47-add5-aea2bf5f9efb\") " pod="kube-system/global-pull-secret-syncer-jt864"
Sep 17 23:10:50.644424 ip-10-0-132-188 kubenswrapper[2562]: E0917 23:10:50.643064    2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/4453ba96-a67f-46af-b61b-9cd8f39b6d36-metrics-certs podName:4453ba96-a67f-46af-b61b-9cd8f39b6d36 nodeName:}" failed. No retries permitted until 2026-09-17 23:10:51.143044643 +0000 UTC m=+3.544941405 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/4453ba96-a67f-46af-b61b-9cd8f39b6d36-metrics-certs") pod "network-metrics-daemon-nbccd" (UID: "4453ba96-a67f-46af-b61b-9cd8f39b6d36") : object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 17 23:10:50.644424 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:10:50.643050    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/cae8ed58-1a8a-40af-a329-a3850797cc58-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-6xqwg\" (UID: \"cae8ed58-1a8a-40af-a329-a3850797cc58\") " pod="openshift-ovn-kubernetes/ovnkube-node-6xqwg"
Sep 17 23:10:50.644424 ip-10-0-132-188 kubenswrapper[2562]: E0917 23:10:50.643122    2562 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Sep 17 23:10:50.644930 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:10:50.643122    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/914194c1-d316-4f82-9beb-af7827f4a570-system-cni-dir\") pod \"multus-s9wtj\" (UID: \"914194c1-d316-4f82-9beb-af7827f4a570\") " pod="openshift-multus/multus-s9wtj"
Sep 17 23:10:50.644930 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:10:50.643160    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qnkg2\" (UniqueName: \"kubernetes.io/projected/9fd2f397-d1b2-4790-914c-4e9b860b827a-kube-api-access-qnkg2\") pod \"iptables-alerter-bvb88\" (UID: \"9fd2f397-d1b2-4790-914c-4e9b860b827a\") " pod="openshift-network-operator/iptables-alerter-bvb88"
Sep 17 23:10:50.644930 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:10:50.643205    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/d1311a9e-f175-47e1-b690-2c4164ad344b-run\") pod \"tuned-g9rzj\" (UID: \"d1311a9e-f175-47e1-b690-2c4164ad344b\") " pod="openshift-cluster-node-tuning-operator/tuned-g9rzj"
Sep 17 23:10:50.644930 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:10:50.643233    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/914194c1-d316-4f82-9beb-af7827f4a570-system-cni-dir\") pod \"multus-s9wtj\" (UID: \"914194c1-d316-4f82-9beb-af7827f4a570\") " pod="openshift-multus/multus-s9wtj"
Sep 17 23:10:50.644930 ip-10-0-132-188 kubenswrapper[2562]: E0917 23:10:50.643241    2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/79765acc-1a30-4a47-add5-aea2bf5f9efb-original-pull-secret podName:79765acc-1a30-4a47-add5-aea2bf5f9efb nodeName:}" failed. No retries permitted until 2026-09-17 23:10:51.143221403 +0000 UTC m=+3.545118165 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/79765acc-1a30-4a47-add5-aea2bf5f9efb-original-pull-secret") pod "global-pull-secret-syncer-jt864" (UID: "79765acc-1a30-4a47-add5-aea2bf5f9efb") : object "kube-system"/"original-pull-secret" not registered
Sep 17 23:10:50.644930 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:10:50.643279    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/33db9205-3be0-47dc-ac16-e8f623e43afb-serviceca\") pod \"node-ca-vkllk\" (UID: \"33db9205-3be0-47dc-ac16-e8f623e43afb\") " pod="openshift-image-registry/node-ca-vkllk"
Sep 17 23:10:50.644930 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:10:50.643306    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/7e452bbb-7dc6-4c9d-a9b1-76400942f5b4-agent-certs\") pod \"konnectivity-agent-fwwc5\" (UID: \"7e452bbb-7dc6-4c9d-a9b1-76400942f5b4\") " pod="kube-system/konnectivity-agent-fwwc5"
Sep 17 23:10:50.644930 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:10:50.643334    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/cae8ed58-1a8a-40af-a329-a3850797cc58-systemd-units\") pod \"ovnkube-node-6xqwg\" (UID: \"cae8ed58-1a8a-40af-a329-a3850797cc58\") " pod="openshift-ovn-kubernetes/ovnkube-node-6xqwg"
Sep 17 23:10:50.644930 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:10:50.643367    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/cae8ed58-1a8a-40af-a329-a3850797cc58-log-socket\") pod \"ovnkube-node-6xqwg\" (UID: \"cae8ed58-1a8a-40af-a329-a3850797cc58\") " pod="openshift-ovn-kubernetes/ovnkube-node-6xqwg"
Sep 17 23:10:50.644930 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:10:50.643388    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/cae8ed58-1a8a-40af-a329-a3850797cc58-systemd-units\") pod \"ovnkube-node-6xqwg\" (UID: \"cae8ed58-1a8a-40af-a329-a3850797cc58\") " pod="openshift-ovn-kubernetes/ovnkube-node-6xqwg"
Sep 17 23:10:50.644930 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:10:50.643450    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/cae8ed58-1a8a-40af-a329-a3850797cc58-log-socket\") pod \"ovnkube-node-6xqwg\" (UID: \"cae8ed58-1a8a-40af-a329-a3850797cc58\") " pod="openshift-ovn-kubernetes/ovnkube-node-6xqwg"
Sep 17 23:10:50.644930 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:10:50.643477    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/914194c1-d316-4f82-9beb-af7827f4a570-cni-binary-copy\") pod \"multus-s9wtj\" (UID: \"914194c1-d316-4f82-9beb-af7827f4a570\") " pod="openshift-multus/multus-s9wtj"
Sep 17 23:10:50.644930 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:10:50.643518    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/215c7cbb-4cd1-47be-9a44-26c1801b3fcb-tuning-conf-dir\") pod \"multus-additional-cni-plugins-srxwj\" (UID: \"215c7cbb-4cd1-47be-9a44-26c1801b3fcb\") " pod="openshift-multus/multus-additional-cni-plugins-srxwj"
Sep 17 23:10:50.644930 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:10:50.643544    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/7e452bbb-7dc6-4c9d-a9b1-76400942f5b4-konnectivity-ca\") pod \"konnectivity-agent-fwwc5\" (UID: \"7e452bbb-7dc6-4c9d-a9b1-76400942f5b4\") " pod="kube-system/konnectivity-agent-fwwc5"
Sep 17 23:10:50.644930 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:10:50.643572    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/cae8ed58-1a8a-40af-a329-a3850797cc58-env-overrides\") pod \"ovnkube-node-6xqwg\" (UID: \"cae8ed58-1a8a-40af-a329-a3850797cc58\") " pod="openshift-ovn-kubernetes/ovnkube-node-6xqwg"
Sep 17 23:10:50.644930 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:10:50.643594    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/d1311a9e-f175-47e1-b690-2c4164ad344b-host\") pod \"tuned-g9rzj\" (UID: \"d1311a9e-f175-47e1-b690-2c4164ad344b\") " pod="openshift-cluster-node-tuning-operator/tuned-g9rzj"
Sep 17 23:10:50.645400 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:10:50.643616    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/215c7cbb-4cd1-47be-9a44-26c1801b3fcb-cni-binary-copy\") pod \"multus-additional-cni-plugins-srxwj\" (UID: \"215c7cbb-4cd1-47be-9a44-26c1801b3fcb\") " pod="openshift-multus/multus-additional-cni-plugins-srxwj"
Sep 17 23:10:50.645400 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:10:50.643638    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/215c7cbb-4cd1-47be-9a44-26c1801b3fcb-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-srxwj\" (UID: \"215c7cbb-4cd1-47be-9a44-26c1801b3fcb\") " pod="openshift-multus/multus-additional-cni-plugins-srxwj"
Sep 17 23:10:50.645400 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:10:50.643664    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/be8438b1-0b71-4af1-9575-1f005b65a607-socket-dir\") pod \"aws-ebs-csi-driver-node-2cj88\" (UID: \"be8438b1-0b71-4af1-9575-1f005b65a607\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-2cj88"
Sep 17 23:10:50.645400 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:10:50.643686    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/be8438b1-0b71-4af1-9575-1f005b65a607-sys-fs\") pod \"aws-ebs-csi-driver-node-2cj88\" (UID: \"be8438b1-0b71-4af1-9575-1f005b65a607\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-2cj88"
Sep 17 23:10:50.645400 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:10:50.643709    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/cae8ed58-1a8a-40af-a329-a3850797cc58-host-run-netns\") pod \"ovnkube-node-6xqwg\" (UID: \"cae8ed58-1a8a-40af-a329-a3850797cc58\") " pod="openshift-ovn-kubernetes/ovnkube-node-6xqwg"
Sep 17 23:10:50.645400 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:10:50.643732    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/cae8ed58-1a8a-40af-a329-a3850797cc58-ovnkube-config\") pod \"ovnkube-node-6xqwg\" (UID: \"cae8ed58-1a8a-40af-a329-a3850797cc58\") " pod="openshift-ovn-kubernetes/ovnkube-node-6xqwg"
Sep 17 23:10:50.645400 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:10:50.643756    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/914194c1-d316-4f82-9beb-af7827f4a570-host-run-k8s-cni-cncf-io\") pod \"multus-s9wtj\" (UID: \"914194c1-d316-4f82-9beb-af7827f4a570\") " pod="openshift-multus/multus-s9wtj"
Sep 17 23:10:50.645400 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:10:50.643783    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/9fd2f397-d1b2-4790-914c-4e9b860b827a-iptables-alerter-script\") pod \"iptables-alerter-bvb88\" (UID: \"9fd2f397-d1b2-4790-914c-4e9b860b827a\") " pod="openshift-network-operator/iptables-alerter-bvb88"
Sep 17 23:10:50.645400 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:10:50.643809    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/d1311a9e-f175-47e1-b690-2c4164ad344b-var-lib-kubelet\") pod \"tuned-g9rzj\" (UID: \"d1311a9e-f175-47e1-b690-2c4164ad344b\") " pod="openshift-cluster-node-tuning-operator/tuned-g9rzj"
Sep 17 23:10:50.645400 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:10:50.643833    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/d1311a9e-f175-47e1-b690-2c4164ad344b-tmp\") pod \"tuned-g9rzj\" (UID: \"d1311a9e-f175-47e1-b690-2c4164ad344b\") " pod="openshift-cluster-node-tuning-operator/tuned-g9rzj"
Sep 17 23:10:50.645400 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:10:50.643868    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/be8438b1-0b71-4af1-9575-1f005b65a607-registration-dir\") pod \"aws-ebs-csi-driver-node-2cj88\" (UID: \"be8438b1-0b71-4af1-9575-1f005b65a607\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-2cj88"
Sep 17 23:10:50.645400 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:10:50.643902    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/be8438b1-0b71-4af1-9575-1f005b65a607-device-dir\") pod \"aws-ebs-csi-driver-node-2cj88\" (UID: \"be8438b1-0b71-4af1-9575-1f005b65a607\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-2cj88"
Sep 17 23:10:50.645400 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:10:50.643907    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/cae8ed58-1a8a-40af-a329-a3850797cc58-host-run-netns\") pod \"ovnkube-node-6xqwg\" (UID: \"cae8ed58-1a8a-40af-a329-a3850797cc58\") " pod="openshift-ovn-kubernetes/ovnkube-node-6xqwg"
Sep 17 23:10:50.645400 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:10:50.643933    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/79765acc-1a30-4a47-add5-aea2bf5f9efb-kubelet-config\") pod \"global-pull-secret-syncer-jt864\" (UID: \"79765acc-1a30-4a47-add5-aea2bf5f9efb\") " pod="kube-system/global-pull-secret-syncer-jt864"
Sep 17 23:10:50.645400 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:10:50.643941    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/914194c1-d316-4f82-9beb-af7827f4a570-host-run-k8s-cni-cncf-io\") pod \"multus-s9wtj\" (UID: \"914194c1-d316-4f82-9beb-af7827f4a570\") " pod="openshift-multus/multus-s9wtj"
Sep 17 23:10:50.645400 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:10:50.643965    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/914194c1-d316-4f82-9beb-af7827f4a570-cni-binary-copy\") pod \"multus-s9wtj\" (UID: \"914194c1-d316-4f82-9beb-af7827f4a570\") " pod="openshift-multus/multus-s9wtj"
Sep 17 23:10:50.645400 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:10:50.643980    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/1d92dc9b-45b6-4912-bf69-f0961e86e8ed-tmp-dir\") pod \"node-resolver-fzpjd\" (UID: \"1d92dc9b-45b6-4912-bf69-f0961e86e8ed\") " pod="openshift-dns/node-resolver-fzpjd"
Sep 17 23:10:50.645864 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:10:50.644040    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-nzkzp\" (UniqueName: \"kubernetes.io/projected/914194c1-d316-4f82-9beb-af7827f4a570-kube-api-access-nzkzp\") pod \"multus-s9wtj\" (UID: \"914194c1-d316-4f82-9beb-af7827f4a570\") " pod="openshift-multus/multus-s9wtj"
Sep 17 23:10:50.645864 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:10:50.644062    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/cae8ed58-1a8a-40af-a329-a3850797cc58-env-overrides\") pod \"ovnkube-node-6xqwg\" (UID: \"cae8ed58-1a8a-40af-a329-a3850797cc58\") " pod="openshift-ovn-kubernetes/ovnkube-node-6xqwg"
Sep 17 23:10:50.645864 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:10:50.644082    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/79765acc-1a30-4a47-add5-aea2bf5f9efb-kubelet-config\") pod \"global-pull-secret-syncer-jt864\" (UID: \"79765acc-1a30-4a47-add5-aea2bf5f9efb\") " pod="kube-system/global-pull-secret-syncer-jt864"
Sep 17 23:10:50.645864 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:10:50.644119    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/d1311a9e-f175-47e1-b690-2c4164ad344b-etc-systemd\") pod \"tuned-g9rzj\" (UID: \"d1311a9e-f175-47e1-b690-2c4164ad344b\") " pod="openshift-cluster-node-tuning-operator/tuned-g9rzj"
Sep 17 23:10:50.645864 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:10:50.644217    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qm6g7\" (UniqueName: \"kubernetes.io/projected/33db9205-3be0-47dc-ac16-e8f623e43afb-kube-api-access-qm6g7\") pod \"node-ca-vkllk\" (UID: \"33db9205-3be0-47dc-ac16-e8f623e43afb\") " pod="openshift-image-registry/node-ca-vkllk"
Sep 17 23:10:50.645864 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:10:50.644247    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/be8438b1-0b71-4af1-9575-1f005b65a607-kubelet-dir\") pod \"aws-ebs-csi-driver-node-2cj88\" (UID: \"be8438b1-0b71-4af1-9575-1f005b65a607\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-2cj88"
Sep 17 23:10:50.645864 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:10:50.644309    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/cae8ed58-1a8a-40af-a329-a3850797cc58-host-kubelet\") pod \"ovnkube-node-6xqwg\" (UID: \"cae8ed58-1a8a-40af-a329-a3850797cc58\") " pod="openshift-ovn-kubernetes/ovnkube-node-6xqwg"
Sep 17 23:10:50.645864 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:10:50.644367    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/cae8ed58-1a8a-40af-a329-a3850797cc58-host-kubelet\") pod \"ovnkube-node-6xqwg\" (UID: \"cae8ed58-1a8a-40af-a329-a3850797cc58\") " pod="openshift-ovn-kubernetes/ovnkube-node-6xqwg"
Sep 17 23:10:50.645864 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:10:50.644384    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/9fd2f397-d1b2-4790-914c-4e9b860b827a-iptables-alerter-script\") pod \"iptables-alerter-bvb88\" (UID: \"9fd2f397-d1b2-4790-914c-4e9b860b827a\") " pod="openshift-network-operator/iptables-alerter-bvb88"
Sep 17 23:10:50.645864 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:10:50.644460    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/cae8ed58-1a8a-40af-a329-a3850797cc58-ovnkube-config\") pod \"ovnkube-node-6xqwg\" (UID: \"cae8ed58-1a8a-40af-a329-a3850797cc58\") " pod="openshift-ovn-kubernetes/ovnkube-node-6xqwg"
Sep 17 23:10:50.646146 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:10:50.646076    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/cae8ed58-1a8a-40af-a329-a3850797cc58-ovn-node-metrics-cert\") pod \"ovnkube-node-6xqwg\" (UID: \"cae8ed58-1a8a-40af-a329-a3850797cc58\") " pod="openshift-ovn-kubernetes/ovnkube-node-6xqwg"
Sep 17 23:10:50.655509 ip-10-0-132-188 kubenswrapper[2562]: E0917 23:10:50.655463    2562 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Sep 17 23:10:50.655509 ip-10-0-132-188 kubenswrapper[2562]: E0917 23:10:50.655490    2562 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Sep 17 23:10:50.655509 ip-10-0-132-188 kubenswrapper[2562]: E0917 23:10:50.655504    2562 projected.go:196] Error preparing data for projected volume kube-api-access-42cvm for pod openshift-network-diagnostics/network-check-target-t9gqw: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 17 23:10:50.655686 ip-10-0-132-188 kubenswrapper[2562]: E0917 23:10:50.655550    2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/440bce33-9c27-46bd-9d1c-0eebcc14fbc3-kube-api-access-42cvm podName:440bce33-9c27-46bd-9d1c-0eebcc14fbc3 nodeName:}" failed. No retries permitted until 2026-09-17 23:10:51.155538369 +0000 UTC m=+3.557435111 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-api-access-42cvm" (UniqueName: "kubernetes.io/projected/440bce33-9c27-46bd-9d1c-0eebcc14fbc3-kube-api-access-42cvm") pod "network-check-target-t9gqw" (UID: "440bce33-9c27-46bd-9d1c-0eebcc14fbc3") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 17 23:10:50.657945 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:10:50.657910    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-g67q7\" (UniqueName: \"kubernetes.io/projected/cae8ed58-1a8a-40af-a329-a3850797cc58-kube-api-access-g67q7\") pod \"ovnkube-node-6xqwg\" (UID: \"cae8ed58-1a8a-40af-a329-a3850797cc58\") " pod="openshift-ovn-kubernetes/ovnkube-node-6xqwg"
Sep 17 23:10:50.658597 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:10:50.658571    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-qnkg2\" (UniqueName: \"kubernetes.io/projected/9fd2f397-d1b2-4790-914c-4e9b860b827a-kube-api-access-qnkg2\") pod \"iptables-alerter-bvb88\" (UID: \"9fd2f397-d1b2-4790-914c-4e9b860b827a\") " pod="openshift-network-operator/iptables-alerter-bvb88"
Sep 17 23:10:50.658724 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:10:50.658703    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-nzkzp\" (UniqueName: \"kubernetes.io/projected/914194c1-d316-4f82-9beb-af7827f4a570-kube-api-access-nzkzp\") pod \"multus-s9wtj\" (UID: \"914194c1-d316-4f82-9beb-af7827f4a570\") " pod="openshift-multus/multus-s9wtj"
Sep 17 23:10:50.658927 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:10:50.658904    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-jrh7f\" (UniqueName: \"kubernetes.io/projected/4453ba96-a67f-46af-b61b-9cd8f39b6d36-kube-api-access-jrh7f\") pod \"network-metrics-daemon-nbccd\" (UID: \"4453ba96-a67f-46af-b61b-9cd8f39b6d36\") " pod="openshift-multus/network-metrics-daemon-nbccd"
Sep 17 23:10:50.678402 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:10:50.678378    2562 reflector.go:439] "Caches populated" type="*v1.Service" reflector="k8s.io/client-go/informers/factory.go:160"
Sep 17 23:10:50.744534 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:10:50.744499    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/d1311a9e-f175-47e1-b690-2c4164ad344b-lib-modules\") pod \"tuned-g9rzj\" (UID: \"d1311a9e-f175-47e1-b690-2c4164ad344b\") " pod="openshift-cluster-node-tuning-operator/tuned-g9rzj"
Sep 17 23:10:50.744534 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:10:50.744533    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/215c7cbb-4cd1-47be-9a44-26c1801b3fcb-system-cni-dir\") pod \"multus-additional-cni-plugins-srxwj\" (UID: \"215c7cbb-4cd1-47be-9a44-26c1801b3fcb\") " pod="openshift-multus/multus-additional-cni-plugins-srxwj"
Sep 17 23:10:50.744771 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:10:50.744553    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-t9nhg\" (UniqueName: \"kubernetes.io/projected/be8438b1-0b71-4af1-9575-1f005b65a607-kube-api-access-t9nhg\") pod \"aws-ebs-csi-driver-node-2cj88\" (UID: \"be8438b1-0b71-4af1-9575-1f005b65a607\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-2cj88"
Sep 17 23:10:50.744771 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:10:50.744568    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/215c7cbb-4cd1-47be-9a44-26c1801b3fcb-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-srxwj\" (UID: \"215c7cbb-4cd1-47be-9a44-26c1801b3fcb\") " pod="openshift-multus/multus-additional-cni-plugins-srxwj"
Sep 17 23:10:50.744771 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:10:50.744584    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/1d92dc9b-45b6-4912-bf69-f0961e86e8ed-hosts-file\") pod \"node-resolver-fzpjd\" (UID: \"1d92dc9b-45b6-4912-bf69-f0961e86e8ed\") " pod="openshift-dns/node-resolver-fzpjd"
Sep 17 23:10:50.744771 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:10:50.744609    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/d1311a9e-f175-47e1-b690-2c4164ad344b-sys\") pod \"tuned-g9rzj\" (UID: \"d1311a9e-f175-47e1-b690-2c4164ad344b\") " pod="openshift-cluster-node-tuning-operator/tuned-g9rzj"
Sep 17 23:10:50.744771 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:10:50.744623    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/215c7cbb-4cd1-47be-9a44-26c1801b3fcb-system-cni-dir\") pod \"multus-additional-cni-plugins-srxwj\" (UID: \"215c7cbb-4cd1-47be-9a44-26c1801b3fcb\") " pod="openshift-multus/multus-additional-cni-plugins-srxwj"
Sep 17 23:10:50.744771 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:10:50.744628    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/d1311a9e-f175-47e1-b690-2c4164ad344b-etc-modprobe-d\") pod \"tuned-g9rzj\" (UID: \"d1311a9e-f175-47e1-b690-2c4164ad344b\") " pod="openshift-cluster-node-tuning-operator/tuned-g9rzj"
Sep 17 23:10:50.744771 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:10:50.744686    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/1d92dc9b-45b6-4912-bf69-f0961e86e8ed-hosts-file\") pod \"node-resolver-fzpjd\" (UID: \"1d92dc9b-45b6-4912-bf69-f0961e86e8ed\") " pod="openshift-dns/node-resolver-fzpjd"
Sep 17 23:10:50.744771 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:10:50.744697    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/d1311a9e-f175-47e1-b690-2c4164ad344b-run\") pod \"tuned-g9rzj\" (UID: \"d1311a9e-f175-47e1-b690-2c4164ad344b\") " pod="openshift-cluster-node-tuning-operator/tuned-g9rzj"
Sep 17 23:10:50.744771 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:10:50.744692    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/d1311a9e-f175-47e1-b690-2c4164ad344b-sys\") pod \"tuned-g9rzj\" (UID: \"d1311a9e-f175-47e1-b690-2c4164ad344b\") " pod="openshift-cluster-node-tuning-operator/tuned-g9rzj"
Sep 17 23:10:50.744771 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:10:50.744718    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/d1311a9e-f175-47e1-b690-2c4164ad344b-etc-modprobe-d\") pod \"tuned-g9rzj\" (UID: \"d1311a9e-f175-47e1-b690-2c4164ad344b\") " pod="openshift-cluster-node-tuning-operator/tuned-g9rzj"
Sep 17 23:10:50.744771 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:10:50.744729    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/d1311a9e-f175-47e1-b690-2c4164ad344b-lib-modules\") pod \"tuned-g9rzj\" (UID: \"d1311a9e-f175-47e1-b690-2c4164ad344b\") " pod="openshift-cluster-node-tuning-operator/tuned-g9rzj"
Sep 17 23:10:50.744771 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:10:50.744739    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/33db9205-3be0-47dc-ac16-e8f623e43afb-serviceca\") pod \"node-ca-vkllk\" (UID: \"33db9205-3be0-47dc-ac16-e8f623e43afb\") " pod="openshift-image-registry/node-ca-vkllk"
Sep 17 23:10:50.744771 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:10:50.744742    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run\" (UniqueName: \"kubernetes.io/host-path/d1311a9e-f175-47e1-b690-2c4164ad344b-run\") pod \"tuned-g9rzj\" (UID: \"d1311a9e-f175-47e1-b690-2c4164ad344b\") " pod="openshift-cluster-node-tuning-operator/tuned-g9rzj"
Sep 17 23:10:50.744771 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:10:50.744774    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/7e452bbb-7dc6-4c9d-a9b1-76400942f5b4-agent-certs\") pod \"konnectivity-agent-fwwc5\" (UID: \"7e452bbb-7dc6-4c9d-a9b1-76400942f5b4\") " pod="kube-system/konnectivity-agent-fwwc5"
Sep 17 23:10:50.745432 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:10:50.744821    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/215c7cbb-4cd1-47be-9a44-26c1801b3fcb-tuning-conf-dir\") pod \"multus-additional-cni-plugins-srxwj\" (UID: \"215c7cbb-4cd1-47be-9a44-26c1801b3fcb\") " pod="openshift-multus/multus-additional-cni-plugins-srxwj"
Sep 17 23:10:50.745432 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:10:50.744845    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/7e452bbb-7dc6-4c9d-a9b1-76400942f5b4-konnectivity-ca\") pod \"konnectivity-agent-fwwc5\" (UID: \"7e452bbb-7dc6-4c9d-a9b1-76400942f5b4\") " pod="kube-system/konnectivity-agent-fwwc5"
Sep 17 23:10:50.745432 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:10:50.744873    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/d1311a9e-f175-47e1-b690-2c4164ad344b-host\") pod \"tuned-g9rzj\" (UID: \"d1311a9e-f175-47e1-b690-2c4164ad344b\") " pod="openshift-cluster-node-tuning-operator/tuned-g9rzj"
Sep 17 23:10:50.745432 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:10:50.744895    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/215c7cbb-4cd1-47be-9a44-26c1801b3fcb-cni-binary-copy\") pod \"multus-additional-cni-plugins-srxwj\" (UID: \"215c7cbb-4cd1-47be-9a44-26c1801b3fcb\") " pod="openshift-multus/multus-additional-cni-plugins-srxwj"
Sep 17 23:10:50.745432 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:10:50.744921    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/215c7cbb-4cd1-47be-9a44-26c1801b3fcb-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-srxwj\" (UID: \"215c7cbb-4cd1-47be-9a44-26c1801b3fcb\") " pod="openshift-multus/multus-additional-cni-plugins-srxwj"
Sep 17 23:10:50.745432 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:10:50.744946    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/be8438b1-0b71-4af1-9575-1f005b65a607-socket-dir\") pod \"aws-ebs-csi-driver-node-2cj88\" (UID: \"be8438b1-0b71-4af1-9575-1f005b65a607\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-2cj88"
Sep 17 23:10:50.745432 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:10:50.744969    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/be8438b1-0b71-4af1-9575-1f005b65a607-sys-fs\") pod \"aws-ebs-csi-driver-node-2cj88\" (UID: \"be8438b1-0b71-4af1-9575-1f005b65a607\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-2cj88"
Sep 17 23:10:50.745432 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:10:50.744995    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/d1311a9e-f175-47e1-b690-2c4164ad344b-var-lib-kubelet\") pod \"tuned-g9rzj\" (UID: \"d1311a9e-f175-47e1-b690-2c4164ad344b\") " pod="openshift-cluster-node-tuning-operator/tuned-g9rzj"
Sep 17 23:10:50.745432 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:10:50.745004    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/215c7cbb-4cd1-47be-9a44-26c1801b3fcb-tuning-conf-dir\") pod \"multus-additional-cni-plugins-srxwj\" (UID: \"215c7cbb-4cd1-47be-9a44-26c1801b3fcb\") " pod="openshift-multus/multus-additional-cni-plugins-srxwj"
Sep 17 23:10:50.745432 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:10:50.745016    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/d1311a9e-f175-47e1-b690-2c4164ad344b-tmp\") pod \"tuned-g9rzj\" (UID: \"d1311a9e-f175-47e1-b690-2c4164ad344b\") " pod="openshift-cluster-node-tuning-operator/tuned-g9rzj"
Sep 17 23:10:50.745432 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:10:50.745049    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/be8438b1-0b71-4af1-9575-1f005b65a607-registration-dir\") pod \"aws-ebs-csi-driver-node-2cj88\" (UID: \"be8438b1-0b71-4af1-9575-1f005b65a607\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-2cj88"
Sep 17 23:10:50.745432 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:10:50.745076    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/be8438b1-0b71-4af1-9575-1f005b65a607-device-dir\") pod \"aws-ebs-csi-driver-node-2cj88\" (UID: \"be8438b1-0b71-4af1-9575-1f005b65a607\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-2cj88"
Sep 17 23:10:50.745432 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:10:50.745101    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/1d92dc9b-45b6-4912-bf69-f0961e86e8ed-tmp-dir\") pod \"node-resolver-fzpjd\" (UID: \"1d92dc9b-45b6-4912-bf69-f0961e86e8ed\") " pod="openshift-dns/node-resolver-fzpjd"
Sep 17 23:10:50.745432 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:10:50.745125    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/d1311a9e-f175-47e1-b690-2c4164ad344b-etc-systemd\") pod \"tuned-g9rzj\" (UID: \"d1311a9e-f175-47e1-b690-2c4164ad344b\") " pod="openshift-cluster-node-tuning-operator/tuned-g9rzj"
Sep 17 23:10:50.745432 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:10:50.745152    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qm6g7\" (UniqueName: \"kubernetes.io/projected/33db9205-3be0-47dc-ac16-e8f623e43afb-kube-api-access-qm6g7\") pod \"node-ca-vkllk\" (UID: \"33db9205-3be0-47dc-ac16-e8f623e43afb\") " pod="openshift-image-registry/node-ca-vkllk"
Sep 17 23:10:50.745432 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:10:50.745159    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/be8438b1-0b71-4af1-9575-1f005b65a607-device-dir\") pod \"aws-ebs-csi-driver-node-2cj88\" (UID: \"be8438b1-0b71-4af1-9575-1f005b65a607\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-2cj88"
Sep 17 23:10:50.745432 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:10:50.745196    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/be8438b1-0b71-4af1-9575-1f005b65a607-kubelet-dir\") pod \"aws-ebs-csi-driver-node-2cj88\" (UID: \"be8438b1-0b71-4af1-9575-1f005b65a607\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-2cj88"
Sep 17 23:10:50.746234 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:10:50.745230    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/be8438b1-0b71-4af1-9575-1f005b65a607-registration-dir\") pod \"aws-ebs-csi-driver-node-2cj88\" (UID: \"be8438b1-0b71-4af1-9575-1f005b65a607\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-2cj88"
Sep 17 23:10:50.746234 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:10:50.745237    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/be8438b1-0b71-4af1-9575-1f005b65a607-etc-selinux\") pod \"aws-ebs-csi-driver-node-2cj88\" (UID: \"be8438b1-0b71-4af1-9575-1f005b65a607\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-2cj88"
Sep 17 23:10:50.746234 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:10:50.745264    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/d1311a9e-f175-47e1-b690-2c4164ad344b-etc-sysconfig\") pod \"tuned-g9rzj\" (UID: \"d1311a9e-f175-47e1-b690-2c4164ad344b\") " pod="openshift-cluster-node-tuning-operator/tuned-g9rzj"
Sep 17 23:10:50.746234 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:10:50.745285    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/d1311a9e-f175-47e1-b690-2c4164ad344b-etc-kubernetes\") pod \"tuned-g9rzj\" (UID: \"d1311a9e-f175-47e1-b690-2c4164ad344b\") " pod="openshift-cluster-node-tuning-operator/tuned-g9rzj"
Sep 17 23:10:50.746234 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:10:50.745305    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/d1311a9e-f175-47e1-b690-2c4164ad344b-etc-sysctl-d\") pod \"tuned-g9rzj\" (UID: \"d1311a9e-f175-47e1-b690-2c4164ad344b\") " pod="openshift-cluster-node-tuning-operator/tuned-g9rzj"
Sep 17 23:10:50.746234 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:10:50.745327    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/d1311a9e-f175-47e1-b690-2c4164ad344b-etc-tuned\") pod \"tuned-g9rzj\" (UID: \"d1311a9e-f175-47e1-b690-2c4164ad344b\") " pod="openshift-cluster-node-tuning-operator/tuned-g9rzj"
Sep 17 23:10:50.746234 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:10:50.745331    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/33db9205-3be0-47dc-ac16-e8f623e43afb-serviceca\") pod \"node-ca-vkllk\" (UID: \"33db9205-3be0-47dc-ac16-e8f623e43afb\") " pod="openshift-image-registry/node-ca-vkllk"
Sep 17 23:10:50.746234 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:10:50.745350    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vmqq9\" (UniqueName: \"kubernetes.io/projected/d1311a9e-f175-47e1-b690-2c4164ad344b-kube-api-access-vmqq9\") pod \"tuned-g9rzj\" (UID: \"d1311a9e-f175-47e1-b690-2c4164ad344b\") " pod="openshift-cluster-node-tuning-operator/tuned-g9rzj"
Sep 17 23:10:50.746234 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:10:50.745391    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/d1311a9e-f175-47e1-b690-2c4164ad344b-etc-sysctl-conf\") pod \"tuned-g9rzj\" (UID: \"d1311a9e-f175-47e1-b690-2c4164ad344b\") " pod="openshift-cluster-node-tuning-operator/tuned-g9rzj"
Sep 17 23:10:50.746234 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:10:50.745415    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/33db9205-3be0-47dc-ac16-e8f623e43afb-host\") pod \"node-ca-vkllk\" (UID: \"33db9205-3be0-47dc-ac16-e8f623e43afb\") " pod="openshift-image-registry/node-ca-vkllk"
Sep 17 23:10:50.746234 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:10:50.745442    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/215c7cbb-4cd1-47be-9a44-26c1801b3fcb-cnibin\") pod \"multus-additional-cni-plugins-srxwj\" (UID: \"215c7cbb-4cd1-47be-9a44-26c1801b3fcb\") " pod="openshift-multus/multus-additional-cni-plugins-srxwj"
Sep 17 23:10:50.746234 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:10:50.745446    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/215c7cbb-4cd1-47be-9a44-26c1801b3fcb-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-srxwj\" (UID: \"215c7cbb-4cd1-47be-9a44-26c1801b3fcb\") " pod="openshift-multus/multus-additional-cni-plugins-srxwj"
Sep 17 23:10:50.746234 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:10:50.745466    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xvqw9\" (UniqueName: \"kubernetes.io/projected/215c7cbb-4cd1-47be-9a44-26c1801b3fcb-kube-api-access-xvqw9\") pod \"multus-additional-cni-plugins-srxwj\" (UID: \"215c7cbb-4cd1-47be-9a44-26c1801b3fcb\") " pod="openshift-multus/multus-additional-cni-plugins-srxwj"
Sep 17 23:10:50.746234 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:10:50.745498    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-h8psz\" (UniqueName: \"kubernetes.io/projected/1d92dc9b-45b6-4912-bf69-f0961e86e8ed-kube-api-access-h8psz\") pod \"node-resolver-fzpjd\" (UID: \"1d92dc9b-45b6-4912-bf69-f0961e86e8ed\") " pod="openshift-dns/node-resolver-fzpjd"
Sep 17 23:10:50.746234 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:10:50.745545    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/be8438b1-0b71-4af1-9575-1f005b65a607-etc-selinux\") pod \"aws-ebs-csi-driver-node-2cj88\" (UID: \"be8438b1-0b71-4af1-9575-1f005b65a607\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-2cj88"
Sep 17 23:10:50.746234 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:10:50.745589    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/7e452bbb-7dc6-4c9d-a9b1-76400942f5b4-konnectivity-ca\") pod \"konnectivity-agent-fwwc5\" (UID: \"7e452bbb-7dc6-4c9d-a9b1-76400942f5b4\") " pod="kube-system/konnectivity-agent-fwwc5"
Sep 17 23:10:50.746234 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:10:50.745610    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/d1311a9e-f175-47e1-b690-2c4164ad344b-etc-sysconfig\") pod \"tuned-g9rzj\" (UID: \"d1311a9e-f175-47e1-b690-2c4164ad344b\") " pod="openshift-cluster-node-tuning-operator/tuned-g9rzj"
Sep 17 23:10:50.746744 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:10:50.745598    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/d1311a9e-f175-47e1-b690-2c4164ad344b-etc-systemd\") pod \"tuned-g9rzj\" (UID: \"d1311a9e-f175-47e1-b690-2c4164ad344b\") " pod="openshift-cluster-node-tuning-operator/tuned-g9rzj"
Sep 17 23:10:50.746744 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:10:50.745660    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/d1311a9e-f175-47e1-b690-2c4164ad344b-etc-kubernetes\") pod \"tuned-g9rzj\" (UID: \"d1311a9e-f175-47e1-b690-2c4164ad344b\") " pod="openshift-cluster-node-tuning-operator/tuned-g9rzj"
Sep 17 23:10:50.746744 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:10:50.745705    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/be8438b1-0b71-4af1-9575-1f005b65a607-kubelet-dir\") pod \"aws-ebs-csi-driver-node-2cj88\" (UID: \"be8438b1-0b71-4af1-9575-1f005b65a607\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-2cj88"
Sep 17 23:10:50.746744 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:10:50.745755    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/d1311a9e-f175-47e1-b690-2c4164ad344b-etc-sysctl-d\") pod \"tuned-g9rzj\" (UID: \"d1311a9e-f175-47e1-b690-2c4164ad344b\") " pod="openshift-cluster-node-tuning-operator/tuned-g9rzj"
Sep 17 23:10:50.746744 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:10:50.745772    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/215c7cbb-4cd1-47be-9a44-26c1801b3fcb-cnibin\") pod \"multus-additional-cni-plugins-srxwj\" (UID: \"215c7cbb-4cd1-47be-9a44-26c1801b3fcb\") " pod="openshift-multus/multus-additional-cni-plugins-srxwj"
Sep 17 23:10:50.746744 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:10:50.745804    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/33db9205-3be0-47dc-ac16-e8f623e43afb-host\") pod \"node-ca-vkllk\" (UID: \"33db9205-3be0-47dc-ac16-e8f623e43afb\") " pod="openshift-image-registry/node-ca-vkllk"
Sep 17 23:10:50.746744 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:10:50.745855    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/be8438b1-0b71-4af1-9575-1f005b65a607-sys-fs\") pod \"aws-ebs-csi-driver-node-2cj88\" (UID: \"be8438b1-0b71-4af1-9575-1f005b65a607\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-2cj88"
Sep 17 23:10:50.746744 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:10:50.745907    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/d1311a9e-f175-47e1-b690-2c4164ad344b-host\") pod \"tuned-g9rzj\" (UID: \"d1311a9e-f175-47e1-b690-2c4164ad344b\") " pod="openshift-cluster-node-tuning-operator/tuned-g9rzj"
Sep 17 23:10:50.746744 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:10:50.745974    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/be8438b1-0b71-4af1-9575-1f005b65a607-socket-dir\") pod \"aws-ebs-csi-driver-node-2cj88\" (UID: \"be8438b1-0b71-4af1-9575-1f005b65a607\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-2cj88"
Sep 17 23:10:50.746744 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:10:50.745974    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/d1311a9e-f175-47e1-b690-2c4164ad344b-var-lib-kubelet\") pod \"tuned-g9rzj\" (UID: \"d1311a9e-f175-47e1-b690-2c4164ad344b\") " pod="openshift-cluster-node-tuning-operator/tuned-g9rzj"
Sep 17 23:10:50.746744 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:10:50.746069    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/1d92dc9b-45b6-4912-bf69-f0961e86e8ed-tmp-dir\") pod \"node-resolver-fzpjd\" (UID: \"1d92dc9b-45b6-4912-bf69-f0961e86e8ed\") " pod="openshift-dns/node-resolver-fzpjd"
Sep 17 23:10:50.746744 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:10:50.746117    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/d1311a9e-f175-47e1-b690-2c4164ad344b-etc-sysctl-conf\") pod \"tuned-g9rzj\" (UID: \"d1311a9e-f175-47e1-b690-2c4164ad344b\") " pod="openshift-cluster-node-tuning-operator/tuned-g9rzj"
Sep 17 23:10:50.746744 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:10:50.746504    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/215c7cbb-4cd1-47be-9a44-26c1801b3fcb-cni-binary-copy\") pod \"multus-additional-cni-plugins-srxwj\" (UID: \"215c7cbb-4cd1-47be-9a44-26c1801b3fcb\") " pod="openshift-multus/multus-additional-cni-plugins-srxwj"
Sep 17 23:10:50.747237 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:10:50.746880    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/215c7cbb-4cd1-47be-9a44-26c1801b3fcb-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-srxwj\" (UID: \"215c7cbb-4cd1-47be-9a44-26c1801b3fcb\") " pod="openshift-multus/multus-additional-cni-plugins-srxwj"
Sep 17 23:10:50.747463 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:10:50.747442    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/7e452bbb-7dc6-4c9d-a9b1-76400942f5b4-agent-certs\") pod \"konnectivity-agent-fwwc5\" (UID: \"7e452bbb-7dc6-4c9d-a9b1-76400942f5b4\") " pod="kube-system/konnectivity-agent-fwwc5"
Sep 17 23:10:50.747602 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:10:50.747586    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/d1311a9e-f175-47e1-b690-2c4164ad344b-tmp\") pod \"tuned-g9rzj\" (UID: \"d1311a9e-f175-47e1-b690-2c4164ad344b\") " pod="openshift-cluster-node-tuning-operator/tuned-g9rzj"
Sep 17 23:10:50.747987 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:10:50.747969    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/d1311a9e-f175-47e1-b690-2c4164ad344b-etc-tuned\") pod \"tuned-g9rzj\" (UID: \"d1311a9e-f175-47e1-b690-2c4164ad344b\") " pod="openshift-cluster-node-tuning-operator/tuned-g9rzj"
Sep 17 23:10:50.753250 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:10:50.753217    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-h8psz\" (UniqueName: \"kubernetes.io/projected/1d92dc9b-45b6-4912-bf69-f0961e86e8ed-kube-api-access-h8psz\") pod \"node-resolver-fzpjd\" (UID: \"1d92dc9b-45b6-4912-bf69-f0961e86e8ed\") " pod="openshift-dns/node-resolver-fzpjd"
Sep 17 23:10:50.753357 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:10:50.753289    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-xvqw9\" (UniqueName: \"kubernetes.io/projected/215c7cbb-4cd1-47be-9a44-26c1801b3fcb-kube-api-access-xvqw9\") pod \"multus-additional-cni-plugins-srxwj\" (UID: \"215c7cbb-4cd1-47be-9a44-26c1801b3fcb\") " pod="openshift-multus/multus-additional-cni-plugins-srxwj"
Sep 17 23:10:50.753357 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:10:50.753308    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-qm6g7\" (UniqueName: \"kubernetes.io/projected/33db9205-3be0-47dc-ac16-e8f623e43afb-kube-api-access-qm6g7\") pod \"node-ca-vkllk\" (UID: \"33db9205-3be0-47dc-ac16-e8f623e43afb\") " pod="openshift-image-registry/node-ca-vkllk"
Sep 17 23:10:50.753885 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:10:50.753865    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-vmqq9\" (UniqueName: \"kubernetes.io/projected/d1311a9e-f175-47e1-b690-2c4164ad344b-kube-api-access-vmqq9\") pod \"tuned-g9rzj\" (UID: \"d1311a9e-f175-47e1-b690-2c4164ad344b\") " pod="openshift-cluster-node-tuning-operator/tuned-g9rzj"
Sep 17 23:10:50.754003 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:10:50.753983    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-t9nhg\" (UniqueName: \"kubernetes.io/projected/be8438b1-0b71-4af1-9575-1f005b65a607-kube-api-access-t9nhg\") pod \"aws-ebs-csi-driver-node-2cj88\" (UID: \"be8438b1-0b71-4af1-9575-1f005b65a607\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-2cj88"
Sep 17 23:10:50.755404 ip-10-0-132-188 kubenswrapper[2562]: W0917 23:10:50.755376    2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod380b7a798346051e115745fb0a3f8e3c.slice/crio-bb17b9e43a3fb8a12c52636c6507b73f5f24e105cc369fd02f5c66be020eb88d WatchSource:0}: Error finding container bb17b9e43a3fb8a12c52636c6507b73f5f24e105cc369fd02f5c66be020eb88d: Status 404 returned error can't find the container with id bb17b9e43a3fb8a12c52636c6507b73f5f24e105cc369fd02f5c66be020eb88d
Sep 17 23:10:50.755906 ip-10-0-132-188 kubenswrapper[2562]: W0917 23:10:50.755886    2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3724b678d11f0286466e70c69db04952.slice/crio-1c02d6a007c3dd3220eb6f9002fb27a8bcd357ea23fc363c844c635a8b185ad3 WatchSource:0}: Error finding container 1c02d6a007c3dd3220eb6f9002fb27a8bcd357ea23fc363c844c635a8b185ad3: Status 404 returned error can't find the container with id 1c02d6a007c3dd3220eb6f9002fb27a8bcd357ea23fc363c844c635a8b185ad3
Sep 17 23:10:50.760779 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:10:50.760759    2562 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Sep 17 23:10:50.863809 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:10:50.863735    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-s9wtj"
Sep 17 23:10:50.873529 ip-10-0-132-188 kubenswrapper[2562]: W0917 23:10:50.873505    2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod914194c1_d316_4f82_9beb_af7827f4a570.slice/crio-559896278e1aeb4746260055e1d398718763cc3fe5641322e0f3e4d1303a9d48 WatchSource:0}: Error finding container 559896278e1aeb4746260055e1d398718763cc3fe5641322e0f3e4d1303a9d48: Status 404 returned error can't find the container with id 559896278e1aeb4746260055e1d398718763cc3fe5641322e0f3e4d1303a9d48
Sep 17 23:10:50.878587 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:10:50.878568    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-bvb88"
Sep 17 23:10:50.886501 ip-10-0-132-188 kubenswrapper[2562]: W0917 23:10:50.886481    2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9fd2f397_d1b2_4790_914c_4e9b860b827a.slice/crio-ec10ad7439e5a7a97367fad8c4d0ae60e051dfc7cf77f4beaeb5eaf873c72c80 WatchSource:0}: Error finding container ec10ad7439e5a7a97367fad8c4d0ae60e051dfc7cf77f4beaeb5eaf873c72c80: Status 404 returned error can't find the container with id ec10ad7439e5a7a97367fad8c4d0ae60e051dfc7cf77f4beaeb5eaf873c72c80
Sep 17 23:10:50.887124 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:10:50.887106    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-6xqwg"
Sep 17 23:10:50.895381 ip-10-0-132-188 kubenswrapper[2562]: W0917 23:10:50.895356    2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podcae8ed58_1a8a_40af_a329_a3850797cc58.slice/crio-1849f62512f25442e1928174eb6ef32aeb0941456bbf7377e90e0da4ec34b0cc WatchSource:0}: Error finding container 1849f62512f25442e1928174eb6ef32aeb0941456bbf7377e90e0da4ec34b0cc: Status 404 returned error can't find the container with id 1849f62512f25442e1928174eb6ef32aeb0941456bbf7377e90e0da4ec34b0cc
Sep 17 23:10:50.904848 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:10:50.904828    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-g9rzj"
Sep 17 23:10:50.912367 ip-10-0-132-188 kubenswrapper[2562]: W0917 23:10:50.912344    2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd1311a9e_f175_47e1_b690_2c4164ad344b.slice/crio-7e187d094c0545fd94c83a3e551ffe51406cbf574fe82970b34cb9583190dc1a WatchSource:0}: Error finding container 7e187d094c0545fd94c83a3e551ffe51406cbf574fe82970b34cb9583190dc1a: Status 404 returned error can't find the container with id 7e187d094c0545fd94c83a3e551ffe51406cbf574fe82970b34cb9583190dc1a
Sep 17 23:10:50.920874 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:10:50.920855    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-fwwc5"
Sep 17 23:10:50.926797 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:10:50.926741    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-2cj88"
Sep 17 23:10:50.928889 ip-10-0-132-188 kubenswrapper[2562]: W0917 23:10:50.928866    2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7e452bbb_7dc6_4c9d_a9b1_76400942f5b4.slice/crio-6a6f97da6e63d4cea72021e8a997f4b70745cd5478c95968d4c4732184d81d9f WatchSource:0}: Error finding container 6a6f97da6e63d4cea72021e8a997f4b70745cd5478c95968d4c4732184d81d9f: Status 404 returned error can't find the container with id 6a6f97da6e63d4cea72021e8a997f4b70745cd5478c95968d4c4732184d81d9f
Sep 17 23:10:50.934848 ip-10-0-132-188 kubenswrapper[2562]: W0917 23:10:50.934822    2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podbe8438b1_0b71_4af1_9575_1f005b65a607.slice/crio-7b22dc97a2a0c1433548b3a31cbc70688760eeb2a6f0c7c892e125e525e655c1 WatchSource:0}: Error finding container 7b22dc97a2a0c1433548b3a31cbc70688760eeb2a6f0c7c892e125e525e655c1: Status 404 returned error can't find the container with id 7b22dc97a2a0c1433548b3a31cbc70688760eeb2a6f0c7c892e125e525e655c1
Sep 17 23:10:50.959684 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:10:50.959659    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-fzpjd"
Sep 17 23:10:50.968782 ip-10-0-132-188 kubenswrapper[2562]: W0917 23:10:50.968760    2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1d92dc9b_45b6_4912_bf69_f0961e86e8ed.slice/crio-0f08b82464d6c036cbc33f849f0fd9fb1a0ad194aa0f22d7342507877ff04fa2 WatchSource:0}: Error finding container 0f08b82464d6c036cbc33f849f0fd9fb1a0ad194aa0f22d7342507877ff04fa2: Status 404 returned error can't find the container with id 0f08b82464d6c036cbc33f849f0fd9fb1a0ad194aa0f22d7342507877ff04fa2
Sep 17 23:10:50.977264 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:10:50.977235    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-vkllk"
Sep 17 23:10:50.982977 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:10:50.982956    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-srxwj"
Sep 17 23:10:50.986712 ip-10-0-132-188 kubenswrapper[2562]: W0917 23:10:50.986686    2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod33db9205_3be0_47dc_ac16_e8f623e43afb.slice/crio-9609e87b52d54648bad7a47282efb0f492365ce3311303b4234515d87951174c WatchSource:0}: Error finding container 9609e87b52d54648bad7a47282efb0f492365ce3311303b4234515d87951174c: Status 404 returned error can't find the container with id 9609e87b52d54648bad7a47282efb0f492365ce3311303b4234515d87951174c
Sep 17 23:10:50.991736 ip-10-0-132-188 kubenswrapper[2562]: W0917 23:10:50.991711    2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod215c7cbb_4cd1_47be_9a44_26c1801b3fcb.slice/crio-95b9a56525509a04fbb4e000cee3e837170a31a52e3b35ef785d622926c6fda7 WatchSource:0}: Error finding container 95b9a56525509a04fbb4e000cee3e837170a31a52e3b35ef785d622926c6fda7: Status 404 returned error can't find the container with id 95b9a56525509a04fbb4e000cee3e837170a31a52e3b35ef785d622926c6fda7
Sep 17 23:10:51.149388 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:10:51.149304    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/4453ba96-a67f-46af-b61b-9cd8f39b6d36-metrics-certs\") pod \"network-metrics-daemon-nbccd\" (UID: \"4453ba96-a67f-46af-b61b-9cd8f39b6d36\") " pod="openshift-multus/network-metrics-daemon-nbccd"
Sep 17 23:10:51.149388 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:10:51.149343    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/79765acc-1a30-4a47-add5-aea2bf5f9efb-original-pull-secret\") pod \"global-pull-secret-syncer-jt864\" (UID: \"79765acc-1a30-4a47-add5-aea2bf5f9efb\") " pod="kube-system/global-pull-secret-syncer-jt864"
Sep 17 23:10:51.149565 ip-10-0-132-188 kubenswrapper[2562]: E0917 23:10:51.149461    2562 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Sep 17 23:10:51.149565 ip-10-0-132-188 kubenswrapper[2562]: E0917 23:10:51.149521    2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/79765acc-1a30-4a47-add5-aea2bf5f9efb-original-pull-secret podName:79765acc-1a30-4a47-add5-aea2bf5f9efb nodeName:}" failed. No retries permitted until 2026-09-17 23:10:52.149507493 +0000 UTC m=+4.551404236 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/79765acc-1a30-4a47-add5-aea2bf5f9efb-original-pull-secret") pod "global-pull-secret-syncer-jt864" (UID: "79765acc-1a30-4a47-add5-aea2bf5f9efb") : object "kube-system"/"original-pull-secret" not registered
Sep 17 23:10:51.149565 ip-10-0-132-188 kubenswrapper[2562]: E0917 23:10:51.149461    2562 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 17 23:10:51.149664 ip-10-0-132-188 kubenswrapper[2562]: E0917 23:10:51.149593    2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/4453ba96-a67f-46af-b61b-9cd8f39b6d36-metrics-certs podName:4453ba96-a67f-46af-b61b-9cd8f39b6d36 nodeName:}" failed. No retries permitted until 2026-09-17 23:10:52.149581199 +0000 UTC m=+4.551477955 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/4453ba96-a67f-46af-b61b-9cd8f39b6d36-metrics-certs") pod "network-metrics-daemon-nbccd" (UID: "4453ba96-a67f-46af-b61b-9cd8f39b6d36") : object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 17 23:10:51.250417 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:10:51.250376    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-42cvm\" (UniqueName: \"kubernetes.io/projected/440bce33-9c27-46bd-9d1c-0eebcc14fbc3-kube-api-access-42cvm\") pod \"network-check-target-t9gqw\" (UID: \"440bce33-9c27-46bd-9d1c-0eebcc14fbc3\") " pod="openshift-network-diagnostics/network-check-target-t9gqw"
Sep 17 23:10:51.250692 ip-10-0-132-188 kubenswrapper[2562]: E0917 23:10:51.250645    2562 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Sep 17 23:10:51.250692 ip-10-0-132-188 kubenswrapper[2562]: E0917 23:10:51.250680    2562 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Sep 17 23:10:51.250692 ip-10-0-132-188 kubenswrapper[2562]: E0917 23:10:51.250695    2562 projected.go:196] Error preparing data for projected volume kube-api-access-42cvm for pod openshift-network-diagnostics/network-check-target-t9gqw: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 17 23:10:51.250965 ip-10-0-132-188 kubenswrapper[2562]: E0917 23:10:51.250760    2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/440bce33-9c27-46bd-9d1c-0eebcc14fbc3-kube-api-access-42cvm podName:440bce33-9c27-46bd-9d1c-0eebcc14fbc3 nodeName:}" failed. No retries permitted until 2026-09-17 23:10:52.250739548 +0000 UTC m=+4.652636293 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-api-access-42cvm" (UniqueName: "kubernetes.io/projected/440bce33-9c27-46bd-9d1c-0eebcc14fbc3-kube-api-access-42cvm") pod "network-check-target-t9gqw" (UID: "440bce33-9c27-46bd-9d1c-0eebcc14fbc3") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 17 23:10:51.580240 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:10:51.580049    2562 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-09-16 23:05:50 +0000 UTC" deadline="2028-03-28 20:42:28.903584449 +0000 UTC"
Sep 17 23:10:51.580240 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:10:51.580113    2562 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="13389h31m37.323476186s"
Sep 17 23:10:51.663878 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:10:51.663319    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-nbccd"
Sep 17 23:10:51.663878 ip-10-0-132-188 kubenswrapper[2562]: E0917 23:10:51.663462    2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-nbccd" podUID="4453ba96-a67f-46af-b61b-9cd8f39b6d36"
Sep 17 23:10:51.710565 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:10:51.710500    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-g9rzj" event={"ID":"d1311a9e-f175-47e1-b690-2c4164ad344b","Type":"ContainerStarted","Data":"7e187d094c0545fd94c83a3e551ffe51406cbf574fe82970b34cb9583190dc1a"}
Sep 17 23:10:51.719035 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:10:51.719002    2562 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160"
Sep 17 23:10:51.738084 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:10:51.738043    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-132-188.ec2.internal" event={"ID":"3724b678d11f0286466e70c69db04952","Type":"ContainerStarted","Data":"1c02d6a007c3dd3220eb6f9002fb27a8bcd357ea23fc363c844c635a8b185ad3"}
Sep 17 23:10:51.742268 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:10:51.742231    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-srxwj" event={"ID":"215c7cbb-4cd1-47be-9a44-26c1801b3fcb","Type":"ContainerStarted","Data":"95b9a56525509a04fbb4e000cee3e837170a31a52e3b35ef785d622926c6fda7"}
Sep 17 23:10:51.747788 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:10:51.747750    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-fzpjd" event={"ID":"1d92dc9b-45b6-4912-bf69-f0961e86e8ed","Type":"ContainerStarted","Data":"0f08b82464d6c036cbc33f849f0fd9fb1a0ad194aa0f22d7342507877ff04fa2"}
Sep 17 23:10:51.759647 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:10:51.759611    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-2cj88" event={"ID":"be8438b1-0b71-4af1-9575-1f005b65a607","Type":"ContainerStarted","Data":"7b22dc97a2a0c1433548b3a31cbc70688760eeb2a6f0c7c892e125e525e655c1"}
Sep 17 23:10:51.776994 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:10:51.776950    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-6xqwg" event={"ID":"cae8ed58-1a8a-40af-a329-a3850797cc58","Type":"ContainerStarted","Data":"1849f62512f25442e1928174eb6ef32aeb0941456bbf7377e90e0da4ec34b0cc"}
Sep 17 23:10:51.785432 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:10:51.784008    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-bvb88" event={"ID":"9fd2f397-d1b2-4790-914c-4e9b860b827a","Type":"ContainerStarted","Data":"ec10ad7439e5a7a97367fad8c4d0ae60e051dfc7cf77f4beaeb5eaf873c72c80"}
Sep 17 23:10:51.812645 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:10:51.812535    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-s9wtj" event={"ID":"914194c1-d316-4f82-9beb-af7827f4a570","Type":"ContainerStarted","Data":"559896278e1aeb4746260055e1d398718763cc3fe5641322e0f3e4d1303a9d48"}
Sep 17 23:10:51.836686 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:10:51.836592    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-132-188.ec2.internal" event={"ID":"380b7a798346051e115745fb0a3f8e3c","Type":"ContainerStarted","Data":"bb17b9e43a3fb8a12c52636c6507b73f5f24e105cc369fd02f5c66be020eb88d"}
Sep 17 23:10:51.860824 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:10:51.860781    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-vkllk" event={"ID":"33db9205-3be0-47dc-ac16-e8f623e43afb","Type":"ContainerStarted","Data":"9609e87b52d54648bad7a47282efb0f492365ce3311303b4234515d87951174c"}
Sep 17 23:10:51.873893 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:10:51.873852    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-fwwc5" event={"ID":"7e452bbb-7dc6-4c9d-a9b1-76400942f5b4","Type":"ContainerStarted","Data":"6a6f97da6e63d4cea72021e8a997f4b70745cd5478c95968d4c4732184d81d9f"}
Sep 17 23:10:52.157663 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:10:52.157577    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/4453ba96-a67f-46af-b61b-9cd8f39b6d36-metrics-certs\") pod \"network-metrics-daemon-nbccd\" (UID: \"4453ba96-a67f-46af-b61b-9cd8f39b6d36\") " pod="openshift-multus/network-metrics-daemon-nbccd"
Sep 17 23:10:52.157663 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:10:52.157629    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/79765acc-1a30-4a47-add5-aea2bf5f9efb-original-pull-secret\") pod \"global-pull-secret-syncer-jt864\" (UID: \"79765acc-1a30-4a47-add5-aea2bf5f9efb\") " pod="kube-system/global-pull-secret-syncer-jt864"
Sep 17 23:10:52.157891 ip-10-0-132-188 kubenswrapper[2562]: E0917 23:10:52.157803    2562 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Sep 17 23:10:52.157891 ip-10-0-132-188 kubenswrapper[2562]: E0917 23:10:52.157869    2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/79765acc-1a30-4a47-add5-aea2bf5f9efb-original-pull-secret podName:79765acc-1a30-4a47-add5-aea2bf5f9efb nodeName:}" failed. No retries permitted until 2026-09-17 23:10:54.157849284 +0000 UTC m=+6.559746049 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/79765acc-1a30-4a47-add5-aea2bf5f9efb-original-pull-secret") pod "global-pull-secret-syncer-jt864" (UID: "79765acc-1a30-4a47-add5-aea2bf5f9efb") : object "kube-system"/"original-pull-secret" not registered
Sep 17 23:10:52.158328 ip-10-0-132-188 kubenswrapper[2562]: E0917 23:10:52.158307    2562 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 17 23:10:52.158412 ip-10-0-132-188 kubenswrapper[2562]: E0917 23:10:52.158370    2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/4453ba96-a67f-46af-b61b-9cd8f39b6d36-metrics-certs podName:4453ba96-a67f-46af-b61b-9cd8f39b6d36 nodeName:}" failed. No retries permitted until 2026-09-17 23:10:54.158354455 +0000 UTC m=+6.560251210 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/4453ba96-a67f-46af-b61b-9cd8f39b6d36-metrics-certs") pod "network-metrics-daemon-nbccd" (UID: "4453ba96-a67f-46af-b61b-9cd8f39b6d36") : object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 17 23:10:52.258307 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:10:52.258266    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-42cvm\" (UniqueName: \"kubernetes.io/projected/440bce33-9c27-46bd-9d1c-0eebcc14fbc3-kube-api-access-42cvm\") pod \"network-check-target-t9gqw\" (UID: \"440bce33-9c27-46bd-9d1c-0eebcc14fbc3\") " pod="openshift-network-diagnostics/network-check-target-t9gqw"
Sep 17 23:10:52.258541 ip-10-0-132-188 kubenswrapper[2562]: E0917 23:10:52.258506    2562 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Sep 17 23:10:52.258541 ip-10-0-132-188 kubenswrapper[2562]: E0917 23:10:52.258531    2562 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Sep 17 23:10:52.258682 ip-10-0-132-188 kubenswrapper[2562]: E0917 23:10:52.258544    2562 projected.go:196] Error preparing data for projected volume kube-api-access-42cvm for pod openshift-network-diagnostics/network-check-target-t9gqw: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 17 23:10:52.258682 ip-10-0-132-188 kubenswrapper[2562]: E0917 23:10:52.258623    2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/440bce33-9c27-46bd-9d1c-0eebcc14fbc3-kube-api-access-42cvm podName:440bce33-9c27-46bd-9d1c-0eebcc14fbc3 nodeName:}" failed. No retries permitted until 2026-09-17 23:10:54.258584481 +0000 UTC m=+6.660481241 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-api-access-42cvm" (UniqueName: "kubernetes.io/projected/440bce33-9c27-46bd-9d1c-0eebcc14fbc3-kube-api-access-42cvm") pod "network-check-target-t9gqw" (UID: "440bce33-9c27-46bd-9d1c-0eebcc14fbc3") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 17 23:10:52.339045 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:10:52.339014    2562 reflector.go:439] "Caches populated" type="*v1.Node" reflector="k8s.io/client-go/informers/factory.go:160"
Sep 17 23:10:52.581053 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:10:52.580968    2562 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-09-16 23:05:50 +0000 UTC" deadline="2028-06-30 04:29:12.818763216 +0000 UTC"
Sep 17 23:10:52.581053 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:10:52.581004    2562 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="15629h18m20.237763951s"
Sep 17 23:10:52.661904 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:10:52.661456    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-jt864"
Sep 17 23:10:52.661904 ip-10-0-132-188 kubenswrapper[2562]: E0917 23:10:52.661592    2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-jt864" podUID="79765acc-1a30-4a47-add5-aea2bf5f9efb"
Sep 17 23:10:52.662128 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:10:52.661998    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-t9gqw"
Sep 17 23:10:52.662128 ip-10-0-132-188 kubenswrapper[2562]: E0917 23:10:52.662087    2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-t9gqw" podUID="440bce33-9c27-46bd-9d1c-0eebcc14fbc3"
Sep 17 23:10:53.661327 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:10:53.661291    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-nbccd"
Sep 17 23:10:53.661766 ip-10-0-132-188 kubenswrapper[2562]: E0917 23:10:53.661511    2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-nbccd" podUID="4453ba96-a67f-46af-b61b-9cd8f39b6d36"
Sep 17 23:10:54.175065 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:10:54.175029    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/4453ba96-a67f-46af-b61b-9cd8f39b6d36-metrics-certs\") pod \"network-metrics-daemon-nbccd\" (UID: \"4453ba96-a67f-46af-b61b-9cd8f39b6d36\") " pod="openshift-multus/network-metrics-daemon-nbccd"
Sep 17 23:10:54.175322 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:10:54.175086    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/79765acc-1a30-4a47-add5-aea2bf5f9efb-original-pull-secret\") pod \"global-pull-secret-syncer-jt864\" (UID: \"79765acc-1a30-4a47-add5-aea2bf5f9efb\") " pod="kube-system/global-pull-secret-syncer-jt864"
Sep 17 23:10:54.175322 ip-10-0-132-188 kubenswrapper[2562]: E0917 23:10:54.175242    2562 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Sep 17 23:10:54.175461 ip-10-0-132-188 kubenswrapper[2562]: E0917 23:10:54.175332    2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/79765acc-1a30-4a47-add5-aea2bf5f9efb-original-pull-secret podName:79765acc-1a30-4a47-add5-aea2bf5f9efb nodeName:}" failed. No retries permitted until 2026-09-17 23:10:58.175314503 +0000 UTC m=+10.577211250 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/79765acc-1a30-4a47-add5-aea2bf5f9efb-original-pull-secret") pod "global-pull-secret-syncer-jt864" (UID: "79765acc-1a30-4a47-add5-aea2bf5f9efb") : object "kube-system"/"original-pull-secret" not registered
Sep 17 23:10:54.175887 ip-10-0-132-188 kubenswrapper[2562]: E0917 23:10:54.175727    2562 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 17 23:10:54.175887 ip-10-0-132-188 kubenswrapper[2562]: E0917 23:10:54.175779    2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/4453ba96-a67f-46af-b61b-9cd8f39b6d36-metrics-certs podName:4453ba96-a67f-46af-b61b-9cd8f39b6d36 nodeName:}" failed. No retries permitted until 2026-09-17 23:10:58.175762776 +0000 UTC m=+10.577659525 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/4453ba96-a67f-46af-b61b-9cd8f39b6d36-metrics-certs") pod "network-metrics-daemon-nbccd" (UID: "4453ba96-a67f-46af-b61b-9cd8f39b6d36") : object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 17 23:10:54.276637 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:10:54.275935    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-42cvm\" (UniqueName: \"kubernetes.io/projected/440bce33-9c27-46bd-9d1c-0eebcc14fbc3-kube-api-access-42cvm\") pod \"network-check-target-t9gqw\" (UID: \"440bce33-9c27-46bd-9d1c-0eebcc14fbc3\") " pod="openshift-network-diagnostics/network-check-target-t9gqw"
Sep 17 23:10:54.276637 ip-10-0-132-188 kubenswrapper[2562]: E0917 23:10:54.276190    2562 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Sep 17 23:10:54.276637 ip-10-0-132-188 kubenswrapper[2562]: E0917 23:10:54.276210    2562 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Sep 17 23:10:54.276637 ip-10-0-132-188 kubenswrapper[2562]: E0917 23:10:54.276223    2562 projected.go:196] Error preparing data for projected volume kube-api-access-42cvm for pod openshift-network-diagnostics/network-check-target-t9gqw: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 17 23:10:54.276637 ip-10-0-132-188 kubenswrapper[2562]: E0917 23:10:54.276281    2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/440bce33-9c27-46bd-9d1c-0eebcc14fbc3-kube-api-access-42cvm podName:440bce33-9c27-46bd-9d1c-0eebcc14fbc3 nodeName:}" failed. No retries permitted until 2026-09-17 23:10:58.276262991 +0000 UTC m=+10.678159737 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-api-access-42cvm" (UniqueName: "kubernetes.io/projected/440bce33-9c27-46bd-9d1c-0eebcc14fbc3-kube-api-access-42cvm") pod "network-check-target-t9gqw" (UID: "440bce33-9c27-46bd-9d1c-0eebcc14fbc3") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 17 23:10:54.662241 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:10:54.661361    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-t9gqw"
Sep 17 23:10:54.662241 ip-10-0-132-188 kubenswrapper[2562]: E0917 23:10:54.661498    2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-t9gqw" podUID="440bce33-9c27-46bd-9d1c-0eebcc14fbc3"
Sep 17 23:10:54.662241 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:10:54.661583    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-jt864"
Sep 17 23:10:54.662241 ip-10-0-132-188 kubenswrapper[2562]: E0917 23:10:54.661665    2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-jt864" podUID="79765acc-1a30-4a47-add5-aea2bf5f9efb"
Sep 17 23:10:55.661452 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:10:55.661416    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-nbccd"
Sep 17 23:10:55.661642 ip-10-0-132-188 kubenswrapper[2562]: E0917 23:10:55.661575    2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-nbccd" podUID="4453ba96-a67f-46af-b61b-9cd8f39b6d36"
Sep 17 23:10:56.661389 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:10:56.661350    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-t9gqw"
Sep 17 23:10:56.661873 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:10:56.661410    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-jt864"
Sep 17 23:10:56.661873 ip-10-0-132-188 kubenswrapper[2562]: E0917 23:10:56.661508    2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-t9gqw" podUID="440bce33-9c27-46bd-9d1c-0eebcc14fbc3"
Sep 17 23:10:56.661873 ip-10-0-132-188 kubenswrapper[2562]: E0917 23:10:56.661724    2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-jt864" podUID="79765acc-1a30-4a47-add5-aea2bf5f9efb"
Sep 17 23:10:57.663544 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:10:57.663030    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-nbccd"
Sep 17 23:10:57.663544 ip-10-0-132-188 kubenswrapper[2562]: E0917 23:10:57.663186    2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-nbccd" podUID="4453ba96-a67f-46af-b61b-9cd8f39b6d36"
Sep 17 23:10:58.213030 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:10:58.212991    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/4453ba96-a67f-46af-b61b-9cd8f39b6d36-metrics-certs\") pod \"network-metrics-daemon-nbccd\" (UID: \"4453ba96-a67f-46af-b61b-9cd8f39b6d36\") " pod="openshift-multus/network-metrics-daemon-nbccd"
Sep 17 23:10:58.213227 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:10:58.213041    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/79765acc-1a30-4a47-add5-aea2bf5f9efb-original-pull-secret\") pod \"global-pull-secret-syncer-jt864\" (UID: \"79765acc-1a30-4a47-add5-aea2bf5f9efb\") " pod="kube-system/global-pull-secret-syncer-jt864"
Sep 17 23:10:58.213304 ip-10-0-132-188 kubenswrapper[2562]: E0917 23:10:58.213225    2562 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Sep 17 23:10:58.213304 ip-10-0-132-188 kubenswrapper[2562]: E0917 23:10:58.213282    2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/79765acc-1a30-4a47-add5-aea2bf5f9efb-original-pull-secret podName:79765acc-1a30-4a47-add5-aea2bf5f9efb nodeName:}" failed. No retries permitted until 2026-09-17 23:11:06.213267608 +0000 UTC m=+18.615164349 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/79765acc-1a30-4a47-add5-aea2bf5f9efb-original-pull-secret") pod "global-pull-secret-syncer-jt864" (UID: "79765acc-1a30-4a47-add5-aea2bf5f9efb") : object "kube-system"/"original-pull-secret" not registered
Sep 17 23:10:58.213692 ip-10-0-132-188 kubenswrapper[2562]: E0917 23:10:58.213669    2562 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 17 23:10:58.213812 ip-10-0-132-188 kubenswrapper[2562]: E0917 23:10:58.213729    2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/4453ba96-a67f-46af-b61b-9cd8f39b6d36-metrics-certs podName:4453ba96-a67f-46af-b61b-9cd8f39b6d36 nodeName:}" failed. No retries permitted until 2026-09-17 23:11:06.213710099 +0000 UTC m=+18.615606846 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/4453ba96-a67f-46af-b61b-9cd8f39b6d36-metrics-certs") pod "network-metrics-daemon-nbccd" (UID: "4453ba96-a67f-46af-b61b-9cd8f39b6d36") : object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 17 23:10:58.313977 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:10:58.313799    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-42cvm\" (UniqueName: \"kubernetes.io/projected/440bce33-9c27-46bd-9d1c-0eebcc14fbc3-kube-api-access-42cvm\") pod \"network-check-target-t9gqw\" (UID: \"440bce33-9c27-46bd-9d1c-0eebcc14fbc3\") " pod="openshift-network-diagnostics/network-check-target-t9gqw"
Sep 17 23:10:58.314202 ip-10-0-132-188 kubenswrapper[2562]: E0917 23:10:58.314007    2562 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Sep 17 23:10:58.314202 ip-10-0-132-188 kubenswrapper[2562]: E0917 23:10:58.314031    2562 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Sep 17 23:10:58.314202 ip-10-0-132-188 kubenswrapper[2562]: E0917 23:10:58.314043    2562 projected.go:196] Error preparing data for projected volume kube-api-access-42cvm for pod openshift-network-diagnostics/network-check-target-t9gqw: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 17 23:10:58.314202 ip-10-0-132-188 kubenswrapper[2562]: E0917 23:10:58.314116    2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/440bce33-9c27-46bd-9d1c-0eebcc14fbc3-kube-api-access-42cvm podName:440bce33-9c27-46bd-9d1c-0eebcc14fbc3 nodeName:}" failed. No retries permitted until 2026-09-17 23:11:06.314096436 +0000 UTC m=+18.715993184 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "kube-api-access-42cvm" (UniqueName: "kubernetes.io/projected/440bce33-9c27-46bd-9d1c-0eebcc14fbc3-kube-api-access-42cvm") pod "network-check-target-t9gqw" (UID: "440bce33-9c27-46bd-9d1c-0eebcc14fbc3") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 17 23:10:58.661471 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:10:58.661373    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-jt864"
Sep 17 23:10:58.661471 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:10:58.661423    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-t9gqw"
Sep 17 23:10:58.661699 ip-10-0-132-188 kubenswrapper[2562]: E0917 23:10:58.661508    2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-jt864" podUID="79765acc-1a30-4a47-add5-aea2bf5f9efb"
Sep 17 23:10:58.661699 ip-10-0-132-188 kubenswrapper[2562]: E0917 23:10:58.661640    2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-t9gqw" podUID="440bce33-9c27-46bd-9d1c-0eebcc14fbc3"
Sep 17 23:10:59.661744 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:10:59.661708    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-nbccd"
Sep 17 23:10:59.662211 ip-10-0-132-188 kubenswrapper[2562]: E0917 23:10:59.661854    2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-nbccd" podUID="4453ba96-a67f-46af-b61b-9cd8f39b6d36"
Sep 17 23:11:00.660954 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:11:00.660860    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-jt864"
Sep 17 23:11:00.661118 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:11:00.660860    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-t9gqw"
Sep 17 23:11:00.661118 ip-10-0-132-188 kubenswrapper[2562]: E0917 23:11:00.660989    2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-jt864" podUID="79765acc-1a30-4a47-add5-aea2bf5f9efb"
Sep 17 23:11:00.661118 ip-10-0-132-188 kubenswrapper[2562]: E0917 23:11:00.661066    2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-t9gqw" podUID="440bce33-9c27-46bd-9d1c-0eebcc14fbc3"
Sep 17 23:11:01.661021 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:11:01.660983    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-nbccd"
Sep 17 23:11:01.661494 ip-10-0-132-188 kubenswrapper[2562]: E0917 23:11:01.661142    2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-nbccd" podUID="4453ba96-a67f-46af-b61b-9cd8f39b6d36"
Sep 17 23:11:02.661544 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:11:02.661504    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-t9gqw"
Sep 17 23:11:02.661969 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:11:02.661621    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-jt864"
Sep 17 23:11:02.661969 ip-10-0-132-188 kubenswrapper[2562]: E0917 23:11:02.661717    2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-jt864" podUID="79765acc-1a30-4a47-add5-aea2bf5f9efb"
Sep 17 23:11:02.661969 ip-10-0-132-188 kubenswrapper[2562]: E0917 23:11:02.661718    2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-t9gqw" podUID="440bce33-9c27-46bd-9d1c-0eebcc14fbc3"
Sep 17 23:11:03.661032 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:11:03.660995    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-nbccd"
Sep 17 23:11:03.661235 ip-10-0-132-188 kubenswrapper[2562]: E0917 23:11:03.661144    2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-nbccd" podUID="4453ba96-a67f-46af-b61b-9cd8f39b6d36"
Sep 17 23:11:04.661346 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:11:04.661307    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-t9gqw"
Sep 17 23:11:04.661787 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:11:04.661310    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-jt864"
Sep 17 23:11:04.661787 ip-10-0-132-188 kubenswrapper[2562]: E0917 23:11:04.661431    2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-t9gqw" podUID="440bce33-9c27-46bd-9d1c-0eebcc14fbc3"
Sep 17 23:11:04.661787 ip-10-0-132-188 kubenswrapper[2562]: E0917 23:11:04.661519    2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-jt864" podUID="79765acc-1a30-4a47-add5-aea2bf5f9efb"
Sep 17 23:11:05.661422 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:11:05.661388    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-nbccd"
Sep 17 23:11:05.661862 ip-10-0-132-188 kubenswrapper[2562]: E0917 23:11:05.661535    2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-nbccd" podUID="4453ba96-a67f-46af-b61b-9cd8f39b6d36"
Sep 17 23:11:06.277272 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:11:06.277228    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/4453ba96-a67f-46af-b61b-9cd8f39b6d36-metrics-certs\") pod \"network-metrics-daemon-nbccd\" (UID: \"4453ba96-a67f-46af-b61b-9cd8f39b6d36\") " pod="openshift-multus/network-metrics-daemon-nbccd"
Sep 17 23:11:06.277272 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:11:06.277282    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/79765acc-1a30-4a47-add5-aea2bf5f9efb-original-pull-secret\") pod \"global-pull-secret-syncer-jt864\" (UID: \"79765acc-1a30-4a47-add5-aea2bf5f9efb\") " pod="kube-system/global-pull-secret-syncer-jt864"
Sep 17 23:11:06.277556 ip-10-0-132-188 kubenswrapper[2562]: E0917 23:11:06.277406    2562 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Sep 17 23:11:06.277556 ip-10-0-132-188 kubenswrapper[2562]: E0917 23:11:06.277418    2562 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 17 23:11:06.277556 ip-10-0-132-188 kubenswrapper[2562]: E0917 23:11:06.277471    2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/79765acc-1a30-4a47-add5-aea2bf5f9efb-original-pull-secret podName:79765acc-1a30-4a47-add5-aea2bf5f9efb nodeName:}" failed. No retries permitted until 2026-09-17 23:11:22.277454036 +0000 UTC m=+34.679350780 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/79765acc-1a30-4a47-add5-aea2bf5f9efb-original-pull-secret") pod "global-pull-secret-syncer-jt864" (UID: "79765acc-1a30-4a47-add5-aea2bf5f9efb") : object "kube-system"/"original-pull-secret" not registered
Sep 17 23:11:06.277556 ip-10-0-132-188 kubenswrapper[2562]: E0917 23:11:06.277486    2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/4453ba96-a67f-46af-b61b-9cd8f39b6d36-metrics-certs podName:4453ba96-a67f-46af-b61b-9cd8f39b6d36 nodeName:}" failed. No retries permitted until 2026-09-17 23:11:22.277478988 +0000 UTC m=+34.679375730 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/4453ba96-a67f-46af-b61b-9cd8f39b6d36-metrics-certs") pod "network-metrics-daemon-nbccd" (UID: "4453ba96-a67f-46af-b61b-9cd8f39b6d36") : object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 17 23:11:06.377784 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:11:06.377740    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-42cvm\" (UniqueName: \"kubernetes.io/projected/440bce33-9c27-46bd-9d1c-0eebcc14fbc3-kube-api-access-42cvm\") pod \"network-check-target-t9gqw\" (UID: \"440bce33-9c27-46bd-9d1c-0eebcc14fbc3\") " pod="openshift-network-diagnostics/network-check-target-t9gqw"
Sep 17 23:11:06.377970 ip-10-0-132-188 kubenswrapper[2562]: E0917 23:11:06.377932    2562 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Sep 17 23:11:06.377970 ip-10-0-132-188 kubenswrapper[2562]: E0917 23:11:06.377963    2562 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Sep 17 23:11:06.378086 ip-10-0-132-188 kubenswrapper[2562]: E0917 23:11:06.377976    2562 projected.go:196] Error preparing data for projected volume kube-api-access-42cvm for pod openshift-network-diagnostics/network-check-target-t9gqw: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 17 23:11:06.378086 ip-10-0-132-188 kubenswrapper[2562]: E0917 23:11:06.378040    2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/440bce33-9c27-46bd-9d1c-0eebcc14fbc3-kube-api-access-42cvm podName:440bce33-9c27-46bd-9d1c-0eebcc14fbc3 nodeName:}" failed. No retries permitted until 2026-09-17 23:11:22.378023324 +0000 UTC m=+34.779920078 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "kube-api-access-42cvm" (UniqueName: "kubernetes.io/projected/440bce33-9c27-46bd-9d1c-0eebcc14fbc3-kube-api-access-42cvm") pod "network-check-target-t9gqw" (UID: "440bce33-9c27-46bd-9d1c-0eebcc14fbc3") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 17 23:11:06.660764 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:11:06.660677    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-t9gqw"
Sep 17 23:11:06.661033 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:11:06.660683    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-jt864"
Sep 17 23:11:06.661033 ip-10-0-132-188 kubenswrapper[2562]: E0917 23:11:06.660815    2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-t9gqw" podUID="440bce33-9c27-46bd-9d1c-0eebcc14fbc3"
Sep 17 23:11:06.661033 ip-10-0-132-188 kubenswrapper[2562]: E0917 23:11:06.660877    2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-jt864" podUID="79765acc-1a30-4a47-add5-aea2bf5f9efb"
Sep 17 23:11:07.661344 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:11:07.661304    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-nbccd"
Sep 17 23:11:07.661819 ip-10-0-132-188 kubenswrapper[2562]: E0917 23:11:07.661423    2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-nbccd" podUID="4453ba96-a67f-46af-b61b-9cd8f39b6d36"
Sep 17 23:11:08.661273 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:11:08.661236    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-jt864"
Sep 17 23:11:08.661481 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:11:08.661232    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-t9gqw"
Sep 17 23:11:08.661481 ip-10-0-132-188 kubenswrapper[2562]: E0917 23:11:08.661387    2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-jt864" podUID="79765acc-1a30-4a47-add5-aea2bf5f9efb"
Sep 17 23:11:08.661481 ip-10-0-132-188 kubenswrapper[2562]: E0917 23:11:08.661435    2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-t9gqw" podUID="440bce33-9c27-46bd-9d1c-0eebcc14fbc3"
Sep 17 23:11:09.663820 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:11:09.663788    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-nbccd"
Sep 17 23:11:09.664288 ip-10-0-132-188 kubenswrapper[2562]: E0917 23:11:09.663905    2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-nbccd" podUID="4453ba96-a67f-46af-b61b-9cd8f39b6d36"
Sep 17 23:11:10.661654 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:11:10.661474    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-jt864"
Sep 17 23:11:10.661750 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:11:10.661498    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-t9gqw"
Sep 17 23:11:10.661750 ip-10-0-132-188 kubenswrapper[2562]: E0917 23:11:10.661733    2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-jt864" podUID="79765acc-1a30-4a47-add5-aea2bf5f9efb"
Sep 17 23:11:10.661881 ip-10-0-132-188 kubenswrapper[2562]: E0917 23:11:10.661857    2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-t9gqw" podUID="440bce33-9c27-46bd-9d1c-0eebcc14fbc3"
Sep 17 23:11:10.918612 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:11:10.918581    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-vkllk" event={"ID":"33db9205-3be0-47dc-ac16-e8f623e43afb","Type":"ContainerStarted","Data":"a2dfc804de4060091c1d02517ce2170ebd8c71d8f94186c561118cf1df99ce16"}
Sep 17 23:11:10.920045 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:11:10.920011    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-fwwc5" event={"ID":"7e452bbb-7dc6-4c9d-a9b1-76400942f5b4","Type":"ContainerStarted","Data":"af7920b3d89c5a1842dcf46b8af78767700b3894b5bd7c75fbc824602ac49879"}
Sep 17 23:11:10.921442 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:11:10.921420    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-g9rzj" event={"ID":"d1311a9e-f175-47e1-b690-2c4164ad344b","Type":"ContainerStarted","Data":"33ee076e9ebd40a0e6d4e600a46b1cfa14c6863e5f4f3daadaaaba87cf318010"}
Sep 17 23:11:10.922867 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:11:10.922843    2562 generic.go:356] "Generic (PLEG): container finished" podID="3724b678d11f0286466e70c69db04952" containerID="b1adf59d2aad550a05d5b6604b89d600d4bfe20c5b55b4d51d07d18a016f747b" exitCode=0
Sep 17 23:11:10.922960 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:11:10.922909    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-132-188.ec2.internal" event={"ID":"3724b678d11f0286466e70c69db04952","Type":"ContainerDied","Data":"b1adf59d2aad550a05d5b6604b89d600d4bfe20c5b55b4d51d07d18a016f747b"}
Sep 17 23:11:10.924482 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:11:10.924456    2562 generic.go:356] "Generic (PLEG): container finished" podID="215c7cbb-4cd1-47be-9a44-26c1801b3fcb" containerID="c97d953f92672baef3879d626b34139c57c8ebf15e0e95ffe41117d4955ae6e9" exitCode=0
Sep 17 23:11:10.924570 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:11:10.924536    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-srxwj" event={"ID":"215c7cbb-4cd1-47be-9a44-26c1801b3fcb","Type":"ContainerDied","Data":"c97d953f92672baef3879d626b34139c57c8ebf15e0e95ffe41117d4955ae6e9"}
Sep 17 23:11:10.926127 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:11:10.926101    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-fzpjd" event={"ID":"1d92dc9b-45b6-4912-bf69-f0961e86e8ed","Type":"ContainerStarted","Data":"a877b2f655b6c34ec4df255f9ca4b366fcf26ce46e57eca1788875b1e63e4f1e"}
Sep 17 23:11:10.927683 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:11:10.927658    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-2cj88" event={"ID":"be8438b1-0b71-4af1-9575-1f005b65a607","Type":"ContainerStarted","Data":"2bd1988dbc1243902b1053a8ce076a5b1319fedacd881d90463ed2876f04c349"}
Sep 17 23:11:10.930510 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:11:10.930487    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-6xqwg" event={"ID":"cae8ed58-1a8a-40af-a329-a3850797cc58","Type":"ContainerStarted","Data":"6528583be63817608bc59cbbc7a3486ffd0f6fd23d5155e71eb2901a0d9fca19"}
Sep 17 23:11:10.930578 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:11:10.930547    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-6xqwg" event={"ID":"cae8ed58-1a8a-40af-a329-a3850797cc58","Type":"ContainerStarted","Data":"52bb4506cc844712f5dd5d12a79c360fd894bcd22208cfcc983f6e793f7aeb9c"}
Sep 17 23:11:10.930630 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:11:10.930582    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-6xqwg" event={"ID":"cae8ed58-1a8a-40af-a329-a3850797cc58","Type":"ContainerStarted","Data":"259b4e0156b5534aaad75848fb4a83cf19b8e3f5ae2bcc89dd37f8c49202b6a4"}
Sep 17 23:11:10.930630 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:11:10.930623    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-6xqwg" event={"ID":"cae8ed58-1a8a-40af-a329-a3850797cc58","Type":"ContainerStarted","Data":"570f44f6f57dee782ea35ee78faefcaf1d89a922690261d7a73628ff6517114e"}
Sep 17 23:11:10.930706 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:11:10.930666    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-6xqwg" event={"ID":"cae8ed58-1a8a-40af-a329-a3850797cc58","Type":"ContainerStarted","Data":"2102750d29d3fb21ccc6b63a06c3ddfddcfab31a5bfaa7044e07fc78de0b5fad"}
Sep 17 23:11:10.931862 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:11:10.931844    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-s9wtj" event={"ID":"914194c1-d316-4f82-9beb-af7827f4a570","Type":"ContainerStarted","Data":"b0ddef80fcf6c2242c4c5f31e8686773a2ef4a77c42a626e35bca040409d8145"}
Sep 17 23:11:10.933086 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:11:10.933069    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-132-188.ec2.internal" event={"ID":"380b7a798346051e115745fb0a3f8e3c","Type":"ContainerStarted","Data":"96c94b42146246f1db03ce49d1f03a9bfc85c579a5d77fc30e35b14ca9915697"}
Sep 17 23:11:10.953756 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:11:10.952609    2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/node-resolver-fzpjd" podStartSLOduration=2.6809682 podStartE2EDuration="21.952590348s" totalImagesPullingTime="19.271622148s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-17 23:10:49 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-17 23:11:10.952276413 +0000 UTC m=+23.354173179" watchObservedRunningTime="2026-09-17 23:11:10.952590348 +0000 UTC m=+23.354487114"
Sep 17 23:11:10.966658 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:11:10.966608    2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/node-ca-vkllk" podStartSLOduration=2.8162786 podStartE2EDuration="21.966593865s" totalImagesPullingTime="19.150315265s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-17 23:10:49 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-17 23:11:10.966504821 +0000 UTC m=+23.368401584" watchObservedRunningTime="2026-09-17 23:11:10.966593865 +0000 UTC m=+23.368490628"
Sep 17 23:11:11.006338 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:11:11.006277    2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/kube-apiserver-proxy-ip-10-0-132-188.ec2.internal" podStartSLOduration=21.006256772 podStartE2EDuration="21.006256772s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-17 23:10:50 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-17 23:11:11.005547632 +0000 UTC m=+23.407444397" watchObservedRunningTime="2026-09-17 23:11:11.006256772 +0000 UTC m=+23.408153537"
Sep 17 23:11:11.047751 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:11:11.047557    2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-node-tuning-operator/tuned-g9rzj" podStartSLOduration=2.723170683 podStartE2EDuration="22.047540578s" totalImagesPullingTime="19.324369895s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-17 23:10:49 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-17 23:11:11.047370356 +0000 UTC m=+23.449267121" watchObservedRunningTime="2026-09-17 23:11:11.047540578 +0000 UTC m=+23.449437341"
Sep 17 23:11:11.048250 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:11:11.048206    2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-s9wtj" podStartSLOduration=2.515501336 podStartE2EDuration="22.048189535s" totalImagesPullingTime="19.532688199s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-17 23:10:49 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-17 23:11:11.027470873 +0000 UTC m=+23.429367628" watchObservedRunningTime="2026-09-17 23:11:11.048189535 +0000 UTC m=+23.450086290"
Sep 17 23:11:11.061231 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:11:11.061145    2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/konnectivity-agent-fwwc5" podStartSLOduration=2.753535687 podStartE2EDuration="22.061123908s" totalImagesPullingTime="19.307588221s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-17 23:10:49 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-17 23:11:11.060582226 +0000 UTC m=+23.462478986" watchObservedRunningTime="2026-09-17 23:11:11.061123908 +0000 UTC m=+23.463020673"
Sep 17 23:11:11.093351 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:11:11.093321    2562 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="kube-system/konnectivity-agent-fwwc5"
Sep 17 23:11:11.094095 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:11:11.094073    2562 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="kube-system/konnectivity-agent-fwwc5"
Sep 17 23:11:11.661316 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:11:11.661273    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-nbccd"
Sep 17 23:11:11.661538 ip-10-0-132-188 kubenswrapper[2562]: E0917 23:11:11.661514    2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-nbccd" podUID="4453ba96-a67f-46af-b61b-9cd8f39b6d36"
Sep 17 23:11:11.938835 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:11:11.938797    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-6xqwg" event={"ID":"cae8ed58-1a8a-40af-a329-a3850797cc58","Type":"ContainerStarted","Data":"e7dd0f210bb296d9fda97fb2b3cff32551282939839b908834782193931744ff"}
Sep 17 23:11:11.941026 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:11:11.940982    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-132-188.ec2.internal" event={"ID":"3724b678d11f0286466e70c69db04952","Type":"ContainerStarted","Data":"9fe74b0e3d77270c1e5047802f30853fbce05d3bb88f637bd8a12016af9bc20d"}
Sep 17 23:11:11.942195 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:11:11.942157    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kube-system/konnectivity-agent-fwwc5"
Sep 17 23:11:11.942313 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:11:11.942235    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kube-system/konnectivity-agent-fwwc5"
Sep 17 23:11:11.955350 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:11:11.955305    2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-132-188.ec2.internal" podStartSLOduration=21.955287929 podStartE2EDuration="21.955287929s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-17 23:10:50 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-17 23:11:11.954689834 +0000 UTC m=+24.356586598" watchObservedRunningTime="2026-09-17 23:11:11.955287929 +0000 UTC m=+24.357184695"
Sep 17 23:11:12.086774 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:11:12.086643    2562 plugin_watcher.go:194] "Adding socket path or updating timestamp to desired state cache" path="/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock"
Sep 17 23:11:12.625468 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:11:12.625334    2562 reconciler.go:161] "OperationExecutor.RegisterPlugin started" plugin={"SocketPath":"/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock","Timestamp":"2026-09-17T23:11:12.086671972Z","UUID":"d1cd3a18-7dd3-416e-9106-278b95d6c99b","Handler":null,"Name":"","Endpoint":""}
Sep 17 23:11:12.627380 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:11:12.627357    2562 csi_plugin.go:106] kubernetes.io/csi: Trying to validate a new CSI Driver with name: ebs.csi.aws.com endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock versions: 1.0.0
Sep 17 23:11:12.627510 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:11:12.627392    2562 csi_plugin.go:119] kubernetes.io/csi: Register new plugin with name: ebs.csi.aws.com at endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock
Sep 17 23:11:12.660826 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:11:12.660803    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-t9gqw"
Sep 17 23:11:12.660977 ip-10-0-132-188 kubenswrapper[2562]: E0917 23:11:12.660911    2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-t9gqw" podUID="440bce33-9c27-46bd-9d1c-0eebcc14fbc3"
Sep 17 23:11:12.661266 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:11:12.661247    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-jt864"
Sep 17 23:11:12.661363 ip-10-0-132-188 kubenswrapper[2562]: E0917 23:11:12.661322    2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-jt864" podUID="79765acc-1a30-4a47-add5-aea2bf5f9efb"
Sep 17 23:11:12.945679 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:11:12.945590    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-2cj88" event={"ID":"be8438b1-0b71-4af1-9575-1f005b65a607","Type":"ContainerStarted","Data":"2885aba7a51733773a916e013a4c5a3980e86e84bed4b53070189047a6bad072"}
Sep 17 23:11:12.947518 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:11:12.947484    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-bvb88" event={"ID":"9fd2f397-d1b2-4790-914c-4e9b860b827a","Type":"ContainerStarted","Data":"d59f7e35132b8ef2e8b6650732e180194270e498e27d83e099c1052935dd27aa"}
Sep 17 23:11:12.960956 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:11:12.960899    2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-operator/iptables-alerter-bvb88" podStartSLOduration=4.710349745 podStartE2EDuration="23.960880245s" totalImagesPullingTime="19.2505305s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-17 23:10:49 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-17 23:11:12.960304568 +0000 UTC m=+25.362201332" watchObservedRunningTime="2026-09-17 23:11:12.960880245 +0000 UTC m=+25.362777010"
Sep 17 23:11:13.661255 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:11:13.661220    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-nbccd"
Sep 17 23:11:13.661496 ip-10-0-132-188 kubenswrapper[2562]: E0917 23:11:13.661369    2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-nbccd" podUID="4453ba96-a67f-46af-b61b-9cd8f39b6d36"
Sep 17 23:11:13.953105 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:11:13.953020    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-2cj88" event={"ID":"be8438b1-0b71-4af1-9575-1f005b65a607","Type":"ContainerStarted","Data":"19c850e4dbe2a46fcdb5f901f7d55296ea60b22fb9600cf32d8912d3a89c987d"}
Sep 17 23:11:13.957070 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:11:13.957039    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-6xqwg" event={"ID":"cae8ed58-1a8a-40af-a329-a3850797cc58","Type":"ContainerStarted","Data":"a5038f15208d54f4c6d8331be42086405ca2ef6672cd3e3e0d62e67aa76c2c2f"}
Sep 17 23:11:13.969956 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:11:13.969902    2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-2cj88" podStartSLOduration=3.571017949 podStartE2EDuration="24.969886708s" totalImagesPullingTime="21.398868759s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-17 23:10:49 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-17 23:11:13.969193416 +0000 UTC m=+26.371090181" watchObservedRunningTime="2026-09-17 23:11:13.969886708 +0000 UTC m=+26.371783472"
Sep 17 23:11:14.661564 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:11:14.661527    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-t9gqw"
Sep 17 23:11:14.661757 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:11:14.661527    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-jt864"
Sep 17 23:11:14.661757 ip-10-0-132-188 kubenswrapper[2562]: E0917 23:11:14.661663    2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-t9gqw" podUID="440bce33-9c27-46bd-9d1c-0eebcc14fbc3"
Sep 17 23:11:14.661757 ip-10-0-132-188 kubenswrapper[2562]: E0917 23:11:14.661744    2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-jt864" podUID="79765acc-1a30-4a47-add5-aea2bf5f9efb"
Sep 17 23:11:15.661421 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:11:15.661237    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-nbccd"
Sep 17 23:11:15.662044 ip-10-0-132-188 kubenswrapper[2562]: E0917 23:11:15.661536    2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-nbccd" podUID="4453ba96-a67f-46af-b61b-9cd8f39b6d36"
Sep 17 23:11:15.889944 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:11:15.889868    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fwwc5_7e452bbb-7dc6-4c9d-a9b1-76400942f5b4/konnectivity-agent/0.log" limit=1024
Sep 17 23:11:15.962262 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:11:15.962229    2562 generic.go:356] "Generic (PLEG): container finished" podID="215c7cbb-4cd1-47be-9a44-26c1801b3fcb" containerID="e5eee316e8fd16e5888c58f618e1d70efc594a17c25d67bf2b4f31402286d0bf" exitCode=0
Sep 17 23:11:15.962429 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:11:15.962315    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-srxwj" event={"ID":"215c7cbb-4cd1-47be-9a44-26c1801b3fcb","Type":"ContainerDied","Data":"e5eee316e8fd16e5888c58f618e1d70efc594a17c25d67bf2b4f31402286d0bf"}
Sep 17 23:11:15.966237 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:11:15.966212    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-6xqwg" event={"ID":"cae8ed58-1a8a-40af-a329-a3850797cc58","Type":"ContainerStarted","Data":"d5187b03fc34c7e980abfc4bb31545b907af675fa7721a4e63339f03ec9891e2"}
Sep 17 23:11:15.966543 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:11:15.966527    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-6xqwg"
Sep 17 23:11:15.972037 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:11:15.972016    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fwwc5_7e452bbb-7dc6-4c9d-a9b1-76400942f5b4/konnectivity-agent/0.log" limit=1024
Sep 17 23:11:15.981721 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:11:15.981699    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-6xqwg"
Sep 17 23:11:16.007263 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:11:16.007213    2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ovn-kubernetes/ovnkube-node-6xqwg" podStartSLOduration=7.667376467 podStartE2EDuration="27.007197994s" totalImagesPullingTime="19.339821527s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-17 23:10:49 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-17 23:11:16.005845045 +0000 UTC m=+28.407741810" watchObservedRunningTime="2026-09-17 23:11:16.007197994 +0000 UTC m=+28.409094751"
Sep 17 23:11:16.356621 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:11:16.356534    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fwwc5_7e452bbb-7dc6-4c9d-a9b1-76400942f5b4/konnectivity-agent/0.log" limit=1024
Sep 17 23:11:16.456567 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:11:16.456530    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fwwc5_7e452bbb-7dc6-4c9d-a9b1-76400942f5b4/konnectivity-agent/0.log" limit=1024
Sep 17 23:11:16.660899 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:11:16.660871    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-t9gqw"
Sep 17 23:11:16.661074 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:11:16.660871    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-jt864"
Sep 17 23:11:16.661074 ip-10-0-132-188 kubenswrapper[2562]: E0917 23:11:16.660959    2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-t9gqw" podUID="440bce33-9c27-46bd-9d1c-0eebcc14fbc3"
Sep 17 23:11:16.661074 ip-10-0-132-188 kubenswrapper[2562]: E0917 23:11:16.661054    2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-jt864" podUID="79765acc-1a30-4a47-add5-aea2bf5f9efb"
Sep 17 23:11:16.872899 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:11:16.872816    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-6xqwg"
Sep 17 23:11:16.970414 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:11:16.970383    2562 generic.go:356] "Generic (PLEG): container finished" podID="215c7cbb-4cd1-47be-9a44-26c1801b3fcb" containerID="72a87d4bc2d948555aacd9f38c2a6964f0d3c990ca0954855b0069d2412f30f2" exitCode=0
Sep 17 23:11:16.970582 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:11:16.970464    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-srxwj" event={"ID":"215c7cbb-4cd1-47be-9a44-26c1801b3fcb","Type":"ContainerDied","Data":"72a87d4bc2d948555aacd9f38c2a6964f0d3c990ca0954855b0069d2412f30f2"}
Sep 17 23:11:16.970931 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:11:16.970908    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-6xqwg"
Sep 17 23:11:16.987055 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:11:16.987030    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-6xqwg"
Sep 17 23:11:17.375497 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:11:17.375265    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-t9gqw"]
Sep 17 23:11:17.375702 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:11:17.375631    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-t9gqw"
Sep 17 23:11:17.375756 ip-10-0-132-188 kubenswrapper[2562]: E0917 23:11:17.375739    2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-t9gqw" podUID="440bce33-9c27-46bd-9d1c-0eebcc14fbc3"
Sep 17 23:11:17.379012 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:11:17.378519    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-jt864"]
Sep 17 23:11:17.379012 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:11:17.378647    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-jt864"
Sep 17 23:11:17.379012 ip-10-0-132-188 kubenswrapper[2562]: E0917 23:11:17.378771    2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-jt864" podUID="79765acc-1a30-4a47-add5-aea2bf5f9efb"
Sep 17 23:11:17.379280 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:11:17.379241    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-nbccd"]
Sep 17 23:11:17.379383 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:11:17.379368    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-nbccd"
Sep 17 23:11:17.379763 ip-10-0-132-188 kubenswrapper[2562]: E0917 23:11:17.379488    2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-nbccd" podUID="4453ba96-a67f-46af-b61b-9cd8f39b6d36"
Sep 17 23:11:17.975473 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:11:17.975441    2562 generic.go:356] "Generic (PLEG): container finished" podID="215c7cbb-4cd1-47be-9a44-26c1801b3fcb" containerID="08ff1d7c78f75f35c4cf20211de2ee13be82066046e8dbd5eeb1f33a158d6b16" exitCode=0
Sep 17 23:11:17.975881 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:11:17.975533    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-srxwj" event={"ID":"215c7cbb-4cd1-47be-9a44-26c1801b3fcb","Type":"ContainerDied","Data":"08ff1d7c78f75f35c4cf20211de2ee13be82066046e8dbd5eeb1f33a158d6b16"}
Sep 17 23:11:18.661257 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:11:18.661224    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-t9gqw"
Sep 17 23:11:18.661480 ip-10-0-132-188 kubenswrapper[2562]: E0917 23:11:18.661353    2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-t9gqw" podUID="440bce33-9c27-46bd-9d1c-0eebcc14fbc3"
Sep 17 23:11:18.747066 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:11:18.747029    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fwwc5_7e452bbb-7dc6-4c9d-a9b1-76400942f5b4/konnectivity-agent/0.log" limit=1024
Sep 17 23:11:18.847066 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:11:18.847030    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fwwc5_7e452bbb-7dc6-4c9d-a9b1-76400942f5b4/konnectivity-agent/0.log" limit=1024
Sep 17 23:11:19.265591 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:11:19.265555    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fwwc5_7e452bbb-7dc6-4c9d-a9b1-76400942f5b4/konnectivity-agent/0.log" limit=1024
Sep 17 23:11:19.319891 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:11:19.319860    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fwwc5_7e452bbb-7dc6-4c9d-a9b1-76400942f5b4/konnectivity-agent/0.log" limit=1024
Sep 17 23:11:19.662060 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:11:19.662025    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-jt864"
Sep 17 23:11:19.662345 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:11:19.662102    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-nbccd"
Sep 17 23:11:19.662485 ip-10-0-132-188 kubenswrapper[2562]: E0917 23:11:19.662377    2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-jt864" podUID="79765acc-1a30-4a47-add5-aea2bf5f9efb"
Sep 17 23:11:19.662485 ip-10-0-132-188 kubenswrapper[2562]: E0917 23:11:19.662432    2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-nbccd" podUID="4453ba96-a67f-46af-b61b-9cd8f39b6d36"
Sep 17 23:11:20.661437 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:11:20.661400    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-t9gqw"
Sep 17 23:11:20.661913 ip-10-0-132-188 kubenswrapper[2562]: E0917 23:11:20.661511    2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-t9gqw" podUID="440bce33-9c27-46bd-9d1c-0eebcc14fbc3"
Sep 17 23:11:21.660999 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:11:21.660967    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-jt864"
Sep 17 23:11:21.660999 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:11:21.660986    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-nbccd"
Sep 17 23:11:21.661214 ip-10-0-132-188 kubenswrapper[2562]: E0917 23:11:21.661108    2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-jt864" podUID="79765acc-1a30-4a47-add5-aea2bf5f9efb"
Sep 17 23:11:21.661260 ip-10-0-132-188 kubenswrapper[2562]: E0917 23:11:21.661229    2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-nbccd" podUID="4453ba96-a67f-46af-b61b-9cd8f39b6d36"
Sep 17 23:11:22.298406 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:11:22.298316    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/4453ba96-a67f-46af-b61b-9cd8f39b6d36-metrics-certs\") pod \"network-metrics-daemon-nbccd\" (UID: \"4453ba96-a67f-46af-b61b-9cd8f39b6d36\") " pod="openshift-multus/network-metrics-daemon-nbccd"
Sep 17 23:11:22.298406 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:11:22.298357    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/79765acc-1a30-4a47-add5-aea2bf5f9efb-original-pull-secret\") pod \"global-pull-secret-syncer-jt864\" (UID: \"79765acc-1a30-4a47-add5-aea2bf5f9efb\") " pod="kube-system/global-pull-secret-syncer-jt864"
Sep 17 23:11:22.298827 ip-10-0-132-188 kubenswrapper[2562]: E0917 23:11:22.298448    2562 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Sep 17 23:11:22.298827 ip-10-0-132-188 kubenswrapper[2562]: E0917 23:11:22.298451    2562 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 17 23:11:22.298827 ip-10-0-132-188 kubenswrapper[2562]: E0917 23:11:22.298503    2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/79765acc-1a30-4a47-add5-aea2bf5f9efb-original-pull-secret podName:79765acc-1a30-4a47-add5-aea2bf5f9efb nodeName:}" failed. No retries permitted until 2026-09-17 23:11:54.298489996 +0000 UTC m=+66.700386738 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/79765acc-1a30-4a47-add5-aea2bf5f9efb-original-pull-secret") pod "global-pull-secret-syncer-jt864" (UID: "79765acc-1a30-4a47-add5-aea2bf5f9efb") : object "kube-system"/"original-pull-secret" not registered
Sep 17 23:11:22.298827 ip-10-0-132-188 kubenswrapper[2562]: E0917 23:11:22.298518    2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/4453ba96-a67f-46af-b61b-9cd8f39b6d36-metrics-certs podName:4453ba96-a67f-46af-b61b-9cd8f39b6d36 nodeName:}" failed. No retries permitted until 2026-09-17 23:11:54.29851134 +0000 UTC m=+66.700408082 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/4453ba96-a67f-46af-b61b-9cd8f39b6d36-metrics-certs") pod "network-metrics-daemon-nbccd" (UID: "4453ba96-a67f-46af-b61b-9cd8f39b6d36") : object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 17 23:11:22.399036 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:11:22.398993    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-42cvm\" (UniqueName: \"kubernetes.io/projected/440bce33-9c27-46bd-9d1c-0eebcc14fbc3-kube-api-access-42cvm\") pod \"network-check-target-t9gqw\" (UID: \"440bce33-9c27-46bd-9d1c-0eebcc14fbc3\") " pod="openshift-network-diagnostics/network-check-target-t9gqw"
Sep 17 23:11:22.399239 ip-10-0-132-188 kubenswrapper[2562]: E0917 23:11:22.399157    2562 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Sep 17 23:11:22.399239 ip-10-0-132-188 kubenswrapper[2562]: E0917 23:11:22.399198    2562 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Sep 17 23:11:22.399239 ip-10-0-132-188 kubenswrapper[2562]: E0917 23:11:22.399207    2562 projected.go:196] Error preparing data for projected volume kube-api-access-42cvm for pod openshift-network-diagnostics/network-check-target-t9gqw: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 17 23:11:22.399364 ip-10-0-132-188 kubenswrapper[2562]: E0917 23:11:22.399257    2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/440bce33-9c27-46bd-9d1c-0eebcc14fbc3-kube-api-access-42cvm podName:440bce33-9c27-46bd-9d1c-0eebcc14fbc3 nodeName:}" failed. No retries permitted until 2026-09-17 23:11:54.399244223 +0000 UTC m=+66.801140983 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "kube-api-access-42cvm" (UniqueName: "kubernetes.io/projected/440bce33-9c27-46bd-9d1c-0eebcc14fbc3-kube-api-access-42cvm") pod "network-check-target-t9gqw" (UID: "440bce33-9c27-46bd-9d1c-0eebcc14fbc3") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 17 23:11:22.660626 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:11:22.660597    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-t9gqw"
Sep 17 23:11:22.660796 ip-10-0-132-188 kubenswrapper[2562]: E0917 23:11:22.660708    2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-t9gqw" podUID="440bce33-9c27-46bd-9d1c-0eebcc14fbc3"
Sep 17 23:11:22.989092 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:11:22.988796    2562 generic.go:356] "Generic (PLEG): container finished" podID="215c7cbb-4cd1-47be-9a44-26c1801b3fcb" containerID="403b9efa6cea8a2f78de3aee8f1ac2de26f30573e7f2c8ad598e6182041c8550" exitCode=0
Sep 17 23:11:22.989092 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:11:22.988879    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-srxwj" event={"ID":"215c7cbb-4cd1-47be-9a44-26c1801b3fcb","Type":"ContainerDied","Data":"403b9efa6cea8a2f78de3aee8f1ac2de26f30573e7f2c8ad598e6182041c8550"}
Sep 17 23:11:23.070841 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:11:23.070820    2562 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-132-188.ec2.internal" event="NodeReady"
Sep 17 23:11:23.070970 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:11:23.070956    2562 kubelet_node_status.go:489] "Fast updating node status as it just became ready"
Sep 17 23:11:23.113753 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:11:23.113726    2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-dns/dns-default-9z4vw"]
Sep 17 23:11:23.128649 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:11:23.128626    2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress-canary/ingress-canary-hqrgl"]
Sep 17 23:11:23.128759 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:11:23.128741    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-9z4vw"
Sep 17 23:11:23.131135 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:11:23.131110    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-default-metrics-tls\""
Sep 17 23:11:23.131278 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:11:23.131263    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"dns-default\""
Sep 17 23:11:23.131732 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:11:23.131713    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-cvr9f\""
Sep 17 23:11:23.146900 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:11:23.146875    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-9z4vw"]
Sep 17 23:11:23.147000 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:11:23.146921    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-hqrgl"]
Sep 17 23:11:23.147000 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:11:23.146987    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-hqrgl"
Sep 17 23:11:23.149239 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:11:23.149215    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"default-dockercfg-fsw47\""
Sep 17 23:11:23.149239 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:11:23.149231    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"canary-serving-cert\""
Sep 17 23:11:23.149379 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:11:23.149242    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"openshift-service-ca.crt\""
Sep 17 23:11:23.149504 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:11:23.149491    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"kube-root-ca.crt\""
Sep 17 23:11:23.205004 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:11:23.204972    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/57e9d0f5-d16a-42d0-ab45-c06fe21e7bbd-metrics-tls\") pod \"dns-default-9z4vw\" (UID: \"57e9d0f5-d16a-42d0-ab45-c06fe21e7bbd\") " pod="openshift-dns/dns-default-9z4vw"
Sep 17 23:11:23.205144 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:11:23.205009    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ghwkv\" (UniqueName: \"kubernetes.io/projected/57e9d0f5-d16a-42d0-ab45-c06fe21e7bbd-kube-api-access-ghwkv\") pod \"dns-default-9z4vw\" (UID: \"57e9d0f5-d16a-42d0-ab45-c06fe21e7bbd\") " pod="openshift-dns/dns-default-9z4vw"
Sep 17 23:11:23.205144 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:11:23.205091    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/57e9d0f5-d16a-42d0-ab45-c06fe21e7bbd-config-volume\") pod \"dns-default-9z4vw\" (UID: \"57e9d0f5-d16a-42d0-ab45-c06fe21e7bbd\") " pod="openshift-dns/dns-default-9z4vw"
Sep 17 23:11:23.205262 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:11:23.205190    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/57e9d0f5-d16a-42d0-ab45-c06fe21e7bbd-tmp-dir\") pod \"dns-default-9z4vw\" (UID: \"57e9d0f5-d16a-42d0-ab45-c06fe21e7bbd\") " pod="openshift-dns/dns-default-9z4vw"
Sep 17 23:11:23.306431 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:11:23.306354    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-m6wrd\" (UniqueName: \"kubernetes.io/projected/5562eaa3-4a89-4ed2-b42b-831c3da5bba3-kube-api-access-m6wrd\") pod \"ingress-canary-hqrgl\" (UID: \"5562eaa3-4a89-4ed2-b42b-831c3da5bba3\") " pod="openshift-ingress-canary/ingress-canary-hqrgl"
Sep 17 23:11:23.306431 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:11:23.306401    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/57e9d0f5-d16a-42d0-ab45-c06fe21e7bbd-config-volume\") pod \"dns-default-9z4vw\" (UID: \"57e9d0f5-d16a-42d0-ab45-c06fe21e7bbd\") " pod="openshift-dns/dns-default-9z4vw"
Sep 17 23:11:23.306887 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:11:23.306448    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/57e9d0f5-d16a-42d0-ab45-c06fe21e7bbd-tmp-dir\") pod \"dns-default-9z4vw\" (UID: \"57e9d0f5-d16a-42d0-ab45-c06fe21e7bbd\") " pod="openshift-dns/dns-default-9z4vw"
Sep 17 23:11:23.306887 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:11:23.306464    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/5562eaa3-4a89-4ed2-b42b-831c3da5bba3-cert\") pod \"ingress-canary-hqrgl\" (UID: \"5562eaa3-4a89-4ed2-b42b-831c3da5bba3\") " pod="openshift-ingress-canary/ingress-canary-hqrgl"
Sep 17 23:11:23.306887 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:11:23.306491    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/57e9d0f5-d16a-42d0-ab45-c06fe21e7bbd-metrics-tls\") pod \"dns-default-9z4vw\" (UID: \"57e9d0f5-d16a-42d0-ab45-c06fe21e7bbd\") " pod="openshift-dns/dns-default-9z4vw"
Sep 17 23:11:23.306887 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:11:23.306513    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ghwkv\" (UniqueName: \"kubernetes.io/projected/57e9d0f5-d16a-42d0-ab45-c06fe21e7bbd-kube-api-access-ghwkv\") pod \"dns-default-9z4vw\" (UID: \"57e9d0f5-d16a-42d0-ab45-c06fe21e7bbd\") " pod="openshift-dns/dns-default-9z4vw"
Sep 17 23:11:23.306887 ip-10-0-132-188 kubenswrapper[2562]: E0917 23:11:23.306609    2562 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Sep 17 23:11:23.306887 ip-10-0-132-188 kubenswrapper[2562]: E0917 23:11:23.306685    2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/57e9d0f5-d16a-42d0-ab45-c06fe21e7bbd-metrics-tls podName:57e9d0f5-d16a-42d0-ab45-c06fe21e7bbd nodeName:}" failed. No retries permitted until 2026-09-17 23:11:23.806666994 +0000 UTC m=+36.208563752 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/57e9d0f5-d16a-42d0-ab45-c06fe21e7bbd-metrics-tls") pod "dns-default-9z4vw" (UID: "57e9d0f5-d16a-42d0-ab45-c06fe21e7bbd") : secret "dns-default-metrics-tls" not found
Sep 17 23:11:23.306887 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:11:23.306840    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/57e9d0f5-d16a-42d0-ab45-c06fe21e7bbd-tmp-dir\") pod \"dns-default-9z4vw\" (UID: \"57e9d0f5-d16a-42d0-ab45-c06fe21e7bbd\") " pod="openshift-dns/dns-default-9z4vw"
Sep 17 23:11:23.307242 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:11:23.307140    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/57e9d0f5-d16a-42d0-ab45-c06fe21e7bbd-config-volume\") pod \"dns-default-9z4vw\" (UID: \"57e9d0f5-d16a-42d0-ab45-c06fe21e7bbd\") " pod="openshift-dns/dns-default-9z4vw"
Sep 17 23:11:23.316907 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:11:23.316881    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-ghwkv\" (UniqueName: \"kubernetes.io/projected/57e9d0f5-d16a-42d0-ab45-c06fe21e7bbd-kube-api-access-ghwkv\") pod \"dns-default-9z4vw\" (UID: \"57e9d0f5-d16a-42d0-ab45-c06fe21e7bbd\") " pod="openshift-dns/dns-default-9z4vw"
Sep 17 23:11:23.407569 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:11:23.407533    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-m6wrd\" (UniqueName: \"kubernetes.io/projected/5562eaa3-4a89-4ed2-b42b-831c3da5bba3-kube-api-access-m6wrd\") pod \"ingress-canary-hqrgl\" (UID: \"5562eaa3-4a89-4ed2-b42b-831c3da5bba3\") " pod="openshift-ingress-canary/ingress-canary-hqrgl"
Sep 17 23:11:23.407708 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:11:23.407590    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/5562eaa3-4a89-4ed2-b42b-831c3da5bba3-cert\") pod \"ingress-canary-hqrgl\" (UID: \"5562eaa3-4a89-4ed2-b42b-831c3da5bba3\") " pod="openshift-ingress-canary/ingress-canary-hqrgl"
Sep 17 23:11:23.410034 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:11:23.410000    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/5562eaa3-4a89-4ed2-b42b-831c3da5bba3-cert\") pod \"ingress-canary-hqrgl\" (UID: \"5562eaa3-4a89-4ed2-b42b-831c3da5bba3\") " pod="openshift-ingress-canary/ingress-canary-hqrgl"
Sep 17 23:11:23.416038 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:11:23.416014    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-m6wrd\" (UniqueName: \"kubernetes.io/projected/5562eaa3-4a89-4ed2-b42b-831c3da5bba3-kube-api-access-m6wrd\") pod \"ingress-canary-hqrgl\" (UID: \"5562eaa3-4a89-4ed2-b42b-831c3da5bba3\") " pod="openshift-ingress-canary/ingress-canary-hqrgl"
Sep 17 23:11:23.504677 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:11:23.504641    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-hqrgl"
Sep 17 23:11:23.660888 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:11:23.660858    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-jt864"
Sep 17 23:11:23.661048 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:11:23.660858    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-nbccd"
Sep 17 23:11:23.663918 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:11:23.663895    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\""
Sep 17 23:11:23.664053 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:11:23.663958    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-mx9lw\""
Sep 17 23:11:23.664053 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:11:23.664011    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\""
Sep 17 23:11:23.810351 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:11:23.810313    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/57e9d0f5-d16a-42d0-ab45-c06fe21e7bbd-metrics-tls\") pod \"dns-default-9z4vw\" (UID: \"57e9d0f5-d16a-42d0-ab45-c06fe21e7bbd\") " pod="openshift-dns/dns-default-9z4vw"
Sep 17 23:11:23.810521 ip-10-0-132-188 kubenswrapper[2562]: E0917 23:11:23.810458    2562 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Sep 17 23:11:23.810521 ip-10-0-132-188 kubenswrapper[2562]: E0917 23:11:23.810518    2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/57e9d0f5-d16a-42d0-ab45-c06fe21e7bbd-metrics-tls podName:57e9d0f5-d16a-42d0-ab45-c06fe21e7bbd nodeName:}" failed. No retries permitted until 2026-09-17 23:11:24.810504179 +0000 UTC m=+37.212400920 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/57e9d0f5-d16a-42d0-ab45-c06fe21e7bbd-metrics-tls") pod "dns-default-9z4vw" (UID: "57e9d0f5-d16a-42d0-ab45-c06fe21e7bbd") : secret "dns-default-metrics-tls" not found
Sep 17 23:11:23.835834 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:11:23.835805    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-hqrgl"]
Sep 17 23:11:23.840147 ip-10-0-132-188 kubenswrapper[2562]: W0917 23:11:23.840112    2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5562eaa3_4a89_4ed2_b42b_831c3da5bba3.slice/crio-c5d14126b23c79cf6de792125037d18e3816684761f1aec0ad3a655be8af0a4f WatchSource:0}: Error finding container c5d14126b23c79cf6de792125037d18e3816684761f1aec0ad3a655be8af0a4f: Status 404 returned error can't find the container with id c5d14126b23c79cf6de792125037d18e3816684761f1aec0ad3a655be8af0a4f
Sep 17 23:11:23.993064 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:11:23.992981    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-hqrgl" event={"ID":"5562eaa3-4a89-4ed2-b42b-831c3da5bba3","Type":"ContainerStarted","Data":"c5d14126b23c79cf6de792125037d18e3816684761f1aec0ad3a655be8af0a4f"}
Sep 17 23:11:23.996058 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:11:23.996032    2562 generic.go:356] "Generic (PLEG): container finished" podID="215c7cbb-4cd1-47be-9a44-26c1801b3fcb" containerID="6c390d65ed4a212b3d1348ecdd015f7d4ecda19c59ec9bf72ac6b0c5d527b737" exitCode=0
Sep 17 23:11:23.996199 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:11:23.996088    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-srxwj" event={"ID":"215c7cbb-4cd1-47be-9a44-26c1801b3fcb","Type":"ContainerDied","Data":"6c390d65ed4a212b3d1348ecdd015f7d4ecda19c59ec9bf72ac6b0c5d527b737"}
Sep 17 23:11:24.661147 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:11:24.660954    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-t9gqw"
Sep 17 23:11:24.663968 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:11:24.663906    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\""
Sep 17 23:11:24.665020 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:11:24.664994    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-qw6rm\""
Sep 17 23:11:24.665020 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:11:24.664996    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\""
Sep 17 23:11:24.819520 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:11:24.819486    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/57e9d0f5-d16a-42d0-ab45-c06fe21e7bbd-metrics-tls\") pod \"dns-default-9z4vw\" (UID: \"57e9d0f5-d16a-42d0-ab45-c06fe21e7bbd\") " pod="openshift-dns/dns-default-9z4vw"
Sep 17 23:11:24.819717 ip-10-0-132-188 kubenswrapper[2562]: E0917 23:11:24.819672    2562 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Sep 17 23:11:24.819783 ip-10-0-132-188 kubenswrapper[2562]: E0917 23:11:24.819756    2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/57e9d0f5-d16a-42d0-ab45-c06fe21e7bbd-metrics-tls podName:57e9d0f5-d16a-42d0-ab45-c06fe21e7bbd nodeName:}" failed. No retries permitted until 2026-09-17 23:11:26.819733807 +0000 UTC m=+39.221630549 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/57e9d0f5-d16a-42d0-ab45-c06fe21e7bbd-metrics-tls") pod "dns-default-9z4vw" (UID: "57e9d0f5-d16a-42d0-ab45-c06fe21e7bbd") : secret "dns-default-metrics-tls" not found
Sep 17 23:11:25.002842 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:11:25.002747    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-srxwj" event={"ID":"215c7cbb-4cd1-47be-9a44-26c1801b3fcb","Type":"ContainerStarted","Data":"59d0caad46bf41524a0edc450493dcb50951e79513a655b075eeda45df89500e"}
Sep 17 23:11:25.022547 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:11:25.022489    2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-additional-cni-plugins-srxwj" podStartSLOduration=6.994197054 podStartE2EDuration="36.022466579s" totalImagesPullingTime="29.028269525s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-17 23:10:49 +0000 UTC" imagePullSessionsCount=5 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-17 23:11:25.021413489 +0000 UTC m=+37.423310252" watchObservedRunningTime="2026-09-17 23:11:25.022466579 +0000 UTC m=+37.424363346"
Sep 17 23:11:25.142905 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:11:25.142869    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fwwc5_7e452bbb-7dc6-4c9d-a9b1-76400942f5b4/konnectivity-agent/0.log" limit=1024
Sep 17 23:11:25.242547 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:11:25.242511    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fwwc5_7e452bbb-7dc6-4c9d-a9b1-76400942f5b4/konnectivity-agent/0.log" limit=1024
Sep 17 23:11:26.834934 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:11:26.834890    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/57e9d0f5-d16a-42d0-ab45-c06fe21e7bbd-metrics-tls\") pod \"dns-default-9z4vw\" (UID: \"57e9d0f5-d16a-42d0-ab45-c06fe21e7bbd\") " pod="openshift-dns/dns-default-9z4vw"
Sep 17 23:11:26.835386 ip-10-0-132-188 kubenswrapper[2562]: E0917 23:11:26.835049    2562 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Sep 17 23:11:26.835386 ip-10-0-132-188 kubenswrapper[2562]: E0917 23:11:26.835125    2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/57e9d0f5-d16a-42d0-ab45-c06fe21e7bbd-metrics-tls podName:57e9d0f5-d16a-42d0-ab45-c06fe21e7bbd nodeName:}" failed. No retries permitted until 2026-09-17 23:11:30.835103849 +0000 UTC m=+43.237000611 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/57e9d0f5-d16a-42d0-ab45-c06fe21e7bbd-metrics-tls") pod "dns-default-9z4vw" (UID: "57e9d0f5-d16a-42d0-ab45-c06fe21e7bbd") : secret "dns-default-metrics-tls" not found
Sep 17 23:11:27.009369 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:11:27.009339    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-hqrgl" event={"ID":"5562eaa3-4a89-4ed2-b42b-831c3da5bba3","Type":"ContainerStarted","Data":"c4501310c695641e35a49661a46c0d445d1d8583051342ade0f0195797e36b80"}
Sep 17 23:11:27.024516 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:11:27.024462    2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress-canary/ingress-canary-hqrgl" podStartSLOduration=1.615954613 podStartE2EDuration="4.024446129s" totalImagesPullingTime="2.408491516s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-17 23:11:23 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-17 23:11:27.023417084 +0000 UTC m=+39.425313848" watchObservedRunningTime="2026-09-17 23:11:27.024446129 +0000 UTC m=+39.426342892"
Sep 17 23:11:28.333188 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:11:28.333142    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fwwc5_7e452bbb-7dc6-4c9d-a9b1-76400942f5b4/konnectivity-agent/0.log" limit=1024
Sep 17 23:11:29.977591 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:11:29.977558    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fwwc5_7e452bbb-7dc6-4c9d-a9b1-76400942f5b4/konnectivity-agent/0.log" limit=1024
Sep 17 23:11:30.078536 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:11:30.078503    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fwwc5_7e452bbb-7dc6-4c9d-a9b1-76400942f5b4/konnectivity-agent/0.log" limit=1024
Sep 17 23:11:30.863566 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:11:30.863532    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/57e9d0f5-d16a-42d0-ab45-c06fe21e7bbd-metrics-tls\") pod \"dns-default-9z4vw\" (UID: \"57e9d0f5-d16a-42d0-ab45-c06fe21e7bbd\") " pod="openshift-dns/dns-default-9z4vw"
Sep 17 23:11:30.863737 ip-10-0-132-188 kubenswrapper[2562]: E0917 23:11:30.863659    2562 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Sep 17 23:11:30.863737 ip-10-0-132-188 kubenswrapper[2562]: E0917 23:11:30.863714    2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/57e9d0f5-d16a-42d0-ab45-c06fe21e7bbd-metrics-tls podName:57e9d0f5-d16a-42d0-ab45-c06fe21e7bbd nodeName:}" failed. No retries permitted until 2026-09-17 23:11:38.863697581 +0000 UTC m=+51.265594323 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/57e9d0f5-d16a-42d0-ab45-c06fe21e7bbd-metrics-tls") pod "dns-default-9z4vw" (UID: "57e9d0f5-d16a-42d0-ab45-c06fe21e7bbd") : secret "dns-default-metrics-tls" not found
Sep 17 23:11:38.858475 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:11:38.858438    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fwwc5_7e452bbb-7dc6-4c9d-a9b1-76400942f5b4/konnectivity-agent/0.log" limit=1024
Sep 17 23:11:38.908435 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:11:38.908408    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fwwc5_7e452bbb-7dc6-4c9d-a9b1-76400942f5b4/konnectivity-agent/0.log" limit=1024
Sep 17 23:11:38.919460 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:11:38.919432    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/57e9d0f5-d16a-42d0-ab45-c06fe21e7bbd-metrics-tls\") pod \"dns-default-9z4vw\" (UID: \"57e9d0f5-d16a-42d0-ab45-c06fe21e7bbd\") " pod="openshift-dns/dns-default-9z4vw"
Sep 17 23:11:38.919599 ip-10-0-132-188 kubenswrapper[2562]: E0917 23:11:38.919581    2562 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Sep 17 23:11:38.919664 ip-10-0-132-188 kubenswrapper[2562]: E0917 23:11:38.919654    2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/57e9d0f5-d16a-42d0-ab45-c06fe21e7bbd-metrics-tls podName:57e9d0f5-d16a-42d0-ab45-c06fe21e7bbd nodeName:}" failed. No retries permitted until 2026-09-17 23:11:54.919637317 +0000 UTC m=+67.321534062 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/57e9d0f5-d16a-42d0-ab45-c06fe21e7bbd-metrics-tls") pod "dns-default-9z4vw" (UID: "57e9d0f5-d16a-42d0-ab45-c06fe21e7bbd") : secret "dns-default-metrics-tls" not found
Sep 17 23:11:39.343076 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:11:39.343043    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fwwc5_7e452bbb-7dc6-4c9d-a9b1-76400942f5b4/konnectivity-agent/0.log" limit=1024
Sep 17 23:11:39.395864 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:11:39.395836    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fwwc5_7e452bbb-7dc6-4c9d-a9b1-76400942f5b4/konnectivity-agent/0.log" limit=1024
Sep 17 23:11:48.988399 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:11:48.988370    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-6xqwg"
Sep 17 23:11:49.104817 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:11:49.104781    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fwwc5_7e452bbb-7dc6-4c9d-a9b1-76400942f5b4/konnectivity-agent/0.log" limit=1024
Sep 17 23:11:49.160717 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:11:49.160679    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fwwc5_7e452bbb-7dc6-4c9d-a9b1-76400942f5b4/konnectivity-agent/0.log" limit=1024
Sep 17 23:11:49.219711 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:11:49.219677    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fwwc5_7e452bbb-7dc6-4c9d-a9b1-76400942f5b4/konnectivity-agent/0.log" limit=1024
Sep 17 23:11:53.896161 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:11:53.896117    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fwwc5_7e452bbb-7dc6-4c9d-a9b1-76400942f5b4/konnectivity-agent/0.log" limit=1024
Sep 17 23:11:53.950397 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:11:53.950368    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fwwc5_7e452bbb-7dc6-4c9d-a9b1-76400942f5b4/konnectivity-agent/0.log" limit=1024
Sep 17 23:11:54.317727 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:11:54.317637    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/4453ba96-a67f-46af-b61b-9cd8f39b6d36-metrics-certs\") pod \"network-metrics-daemon-nbccd\" (UID: \"4453ba96-a67f-46af-b61b-9cd8f39b6d36\") " pod="openshift-multus/network-metrics-daemon-nbccd"
Sep 17 23:11:54.317727 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:11:54.317682    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/79765acc-1a30-4a47-add5-aea2bf5f9efb-original-pull-secret\") pod \"global-pull-secret-syncer-jt864\" (UID: \"79765acc-1a30-4a47-add5-aea2bf5f9efb\") " pod="kube-system/global-pull-secret-syncer-jt864"
Sep 17 23:11:54.320416 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:11:54.320388    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\""
Sep 17 23:11:54.320416 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:11:54.320412    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\""
Sep 17 23:11:54.328197 ip-10-0-132-188 kubenswrapper[2562]: E0917 23:11:54.328156    2562 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: secret "metrics-daemon-secret" not found
Sep 17 23:11:54.328311 ip-10-0-132-188 kubenswrapper[2562]: E0917 23:11:54.328250    2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/4453ba96-a67f-46af-b61b-9cd8f39b6d36-metrics-certs podName:4453ba96-a67f-46af-b61b-9cd8f39b6d36 nodeName:}" failed. No retries permitted until 2026-09-17 23:12:58.328228346 +0000 UTC m=+130.730125102 (durationBeforeRetry 1m4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/4453ba96-a67f-46af-b61b-9cd8f39b6d36-metrics-certs") pod "network-metrics-daemon-nbccd" (UID: "4453ba96-a67f-46af-b61b-9cd8f39b6d36") : secret "metrics-daemon-secret" not found
Sep 17 23:11:54.331486 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:11:54.331460    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/79765acc-1a30-4a47-add5-aea2bf5f9efb-original-pull-secret\") pod \"global-pull-secret-syncer-jt864\" (UID: \"79765acc-1a30-4a47-add5-aea2bf5f9efb\") " pod="kube-system/global-pull-secret-syncer-jt864"
Sep 17 23:11:54.418848 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:11:54.418794    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-42cvm\" (UniqueName: \"kubernetes.io/projected/440bce33-9c27-46bd-9d1c-0eebcc14fbc3-kube-api-access-42cvm\") pod \"network-check-target-t9gqw\" (UID: \"440bce33-9c27-46bd-9d1c-0eebcc14fbc3\") " pod="openshift-network-diagnostics/network-check-target-t9gqw"
Sep 17 23:11:54.421678 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:11:54.421658    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\""
Sep 17 23:11:54.431532 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:11:54.431513    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\""
Sep 17 23:11:54.442567 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:11:54.442541    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-42cvm\" (UniqueName: \"kubernetes.io/projected/440bce33-9c27-46bd-9d1c-0eebcc14fbc3-kube-api-access-42cvm\") pod \"network-check-target-t9gqw\" (UID: \"440bce33-9c27-46bd-9d1c-0eebcc14fbc3\") " pod="openshift-network-diagnostics/network-check-target-t9gqw"
Sep 17 23:11:54.571786 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:11:54.571706    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-jt864"
Sep 17 23:11:54.675391 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:11:54.675356    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-qw6rm\""
Sep 17 23:11:54.683697 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:11:54.683670    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-t9gqw"
Sep 17 23:11:54.841829 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:11:54.841665    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-jt864"]
Sep 17 23:11:54.844473 ip-10-0-132-188 kubenswrapper[2562]: W0917 23:11:54.844439    2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod79765acc_1a30_4a47_add5_aea2bf5f9efb.slice/crio-f787e3545835af31d9594b79c80d1803e1c8d6bef5c2450d16fd5c58724f0dd0 WatchSource:0}: Error finding container f787e3545835af31d9594b79c80d1803e1c8d6bef5c2450d16fd5c58724f0dd0: Status 404 returned error can't find the container with id f787e3545835af31d9594b79c80d1803e1c8d6bef5c2450d16fd5c58724f0dd0
Sep 17 23:11:54.923191 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:11:54.923131    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/57e9d0f5-d16a-42d0-ab45-c06fe21e7bbd-metrics-tls\") pod \"dns-default-9z4vw\" (UID: \"57e9d0f5-d16a-42d0-ab45-c06fe21e7bbd\") " pod="openshift-dns/dns-default-9z4vw"
Sep 17 23:11:54.923546 ip-10-0-132-188 kubenswrapper[2562]: E0917 23:11:54.923293    2562 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Sep 17 23:11:54.923546 ip-10-0-132-188 kubenswrapper[2562]: E0917 23:11:54.923378    2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/57e9d0f5-d16a-42d0-ab45-c06fe21e7bbd-metrics-tls podName:57e9d0f5-d16a-42d0-ab45-c06fe21e7bbd nodeName:}" failed. No retries permitted until 2026-09-17 23:12:26.923354937 +0000 UTC m=+99.325251683 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/57e9d0f5-d16a-42d0-ab45-c06fe21e7bbd-metrics-tls") pod "dns-default-9z4vw" (UID: "57e9d0f5-d16a-42d0-ab45-c06fe21e7bbd") : secret "dns-default-metrics-tls" not found
Sep 17 23:11:54.949810 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:11:54.949782    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-t9gqw"]
Sep 17 23:11:54.952571 ip-10-0-132-188 kubenswrapper[2562]: W0917 23:11:54.952544    2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod440bce33_9c27_46bd_9d1c_0eebcc14fbc3.slice/crio-767bac932670ee026100f13dff78f958406308aee52ea036c1999c6132a1c126 WatchSource:0}: Error finding container 767bac932670ee026100f13dff78f958406308aee52ea036c1999c6132a1c126: Status 404 returned error can't find the container with id 767bac932670ee026100f13dff78f958406308aee52ea036c1999c6132a1c126
Sep 17 23:11:55.075185 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:11:55.075134    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-t9gqw" event={"ID":"440bce33-9c27-46bd-9d1c-0eebcc14fbc3","Type":"ContainerStarted","Data":"767bac932670ee026100f13dff78f958406308aee52ea036c1999c6132a1c126"}
Sep 17 23:11:55.076263 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:11:55.076237    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-jt864" event={"ID":"79765acc-1a30-4a47-add5-aea2bf5f9efb","Type":"ContainerStarted","Data":"f787e3545835af31d9594b79c80d1803e1c8d6bef5c2450d16fd5c58724f0dd0"}
Sep 17 23:12:00.090219 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:12:00.090166    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-t9gqw" event={"ID":"440bce33-9c27-46bd-9d1c-0eebcc14fbc3","Type":"ContainerStarted","Data":"01e60f32452c007eb44d63c8c714a103f09c174b8251735308316d0d3229dca6"}
Sep 17 23:12:00.090816 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:12:00.090278    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-network-diagnostics/network-check-target-t9gqw"
Sep 17 23:12:00.091683 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:12:00.091663    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-jt864" event={"ID":"79765acc-1a30-4a47-add5-aea2bf5f9efb","Type":"ContainerStarted","Data":"68800cd0fb9e5a06f1b6d3bba09f3094db827afd0a74a10bc285458392196ef8"}
Sep 17 23:12:00.104885 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:12:00.104835    2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-diagnostics/network-check-target-t9gqw" podStartSLOduration=66.939739626 podStartE2EDuration="1m11.104821564s" totalImagesPullingTime="4.165081938s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-17 23:10:49 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-17 23:12:00.103913121 +0000 UTC m=+72.505809889" watchObservedRunningTime="2026-09-17 23:12:00.104821564 +0000 UTC m=+72.506718325"
Sep 17 23:12:00.117349 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:12:00.117309    2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/global-pull-secret-syncer-jt864" podStartSLOduration=66.840094199 podStartE2EDuration="1m11.117292808s" totalImagesPullingTime="4.277198609s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-17 23:10:49 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-17 23:12:00.116965103 +0000 UTC m=+72.518861868" watchObservedRunningTime="2026-09-17 23:12:00.117292808 +0000 UTC m=+72.519189570"
Sep 17 23:12:17.902607 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:12:17.902578    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fwwc5_7e452bbb-7dc6-4c9d-a9b1-76400942f5b4/konnectivity-agent/0.log" limit=1024
Sep 17 23:12:17.950642 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:12:17.950613    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fwwc5_7e452bbb-7dc6-4c9d-a9b1-76400942f5b4/konnectivity-agent/0.log" limit=1024
Sep 17 23:12:18.547290 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:12:18.547263    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fwwc5_7e452bbb-7dc6-4c9d-a9b1-76400942f5b4/konnectivity-agent/0.log" limit=1024
Sep 17 23:12:26.944526 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:12:26.944492    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/57e9d0f5-d16a-42d0-ab45-c06fe21e7bbd-metrics-tls\") pod \"dns-default-9z4vw\" (UID: \"57e9d0f5-d16a-42d0-ab45-c06fe21e7bbd\") " pod="openshift-dns/dns-default-9z4vw"
Sep 17 23:12:26.944871 ip-10-0-132-188 kubenswrapper[2562]: E0917 23:12:26.944639    2562 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Sep 17 23:12:26.944871 ip-10-0-132-188 kubenswrapper[2562]: E0917 23:12:26.944707    2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/57e9d0f5-d16a-42d0-ab45-c06fe21e7bbd-metrics-tls podName:57e9d0f5-d16a-42d0-ab45-c06fe21e7bbd nodeName:}" failed. No retries permitted until 2026-09-17 23:13:30.944691902 +0000 UTC m=+163.346588644 (durationBeforeRetry 1m4s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/57e9d0f5-d16a-42d0-ab45-c06fe21e7bbd-metrics-tls") pod "dns-default-9z4vw" (UID: "57e9d0f5-d16a-42d0-ab45-c06fe21e7bbd") : secret "dns-default-metrics-tls" not found
Sep 17 23:12:31.096116 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:12:31.096085    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-network-diagnostics/network-check-target-t9gqw"
Sep 17 23:12:58.361803 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:12:58.361769    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/4453ba96-a67f-46af-b61b-9cd8f39b6d36-metrics-certs\") pod \"network-metrics-daemon-nbccd\" (UID: \"4453ba96-a67f-46af-b61b-9cd8f39b6d36\") " pod="openshift-multus/network-metrics-daemon-nbccd"
Sep 17 23:12:58.362316 ip-10-0-132-188 kubenswrapper[2562]: E0917 23:12:58.361902    2562 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: secret "metrics-daemon-secret" not found
Sep 17 23:12:58.362316 ip-10-0-132-188 kubenswrapper[2562]: E0917 23:12:58.361959    2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/4453ba96-a67f-46af-b61b-9cd8f39b6d36-metrics-certs podName:4453ba96-a67f-46af-b61b-9cd8f39b6d36 nodeName:}" failed. No retries permitted until 2026-09-17 23:15:00.361944157 +0000 UTC m=+252.763840899 (durationBeforeRetry 2m2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/4453ba96-a67f-46af-b61b-9cd8f39b6d36-metrics-certs") pod "network-metrics-daemon-nbccd" (UID: "4453ba96-a67f-46af-b61b-9cd8f39b6d36") : secret "metrics-daemon-secret" not found
Sep 17 23:13:22.034211 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:13:22.034164    2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["open-cluster-management-agent-addon/cluster-proxy-proxy-agent-64d77d549c-dndvm"]
Sep 17 23:13:22.036009 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:13:22.035993    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-64d77d549c-dndvm"
Sep 17 23:13:22.040531 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:13:22.040273    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"cluster-proxy-ca\""
Sep 17 23:13:22.041127 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:13:22.040983    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"cluster-proxy-open-cluster-management.io-proxy-agent-signer-client-cert\""
Sep 17 23:13:22.041127 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:13:22.041003    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"open-cluster-management-image-pull-credentials\""
Sep 17 23:13:22.041127 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:13:22.041014    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"cluster-proxy-hub-kubeconfig\""
Sep 17 23:13:22.041127 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:13:22.041029    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"open-cluster-management-agent-addon\"/\"kube-root-ca.crt\""
Sep 17 23:13:22.041127 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:13:22.041027    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"open-cluster-management-agent-addon\"/\"openshift-service-ca.crt\""
Sep 17 23:13:22.041462 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:13:22.041136    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"cluster-proxy-service-proxy-server-certificates\""
Sep 17 23:13:22.051095 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:13:22.051071    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/cluster-proxy-proxy-agent-64d77d549c-dndvm"]
Sep 17 23:13:22.220330 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:13:22.220291    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/204204da-e53d-4c6b-93ed-a6271c85ac27-hub-kubeconfig\") pod \"cluster-proxy-proxy-agent-64d77d549c-dndvm\" (UID: \"204204da-e53d-4c6b-93ed-a6271c85ac27\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-64d77d549c-dndvm"
Sep 17 23:13:22.220330 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:13:22.220332    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"additional-service-ca\" (UniqueName: \"kubernetes.io/configmap/204204da-e53d-4c6b-93ed-a6271c85ac27-additional-service-ca\") pod \"cluster-proxy-proxy-agent-64d77d549c-dndvm\" (UID: \"204204da-e53d-4c6b-93ed-a6271c85ac27\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-64d77d549c-dndvm"
Sep 17 23:13:22.220614 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:13:22.220412    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ca\" (UniqueName: \"kubernetes.io/secret/204204da-e53d-4c6b-93ed-a6271c85ac27-ca\") pod \"cluster-proxy-proxy-agent-64d77d549c-dndvm\" (UID: \"204204da-e53d-4c6b-93ed-a6271c85ac27\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-64d77d549c-dndvm"
Sep 17 23:13:22.220614 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:13:22.220451    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-proxy-server-cert\" (UniqueName: \"kubernetes.io/secret/204204da-e53d-4c6b-93ed-a6271c85ac27-service-proxy-server-cert\") pod \"cluster-proxy-proxy-agent-64d77d549c-dndvm\" (UID: \"204204da-e53d-4c6b-93ed-a6271c85ac27\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-64d77d549c-dndvm"
Sep 17 23:13:22.220614 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:13:22.220500    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-c55hj\" (UniqueName: \"kubernetes.io/projected/204204da-e53d-4c6b-93ed-a6271c85ac27-kube-api-access-c55hj\") pod \"cluster-proxy-proxy-agent-64d77d549c-dndvm\" (UID: \"204204da-e53d-4c6b-93ed-a6271c85ac27\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-64d77d549c-dndvm"
Sep 17 23:13:22.220614 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:13:22.220524    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hub\" (UniqueName: \"kubernetes.io/secret/204204da-e53d-4c6b-93ed-a6271c85ac27-hub\") pod \"cluster-proxy-proxy-agent-64d77d549c-dndvm\" (UID: \"204204da-e53d-4c6b-93ed-a6271c85ac27\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-64d77d549c-dndvm"
Sep 17 23:13:22.321489 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:13:22.321402    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/204204da-e53d-4c6b-93ed-a6271c85ac27-hub-kubeconfig\") pod \"cluster-proxy-proxy-agent-64d77d549c-dndvm\" (UID: \"204204da-e53d-4c6b-93ed-a6271c85ac27\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-64d77d549c-dndvm"
Sep 17 23:13:22.321489 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:13:22.321448    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"additional-service-ca\" (UniqueName: \"kubernetes.io/configmap/204204da-e53d-4c6b-93ed-a6271c85ac27-additional-service-ca\") pod \"cluster-proxy-proxy-agent-64d77d549c-dndvm\" (UID: \"204204da-e53d-4c6b-93ed-a6271c85ac27\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-64d77d549c-dndvm"
Sep 17 23:13:22.321489 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:13:22.321484    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ca\" (UniqueName: \"kubernetes.io/secret/204204da-e53d-4c6b-93ed-a6271c85ac27-ca\") pod \"cluster-proxy-proxy-agent-64d77d549c-dndvm\" (UID: \"204204da-e53d-4c6b-93ed-a6271c85ac27\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-64d77d549c-dndvm"
Sep 17 23:13:22.321750 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:13:22.321500    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-proxy-server-cert\" (UniqueName: \"kubernetes.io/secret/204204da-e53d-4c6b-93ed-a6271c85ac27-service-proxy-server-cert\") pod \"cluster-proxy-proxy-agent-64d77d549c-dndvm\" (UID: \"204204da-e53d-4c6b-93ed-a6271c85ac27\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-64d77d549c-dndvm"
Sep 17 23:13:22.321750 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:13:22.321526    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-c55hj\" (UniqueName: \"kubernetes.io/projected/204204da-e53d-4c6b-93ed-a6271c85ac27-kube-api-access-c55hj\") pod \"cluster-proxy-proxy-agent-64d77d549c-dndvm\" (UID: \"204204da-e53d-4c6b-93ed-a6271c85ac27\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-64d77d549c-dndvm"
Sep 17 23:13:22.321750 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:13:22.321548    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hub\" (UniqueName: \"kubernetes.io/secret/204204da-e53d-4c6b-93ed-a6271c85ac27-hub\") pod \"cluster-proxy-proxy-agent-64d77d549c-dndvm\" (UID: \"204204da-e53d-4c6b-93ed-a6271c85ac27\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-64d77d549c-dndvm"
Sep 17 23:13:22.322451 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:13:22.322418    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"additional-service-ca\" (UniqueName: \"kubernetes.io/configmap/204204da-e53d-4c6b-93ed-a6271c85ac27-additional-service-ca\") pod \"cluster-proxy-proxy-agent-64d77d549c-dndvm\" (UID: \"204204da-e53d-4c6b-93ed-a6271c85ac27\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-64d77d549c-dndvm"
Sep 17 23:13:22.324075 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:13:22.324048    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-proxy-server-cert\" (UniqueName: \"kubernetes.io/secret/204204da-e53d-4c6b-93ed-a6271c85ac27-service-proxy-server-cert\") pod \"cluster-proxy-proxy-agent-64d77d549c-dndvm\" (UID: \"204204da-e53d-4c6b-93ed-a6271c85ac27\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-64d77d549c-dndvm"
Sep 17 23:13:22.324179 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:13:22.324133    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ca\" (UniqueName: \"kubernetes.io/secret/204204da-e53d-4c6b-93ed-a6271c85ac27-ca\") pod \"cluster-proxy-proxy-agent-64d77d549c-dndvm\" (UID: \"204204da-e53d-4c6b-93ed-a6271c85ac27\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-64d77d549c-dndvm"
Sep 17 23:13:22.324333 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:13:22.324312    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hub\" (UniqueName: \"kubernetes.io/secret/204204da-e53d-4c6b-93ed-a6271c85ac27-hub\") pod \"cluster-proxy-proxy-agent-64d77d549c-dndvm\" (UID: \"204204da-e53d-4c6b-93ed-a6271c85ac27\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-64d77d549c-dndvm"
Sep 17 23:13:22.324373 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:13:22.324337    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/204204da-e53d-4c6b-93ed-a6271c85ac27-hub-kubeconfig\") pod \"cluster-proxy-proxy-agent-64d77d549c-dndvm\" (UID: \"204204da-e53d-4c6b-93ed-a6271c85ac27\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-64d77d549c-dndvm"
Sep 17 23:13:22.330253 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:13:22.330231    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-c55hj\" (UniqueName: \"kubernetes.io/projected/204204da-e53d-4c6b-93ed-a6271c85ac27-kube-api-access-c55hj\") pod \"cluster-proxy-proxy-agent-64d77d549c-dndvm\" (UID: \"204204da-e53d-4c6b-93ed-a6271c85ac27\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-64d77d549c-dndvm"
Sep 17 23:13:22.358163 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:13:22.358125    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-64d77d549c-dndvm"
Sep 17 23:13:22.633117 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:13:22.633084    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/cluster-proxy-proxy-agent-64d77d549c-dndvm"]
Sep 17 23:13:22.636533 ip-10-0-132-188 kubenswrapper[2562]: W0917 23:13:22.636506    2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod204204da_e53d_4c6b_93ed_a6271c85ac27.slice/crio-5516ce065f0447645ddf9ff8ab5785ebad0eb3895dbaef24728c6bb37f67839c WatchSource:0}: Error finding container 5516ce065f0447645ddf9ff8ab5785ebad0eb3895dbaef24728c6bb37f67839c: Status 404 returned error can't find the container with id 5516ce065f0447645ddf9ff8ab5785ebad0eb3895dbaef24728c6bb37f67839c
Sep 17 23:13:23.291852 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:13:23.291815    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-64d77d549c-dndvm" event={"ID":"204204da-e53d-4c6b-93ed-a6271c85ac27","Type":"ContainerStarted","Data":"5516ce065f0447645ddf9ff8ab5785ebad0eb3895dbaef24728c6bb37f67839c"}
Sep 17 23:13:26.139011 ip-10-0-132-188 kubenswrapper[2562]: E0917 23:13:26.138962    2562 pod_workers.go:1324] "Error syncing pod, skipping" err="unmounted volumes=[metrics-tls], unattached volumes=[], failed to process volumes=[]: context deadline exceeded" pod="openshift-dns/dns-default-9z4vw" podUID="57e9d0f5-d16a-42d0-ab45-c06fe21e7bbd"
Sep 17 23:13:26.300669 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:13:26.300642    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-9z4vw"
Sep 17 23:13:26.675973 ip-10-0-132-188 kubenswrapper[2562]: E0917 23:13:26.675878    2562 pod_workers.go:1324] "Error syncing pod, skipping" err="unmounted volumes=[metrics-certs], unattached volumes=[], failed to process volumes=[]: context deadline exceeded" pod="openshift-multus/network-metrics-daemon-nbccd" podUID="4453ba96-a67f-46af-b61b-9cd8f39b6d36"
Sep 17 23:13:27.305110 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:13:27.305074    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-64d77d549c-dndvm" event={"ID":"204204da-e53d-4c6b-93ed-a6271c85ac27","Type":"ContainerStarted","Data":"a3fb94cc577c91226e36070d0fc0c23adbfb2d08783a3ad8e99a4955a1b14571"}
Sep 17 23:13:27.305590 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:13:27.305120    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-64d77d549c-dndvm" event={"ID":"204204da-e53d-4c6b-93ed-a6271c85ac27","Type":"ContainerStarted","Data":"66b6ce6c335bd16f4585dbfcdcab771d17adf304d6b13b7db58ac10e1768f9db"}
Sep 17 23:13:27.305590 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:13:27.305137    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-64d77d549c-dndvm" event={"ID":"204204da-e53d-4c6b-93ed-a6271c85ac27","Type":"ContainerStarted","Data":"0742d74f376272d5d266ca45ecbb7e3c74368186f0653664f9444e20122fb41b"}
Sep 17 23:13:30.987821 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:13:30.987715    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/57e9d0f5-d16a-42d0-ab45-c06fe21e7bbd-metrics-tls\") pod \"dns-default-9z4vw\" (UID: \"57e9d0f5-d16a-42d0-ab45-c06fe21e7bbd\") " pod="openshift-dns/dns-default-9z4vw"
Sep 17 23:13:30.988329 ip-10-0-132-188 kubenswrapper[2562]: E0917 23:13:30.987892    2562 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Sep 17 23:13:30.988329 ip-10-0-132-188 kubenswrapper[2562]: E0917 23:13:30.987981    2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/57e9d0f5-d16a-42d0-ab45-c06fe21e7bbd-metrics-tls podName:57e9d0f5-d16a-42d0-ab45-c06fe21e7bbd nodeName:}" failed. No retries permitted until 2026-09-17 23:15:32.987958382 +0000 UTC m=+285.389855138 (durationBeforeRetry 2m2s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/57e9d0f5-d16a-42d0-ab45-c06fe21e7bbd-metrics-tls") pod "dns-default-9z4vw" (UID: "57e9d0f5-d16a-42d0-ab45-c06fe21e7bbd") : secret "dns-default-metrics-tls" not found
Sep 17 23:13:38.661399 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:13:38.661360    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-nbccd"
Sep 17 23:14:09.635034 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:14:09.634999    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fwwc5_7e452bbb-7dc6-4c9d-a9b1-76400942f5b4/konnectivity-agent/0.log" limit=1024
Sep 17 23:14:30.178507 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:14:30.178452    2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-64d77d549c-dndvm" podStartSLOduration=64.509519343 podStartE2EDuration="1m8.178436417s" totalImagesPullingTime="3.668917074s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-17 23:13:22 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-17 23:13:27.322225411 +0000 UTC m=+159.724122176" watchObservedRunningTime="2026-09-17 23:14:30.178436417 +0000 UTC m=+222.580333180"
Sep 17 23:14:30.179157 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:14:30.179141    2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-network-console/networking-console-plugin-6446c8d75d-9gc6s"]
Sep 17 23:14:30.181271 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:14:30.181256    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-console/networking-console-plugin-6446c8d75d-9gc6s"
Sep 17 23:14:30.183664 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:14:30.183639    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-console\"/\"networking-console-plugin-cert\""
Sep 17 23:14:30.183891 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:14:30.183874    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-console\"/\"networking-console-plugin\""
Sep 17 23:14:30.183956 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:14:30.183943    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-console\"/\"default-dockercfg-l459d\""
Sep 17 23:14:30.191692 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:14:30.191668    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-console/networking-console-plugin-6446c8d75d-9gc6s"]
Sep 17 23:14:30.286301 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:14:30.286266    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/68dcc3ba-959d-419d-a77b-31b95d377616-nginx-conf\") pod \"networking-console-plugin-6446c8d75d-9gc6s\" (UID: \"68dcc3ba-959d-419d-a77b-31b95d377616\") " pod="openshift-network-console/networking-console-plugin-6446c8d75d-9gc6s"
Sep 17 23:14:30.286301 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:14:30.286304    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/68dcc3ba-959d-419d-a77b-31b95d377616-networking-console-plugin-cert\") pod \"networking-console-plugin-6446c8d75d-9gc6s\" (UID: \"68dcc3ba-959d-419d-a77b-31b95d377616\") " pod="openshift-network-console/networking-console-plugin-6446c8d75d-9gc6s"
Sep 17 23:14:30.359980 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:14:30.359933    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fwwc5_7e452bbb-7dc6-4c9d-a9b1-76400942f5b4/konnectivity-agent/0.log" limit=1024
Sep 17 23:14:30.386868 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:14:30.386841    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/68dcc3ba-959d-419d-a77b-31b95d377616-nginx-conf\") pod \"networking-console-plugin-6446c8d75d-9gc6s\" (UID: \"68dcc3ba-959d-419d-a77b-31b95d377616\") " pod="openshift-network-console/networking-console-plugin-6446c8d75d-9gc6s"
Sep 17 23:14:30.386961 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:14:30.386878    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/68dcc3ba-959d-419d-a77b-31b95d377616-networking-console-plugin-cert\") pod \"networking-console-plugin-6446c8d75d-9gc6s\" (UID: \"68dcc3ba-959d-419d-a77b-31b95d377616\") " pod="openshift-network-console/networking-console-plugin-6446c8d75d-9gc6s"
Sep 17 23:14:30.386998 ip-10-0-132-188 kubenswrapper[2562]: E0917 23:14:30.386980    2562 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found
Sep 17 23:14:30.387051 ip-10-0-132-188 kubenswrapper[2562]: E0917 23:14:30.387041    2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/68dcc3ba-959d-419d-a77b-31b95d377616-networking-console-plugin-cert podName:68dcc3ba-959d-419d-a77b-31b95d377616 nodeName:}" failed. No retries permitted until 2026-09-17 23:14:30.887027466 +0000 UTC m=+223.288924207 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/68dcc3ba-959d-419d-a77b-31b95d377616-networking-console-plugin-cert") pod "networking-console-plugin-6446c8d75d-9gc6s" (UID: "68dcc3ba-959d-419d-a77b-31b95d377616") : secret "networking-console-plugin-cert" not found
Sep 17 23:14:30.387473 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:14:30.387454    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/68dcc3ba-959d-419d-a77b-31b95d377616-nginx-conf\") pod \"networking-console-plugin-6446c8d75d-9gc6s\" (UID: \"68dcc3ba-959d-419d-a77b-31b95d377616\") " pod="openshift-network-console/networking-console-plugin-6446c8d75d-9gc6s"
Sep 17 23:14:30.461003 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:14:30.460900    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fwwc5_7e452bbb-7dc6-4c9d-a9b1-76400942f5b4/konnectivity-agent/0.log" limit=1024
Sep 17 23:14:30.891085 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:14:30.891041    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/68dcc3ba-959d-419d-a77b-31b95d377616-networking-console-plugin-cert\") pod \"networking-console-plugin-6446c8d75d-9gc6s\" (UID: \"68dcc3ba-959d-419d-a77b-31b95d377616\") " pod="openshift-network-console/networking-console-plugin-6446c8d75d-9gc6s"
Sep 17 23:14:30.891273 ip-10-0-132-188 kubenswrapper[2562]: E0917 23:14:30.891157    2562 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found
Sep 17 23:14:30.891273 ip-10-0-132-188 kubenswrapper[2562]: E0917 23:14:30.891237    2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/68dcc3ba-959d-419d-a77b-31b95d377616-networking-console-plugin-cert podName:68dcc3ba-959d-419d-a77b-31b95d377616 nodeName:}" failed. No retries permitted until 2026-09-17 23:14:31.891222866 +0000 UTC m=+224.293119608 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/68dcc3ba-959d-419d-a77b-31b95d377616-networking-console-plugin-cert") pod "networking-console-plugin-6446c8d75d-9gc6s" (UID: "68dcc3ba-959d-419d-a77b-31b95d377616") : secret "networking-console-plugin-cert" not found
Sep 17 23:14:31.897828 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:14:31.897778    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/68dcc3ba-959d-419d-a77b-31b95d377616-networking-console-plugin-cert\") pod \"networking-console-plugin-6446c8d75d-9gc6s\" (UID: \"68dcc3ba-959d-419d-a77b-31b95d377616\") " pod="openshift-network-console/networking-console-plugin-6446c8d75d-9gc6s"
Sep 17 23:14:31.898267 ip-10-0-132-188 kubenswrapper[2562]: E0917 23:14:31.897939    2562 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found
Sep 17 23:14:31.898267 ip-10-0-132-188 kubenswrapper[2562]: E0917 23:14:31.898024    2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/68dcc3ba-959d-419d-a77b-31b95d377616-networking-console-plugin-cert podName:68dcc3ba-959d-419d-a77b-31b95d377616 nodeName:}" failed. No retries permitted until 2026-09-17 23:14:33.898005917 +0000 UTC m=+226.299902677 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/68dcc3ba-959d-419d-a77b-31b95d377616-networking-console-plugin-cert") pod "networking-console-plugin-6446c8d75d-9gc6s" (UID: "68dcc3ba-959d-419d-a77b-31b95d377616") : secret "networking-console-plugin-cert" not found
Sep 17 23:14:33.311456 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:14:33.311401    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fwwc5_7e452bbb-7dc6-4c9d-a9b1-76400942f5b4/konnectivity-agent/0.log" limit=1024
Sep 17 23:14:33.360162 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:14:33.360126    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fwwc5_7e452bbb-7dc6-4c9d-a9b1-76400942f5b4/konnectivity-agent/0.log" limit=1024
Sep 17 23:14:33.757723 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:14:33.757688    2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-insights/insights-runtime-extractor-59djn"]
Sep 17 23:14:33.759804 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:14:33.759786    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-59djn"
Sep 17 23:14:33.763711 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:14:33.763681    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-tls\""
Sep 17 23:14:33.764005 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:14:33.763988    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-sa-dockercfg-9v429\""
Sep 17 23:14:33.764248 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:14:33.764233    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"openshift-service-ca.crt\""
Sep 17 23:14:33.764563 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:14:33.764548    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-rbac-proxy\""
Sep 17 23:14:33.764825 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:14:33.764810    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-root-ca.crt\""
Sep 17 23:14:33.769712 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:14:33.769682    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-59djn"]
Sep 17 23:14:33.774517 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:14:33.774491    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fwwc5_7e452bbb-7dc6-4c9d-a9b1-76400942f5b4/konnectivity-agent/0.log" limit=1024
Sep 17 23:14:33.831612 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:14:33.831572    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fwwc5_7e452bbb-7dc6-4c9d-a9b1-76400942f5b4/konnectivity-agent/0.log" limit=1024
Sep 17 23:14:33.914863 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:14:33.914830    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/062c04b5-55f2-49ca-aff9-19fe1c0a20b8-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-59djn\" (UID: \"062c04b5-55f2-49ca-aff9-19fe1c0a20b8\") " pod="openshift-insights/insights-runtime-extractor-59djn"
Sep 17 23:14:33.914991 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:14:33.914867    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9hbrf\" (UniqueName: \"kubernetes.io/projected/062c04b5-55f2-49ca-aff9-19fe1c0a20b8-kube-api-access-9hbrf\") pod \"insights-runtime-extractor-59djn\" (UID: \"062c04b5-55f2-49ca-aff9-19fe1c0a20b8\") " pod="openshift-insights/insights-runtime-extractor-59djn"
Sep 17 23:14:33.915041 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:14:33.914993    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/062c04b5-55f2-49ca-aff9-19fe1c0a20b8-crio-socket\") pod \"insights-runtime-extractor-59djn\" (UID: \"062c04b5-55f2-49ca-aff9-19fe1c0a20b8\") " pod="openshift-insights/insights-runtime-extractor-59djn"
Sep 17 23:14:33.915041 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:14:33.915021    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/062c04b5-55f2-49ca-aff9-19fe1c0a20b8-data-volume\") pod \"insights-runtime-extractor-59djn\" (UID: \"062c04b5-55f2-49ca-aff9-19fe1c0a20b8\") " pod="openshift-insights/insights-runtime-extractor-59djn"
Sep 17 23:14:33.915113 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:14:33.915044    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/062c04b5-55f2-49ca-aff9-19fe1c0a20b8-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-59djn\" (UID: \"062c04b5-55f2-49ca-aff9-19fe1c0a20b8\") " pod="openshift-insights/insights-runtime-extractor-59djn"
Sep 17 23:14:33.915113 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:14:33.915096    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/68dcc3ba-959d-419d-a77b-31b95d377616-networking-console-plugin-cert\") pod \"networking-console-plugin-6446c8d75d-9gc6s\" (UID: \"68dcc3ba-959d-419d-a77b-31b95d377616\") " pod="openshift-network-console/networking-console-plugin-6446c8d75d-9gc6s"
Sep 17 23:14:33.915221 ip-10-0-132-188 kubenswrapper[2562]: E0917 23:14:33.915208    2562 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found
Sep 17 23:14:33.915268 ip-10-0-132-188 kubenswrapper[2562]: E0917 23:14:33.915263    2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/68dcc3ba-959d-419d-a77b-31b95d377616-networking-console-plugin-cert podName:68dcc3ba-959d-419d-a77b-31b95d377616 nodeName:}" failed. No retries permitted until 2026-09-17 23:14:37.915247597 +0000 UTC m=+230.317144340 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/68dcc3ba-959d-419d-a77b-31b95d377616-networking-console-plugin-cert") pod "networking-console-plugin-6446c8d75d-9gc6s" (UID: "68dcc3ba-959d-419d-a77b-31b95d377616") : secret "networking-console-plugin-cert" not found
Sep 17 23:14:34.015707 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:14:34.015633    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/062c04b5-55f2-49ca-aff9-19fe1c0a20b8-crio-socket\") pod \"insights-runtime-extractor-59djn\" (UID: \"062c04b5-55f2-49ca-aff9-19fe1c0a20b8\") " pod="openshift-insights/insights-runtime-extractor-59djn"
Sep 17 23:14:34.015707 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:14:34.015669    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/062c04b5-55f2-49ca-aff9-19fe1c0a20b8-data-volume\") pod \"insights-runtime-extractor-59djn\" (UID: \"062c04b5-55f2-49ca-aff9-19fe1c0a20b8\") " pod="openshift-insights/insights-runtime-extractor-59djn"
Sep 17 23:14:34.015707 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:14:34.015690    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/062c04b5-55f2-49ca-aff9-19fe1c0a20b8-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-59djn\" (UID: \"062c04b5-55f2-49ca-aff9-19fe1c0a20b8\") " pod="openshift-insights/insights-runtime-extractor-59djn"
Sep 17 23:14:34.015922 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:14:34.015719    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/062c04b5-55f2-49ca-aff9-19fe1c0a20b8-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-59djn\" (UID: \"062c04b5-55f2-49ca-aff9-19fe1c0a20b8\") " pod="openshift-insights/insights-runtime-extractor-59djn"
Sep 17 23:14:34.015922 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:14:34.015736    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9hbrf\" (UniqueName: \"kubernetes.io/projected/062c04b5-55f2-49ca-aff9-19fe1c0a20b8-kube-api-access-9hbrf\") pod \"insights-runtime-extractor-59djn\" (UID: \"062c04b5-55f2-49ca-aff9-19fe1c0a20b8\") " pod="openshift-insights/insights-runtime-extractor-59djn"
Sep 17 23:14:34.015922 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:14:34.015772    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/062c04b5-55f2-49ca-aff9-19fe1c0a20b8-crio-socket\") pod \"insights-runtime-extractor-59djn\" (UID: \"062c04b5-55f2-49ca-aff9-19fe1c0a20b8\") " pod="openshift-insights/insights-runtime-extractor-59djn"
Sep 17 23:14:34.015922 ip-10-0-132-188 kubenswrapper[2562]: E0917 23:14:34.015846    2562 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Sep 17 23:14:34.015922 ip-10-0-132-188 kubenswrapper[2562]: E0917 23:14:34.015862    2562 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Sep 17 23:14:34.015922 ip-10-0-132-188 kubenswrapper[2562]: E0917 23:14:34.015903    2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/062c04b5-55f2-49ca-aff9-19fe1c0a20b8-insights-runtime-extractor-tls podName:062c04b5-55f2-49ca-aff9-19fe1c0a20b8 nodeName:}" failed. No retries permitted until 2026-09-17 23:14:34.515887949 +0000 UTC m=+226.917784691 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/062c04b5-55f2-49ca-aff9-19fe1c0a20b8-insights-runtime-extractor-tls") pod "insights-runtime-extractor-59djn" (UID: "062c04b5-55f2-49ca-aff9-19fe1c0a20b8") : secret "insights-runtime-extractor-tls" not found
Sep 17 23:14:34.016195 ip-10-0-132-188 kubenswrapper[2562]: E0917 23:14:34.015932    2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/062c04b5-55f2-49ca-aff9-19fe1c0a20b8-kube-rbac-proxy-cm podName:062c04b5-55f2-49ca-aff9-19fe1c0a20b8 nodeName:}" failed. No retries permitted until 2026-09-17 23:14:34.515914493 +0000 UTC m=+226.917811240 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/062c04b5-55f2-49ca-aff9-19fe1c0a20b8-kube-rbac-proxy-cm") pod "insights-runtime-extractor-59djn" (UID: "062c04b5-55f2-49ca-aff9-19fe1c0a20b8") : configmap "kube-rbac-proxy" not found
Sep 17 23:14:34.016195 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:14:34.015997    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/062c04b5-55f2-49ca-aff9-19fe1c0a20b8-data-volume\") pod \"insights-runtime-extractor-59djn\" (UID: \"062c04b5-55f2-49ca-aff9-19fe1c0a20b8\") " pod="openshift-insights/insights-runtime-extractor-59djn"
Sep 17 23:14:34.026364 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:14:34.026341    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-9hbrf\" (UniqueName: \"kubernetes.io/projected/062c04b5-55f2-49ca-aff9-19fe1c0a20b8-kube-api-access-9hbrf\") pod \"insights-runtime-extractor-59djn\" (UID: \"062c04b5-55f2-49ca-aff9-19fe1c0a20b8\") " pod="openshift-insights/insights-runtime-extractor-59djn"
Sep 17 23:14:34.519034 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:14:34.518984    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/062c04b5-55f2-49ca-aff9-19fe1c0a20b8-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-59djn\" (UID: \"062c04b5-55f2-49ca-aff9-19fe1c0a20b8\") " pod="openshift-insights/insights-runtime-extractor-59djn"
Sep 17 23:14:34.519460 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:14:34.519057    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/062c04b5-55f2-49ca-aff9-19fe1c0a20b8-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-59djn\" (UID: \"062c04b5-55f2-49ca-aff9-19fe1c0a20b8\") " pod="openshift-insights/insights-runtime-extractor-59djn"
Sep 17 23:14:34.519460 ip-10-0-132-188 kubenswrapper[2562]: E0917 23:14:34.519142    2562 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Sep 17 23:14:34.519460 ip-10-0-132-188 kubenswrapper[2562]: E0917 23:14:34.519197    2562 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Sep 17 23:14:34.519460 ip-10-0-132-188 kubenswrapper[2562]: E0917 23:14:34.519224    2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/062c04b5-55f2-49ca-aff9-19fe1c0a20b8-insights-runtime-extractor-tls podName:062c04b5-55f2-49ca-aff9-19fe1c0a20b8 nodeName:}" failed. No retries permitted until 2026-09-17 23:14:35.519206361 +0000 UTC m=+227.921103120 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/062c04b5-55f2-49ca-aff9-19fe1c0a20b8-insights-runtime-extractor-tls") pod "insights-runtime-extractor-59djn" (UID: "062c04b5-55f2-49ca-aff9-19fe1c0a20b8") : secret "insights-runtime-extractor-tls" not found
Sep 17 23:14:34.519460 ip-10-0-132-188 kubenswrapper[2562]: E0917 23:14:34.519246    2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/062c04b5-55f2-49ca-aff9-19fe1c0a20b8-kube-rbac-proxy-cm podName:062c04b5-55f2-49ca-aff9-19fe1c0a20b8 nodeName:}" failed. No retries permitted until 2026-09-17 23:14:35.519232244 +0000 UTC m=+227.921128987 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/062c04b5-55f2-49ca-aff9-19fe1c0a20b8-kube-rbac-proxy-cm") pod "insights-runtime-extractor-59djn" (UID: "062c04b5-55f2-49ca-aff9-19fe1c0a20b8") : configmap "kube-rbac-proxy" not found
Sep 17 23:14:35.262385 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:14:35.262349    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fwwc5_7e452bbb-7dc6-4c9d-a9b1-76400942f5b4/konnectivity-agent/0.log" limit=1024
Sep 17 23:14:35.526136 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:14:35.526044    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/062c04b5-55f2-49ca-aff9-19fe1c0a20b8-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-59djn\" (UID: \"062c04b5-55f2-49ca-aff9-19fe1c0a20b8\") " pod="openshift-insights/insights-runtime-extractor-59djn"
Sep 17 23:14:35.526136 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:14:35.526093    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/062c04b5-55f2-49ca-aff9-19fe1c0a20b8-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-59djn\" (UID: \"062c04b5-55f2-49ca-aff9-19fe1c0a20b8\") " pod="openshift-insights/insights-runtime-extractor-59djn"
Sep 17 23:14:35.526566 ip-10-0-132-188 kubenswrapper[2562]: E0917 23:14:35.526189    2562 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Sep 17 23:14:35.526566 ip-10-0-132-188 kubenswrapper[2562]: E0917 23:14:35.526206    2562 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Sep 17 23:14:35.526566 ip-10-0-132-188 kubenswrapper[2562]: E0917 23:14:35.526241    2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/062c04b5-55f2-49ca-aff9-19fe1c0a20b8-kube-rbac-proxy-cm podName:062c04b5-55f2-49ca-aff9-19fe1c0a20b8 nodeName:}" failed. No retries permitted until 2026-09-17 23:14:37.526227634 +0000 UTC m=+229.928124376 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/062c04b5-55f2-49ca-aff9-19fe1c0a20b8-kube-rbac-proxy-cm") pod "insights-runtime-extractor-59djn" (UID: "062c04b5-55f2-49ca-aff9-19fe1c0a20b8") : configmap "kube-rbac-proxy" not found
Sep 17 23:14:35.526566 ip-10-0-132-188 kubenswrapper[2562]: E0917 23:14:35.526259    2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/062c04b5-55f2-49ca-aff9-19fe1c0a20b8-insights-runtime-extractor-tls podName:062c04b5-55f2-49ca-aff9-19fe1c0a20b8 nodeName:}" failed. No retries permitted until 2026-09-17 23:14:37.526246553 +0000 UTC m=+229.928143298 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/062c04b5-55f2-49ca-aff9-19fe1c0a20b8-insights-runtime-extractor-tls") pod "insights-runtime-extractor-59djn" (UID: "062c04b5-55f2-49ca-aff9-19fe1c0a20b8") : secret "insights-runtime-extractor-tls" not found
Sep 17 23:14:35.723598 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:14:35.723571    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-fzpjd_1d92dc9b-45b6-4912-bf69-f0961e86e8ed/dns-node-resolver/0.log"
Sep 17 23:14:36.815483 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:14:36.815450    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fwwc5_7e452bbb-7dc6-4c9d-a9b1-76400942f5b4/konnectivity-agent/0.log" limit=1024
Sep 17 23:14:36.914064 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:14:36.914033    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fwwc5_7e452bbb-7dc6-4c9d-a9b1-76400942f5b4/konnectivity-agent/0.log" limit=1024
Sep 17 23:14:36.925514 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:14:36.925489    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-vkllk_33db9205-3be0-47dc-ac16-e8f623e43afb/node-ca/0.log"
Sep 17 23:14:37.308895 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:14:37.308857    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fwwc5_7e452bbb-7dc6-4c9d-a9b1-76400942f5b4/konnectivity-agent/0.log" limit=1024
Sep 17 23:14:37.359331 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:14:37.359299    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fwwc5_7e452bbb-7dc6-4c9d-a9b1-76400942f5b4/konnectivity-agent/0.log" limit=1024
Sep 17 23:14:37.523215 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:14:37.523164    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-hqrgl_5562eaa3-4a89-4ed2-b42b-831c3da5bba3/serve-healthcheck-canary/0.log"
Sep 17 23:14:37.543457 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:14:37.543426    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/062c04b5-55f2-49ca-aff9-19fe1c0a20b8-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-59djn\" (UID: \"062c04b5-55f2-49ca-aff9-19fe1c0a20b8\") " pod="openshift-insights/insights-runtime-extractor-59djn"
Sep 17 23:14:37.543582 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:14:37.543479    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/062c04b5-55f2-49ca-aff9-19fe1c0a20b8-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-59djn\" (UID: \"062c04b5-55f2-49ca-aff9-19fe1c0a20b8\") " pod="openshift-insights/insights-runtime-extractor-59djn"
Sep 17 23:14:37.543640 ip-10-0-132-188 kubenswrapper[2562]: E0917 23:14:37.543605    2562 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Sep 17 23:14:37.543640 ip-10-0-132-188 kubenswrapper[2562]: E0917 23:14:37.543630    2562 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Sep 17 23:14:37.543739 ip-10-0-132-188 kubenswrapper[2562]: E0917 23:14:37.543685    2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/062c04b5-55f2-49ca-aff9-19fe1c0a20b8-insights-runtime-extractor-tls podName:062c04b5-55f2-49ca-aff9-19fe1c0a20b8 nodeName:}" failed. No retries permitted until 2026-09-17 23:14:41.543663098 +0000 UTC m=+233.945559855 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/062c04b5-55f2-49ca-aff9-19fe1c0a20b8-insights-runtime-extractor-tls") pod "insights-runtime-extractor-59djn" (UID: "062c04b5-55f2-49ca-aff9-19fe1c0a20b8") : secret "insights-runtime-extractor-tls" not found
Sep 17 23:14:37.543739 ip-10-0-132-188 kubenswrapper[2562]: E0917 23:14:37.543707    2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/062c04b5-55f2-49ca-aff9-19fe1c0a20b8-kube-rbac-proxy-cm podName:062c04b5-55f2-49ca-aff9-19fe1c0a20b8 nodeName:}" failed. No retries permitted until 2026-09-17 23:14:41.543695075 +0000 UTC m=+233.945591823 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/062c04b5-55f2-49ca-aff9-19fe1c0a20b8-kube-rbac-proxy-cm") pod "insights-runtime-extractor-59djn" (UID: "062c04b5-55f2-49ca-aff9-19fe1c0a20b8") : configmap "kube-rbac-proxy" not found
Sep 17 23:14:37.947372 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:14:37.947325    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/68dcc3ba-959d-419d-a77b-31b95d377616-networking-console-plugin-cert\") pod \"networking-console-plugin-6446c8d75d-9gc6s\" (UID: \"68dcc3ba-959d-419d-a77b-31b95d377616\") " pod="openshift-network-console/networking-console-plugin-6446c8d75d-9gc6s"
Sep 17 23:14:37.947789 ip-10-0-132-188 kubenswrapper[2562]: E0917 23:14:37.947474    2562 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found
Sep 17 23:14:37.947789 ip-10-0-132-188 kubenswrapper[2562]: E0917 23:14:37.947532    2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/68dcc3ba-959d-419d-a77b-31b95d377616-networking-console-plugin-cert podName:68dcc3ba-959d-419d-a77b-31b95d377616 nodeName:}" failed. No retries permitted until 2026-09-17 23:14:45.947518153 +0000 UTC m=+238.349414900 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/68dcc3ba-959d-419d-a77b-31b95d377616-networking-console-plugin-cert") pod "networking-console-plugin-6446c8d75d-9gc6s" (UID: "68dcc3ba-959d-419d-a77b-31b95d377616") : secret "networking-console-plugin-cert" not found
Sep 17 23:14:38.472353 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:14:38.472309    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fwwc5_7e452bbb-7dc6-4c9d-a9b1-76400942f5b4/konnectivity-agent/0.log" limit=1024
Sep 17 23:14:38.521364 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:14:38.521326    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fwwc5_7e452bbb-7dc6-4c9d-a9b1-76400942f5b4/konnectivity-agent/0.log" limit=1024
Sep 17 23:14:39.390344 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:14:39.390308    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fwwc5_7e452bbb-7dc6-4c9d-a9b1-76400942f5b4/konnectivity-agent/0.log" limit=1024
Sep 17 23:14:41.576134 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:14:41.576085    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/062c04b5-55f2-49ca-aff9-19fe1c0a20b8-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-59djn\" (UID: \"062c04b5-55f2-49ca-aff9-19fe1c0a20b8\") " pod="openshift-insights/insights-runtime-extractor-59djn"
Sep 17 23:14:41.576675 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:14:41.576345    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/062c04b5-55f2-49ca-aff9-19fe1c0a20b8-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-59djn\" (UID: \"062c04b5-55f2-49ca-aff9-19fe1c0a20b8\") " pod="openshift-insights/insights-runtime-extractor-59djn"
Sep 17 23:14:41.576675 ip-10-0-132-188 kubenswrapper[2562]: E0917 23:14:41.576354    2562 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Sep 17 23:14:41.576675 ip-10-0-132-188 kubenswrapper[2562]: E0917 23:14:41.576447    2562 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Sep 17 23:14:41.576675 ip-10-0-132-188 kubenswrapper[2562]: E0917 23:14:41.576512    2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/062c04b5-55f2-49ca-aff9-19fe1c0a20b8-kube-rbac-proxy-cm podName:062c04b5-55f2-49ca-aff9-19fe1c0a20b8 nodeName:}" failed. No retries permitted until 2026-09-17 23:14:49.576489987 +0000 UTC m=+241.978386731 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/062c04b5-55f2-49ca-aff9-19fe1c0a20b8-kube-rbac-proxy-cm") pod "insights-runtime-extractor-59djn" (UID: "062c04b5-55f2-49ca-aff9-19fe1c0a20b8") : configmap "kube-rbac-proxy" not found
Sep 17 23:14:41.576925 ip-10-0-132-188 kubenswrapper[2562]: E0917 23:14:41.576905    2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/062c04b5-55f2-49ca-aff9-19fe1c0a20b8-insights-runtime-extractor-tls podName:062c04b5-55f2-49ca-aff9-19fe1c0a20b8 nodeName:}" failed. No retries permitted until 2026-09-17 23:14:49.576878625 +0000 UTC m=+241.978775381 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/062c04b5-55f2-49ca-aff9-19fe1c0a20b8-insights-runtime-extractor-tls") pod "insights-runtime-extractor-59djn" (UID: "062c04b5-55f2-49ca-aff9-19fe1c0a20b8") : secret "insights-runtime-extractor-tls" not found
Sep 17 23:14:41.774315 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:14:41.774278    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fwwc5_7e452bbb-7dc6-4c9d-a9b1-76400942f5b4/konnectivity-agent/0.log" limit=1024
Sep 17 23:14:41.970666 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:14:41.970582    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fwwc5_7e452bbb-7dc6-4c9d-a9b1-76400942f5b4/konnectivity-agent/0.log" limit=1024
Sep 17 23:14:42.070892 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:14:42.070860    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fwwc5_7e452bbb-7dc6-4c9d-a9b1-76400942f5b4/konnectivity-agent/0.log" limit=1024
Sep 17 23:14:42.173698 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:14:42.173662    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fwwc5_7e452bbb-7dc6-4c9d-a9b1-76400942f5b4/konnectivity-agent/0.log" limit=1024
Sep 17 23:14:42.277965 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:14:42.277879    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fwwc5_7e452bbb-7dc6-4c9d-a9b1-76400942f5b4/konnectivity-agent/0.log" limit=1024
Sep 17 23:14:42.377564 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:14:42.377524    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fwwc5_7e452bbb-7dc6-4c9d-a9b1-76400942f5b4/konnectivity-agent/0.log" limit=1024
Sep 17 23:14:42.472097 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:14:42.472061    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fwwc5_7e452bbb-7dc6-4c9d-a9b1-76400942f5b4/konnectivity-agent/0.log" limit=1024
Sep 17 23:14:43.269156 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:14:43.269122    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fwwc5_7e452bbb-7dc6-4c9d-a9b1-76400942f5b4/konnectivity-agent/0.log" limit=1024
Sep 17 23:14:43.865738 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:14:43.865698    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fwwc5_7e452bbb-7dc6-4c9d-a9b1-76400942f5b4/konnectivity-agent/0.log" limit=1024
Sep 17 23:14:44.067390 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:14:44.067348    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fwwc5_7e452bbb-7dc6-4c9d-a9b1-76400942f5b4/konnectivity-agent/0.log" limit=1024
Sep 17 23:14:44.468584 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:14:44.468543    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fwwc5_7e452bbb-7dc6-4c9d-a9b1-76400942f5b4/konnectivity-agent/0.log" limit=1024
Sep 17 23:14:44.663828 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:14:44.663786    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fwwc5_7e452bbb-7dc6-4c9d-a9b1-76400942f5b4/konnectivity-agent/0.log" limit=1024
Sep 17 23:14:45.266972 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:14:45.266940    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fwwc5_7e452bbb-7dc6-4c9d-a9b1-76400942f5b4/konnectivity-agent/0.log" limit=1024
Sep 17 23:14:45.641313 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:14:45.641281    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fwwc5_7e452bbb-7dc6-4c9d-a9b1-76400942f5b4/konnectivity-agent/0.log" limit=1024
Sep 17 23:14:45.689110 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:14:45.689057    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fwwc5_7e452bbb-7dc6-4c9d-a9b1-76400942f5b4/konnectivity-agent/0.log" limit=1024
Sep 17 23:14:46.012948 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:14:46.012865    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/68dcc3ba-959d-419d-a77b-31b95d377616-networking-console-plugin-cert\") pod \"networking-console-plugin-6446c8d75d-9gc6s\" (UID: \"68dcc3ba-959d-419d-a77b-31b95d377616\") " pod="openshift-network-console/networking-console-plugin-6446c8d75d-9gc6s"
Sep 17 23:14:46.013094 ip-10-0-132-188 kubenswrapper[2562]: E0917 23:14:46.013016    2562 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found
Sep 17 23:14:46.013094 ip-10-0-132-188 kubenswrapper[2562]: E0917 23:14:46.013082    2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/68dcc3ba-959d-419d-a77b-31b95d377616-networking-console-plugin-cert podName:68dcc3ba-959d-419d-a77b-31b95d377616 nodeName:}" failed. No retries permitted until 2026-09-17 23:15:02.01306729 +0000 UTC m=+254.414964037 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/68dcc3ba-959d-419d-a77b-31b95d377616-networking-console-plugin-cert") pod "networking-console-plugin-6446c8d75d-9gc6s" (UID: "68dcc3ba-959d-419d-a77b-31b95d377616") : secret "networking-console-plugin-cert" not found
Sep 17 23:14:46.067450 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:14:46.067389    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fwwc5_7e452bbb-7dc6-4c9d-a9b1-76400942f5b4/konnectivity-agent/0.log" limit=1024
Sep 17 23:14:46.271434 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:14:46.271321    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fwwc5_7e452bbb-7dc6-4c9d-a9b1-76400942f5b4/konnectivity-agent/0.log" limit=1024
Sep 17 23:14:46.665163 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:14:46.665107    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fwwc5_7e452bbb-7dc6-4c9d-a9b1-76400942f5b4/konnectivity-agent/0.log" limit=1024
Sep 17 23:14:46.986138 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:14:46.986049    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fwwc5_7e452bbb-7dc6-4c9d-a9b1-76400942f5b4/konnectivity-agent/0.log" limit=1024
Sep 17 23:14:47.267109 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:14:47.267019    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fwwc5_7e452bbb-7dc6-4c9d-a9b1-76400942f5b4/konnectivity-agent/0.log" limit=1024
Sep 17 23:14:47.663491 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:14:47.663446    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fwwc5_7e452bbb-7dc6-4c9d-a9b1-76400942f5b4/konnectivity-agent/0.log" limit=1024
Sep 17 23:14:47.865233 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:14:47.865166    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fwwc5_7e452bbb-7dc6-4c9d-a9b1-76400942f5b4/konnectivity-agent/0.log" limit=1024
Sep 17 23:14:48.267772 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:14:48.267736    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fwwc5_7e452bbb-7dc6-4c9d-a9b1-76400942f5b4/konnectivity-agent/0.log" limit=1024
Sep 17 23:14:48.368478 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:14:48.368447    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fwwc5_7e452bbb-7dc6-4c9d-a9b1-76400942f5b4/konnectivity-agent/0.log" limit=1024
Sep 17 23:14:48.862740 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:14:48.862709    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fwwc5_7e452bbb-7dc6-4c9d-a9b1-76400942f5b4/konnectivity-agent/0.log" limit=1024
Sep 17 23:14:49.264501 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:14:49.264410    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fwwc5_7e452bbb-7dc6-4c9d-a9b1-76400942f5b4/konnectivity-agent/0.log" limit=1024
Sep 17 23:14:49.642308 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:14:49.642265    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/062c04b5-55f2-49ca-aff9-19fe1c0a20b8-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-59djn\" (UID: \"062c04b5-55f2-49ca-aff9-19fe1c0a20b8\") " pod="openshift-insights/insights-runtime-extractor-59djn"
Sep 17 23:14:49.642518 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:14:49.642318    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/062c04b5-55f2-49ca-aff9-19fe1c0a20b8-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-59djn\" (UID: \"062c04b5-55f2-49ca-aff9-19fe1c0a20b8\") " pod="openshift-insights/insights-runtime-extractor-59djn"
Sep 17 23:14:49.642518 ip-10-0-132-188 kubenswrapper[2562]: E0917 23:14:49.642456    2562 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Sep 17 23:14:49.642640 ip-10-0-132-188 kubenswrapper[2562]: E0917 23:14:49.642548    2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/062c04b5-55f2-49ca-aff9-19fe1c0a20b8-insights-runtime-extractor-tls podName:062c04b5-55f2-49ca-aff9-19fe1c0a20b8 nodeName:}" failed. No retries permitted until 2026-09-17 23:15:05.642519218 +0000 UTC m=+258.044415976 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/062c04b5-55f2-49ca-aff9-19fe1c0a20b8-insights-runtime-extractor-tls") pod "insights-runtime-extractor-59djn" (UID: "062c04b5-55f2-49ca-aff9-19fe1c0a20b8") : secret "insights-runtime-extractor-tls" not found
Sep 17 23:14:49.642813 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:14:49.642794    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/062c04b5-55f2-49ca-aff9-19fe1c0a20b8-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-59djn\" (UID: \"062c04b5-55f2-49ca-aff9-19fe1c0a20b8\") " pod="openshift-insights/insights-runtime-extractor-59djn"
Sep 17 23:14:49.666892 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:14:49.666859    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fwwc5_7e452bbb-7dc6-4c9d-a9b1-76400942f5b4/konnectivity-agent/0.log" limit=1024
Sep 17 23:14:49.866479 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:14:49.866447    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fwwc5_7e452bbb-7dc6-4c9d-a9b1-76400942f5b4/konnectivity-agent/0.log" limit=1024
Sep 17 23:14:50.268469 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:14:50.268425    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fwwc5_7e452bbb-7dc6-4c9d-a9b1-76400942f5b4/konnectivity-agent/0.log" limit=1024
Sep 17 23:14:50.845964 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:14:50.845923    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fwwc5_7e452bbb-7dc6-4c9d-a9b1-76400942f5b4/konnectivity-agent/0.log" limit=1024
Sep 17 23:14:50.898066 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:14:50.898027    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fwwc5_7e452bbb-7dc6-4c9d-a9b1-76400942f5b4/konnectivity-agent/0.log" limit=1024
Sep 17 23:14:51.270657 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:14:51.270561    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fwwc5_7e452bbb-7dc6-4c9d-a9b1-76400942f5b4/konnectivity-agent/0.log" limit=1024
Sep 17 23:14:51.465801 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:14:51.465768    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fwwc5_7e452bbb-7dc6-4c9d-a9b1-76400942f5b4/konnectivity-agent/0.log" limit=1024
Sep 17 23:14:51.864446 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:14:51.864404    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fwwc5_7e452bbb-7dc6-4c9d-a9b1-76400942f5b4/konnectivity-agent/0.log" limit=1024
Sep 17 23:14:52.269565 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:14:52.269478    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fwwc5_7e452bbb-7dc6-4c9d-a9b1-76400942f5b4/konnectivity-agent/0.log" limit=1024
Sep 17 23:14:52.466742 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:14:52.466711    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fwwc5_7e452bbb-7dc6-4c9d-a9b1-76400942f5b4/konnectivity-agent/0.log" limit=1024
Sep 17 23:14:59.628043 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:14:59.628008    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fwwc5_7e452bbb-7dc6-4c9d-a9b1-76400942f5b4/konnectivity-agent/0.log" limit=1024
Sep 17 23:14:59.637588 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:14:59.637558    2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-image-registry/image-registry-64c75f9db5-qc6hf"]
Sep 17 23:14:59.639839 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:14:59.639814    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-64c75f9db5-qc6hf"
Sep 17 23:14:59.642496 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:14:59.642468    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-tls\""
Sep 17 23:14:59.642651 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:14:59.642517    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"registry-dockercfg-9dbhz\""
Sep 17 23:14:59.642651 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:14:59.642558    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-private-configuration\""
Sep 17 23:14:59.643875 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:14:59.643858    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"installation-pull-secrets\""
Sep 17 23:14:59.647862 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:14:59.647844    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"trusted-ca\""
Sep 17 23:14:59.654242 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:14:59.654218    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-64c75f9db5-qc6hf"]
Sep 17 23:14:59.719034 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:14:59.719002    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/51d624a1-203a-4888-be49-4a3c375984a3-image-registry-private-configuration\") pod \"image-registry-64c75f9db5-qc6hf\" (UID: \"51d624a1-203a-4888-be49-4a3c375984a3\") " pod="openshift-image-registry/image-registry-64c75f9db5-qc6hf"
Sep 17 23:14:59.719256 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:14:59.719047    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/51d624a1-203a-4888-be49-4a3c375984a3-registry-certificates\") pod \"image-registry-64c75f9db5-qc6hf\" (UID: \"51d624a1-203a-4888-be49-4a3c375984a3\") " pod="openshift-image-registry/image-registry-64c75f9db5-qc6hf"
Sep 17 23:14:59.719256 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:14:59.719103    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/51d624a1-203a-4888-be49-4a3c375984a3-bound-sa-token\") pod \"image-registry-64c75f9db5-qc6hf\" (UID: \"51d624a1-203a-4888-be49-4a3c375984a3\") " pod="openshift-image-registry/image-registry-64c75f9db5-qc6hf"
Sep 17 23:14:59.719256 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:14:59.719141    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ljssz\" (UniqueName: \"kubernetes.io/projected/51d624a1-203a-4888-be49-4a3c375984a3-kube-api-access-ljssz\") pod \"image-registry-64c75f9db5-qc6hf\" (UID: \"51d624a1-203a-4888-be49-4a3c375984a3\") " pod="openshift-image-registry/image-registry-64c75f9db5-qc6hf"
Sep 17 23:14:59.719417 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:14:59.719288    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fwwc5_7e452bbb-7dc6-4c9d-a9b1-76400942f5b4/konnectivity-agent/0.log" limit=1024
Sep 17 23:14:59.719417 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:14:59.719292    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/51d624a1-203a-4888-be49-4a3c375984a3-trusted-ca\") pod \"image-registry-64c75f9db5-qc6hf\" (UID: \"51d624a1-203a-4888-be49-4a3c375984a3\") " pod="openshift-image-registry/image-registry-64c75f9db5-qc6hf"
Sep 17 23:14:59.719489 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:14:59.719424    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/51d624a1-203a-4888-be49-4a3c375984a3-ca-trust-extracted\") pod \"image-registry-64c75f9db5-qc6hf\" (UID: \"51d624a1-203a-4888-be49-4a3c375984a3\") " pod="openshift-image-registry/image-registry-64c75f9db5-qc6hf"
Sep 17 23:14:59.719489 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:14:59.719454    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/51d624a1-203a-4888-be49-4a3c375984a3-registry-tls\") pod \"image-registry-64c75f9db5-qc6hf\" (UID: \"51d624a1-203a-4888-be49-4a3c375984a3\") " pod="openshift-image-registry/image-registry-64c75f9db5-qc6hf"
Sep 17 23:14:59.719489 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:14:59.719475    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/51d624a1-203a-4888-be49-4a3c375984a3-installation-pull-secrets\") pod \"image-registry-64c75f9db5-qc6hf\" (UID: \"51d624a1-203a-4888-be49-4a3c375984a3\") " pod="openshift-image-registry/image-registry-64c75f9db5-qc6hf"
Sep 17 23:14:59.820155 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:14:59.820115    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/51d624a1-203a-4888-be49-4a3c375984a3-registry-tls\") pod \"image-registry-64c75f9db5-qc6hf\" (UID: \"51d624a1-203a-4888-be49-4a3c375984a3\") " pod="openshift-image-registry/image-registry-64c75f9db5-qc6hf"
Sep 17 23:14:59.820155 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:14:59.820153    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/51d624a1-203a-4888-be49-4a3c375984a3-installation-pull-secrets\") pod \"image-registry-64c75f9db5-qc6hf\" (UID: \"51d624a1-203a-4888-be49-4a3c375984a3\") " pod="openshift-image-registry/image-registry-64c75f9db5-qc6hf"
Sep 17 23:14:59.820391 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:14:59.820204    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/51d624a1-203a-4888-be49-4a3c375984a3-image-registry-private-configuration\") pod \"image-registry-64c75f9db5-qc6hf\" (UID: \"51d624a1-203a-4888-be49-4a3c375984a3\") " pod="openshift-image-registry/image-registry-64c75f9db5-qc6hf"
Sep 17 23:14:59.820391 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:14:59.820232    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/51d624a1-203a-4888-be49-4a3c375984a3-registry-certificates\") pod \"image-registry-64c75f9db5-qc6hf\" (UID: \"51d624a1-203a-4888-be49-4a3c375984a3\") " pod="openshift-image-registry/image-registry-64c75f9db5-qc6hf"
Sep 17 23:14:59.820391 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:14:59.820254    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/51d624a1-203a-4888-be49-4a3c375984a3-bound-sa-token\") pod \"image-registry-64c75f9db5-qc6hf\" (UID: \"51d624a1-203a-4888-be49-4a3c375984a3\") " pod="openshift-image-registry/image-registry-64c75f9db5-qc6hf"
Sep 17 23:14:59.820391 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:14:59.820279    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ljssz\" (UniqueName: \"kubernetes.io/projected/51d624a1-203a-4888-be49-4a3c375984a3-kube-api-access-ljssz\") pod \"image-registry-64c75f9db5-qc6hf\" (UID: \"51d624a1-203a-4888-be49-4a3c375984a3\") " pod="openshift-image-registry/image-registry-64c75f9db5-qc6hf"
Sep 17 23:14:59.820575 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:14:59.820518    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/51d624a1-203a-4888-be49-4a3c375984a3-trusted-ca\") pod \"image-registry-64c75f9db5-qc6hf\" (UID: \"51d624a1-203a-4888-be49-4a3c375984a3\") " pod="openshift-image-registry/image-registry-64c75f9db5-qc6hf"
Sep 17 23:14:59.820632 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:14:59.820600    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/51d624a1-203a-4888-be49-4a3c375984a3-ca-trust-extracted\") pod \"image-registry-64c75f9db5-qc6hf\" (UID: \"51d624a1-203a-4888-be49-4a3c375984a3\") " pod="openshift-image-registry/image-registry-64c75f9db5-qc6hf"
Sep 17 23:14:59.821015 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:14:59.820946    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/51d624a1-203a-4888-be49-4a3c375984a3-ca-trust-extracted\") pod \"image-registry-64c75f9db5-qc6hf\" (UID: \"51d624a1-203a-4888-be49-4a3c375984a3\") " pod="openshift-image-registry/image-registry-64c75f9db5-qc6hf"
Sep 17 23:14:59.821248 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:14:59.821225    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/51d624a1-203a-4888-be49-4a3c375984a3-registry-certificates\") pod \"image-registry-64c75f9db5-qc6hf\" (UID: \"51d624a1-203a-4888-be49-4a3c375984a3\") " pod="openshift-image-registry/image-registry-64c75f9db5-qc6hf"
Sep 17 23:14:59.821676 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:14:59.821653    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/51d624a1-203a-4888-be49-4a3c375984a3-trusted-ca\") pod \"image-registry-64c75f9db5-qc6hf\" (UID: \"51d624a1-203a-4888-be49-4a3c375984a3\") " pod="openshift-image-registry/image-registry-64c75f9db5-qc6hf"
Sep 17 23:14:59.822748 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:14:59.822725    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/51d624a1-203a-4888-be49-4a3c375984a3-image-registry-private-configuration\") pod \"image-registry-64c75f9db5-qc6hf\" (UID: \"51d624a1-203a-4888-be49-4a3c375984a3\") " pod="openshift-image-registry/image-registry-64c75f9db5-qc6hf"
Sep 17 23:14:59.822856 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:14:59.822794    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/51d624a1-203a-4888-be49-4a3c375984a3-installation-pull-secrets\") pod \"image-registry-64c75f9db5-qc6hf\" (UID: \"51d624a1-203a-4888-be49-4a3c375984a3\") " pod="openshift-image-registry/image-registry-64c75f9db5-qc6hf"
Sep 17 23:14:59.822901 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:14:59.822885    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/51d624a1-203a-4888-be49-4a3c375984a3-registry-tls\") pod \"image-registry-64c75f9db5-qc6hf\" (UID: \"51d624a1-203a-4888-be49-4a3c375984a3\") " pod="openshift-image-registry/image-registry-64c75f9db5-qc6hf"
Sep 17 23:14:59.827572 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:14:59.827551    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/51d624a1-203a-4888-be49-4a3c375984a3-bound-sa-token\") pod \"image-registry-64c75f9db5-qc6hf\" (UID: \"51d624a1-203a-4888-be49-4a3c375984a3\") " pod="openshift-image-registry/image-registry-64c75f9db5-qc6hf"
Sep 17 23:14:59.827786 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:14:59.827760    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-ljssz\" (UniqueName: \"kubernetes.io/projected/51d624a1-203a-4888-be49-4a3c375984a3-kube-api-access-ljssz\") pod \"image-registry-64c75f9db5-qc6hf\" (UID: \"51d624a1-203a-4888-be49-4a3c375984a3\") " pod="openshift-image-registry/image-registry-64c75f9db5-qc6hf"
Sep 17 23:14:59.834229 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:14:59.834199    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fwwc5_7e452bbb-7dc6-4c9d-a9b1-76400942f5b4/konnectivity-agent/0.log" limit=1024
Sep 17 23:14:59.949366 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:14:59.949279    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-64c75f9db5-qc6hf"
Sep 17 23:14:59.980330 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:14:59.980296    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fwwc5_7e452bbb-7dc6-4c9d-a9b1-76400942f5b4/konnectivity-agent/0.log" limit=1024
Sep 17 23:15:00.080668 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:15:00.080632    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fwwc5_7e452bbb-7dc6-4c9d-a9b1-76400942f5b4/konnectivity-agent/0.log" limit=1024
Sep 17 23:15:00.222722 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:15:00.222644    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-64c75f9db5-qc6hf"]
Sep 17 23:15:00.226126 ip-10-0-132-188 kubenswrapper[2562]: W0917 23:15:00.226099    2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod51d624a1_203a_4888_be49_4a3c375984a3.slice/crio-7b9c713c2268c6a9291badcae1a377b8419d281c4049c3ba9582a6fed7eca50b WatchSource:0}: Error finding container 7b9c713c2268c6a9291badcae1a377b8419d281c4049c3ba9582a6fed7eca50b: Status 404 returned error can't find the container with id 7b9c713c2268c6a9291badcae1a377b8419d281c4049c3ba9582a6fed7eca50b
Sep 17 23:15:00.423912 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:15:00.423878    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/4453ba96-a67f-46af-b61b-9cd8f39b6d36-metrics-certs\") pod \"network-metrics-daemon-nbccd\" (UID: \"4453ba96-a67f-46af-b61b-9cd8f39b6d36\") " pod="openshift-multus/network-metrics-daemon-nbccd"
Sep 17 23:15:00.426198 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:15:00.426155    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/4453ba96-a67f-46af-b61b-9cd8f39b6d36-metrics-certs\") pod \"network-metrics-daemon-nbccd\" (UID: \"4453ba96-a67f-46af-b61b-9cd8f39b6d36\") " pod="openshift-multus/network-metrics-daemon-nbccd"
Sep 17 23:15:00.565548 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:15:00.565466    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-mx9lw\""
Sep 17 23:15:00.572811 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:15:00.572790    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-nbccd"
Sep 17 23:15:00.576976 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:15:00.576940    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-64c75f9db5-qc6hf" event={"ID":"51d624a1-203a-4888-be49-4a3c375984a3","Type":"ContainerStarted","Data":"1b840b07dde15d8d80599f875486b5027a34d845148bba04d761834e98eee6dd"}
Sep 17 23:15:00.577074 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:15:00.577034    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-64c75f9db5-qc6hf" event={"ID":"51d624a1-203a-4888-be49-4a3c375984a3","Type":"ContainerStarted","Data":"7b9c713c2268c6a9291badcae1a377b8419d281c4049c3ba9582a6fed7eca50b"}
Sep 17 23:15:00.577127 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:15:00.577094    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-image-registry/image-registry-64c75f9db5-qc6hf"
Sep 17 23:15:00.596044 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:15:00.595997    2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/image-registry-64c75f9db5-qc6hf" podStartSLOduration=1.5959795639999999 podStartE2EDuration="1.595979564s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-17 23:14:59 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-17 23:15:00.595131442 +0000 UTC m=+252.997028208" watchObservedRunningTime="2026-09-17 23:15:00.595979564 +0000 UTC m=+252.997876327"
Sep 17 23:15:00.847022 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:15:00.846989    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-nbccd"]
Sep 17 23:15:00.850638 ip-10-0-132-188 kubenswrapper[2562]: W0917 23:15:00.850610    2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4453ba96_a67f_46af_b61b_9cd8f39b6d36.slice/crio-1858a46faed9456a0071de30d7bca67b98719ec59254d0c996946d19fcb950f5 WatchSource:0}: Error finding container 1858a46faed9456a0071de30d7bca67b98719ec59254d0c996946d19fcb950f5: Status 404 returned error can't find the container with id 1858a46faed9456a0071de30d7bca67b98719ec59254d0c996946d19fcb950f5
Sep 17 23:15:01.581989 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:15:01.581943    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-nbccd" event={"ID":"4453ba96-a67f-46af-b61b-9cd8f39b6d36","Type":"ContainerStarted","Data":"1858a46faed9456a0071de30d7bca67b98719ec59254d0c996946d19fcb950f5"}
Sep 17 23:15:02.036702 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:15:02.036664    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/68dcc3ba-959d-419d-a77b-31b95d377616-networking-console-plugin-cert\") pod \"networking-console-plugin-6446c8d75d-9gc6s\" (UID: \"68dcc3ba-959d-419d-a77b-31b95d377616\") " pod="openshift-network-console/networking-console-plugin-6446c8d75d-9gc6s"
Sep 17 23:15:02.039575 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:15:02.039543    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/68dcc3ba-959d-419d-a77b-31b95d377616-networking-console-plugin-cert\") pod \"networking-console-plugin-6446c8d75d-9gc6s\" (UID: \"68dcc3ba-959d-419d-a77b-31b95d377616\") " pod="openshift-network-console/networking-console-plugin-6446c8d75d-9gc6s"
Sep 17 23:15:02.292300 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:15:02.292210    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-console\"/\"default-dockercfg-l459d\""
Sep 17 23:15:02.300197 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:15:02.300159    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-console/networking-console-plugin-6446c8d75d-9gc6s"
Sep 17 23:15:02.359406 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:15:02.359366    2562 prober.go:120] "Probe failed" probeType="Liveness" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-64d77d549c-dndvm" podUID="204204da-e53d-4c6b-93ed-a6271c85ac27" containerName="service-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 500"
Sep 17 23:15:02.565363 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:15:02.565284    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-console/networking-console-plugin-6446c8d75d-9gc6s"]
Sep 17 23:15:02.567902 ip-10-0-132-188 kubenswrapper[2562]: W0917 23:15:02.567877    2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod68dcc3ba_959d_419d_a77b_31b95d377616.slice/crio-1ec346c91ab0c978c48810e6738cacc735e2402d427a4ea3e2f788fd7a5143b1 WatchSource:0}: Error finding container 1ec346c91ab0c978c48810e6738cacc735e2402d427a4ea3e2f788fd7a5143b1: Status 404 returned error can't find the container with id 1ec346c91ab0c978c48810e6738cacc735e2402d427a4ea3e2f788fd7a5143b1
Sep 17 23:15:02.586055 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:15:02.586025    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-nbccd" event={"ID":"4453ba96-a67f-46af-b61b-9cd8f39b6d36","Type":"ContainerStarted","Data":"80ea0ceb4f9393b5ea2e0f92a24b4cdd6593a51121829b6a791d8db09af6f9bf"}
Sep 17 23:15:02.586198 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:15:02.586083    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-nbccd" event={"ID":"4453ba96-a67f-46af-b61b-9cd8f39b6d36","Type":"ContainerStarted","Data":"4da3f598778a5aa9e19e8fe9d3dcdeb54cc14112718b9be881aab44bbe972fb3"}
Sep 17 23:15:02.587213 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:15:02.587189    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-console/networking-console-plugin-6446c8d75d-9gc6s" event={"ID":"68dcc3ba-959d-419d-a77b-31b95d377616","Type":"ContainerStarted","Data":"1ec346c91ab0c978c48810e6738cacc735e2402d427a4ea3e2f788fd7a5143b1"}
Sep 17 23:15:02.600670 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:15:02.600617    2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/network-metrics-daemon-nbccd" podStartSLOduration=252.590072785 podStartE2EDuration="4m13.600597398s" totalImagesPullingTime="1.010524613s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-17 23:10:49 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-17 23:15:02.600329672 +0000 UTC m=+255.002226437" watchObservedRunningTime="2026-09-17 23:15:02.600597398 +0000 UTC m=+255.002494159"
Sep 17 23:15:02.744870 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:15:02.744841    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fwwc5_7e452bbb-7dc6-4c9d-a9b1-76400942f5b4/konnectivity-agent/0.log" limit=1024
Sep 17 23:15:02.846195 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:15:02.846142    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fwwc5_7e452bbb-7dc6-4c9d-a9b1-76400942f5b4/konnectivity-agent/0.log" limit=1024
Sep 17 23:15:03.662861 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:15:03.662827    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fwwc5_7e452bbb-7dc6-4c9d-a9b1-76400942f5b4/konnectivity-agent/0.log" limit=1024
Sep 17 23:15:04.599874 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:15:04.599836    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-console/networking-console-plugin-6446c8d75d-9gc6s" event={"ID":"68dcc3ba-959d-419d-a77b-31b95d377616","Type":"ContainerStarted","Data":"e6aa5dbd6480ec63a94d946d5966412ffc516ad03177b2d263cb8f9f4eb5df17"}
Sep 17 23:15:04.614795 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:15:04.614739    2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-console/networking-console-plugin-6446c8d75d-9gc6s" podStartSLOduration=33.543828427 podStartE2EDuration="34.614725028s" totalImagesPullingTime="1.070896601s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-17 23:14:30 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-17 23:15:04.613284686 +0000 UTC m=+257.015181476" watchObservedRunningTime="2026-09-17 23:15:04.614725028 +0000 UTC m=+257.016621792"
Sep 17 23:15:04.717688 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:15:04.717652    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fwwc5_7e452bbb-7dc6-4c9d-a9b1-76400942f5b4/konnectivity-agent/0.log" limit=1024
Sep 17 23:15:04.817782 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:15:04.817749    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fwwc5_7e452bbb-7dc6-4c9d-a9b1-76400942f5b4/konnectivity-agent/0.log" limit=1024
Sep 17 23:15:04.889618 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:15:04.889513    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fwwc5_7e452bbb-7dc6-4c9d-a9b1-76400942f5b4/konnectivity-agent/0.log" limit=1024
Sep 17 23:15:05.078963 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:15:05.078933    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fwwc5_7e452bbb-7dc6-4c9d-a9b1-76400942f5b4/konnectivity-agent/0.log" limit=1024
Sep 17 23:15:05.660348 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:15:05.660293    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/062c04b5-55f2-49ca-aff9-19fe1c0a20b8-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-59djn\" (UID: \"062c04b5-55f2-49ca-aff9-19fe1c0a20b8\") " pod="openshift-insights/insights-runtime-extractor-59djn"
Sep 17 23:15:05.662804 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:15:05.662766    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/062c04b5-55f2-49ca-aff9-19fe1c0a20b8-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-59djn\" (UID: \"062c04b5-55f2-49ca-aff9-19fe1c0a20b8\") " pod="openshift-insights/insights-runtime-extractor-59djn"
Sep 17 23:15:05.878292 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:15:05.878262    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-sa-dockercfg-9v429\""
Sep 17 23:15:05.886441 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:15:05.886420    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-59djn"
Sep 17 23:15:06.158869 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:15:06.158830    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-59djn"]
Sep 17 23:15:06.162130 ip-10-0-132-188 kubenswrapper[2562]: W0917 23:15:06.162104    2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod062c04b5_55f2_49ca_aff9_19fe1c0a20b8.slice/crio-100e9708db23944a2386f854cc6bef667a77e194041e75e56a4e5362edecbec5 WatchSource:0}: Error finding container 100e9708db23944a2386f854cc6bef667a77e194041e75e56a4e5362edecbec5: Status 404 returned error can't find the container with id 100e9708db23944a2386f854cc6bef667a77e194041e75e56a4e5362edecbec5
Sep 17 23:15:06.607194 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:15:06.607153    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-59djn" event={"ID":"062c04b5-55f2-49ca-aff9-19fe1c0a20b8","Type":"ContainerStarted","Data":"e1cae5b163c6076acf8d6cc7bc4bd406cb6c0cce29d407df82ff29f7e426aa3f"}
Sep 17 23:15:06.607337 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:15:06.607218    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-59djn" event={"ID":"062c04b5-55f2-49ca-aff9-19fe1c0a20b8","Type":"ContainerStarted","Data":"100e9708db23944a2386f854cc6bef667a77e194041e75e56a4e5362edecbec5"}
Sep 17 23:15:07.613771 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:15:07.613729    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-59djn" event={"ID":"062c04b5-55f2-49ca-aff9-19fe1c0a20b8","Type":"ContainerStarted","Data":"27a3af98dc9eece9d5a35c4a2d805d22cec9eba16fe01acd9f27122315202fcf"}
Sep 17 23:15:07.918130 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:15:07.918025    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fwwc5_7e452bbb-7dc6-4c9d-a9b1-76400942f5b4/konnectivity-agent/0.log" limit=1024
Sep 17 23:15:07.955394 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:15:07.955366    2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/prometheus-operator-7bd5d56d6-x6qkt"]
Sep 17 23:15:07.957613 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:15:07.957591    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-operator-7bd5d56d6-x6qkt"
Sep 17 23:15:07.960091 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:15:07.960063    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-tls\""
Sep 17 23:15:07.961041 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:15:07.960986    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kube-root-ca.crt\""
Sep 17 23:15:07.961041 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:15:07.960998    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"openshift-service-ca.crt\""
Sep 17 23:15:07.961041 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:15:07.961008    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"metrics-client-ca\""
Sep 17 23:15:07.961292 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:15:07.961076    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-kube-rbac-proxy-config\""
Sep 17 23:15:07.961292 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:15:07.961095    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-dockercfg-wm2ff\""
Sep 17 23:15:07.967236 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:15:07.967212    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-operator-7bd5d56d6-x6qkt"]
Sep 17 23:15:07.975924 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:15:07.975876    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/0d951367-db80-4aee-9177-dd51602c88ce-metrics-client-ca\") pod \"prometheus-operator-7bd5d56d6-x6qkt\" (UID: \"0d951367-db80-4aee-9177-dd51602c88ce\") " pod="openshift-monitoring/prometheus-operator-7bd5d56d6-x6qkt"
Sep 17 23:15:07.976035 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:15:07.975933    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-z8prh\" (UniqueName: \"kubernetes.io/projected/0d951367-db80-4aee-9177-dd51602c88ce-kube-api-access-z8prh\") pod \"prometheus-operator-7bd5d56d6-x6qkt\" (UID: \"0d951367-db80-4aee-9177-dd51602c88ce\") " pod="openshift-monitoring/prometheus-operator-7bd5d56d6-x6qkt"
Sep 17 23:15:07.976035 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:15:07.975995    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-operator-tls\" (UniqueName: \"kubernetes.io/secret/0d951367-db80-4aee-9177-dd51602c88ce-prometheus-operator-tls\") pod \"prometheus-operator-7bd5d56d6-x6qkt\" (UID: \"0d951367-db80-4aee-9177-dd51602c88ce\") " pod="openshift-monitoring/prometheus-operator-7bd5d56d6-x6qkt"
Sep 17 23:15:07.976159 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:15:07.976109    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-operator-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/0d951367-db80-4aee-9177-dd51602c88ce-prometheus-operator-kube-rbac-proxy-config\") pod \"prometheus-operator-7bd5d56d6-x6qkt\" (UID: \"0d951367-db80-4aee-9177-dd51602c88ce\") " pod="openshift-monitoring/prometheus-operator-7bd5d56d6-x6qkt"
Sep 17 23:15:07.989935 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:15:07.989896    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fwwc5_7e452bbb-7dc6-4c9d-a9b1-76400942f5b4/konnectivity-agent/0.log" limit=1024
Sep 17 23:15:08.077065 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:15:08.077036    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-operator-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/0d951367-db80-4aee-9177-dd51602c88ce-prometheus-operator-kube-rbac-proxy-config\") pod \"prometheus-operator-7bd5d56d6-x6qkt\" (UID: \"0d951367-db80-4aee-9177-dd51602c88ce\") " pod="openshift-monitoring/prometheus-operator-7bd5d56d6-x6qkt"
Sep 17 23:15:08.077185 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:15:08.077087    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/0d951367-db80-4aee-9177-dd51602c88ce-metrics-client-ca\") pod \"prometheus-operator-7bd5d56d6-x6qkt\" (UID: \"0d951367-db80-4aee-9177-dd51602c88ce\") " pod="openshift-monitoring/prometheus-operator-7bd5d56d6-x6qkt"
Sep 17 23:15:08.077185 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:15:08.077124    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-z8prh\" (UniqueName: \"kubernetes.io/projected/0d951367-db80-4aee-9177-dd51602c88ce-kube-api-access-z8prh\") pod \"prometheus-operator-7bd5d56d6-x6qkt\" (UID: \"0d951367-db80-4aee-9177-dd51602c88ce\") " pod="openshift-monitoring/prometheus-operator-7bd5d56d6-x6qkt"
Sep 17 23:15:08.077311 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:15:08.077199    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-operator-tls\" (UniqueName: \"kubernetes.io/secret/0d951367-db80-4aee-9177-dd51602c88ce-prometheus-operator-tls\") pod \"prometheus-operator-7bd5d56d6-x6qkt\" (UID: \"0d951367-db80-4aee-9177-dd51602c88ce\") " pod="openshift-monitoring/prometheus-operator-7bd5d56d6-x6qkt"
Sep 17 23:15:08.077371 ip-10-0-132-188 kubenswrapper[2562]: E0917 23:15:08.077325    2562 secret.go:189] Couldn't get secret openshift-monitoring/prometheus-operator-tls: secret "prometheus-operator-tls" not found
Sep 17 23:15:08.077422 ip-10-0-132-188 kubenswrapper[2562]: E0917 23:15:08.077382    2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/0d951367-db80-4aee-9177-dd51602c88ce-prometheus-operator-tls podName:0d951367-db80-4aee-9177-dd51602c88ce nodeName:}" failed. No retries permitted until 2026-09-17 23:15:08.577363334 +0000 UTC m=+260.979260100 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "prometheus-operator-tls" (UniqueName: "kubernetes.io/secret/0d951367-db80-4aee-9177-dd51602c88ce-prometheus-operator-tls") pod "prometheus-operator-7bd5d56d6-x6qkt" (UID: "0d951367-db80-4aee-9177-dd51602c88ce") : secret "prometheus-operator-tls" not found
Sep 17 23:15:08.077969 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:15:08.077938    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/0d951367-db80-4aee-9177-dd51602c88ce-metrics-client-ca\") pod \"prometheus-operator-7bd5d56d6-x6qkt\" (UID: \"0d951367-db80-4aee-9177-dd51602c88ce\") " pod="openshift-monitoring/prometheus-operator-7bd5d56d6-x6qkt"
Sep 17 23:15:08.079585 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:15:08.079565    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-operator-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/0d951367-db80-4aee-9177-dd51602c88ce-prometheus-operator-kube-rbac-proxy-config\") pod \"prometheus-operator-7bd5d56d6-x6qkt\" (UID: \"0d951367-db80-4aee-9177-dd51602c88ce\") " pod="openshift-monitoring/prometheus-operator-7bd5d56d6-x6qkt"
Sep 17 23:15:08.085671 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:15:08.085650    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-z8prh\" (UniqueName: \"kubernetes.io/projected/0d951367-db80-4aee-9177-dd51602c88ce-kube-api-access-z8prh\") pod \"prometheus-operator-7bd5d56d6-x6qkt\" (UID: \"0d951367-db80-4aee-9177-dd51602c88ce\") " pod="openshift-monitoring/prometheus-operator-7bd5d56d6-x6qkt"
Sep 17 23:15:08.218862 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:15:08.218772    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fwwc5_7e452bbb-7dc6-4c9d-a9b1-76400942f5b4/konnectivity-agent/0.log" limit=1024
Sep 17 23:15:08.580423 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:15:08.580389    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-operator-tls\" (UniqueName: \"kubernetes.io/secret/0d951367-db80-4aee-9177-dd51602c88ce-prometheus-operator-tls\") pod \"prometheus-operator-7bd5d56d6-x6qkt\" (UID: \"0d951367-db80-4aee-9177-dd51602c88ce\") " pod="openshift-monitoring/prometheus-operator-7bd5d56d6-x6qkt"
Sep 17 23:15:08.582805 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:15:08.582785    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-operator-tls\" (UniqueName: \"kubernetes.io/secret/0d951367-db80-4aee-9177-dd51602c88ce-prometheus-operator-tls\") pod \"prometheus-operator-7bd5d56d6-x6qkt\" (UID: \"0d951367-db80-4aee-9177-dd51602c88ce\") " pod="openshift-monitoring/prometheus-operator-7bd5d56d6-x6qkt"
Sep 17 23:15:08.618969 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:15:08.618934    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-59djn" event={"ID":"062c04b5-55f2-49ca-aff9-19fe1c0a20b8","Type":"ContainerStarted","Data":"1fb30257bd959da358223186be1ac406d0e94c02612a696a2a1189e8a312a046"}
Sep 17 23:15:08.637611 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:15:08.637564    2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-insights/insights-runtime-extractor-59djn" podStartSLOduration=33.844758649 podStartE2EDuration="35.637548744s" totalImagesPullingTime="1.792790095s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-17 23:14:33 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-17 23:15:08.637306126 +0000 UTC m=+261.039202890" watchObservedRunningTime="2026-09-17 23:15:08.637548744 +0000 UTC m=+261.039445509"
Sep 17 23:15:08.868513 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:15:08.868441    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-operator-7bd5d56d6-x6qkt"
Sep 17 23:15:09.136922 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:15:09.136851    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-operator-7bd5d56d6-x6qkt"]
Sep 17 23:15:09.140150 ip-10-0-132-188 kubenswrapper[2562]: W0917 23:15:09.140121    2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0d951367_db80_4aee_9177_dd51602c88ce.slice/crio-78b40f1050eb48456fa89ced050c00dda0c3cd806ebd62e9afa454dab8243c7d WatchSource:0}: Error finding container 78b40f1050eb48456fa89ced050c00dda0c3cd806ebd62e9afa454dab8243c7d: Status 404 returned error can't find the container with id 78b40f1050eb48456fa89ced050c00dda0c3cd806ebd62e9afa454dab8243c7d
Sep 17 23:15:09.623779 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:15:09.623741    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-7bd5d56d6-x6qkt" event={"ID":"0d951367-db80-4aee-9177-dd51602c88ce","Type":"ContainerStarted","Data":"78b40f1050eb48456fa89ced050c00dda0c3cd806ebd62e9afa454dab8243c7d"}
Sep 17 23:15:10.628932 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:15:10.628898    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-7bd5d56d6-x6qkt" event={"ID":"0d951367-db80-4aee-9177-dd51602c88ce","Type":"ContainerStarted","Data":"e7e73a573ed89eb2f585da25eed782aad60ad8648be719140a52d6d1de9a1d02"}
Sep 17 23:15:10.629408 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:15:10.628955    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-7bd5d56d6-x6qkt" event={"ID":"0d951367-db80-4aee-9177-dd51602c88ce","Type":"ContainerStarted","Data":"f2eb72ab148c5441427237c8483c2bd61ca31922732284e69ba2f44b2090086b"}
Sep 17 23:15:10.645920 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:15:10.645870    2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/prometheus-operator-7bd5d56d6-x6qkt" podStartSLOduration=2.329903255 podStartE2EDuration="3.645852965s" totalImagesPullingTime="1.31594971s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-17 23:15:07 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-17 23:15:10.643860017 +0000 UTC m=+263.045756785" watchObservedRunningTime="2026-09-17 23:15:10.645852965 +0000 UTC m=+263.047749753"
Sep 17 23:15:10.700675 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:15:10.700644    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fwwc5_7e452bbb-7dc6-4c9d-a9b1-76400942f5b4/konnectivity-agent/0.log" limit=1024
Sep 17 23:15:12.053276 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:15:12.053243    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fwwc5_7e452bbb-7dc6-4c9d-a9b1-76400942f5b4/konnectivity-agent/0.log" limit=1024
Sep 17 23:15:12.280858 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:15:12.280824    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fwwc5_7e452bbb-7dc6-4c9d-a9b1-76400942f5b4/konnectivity-agent/0.log" limit=1024
Sep 17 23:15:12.287649 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:15:12.287609    2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/openshift-state-metrics-86c5bc79d6-bwj6t"]
Sep 17 23:15:12.289858 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:15:12.289839    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/openshift-state-metrics-86c5bc79d6-bwj6t"
Sep 17 23:15:12.292062 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:15:12.292034    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"openshift-state-metrics-dockercfg-rs4n2\""
Sep 17 23:15:12.292229 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:15:12.292094    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"openshift-state-metrics-kube-rbac-proxy-config\""
Sep 17 23:15:12.292229 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:15:12.292034    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"openshift-state-metrics-tls\""
Sep 17 23:15:12.301265 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:15:12.301227    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/openshift-state-metrics-86c5bc79d6-bwj6t"]
Sep 17 23:15:12.307459 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:15:12.307382    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rw577\" (UniqueName: \"kubernetes.io/projected/c5c362bf-e3b6-41f3-85c6-4a7d2a3e4b87-kube-api-access-rw577\") pod \"openshift-state-metrics-86c5bc79d6-bwj6t\" (UID: \"c5c362bf-e3b6-41f3-85c6-4a7d2a3e4b87\") " pod="openshift-monitoring/openshift-state-metrics-86c5bc79d6-bwj6t"
Sep 17 23:15:12.307459 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:15:12.307428    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/c5c362bf-e3b6-41f3-85c6-4a7d2a3e4b87-openshift-state-metrics-tls\") pod \"openshift-state-metrics-86c5bc79d6-bwj6t\" (UID: \"c5c362bf-e3b6-41f3-85c6-4a7d2a3e4b87\") " pod="openshift-monitoring/openshift-state-metrics-86c5bc79d6-bwj6t"
Sep 17 23:15:12.307680 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:15:12.307513    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/c5c362bf-e3b6-41f3-85c6-4a7d2a3e4b87-openshift-state-metrics-kube-rbac-proxy-config\") pod \"openshift-state-metrics-86c5bc79d6-bwj6t\" (UID: \"c5c362bf-e3b6-41f3-85c6-4a7d2a3e4b87\") " pod="openshift-monitoring/openshift-state-metrics-86c5bc79d6-bwj6t"
Sep 17 23:15:12.307680 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:15:12.307553    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/c5c362bf-e3b6-41f3-85c6-4a7d2a3e4b87-metrics-client-ca\") pod \"openshift-state-metrics-86c5bc79d6-bwj6t\" (UID: \"c5c362bf-e3b6-41f3-85c6-4a7d2a3e4b87\") " pod="openshift-monitoring/openshift-state-metrics-86c5bc79d6-bwj6t"
Sep 17 23:15:12.324161 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:15:12.324128    2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/node-exporter-dllnm"]
Sep 17 23:15:12.326154 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:15:12.326137    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-dllnm"
Sep 17 23:15:12.328542 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:15:12.328507    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-kube-rbac-proxy-config\""
Sep 17 23:15:12.328542 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:15:12.328508    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-tls\""
Sep 17 23:15:12.328750 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:15:12.328547    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"node-exporter-accelerators-collector-config\""
Sep 17 23:15:12.328750 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:15:12.328518    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-dockercfg-bbsjl\""
Sep 17 23:15:12.358186 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:15:12.358135    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fwwc5_7e452bbb-7dc6-4c9d-a9b1-76400942f5b4/konnectivity-agent/0.log" limit=1024
Sep 17 23:15:12.359059 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:15:12.359032    2562 prober.go:120] "Probe failed" probeType="Liveness" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-64d77d549c-dndvm" podUID="204204da-e53d-4c6b-93ed-a6271c85ac27" containerName="service-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 500"
Sep 17 23:15:12.408146 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:15:12.408111    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/34ae65de-7218-4b53-9a3e-b223c5338b11-node-exporter-tls\") pod \"node-exporter-dllnm\" (UID: \"34ae65de-7218-4b53-9a3e-b223c5338b11\") " pod="openshift-monitoring/node-exporter-dllnm"
Sep 17 23:15:12.408146 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:15:12.408148    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/c5c362bf-e3b6-41f3-85c6-4a7d2a3e4b87-openshift-state-metrics-kube-rbac-proxy-config\") pod \"openshift-state-metrics-86c5bc79d6-bwj6t\" (UID: \"c5c362bf-e3b6-41f3-85c6-4a7d2a3e4b87\") " pod="openshift-monitoring/openshift-state-metrics-86c5bc79d6-bwj6t"
Sep 17 23:15:12.408379 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:15:12.408188    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/34ae65de-7218-4b53-9a3e-b223c5338b11-node-exporter-accelerators-collector-config\") pod \"node-exporter-dllnm\" (UID: \"34ae65de-7218-4b53-9a3e-b223c5338b11\") " pod="openshift-monitoring/node-exporter-dllnm"
Sep 17 23:15:12.408379 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:15:12.408273    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/c5c362bf-e3b6-41f3-85c6-4a7d2a3e4b87-metrics-client-ca\") pod \"openshift-state-metrics-86c5bc79d6-bwj6t\" (UID: \"c5c362bf-e3b6-41f3-85c6-4a7d2a3e4b87\") " pod="openshift-monitoring/openshift-state-metrics-86c5bc79d6-bwj6t"
Sep 17 23:15:12.408379 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:15:12.408331    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/34ae65de-7218-4b53-9a3e-b223c5338b11-root\") pod \"node-exporter-dllnm\" (UID: \"34ae65de-7218-4b53-9a3e-b223c5338b11\") " pod="openshift-monitoring/node-exporter-dllnm"
Sep 17 23:15:12.408379 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:15:12.408360    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rw577\" (UniqueName: \"kubernetes.io/projected/c5c362bf-e3b6-41f3-85c6-4a7d2a3e4b87-kube-api-access-rw577\") pod \"openshift-state-metrics-86c5bc79d6-bwj6t\" (UID: \"c5c362bf-e3b6-41f3-85c6-4a7d2a3e4b87\") " pod="openshift-monitoring/openshift-state-metrics-86c5bc79d6-bwj6t"
Sep 17 23:15:12.408566 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:15:12.408388    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/34ae65de-7218-4b53-9a3e-b223c5338b11-sys\") pod \"node-exporter-dllnm\" (UID: \"34ae65de-7218-4b53-9a3e-b223c5338b11\") " pod="openshift-monitoring/node-exporter-dllnm"
Sep 17 23:15:12.408566 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:15:12.408411    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/34ae65de-7218-4b53-9a3e-b223c5338b11-node-exporter-wtmp\") pod \"node-exporter-dllnm\" (UID: \"34ae65de-7218-4b53-9a3e-b223c5338b11\") " pod="openshift-monitoring/node-exporter-dllnm"
Sep 17 23:15:12.408566 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:15:12.408502    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/c5c362bf-e3b6-41f3-85c6-4a7d2a3e4b87-openshift-state-metrics-tls\") pod \"openshift-state-metrics-86c5bc79d6-bwj6t\" (UID: \"c5c362bf-e3b6-41f3-85c6-4a7d2a3e4b87\") " pod="openshift-monitoring/openshift-state-metrics-86c5bc79d6-bwj6t"
Sep 17 23:15:12.408717 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:15:12.408565    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/34ae65de-7218-4b53-9a3e-b223c5338b11-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-dllnm\" (UID: \"34ae65de-7218-4b53-9a3e-b223c5338b11\") " pod="openshift-monitoring/node-exporter-dllnm"
Sep 17 23:15:12.408717 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:15:12.408599    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8cbd6\" (UniqueName: \"kubernetes.io/projected/34ae65de-7218-4b53-9a3e-b223c5338b11-kube-api-access-8cbd6\") pod \"node-exporter-dllnm\" (UID: \"34ae65de-7218-4b53-9a3e-b223c5338b11\") " pod="openshift-monitoring/node-exporter-dllnm"
Sep 17 23:15:12.408717 ip-10-0-132-188 kubenswrapper[2562]: E0917 23:15:12.408616    2562 secret.go:189] Couldn't get secret openshift-monitoring/openshift-state-metrics-tls: secret "openshift-state-metrics-tls" not found
Sep 17 23:15:12.408717 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:15:12.408683    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/34ae65de-7218-4b53-9a3e-b223c5338b11-node-exporter-textfile\") pod \"node-exporter-dllnm\" (UID: \"34ae65de-7218-4b53-9a3e-b223c5338b11\") " pod="openshift-monitoring/node-exporter-dllnm"
Sep 17 23:15:12.408883 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:15:12.408713    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/34ae65de-7218-4b53-9a3e-b223c5338b11-metrics-client-ca\") pod \"node-exporter-dllnm\" (UID: \"34ae65de-7218-4b53-9a3e-b223c5338b11\") " pod="openshift-monitoring/node-exporter-dllnm"
Sep 17 23:15:12.408883 ip-10-0-132-188 kubenswrapper[2562]: E0917 23:15:12.408775    2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/c5c362bf-e3b6-41f3-85c6-4a7d2a3e4b87-openshift-state-metrics-tls podName:c5c362bf-e3b6-41f3-85c6-4a7d2a3e4b87 nodeName:}" failed. No retries permitted until 2026-09-17 23:15:12.908751048 +0000 UTC m=+265.310647793 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "openshift-state-metrics-tls" (UniqueName: "kubernetes.io/secret/c5c362bf-e3b6-41f3-85c6-4a7d2a3e4b87-openshift-state-metrics-tls") pod "openshift-state-metrics-86c5bc79d6-bwj6t" (UID: "c5c362bf-e3b6-41f3-85c6-4a7d2a3e4b87") : secret "openshift-state-metrics-tls" not found
Sep 17 23:15:12.408980 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:15:12.408963    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/c5c362bf-e3b6-41f3-85c6-4a7d2a3e4b87-metrics-client-ca\") pod \"openshift-state-metrics-86c5bc79d6-bwj6t\" (UID: \"c5c362bf-e3b6-41f3-85c6-4a7d2a3e4b87\") " pod="openshift-monitoring/openshift-state-metrics-86c5bc79d6-bwj6t"
Sep 17 23:15:12.410667 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:15:12.410648    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/c5c362bf-e3b6-41f3-85c6-4a7d2a3e4b87-openshift-state-metrics-kube-rbac-proxy-config\") pod \"openshift-state-metrics-86c5bc79d6-bwj6t\" (UID: \"c5c362bf-e3b6-41f3-85c6-4a7d2a3e4b87\") " pod="openshift-monitoring/openshift-state-metrics-86c5bc79d6-bwj6t"
Sep 17 23:15:12.419149 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:15:12.419124    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-rw577\" (UniqueName: \"kubernetes.io/projected/c5c362bf-e3b6-41f3-85c6-4a7d2a3e4b87-kube-api-access-rw577\") pod \"openshift-state-metrics-86c5bc79d6-bwj6t\" (UID: \"c5c362bf-e3b6-41f3-85c6-4a7d2a3e4b87\") " pod="openshift-monitoring/openshift-state-metrics-86c5bc79d6-bwj6t"
Sep 17 23:15:12.420805 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:15:12.420777    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fwwc5_7e452bbb-7dc6-4c9d-a9b1-76400942f5b4/konnectivity-agent/0.log" limit=1024
Sep 17 23:15:12.509114 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:15:12.509073    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/34ae65de-7218-4b53-9a3e-b223c5338b11-node-exporter-textfile\") pod \"node-exporter-dllnm\" (UID: \"34ae65de-7218-4b53-9a3e-b223c5338b11\") " pod="openshift-monitoring/node-exporter-dllnm"
Sep 17 23:15:12.509114 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:15:12.509109    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/34ae65de-7218-4b53-9a3e-b223c5338b11-metrics-client-ca\") pod \"node-exporter-dllnm\" (UID: \"34ae65de-7218-4b53-9a3e-b223c5338b11\") " pod="openshift-monitoring/node-exporter-dllnm"
Sep 17 23:15:12.509361 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:15:12.509139    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/34ae65de-7218-4b53-9a3e-b223c5338b11-node-exporter-tls\") pod \"node-exporter-dllnm\" (UID: \"34ae65de-7218-4b53-9a3e-b223c5338b11\") " pod="openshift-monitoring/node-exporter-dllnm"
Sep 17 23:15:12.509361 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:15:12.509189    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/34ae65de-7218-4b53-9a3e-b223c5338b11-node-exporter-accelerators-collector-config\") pod \"node-exporter-dllnm\" (UID: \"34ae65de-7218-4b53-9a3e-b223c5338b11\") " pod="openshift-monitoring/node-exporter-dllnm"
Sep 17 23:15:12.509361 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:15:12.509230    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/34ae65de-7218-4b53-9a3e-b223c5338b11-root\") pod \"node-exporter-dllnm\" (UID: \"34ae65de-7218-4b53-9a3e-b223c5338b11\") " pod="openshift-monitoring/node-exporter-dllnm"
Sep 17 23:15:12.509361 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:15:12.509258    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/34ae65de-7218-4b53-9a3e-b223c5338b11-sys\") pod \"node-exporter-dllnm\" (UID: \"34ae65de-7218-4b53-9a3e-b223c5338b11\") " pod="openshift-monitoring/node-exporter-dllnm"
Sep 17 23:15:12.509361 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:15:12.509281    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/34ae65de-7218-4b53-9a3e-b223c5338b11-node-exporter-wtmp\") pod \"node-exporter-dllnm\" (UID: \"34ae65de-7218-4b53-9a3e-b223c5338b11\") " pod="openshift-monitoring/node-exporter-dllnm"
Sep 17 23:15:12.509361 ip-10-0-132-188 kubenswrapper[2562]: E0917 23:15:12.509299    2562 secret.go:189] Couldn't get secret openshift-monitoring/node-exporter-tls: secret "node-exporter-tls" not found
Sep 17 23:15:12.509361 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:15:12.509335    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/34ae65de-7218-4b53-9a3e-b223c5338b11-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-dllnm\" (UID: \"34ae65de-7218-4b53-9a3e-b223c5338b11\") " pod="openshift-monitoring/node-exporter-dllnm"
Sep 17 23:15:12.509361 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:15:12.509347    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"root\" (UniqueName: \"kubernetes.io/host-path/34ae65de-7218-4b53-9a3e-b223c5338b11-root\") pod \"node-exporter-dllnm\" (UID: \"34ae65de-7218-4b53-9a3e-b223c5338b11\") " pod="openshift-monitoring/node-exporter-dllnm"
Sep 17 23:15:12.509361 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:15:12.509359    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8cbd6\" (UniqueName: \"kubernetes.io/projected/34ae65de-7218-4b53-9a3e-b223c5338b11-kube-api-access-8cbd6\") pod \"node-exporter-dllnm\" (UID: \"34ae65de-7218-4b53-9a3e-b223c5338b11\") " pod="openshift-monitoring/node-exporter-dllnm"
Sep 17 23:15:12.509792 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:15:12.509373    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/34ae65de-7218-4b53-9a3e-b223c5338b11-sys\") pod \"node-exporter-dllnm\" (UID: \"34ae65de-7218-4b53-9a3e-b223c5338b11\") " pod="openshift-monitoring/node-exporter-dllnm"
Sep 17 23:15:12.509792 ip-10-0-132-188 kubenswrapper[2562]: E0917 23:15:12.509378    2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/34ae65de-7218-4b53-9a3e-b223c5338b11-node-exporter-tls podName:34ae65de-7218-4b53-9a3e-b223c5338b11 nodeName:}" failed. No retries permitted until 2026-09-17 23:15:13.009355106 +0000 UTC m=+265.411251862 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "node-exporter-tls" (UniqueName: "kubernetes.io/secret/34ae65de-7218-4b53-9a3e-b223c5338b11-node-exporter-tls") pod "node-exporter-dllnm" (UID: "34ae65de-7218-4b53-9a3e-b223c5338b11") : secret "node-exporter-tls" not found
Sep 17 23:15:12.509792 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:15:12.509491    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/34ae65de-7218-4b53-9a3e-b223c5338b11-node-exporter-wtmp\") pod \"node-exporter-dllnm\" (UID: \"34ae65de-7218-4b53-9a3e-b223c5338b11\") " pod="openshift-monitoring/node-exporter-dllnm"
Sep 17 23:15:12.509792 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:15:12.509504    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/34ae65de-7218-4b53-9a3e-b223c5338b11-node-exporter-textfile\") pod \"node-exporter-dllnm\" (UID: \"34ae65de-7218-4b53-9a3e-b223c5338b11\") " pod="openshift-monitoring/node-exporter-dllnm"
Sep 17 23:15:12.509792 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:15:12.509749    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/34ae65de-7218-4b53-9a3e-b223c5338b11-metrics-client-ca\") pod \"node-exporter-dllnm\" (UID: \"34ae65de-7218-4b53-9a3e-b223c5338b11\") " pod="openshift-monitoring/node-exporter-dllnm"
Sep 17 23:15:12.509994 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:15:12.509798    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/34ae65de-7218-4b53-9a3e-b223c5338b11-node-exporter-accelerators-collector-config\") pod \"node-exporter-dllnm\" (UID: \"34ae65de-7218-4b53-9a3e-b223c5338b11\") " pod="openshift-monitoring/node-exporter-dllnm"
Sep 17 23:15:12.511821 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:15:12.511794    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/34ae65de-7218-4b53-9a3e-b223c5338b11-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-dllnm\" (UID: \"34ae65de-7218-4b53-9a3e-b223c5338b11\") " pod="openshift-monitoring/node-exporter-dllnm"
Sep 17 23:15:12.520625 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:15:12.520589    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-8cbd6\" (UniqueName: \"kubernetes.io/projected/34ae65de-7218-4b53-9a3e-b223c5338b11-kube-api-access-8cbd6\") pod \"node-exporter-dllnm\" (UID: \"34ae65de-7218-4b53-9a3e-b223c5338b11\") " pod="openshift-monitoring/node-exporter-dllnm"
Sep 17 23:15:12.522327 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:15:12.522284    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fwwc5_7e452bbb-7dc6-4c9d-a9b1-76400942f5b4/konnectivity-agent/0.log" limit=1024
Sep 17 23:15:12.641464 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:15:12.641421    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fwwc5_7e452bbb-7dc6-4c9d-a9b1-76400942f5b4/konnectivity-agent/0.log" limit=1024
Sep 17 23:15:12.726658 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:15:12.726625    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fwwc5_7e452bbb-7dc6-4c9d-a9b1-76400942f5b4/konnectivity-agent/0.log" limit=1024
Sep 17 23:15:12.823893 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:15:12.823858    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fwwc5_7e452bbb-7dc6-4c9d-a9b1-76400942f5b4/konnectivity-agent/0.log" limit=1024
Sep 17 23:15:12.911995 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:15:12.911897    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/c5c362bf-e3b6-41f3-85c6-4a7d2a3e4b87-openshift-state-metrics-tls\") pod \"openshift-state-metrics-86c5bc79d6-bwj6t\" (UID: \"c5c362bf-e3b6-41f3-85c6-4a7d2a3e4b87\") " pod="openshift-monitoring/openshift-state-metrics-86c5bc79d6-bwj6t"
Sep 17 23:15:12.914318 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:15:12.914292    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/c5c362bf-e3b6-41f3-85c6-4a7d2a3e4b87-openshift-state-metrics-tls\") pod \"openshift-state-metrics-86c5bc79d6-bwj6t\" (UID: \"c5c362bf-e3b6-41f3-85c6-4a7d2a3e4b87\") " pod="openshift-monitoring/openshift-state-metrics-86c5bc79d6-bwj6t"
Sep 17 23:15:12.923288 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:15:12.923255    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fwwc5_7e452bbb-7dc6-4c9d-a9b1-76400942f5b4/konnectivity-agent/0.log" limit=1024
Sep 17 23:15:13.013393 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:15:13.013346    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/34ae65de-7218-4b53-9a3e-b223c5338b11-node-exporter-tls\") pod \"node-exporter-dllnm\" (UID: \"34ae65de-7218-4b53-9a3e-b223c5338b11\") " pod="openshift-monitoring/node-exporter-dllnm"
Sep 17 23:15:13.017033 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:15:13.016542    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/34ae65de-7218-4b53-9a3e-b223c5338b11-node-exporter-tls\") pod \"node-exporter-dllnm\" (UID: \"34ae65de-7218-4b53-9a3e-b223c5338b11\") " pod="openshift-monitoring/node-exporter-dllnm"
Sep 17 23:15:13.199447 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:15:13.199350    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/openshift-state-metrics-86c5bc79d6-bwj6t"
Sep 17 23:15:13.235550 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:15:13.235504    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-dllnm"
Sep 17 23:15:13.246445 ip-10-0-132-188 kubenswrapper[2562]: W0917 23:15:13.246341    2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod34ae65de_7218_4b53_9a3e_b223c5338b11.slice/crio-a6773ce74ae01a15f61e029bf63a991a2e61729b1ebea6f18da22c6326bb37c6 WatchSource:0}: Error finding container a6773ce74ae01a15f61e029bf63a991a2e61729b1ebea6f18da22c6326bb37c6: Status 404 returned error can't find the container with id a6773ce74ae01a15f61e029bf63a991a2e61729b1ebea6f18da22c6326bb37c6
Sep 17 23:15:13.307416 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:15:13.307380    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fwwc5_7e452bbb-7dc6-4c9d-a9b1-76400942f5b4/konnectivity-agent/0.log" limit=1024
Sep 17 23:15:13.395362 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:15:13.395332    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fwwc5_7e452bbb-7dc6-4c9d-a9b1-76400942f5b4/konnectivity-agent/0.log" limit=1024
Sep 17 23:15:13.476360 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:15:13.476282    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/openshift-state-metrics-86c5bc79d6-bwj6t"]
Sep 17 23:15:13.479347 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:15:13.479316    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fwwc5_7e452bbb-7dc6-4c9d-a9b1-76400942f5b4/konnectivity-agent/0.log" limit=1024
Sep 17 23:15:13.480197 ip-10-0-132-188 kubenswrapper[2562]: W0917 23:15:13.480153    2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc5c362bf_e3b6_41f3_85c6_4a7d2a3e4b87.slice/crio-3722400d19ff02d2431fa7b2c4eaaffe95a8dc7a815d3c42ae01ff77805beac3 WatchSource:0}: Error finding container 3722400d19ff02d2431fa7b2c4eaaffe95a8dc7a815d3c42ae01ff77805beac3: Status 404 returned error can't find the container with id 3722400d19ff02d2431fa7b2c4eaaffe95a8dc7a815d3c42ae01ff77805beac3
Sep 17 23:15:13.639991 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:15:13.639946    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-dllnm" event={"ID":"34ae65de-7218-4b53-9a3e-b223c5338b11","Type":"ContainerStarted","Data":"a6773ce74ae01a15f61e029bf63a991a2e61729b1ebea6f18da22c6326bb37c6"}
Sep 17 23:15:13.642053 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:15:13.642020    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/openshift-state-metrics-86c5bc79d6-bwj6t" event={"ID":"c5c362bf-e3b6-41f3-85c6-4a7d2a3e4b87","Type":"ContainerStarted","Data":"9da3510d80a9db766a7505de772f21c593ce2025675cdcf36ab68179241581b3"}
Sep 17 23:15:13.642241 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:15:13.642075    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/openshift-state-metrics-86c5bc79d6-bwj6t" event={"ID":"c5c362bf-e3b6-41f3-85c6-4a7d2a3e4b87","Type":"ContainerStarted","Data":"07173364e2e83bd9ee1ab9d9c1d998353a96190cf13ed3cee9ed8f4ba9031109"}
Sep 17 23:15:13.642241 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:15:13.642102    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/openshift-state-metrics-86c5bc79d6-bwj6t" event={"ID":"c5c362bf-e3b6-41f3-85c6-4a7d2a3e4b87","Type":"ContainerStarted","Data":"3722400d19ff02d2431fa7b2c4eaaffe95a8dc7a815d3c42ae01ff77805beac3"}
Sep 17 23:15:14.648040 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:15:14.647998    2562 generic.go:356] "Generic (PLEG): container finished" podID="34ae65de-7218-4b53-9a3e-b223c5338b11" containerID="d3bc61096c6cda385f6390e4fc74642690de1002959bd550151aced7ac5ab6d3" exitCode=0
Sep 17 23:15:14.648499 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:15:14.648068    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-dllnm" event={"ID":"34ae65de-7218-4b53-9a3e-b223c5338b11","Type":"ContainerDied","Data":"d3bc61096c6cda385f6390e4fc74642690de1002959bd550151aced7ac5ab6d3"}
Sep 17 23:15:15.209105 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:15:15.209064    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fwwc5_7e452bbb-7dc6-4c9d-a9b1-76400942f5b4/konnectivity-agent/0.log" limit=1024
Sep 17 23:15:15.261560 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:15:15.261516    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fwwc5_7e452bbb-7dc6-4c9d-a9b1-76400942f5b4/konnectivity-agent/0.log" limit=1024
Sep 17 23:15:15.364882 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:15:15.364844    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fwwc5_7e452bbb-7dc6-4c9d-a9b1-76400942f5b4/konnectivity-agent/0.log" limit=1024
Sep 17 23:15:15.435776 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:15:15.435736    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fwwc5_7e452bbb-7dc6-4c9d-a9b1-76400942f5b4/konnectivity-agent/0.log" limit=1024
Sep 17 23:15:15.653506 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:15:15.653467    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/openshift-state-metrics-86c5bc79d6-bwj6t" event={"ID":"c5c362bf-e3b6-41f3-85c6-4a7d2a3e4b87","Type":"ContainerStarted","Data":"ab30f4869c5a9870303ecd08eb35f8d5143183a443c8ed7fdd1f4978a378436f"}
Sep 17 23:15:15.655698 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:15:15.655670    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-dllnm" event={"ID":"34ae65de-7218-4b53-9a3e-b223c5338b11","Type":"ContainerStarted","Data":"49ca3b391a2d5cecdc6385b1b7483997d3f888a125d557d842ab7a3f500246e9"}
Sep 17 23:15:15.655820 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:15:15.655717    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-dllnm" event={"ID":"34ae65de-7218-4b53-9a3e-b223c5338b11","Type":"ContainerStarted","Data":"ff1dae52b580fcfd21ef993c259059d6dc7e522bb036f382e3d9831f52cffeeb"}
Sep 17 23:15:15.670738 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:15:15.670686    2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/openshift-state-metrics-86c5bc79d6-bwj6t" podStartSLOduration=2.534751589 podStartE2EDuration="3.670668963s" totalImagesPullingTime="1.135917374s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-17 23:15:12 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-17 23:15:15.669249126 +0000 UTC m=+268.071145892" watchObservedRunningTime="2026-09-17 23:15:15.670668963 +0000 UTC m=+268.072565759"
Sep 17 23:15:15.688246 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:15:15.688155    2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/node-exporter-dllnm" podStartSLOduration=2.686166916 podStartE2EDuration="3.688135317s" totalImagesPullingTime="935.831345ms" totalInitContainerRuntime="66.137056ms" isStatefulPod=true podCreationTimestamp="2026-09-17 23:15:12 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-17 23:15:15.686994364 +0000 UTC m=+268.088891129" watchObservedRunningTime="2026-09-17 23:15:15.688135317 +0000 UTC m=+268.090032091"
Sep 17 23:15:16.608507 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:15:16.608470    2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/metrics-server-5d89bb6d78-5kftc"]
Sep 17 23:15:16.610647 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:15:16.610630    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/metrics-server-5d89bb6d78-5kftc"
Sep 17 23:15:16.613822 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:15:16.613790    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kubelet-serving-ca-bundle\""
Sep 17 23:15:16.613932 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:15:16.613792    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-server-dockercfg-cvsfm\""
Sep 17 23:15:16.613932 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:15:16.613897    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-server-av6cngf2q7pa8\""
Sep 17 23:15:16.614046 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:15:16.613930    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-server-tls\""
Sep 17 23:15:16.614046 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:15:16.614039    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"metrics-server-audit-profiles\""
Sep 17 23:15:16.614155 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:15:16.614138    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-server-client-certs\""
Sep 17 23:15:16.625767 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:15:16.623567    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/metrics-server-5d89bb6d78-5kftc"]
Sep 17 23:15:16.640774 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:15:16.640745    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-metrics-server-tls\" (UniqueName: \"kubernetes.io/secret/8fcaefb7-7432-4b32-95ae-09870e558cdd-secret-metrics-server-tls\") pod \"metrics-server-5d89bb6d78-5kftc\" (UID: \"8fcaefb7-7432-4b32-95ae-09870e558cdd\") " pod="openshift-monitoring/metrics-server-5d89bb6d78-5kftc"
Sep 17 23:15:16.640921 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:15:16.640781    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"audit-log\" (UniqueName: \"kubernetes.io/empty-dir/8fcaefb7-7432-4b32-95ae-09870e558cdd-audit-log\") pod \"metrics-server-5d89bb6d78-5kftc\" (UID: \"8fcaefb7-7432-4b32-95ae-09870e558cdd\") " pod="openshift-monitoring/metrics-server-5d89bb6d78-5kftc"
Sep 17 23:15:16.640921 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:15:16.640798    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-96npf\" (UniqueName: \"kubernetes.io/projected/8fcaefb7-7432-4b32-95ae-09870e558cdd-kube-api-access-96npf\") pod \"metrics-server-5d89bb6d78-5kftc\" (UID: \"8fcaefb7-7432-4b32-95ae-09870e558cdd\") " pod="openshift-monitoring/metrics-server-5d89bb6d78-5kftc"
Sep 17 23:15:16.640921 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:15:16.640827    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-metrics-server-client-certs\" (UniqueName: \"kubernetes.io/secret/8fcaefb7-7432-4b32-95ae-09870e558cdd-secret-metrics-server-client-certs\") pod \"metrics-server-5d89bb6d78-5kftc\" (UID: \"8fcaefb7-7432-4b32-95ae-09870e558cdd\") " pod="openshift-monitoring/metrics-server-5d89bb6d78-5kftc"
Sep 17 23:15:16.640921 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:15:16.640845    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/8fcaefb7-7432-4b32-95ae-09870e558cdd-configmap-kubelet-serving-ca-bundle\") pod \"metrics-server-5d89bb6d78-5kftc\" (UID: \"8fcaefb7-7432-4b32-95ae-09870e558cdd\") " pod="openshift-monitoring/metrics-server-5d89bb6d78-5kftc"
Sep 17 23:15:16.640921 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:15:16.640870    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-server-audit-profiles\" (UniqueName: \"kubernetes.io/configmap/8fcaefb7-7432-4b32-95ae-09870e558cdd-metrics-server-audit-profiles\") pod \"metrics-server-5d89bb6d78-5kftc\" (UID: \"8fcaefb7-7432-4b32-95ae-09870e558cdd\") " pod="openshift-monitoring/metrics-server-5d89bb6d78-5kftc"
Sep 17 23:15:16.640921 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:15:16.640918    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"client-ca-bundle\" (UniqueName: \"kubernetes.io/secret/8fcaefb7-7432-4b32-95ae-09870e558cdd-client-ca-bundle\") pod \"metrics-server-5d89bb6d78-5kftc\" (UID: \"8fcaefb7-7432-4b32-95ae-09870e558cdd\") " pod="openshift-monitoring/metrics-server-5d89bb6d78-5kftc"
Sep 17 23:15:16.741902 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:15:16.741853    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"audit-log\" (UniqueName: \"kubernetes.io/empty-dir/8fcaefb7-7432-4b32-95ae-09870e558cdd-audit-log\") pod \"metrics-server-5d89bb6d78-5kftc\" (UID: \"8fcaefb7-7432-4b32-95ae-09870e558cdd\") " pod="openshift-monitoring/metrics-server-5d89bb6d78-5kftc"
Sep 17 23:15:16.741902 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:15:16.741916    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-96npf\" (UniqueName: \"kubernetes.io/projected/8fcaefb7-7432-4b32-95ae-09870e558cdd-kube-api-access-96npf\") pod \"metrics-server-5d89bb6d78-5kftc\" (UID: \"8fcaefb7-7432-4b32-95ae-09870e558cdd\") " pod="openshift-monitoring/metrics-server-5d89bb6d78-5kftc"
Sep 17 23:15:16.742477 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:15:16.741938    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-metrics-server-client-certs\" (UniqueName: \"kubernetes.io/secret/8fcaefb7-7432-4b32-95ae-09870e558cdd-secret-metrics-server-client-certs\") pod \"metrics-server-5d89bb6d78-5kftc\" (UID: \"8fcaefb7-7432-4b32-95ae-09870e558cdd\") " pod="openshift-monitoring/metrics-server-5d89bb6d78-5kftc"
Sep 17 23:15:16.742477 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:15:16.741956    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/8fcaefb7-7432-4b32-95ae-09870e558cdd-configmap-kubelet-serving-ca-bundle\") pod \"metrics-server-5d89bb6d78-5kftc\" (UID: \"8fcaefb7-7432-4b32-95ae-09870e558cdd\") " pod="openshift-monitoring/metrics-server-5d89bb6d78-5kftc"
Sep 17 23:15:16.742477 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:15:16.742001    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-server-audit-profiles\" (UniqueName: \"kubernetes.io/configmap/8fcaefb7-7432-4b32-95ae-09870e558cdd-metrics-server-audit-profiles\") pod \"metrics-server-5d89bb6d78-5kftc\" (UID: \"8fcaefb7-7432-4b32-95ae-09870e558cdd\") " pod="openshift-monitoring/metrics-server-5d89bb6d78-5kftc"
Sep 17 23:15:16.742477 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:15:16.742266    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"client-ca-bundle\" (UniqueName: \"kubernetes.io/secret/8fcaefb7-7432-4b32-95ae-09870e558cdd-client-ca-bundle\") pod \"metrics-server-5d89bb6d78-5kftc\" (UID: \"8fcaefb7-7432-4b32-95ae-09870e558cdd\") " pod="openshift-monitoring/metrics-server-5d89bb6d78-5kftc"
Sep 17 23:15:16.742477 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:15:16.742363    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"audit-log\" (UniqueName: \"kubernetes.io/empty-dir/8fcaefb7-7432-4b32-95ae-09870e558cdd-audit-log\") pod \"metrics-server-5d89bb6d78-5kftc\" (UID: \"8fcaefb7-7432-4b32-95ae-09870e558cdd\") " pod="openshift-monitoring/metrics-server-5d89bb6d78-5kftc"
Sep 17 23:15:16.742477 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:15:16.742405    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-metrics-server-tls\" (UniqueName: \"kubernetes.io/secret/8fcaefb7-7432-4b32-95ae-09870e558cdd-secret-metrics-server-tls\") pod \"metrics-server-5d89bb6d78-5kftc\" (UID: \"8fcaefb7-7432-4b32-95ae-09870e558cdd\") " pod="openshift-monitoring/metrics-server-5d89bb6d78-5kftc"
Sep 17 23:15:16.742797 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:15:16.742767    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/8fcaefb7-7432-4b32-95ae-09870e558cdd-configmap-kubelet-serving-ca-bundle\") pod \"metrics-server-5d89bb6d78-5kftc\" (UID: \"8fcaefb7-7432-4b32-95ae-09870e558cdd\") " pod="openshift-monitoring/metrics-server-5d89bb6d78-5kftc"
Sep 17 23:15:16.743125 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:15:16.743100    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-server-audit-profiles\" (UniqueName: \"kubernetes.io/configmap/8fcaefb7-7432-4b32-95ae-09870e558cdd-metrics-server-audit-profiles\") pod \"metrics-server-5d89bb6d78-5kftc\" (UID: \"8fcaefb7-7432-4b32-95ae-09870e558cdd\") " pod="openshift-monitoring/metrics-server-5d89bb6d78-5kftc"
Sep 17 23:15:16.744910 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:15:16.744889    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-metrics-server-client-certs\" (UniqueName: \"kubernetes.io/secret/8fcaefb7-7432-4b32-95ae-09870e558cdd-secret-metrics-server-client-certs\") pod \"metrics-server-5d89bb6d78-5kftc\" (UID: \"8fcaefb7-7432-4b32-95ae-09870e558cdd\") " pod="openshift-monitoring/metrics-server-5d89bb6d78-5kftc"
Sep 17 23:15:16.745791 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:15:16.745764    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"client-ca-bundle\" (UniqueName: \"kubernetes.io/secret/8fcaefb7-7432-4b32-95ae-09870e558cdd-client-ca-bundle\") pod \"metrics-server-5d89bb6d78-5kftc\" (UID: \"8fcaefb7-7432-4b32-95ae-09870e558cdd\") " pod="openshift-monitoring/metrics-server-5d89bb6d78-5kftc"
Sep 17 23:15:16.745910 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:15:16.745887    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-metrics-server-tls\" (UniqueName: \"kubernetes.io/secret/8fcaefb7-7432-4b32-95ae-09870e558cdd-secret-metrics-server-tls\") pod \"metrics-server-5d89bb6d78-5kftc\" (UID: \"8fcaefb7-7432-4b32-95ae-09870e558cdd\") " pod="openshift-monitoring/metrics-server-5d89bb6d78-5kftc"
Sep 17 23:15:16.749994 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:15:16.749977    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-96npf\" (UniqueName: \"kubernetes.io/projected/8fcaefb7-7432-4b32-95ae-09870e558cdd-kube-api-access-96npf\") pod \"metrics-server-5d89bb6d78-5kftc\" (UID: \"8fcaefb7-7432-4b32-95ae-09870e558cdd\") " pod="openshift-monitoring/metrics-server-5d89bb6d78-5kftc"
Sep 17 23:15:16.920598 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:15:16.920503    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/metrics-server-5d89bb6d78-5kftc"
Sep 17 23:15:17.077943 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:15:17.077885    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fwwc5_7e452bbb-7dc6-4c9d-a9b1-76400942f5b4/konnectivity-agent/0.log" limit=1024
Sep 17 23:15:17.197924 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:15:17.197848    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/metrics-server-5d89bb6d78-5kftc"]
Sep 17 23:15:17.201235 ip-10-0-132-188 kubenswrapper[2562]: W0917 23:15:17.201206    2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8fcaefb7_7432_4b32_95ae_09870e558cdd.slice/crio-e6bdfe3daacb68156d09e5babec6bfeb2b048bbcd248ce9510d24df85f9e7f9e WatchSource:0}: Error finding container e6bdfe3daacb68156d09e5babec6bfeb2b048bbcd248ce9510d24df85f9e7f9e: Status 404 returned error can't find the container with id e6bdfe3daacb68156d09e5babec6bfeb2b048bbcd248ce9510d24df85f9e7f9e
Sep 17 23:15:17.665276 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:15:17.665241    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/metrics-server-5d89bb6d78-5kftc" event={"ID":"8fcaefb7-7432-4b32-95ae-09870e558cdd","Type":"ContainerStarted","Data":"e6bdfe3daacb68156d09e5babec6bfeb2b048bbcd248ce9510d24df85f9e7f9e"}
Sep 17 23:15:17.671870 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:15:17.671842    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fwwc5_7e452bbb-7dc6-4c9d-a9b1-76400942f5b4/konnectivity-agent/0.log" limit=1024
Sep 17 23:15:18.476380 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:15:18.476324    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fwwc5_7e452bbb-7dc6-4c9d-a9b1-76400942f5b4/konnectivity-agent/0.log" limit=1024
Sep 17 23:15:19.673249 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:15:19.673213    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/metrics-server-5d89bb6d78-5kftc" event={"ID":"8fcaefb7-7432-4b32-95ae-09870e558cdd","Type":"ContainerStarted","Data":"9fbd69a746fa43ba6cf40af27900887934a3649bf64c0b384d5d028f46154e58"}
Sep 17 23:15:19.690114 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:15:19.690066    2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/metrics-server-5d89bb6d78-5kftc" podStartSLOduration=2.13296517 podStartE2EDuration="3.690050706s" totalImagesPullingTime="1.557085536s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-17 23:15:16 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-17 23:15:19.688557174 +0000 UTC m=+272.090453939" watchObservedRunningTime="2026-09-17 23:15:19.690050706 +0000 UTC m=+272.091947470"
Sep 17 23:15:19.953712 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:15:19.953627    2562 patch_prober.go:28] interesting pod/image-registry-64c75f9db5-qc6hf container/registry namespace/openshift-image-registry: Liveness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body={"errors":[{"code":"UNAVAILABLE","message":"service unavailable","detail":"health check failed: please see /debug/health"}]}
Sep 17 23:15:19.953712 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:15:19.953691    2562 prober.go:120] "Probe failed" probeType="Liveness" pod="openshift-image-registry/image-registry-64c75f9db5-qc6hf" podUID="51d624a1-203a-4888-be49-4a3c375984a3" containerName="registry" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Sep 17 23:15:21.586442 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:15:21.586407    2562 patch_prober.go:28] interesting pod/image-registry-64c75f9db5-qc6hf container/registry namespace/openshift-image-registry: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body={"errors":[{"code":"UNAVAILABLE","message":"service unavailable","detail":"health check failed: please see /debug/health"}]}
Sep 17 23:15:21.586832 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:15:21.586474    2562 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-image-registry/image-registry-64c75f9db5-qc6hf" podUID="51d624a1-203a-4888-be49-4a3c375984a3" containerName="registry" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Sep 17 23:15:22.359346 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:15:22.359304    2562 prober.go:120] "Probe failed" probeType="Liveness" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-64d77d549c-dndvm" podUID="204204da-e53d-4c6b-93ed-a6271c85ac27" containerName="service-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 500"
Sep 17 23:15:22.359515 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:15:22.359381    2562 kubelet.go:2654] "SyncLoop (probe)" probe="liveness" status="unhealthy" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-64d77d549c-dndvm"
Sep 17 23:15:22.359879 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:15:22.359849    2562 kuberuntime_manager.go:1192] "Message for Container of pod" containerName="service-proxy" containerStatusID={"Type":"cri-o","ID":"a3fb94cc577c91226e36070d0fc0c23adbfb2d08783a3ad8e99a4955a1b14571"} pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-64d77d549c-dndvm" containerMessage="Container service-proxy failed liveness probe, will be restarted"
Sep 17 23:15:22.359919 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:15:22.359901    2562 kuberuntime_container.go:873] "Killing container with a grace period" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-64d77d549c-dndvm" podUID="204204da-e53d-4c6b-93ed-a6271c85ac27" containerName="service-proxy" containerID="cri-o://a3fb94cc577c91226e36070d0fc0c23adbfb2d08783a3ad8e99a4955a1b14571" gracePeriod=30
Sep 17 23:15:22.686153 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:15:22.686066    2562 generic.go:356] "Generic (PLEG): container finished" podID="204204da-e53d-4c6b-93ed-a6271c85ac27" containerID="a3fb94cc577c91226e36070d0fc0c23adbfb2d08783a3ad8e99a4955a1b14571" exitCode=2
Sep 17 23:15:22.686153 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:15:22.686138    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-64d77d549c-dndvm" event={"ID":"204204da-e53d-4c6b-93ed-a6271c85ac27","Type":"ContainerDied","Data":"a3fb94cc577c91226e36070d0fc0c23adbfb2d08783a3ad8e99a4955a1b14571"}
Sep 17 23:15:22.686579 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:15:22.686209    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-64d77d549c-dndvm" event={"ID":"204204da-e53d-4c6b-93ed-a6271c85ac27","Type":"ContainerStarted","Data":"acf50c6db293f3c86c0a5051e8a1b5a31d5b6c7e36c89633e5d0d45ac9927f33"}
Sep 17 23:15:25.691540 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:15:25.691498    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fwwc5_7e452bbb-7dc6-4c9d-a9b1-76400942f5b4/konnectivity-agent/0.log" limit=1024
Sep 17 23:15:25.788371 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:15:25.788333    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fwwc5_7e452bbb-7dc6-4c9d-a9b1-76400942f5b4/konnectivity-agent/0.log" limit=1024
Sep 17 23:15:26.023674 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:15:26.023577    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fwwc5_7e452bbb-7dc6-4c9d-a9b1-76400942f5b4/konnectivity-agent/0.log" limit=1024
Sep 17 23:15:26.342217 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:15:26.342180    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fwwc5_7e452bbb-7dc6-4c9d-a9b1-76400942f5b4/konnectivity-agent/0.log" limit=1024
Sep 17 23:15:26.443256 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:15:26.443211    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fwwc5_7e452bbb-7dc6-4c9d-a9b1-76400942f5b4/konnectivity-agent/0.log" limit=1024
Sep 17 23:15:26.747020 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:15:26.746918    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fwwc5_7e452bbb-7dc6-4c9d-a9b1-76400942f5b4/konnectivity-agent/0.log" limit=1024
Sep 17 23:15:26.846162 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:15:26.846127    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fwwc5_7e452bbb-7dc6-4c9d-a9b1-76400942f5b4/konnectivity-agent/0.log" limit=1024
Sep 17 23:15:27.553646 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:15:27.553604    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fwwc5_7e452bbb-7dc6-4c9d-a9b1-76400942f5b4/konnectivity-agent/0.log" limit=1024
Sep 17 23:15:27.653878 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:15:27.653836    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fwwc5_7e452bbb-7dc6-4c9d-a9b1-76400942f5b4/konnectivity-agent/0.log" limit=1024
Sep 17 23:15:27.807304 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:15:27.807217    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fwwc5_7e452bbb-7dc6-4c9d-a9b1-76400942f5b4/konnectivity-agent/0.log" limit=1024
Sep 17 23:15:28.420678 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:15:28.420639    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fwwc5_7e452bbb-7dc6-4c9d-a9b1-76400942f5b4/konnectivity-agent/0.log" limit=1024
Sep 17 23:15:29.301435 ip-10-0-132-188 kubenswrapper[2562]: E0917 23:15:29.301390    2562 pod_workers.go:1324] "Error syncing pod, skipping" err="unmounted volumes=[metrics-tls], unattached volumes=[], failed to process volumes=[]: context deadline exceeded" pod="openshift-dns/dns-default-9z4vw" podUID="57e9d0f5-d16a-42d0-ab45-c06fe21e7bbd"
Sep 17 23:15:29.711057 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:15:29.711026    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-9z4vw"
Sep 17 23:15:29.953738 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:15:29.953698    2562 patch_prober.go:28] interesting pod/image-registry-64c75f9db5-qc6hf container/registry namespace/openshift-image-registry: Liveness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body={"errors":[{"code":"UNAVAILABLE","message":"service unavailable","detail":"health check failed: please see /debug/health"}]}
Sep 17 23:15:29.953889 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:15:29.953749    2562 prober.go:120] "Probe failed" probeType="Liveness" pod="openshift-image-registry/image-registry-64c75f9db5-qc6hf" podUID="51d624a1-203a-4888-be49-4a3c375984a3" containerName="registry" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Sep 17 23:15:31.586572 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:15:31.586536    2562 patch_prober.go:28] interesting pod/image-registry-64c75f9db5-qc6hf container/registry namespace/openshift-image-registry: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body={"errors":[{"code":"UNAVAILABLE","message":"service unavailable","detail":"health check failed: please see /debug/health"}]}
Sep 17 23:15:31.586936 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:15:31.586594    2562 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-image-registry/image-registry-64c75f9db5-qc6hf" podUID="51d624a1-203a-4888-be49-4a3c375984a3" containerName="registry" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Sep 17 23:15:33.084820 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:15:33.084778    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/57e9d0f5-d16a-42d0-ab45-c06fe21e7bbd-metrics-tls\") pod \"dns-default-9z4vw\" (UID: \"57e9d0f5-d16a-42d0-ab45-c06fe21e7bbd\") " pod="openshift-dns/dns-default-9z4vw"
Sep 17 23:15:33.087126 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:15:33.087087    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/57e9d0f5-d16a-42d0-ab45-c06fe21e7bbd-metrics-tls\") pod \"dns-default-9z4vw\" (UID: \"57e9d0f5-d16a-42d0-ab45-c06fe21e7bbd\") " pod="openshift-dns/dns-default-9z4vw"
Sep 17 23:15:33.126651 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:15:33.126615    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fwwc5_7e452bbb-7dc6-4c9d-a9b1-76400942f5b4/konnectivity-agent/0.log" limit=1024
Sep 17 23:15:33.183343 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:15:33.183309    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fwwc5_7e452bbb-7dc6-4c9d-a9b1-76400942f5b4/konnectivity-agent/0.log" limit=1024
Sep 17 23:15:33.313896 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:15:33.313869    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-cvr9f\""
Sep 17 23:15:33.321838 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:15:33.321819    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-9z4vw"
Sep 17 23:15:33.591831 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:15:33.591800    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-9z4vw"]
Sep 17 23:15:33.594918 ip-10-0-132-188 kubenswrapper[2562]: W0917 23:15:33.594889    2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod57e9d0f5_d16a_42d0_ab45_c06fe21e7bbd.slice/crio-5f1846207e15f1f28c6970af1bf677b0b53917ff3810231036113022bd72b947 WatchSource:0}: Error finding container 5f1846207e15f1f28c6970af1bf677b0b53917ff3810231036113022bd72b947: Status 404 returned error can't find the container with id 5f1846207e15f1f28c6970af1bf677b0b53917ff3810231036113022bd72b947
Sep 17 23:15:33.730611 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:15:33.730503    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-9z4vw" event={"ID":"57e9d0f5-d16a-42d0-ab45-c06fe21e7bbd","Type":"ContainerStarted","Data":"5f1846207e15f1f28c6970af1bf677b0b53917ff3810231036113022bd72b947"}
Sep 17 23:15:35.739524 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:15:35.739490    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-9z4vw" event={"ID":"57e9d0f5-d16a-42d0-ab45-c06fe21e7bbd","Type":"ContainerStarted","Data":"bf8943c334f0f7c8b9434bc256a4111247d6e86da0ed025dff9cf9c5a79565ba"}
Sep 17 23:15:35.739904 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:15:35.739541    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-9z4vw" event={"ID":"57e9d0f5-d16a-42d0-ab45-c06fe21e7bbd","Type":"ContainerStarted","Data":"2d14645f7beef3369efce51d8da9c8ba8bb2b0e85ea8357753958bc491766196"}
Sep 17 23:15:35.739904 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:15:35.739592    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-dns/dns-default-9z4vw"
Sep 17 23:15:35.756007 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:15:35.755948    2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/dns-default-9z4vw" podStartSLOduration=251.198600246 podStartE2EDuration="4m12.755930917s" totalImagesPullingTime="1.557330671s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-17 23:11:23 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-17 23:15:35.755213902 +0000 UTC m=+288.157110667" watchObservedRunningTime="2026-09-17 23:15:35.755930917 +0000 UTC m=+288.157827717"
Sep 17 23:15:36.920947 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:15:36.920899    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/metrics-server-5d89bb6d78-5kftc"
Sep 17 23:15:36.921376 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:15:36.920970    2562 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-monitoring/metrics-server-5d89bb6d78-5kftc"
Sep 17 23:15:38.776162 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:15:38.776132    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fwwc5_7e452bbb-7dc6-4c9d-a9b1-76400942f5b4/konnectivity-agent/0.log" limit=1024
Sep 17 23:15:39.954331 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:15:39.954287    2562 patch_prober.go:28] interesting pod/image-registry-64c75f9db5-qc6hf container/registry namespace/openshift-image-registry: Liveness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body={"errors":[{"code":"UNAVAILABLE","message":"service unavailable","detail":"health check failed: please see /debug/health"}]}
Sep 17 23:15:39.954847 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:15:39.954347    2562 prober.go:120] "Probe failed" probeType="Liveness" pod="openshift-image-registry/image-registry-64c75f9db5-qc6hf" podUID="51d624a1-203a-4888-be49-4a3c375984a3" containerName="registry" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Sep 17 23:15:39.954847 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:15:39.954392    2562 kubelet.go:2654] "SyncLoop (probe)" probe="liveness" status="unhealthy" pod="openshift-image-registry/image-registry-64c75f9db5-qc6hf"
Sep 17 23:15:39.954985 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:15:39.954961    2562 kuberuntime_manager.go:1192] "Message for Container of pod" containerName="registry" containerStatusID={"Type":"cri-o","ID":"1b840b07dde15d8d80599f875486b5027a34d845148bba04d761834e98eee6dd"} pod="openshift-image-registry/image-registry-64c75f9db5-qc6hf" containerMessage="Container registry failed liveness probe, will be restarted"
Sep 17 23:15:39.958556 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:15:39.958532    2562 patch_prober.go:28] interesting pod/image-registry-64c75f9db5-qc6hf container/registry namespace/openshift-image-registry: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body={"errors":[{"code":"UNAVAILABLE","message":"service unavailable","detail":"health check failed: please see /debug/health"}]}
Sep 17 23:15:39.958666 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:15:39.958572    2562 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-image-registry/image-registry-64c75f9db5-qc6hf" podUID="51d624a1-203a-4888-be49-4a3c375984a3" containerName="registry" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Sep 17 23:15:39.974751 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:15:39.974720    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fwwc5_7e452bbb-7dc6-4c9d-a9b1-76400942f5b4/konnectivity-agent/0.log" limit=1024
Sep 17 23:15:40.075889 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:15:40.075841    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fwwc5_7e452bbb-7dc6-4c9d-a9b1-76400942f5b4/konnectivity-agent/0.log" limit=1024
Sep 17 23:15:40.742064 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:15:40.742028    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fwwc5_7e452bbb-7dc6-4c9d-a9b1-76400942f5b4/konnectivity-agent/0.log" limit=1024
Sep 17 23:15:40.843057 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:15:40.843020    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fwwc5_7e452bbb-7dc6-4c9d-a9b1-76400942f5b4/konnectivity-agent/0.log" limit=1024
Sep 17 23:15:43.170768 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:15:43.170731    2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-6f579bdb4b-9glh9"]
Sep 17 23:15:43.175222 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:15:43.175198    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-6f579bdb4b-9glh9"
Sep 17 23:15:43.178424 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:15:43.178397    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-serving-cert\""
Sep 17 23:15:43.178549 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:15:43.178478    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-dockercfg-hjfdq\""
Sep 17 23:15:43.178549 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:15:43.178492    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"service-ca\""
Sep 17 23:15:43.178549 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:15:43.178523    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"openshift-service-ca.crt\""
Sep 17 23:15:43.178749 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:15:43.178633    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"kube-root-ca.crt\""
Sep 17 23:15:43.178749 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:15:43.178668    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"console-config\""
Sep 17 23:15:43.179027 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:15:43.179010    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-oauth-config\""
Sep 17 23:15:43.179088 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:15:43.179038    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"oauth-serving-cert\""
Sep 17 23:15:43.182856 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:15:43.182832    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-6f579bdb4b-9glh9"]
Sep 17 23:15:43.183064 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:15:43.183046    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"trusted-ca-bundle\""
Sep 17 23:15:43.266642 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:15:43.266596    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/40e3987b-5dac-4ecc-90ca-8b703295acd7-console-serving-cert\") pod \"console-6f579bdb4b-9glh9\" (UID: \"40e3987b-5dac-4ecc-90ca-8b703295acd7\") " pod="openshift-console/console-6f579bdb4b-9glh9"
Sep 17 23:15:43.266828 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:15:43.266659    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/40e3987b-5dac-4ecc-90ca-8b703295acd7-oauth-serving-cert\") pod \"console-6f579bdb4b-9glh9\" (UID: \"40e3987b-5dac-4ecc-90ca-8b703295acd7\") " pod="openshift-console/console-6f579bdb4b-9glh9"
Sep 17 23:15:43.266828 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:15:43.266687    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/40e3987b-5dac-4ecc-90ca-8b703295acd7-service-ca\") pod \"console-6f579bdb4b-9glh9\" (UID: \"40e3987b-5dac-4ecc-90ca-8b703295acd7\") " pod="openshift-console/console-6f579bdb4b-9glh9"
Sep 17 23:15:43.266828 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:15:43.266713    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-r42pv\" (UniqueName: \"kubernetes.io/projected/40e3987b-5dac-4ecc-90ca-8b703295acd7-kube-api-access-r42pv\") pod \"console-6f579bdb4b-9glh9\" (UID: \"40e3987b-5dac-4ecc-90ca-8b703295acd7\") " pod="openshift-console/console-6f579bdb4b-9glh9"
Sep 17 23:15:43.266992 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:15:43.266847    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/40e3987b-5dac-4ecc-90ca-8b703295acd7-trusted-ca-bundle\") pod \"console-6f579bdb4b-9glh9\" (UID: \"40e3987b-5dac-4ecc-90ca-8b703295acd7\") " pod="openshift-console/console-6f579bdb4b-9glh9"
Sep 17 23:15:43.266992 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:15:43.266892    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/40e3987b-5dac-4ecc-90ca-8b703295acd7-console-config\") pod \"console-6f579bdb4b-9glh9\" (UID: \"40e3987b-5dac-4ecc-90ca-8b703295acd7\") " pod="openshift-console/console-6f579bdb4b-9glh9"
Sep 17 23:15:43.266992 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:15:43.266926    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/40e3987b-5dac-4ecc-90ca-8b703295acd7-console-oauth-config\") pod \"console-6f579bdb4b-9glh9\" (UID: \"40e3987b-5dac-4ecc-90ca-8b703295acd7\") " pod="openshift-console/console-6f579bdb4b-9glh9"
Sep 17 23:15:43.368036 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:15:43.367991    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/40e3987b-5dac-4ecc-90ca-8b703295acd7-trusted-ca-bundle\") pod \"console-6f579bdb4b-9glh9\" (UID: \"40e3987b-5dac-4ecc-90ca-8b703295acd7\") " pod="openshift-console/console-6f579bdb4b-9glh9"
Sep 17 23:15:43.368036 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:15:43.368048    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/40e3987b-5dac-4ecc-90ca-8b703295acd7-console-config\") pod \"console-6f579bdb4b-9glh9\" (UID: \"40e3987b-5dac-4ecc-90ca-8b703295acd7\") " pod="openshift-console/console-6f579bdb4b-9glh9"
Sep 17 23:15:43.368305 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:15:43.368075    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/40e3987b-5dac-4ecc-90ca-8b703295acd7-console-oauth-config\") pod \"console-6f579bdb4b-9glh9\" (UID: \"40e3987b-5dac-4ecc-90ca-8b703295acd7\") " pod="openshift-console/console-6f579bdb4b-9glh9"
Sep 17 23:15:43.368305 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:15:43.368100    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/40e3987b-5dac-4ecc-90ca-8b703295acd7-console-serving-cert\") pod \"console-6f579bdb4b-9glh9\" (UID: \"40e3987b-5dac-4ecc-90ca-8b703295acd7\") " pod="openshift-console/console-6f579bdb4b-9glh9"
Sep 17 23:15:43.368305 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:15:43.368140    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/40e3987b-5dac-4ecc-90ca-8b703295acd7-oauth-serving-cert\") pod \"console-6f579bdb4b-9glh9\" (UID: \"40e3987b-5dac-4ecc-90ca-8b703295acd7\") " pod="openshift-console/console-6f579bdb4b-9glh9"
Sep 17 23:15:43.368305 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:15:43.368192    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/40e3987b-5dac-4ecc-90ca-8b703295acd7-service-ca\") pod \"console-6f579bdb4b-9glh9\" (UID: \"40e3987b-5dac-4ecc-90ca-8b703295acd7\") " pod="openshift-console/console-6f579bdb4b-9glh9"
Sep 17 23:15:43.368305 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:15:43.368224    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-r42pv\" (UniqueName: \"kubernetes.io/projected/40e3987b-5dac-4ecc-90ca-8b703295acd7-kube-api-access-r42pv\") pod \"console-6f579bdb4b-9glh9\" (UID: \"40e3987b-5dac-4ecc-90ca-8b703295acd7\") " pod="openshift-console/console-6f579bdb4b-9glh9"
Sep 17 23:15:43.368935 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:15:43.368906    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/40e3987b-5dac-4ecc-90ca-8b703295acd7-console-config\") pod \"console-6f579bdb4b-9glh9\" (UID: \"40e3987b-5dac-4ecc-90ca-8b703295acd7\") " pod="openshift-console/console-6f579bdb4b-9glh9"
Sep 17 23:15:43.368935 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:15:43.368909    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/40e3987b-5dac-4ecc-90ca-8b703295acd7-service-ca\") pod \"console-6f579bdb4b-9glh9\" (UID: \"40e3987b-5dac-4ecc-90ca-8b703295acd7\") " pod="openshift-console/console-6f579bdb4b-9glh9"
Sep 17 23:15:43.369138 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:15:43.368989    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/40e3987b-5dac-4ecc-90ca-8b703295acd7-oauth-serving-cert\") pod \"console-6f579bdb4b-9glh9\" (UID: \"40e3987b-5dac-4ecc-90ca-8b703295acd7\") " pod="openshift-console/console-6f579bdb4b-9glh9"
Sep 17 23:15:43.369138 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:15:43.368996    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/40e3987b-5dac-4ecc-90ca-8b703295acd7-trusted-ca-bundle\") pod \"console-6f579bdb4b-9glh9\" (UID: \"40e3987b-5dac-4ecc-90ca-8b703295acd7\") " pod="openshift-console/console-6f579bdb4b-9glh9"
Sep 17 23:15:43.370681 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:15:43.370658    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/40e3987b-5dac-4ecc-90ca-8b703295acd7-console-serving-cert\") pod \"console-6f579bdb4b-9glh9\" (UID: \"40e3987b-5dac-4ecc-90ca-8b703295acd7\") " pod="openshift-console/console-6f579bdb4b-9glh9"
Sep 17 23:15:43.370792 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:15:43.370719    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/40e3987b-5dac-4ecc-90ca-8b703295acd7-console-oauth-config\") pod \"console-6f579bdb4b-9glh9\" (UID: \"40e3987b-5dac-4ecc-90ca-8b703295acd7\") " pod="openshift-console/console-6f579bdb4b-9glh9"
Sep 17 23:15:43.375902 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:15:43.375874    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fwwc5_7e452bbb-7dc6-4c9d-a9b1-76400942f5b4/konnectivity-agent/0.log" limit=1024
Sep 17 23:15:43.376928 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:15:43.376909    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-r42pv\" (UniqueName: \"kubernetes.io/projected/40e3987b-5dac-4ecc-90ca-8b703295acd7-kube-api-access-r42pv\") pod \"console-6f579bdb4b-9glh9\" (UID: \"40e3987b-5dac-4ecc-90ca-8b703295acd7\") " pod="openshift-console/console-6f579bdb4b-9glh9"
Sep 17 23:15:43.429603 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:15:43.429512    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fwwc5_7e452bbb-7dc6-4c9d-a9b1-76400942f5b4/konnectivity-agent/0.log" limit=1024
Sep 17 23:15:43.486742 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:15:43.486710    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-6f579bdb4b-9glh9"
Sep 17 23:15:43.758206 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:15:43.758111    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-6f579bdb4b-9glh9"]
Sep 17 23:15:43.761349 ip-10-0-132-188 kubenswrapper[2562]: W0917 23:15:43.761319    2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod40e3987b_5dac_4ecc_90ca_8b703295acd7.slice/crio-e2e20717658ccc1c6eff759f217e54dc55258ecf2471993d7ec87a9d9599365d WatchSource:0}: Error finding container e2e20717658ccc1c6eff759f217e54dc55258ecf2471993d7ec87a9d9599365d: Status 404 returned error can't find the container with id e2e20717658ccc1c6eff759f217e54dc55258ecf2471993d7ec87a9d9599365d
Sep 17 23:15:43.770022 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:15:43.769995    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-6f579bdb4b-9glh9" event={"ID":"40e3987b-5dac-4ecc-90ca-8b703295acd7","Type":"ContainerStarted","Data":"e2e20717658ccc1c6eff759f217e54dc55258ecf2471993d7ec87a9d9599365d"}
Sep 17 23:15:45.590458 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:15:45.590408    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fwwc5_7e452bbb-7dc6-4c9d-a9b1-76400942f5b4/konnectivity-agent/0.log" limit=1024
Sep 17 23:15:45.646492 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:15:45.646449    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fwwc5_7e452bbb-7dc6-4c9d-a9b1-76400942f5b4/konnectivity-agent/0.log" limit=1024
Sep 17 23:15:45.713211 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:15:45.713160    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fwwc5_7e452bbb-7dc6-4c9d-a9b1-76400942f5b4/konnectivity-agent/0.log" limit=1024
Sep 17 23:15:45.746503 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:15:45.746472    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-dns/dns-default-9z4vw"
Sep 17 23:15:45.768889 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:15:45.768855    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fwwc5_7e452bbb-7dc6-4c9d-a9b1-76400942f5b4/konnectivity-agent/0.log" limit=1024
Sep 17 23:15:47.789711 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:15:47.789670    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-6f579bdb4b-9glh9" event={"ID":"40e3987b-5dac-4ecc-90ca-8b703295acd7","Type":"ContainerStarted","Data":"157c9bb3b591bc0f70c74abad6519737ba07624b54f7b374adb7db6441ce7275"}
Sep 17 23:15:47.807269 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:15:47.807209    2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-6f579bdb4b-9glh9" podStartSLOduration=0.874050056 podStartE2EDuration="4.807191389s" totalImagesPullingTime="3.933141333s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-17 23:15:43 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-17 23:15:47.805446565 +0000 UTC m=+300.207343364" watchObservedRunningTime="2026-09-17 23:15:47.807191389 +0000 UTC m=+300.209088145"
Sep 17 23:15:48.382189 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:15:48.382133    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fwwc5_7e452bbb-7dc6-4c9d-a9b1-76400942f5b4/konnectivity-agent/0.log" limit=1024
Sep 17 23:15:49.558879 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:15:49.558849    2562 kubelet.go:1648] "Image garbage collection succeeded"
Sep 17 23:15:49.958865 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:15:49.958831    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-image-registry/image-registry-64c75f9db5-qc6hf"
Sep 17 23:15:50.074638 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:15:50.074590    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fwwc5_7e452bbb-7dc6-4c9d-a9b1-76400942f5b4/konnectivity-agent/0.log" limit=1024
Sep 17 23:15:50.175901 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:15:50.175865    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fwwc5_7e452bbb-7dc6-4c9d-a9b1-76400942f5b4/konnectivity-agent/0.log" limit=1024
Sep 17 23:15:50.276515 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:15:50.276426    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fwwc5_7e452bbb-7dc6-4c9d-a9b1-76400942f5b4/konnectivity-agent/0.log" limit=1024
Sep 17 23:15:51.590446 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:15:51.590408    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fwwc5_7e452bbb-7dc6-4c9d-a9b1-76400942f5b4/konnectivity-agent/0.log" limit=1024
Sep 17 23:15:51.690498 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:15:51.690455    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fwwc5_7e452bbb-7dc6-4c9d-a9b1-76400942f5b4/konnectivity-agent/0.log" limit=1024
Sep 17 23:15:52.533002 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:15:52.532966    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fwwc5_7e452bbb-7dc6-4c9d-a9b1-76400942f5b4/konnectivity-agent/0.log" limit=1024
Sep 17 23:15:52.591806 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:15:52.591776    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fwwc5_7e452bbb-7dc6-4c9d-a9b1-76400942f5b4/konnectivity-agent/0.log" limit=1024
Sep 17 23:15:53.487161 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:15:53.487115    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-6f579bdb4b-9glh9"
Sep 17 23:15:53.487357 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:15:53.487213    2562 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-6f579bdb4b-9glh9"
Sep 17 23:15:53.492028 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:15:53.492007    2562 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-6f579bdb4b-9glh9"
Sep 17 23:15:53.813748 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:15:53.813663    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-6f579bdb4b-9glh9"
Sep 17 23:15:53.955264 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:15:53.955224    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fwwc5_7e452bbb-7dc6-4c9d-a9b1-76400942f5b4/konnectivity-agent/0.log" limit=1024
Sep 17 23:15:54.057444 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:15:54.057408    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fwwc5_7e452bbb-7dc6-4c9d-a9b1-76400942f5b4/konnectivity-agent/0.log" limit=1024
Sep 17 23:15:54.119443 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:15:54.119412    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fwwc5_7e452bbb-7dc6-4c9d-a9b1-76400942f5b4/konnectivity-agent/0.log" limit=1024
Sep 17 23:15:54.721459 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:15:54.721401    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fwwc5_7e452bbb-7dc6-4c9d-a9b1-76400942f5b4/konnectivity-agent/0.log" limit=1024
Sep 17 23:15:54.823916 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:15:54.823883    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fwwc5_7e452bbb-7dc6-4c9d-a9b1-76400942f5b4/konnectivity-agent/0.log" limit=1024
Sep 17 23:15:54.923286 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:15:54.923242    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fwwc5_7e452bbb-7dc6-4c9d-a9b1-76400942f5b4/konnectivity-agent/0.log" limit=1024
Sep 17 23:15:56.927055 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:15:56.927027    2562 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-monitoring/metrics-server-5d89bb6d78-5kftc"
Sep 17 23:15:56.930989 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:15:56.930960    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/metrics-server-5d89bb6d78-5kftc"
Sep 17 23:15:59.978630 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:15:59.978583    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fwwc5_7e452bbb-7dc6-4c9d-a9b1-76400942f5b4/konnectivity-agent/0.log" limit=1024
Sep 17 23:16:00.079144 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:16:00.079109    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fwwc5_7e452bbb-7dc6-4c9d-a9b1-76400942f5b4/konnectivity-agent/0.log" limit=1024
Sep 17 23:16:00.689162 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:16:00.689132    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fwwc5_7e452bbb-7dc6-4c9d-a9b1-76400942f5b4/konnectivity-agent/0.log" limit=1024
Sep 17 23:16:00.746181 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:16:00.746147    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fwwc5_7e452bbb-7dc6-4c9d-a9b1-76400942f5b4/konnectivity-agent/0.log" limit=1024
Sep 17 23:16:04.973788 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:16:04.973740    2562 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-image-registry/image-registry-64c75f9db5-qc6hf" podUID="51d624a1-203a-4888-be49-4a3c375984a3" containerName="registry" containerID="cri-o://1b840b07dde15d8d80599f875486b5027a34d845148bba04d761834e98eee6dd" gracePeriod=30
Sep 17 23:16:06.854804 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:16:06.854768    2562 generic.go:356] "Generic (PLEG): container finished" podID="51d624a1-203a-4888-be49-4a3c375984a3" containerID="1b840b07dde15d8d80599f875486b5027a34d845148bba04d761834e98eee6dd" exitCode=0
Sep 17 23:16:06.855256 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:16:06.854840    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-64c75f9db5-qc6hf" event={"ID":"51d624a1-203a-4888-be49-4a3c375984a3","Type":"ContainerDied","Data":"1b840b07dde15d8d80599f875486b5027a34d845148bba04d761834e98eee6dd"}
Sep 17 23:16:06.855256 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:16:06.854900    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-64c75f9db5-qc6hf" event={"ID":"51d624a1-203a-4888-be49-4a3c375984a3","Type":"ContainerStarted","Data":"79d8277f47be67b67047cdd93ef76f2bc399e2bec9e2de866f84f3624e1b583e"}
Sep 17 23:16:06.855256 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:16:06.854947    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-image-registry/image-registry-64c75f9db5-qc6hf"
Sep 17 23:16:07.081963 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:16:07.081919    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fwwc5_7e452bbb-7dc6-4c9d-a9b1-76400942f5b4/konnectivity-agent/0.log" limit=1024
Sep 17 23:16:07.182929 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:16:07.182837    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fwwc5_7e452bbb-7dc6-4c9d-a9b1-76400942f5b4/konnectivity-agent/0.log" limit=1024
Sep 17 23:16:14.976020 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:16:14.975986    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fwwc5_7e452bbb-7dc6-4c9d-a9b1-76400942f5b4/konnectivity-agent/0.log" limit=1024
Sep 17 23:16:15.049051 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:16:15.049016    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fwwc5_7e452bbb-7dc6-4c9d-a9b1-76400942f5b4/konnectivity-agent/0.log" limit=1024
Sep 17 23:16:22.214303 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:16:22.214265    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fwwc5_7e452bbb-7dc6-4c9d-a9b1-76400942f5b4/konnectivity-agent/0.log" limit=1024
Sep 17 23:16:22.312583 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:16:22.312543    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fwwc5_7e452bbb-7dc6-4c9d-a9b1-76400942f5b4/konnectivity-agent/0.log" limit=1024
Sep 17 23:16:22.365815 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:16:22.365783    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fwwc5_7e452bbb-7dc6-4c9d-a9b1-76400942f5b4/konnectivity-agent/0.log" limit=1024
Sep 17 23:16:27.863240 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:16:27.863212    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-image-registry/image-registry-64c75f9db5-qc6hf"
Sep 17 23:16:28.080058 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:16:28.080021    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fwwc5_7e452bbb-7dc6-4c9d-a9b1-76400942f5b4/konnectivity-agent/0.log" limit=1024
Sep 17 23:16:28.145041 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:16:28.144963    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fwwc5_7e452bbb-7dc6-4c9d-a9b1-76400942f5b4/konnectivity-agent/0.log" limit=1024
Sep 17 23:16:28.346493 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:16:28.346454    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fwwc5_7e452bbb-7dc6-4c9d-a9b1-76400942f5b4/konnectivity-agent/0.log" limit=1024
Sep 17 23:16:28.448895 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:16:28.448799    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fwwc5_7e452bbb-7dc6-4c9d-a9b1-76400942f5b4/konnectivity-agent/0.log" limit=1024
Sep 17 23:16:29.975095 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:16:29.975056    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fwwc5_7e452bbb-7dc6-4c9d-a9b1-76400942f5b4/konnectivity-agent/0.log" limit=1024
Sep 17 23:16:30.074610 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:16:30.074579    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fwwc5_7e452bbb-7dc6-4c9d-a9b1-76400942f5b4/konnectivity-agent/0.log" limit=1024
Sep 17 23:16:37.454998 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:16:37.454911    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fwwc5_7e452bbb-7dc6-4c9d-a9b1-76400942f5b4/konnectivity-agent/0.log" limit=1024
Sep 17 23:16:37.504242 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:16:37.504212    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fwwc5_7e452bbb-7dc6-4c9d-a9b1-76400942f5b4/konnectivity-agent/0.log" limit=1024
Sep 17 23:16:37.595912 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:16:37.595876    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fwwc5_7e452bbb-7dc6-4c9d-a9b1-76400942f5b4/konnectivity-agent/0.log" limit=1024
Sep 17 23:16:37.656689 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:16:37.656648    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fwwc5_7e452bbb-7dc6-4c9d-a9b1-76400942f5b4/konnectivity-agent/0.log" limit=1024
Sep 17 23:16:38.192603 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:16:38.192573    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fwwc5_7e452bbb-7dc6-4c9d-a9b1-76400942f5b4/konnectivity-agent/0.log" limit=1024
Sep 17 23:16:38.247161 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:16:38.247117    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fwwc5_7e452bbb-7dc6-4c9d-a9b1-76400942f5b4/konnectivity-agent/0.log" limit=1024
Sep 17 23:16:42.096735 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:16:42.096691    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fwwc5_7e452bbb-7dc6-4c9d-a9b1-76400942f5b4/konnectivity-agent/0.log" limit=1024
Sep 17 23:16:42.146686 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:16:42.146652    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fwwc5_7e452bbb-7dc6-4c9d-a9b1-76400942f5b4/konnectivity-agent/0.log" limit=1024
Sep 17 23:16:44.985537 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:16:44.985503    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fwwc5_7e452bbb-7dc6-4c9d-a9b1-76400942f5b4/konnectivity-agent/0.log" limit=1024
Sep 17 23:16:45.086501 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:16:45.086466    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fwwc5_7e452bbb-7dc6-4c9d-a9b1-76400942f5b4/konnectivity-agent/0.log" limit=1024
Sep 17 23:16:47.896517 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:16:47.896475    2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-6f579bdb4b-9glh9"]
Sep 17 23:16:48.038698 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:16:48.038662    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fwwc5_7e452bbb-7dc6-4c9d-a9b1-76400942f5b4/konnectivity-agent/0.log" limit=1024
Sep 17 23:16:48.140111 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:16:48.140072    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fwwc5_7e452bbb-7dc6-4c9d-a9b1-76400942f5b4/konnectivity-agent/0.log" limit=1024
Sep 17 23:17:07.533435 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:17:07.533400    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fwwc5_7e452bbb-7dc6-4c9d-a9b1-76400942f5b4/konnectivity-agent/0.log" limit=1024
Sep 17 23:17:12.915757 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:17:12.915715    2562 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-console/console-6f579bdb4b-9glh9" podUID="40e3987b-5dac-4ecc-90ca-8b703295acd7" containerName="console" containerID="cri-o://157c9bb3b591bc0f70c74abad6519737ba07624b54f7b374adb7db6441ce7275" gracePeriod=15
Sep 17 23:17:13.092558 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:17:13.092527    2562 generic.go:356] "Generic (PLEG): container finished" podID="40e3987b-5dac-4ecc-90ca-8b703295acd7" containerID="157c9bb3b591bc0f70c74abad6519737ba07624b54f7b374adb7db6441ce7275" exitCode=0
Sep 17 23:17:13.092966 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:17:13.092577    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-6f579bdb4b-9glh9" event={"ID":"40e3987b-5dac-4ecc-90ca-8b703295acd7","Type":"ContainerDied","Data":"157c9bb3b591bc0f70c74abad6519737ba07624b54f7b374adb7db6441ce7275"}
Sep 17 23:17:13.151952 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:17:13.151923    2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-6f579bdb4b-9glh9"
Sep 17 23:17:13.255143 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:17:13.255052    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/40e3987b-5dac-4ecc-90ca-8b703295acd7-trusted-ca-bundle\") pod \"40e3987b-5dac-4ecc-90ca-8b703295acd7\" (UID: \"40e3987b-5dac-4ecc-90ca-8b703295acd7\") "
Sep 17 23:17:13.255143 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:17:13.255094    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/40e3987b-5dac-4ecc-90ca-8b703295acd7-console-oauth-config\") pod \"40e3987b-5dac-4ecc-90ca-8b703295acd7\" (UID: \"40e3987b-5dac-4ecc-90ca-8b703295acd7\") "
Sep 17 23:17:13.255143 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:17:13.255124    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/40e3987b-5dac-4ecc-90ca-8b703295acd7-console-config\") pod \"40e3987b-5dac-4ecc-90ca-8b703295acd7\" (UID: \"40e3987b-5dac-4ecc-90ca-8b703295acd7\") "
Sep 17 23:17:13.255459 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:17:13.255209    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/40e3987b-5dac-4ecc-90ca-8b703295acd7-service-ca\") pod \"40e3987b-5dac-4ecc-90ca-8b703295acd7\" (UID: \"40e3987b-5dac-4ecc-90ca-8b703295acd7\") "
Sep 17 23:17:13.255459 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:17:13.255236    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/40e3987b-5dac-4ecc-90ca-8b703295acd7-console-serving-cert\") pod \"40e3987b-5dac-4ecc-90ca-8b703295acd7\" (UID: \"40e3987b-5dac-4ecc-90ca-8b703295acd7\") "
Sep 17 23:17:13.255459 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:17:13.255251    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-r42pv\" (UniqueName: \"kubernetes.io/projected/40e3987b-5dac-4ecc-90ca-8b703295acd7-kube-api-access-r42pv\") pod \"40e3987b-5dac-4ecc-90ca-8b703295acd7\" (UID: \"40e3987b-5dac-4ecc-90ca-8b703295acd7\") "
Sep 17 23:17:13.255459 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:17:13.255373    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/40e3987b-5dac-4ecc-90ca-8b703295acd7-oauth-serving-cert\") pod \"40e3987b-5dac-4ecc-90ca-8b703295acd7\" (UID: \"40e3987b-5dac-4ecc-90ca-8b703295acd7\") "
Sep 17 23:17:13.255667 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:17:13.255558    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/40e3987b-5dac-4ecc-90ca-8b703295acd7-trusted-ca-bundle" (OuterVolumeSpecName: "trusted-ca-bundle") pod "40e3987b-5dac-4ecc-90ca-8b703295acd7" (UID: "40e3987b-5dac-4ecc-90ca-8b703295acd7"). InnerVolumeSpecName "trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 17 23:17:13.255667 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:17:13.255634    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/40e3987b-5dac-4ecc-90ca-8b703295acd7-console-config" (OuterVolumeSpecName: "console-config") pod "40e3987b-5dac-4ecc-90ca-8b703295acd7" (UID: "40e3987b-5dac-4ecc-90ca-8b703295acd7"). InnerVolumeSpecName "console-config". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 17 23:17:13.255743 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:17:13.255708    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/40e3987b-5dac-4ecc-90ca-8b703295acd7-service-ca" (OuterVolumeSpecName: "service-ca") pod "40e3987b-5dac-4ecc-90ca-8b703295acd7" (UID: "40e3987b-5dac-4ecc-90ca-8b703295acd7"). InnerVolumeSpecName "service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 17 23:17:13.255804 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:17:13.255738    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/40e3987b-5dac-4ecc-90ca-8b703295acd7-oauth-serving-cert" (OuterVolumeSpecName: "oauth-serving-cert") pod "40e3987b-5dac-4ecc-90ca-8b703295acd7" (UID: "40e3987b-5dac-4ecc-90ca-8b703295acd7"). InnerVolumeSpecName "oauth-serving-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 17 23:17:13.257710 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:17:13.257671    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/40e3987b-5dac-4ecc-90ca-8b703295acd7-console-serving-cert" (OuterVolumeSpecName: "console-serving-cert") pod "40e3987b-5dac-4ecc-90ca-8b703295acd7" (UID: "40e3987b-5dac-4ecc-90ca-8b703295acd7"). InnerVolumeSpecName "console-serving-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 17 23:17:13.257842 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:17:13.257711    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/40e3987b-5dac-4ecc-90ca-8b703295acd7-console-oauth-config" (OuterVolumeSpecName: "console-oauth-config") pod "40e3987b-5dac-4ecc-90ca-8b703295acd7" (UID: "40e3987b-5dac-4ecc-90ca-8b703295acd7"). InnerVolumeSpecName "console-oauth-config". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 17 23:17:13.257842 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:17:13.257772    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/40e3987b-5dac-4ecc-90ca-8b703295acd7-kube-api-access-r42pv" (OuterVolumeSpecName: "kube-api-access-r42pv") pod "40e3987b-5dac-4ecc-90ca-8b703295acd7" (UID: "40e3987b-5dac-4ecc-90ca-8b703295acd7"). InnerVolumeSpecName "kube-api-access-r42pv". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 17 23:17:13.356468 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:17:13.356430    2562 reconciler_common.go:299] "Volume detached for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/40e3987b-5dac-4ecc-90ca-8b703295acd7-console-serving-cert\") on node \"ip-10-0-132-188.ec2.internal\" DevicePath \"\""
Sep 17 23:17:13.356468 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:17:13.356464    2562 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-r42pv\" (UniqueName: \"kubernetes.io/projected/40e3987b-5dac-4ecc-90ca-8b703295acd7-kube-api-access-r42pv\") on node \"ip-10-0-132-188.ec2.internal\" DevicePath \"\""
Sep 17 23:17:13.356468 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:17:13.356474    2562 reconciler_common.go:299] "Volume detached for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/40e3987b-5dac-4ecc-90ca-8b703295acd7-oauth-serving-cert\") on node \"ip-10-0-132-188.ec2.internal\" DevicePath \"\""
Sep 17 23:17:13.356716 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:17:13.356483    2562 reconciler_common.go:299] "Volume detached for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/40e3987b-5dac-4ecc-90ca-8b703295acd7-trusted-ca-bundle\") on node \"ip-10-0-132-188.ec2.internal\" DevicePath \"\""
Sep 17 23:17:13.356716 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:17:13.356491    2562 reconciler_common.go:299] "Volume detached for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/40e3987b-5dac-4ecc-90ca-8b703295acd7-console-oauth-config\") on node \"ip-10-0-132-188.ec2.internal\" DevicePath \"\""
Sep 17 23:17:13.356716 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:17:13.356500    2562 reconciler_common.go:299] "Volume detached for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/40e3987b-5dac-4ecc-90ca-8b703295acd7-console-config\") on node \"ip-10-0-132-188.ec2.internal\" DevicePath \"\""
Sep 17 23:17:13.356716 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:17:13.356508    2562 reconciler_common.go:299] "Volume detached for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/40e3987b-5dac-4ecc-90ca-8b703295acd7-service-ca\") on node \"ip-10-0-132-188.ec2.internal\" DevicePath \"\""
Sep 17 23:17:14.098052 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:17:14.098015    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-6f579bdb4b-9glh9" event={"ID":"40e3987b-5dac-4ecc-90ca-8b703295acd7","Type":"ContainerDied","Data":"e2e20717658ccc1c6eff759f217e54dc55258ecf2471993d7ec87a9d9599365d"}
Sep 17 23:17:14.098522 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:17:14.098079    2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-6f579bdb4b-9glh9"
Sep 17 23:17:14.098522 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:17:14.098080    2562 scope.go:117] "RemoveContainer" containerID="157c9bb3b591bc0f70c74abad6519737ba07624b54f7b374adb7db6441ce7275"
Sep 17 23:17:14.113505 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:17:14.113477    2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-6f579bdb4b-9glh9"]
Sep 17 23:17:14.116859 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:17:14.116836    2562 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-console/console-6f579bdb4b-9glh9"]
Sep 17 23:17:15.669457 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:17:15.669423    2562 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="40e3987b-5dac-4ecc-90ca-8b703295acd7" path="/var/lib/kubelet/pods/40e3987b-5dac-4ecc-90ca-8b703295acd7/volumes"
Sep 17 23:17:36.820671 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:17:36.820619    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fwwc5_7e452bbb-7dc6-4c9d-a9b1-76400942f5b4/konnectivity-agent/0.log" limit=1024
Sep 17 23:17:36.920690 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:17:36.920630    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fwwc5_7e452bbb-7dc6-4c9d-a9b1-76400942f5b4/konnectivity-agent/0.log" limit=1024
Sep 17 23:17:36.971209 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:17:36.971154    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fwwc5_7e452bbb-7dc6-4c9d-a9b1-76400942f5b4/konnectivity-agent/0.log" limit=1024
Sep 17 23:17:50.595069 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:17:50.595033    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fwwc5_7e452bbb-7dc6-4c9d-a9b1-76400942f5b4/konnectivity-agent/0.log" limit=1024
Sep 17 23:17:58.710196 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:17:58.710139    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fwwc5_7e452bbb-7dc6-4c9d-a9b1-76400942f5b4/konnectivity-agent/0.log" limit=1024
Sep 17 23:17:58.807542 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:17:58.807506    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fwwc5_7e452bbb-7dc6-4c9d-a9b1-76400942f5b4/konnectivity-agent/0.log" limit=1024
Sep 17 23:18:03.554557 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:18:03.554478    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fwwc5_7e452bbb-7dc6-4c9d-a9b1-76400942f5b4/konnectivity-agent/0.log" limit=1024
Sep 17 23:18:03.831432 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:18:03.831396    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fwwc5_7e452bbb-7dc6-4c9d-a9b1-76400942f5b4/konnectivity-agent/0.log" limit=1024
Sep 17 23:18:04.026864 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:18:04.026829    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fwwc5_7e452bbb-7dc6-4c9d-a9b1-76400942f5b4/konnectivity-agent/0.log" limit=1024
Sep 17 23:18:04.029827 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:18:04.029802    2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-keda/keda-metrics-apiserver-77857b7b98-87k7w"]
Sep 17 23:18:04.030058 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:18:04.030047    2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="40e3987b-5dac-4ecc-90ca-8b703295acd7" containerName="console"
Sep 17 23:18:04.030114 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:18:04.030061    2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="40e3987b-5dac-4ecc-90ca-8b703295acd7" containerName="console"
Sep 17 23:18:04.030114 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:18:04.030111    2562 memory_manager.go:368] "RemoveStaleState removing state" podUID="40e3987b-5dac-4ecc-90ca-8b703295acd7" containerName="console"
Sep 17 23:18:04.032901 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:18:04.032886    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-keda/keda-metrics-apiserver-77857b7b98-87k7w"
Sep 17 23:18:04.035372 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:18:04.035338    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-keda\"/\"keda-metrics-apiserver-certs\""
Sep 17 23:18:04.035508 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:18:04.035377    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-keda\"/\"keda-ocp-cabundle\""
Sep 17 23:18:04.035508 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:18:04.035371    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-keda\"/\"kedaorg-certs\""
Sep 17 23:18:04.035508 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:18:04.035439    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-keda\"/\"openshift-service-ca.crt\""
Sep 17 23:18:04.035508 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:18:04.035342    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-keda\"/\"kube-root-ca.crt\""
Sep 17 23:18:04.036319 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:18:04.036305    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-keda\"/\"keda-operator-dockercfg-tnmsm\""
Sep 17 23:18:04.042912 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:18:04.042887    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-keda/keda-metrics-apiserver-77857b7b98-87k7w"]
Sep 17 23:18:04.126385 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:18:04.126289    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fwwc5_7e452bbb-7dc6-4c9d-a9b1-76400942f5b4/konnectivity-agent/0.log" limit=1024
Sep 17 23:18:04.149294 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:18:04.149261    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2b6l2\" (UniqueName: \"kubernetes.io/projected/763cdefe-5de7-4a6d-856a-9048425082fb-kube-api-access-2b6l2\") pod \"keda-metrics-apiserver-77857b7b98-87k7w\" (UID: \"763cdefe-5de7-4a6d-856a-9048425082fb\") " pod="openshift-keda/keda-metrics-apiserver-77857b7b98-87k7w"
Sep 17 23:18:04.149472 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:18:04.149309    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"certificates\" (UniqueName: \"kubernetes.io/projected/763cdefe-5de7-4a6d-856a-9048425082fb-certificates\") pod \"keda-metrics-apiserver-77857b7b98-87k7w\" (UID: \"763cdefe-5de7-4a6d-856a-9048425082fb\") " pod="openshift-keda/keda-metrics-apiserver-77857b7b98-87k7w"
Sep 17 23:18:04.149472 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:18:04.149374    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"temp-vol\" (UniqueName: \"kubernetes.io/empty-dir/763cdefe-5de7-4a6d-856a-9048425082fb-temp-vol\") pod \"keda-metrics-apiserver-77857b7b98-87k7w\" (UID: \"763cdefe-5de7-4a6d-856a-9048425082fb\") " pod="openshift-keda/keda-metrics-apiserver-77857b7b98-87k7w"
Sep 17 23:18:04.233472 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:18:04.233440    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fwwc5_7e452bbb-7dc6-4c9d-a9b1-76400942f5b4/konnectivity-agent/0.log" limit=1024
Sep 17 23:18:04.250456 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:18:04.250420    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2b6l2\" (UniqueName: \"kubernetes.io/projected/763cdefe-5de7-4a6d-856a-9048425082fb-kube-api-access-2b6l2\") pod \"keda-metrics-apiserver-77857b7b98-87k7w\" (UID: \"763cdefe-5de7-4a6d-856a-9048425082fb\") " pod="openshift-keda/keda-metrics-apiserver-77857b7b98-87k7w"
Sep 17 23:18:04.250630 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:18:04.250463    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"certificates\" (UniqueName: \"kubernetes.io/projected/763cdefe-5de7-4a6d-856a-9048425082fb-certificates\") pod \"keda-metrics-apiserver-77857b7b98-87k7w\" (UID: \"763cdefe-5de7-4a6d-856a-9048425082fb\") " pod="openshift-keda/keda-metrics-apiserver-77857b7b98-87k7w"
Sep 17 23:18:04.250630 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:18:04.250585    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"temp-vol\" (UniqueName: \"kubernetes.io/empty-dir/763cdefe-5de7-4a6d-856a-9048425082fb-temp-vol\") pod \"keda-metrics-apiserver-77857b7b98-87k7w\" (UID: \"763cdefe-5de7-4a6d-856a-9048425082fb\") " pod="openshift-keda/keda-metrics-apiserver-77857b7b98-87k7w"
Sep 17 23:18:04.250956 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:18:04.250936    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"temp-vol\" (UniqueName: \"kubernetes.io/empty-dir/763cdefe-5de7-4a6d-856a-9048425082fb-temp-vol\") pod \"keda-metrics-apiserver-77857b7b98-87k7w\" (UID: \"763cdefe-5de7-4a6d-856a-9048425082fb\") " pod="openshift-keda/keda-metrics-apiserver-77857b7b98-87k7w"
Sep 17 23:18:04.253272 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:18:04.253248    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"certificates\" (UniqueName: \"kubernetes.io/projected/763cdefe-5de7-4a6d-856a-9048425082fb-certificates\") pod \"keda-metrics-apiserver-77857b7b98-87k7w\" (UID: \"763cdefe-5de7-4a6d-856a-9048425082fb\") " pod="openshift-keda/keda-metrics-apiserver-77857b7b98-87k7w"
Sep 17 23:18:04.260706 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:18:04.260678    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-2b6l2\" (UniqueName: \"kubernetes.io/projected/763cdefe-5de7-4a6d-856a-9048425082fb-kube-api-access-2b6l2\") pod \"keda-metrics-apiserver-77857b7b98-87k7w\" (UID: \"763cdefe-5de7-4a6d-856a-9048425082fb\") " pod="openshift-keda/keda-metrics-apiserver-77857b7b98-87k7w"
Sep 17 23:18:04.331385 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:18:04.331349    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fwwc5_7e452bbb-7dc6-4c9d-a9b1-76400942f5b4/konnectivity-agent/0.log" limit=1024
Sep 17 23:18:04.343213 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:18:04.343182    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-keda/keda-metrics-apiserver-77857b7b98-87k7w"
Sep 17 23:18:04.431642 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:18:04.431607    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fwwc5_7e452bbb-7dc6-4c9d-a9b1-76400942f5b4/konnectivity-agent/0.log" limit=1024
Sep 17 23:18:04.533220 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:18:04.533133    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fwwc5_7e452bbb-7dc6-4c9d-a9b1-76400942f5b4/konnectivity-agent/0.log" limit=1024
Sep 17 23:18:04.622158 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:18:04.622122    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-keda/keda-metrics-apiserver-77857b7b98-87k7w"]
Sep 17 23:18:04.624768 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:18:04.624738    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fwwc5_7e452bbb-7dc6-4c9d-a9b1-76400942f5b4/konnectivity-agent/0.log" limit=1024
Sep 17 23:18:04.625712 ip-10-0-132-188 kubenswrapper[2562]: W0917 23:18:04.625688    2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod763cdefe_5de7_4a6d_856a_9048425082fb.slice/crio-6e955e6963fc1dae4b61fef792698f49bf09c020facaa52198dace07f3cdc2d3 WatchSource:0}: Error finding container 6e955e6963fc1dae4b61fef792698f49bf09c020facaa52198dace07f3cdc2d3: Status 404 returned error can't find the container with id 6e955e6963fc1dae4b61fef792698f49bf09c020facaa52198dace07f3cdc2d3
Sep 17 23:18:04.627034 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:18:04.627018    2562 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Sep 17 23:18:04.766402 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:18:04.766320    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fwwc5_7e452bbb-7dc6-4c9d-a9b1-76400942f5b4/konnectivity-agent/0.log" limit=1024
Sep 17 23:18:04.945494 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:18:04.945463    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fwwc5_7e452bbb-7dc6-4c9d-a9b1-76400942f5b4/konnectivity-agent/0.log" limit=1024
Sep 17 23:18:05.048913 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:18:05.048827    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fwwc5_7e452bbb-7dc6-4c9d-a9b1-76400942f5b4/konnectivity-agent/0.log" limit=1024
Sep 17 23:18:05.279165 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:18:05.279134    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-keda/keda-metrics-apiserver-77857b7b98-87k7w" event={"ID":"763cdefe-5de7-4a6d-856a-9048425082fb","Type":"ContainerStarted","Data":"6e955e6963fc1dae4b61fef792698f49bf09c020facaa52198dace07f3cdc2d3"}
Sep 17 23:18:08.294601 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:18:08.294554    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-keda/keda-metrics-apiserver-77857b7b98-87k7w" event={"ID":"763cdefe-5de7-4a6d-856a-9048425082fb","Type":"ContainerStarted","Data":"43f54fd620b1cb72bdfea67ba80dfaa030c454071120140257bffd9bb91ce65e"}
Sep 17 23:18:08.294996 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:18:08.294697    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-keda/keda-metrics-apiserver-77857b7b98-87k7w"
Sep 17 23:18:08.312524 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:18:08.312473    2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-keda/keda-metrics-apiserver-77857b7b98-87k7w" podStartSLOduration=1.23456791 podStartE2EDuration="4.312456231s" totalImagesPullingTime="3.077888321s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-17 23:18:04 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-17 23:18:08.311499639 +0000 UTC m=+440.713396404" watchObservedRunningTime="2026-09-17 23:18:08.312456231 +0000 UTC m=+440.714352995"
Sep 17 23:18:19.303445 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:18:19.303415    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-keda/keda-metrics-apiserver-77857b7b98-87k7w"
Sep 17 23:18:48.267348 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:18:48.267300    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fwwc5_7e452bbb-7dc6-4c9d-a9b1-76400942f5b4/konnectivity-agent/0.log" limit=1024
Sep 17 23:18:48.329315 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:18:48.329285    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fwwc5_7e452bbb-7dc6-4c9d-a9b1-76400942f5b4/konnectivity-agent/0.log" limit=1024
Sep 17 23:18:57.378016 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:18:57.377977    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fwwc5_7e452bbb-7dc6-4c9d-a9b1-76400942f5b4/konnectivity-agent/0.log" limit=1024
Sep 17 23:18:57.478032 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:18:57.478000    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fwwc5_7e452bbb-7dc6-4c9d-a9b1-76400942f5b4/konnectivity-agent/0.log" limit=1024
Sep 17 23:19:07.901137 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:19:07.901096    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fwwc5_7e452bbb-7dc6-4c9d-a9b1-76400942f5b4/konnectivity-agent/0.log" limit=1024
Sep 17 23:19:07.961316 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:19:07.961285    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fwwc5_7e452bbb-7dc6-4c9d-a9b1-76400942f5b4/konnectivity-agent/0.log" limit=1024
Sep 17 23:19:08.506881 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:19:08.506848    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fwwc5_7e452bbb-7dc6-4c9d-a9b1-76400942f5b4/konnectivity-agent/0.log" limit=1024
Sep 17 23:19:09.265747 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:19:09.265712    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fwwc5_7e452bbb-7dc6-4c9d-a9b1-76400942f5b4/konnectivity-agent/0.log" limit=1024
Sep 17 23:19:09.849823 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:19:09.849786    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fwwc5_7e452bbb-7dc6-4c9d-a9b1-76400942f5b4/konnectivity-agent/0.log" limit=1024
Sep 17 23:19:09.951964 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:19:09.951930    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fwwc5_7e452bbb-7dc6-4c9d-a9b1-76400942f5b4/konnectivity-agent/0.log" limit=1024
Sep 17 23:19:10.435959 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:19:10.435925    2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["cert-manager-operator/cert-manager-operator-controller-manager-684f79c9b9-jzcmn"]
Sep 17 23:19:10.439156 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:19:10.439130    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager-operator/cert-manager-operator-controller-manager-684f79c9b9-jzcmn"
Sep 17 23:19:10.441598 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:19:10.441565    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager-operator\"/\"openshift-service-ca.crt\""
Sep 17 23:19:10.441598 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:19:10.441591    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager-operator\"/\"kube-root-ca.crt\""
Sep 17 23:19:10.441783 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:19:10.441691    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"cert-manager-operator\"/\"cert-manager-operator-controller-manager-dockercfg-gjf7v\""
Sep 17 23:19:10.450872 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:19:10.450845    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager-operator/cert-manager-operator-controller-manager-684f79c9b9-jzcmn"]
Sep 17 23:19:10.585307 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:19:10.585269    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hgn58\" (UniqueName: \"kubernetes.io/projected/60933d33-03ca-491d-b27f-b6fd08ff531e-kube-api-access-hgn58\") pod \"cert-manager-operator-controller-manager-684f79c9b9-jzcmn\" (UID: \"60933d33-03ca-491d-b27f-b6fd08ff531e\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-684f79c9b9-jzcmn"
Sep 17 23:19:10.585307 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:19:10.585311    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/60933d33-03ca-491d-b27f-b6fd08ff531e-tmp\") pod \"cert-manager-operator-controller-manager-684f79c9b9-jzcmn\" (UID: \"60933d33-03ca-491d-b27f-b6fd08ff531e\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-684f79c9b9-jzcmn"
Sep 17 23:19:10.686512 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:19:10.686415    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hgn58\" (UniqueName: \"kubernetes.io/projected/60933d33-03ca-491d-b27f-b6fd08ff531e-kube-api-access-hgn58\") pod \"cert-manager-operator-controller-manager-684f79c9b9-jzcmn\" (UID: \"60933d33-03ca-491d-b27f-b6fd08ff531e\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-684f79c9b9-jzcmn"
Sep 17 23:19:10.686512 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:19:10.686464    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/60933d33-03ca-491d-b27f-b6fd08ff531e-tmp\") pod \"cert-manager-operator-controller-manager-684f79c9b9-jzcmn\" (UID: \"60933d33-03ca-491d-b27f-b6fd08ff531e\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-684f79c9b9-jzcmn"
Sep 17 23:19:10.686922 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:19:10.686902    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/60933d33-03ca-491d-b27f-b6fd08ff531e-tmp\") pod \"cert-manager-operator-controller-manager-684f79c9b9-jzcmn\" (UID: \"60933d33-03ca-491d-b27f-b6fd08ff531e\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-684f79c9b9-jzcmn"
Sep 17 23:19:10.696217 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:19:10.696166    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-hgn58\" (UniqueName: \"kubernetes.io/projected/60933d33-03ca-491d-b27f-b6fd08ff531e-kube-api-access-hgn58\") pod \"cert-manager-operator-controller-manager-684f79c9b9-jzcmn\" (UID: \"60933d33-03ca-491d-b27f-b6fd08ff531e\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-684f79c9b9-jzcmn"
Sep 17 23:19:10.748573 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:19:10.748522    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager-operator/cert-manager-operator-controller-manager-684f79c9b9-jzcmn"
Sep 17 23:19:11.039306 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:19:11.039224    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager-operator/cert-manager-operator-controller-manager-684f79c9b9-jzcmn"]
Sep 17 23:19:11.044190 ip-10-0-132-188 kubenswrapper[2562]: W0917 23:19:11.044148    2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod60933d33_03ca_491d_b27f_b6fd08ff531e.slice/crio-2978e290b73027c26c0867054c5fc58ac7adb68f235af9c279faa64bc4da497d WatchSource:0}: Error finding container 2978e290b73027c26c0867054c5fc58ac7adb68f235af9c279faa64bc4da497d: Status 404 returned error can't find the container with id 2978e290b73027c26c0867054c5fc58ac7adb68f235af9c279faa64bc4da497d
Sep 17 23:19:11.526736 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:19:11.526702    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager-operator/cert-manager-operator-controller-manager-684f79c9b9-jzcmn" event={"ID":"60933d33-03ca-491d-b27f-b6fd08ff531e","Type":"ContainerStarted","Data":"2978e290b73027c26c0867054c5fc58ac7adb68f235af9c279faa64bc4da497d"}
Sep 17 23:19:12.329599 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:19:12.329537    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fwwc5_7e452bbb-7dc6-4c9d-a9b1-76400942f5b4/konnectivity-agent/0.log" limit=1024
Sep 17 23:19:12.427248 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:19:12.427201    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fwwc5_7e452bbb-7dc6-4c9d-a9b1-76400942f5b4/konnectivity-agent/0.log" limit=1024
Sep 17 23:19:14.541549 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:19:14.541512    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager-operator/cert-manager-operator-controller-manager-684f79c9b9-jzcmn" event={"ID":"60933d33-03ca-491d-b27f-b6fd08ff531e","Type":"ContainerStarted","Data":"df3b212878bb25505d7d11025bf36399a8a5442d4e5e880940e789d5668d9ea3"}
Sep 17 23:19:14.566013 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:19:14.565951    2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="cert-manager-operator/cert-manager-operator-controller-manager-684f79c9b9-jzcmn" podStartSLOduration=1.755402865 podStartE2EDuration="4.565930422s" totalImagesPullingTime="2.810527557s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-17 23:19:10 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-17 23:19:14.563887155 +0000 UTC m=+506.965783932" watchObservedRunningTime="2026-09-17 23:19:14.565930422 +0000 UTC m=+506.967827187"
Sep 17 23:19:14.727901 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:19:14.727862    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fwwc5_7e452bbb-7dc6-4c9d-a9b1-76400942f5b4/konnectivity-agent/0.log" limit=1024
Sep 17 23:19:14.828242 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:19:14.828214    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fwwc5_7e452bbb-7dc6-4c9d-a9b1-76400942f5b4/konnectivity-agent/0.log" limit=1024
Sep 17 23:19:17.115124 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:19:17.115093    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fwwc5_7e452bbb-7dc6-4c9d-a9b1-76400942f5b4/konnectivity-agent/0.log" limit=1024
Sep 17 23:19:17.346723 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:19:17.346672    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fwwc5_7e452bbb-7dc6-4c9d-a9b1-76400942f5b4/konnectivity-agent/0.log" limit=1024
Sep 17 23:19:18.778241 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:19:18.778205    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fwwc5_7e452bbb-7dc6-4c9d-a9b1-76400942f5b4/konnectivity-agent/0.log" limit=1024
Sep 17 23:19:18.881209 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:19:18.881155    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fwwc5_7e452bbb-7dc6-4c9d-a9b1-76400942f5b4/konnectivity-agent/0.log" limit=1024
Sep 17 23:19:18.979469 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:19:18.979424    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fwwc5_7e452bbb-7dc6-4c9d-a9b1-76400942f5b4/konnectivity-agent/0.log" limit=1024
Sep 17 23:19:20.118394 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:19:20.118355    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fwwc5_7e452bbb-7dc6-4c9d-a9b1-76400942f5b4/konnectivity-agent/0.log" limit=1024
Sep 17 23:19:20.179191 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:19:20.179130    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fwwc5_7e452bbb-7dc6-4c9d-a9b1-76400942f5b4/konnectivity-agent/0.log" limit=1024
Sep 17 23:19:24.342696 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:19:24.342658    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fwwc5_7e452bbb-7dc6-4c9d-a9b1-76400942f5b4/konnectivity-agent/0.log" limit=1024
Sep 17 23:19:27.570492 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:19:27.570456    2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["cert-manager/cert-manager-55b4d954d5-tznmb"]
Sep 17 23:19:27.575244 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:19:27.575218    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-55b4d954d5-tznmb"
Sep 17 23:19:27.577942 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:19:27.577921    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager\"/\"openshift-service-ca.crt\""
Sep 17 23:19:27.578766 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:19:27.578750    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager\"/\"kube-root-ca.crt\""
Sep 17 23:19:27.578856 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:19:27.578831    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"cert-manager\"/\"cert-manager-dockercfg-j7b6k\""
Sep 17 23:19:27.583000 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:19:27.582973    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-55b4d954d5-tznmb"]
Sep 17 23:19:27.726572 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:19:27.726527    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jq7z7\" (UniqueName: \"kubernetes.io/projected/aa02c7a3-6c2c-48f3-878c-5bfca0d5bd53-kube-api-access-jq7z7\") pod \"cert-manager-55b4d954d5-tznmb\" (UID: \"aa02c7a3-6c2c-48f3-878c-5bfca0d5bd53\") " pod="cert-manager/cert-manager-55b4d954d5-tznmb"
Sep 17 23:19:27.726772 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:19:27.726588    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/aa02c7a3-6c2c-48f3-878c-5bfca0d5bd53-bound-sa-token\") pod \"cert-manager-55b4d954d5-tznmb\" (UID: \"aa02c7a3-6c2c-48f3-878c-5bfca0d5bd53\") " pod="cert-manager/cert-manager-55b4d954d5-tznmb"
Sep 17 23:19:27.827222 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:19:27.827191    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jq7z7\" (UniqueName: \"kubernetes.io/projected/aa02c7a3-6c2c-48f3-878c-5bfca0d5bd53-kube-api-access-jq7z7\") pod \"cert-manager-55b4d954d5-tznmb\" (UID: \"aa02c7a3-6c2c-48f3-878c-5bfca0d5bd53\") " pod="cert-manager/cert-manager-55b4d954d5-tznmb"
Sep 17 23:19:27.827357 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:19:27.827247    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/aa02c7a3-6c2c-48f3-878c-5bfca0d5bd53-bound-sa-token\") pod \"cert-manager-55b4d954d5-tznmb\" (UID: \"aa02c7a3-6c2c-48f3-878c-5bfca0d5bd53\") " pod="cert-manager/cert-manager-55b4d954d5-tznmb"
Sep 17 23:19:27.835326 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:19:27.835293    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/aa02c7a3-6c2c-48f3-878c-5bfca0d5bd53-bound-sa-token\") pod \"cert-manager-55b4d954d5-tznmb\" (UID: \"aa02c7a3-6c2c-48f3-878c-5bfca0d5bd53\") " pod="cert-manager/cert-manager-55b4d954d5-tznmb"
Sep 17 23:19:27.835440 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:19:27.835353    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-jq7z7\" (UniqueName: \"kubernetes.io/projected/aa02c7a3-6c2c-48f3-878c-5bfca0d5bd53-kube-api-access-jq7z7\") pod \"cert-manager-55b4d954d5-tznmb\" (UID: \"aa02c7a3-6c2c-48f3-878c-5bfca0d5bd53\") " pod="cert-manager/cert-manager-55b4d954d5-tznmb"
Sep 17 23:19:27.884066 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:19:27.884034    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-55b4d954d5-tznmb"
Sep 17 23:19:28.153217 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:19:28.153040    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-55b4d954d5-tznmb"]
Sep 17 23:19:28.155876 ip-10-0-132-188 kubenswrapper[2562]: W0917 23:19:28.155843    2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podaa02c7a3_6c2c_48f3_878c_5bfca0d5bd53.slice/crio-64ca1a8bac5f2115229877b42a0a6a10fea99cb312ad23173ff4e767d3142cd1 WatchSource:0}: Error finding container 64ca1a8bac5f2115229877b42a0a6a10fea99cb312ad23173ff4e767d3142cd1: Status 404 returned error can't find the container with id 64ca1a8bac5f2115229877b42a0a6a10fea99cb312ad23173ff4e767d3142cd1
Sep 17 23:19:28.597472 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:19:28.597438    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-55b4d954d5-tznmb" event={"ID":"aa02c7a3-6c2c-48f3-878c-5bfca0d5bd53","Type":"ContainerStarted","Data":"64ca1a8bac5f2115229877b42a0a6a10fea99cb312ad23173ff4e767d3142cd1"}
Sep 17 23:19:31.613066 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:19:31.613022    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-55b4d954d5-tznmb" event={"ID":"aa02c7a3-6c2c-48f3-878c-5bfca0d5bd53","Type":"ContainerStarted","Data":"d8c3f59cacd3ba4328e715b447098c7b56a6579cdf359dd0f077db3685571ec3"}
Sep 17 23:19:31.627546 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:19:31.627494    2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="cert-manager/cert-manager-55b4d954d5-tznmb" podStartSLOduration=1.894842078 podStartE2EDuration="4.627475822s" totalImagesPullingTime="2.732633744s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-17 23:19:27 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-17 23:19:31.626912369 +0000 UTC m=+524.028809133" watchObservedRunningTime="2026-09-17 23:19:31.627475822 +0000 UTC m=+524.029372586"
Sep 17 23:19:40.180863 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:19:40.180818    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fwwc5_7e452bbb-7dc6-4c9d-a9b1-76400942f5b4/konnectivity-agent/0.log" limit=1024
Sep 17 23:19:41.215590 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:19:41.215556    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fwwc5_7e452bbb-7dc6-4c9d-a9b1-76400942f5b4/konnectivity-agent/0.log" limit=1024
Sep 17 23:19:45.914412 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:19:45.914379    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fwwc5_7e452bbb-7dc6-4c9d-a9b1-76400942f5b4/konnectivity-agent/0.log" limit=1024
Sep 17 23:19:46.007845 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:19:46.007806    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fwwc5_7e452bbb-7dc6-4c9d-a9b1-76400942f5b4/konnectivity-agent/0.log" limit=1024
Sep 17 23:19:51.398015 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:19:51.397977    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fwwc5_7e452bbb-7dc6-4c9d-a9b1-76400942f5b4/konnectivity-agent/0.log" limit=1024
Sep 17 23:19:52.428976 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:19:52.428934    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fwwc5_7e452bbb-7dc6-4c9d-a9b1-76400942f5b4/konnectivity-agent/0.log" limit=1024
Sep 17 23:19:56.653253 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:19:56.653218    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fwwc5_7e452bbb-7dc6-4c9d-a9b1-76400942f5b4/konnectivity-agent/0.log" limit=1024
Sep 17 23:19:56.733390 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:19:56.733354    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fwwc5_7e452bbb-7dc6-4c9d-a9b1-76400942f5b4/konnectivity-agent/0.log" limit=1024
Sep 17 23:19:57.214015 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:19:57.213979    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fwwc5_7e452bbb-7dc6-4c9d-a9b1-76400942f5b4/konnectivity-agent/0.log" limit=1024
Sep 17 23:19:57.229531 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:19:57.229500    2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7b7995d5cd-nt48c"]
Sep 17 23:19:57.234110 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:19:57.234091    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-nt48c"
Sep 17 23:19:57.239381 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:19:57.239355    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istiod-tls\""
Sep 17 23:19:57.239508 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:19:57.239359    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istiod-openshift-gateway-dockercfg-sbqnz\""
Sep 17 23:19:57.239663 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:19:57.239646    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"cacerts\""
Sep 17 23:19:57.240087 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:19:57.240073    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"kube-root-ca.crt\""
Sep 17 23:19:57.240456 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:19:57.240388    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-gw-ca-root-cert\""
Sep 17 23:19:57.242376 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:19:57.242356    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istio-kubeconfig\""
Sep 17 23:19:57.245436 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:19:57.245423    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-service-ca.crt\""
Sep 17 23:19:57.259046 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:19:57.258655    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7b7995d5cd-nt48c"]
Sep 17 23:19:57.262335 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:19:57.262310    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/cb721446-8bdd-4787-93e7-257294b551f3-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-7b7995d5cd-nt48c\" (UID: \"cb721446-8bdd-4787-93e7-257294b551f3\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-nt48c"
Sep 17 23:19:57.262448 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:19:57.262361    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-chtj6\" (UniqueName: \"kubernetes.io/projected/cb721446-8bdd-4787-93e7-257294b551f3-kube-api-access-chtj6\") pod \"istiod-openshift-gateway-7b7995d5cd-nt48c\" (UID: \"cb721446-8bdd-4787-93e7-257294b551f3\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-nt48c"
Sep 17 23:19:57.262448 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:19:57.262397    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/cb721446-8bdd-4787-93e7-257294b551f3-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-7b7995d5cd-nt48c\" (UID: \"cb721446-8bdd-4787-93e7-257294b551f3\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-nt48c"
Sep 17 23:19:57.262448 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:19:57.262419    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/cb721446-8bdd-4787-93e7-257294b551f3-cacerts\") pod \"istiod-openshift-gateway-7b7995d5cd-nt48c\" (UID: \"cb721446-8bdd-4787-93e7-257294b551f3\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-nt48c"
Sep 17 23:19:57.262593 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:19:57.262439    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/cb721446-8bdd-4787-93e7-257294b551f3-local-certs\") pod \"istiod-openshift-gateway-7b7995d5cd-nt48c\" (UID: \"cb721446-8bdd-4787-93e7-257294b551f3\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-nt48c"
Sep 17 23:19:57.262593 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:19:57.262483    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/cb721446-8bdd-4787-93e7-257294b551f3-istio-kubeconfig\") pod \"istiod-openshift-gateway-7b7995d5cd-nt48c\" (UID: \"cb721446-8bdd-4787-93e7-257294b551f3\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-nt48c"
Sep 17 23:19:57.262691 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:19:57.262599    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/cb721446-8bdd-4787-93e7-257294b551f3-istio-token\") pod \"istiod-openshift-gateway-7b7995d5cd-nt48c\" (UID: \"cb721446-8bdd-4787-93e7-257294b551f3\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-nt48c"
Sep 17 23:19:57.352062 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:19:57.352026    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fwwc5_7e452bbb-7dc6-4c9d-a9b1-76400942f5b4/konnectivity-agent/0.log" limit=1024
Sep 17 23:19:57.363864 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:19:57.363840    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/cb721446-8bdd-4787-93e7-257294b551f3-local-certs\") pod \"istiod-openshift-gateway-7b7995d5cd-nt48c\" (UID: \"cb721446-8bdd-4787-93e7-257294b551f3\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-nt48c"
Sep 17 23:19:57.364004 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:19:57.363874    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/cb721446-8bdd-4787-93e7-257294b551f3-istio-kubeconfig\") pod \"istiod-openshift-gateway-7b7995d5cd-nt48c\" (UID: \"cb721446-8bdd-4787-93e7-257294b551f3\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-nt48c"
Sep 17 23:19:57.364004 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:19:57.363904    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/cb721446-8bdd-4787-93e7-257294b551f3-istio-token\") pod \"istiod-openshift-gateway-7b7995d5cd-nt48c\" (UID: \"cb721446-8bdd-4787-93e7-257294b551f3\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-nt48c"
Sep 17 23:19:57.364004 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:19:57.363929    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/cb721446-8bdd-4787-93e7-257294b551f3-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-7b7995d5cd-nt48c\" (UID: \"cb721446-8bdd-4787-93e7-257294b551f3\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-nt48c"
Sep 17 23:19:57.364163 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:19:57.364035    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-chtj6\" (UniqueName: \"kubernetes.io/projected/cb721446-8bdd-4787-93e7-257294b551f3-kube-api-access-chtj6\") pod \"istiod-openshift-gateway-7b7995d5cd-nt48c\" (UID: \"cb721446-8bdd-4787-93e7-257294b551f3\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-nt48c"
Sep 17 23:19:57.364163 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:19:57.364075    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/cb721446-8bdd-4787-93e7-257294b551f3-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-7b7995d5cd-nt48c\" (UID: \"cb721446-8bdd-4787-93e7-257294b551f3\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-nt48c"
Sep 17 23:19:57.364163 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:19:57.364097    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/cb721446-8bdd-4787-93e7-257294b551f3-cacerts\") pod \"istiod-openshift-gateway-7b7995d5cd-nt48c\" (UID: \"cb721446-8bdd-4787-93e7-257294b551f3\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-nt48c"
Sep 17 23:19:57.364626 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:19:57.364594    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/cb721446-8bdd-4787-93e7-257294b551f3-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-7b7995d5cd-nt48c\" (UID: \"cb721446-8bdd-4787-93e7-257294b551f3\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-nt48c"
Sep 17 23:19:57.366363 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:19:57.366340    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/cb721446-8bdd-4787-93e7-257294b551f3-local-certs\") pod \"istiod-openshift-gateway-7b7995d5cd-nt48c\" (UID: \"cb721446-8bdd-4787-93e7-257294b551f3\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-nt48c"
Sep 17 23:19:57.366487 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:19:57.366467    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/cb721446-8bdd-4787-93e7-257294b551f3-istio-kubeconfig\") pod \"istiod-openshift-gateway-7b7995d5cd-nt48c\" (UID: \"cb721446-8bdd-4787-93e7-257294b551f3\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-nt48c"
Sep 17 23:19:57.366530 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:19:57.366513    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/cb721446-8bdd-4787-93e7-257294b551f3-cacerts\") pod \"istiod-openshift-gateway-7b7995d5cd-nt48c\" (UID: \"cb721446-8bdd-4787-93e7-257294b551f3\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-nt48c"
Sep 17 23:19:57.366606 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:19:57.366587    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/cb721446-8bdd-4787-93e7-257294b551f3-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-7b7995d5cd-nt48c\" (UID: \"cb721446-8bdd-4787-93e7-257294b551f3\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-nt48c"
Sep 17 23:19:57.373156 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:19:57.373125    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-chtj6\" (UniqueName: \"kubernetes.io/projected/cb721446-8bdd-4787-93e7-257294b551f3-kube-api-access-chtj6\") pod \"istiod-openshift-gateway-7b7995d5cd-nt48c\" (UID: \"cb721446-8bdd-4787-93e7-257294b551f3\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-nt48c"
Sep 17 23:19:57.373415 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:19:57.373395    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/cb721446-8bdd-4787-93e7-257294b551f3-istio-token\") pod \"istiod-openshift-gateway-7b7995d5cd-nt48c\" (UID: \"cb721446-8bdd-4787-93e7-257294b551f3\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-nt48c"
Sep 17 23:19:57.543057 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:19:57.542958    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-nt48c"
Sep 17 23:19:57.814766 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:19:57.814680    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7b7995d5cd-nt48c"]
Sep 17 23:19:57.817774 ip-10-0-132-188 kubenswrapper[2562]: W0917 23:19:57.817749    2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podcb721446_8bdd_4787_93e7_257294b551f3.slice/crio-2d5d3131c5b458222d9d731d466fc0b7f43a6bf245e3838d794107f3e481405a WatchSource:0}: Error finding container 2d5d3131c5b458222d9d731d466fc0b7f43a6bf245e3838d794107f3e481405a: Status 404 returned error can't find the container with id 2d5d3131c5b458222d9d731d466fc0b7f43a6bf245e3838d794107f3e481405a
Sep 17 23:19:58.268487 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:19:58.268427    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fwwc5_7e452bbb-7dc6-4c9d-a9b1-76400942f5b4/konnectivity-agent/0.log" limit=1024
Sep 17 23:19:58.718351 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:19:58.718315    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-nt48c" event={"ID":"cb721446-8bdd-4787-93e7-257294b551f3","Type":"ContainerStarted","Data":"2d5d3131c5b458222d9d731d466fc0b7f43a6bf245e3838d794107f3e481405a"}
Sep 17 23:20:02.039671 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:20:02.039633    2562 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"}
Sep 17 23:20:02.039957 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:20:02.039702    2562 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"}
Sep 17 23:20:02.735894 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:20:02.735855    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-nt48c" event={"ID":"cb721446-8bdd-4787-93e7-257294b551f3","Type":"ContainerStarted","Data":"2e7d49868dde51623999031e4408beb85223f9429e6578d4ca64106ac646501c"}
Sep 17 23:20:02.736096 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:20:02.735973    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-nt48c"
Sep 17 23:20:02.755930 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:20:02.755864    2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-nt48c" podStartSLOduration=1.5362280510000001 podStartE2EDuration="5.75584372s" totalImagesPullingTime="4.219615669s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-17 23:19:57 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-17 23:20:02.753010041 +0000 UTC m=+555.154906805" watchObservedRunningTime="2026-09-17 23:20:02.75584372 +0000 UTC m=+555.157740486"
Sep 17 23:20:03.077608 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:20:03.077562    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fwwc5_7e452bbb-7dc6-4c9d-a9b1-76400942f5b4/konnectivity-agent/0.log" limit=1024
Sep 17 23:20:03.176859 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:20:03.176824    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fwwc5_7e452bbb-7dc6-4c9d-a9b1-76400942f5b4/konnectivity-agent/0.log" limit=1024
Sep 17 23:20:03.741892 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:20:03.741862    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-nt48c"
Sep 17 23:20:04.176582 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:20:04.176547    2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/openshift-ai-inference-openshift-default-58c5cd85c9-9wjvx"]
Sep 17 23:20:04.178805 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:20:04.178781    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/openshift-ai-inference-openshift-default-58c5cd85c9-9wjvx"
Sep 17 23:20:04.181417 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:20:04.181395    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"openshift-ai-inference-openshift-default-dockercfg-knpf7\""
Sep 17 23:20:04.192903 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:20:04.192874    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/openshift-ai-inference-openshift-default-58c5cd85c9-9wjvx"]
Sep 17 23:20:04.218321 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:20:04.218271    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fwwc5_7e452bbb-7dc6-4c9d-a9b1-76400942f5b4/konnectivity-agent/0.log" limit=1024
Sep 17 23:20:04.224080 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:20:04.224055    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/bafca983-9e23-438d-b13c-ff56f8b81b6b-credential-socket\") pod \"openshift-ai-inference-openshift-default-58c5cd85c9-9wjvx\" (UID: \"bafca983-9e23-438d-b13c-ff56f8b81b6b\") " pod="openshift-ingress/openshift-ai-inference-openshift-default-58c5cd85c9-9wjvx"
Sep 17 23:20:04.224235 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:20:04.224101    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/bafca983-9e23-438d-b13c-ff56f8b81b6b-istio-envoy\") pod \"openshift-ai-inference-openshift-default-58c5cd85c9-9wjvx\" (UID: \"bafca983-9e23-438d-b13c-ff56f8b81b6b\") " pod="openshift-ingress/openshift-ai-inference-openshift-default-58c5cd85c9-9wjvx"
Sep 17 23:20:04.224235 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:20:04.224128    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vqxr9\" (UniqueName: \"kubernetes.io/projected/bafca983-9e23-438d-b13c-ff56f8b81b6b-kube-api-access-vqxr9\") pod \"openshift-ai-inference-openshift-default-58c5cd85c9-9wjvx\" (UID: \"bafca983-9e23-438d-b13c-ff56f8b81b6b\") " pod="openshift-ingress/openshift-ai-inference-openshift-default-58c5cd85c9-9wjvx"
Sep 17 23:20:04.224235 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:20:04.224219    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/bafca983-9e23-438d-b13c-ff56f8b81b6b-istio-token\") pod \"openshift-ai-inference-openshift-default-58c5cd85c9-9wjvx\" (UID: \"bafca983-9e23-438d-b13c-ff56f8b81b6b\") " pod="openshift-ingress/openshift-ai-inference-openshift-default-58c5cd85c9-9wjvx"
Sep 17 23:20:04.224369 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:20:04.224254    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/bafca983-9e23-438d-b13c-ff56f8b81b6b-istio-podinfo\") pod \"openshift-ai-inference-openshift-default-58c5cd85c9-9wjvx\" (UID: \"bafca983-9e23-438d-b13c-ff56f8b81b6b\") " pod="openshift-ingress/openshift-ai-inference-openshift-default-58c5cd85c9-9wjvx"
Sep 17 23:20:04.224369 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:20:04.224278    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/bafca983-9e23-438d-b13c-ff56f8b81b6b-workload-socket\") pod \"openshift-ai-inference-openshift-default-58c5cd85c9-9wjvx\" (UID: \"bafca983-9e23-438d-b13c-ff56f8b81b6b\") " pod="openshift-ingress/openshift-ai-inference-openshift-default-58c5cd85c9-9wjvx"
Sep 17 23:20:04.224369 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:20:04.224294    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/bafca983-9e23-438d-b13c-ff56f8b81b6b-workload-certs\") pod \"openshift-ai-inference-openshift-default-58c5cd85c9-9wjvx\" (UID: \"bafca983-9e23-438d-b13c-ff56f8b81b6b\") " pod="openshift-ingress/openshift-ai-inference-openshift-default-58c5cd85c9-9wjvx"
Sep 17 23:20:04.224369 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:20:04.224328    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/bafca983-9e23-438d-b13c-ff56f8b81b6b-istio-data\") pod \"openshift-ai-inference-openshift-default-58c5cd85c9-9wjvx\" (UID: \"bafca983-9e23-438d-b13c-ff56f8b81b6b\") " pod="openshift-ingress/openshift-ai-inference-openshift-default-58c5cd85c9-9wjvx"
Sep 17 23:20:04.224507 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:20:04.224369    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/bafca983-9e23-438d-b13c-ff56f8b81b6b-istiod-ca-cert\") pod \"openshift-ai-inference-openshift-default-58c5cd85c9-9wjvx\" (UID: \"bafca983-9e23-438d-b13c-ff56f8b81b6b\") " pod="openshift-ingress/openshift-ai-inference-openshift-default-58c5cd85c9-9wjvx"
Sep 17 23:20:04.304053 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:20:04.304017    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fwwc5_7e452bbb-7dc6-4c9d-a9b1-76400942f5b4/konnectivity-agent/0.log" limit=1024
Sep 17 23:20:04.325102 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:20:04.325069    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vqxr9\" (UniqueName: \"kubernetes.io/projected/bafca983-9e23-438d-b13c-ff56f8b81b6b-kube-api-access-vqxr9\") pod \"openshift-ai-inference-openshift-default-58c5cd85c9-9wjvx\" (UID: \"bafca983-9e23-438d-b13c-ff56f8b81b6b\") " pod="openshift-ingress/openshift-ai-inference-openshift-default-58c5cd85c9-9wjvx"
Sep 17 23:20:04.325225 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:20:04.325117    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/bafca983-9e23-438d-b13c-ff56f8b81b6b-istio-token\") pod \"openshift-ai-inference-openshift-default-58c5cd85c9-9wjvx\" (UID: \"bafca983-9e23-438d-b13c-ff56f8b81b6b\") " pod="openshift-ingress/openshift-ai-inference-openshift-default-58c5cd85c9-9wjvx"
Sep 17 23:20:04.325274 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:20:04.325233    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/bafca983-9e23-438d-b13c-ff56f8b81b6b-istio-podinfo\") pod \"openshift-ai-inference-openshift-default-58c5cd85c9-9wjvx\" (UID: \"bafca983-9e23-438d-b13c-ff56f8b81b6b\") " pod="openshift-ingress/openshift-ai-inference-openshift-default-58c5cd85c9-9wjvx"
Sep 17 23:20:04.325318 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:20:04.325284    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/bafca983-9e23-438d-b13c-ff56f8b81b6b-workload-socket\") pod \"openshift-ai-inference-openshift-default-58c5cd85c9-9wjvx\" (UID: \"bafca983-9e23-438d-b13c-ff56f8b81b6b\") " pod="openshift-ingress/openshift-ai-inference-openshift-default-58c5cd85c9-9wjvx"
Sep 17 23:20:04.325318 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:20:04.325310    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/bafca983-9e23-438d-b13c-ff56f8b81b6b-workload-certs\") pod \"openshift-ai-inference-openshift-default-58c5cd85c9-9wjvx\" (UID: \"bafca983-9e23-438d-b13c-ff56f8b81b6b\") " pod="openshift-ingress/openshift-ai-inference-openshift-default-58c5cd85c9-9wjvx"
Sep 17 23:20:04.325419 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:20:04.325377    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/bafca983-9e23-438d-b13c-ff56f8b81b6b-istio-data\") pod \"openshift-ai-inference-openshift-default-58c5cd85c9-9wjvx\" (UID: \"bafca983-9e23-438d-b13c-ff56f8b81b6b\") " pod="openshift-ingress/openshift-ai-inference-openshift-default-58c5cd85c9-9wjvx"
Sep 17 23:20:04.325419 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:20:04.325412    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/bafca983-9e23-438d-b13c-ff56f8b81b6b-istiod-ca-cert\") pod \"openshift-ai-inference-openshift-default-58c5cd85c9-9wjvx\" (UID: \"bafca983-9e23-438d-b13c-ff56f8b81b6b\") " pod="openshift-ingress/openshift-ai-inference-openshift-default-58c5cd85c9-9wjvx"
Sep 17 23:20:04.325547 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:20:04.325525    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/bafca983-9e23-438d-b13c-ff56f8b81b6b-credential-socket\") pod \"openshift-ai-inference-openshift-default-58c5cd85c9-9wjvx\" (UID: \"bafca983-9e23-438d-b13c-ff56f8b81b6b\") " pod="openshift-ingress/openshift-ai-inference-openshift-default-58c5cd85c9-9wjvx"
Sep 17 23:20:04.325631 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:20:04.325608    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/bafca983-9e23-438d-b13c-ff56f8b81b6b-istio-envoy\") pod \"openshift-ai-inference-openshift-default-58c5cd85c9-9wjvx\" (UID: \"bafca983-9e23-438d-b13c-ff56f8b81b6b\") " pod="openshift-ingress/openshift-ai-inference-openshift-default-58c5cd85c9-9wjvx"
Sep 17 23:20:04.325775 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:20:04.325751    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/bafca983-9e23-438d-b13c-ff56f8b81b6b-workload-socket\") pod \"openshift-ai-inference-openshift-default-58c5cd85c9-9wjvx\" (UID: \"bafca983-9e23-438d-b13c-ff56f8b81b6b\") " pod="openshift-ingress/openshift-ai-inference-openshift-default-58c5cd85c9-9wjvx"
Sep 17 23:20:04.325914 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:20:04.325799    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/bafca983-9e23-438d-b13c-ff56f8b81b6b-workload-certs\") pod \"openshift-ai-inference-openshift-default-58c5cd85c9-9wjvx\" (UID: \"bafca983-9e23-438d-b13c-ff56f8b81b6b\") " pod="openshift-ingress/openshift-ai-inference-openshift-default-58c5cd85c9-9wjvx"
Sep 17 23:20:04.325914 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:20:04.325889    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/bafca983-9e23-438d-b13c-ff56f8b81b6b-istio-data\") pod \"openshift-ai-inference-openshift-default-58c5cd85c9-9wjvx\" (UID: \"bafca983-9e23-438d-b13c-ff56f8b81b6b\") " pod="openshift-ingress/openshift-ai-inference-openshift-default-58c5cd85c9-9wjvx"
Sep 17 23:20:04.326103 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:20:04.325972    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/bafca983-9e23-438d-b13c-ff56f8b81b6b-credential-socket\") pod \"openshift-ai-inference-openshift-default-58c5cd85c9-9wjvx\" (UID: \"bafca983-9e23-438d-b13c-ff56f8b81b6b\") " pod="openshift-ingress/openshift-ai-inference-openshift-default-58c5cd85c9-9wjvx"
Sep 17 23:20:04.326216 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:20:04.326195    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/bafca983-9e23-438d-b13c-ff56f8b81b6b-istiod-ca-cert\") pod \"openshift-ai-inference-openshift-default-58c5cd85c9-9wjvx\" (UID: \"bafca983-9e23-438d-b13c-ff56f8b81b6b\") " pod="openshift-ingress/openshift-ai-inference-openshift-default-58c5cd85c9-9wjvx"
Sep 17 23:20:04.327757 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:20:04.327738    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/bafca983-9e23-438d-b13c-ff56f8b81b6b-istio-envoy\") pod \"openshift-ai-inference-openshift-default-58c5cd85c9-9wjvx\" (UID: \"bafca983-9e23-438d-b13c-ff56f8b81b6b\") " pod="openshift-ingress/openshift-ai-inference-openshift-default-58c5cd85c9-9wjvx"
Sep 17 23:20:04.327824 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:20:04.327813    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/bafca983-9e23-438d-b13c-ff56f8b81b6b-istio-podinfo\") pod \"openshift-ai-inference-openshift-default-58c5cd85c9-9wjvx\" (UID: \"bafca983-9e23-438d-b13c-ff56f8b81b6b\") " pod="openshift-ingress/openshift-ai-inference-openshift-default-58c5cd85c9-9wjvx"
Sep 17 23:20:04.333090 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:20:04.333066    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/bafca983-9e23-438d-b13c-ff56f8b81b6b-istio-token\") pod \"openshift-ai-inference-openshift-default-58c5cd85c9-9wjvx\" (UID: \"bafca983-9e23-438d-b13c-ff56f8b81b6b\") " pod="openshift-ingress/openshift-ai-inference-openshift-default-58c5cd85c9-9wjvx"
Sep 17 23:20:04.333401 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:20:04.333381    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-vqxr9\" (UniqueName: \"kubernetes.io/projected/bafca983-9e23-438d-b13c-ff56f8b81b6b-kube-api-access-vqxr9\") pod \"openshift-ai-inference-openshift-default-58c5cd85c9-9wjvx\" (UID: \"bafca983-9e23-438d-b13c-ff56f8b81b6b\") " pod="openshift-ingress/openshift-ai-inference-openshift-default-58c5cd85c9-9wjvx"
Sep 17 23:20:04.491978 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:20:04.491879    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/openshift-ai-inference-openshift-default-58c5cd85c9-9wjvx"
Sep 17 23:20:04.768073 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:20:04.767993    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/openshift-ai-inference-openshift-default-58c5cd85c9-9wjvx"]
Sep 17 23:20:04.771814 ip-10-0-132-188 kubenswrapper[2562]: W0917 23:20:04.771780    2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podbafca983_9e23_438d_b13c_ff56f8b81b6b.slice/crio-d3726d471902d0681ea492acff99aedae3e4465936bcc47ce09446875a050d76 WatchSource:0}: Error finding container d3726d471902d0681ea492acff99aedae3e4465936bcc47ce09446875a050d76: Status 404 returned error can't find the container with id d3726d471902d0681ea492acff99aedae3e4465936bcc47ce09446875a050d76
Sep 17 23:20:05.750974 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:20:05.750936    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/openshift-ai-inference-openshift-default-58c5cd85c9-9wjvx" event={"ID":"bafca983-9e23-438d-b13c-ff56f8b81b6b","Type":"ContainerStarted","Data":"d3726d471902d0681ea492acff99aedae3e4465936bcc47ce09446875a050d76"}
Sep 17 23:20:07.124399 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:20:07.124297    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fwwc5_7e452bbb-7dc6-4c9d-a9b1-76400942f5b4/konnectivity-agent/0.log" limit=1024
Sep 17 23:20:07.463215 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:20:07.463163    2562 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"}
Sep 17 23:20:07.463316 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:20:07.463258    2562 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"}
Sep 17 23:20:07.463316 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:20:07.463293    2562 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"}
Sep 17 23:20:07.767655 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:20:07.767558    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/openshift-ai-inference-openshift-default-58c5cd85c9-9wjvx" event={"ID":"bafca983-9e23-438d-b13c-ff56f8b81b6b","Type":"ContainerStarted","Data":"7301d5024fc3878457bd3445e772e22c7e740b1d2d1aeef66db3c53552d0af76"}
Sep 17 23:20:07.788329 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:20:07.788264    2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/openshift-ai-inference-openshift-default-58c5cd85c9-9wjvx" podStartSLOduration=1.099317368 podStartE2EDuration="3.788242136s" totalImagesPullingTime="2.688924768s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-17 23:20:04 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-17 23:20:07.787349989 +0000 UTC m=+560.189246748" watchObservedRunningTime="2026-09-17 23:20:07.788242136 +0000 UTC m=+560.190138897"
Sep 17 23:20:08.492853 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:20:08.492809    2562 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/openshift-ai-inference-openshift-default-58c5cd85c9-9wjvx"
Sep 17 23:20:08.497665 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:20:08.497637    2562 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/openshift-ai-inference-openshift-default-58c5cd85c9-9wjvx"
Sep 17 23:20:08.772431 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:20:08.772339    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/openshift-ai-inference-openshift-default-58c5cd85c9-9wjvx"
Sep 17 23:20:08.773479 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:20:08.773461    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/openshift-ai-inference-openshift-default-58c5cd85c9-9wjvx"
Sep 17 23:20:09.678670 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:20:09.678629    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fwwc5_7e452bbb-7dc6-4c9d-a9b1-76400942f5b4/konnectivity-agent/0.log" limit=1024
Sep 17 23:20:09.783266 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:20:09.783221    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fwwc5_7e452bbb-7dc6-4c9d-a9b1-76400942f5b4/konnectivity-agent/0.log" limit=1024
Sep 17 23:20:34.618662 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:20:34.618624    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fwwc5_7e452bbb-7dc6-4c9d-a9b1-76400942f5b4/konnectivity-agent/0.log" limit=1024
Sep 17 23:20:40.549087 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:20:40.549052    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fwwc5_7e452bbb-7dc6-4c9d-a9b1-76400942f5b4/konnectivity-agent/0.log" limit=1024
Sep 17 23:20:41.341467 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:20:41.341427    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fwwc5_7e452bbb-7dc6-4c9d-a9b1-76400942f5b4/konnectivity-agent/0.log" limit=1024
Sep 17 23:20:42.613408 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:20:42.613372    2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-84794c78d6-brcjh"]
Sep 17 23:20:42.626331 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:20:42.626304    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-84794c78d6-brcjh"
Sep 17 23:20:42.627141 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:20:42.627103    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-84794c78d6-brcjh"]
Sep 17 23:20:42.628955 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:20:42.628931    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"kube-root-ca.crt\""
Sep 17 23:20:42.629076 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:20:42.628931    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"service-ca\""
Sep 17 23:20:42.629236 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:20:42.629220    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"oauth-serving-cert\""
Sep 17 23:20:42.629331 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:20:42.629254    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"console-config\""
Sep 17 23:20:42.630921 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:20:42.630903    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-serving-cert\""
Sep 17 23:20:42.631299 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:20:42.631162    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-dockercfg-hjfdq\""
Sep 17 23:20:42.631299 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:20:42.631220    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"openshift-service-ca.crt\""
Sep 17 23:20:42.631299 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:20:42.631220    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-oauth-config\""
Sep 17 23:20:42.634885 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:20:42.634866    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"trusted-ca-bundle\""
Sep 17 23:20:42.649883 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:20:42.649851    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/2a8e168f-81be-41c4-8635-2ba879bdce47-console-oauth-config\") pod \"console-84794c78d6-brcjh\" (UID: \"2a8e168f-81be-41c4-8635-2ba879bdce47\") " pod="openshift-console/console-84794c78d6-brcjh"
Sep 17 23:20:42.650031 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:20:42.649888    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/2a8e168f-81be-41c4-8635-2ba879bdce47-oauth-serving-cert\") pod \"console-84794c78d6-brcjh\" (UID: \"2a8e168f-81be-41c4-8635-2ba879bdce47\") " pod="openshift-console/console-84794c78d6-brcjh"
Sep 17 23:20:42.650031 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:20:42.649908    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/2a8e168f-81be-41c4-8635-2ba879bdce47-trusted-ca-bundle\") pod \"console-84794c78d6-brcjh\" (UID: \"2a8e168f-81be-41c4-8635-2ba879bdce47\") " pod="openshift-console/console-84794c78d6-brcjh"
Sep 17 23:20:42.650031 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:20:42.649971    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-cg82g\" (UniqueName: \"kubernetes.io/projected/2a8e168f-81be-41c4-8635-2ba879bdce47-kube-api-access-cg82g\") pod \"console-84794c78d6-brcjh\" (UID: \"2a8e168f-81be-41c4-8635-2ba879bdce47\") " pod="openshift-console/console-84794c78d6-brcjh"
Sep 17 23:20:42.650031 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:20:42.649997    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/2a8e168f-81be-41c4-8635-2ba879bdce47-service-ca\") pod \"console-84794c78d6-brcjh\" (UID: \"2a8e168f-81be-41c4-8635-2ba879bdce47\") " pod="openshift-console/console-84794c78d6-brcjh"
Sep 17 23:20:42.650342 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:20:42.650056    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/2a8e168f-81be-41c4-8635-2ba879bdce47-console-serving-cert\") pod \"console-84794c78d6-brcjh\" (UID: \"2a8e168f-81be-41c4-8635-2ba879bdce47\") " pod="openshift-console/console-84794c78d6-brcjh"
Sep 17 23:20:42.650342 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:20:42.650075    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/2a8e168f-81be-41c4-8635-2ba879bdce47-console-config\") pod \"console-84794c78d6-brcjh\" (UID: \"2a8e168f-81be-41c4-8635-2ba879bdce47\") " pod="openshift-console/console-84794c78d6-brcjh"
Sep 17 23:20:42.663244 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:20:42.663210    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fwwc5_7e452bbb-7dc6-4c9d-a9b1-76400942f5b4/konnectivity-agent/0.log" limit=1024
Sep 17 23:20:42.750971 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:20:42.750938    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/2a8e168f-81be-41c4-8635-2ba879bdce47-console-oauth-config\") pod \"console-84794c78d6-brcjh\" (UID: \"2a8e168f-81be-41c4-8635-2ba879bdce47\") " pod="openshift-console/console-84794c78d6-brcjh"
Sep 17 23:20:42.750971 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:20:42.750978    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/2a8e168f-81be-41c4-8635-2ba879bdce47-oauth-serving-cert\") pod \"console-84794c78d6-brcjh\" (UID: \"2a8e168f-81be-41c4-8635-2ba879bdce47\") " pod="openshift-console/console-84794c78d6-brcjh"
Sep 17 23:20:42.751270 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:20:42.750996    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/2a8e168f-81be-41c4-8635-2ba879bdce47-trusted-ca-bundle\") pod \"console-84794c78d6-brcjh\" (UID: \"2a8e168f-81be-41c4-8635-2ba879bdce47\") " pod="openshift-console/console-84794c78d6-brcjh"
Sep 17 23:20:42.751270 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:20:42.751074    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-cg82g\" (UniqueName: \"kubernetes.io/projected/2a8e168f-81be-41c4-8635-2ba879bdce47-kube-api-access-cg82g\") pod \"console-84794c78d6-brcjh\" (UID: \"2a8e168f-81be-41c4-8635-2ba879bdce47\") " pod="openshift-console/console-84794c78d6-brcjh"
Sep 17 23:20:42.751270 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:20:42.751111    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/2a8e168f-81be-41c4-8635-2ba879bdce47-service-ca\") pod \"console-84794c78d6-brcjh\" (UID: \"2a8e168f-81be-41c4-8635-2ba879bdce47\") " pod="openshift-console/console-84794c78d6-brcjh"
Sep 17 23:20:42.751270 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:20:42.751216    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/2a8e168f-81be-41c4-8635-2ba879bdce47-console-serving-cert\") pod \"console-84794c78d6-brcjh\" (UID: \"2a8e168f-81be-41c4-8635-2ba879bdce47\") " pod="openshift-console/console-84794c78d6-brcjh"
Sep 17 23:20:42.751468 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:20:42.751296    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/2a8e168f-81be-41c4-8635-2ba879bdce47-console-config\") pod \"console-84794c78d6-brcjh\" (UID: \"2a8e168f-81be-41c4-8635-2ba879bdce47\") " pod="openshift-console/console-84794c78d6-brcjh"
Sep 17 23:20:42.751811 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:20:42.751781    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/2a8e168f-81be-41c4-8635-2ba879bdce47-oauth-serving-cert\") pod \"console-84794c78d6-brcjh\" (UID: \"2a8e168f-81be-41c4-8635-2ba879bdce47\") " pod="openshift-console/console-84794c78d6-brcjh"
Sep 17 23:20:42.751925 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:20:42.751857    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/2a8e168f-81be-41c4-8635-2ba879bdce47-service-ca\") pod \"console-84794c78d6-brcjh\" (UID: \"2a8e168f-81be-41c4-8635-2ba879bdce47\") " pod="openshift-console/console-84794c78d6-brcjh"
Sep 17 23:20:42.751925 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:20:42.751916    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/2a8e168f-81be-41c4-8635-2ba879bdce47-console-config\") pod \"console-84794c78d6-brcjh\" (UID: \"2a8e168f-81be-41c4-8635-2ba879bdce47\") " pod="openshift-console/console-84794c78d6-brcjh"
Sep 17 23:20:42.752183 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:20:42.752141    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/2a8e168f-81be-41c4-8635-2ba879bdce47-trusted-ca-bundle\") pod \"console-84794c78d6-brcjh\" (UID: \"2a8e168f-81be-41c4-8635-2ba879bdce47\") " pod="openshift-console/console-84794c78d6-brcjh"
Sep 17 23:20:42.753731 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:20:42.753710    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/2a8e168f-81be-41c4-8635-2ba879bdce47-console-serving-cert\") pod \"console-84794c78d6-brcjh\" (UID: \"2a8e168f-81be-41c4-8635-2ba879bdce47\") " pod="openshift-console/console-84794c78d6-brcjh"
Sep 17 23:20:42.753816 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:20:42.753794    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/2a8e168f-81be-41c4-8635-2ba879bdce47-console-oauth-config\") pod \"console-84794c78d6-brcjh\" (UID: \"2a8e168f-81be-41c4-8635-2ba879bdce47\") " pod="openshift-console/console-84794c78d6-brcjh"
Sep 17 23:20:42.759241 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:20:42.759217    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-cg82g\" (UniqueName: \"kubernetes.io/projected/2a8e168f-81be-41c4-8635-2ba879bdce47-kube-api-access-cg82g\") pod \"console-84794c78d6-brcjh\" (UID: \"2a8e168f-81be-41c4-8635-2ba879bdce47\") " pod="openshift-console/console-84794c78d6-brcjh"
Sep 17 23:20:42.764383 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:20:42.764355    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fwwc5_7e452bbb-7dc6-4c9d-a9b1-76400942f5b4/konnectivity-agent/0.log" limit=1024
Sep 17 23:20:42.937336 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:20:42.937235    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-84794c78d6-brcjh"
Sep 17 23:20:43.208693 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:20:43.208666    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-84794c78d6-brcjh"]
Sep 17 23:20:43.211213 ip-10-0-132-188 kubenswrapper[2562]: W0917 23:20:43.211159    2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2a8e168f_81be_41c4_8635_2ba879bdce47.slice/crio-4e39ce23e96e307b3a7e72b5fcd7fc7aac82c85413aee90d9f2fc92ec0399e82 WatchSource:0}: Error finding container 4e39ce23e96e307b3a7e72b5fcd7fc7aac82c85413aee90d9f2fc92ec0399e82: Status 404 returned error can't find the container with id 4e39ce23e96e307b3a7e72b5fcd7fc7aac82c85413aee90d9f2fc92ec0399e82
Sep 17 23:20:43.875673 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:20:43.875634    2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-operator-6fc8f864f4-tbvdm"]
Sep 17 23:20:43.879031 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:20:43.879013    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-operator-6fc8f864f4-tbvdm"
Sep 17 23:20:43.881866 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:20:43.881841    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"authorino-operator-dockercfg-57jjz\""
Sep 17 23:20:43.881998 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:20:43.881845    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"kube-root-ca.crt\""
Sep 17 23:20:43.881998 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:20:43.881931    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"openshift-service-ca.crt\""
Sep 17 23:20:43.895701 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:20:43.895667    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-operator-6fc8f864f4-tbvdm"]
Sep 17 23:20:43.921013 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:20:43.920972    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-84794c78d6-brcjh" event={"ID":"2a8e168f-81be-41c4-8635-2ba879bdce47","Type":"ContainerStarted","Data":"8e9fca2b1675492b0196031e05d4d7fa43f19f9c235d9585914b5b222c788aef"}
Sep 17 23:20:43.921200 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:20:43.921046    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-84794c78d6-brcjh" event={"ID":"2a8e168f-81be-41c4-8635-2ba879bdce47","Type":"ContainerStarted","Data":"4e39ce23e96e307b3a7e72b5fcd7fc7aac82c85413aee90d9f2fc92ec0399e82"}
Sep 17 23:20:43.942146 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:20:43.942087    2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-84794c78d6-brcjh" podStartSLOduration=1.9420670599999998 podStartE2EDuration="1.94206706s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-17 23:20:42 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-17 23:20:43.940478269 +0000 UTC m=+596.342375028" watchObservedRunningTime="2026-09-17 23:20:43.94206706 +0000 UTC m=+596.343963824"
Sep 17 23:20:43.962857 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:20:43.962810    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wbzxr\" (UniqueName: \"kubernetes.io/projected/cc89fb3b-161e-46c8-b343-f221ce5fedd1-kube-api-access-wbzxr\") pod \"authorino-operator-6fc8f864f4-tbvdm\" (UID: \"cc89fb3b-161e-46c8-b343-f221ce5fedd1\") " pod="kuadrant-system/authorino-operator-6fc8f864f4-tbvdm"
Sep 17 23:20:44.063434 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:20:44.063394    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wbzxr\" (UniqueName: \"kubernetes.io/projected/cc89fb3b-161e-46c8-b343-f221ce5fedd1-kube-api-access-wbzxr\") pod \"authorino-operator-6fc8f864f4-tbvdm\" (UID: \"cc89fb3b-161e-46c8-b343-f221ce5fedd1\") " pod="kuadrant-system/authorino-operator-6fc8f864f4-tbvdm"
Sep 17 23:20:44.072969 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:20:44.072928    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-wbzxr\" (UniqueName: \"kubernetes.io/projected/cc89fb3b-161e-46c8-b343-f221ce5fedd1-kube-api-access-wbzxr\") pod \"authorino-operator-6fc8f864f4-tbvdm\" (UID: \"cc89fb3b-161e-46c8-b343-f221ce5fedd1\") " pod="kuadrant-system/authorino-operator-6fc8f864f4-tbvdm"
Sep 17 23:20:44.189859 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:20:44.189771    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-operator-6fc8f864f4-tbvdm"
Sep 17 23:20:44.467599 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:20:44.467444    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-operator-6fc8f864f4-tbvdm"]
Sep 17 23:20:44.470233 ip-10-0-132-188 kubenswrapper[2562]: W0917 23:20:44.470205    2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podcc89fb3b_161e_46c8_b343_f221ce5fedd1.slice/crio-f71add56ae53c40ac694e554cce855d5213c870d184cc907e2933d48ba39f125 WatchSource:0}: Error finding container f71add56ae53c40ac694e554cce855d5213c870d184cc907e2933d48ba39f125: Status 404 returned error can't find the container with id f71add56ae53c40ac694e554cce855d5213c870d184cc907e2933d48ba39f125
Sep 17 23:20:44.551341 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:20:44.551303    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fwwc5_7e452bbb-7dc6-4c9d-a9b1-76400942f5b4/konnectivity-agent/0.log" limit=1024
Sep 17 23:20:44.782056 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:20:44.781969    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fwwc5_7e452bbb-7dc6-4c9d-a9b1-76400942f5b4/konnectivity-agent/0.log" limit=1024
Sep 17 23:20:44.926641 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:20:44.926602    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-operator-6fc8f864f4-tbvdm" event={"ID":"cc89fb3b-161e-46c8-b343-f221ce5fedd1","Type":"ContainerStarted","Data":"f71add56ae53c40ac694e554cce855d5213c870d184cc907e2933d48ba39f125"}
Sep 17 23:20:46.191399 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:20:46.191368    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fwwc5_7e452bbb-7dc6-4c9d-a9b1-76400942f5b4/konnectivity-agent/0.log" limit=1024
Sep 17 23:20:46.938830 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:20:46.938797    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-operator-6fc8f864f4-tbvdm" event={"ID":"cc89fb3b-161e-46c8-b343-f221ce5fedd1","Type":"ContainerStarted","Data":"cc3097815beb8ebb2d28711cb205f7d1c4561e61dce7f30fdefb23ba5c55852f"}
Sep 17 23:20:46.939001 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:20:46.938866    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/authorino-operator-6fc8f864f4-tbvdm"
Sep 17 23:20:46.960277 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:20:46.960221    2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/authorino-operator-6fc8f864f4-tbvdm" podStartSLOduration=2.360013689 podStartE2EDuration="3.960203627s" totalImagesPullingTime="1.600189938s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-17 23:20:43 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-17 23:20:46.959351576 +0000 UTC m=+599.361248340" watchObservedRunningTime="2026-09-17 23:20:46.960203627 +0000 UTC m=+599.362100389"
Sep 17 23:20:46.971417 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:20:46.971379    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fwwc5_7e452bbb-7dc6-4c9d-a9b1-76400942f5b4/konnectivity-agent/0.log" limit=1024
Sep 17 23:20:47.589012 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:20:47.588982    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fwwc5_7e452bbb-7dc6-4c9d-a9b1-76400942f5b4/konnectivity-agent/0.log" limit=1024
Sep 17 23:20:48.573588 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:20:48.573552    2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-5ffb4b4ccb-l2khb"]
Sep 17 23:20:48.576852 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:20:48.576828    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-5ffb4b4ccb-l2khb"
Sep 17 23:20:48.583511 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:20:48.583487    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"limitador-operator-controller-manager-dockercfg-2zthr\""
Sep 17 23:20:48.603119 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:20:48.603083    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-5ffb4b4ccb-l2khb"]
Sep 17 23:20:48.608577 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:20:48.608548    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-w7q7j\" (UniqueName: \"kubernetes.io/projected/58552d00-05c8-4a22-b1ae-1446a33743bd-kube-api-access-w7q7j\") pod \"limitador-operator-controller-manager-5ffb4b4ccb-l2khb\" (UID: \"58552d00-05c8-4a22-b1ae-1446a33743bd\") " pod="kuadrant-system/limitador-operator-controller-manager-5ffb4b4ccb-l2khb"
Sep 17 23:20:48.709366 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:20:48.709334    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-w7q7j\" (UniqueName: \"kubernetes.io/projected/58552d00-05c8-4a22-b1ae-1446a33743bd-kube-api-access-w7q7j\") pod \"limitador-operator-controller-manager-5ffb4b4ccb-l2khb\" (UID: \"58552d00-05c8-4a22-b1ae-1446a33743bd\") " pod="kuadrant-system/limitador-operator-controller-manager-5ffb4b4ccb-l2khb"
Sep 17 23:20:48.717551 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:20:48.717522    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-w7q7j\" (UniqueName: \"kubernetes.io/projected/58552d00-05c8-4a22-b1ae-1446a33743bd-kube-api-access-w7q7j\") pod \"limitador-operator-controller-manager-5ffb4b4ccb-l2khb\" (UID: \"58552d00-05c8-4a22-b1ae-1446a33743bd\") " pod="kuadrant-system/limitador-operator-controller-manager-5ffb4b4ccb-l2khb"
Sep 17 23:20:48.886978 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:20:48.886948    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-5ffb4b4ccb-l2khb"
Sep 17 23:20:48.947133 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:20:48.947091    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fwwc5_7e452bbb-7dc6-4c9d-a9b1-76400942f5b4/konnectivity-agent/0.log" limit=1024
Sep 17 23:20:49.163747 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:20:49.163717    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-5ffb4b4ccb-l2khb"]
Sep 17 23:20:49.166158 ip-10-0-132-188 kubenswrapper[2562]: W0917 23:20:49.166124    2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod58552d00_05c8_4a22_b1ae_1446a33743bd.slice/crio-6b1e0c210662b819971622ab9e51f2d41bd2057b51efcc099b824886e3c308ca WatchSource:0}: Error finding container 6b1e0c210662b819971622ab9e51f2d41bd2057b51efcc099b824886e3c308ca: Status 404 returned error can't find the container with id 6b1e0c210662b819971622ab9e51f2d41bd2057b51efcc099b824886e3c308ca
Sep 17 23:20:49.351185 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:20:49.351135    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fwwc5_7e452bbb-7dc6-4c9d-a9b1-76400942f5b4/konnectivity-agent/0.log" limit=1024
Sep 17 23:20:49.753021 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:20:49.752979    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fwwc5_7e452bbb-7dc6-4c9d-a9b1-76400942f5b4/konnectivity-agent/0.log" limit=1024
Sep 17 23:20:49.955906 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:20:49.955866    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-operator-controller-manager-5ffb4b4ccb-l2khb" event={"ID":"58552d00-05c8-4a22-b1ae-1446a33743bd","Type":"ContainerStarted","Data":"6b1e0c210662b819971622ab9e51f2d41bd2057b51efcc099b824886e3c308ca"}
Sep 17 23:20:50.218064 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:20:50.218029    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fwwc5_7e452bbb-7dc6-4c9d-a9b1-76400942f5b4/konnectivity-agent/0.log" limit=1024
Sep 17 23:20:51.967060 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:20:51.967021    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-operator-controller-manager-5ffb4b4ccb-l2khb" event={"ID":"58552d00-05c8-4a22-b1ae-1446a33743bd","Type":"ContainerStarted","Data":"6bfff507bf5747e9c9d176746c6761c0f08990b1b17927ff01e813e0a948c250"}
Sep 17 23:20:51.967483 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:20:51.967096    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/limitador-operator-controller-manager-5ffb4b4ccb-l2khb"
Sep 17 23:20:51.991070 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:20:51.990995    2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/limitador-operator-controller-manager-5ffb4b4ccb-l2khb" podStartSLOduration=2.126426571 podStartE2EDuration="3.990977676s" totalImagesPullingTime="1.864551105s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-17 23:20:48 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-17 23:20:51.989738071 +0000 UTC m=+604.391634842" watchObservedRunningTime="2026-09-17 23:20:51.990977676 +0000 UTC m=+604.392874442"
Sep 17 23:20:52.938255 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:20:52.938165    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-84794c78d6-brcjh"
Sep 17 23:20:52.938451 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:20:52.938287    2562 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-84794c78d6-brcjh"
Sep 17 23:20:52.943962 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:20:52.943936    2562 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-84794c78d6-brcjh"
Sep 17 23:20:52.975530 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:20:52.975503    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-84794c78d6-brcjh"
Sep 17 23:20:53.144432 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:20:53.144391    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fwwc5_7e452bbb-7dc6-4c9d-a9b1-76400942f5b4/konnectivity-agent/0.log" limit=1024
Sep 17 23:20:53.255058 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:20:53.254967    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fwwc5_7e452bbb-7dc6-4c9d-a9b1-76400942f5b4/konnectivity-agent/0.log" limit=1024
Sep 17 23:20:53.351916 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:20:53.351883    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fwwc5_7e452bbb-7dc6-4c9d-a9b1-76400942f5b4/konnectivity-agent/0.log" limit=1024
Sep 17 23:20:53.452313 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:20:53.452277    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fwwc5_7e452bbb-7dc6-4c9d-a9b1-76400942f5b4/konnectivity-agent/0.log" limit=1024
Sep 17 23:20:54.357451 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:20:54.357415    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fwwc5_7e452bbb-7dc6-4c9d-a9b1-76400942f5b4/konnectivity-agent/0.log" limit=1024
Sep 17 23:20:54.457847 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:20:54.457804    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fwwc5_7e452bbb-7dc6-4c9d-a9b1-76400942f5b4/konnectivity-agent/0.log" limit=1024
Sep 17 23:20:55.153198 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:20:55.153143    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fwwc5_7e452bbb-7dc6-4c9d-a9b1-76400942f5b4/konnectivity-agent/0.log" limit=1024
Sep 17 23:20:55.254300 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:20:55.254263    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fwwc5_7e452bbb-7dc6-4c9d-a9b1-76400942f5b4/konnectivity-agent/0.log" limit=1024
Sep 17 23:20:57.946016 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:20:57.945985    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/authorino-operator-6fc8f864f4-tbvdm"
Sep 17 23:20:58.956961 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:20:58.956919    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fwwc5_7e452bbb-7dc6-4c9d-a9b1-76400942f5b4/konnectivity-agent/0.log" limit=1024
Sep 17 23:20:59.059744 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:20:59.059708    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fwwc5_7e452bbb-7dc6-4c9d-a9b1-76400942f5b4/konnectivity-agent/0.log" limit=1024
Sep 17 23:21:02.554388 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:21:02.554275    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fwwc5_7e452bbb-7dc6-4c9d-a9b1-76400942f5b4/konnectivity-agent/0.log" limit=1024
Sep 17 23:21:02.974240 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:21:02.974211    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/limitador-operator-controller-manager-5ffb4b4ccb-l2khb"
Sep 17 23:21:03.033491 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:21:03.033456    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fwwc5_7e452bbb-7dc6-4c9d-a9b1-76400942f5b4/konnectivity-agent/0.log" limit=1024
Sep 17 23:21:03.556191 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:21:03.556140    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fwwc5_7e452bbb-7dc6-4c9d-a9b1-76400942f5b4/konnectivity-agent/0.log" limit=1024
Sep 17 23:21:03.660951 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:21:03.660917    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fwwc5_7e452bbb-7dc6-4c9d-a9b1-76400942f5b4/konnectivity-agent/0.log" limit=1024
Sep 17 23:21:07.700656 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:21:07.700620    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fwwc5_7e452bbb-7dc6-4c9d-a9b1-76400942f5b4/konnectivity-agent/0.log" limit=1024
Sep 17 23:21:07.796772 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:21:07.796741    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fwwc5_7e452bbb-7dc6-4c9d-a9b1-76400942f5b4/konnectivity-agent/0.log" limit=1024
Sep 17 23:21:07.852423 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:21:07.852372    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fwwc5_7e452bbb-7dc6-4c9d-a9b1-76400942f5b4/konnectivity-agent/0.log" limit=1024
Sep 17 23:21:15.526918 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:21:15.526884    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fwwc5_7e452bbb-7dc6-4c9d-a9b1-76400942f5b4/konnectivity-agent/0.log" limit=1024
Sep 17 23:21:15.580708 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:21:15.580669    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fwwc5_7e452bbb-7dc6-4c9d-a9b1-76400942f5b4/konnectivity-agent/0.log" limit=1024
Sep 17 23:21:16.131308 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:21:16.131275    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fwwc5_7e452bbb-7dc6-4c9d-a9b1-76400942f5b4/konnectivity-agent/0.log" limit=1024
Sep 17 23:21:16.197778 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:21:16.197743    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fwwc5_7e452bbb-7dc6-4c9d-a9b1-76400942f5b4/konnectivity-agent/0.log" limit=1024
Sep 17 23:21:18.532935 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:21:18.532881    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fwwc5_7e452bbb-7dc6-4c9d-a9b1-76400942f5b4/konnectivity-agent/0.log" limit=1024
Sep 17 23:21:18.582624 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:21:18.582583    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fwwc5_7e452bbb-7dc6-4c9d-a9b1-76400942f5b4/konnectivity-agent/0.log" limit=1024
Sep 17 23:21:23.739339 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:21:23.739295    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fwwc5_7e452bbb-7dc6-4c9d-a9b1-76400942f5b4/konnectivity-agent/0.log" limit=1024
Sep 17 23:21:23.837714 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:21:23.837672    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fwwc5_7e452bbb-7dc6-4c9d-a9b1-76400942f5b4/konnectivity-agent/0.log" limit=1024
Sep 17 23:21:25.672626 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:21:25.672589    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fwwc5_7e452bbb-7dc6-4c9d-a9b1-76400942f5b4/konnectivity-agent/0.log" limit=1024
Sep 17 23:21:25.773232 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:21:25.773197    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fwwc5_7e452bbb-7dc6-4c9d-a9b1-76400942f5b4/konnectivity-agent/0.log" limit=1024
Sep 17 23:21:27.490096 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:21:27.490059    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fwwc5_7e452bbb-7dc6-4c9d-a9b1-76400942f5b4/konnectivity-agent/0.log" limit=1024
Sep 17 23:21:27.589928 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:21:27.589896    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fwwc5_7e452bbb-7dc6-4c9d-a9b1-76400942f5b4/konnectivity-agent/0.log" limit=1024
Sep 17 23:21:27.646655 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:21:27.646617    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fwwc5_7e452bbb-7dc6-4c9d-a9b1-76400942f5b4/konnectivity-agent/0.log" limit=1024
Sep 17 23:21:42.665611 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:21:42.665567    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fwwc5_7e452bbb-7dc6-4c9d-a9b1-76400942f5b4/konnectivity-agent/0.log" limit=1024
Sep 17 23:21:42.675029 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:21:42.674996    2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/limitador-limitador-8588749977-rb8v8"]
Sep 17 23:21:42.678141 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:21:42.678123    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-8588749977-rb8v8"
Sep 17 23:21:42.680442 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:21:42.680416    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"limitador-limits-config-limitador\""
Sep 17 23:21:42.680566 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:21:42.680448    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"default-dockercfg-fpbxh\""
Sep 17 23:21:42.686588 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:21:42.686555    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-8588749977-rb8v8"]
Sep 17 23:21:42.776504 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:21:42.776458    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-8588749977-rb8v8"]
Sep 17 23:21:42.777326 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:21:42.777285    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fwwc5_7e452bbb-7dc6-4c9d-a9b1-76400942f5b4/konnectivity-agent/0.log" limit=1024
Sep 17 23:21:42.791411 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:21:42.791368    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hmvnt\" (UniqueName: \"kubernetes.io/projected/4ce279f4-7098-4da7-a4d4-42b6d44a37dc-kube-api-access-hmvnt\") pod \"limitador-limitador-8588749977-rb8v8\" (UID: \"4ce279f4-7098-4da7-a4d4-42b6d44a37dc\") " pod="kuadrant-system/limitador-limitador-8588749977-rb8v8"
Sep 17 23:21:42.791574 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:21:42.791420    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/4ce279f4-7098-4da7-a4d4-42b6d44a37dc-config-file\") pod \"limitador-limitador-8588749977-rb8v8\" (UID: \"4ce279f4-7098-4da7-a4d4-42b6d44a37dc\") " pod="kuadrant-system/limitador-limitador-8588749977-rb8v8"
Sep 17 23:21:42.873378 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:21:42.873335    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fwwc5_7e452bbb-7dc6-4c9d-a9b1-76400942f5b4/konnectivity-agent/0.log" limit=1024
Sep 17 23:21:42.892439 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:21:42.892383    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hmvnt\" (UniqueName: \"kubernetes.io/projected/4ce279f4-7098-4da7-a4d4-42b6d44a37dc-kube-api-access-hmvnt\") pod \"limitador-limitador-8588749977-rb8v8\" (UID: \"4ce279f4-7098-4da7-a4d4-42b6d44a37dc\") " pod="kuadrant-system/limitador-limitador-8588749977-rb8v8"
Sep 17 23:21:42.892439 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:21:42.892443    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/4ce279f4-7098-4da7-a4d4-42b6d44a37dc-config-file\") pod \"limitador-limitador-8588749977-rb8v8\" (UID: \"4ce279f4-7098-4da7-a4d4-42b6d44a37dc\") " pod="kuadrant-system/limitador-limitador-8588749977-rb8v8"
Sep 17 23:21:42.893303 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:21:42.893282    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/4ce279f4-7098-4da7-a4d4-42b6d44a37dc-config-file\") pod \"limitador-limitador-8588749977-rb8v8\" (UID: \"4ce279f4-7098-4da7-a4d4-42b6d44a37dc\") " pod="kuadrant-system/limitador-limitador-8588749977-rb8v8"
Sep 17 23:21:42.900551 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:21:42.900527    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-hmvnt\" (UniqueName: \"kubernetes.io/projected/4ce279f4-7098-4da7-a4d4-42b6d44a37dc-kube-api-access-hmvnt\") pod \"limitador-limitador-8588749977-rb8v8\" (UID: \"4ce279f4-7098-4da7-a4d4-42b6d44a37dc\") " pod="kuadrant-system/limitador-limitador-8588749977-rb8v8"
Sep 17 23:21:42.989942 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:21:42.989826    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-8588749977-rb8v8"
Sep 17 23:21:43.261844 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:21:43.261794    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-8588749977-rb8v8"]
Sep 17 23:21:43.264607 ip-10-0-132-188 kubenswrapper[2562]: W0917 23:21:43.264573    2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4ce279f4_7098_4da7_a4d4_42b6d44a37dc.slice/crio-a8e8a464560b3affb1f407c6e4edc68206754a20c351a19fc078827748728600 WatchSource:0}: Error finding container a8e8a464560b3affb1f407c6e4edc68206754a20c351a19fc078827748728600: Status 404 returned error can't find the container with id a8e8a464560b3affb1f407c6e4edc68206754a20c351a19fc078827748728600
Sep 17 23:21:43.471958 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:21:43.471915    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fwwc5_7e452bbb-7dc6-4c9d-a9b1-76400942f5b4/konnectivity-agent/0.log" limit=1024
Sep 17 23:21:43.550213 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:21:43.550116    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fwwc5_7e452bbb-7dc6-4c9d-a9b1-76400942f5b4/konnectivity-agent/0.log" limit=1024
Sep 17 23:21:44.187464 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:21:44.187407    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-8588749977-rb8v8" event={"ID":"4ce279f4-7098-4da7-a4d4-42b6d44a37dc","Type":"ContainerStarted","Data":"a8e8a464560b3affb1f407c6e4edc68206754a20c351a19fc078827748728600"}
Sep 17 23:21:45.195778 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:21:45.195745    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-8588749977-rb8v8" event={"ID":"4ce279f4-7098-4da7-a4d4-42b6d44a37dc","Type":"ContainerStarted","Data":"28188b7f278b610d8aa013c53919dde88820ebbc5ef0588b57184beb9bc02e23"}
Sep 17 23:21:45.196231 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:21:45.195876    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/limitador-limitador-8588749977-rb8v8"
Sep 17 23:21:45.211589 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:21:45.211532    2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/limitador-limitador-8588749977-rb8v8" podStartSLOduration=2.078448545 podStartE2EDuration="3.211516166s" totalImagesPullingTime="1.133067621s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-17 23:21:42 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-17 23:21:45.210987008 +0000 UTC m=+657.612883771" watchObservedRunningTime="2026-09-17 23:21:45.211516166 +0000 UTC m=+657.613412930"
Sep 17 23:21:56.201437 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:21:56.201400    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/limitador-limitador-8588749977-rb8v8"
Sep 17 23:21:56.373962 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:21:56.373921    2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/limitador-limitador-8588749977-rb8v8"]
Sep 17 23:21:56.374192 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:21:56.374130    2562 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/limitador-limitador-8588749977-rb8v8" podUID="4ce279f4-7098-4da7-a4d4-42b6d44a37dc" containerName="limitador" containerID="cri-o://28188b7f278b610d8aa013c53919dde88820ebbc5ef0588b57184beb9bc02e23" gracePeriod=30
Sep 17 23:21:57.250441 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:21:57.250410    2562 generic.go:356] "Generic (PLEG): container finished" podID="4ce279f4-7098-4da7-a4d4-42b6d44a37dc" containerID="28188b7f278b610d8aa013c53919dde88820ebbc5ef0588b57184beb9bc02e23" exitCode=0
Sep 17 23:21:57.250821 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:21:57.250486    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-8588749977-rb8v8" event={"ID":"4ce279f4-7098-4da7-a4d4-42b6d44a37dc","Type":"ContainerDied","Data":"28188b7f278b610d8aa013c53919dde88820ebbc5ef0588b57184beb9bc02e23"}
Sep 17 23:21:57.321862 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:21:57.321837    2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-8588749977-rb8v8"
Sep 17 23:21:57.424166 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:21:57.424126    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/4ce279f4-7098-4da7-a4d4-42b6d44a37dc-config-file\") pod \"4ce279f4-7098-4da7-a4d4-42b6d44a37dc\" (UID: \"4ce279f4-7098-4da7-a4d4-42b6d44a37dc\") "
Sep 17 23:21:57.424389 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:21:57.424233    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-hmvnt\" (UniqueName: \"kubernetes.io/projected/4ce279f4-7098-4da7-a4d4-42b6d44a37dc-kube-api-access-hmvnt\") pod \"4ce279f4-7098-4da7-a4d4-42b6d44a37dc\" (UID: \"4ce279f4-7098-4da7-a4d4-42b6d44a37dc\") "
Sep 17 23:21:57.424524 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:21:57.424501    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/4ce279f4-7098-4da7-a4d4-42b6d44a37dc-config-file" (OuterVolumeSpecName: "config-file") pod "4ce279f4-7098-4da7-a4d4-42b6d44a37dc" (UID: "4ce279f4-7098-4da7-a4d4-42b6d44a37dc"). InnerVolumeSpecName "config-file". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 17 23:21:57.426403 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:21:57.426376    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/4ce279f4-7098-4da7-a4d4-42b6d44a37dc-kube-api-access-hmvnt" (OuterVolumeSpecName: "kube-api-access-hmvnt") pod "4ce279f4-7098-4da7-a4d4-42b6d44a37dc" (UID: "4ce279f4-7098-4da7-a4d4-42b6d44a37dc"). InnerVolumeSpecName "kube-api-access-hmvnt". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 17 23:21:57.525394 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:21:57.525294    2562 reconciler_common.go:299] "Volume detached for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/4ce279f4-7098-4da7-a4d4-42b6d44a37dc-config-file\") on node \"ip-10-0-132-188.ec2.internal\" DevicePath \"\""
Sep 17 23:21:57.525394 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:21:57.525331    2562 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-hmvnt\" (UniqueName: \"kubernetes.io/projected/4ce279f4-7098-4da7-a4d4-42b6d44a37dc-kube-api-access-hmvnt\") on node \"ip-10-0-132-188.ec2.internal\" DevicePath \"\""
Sep 17 23:21:58.256122 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:21:58.256089    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-8588749977-rb8v8" event={"ID":"4ce279f4-7098-4da7-a4d4-42b6d44a37dc","Type":"ContainerDied","Data":"a8e8a464560b3affb1f407c6e4edc68206754a20c351a19fc078827748728600"}
Sep 17 23:21:58.256122 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:21:58.256110    2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-8588749977-rb8v8"
Sep 17 23:21:58.256623 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:21:58.256152    2562 scope.go:117] "RemoveContainer" containerID="28188b7f278b610d8aa013c53919dde88820ebbc5ef0588b57184beb9bc02e23"
Sep 17 23:21:58.271000 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:21:58.270972    2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/limitador-limitador-8588749977-rb8v8"]
Sep 17 23:21:58.274495 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:21:58.274469    2562 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/limitador-limitador-8588749977-rb8v8"]
Sep 17 23:21:59.671403 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:21:59.671367    2562 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="4ce279f4-7098-4da7-a4d4-42b6d44a37dc" path="/var/lib/kubelet/pods/4ce279f4-7098-4da7-a4d4-42b6d44a37dc/volumes"
Sep 17 23:22:02.320284 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:22:02.320243    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fwwc5_7e452bbb-7dc6-4c9d-a9b1-76400942f5b4/konnectivity-agent/0.log" limit=1024
Sep 17 23:22:02.458810 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:22:02.458780    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fwwc5_7e452bbb-7dc6-4c9d-a9b1-76400942f5b4/konnectivity-agent/0.log" limit=1024
Sep 17 23:22:10.068380 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:22:10.068334    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fwwc5_7e452bbb-7dc6-4c9d-a9b1-76400942f5b4/konnectivity-agent/0.log" limit=1024
Sep 17 23:22:10.169016 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:22:10.168986    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fwwc5_7e452bbb-7dc6-4c9d-a9b1-76400942f5b4/konnectivity-agent/0.log" limit=1024
Sep 17 23:22:23.223332 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:22:23.223297    2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/istiod-openshift-gateway-5677ccb49b-np75w"]
Sep 17 23:22:23.223743 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:22:23.223623    2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="4ce279f4-7098-4da7-a4d4-42b6d44a37dc" containerName="limitador"
Sep 17 23:22:23.223743 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:22:23.223634    2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="4ce279f4-7098-4da7-a4d4-42b6d44a37dc" containerName="limitador"
Sep 17 23:22:23.223743 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:22:23.223696    2562 memory_manager.go:368] "RemoveStaleState removing state" podUID="4ce279f4-7098-4da7-a4d4-42b6d44a37dc" containerName="limitador"
Sep 17 23:22:23.227673 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:22:23.227655    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-np75w"
Sep 17 23:22:23.237558 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:22:23.237528    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-5677ccb49b-np75w"]
Sep 17 23:22:23.343340 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:22:23.343291    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/eb73a80f-c670-434f-9ddb-7953608877b2-local-certs\") pod \"istiod-openshift-gateway-5677ccb49b-np75w\" (UID: \"eb73a80f-c670-434f-9ddb-7953608877b2\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-np75w"
Sep 17 23:22:23.343340 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:22:23.343340    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/eb73a80f-c670-434f-9ddb-7953608877b2-istio-kubeconfig\") pod \"istiod-openshift-gateway-5677ccb49b-np75w\" (UID: \"eb73a80f-c670-434f-9ddb-7953608877b2\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-np75w"
Sep 17 23:22:23.343590 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:22:23.343370    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/eb73a80f-c670-434f-9ddb-7953608877b2-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-5677ccb49b-np75w\" (UID: \"eb73a80f-c670-434f-9ddb-7953608877b2\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-np75w"
Sep 17 23:22:23.343590 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:22:23.343437    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/eb73a80f-c670-434f-9ddb-7953608877b2-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-5677ccb49b-np75w\" (UID: \"eb73a80f-c670-434f-9ddb-7953608877b2\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-np75w"
Sep 17 23:22:23.343590 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:22:23.343491    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/eb73a80f-c670-434f-9ddb-7953608877b2-istio-token\") pod \"istiod-openshift-gateway-5677ccb49b-np75w\" (UID: \"eb73a80f-c670-434f-9ddb-7953608877b2\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-np75w"
Sep 17 23:22:23.343590 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:22:23.343524    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-spdb4\" (UniqueName: \"kubernetes.io/projected/eb73a80f-c670-434f-9ddb-7953608877b2-kube-api-access-spdb4\") pod \"istiod-openshift-gateway-5677ccb49b-np75w\" (UID: \"eb73a80f-c670-434f-9ddb-7953608877b2\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-np75w"
Sep 17 23:22:23.343590 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:22:23.343552    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/eb73a80f-c670-434f-9ddb-7953608877b2-cacerts\") pod \"istiod-openshift-gateway-5677ccb49b-np75w\" (UID: \"eb73a80f-c670-434f-9ddb-7953608877b2\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-np75w"
Sep 17 23:22:23.444419 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:22:23.444354    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/eb73a80f-c670-434f-9ddb-7953608877b2-local-certs\") pod \"istiod-openshift-gateway-5677ccb49b-np75w\" (UID: \"eb73a80f-c670-434f-9ddb-7953608877b2\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-np75w"
Sep 17 23:22:23.444419 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:22:23.444421    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/eb73a80f-c670-434f-9ddb-7953608877b2-istio-kubeconfig\") pod \"istiod-openshift-gateway-5677ccb49b-np75w\" (UID: \"eb73a80f-c670-434f-9ddb-7953608877b2\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-np75w"
Sep 17 23:22:23.444667 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:22:23.444455    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/eb73a80f-c670-434f-9ddb-7953608877b2-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-5677ccb49b-np75w\" (UID: \"eb73a80f-c670-434f-9ddb-7953608877b2\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-np75w"
Sep 17 23:22:23.444667 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:22:23.444491    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/eb73a80f-c670-434f-9ddb-7953608877b2-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-5677ccb49b-np75w\" (UID: \"eb73a80f-c670-434f-9ddb-7953608877b2\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-np75w"
Sep 17 23:22:23.444667 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:22:23.444535    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/eb73a80f-c670-434f-9ddb-7953608877b2-istio-token\") pod \"istiod-openshift-gateway-5677ccb49b-np75w\" (UID: \"eb73a80f-c670-434f-9ddb-7953608877b2\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-np75w"
Sep 17 23:22:23.444667 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:22:23.444578    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-spdb4\" (UniqueName: \"kubernetes.io/projected/eb73a80f-c670-434f-9ddb-7953608877b2-kube-api-access-spdb4\") pod \"istiod-openshift-gateway-5677ccb49b-np75w\" (UID: \"eb73a80f-c670-434f-9ddb-7953608877b2\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-np75w"
Sep 17 23:22:23.444667 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:22:23.444617    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/eb73a80f-c670-434f-9ddb-7953608877b2-cacerts\") pod \"istiod-openshift-gateway-5677ccb49b-np75w\" (UID: \"eb73a80f-c670-434f-9ddb-7953608877b2\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-np75w"
Sep 17 23:22:23.445352 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:22:23.445322    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/eb73a80f-c670-434f-9ddb-7953608877b2-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-5677ccb49b-np75w\" (UID: \"eb73a80f-c670-434f-9ddb-7953608877b2\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-np75w"
Sep 17 23:22:23.446768 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:22:23.446737    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/eb73a80f-c670-434f-9ddb-7953608877b2-local-certs\") pod \"istiod-openshift-gateway-5677ccb49b-np75w\" (UID: \"eb73a80f-c670-434f-9ddb-7953608877b2\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-np75w"
Sep 17 23:22:23.446929 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:22:23.446908    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/eb73a80f-c670-434f-9ddb-7953608877b2-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-5677ccb49b-np75w\" (UID: \"eb73a80f-c670-434f-9ddb-7953608877b2\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-np75w"
Sep 17 23:22:23.446997 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:22:23.446977    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/eb73a80f-c670-434f-9ddb-7953608877b2-cacerts\") pod \"istiod-openshift-gateway-5677ccb49b-np75w\" (UID: \"eb73a80f-c670-434f-9ddb-7953608877b2\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-np75w"
Sep 17 23:22:23.447035 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:22:23.447003    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/eb73a80f-c670-434f-9ddb-7953608877b2-istio-kubeconfig\") pod \"istiod-openshift-gateway-5677ccb49b-np75w\" (UID: \"eb73a80f-c670-434f-9ddb-7953608877b2\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-np75w"
Sep 17 23:22:23.452569 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:22:23.452542    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/eb73a80f-c670-434f-9ddb-7953608877b2-istio-token\") pod \"istiod-openshift-gateway-5677ccb49b-np75w\" (UID: \"eb73a80f-c670-434f-9ddb-7953608877b2\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-np75w"
Sep 17 23:22:23.453230 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:22:23.453212    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-spdb4\" (UniqueName: \"kubernetes.io/projected/eb73a80f-c670-434f-9ddb-7953608877b2-kube-api-access-spdb4\") pod \"istiod-openshift-gateway-5677ccb49b-np75w\" (UID: \"eb73a80f-c670-434f-9ddb-7953608877b2\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-np75w"
Sep 17 23:22:23.537752 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:22:23.537648    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-np75w"
Sep 17 23:22:23.828227 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:22:23.828194    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-5677ccb49b-np75w"]
Sep 17 23:22:23.830592 ip-10-0-132-188 kubenswrapper[2562]: W0917 23:22:23.830564    2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podeb73a80f_c670_434f_9ddb_7953608877b2.slice/crio-92d225cbd7aaebd27b9e99bc15d897678353c235f9b135d07d35511fc659524e WatchSource:0}: Error finding container 92d225cbd7aaebd27b9e99bc15d897678353c235f9b135d07d35511fc659524e: Status 404 returned error can't find the container with id 92d225cbd7aaebd27b9e99bc15d897678353c235f9b135d07d35511fc659524e
Sep 17 23:22:23.832708 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:22:23.832670    2562 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"}
Sep 17 23:22:23.832787 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:22:23.832751    2562 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"}
Sep 17 23:22:24.371904 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:22:24.371866    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-np75w" event={"ID":"eb73a80f-c670-434f-9ddb-7953608877b2","Type":"ContainerStarted","Data":"b818403cfcf161a11f9e685c0899e1a18ad7ae504749dec29f2943978a229258"}
Sep 17 23:22:24.372451 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:22:24.371921    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-np75w" event={"ID":"eb73a80f-c670-434f-9ddb-7953608877b2","Type":"ContainerStarted","Data":"92d225cbd7aaebd27b9e99bc15d897678353c235f9b135d07d35511fc659524e"}
Sep 17 23:22:24.372451 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:22:24.372037    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-np75w"
Sep 17 23:22:24.374084 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:22:24.374051    2562 patch_prober.go:28] interesting pod/istiod-openshift-gateway-5677ccb49b-np75w container/discovery namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body=
Sep 17 23:22:24.374233 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:22:24.374161    2562 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-np75w" podUID="eb73a80f-c670-434f-9ddb-7953608877b2" containerName="discovery" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Sep 17 23:22:24.395271 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:22:24.395219    2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-np75w" podStartSLOduration=1.3952006639999999 podStartE2EDuration="1.395200664s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-17 23:22:23 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-17 23:22:24.392732363 +0000 UTC m=+696.794629128" watchObservedRunningTime="2026-09-17 23:22:24.395200664 +0000 UTC m=+696.797097427"
Sep 17 23:22:25.376649 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:22:25.376612    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-np75w"
Sep 17 23:22:25.425762 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:22:25.425727    2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7b7995d5cd-nt48c"]
Sep 17 23:22:25.426024 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:22:25.425996    2562 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-nt48c" podUID="cb721446-8bdd-4787-93e7-257294b551f3" containerName="discovery" containerID="cri-o://2e7d49868dde51623999031e4408beb85223f9429e6578d4ca64106ac646501c" gracePeriod=30
Sep 17 23:22:25.685697 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:22:25.685667    2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-nt48c"
Sep 17 23:22:25.762422 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:22:25.762390    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/cb721446-8bdd-4787-93e7-257294b551f3-local-certs\") pod \"cb721446-8bdd-4787-93e7-257294b551f3\" (UID: \"cb721446-8bdd-4787-93e7-257294b551f3\") "
Sep 17 23:22:25.762630 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:22:25.762446    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-chtj6\" (UniqueName: \"kubernetes.io/projected/cb721446-8bdd-4787-93e7-257294b551f3-kube-api-access-chtj6\") pod \"cb721446-8bdd-4787-93e7-257294b551f3\" (UID: \"cb721446-8bdd-4787-93e7-257294b551f3\") "
Sep 17 23:22:25.762630 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:22:25.762467    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/cb721446-8bdd-4787-93e7-257294b551f3-cacerts\") pod \"cb721446-8bdd-4787-93e7-257294b551f3\" (UID: \"cb721446-8bdd-4787-93e7-257294b551f3\") "
Sep 17 23:22:25.762630 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:22:25.762534    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/cb721446-8bdd-4787-93e7-257294b551f3-istio-csr-dns-cert\") pod \"cb721446-8bdd-4787-93e7-257294b551f3\" (UID: \"cb721446-8bdd-4787-93e7-257294b551f3\") "
Sep 17 23:22:25.762778 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:22:25.762674    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/cb721446-8bdd-4787-93e7-257294b551f3-istio-csr-ca-configmap\") pod \"cb721446-8bdd-4787-93e7-257294b551f3\" (UID: \"cb721446-8bdd-4787-93e7-257294b551f3\") "
Sep 17 23:22:25.762778 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:22:25.762740    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/cb721446-8bdd-4787-93e7-257294b551f3-istio-token\") pod \"cb721446-8bdd-4787-93e7-257294b551f3\" (UID: \"cb721446-8bdd-4787-93e7-257294b551f3\") "
Sep 17 23:22:25.762883 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:22:25.762777    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/cb721446-8bdd-4787-93e7-257294b551f3-istio-kubeconfig\") pod \"cb721446-8bdd-4787-93e7-257294b551f3\" (UID: \"cb721446-8bdd-4787-93e7-257294b551f3\") "
Sep 17 23:22:25.763235 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:22:25.763045    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/cb721446-8bdd-4787-93e7-257294b551f3-istio-csr-ca-configmap" (OuterVolumeSpecName: "istio-csr-ca-configmap") pod "cb721446-8bdd-4787-93e7-257294b551f3" (UID: "cb721446-8bdd-4787-93e7-257294b551f3"). InnerVolumeSpecName "istio-csr-ca-configmap". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 17 23:22:25.763235 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:22:25.763140    2562 reconciler_common.go:299] "Volume detached for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/cb721446-8bdd-4787-93e7-257294b551f3-istio-csr-ca-configmap\") on node \"ip-10-0-132-188.ec2.internal\" DevicePath \"\""
Sep 17 23:22:25.765002 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:22:25.764973    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/cb721446-8bdd-4787-93e7-257294b551f3-cacerts" (OuterVolumeSpecName: "cacerts") pod "cb721446-8bdd-4787-93e7-257294b551f3" (UID: "cb721446-8bdd-4787-93e7-257294b551f3"). InnerVolumeSpecName "cacerts". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 17 23:22:25.765233 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:22:25.765203    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/cb721446-8bdd-4787-93e7-257294b551f3-local-certs" (OuterVolumeSpecName: "local-certs") pod "cb721446-8bdd-4787-93e7-257294b551f3" (UID: "cb721446-8bdd-4787-93e7-257294b551f3"). InnerVolumeSpecName "local-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 17 23:22:25.765337 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:22:25.765293    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/cb721446-8bdd-4787-93e7-257294b551f3-istio-kubeconfig" (OuterVolumeSpecName: "istio-kubeconfig") pod "cb721446-8bdd-4787-93e7-257294b551f3" (UID: "cb721446-8bdd-4787-93e7-257294b551f3"). InnerVolumeSpecName "istio-kubeconfig". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 17 23:22:25.765337 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:22:25.765322    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/cb721446-8bdd-4787-93e7-257294b551f3-istio-csr-dns-cert" (OuterVolumeSpecName: "istio-csr-dns-cert") pod "cb721446-8bdd-4787-93e7-257294b551f3" (UID: "cb721446-8bdd-4787-93e7-257294b551f3"). InnerVolumeSpecName "istio-csr-dns-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 17 23:22:25.765413 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:22:25.765338    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/cb721446-8bdd-4787-93e7-257294b551f3-kube-api-access-chtj6" (OuterVolumeSpecName: "kube-api-access-chtj6") pod "cb721446-8bdd-4787-93e7-257294b551f3" (UID: "cb721446-8bdd-4787-93e7-257294b551f3"). InnerVolumeSpecName "kube-api-access-chtj6". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 17 23:22:25.765413 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:22:25.765335    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/cb721446-8bdd-4787-93e7-257294b551f3-istio-token" (OuterVolumeSpecName: "istio-token") pod "cb721446-8bdd-4787-93e7-257294b551f3" (UID: "cb721446-8bdd-4787-93e7-257294b551f3"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 17 23:22:25.863578 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:22:25.863539    2562 reconciler_common.go:299] "Volume detached for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/cb721446-8bdd-4787-93e7-257294b551f3-istio-csr-dns-cert\") on node \"ip-10-0-132-188.ec2.internal\" DevicePath \"\""
Sep 17 23:22:25.863578 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:22:25.863572    2562 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/cb721446-8bdd-4787-93e7-257294b551f3-istio-token\") on node \"ip-10-0-132-188.ec2.internal\" DevicePath \"\""
Sep 17 23:22:25.863578 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:22:25.863582    2562 reconciler_common.go:299] "Volume detached for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/cb721446-8bdd-4787-93e7-257294b551f3-istio-kubeconfig\") on node \"ip-10-0-132-188.ec2.internal\" DevicePath \"\""
Sep 17 23:22:25.863817 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:22:25.863591    2562 reconciler_common.go:299] "Volume detached for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/cb721446-8bdd-4787-93e7-257294b551f3-local-certs\") on node \"ip-10-0-132-188.ec2.internal\" DevicePath \"\""
Sep 17 23:22:25.863817 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:22:25.863600    2562 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-chtj6\" (UniqueName: \"kubernetes.io/projected/cb721446-8bdd-4787-93e7-257294b551f3-kube-api-access-chtj6\") on node \"ip-10-0-132-188.ec2.internal\" DevicePath \"\""
Sep 17 23:22:25.863817 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:22:25.863609    2562 reconciler_common.go:299] "Volume detached for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/cb721446-8bdd-4787-93e7-257294b551f3-cacerts\") on node \"ip-10-0-132-188.ec2.internal\" DevicePath \"\""
Sep 17 23:22:26.381396 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:22:26.381361    2562 generic.go:356] "Generic (PLEG): container finished" podID="cb721446-8bdd-4787-93e7-257294b551f3" containerID="2e7d49868dde51623999031e4408beb85223f9429e6578d4ca64106ac646501c" exitCode=0
Sep 17 23:22:26.381891 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:22:26.381411    2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-nt48c"
Sep 17 23:22:26.381891 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:22:26.381446    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-nt48c" event={"ID":"cb721446-8bdd-4787-93e7-257294b551f3","Type":"ContainerDied","Data":"2e7d49868dde51623999031e4408beb85223f9429e6578d4ca64106ac646501c"}
Sep 17 23:22:26.381891 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:22:26.381496    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-nt48c" event={"ID":"cb721446-8bdd-4787-93e7-257294b551f3","Type":"ContainerDied","Data":"2d5d3131c5b458222d9d731d466fc0b7f43a6bf245e3838d794107f3e481405a"}
Sep 17 23:22:26.381891 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:22:26.381517    2562 scope.go:117] "RemoveContainer" containerID="2e7d49868dde51623999031e4408beb85223f9429e6578d4ca64106ac646501c"
Sep 17 23:22:26.391238 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:22:26.391218    2562 scope.go:117] "RemoveContainer" containerID="2e7d49868dde51623999031e4408beb85223f9429e6578d4ca64106ac646501c"
Sep 17 23:22:26.391587 ip-10-0-132-188 kubenswrapper[2562]: E0917 23:22:26.391566    2562 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"2e7d49868dde51623999031e4408beb85223f9429e6578d4ca64106ac646501c\": container with ID starting with 2e7d49868dde51623999031e4408beb85223f9429e6578d4ca64106ac646501c not found: ID does not exist" containerID="2e7d49868dde51623999031e4408beb85223f9429e6578d4ca64106ac646501c"
Sep 17 23:22:26.391640 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:22:26.391597    2562 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"2e7d49868dde51623999031e4408beb85223f9429e6578d4ca64106ac646501c"} err="failed to get container status \"2e7d49868dde51623999031e4408beb85223f9429e6578d4ca64106ac646501c\": rpc error: code = NotFound desc = could not find container \"2e7d49868dde51623999031e4408beb85223f9429e6578d4ca64106ac646501c\": container with ID starting with 2e7d49868dde51623999031e4408beb85223f9429e6578d4ca64106ac646501c not found: ID does not exist"
Sep 17 23:22:26.406459 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:22:26.406426    2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7b7995d5cd-nt48c"]
Sep 17 23:22:26.412834 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:22:26.412803    2562 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7b7995d5cd-nt48c"]
Sep 17 23:22:27.666857 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:22:27.666823    2562 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="cb721446-8bdd-4787-93e7-257294b551f3" path="/var/lib/kubelet/pods/cb721446-8bdd-4787-93e7-257294b551f3/volumes"
Sep 17 23:22:33.779688 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:22:33.779599    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fwwc5_7e452bbb-7dc6-4c9d-a9b1-76400942f5b4/konnectivity-agent/0.log" limit=1024
Sep 17 23:22:33.843648 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:22:33.843615    2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kserve/kserve-controller-manager-5c5fcd7899-7tdw7"]
Sep 17 23:22:33.843989 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:22:33.843973    2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="cb721446-8bdd-4787-93e7-257294b551f3" containerName="discovery"
Sep 17 23:22:33.844068 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:22:33.844010    2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="cb721446-8bdd-4787-93e7-257294b551f3" containerName="discovery"
Sep 17 23:22:33.844123 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:22:33.844116    2562 memory_manager.go:368] "RemoveStaleState removing state" podUID="cb721446-8bdd-4787-93e7-257294b551f3" containerName="discovery"
Sep 17 23:22:33.846537 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:22:33.846513    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve/kserve-controller-manager-5c5fcd7899-7tdw7"
Sep 17 23:22:33.848993 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:22:33.848967    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kserve\"/\"kserve-webhook-server-cert\""
Sep 17 23:22:33.849149 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:22:33.849012    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kserve\"/\"kube-root-ca.crt\""
Sep 17 23:22:33.849149 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:22:33.849012    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kserve\"/\"openshift-service-ca.crt\""
Sep 17 23:22:33.849738 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:22:33.849720    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kserve\"/\"kserve-controller-manager-dockercfg-bbmg2\""
Sep 17 23:22:33.856031 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:22:33.856001    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve/kserve-controller-manager-5c5fcd7899-7tdw7"]
Sep 17 23:22:33.864119 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:22:33.864093    2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kserve/llmisvc-controller-manager-5cbcfb6fbf-n7654"]
Sep 17 23:22:33.866465 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:22:33.866430    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fwwc5_7e452bbb-7dc6-4c9d-a9b1-76400942f5b4/konnectivity-agent/0.log" limit=1024
Sep 17 23:22:33.866843 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:22:33.866823    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve/llmisvc-controller-manager-5cbcfb6fbf-n7654"
Sep 17 23:22:33.869081 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:22:33.869054    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kserve\"/\"llmisvc-webhook-server-cert\""
Sep 17 23:22:33.869207 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:22:33.869089    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kserve\"/\"llmisvc-controller-manager-dockercfg-hrphh\""
Sep 17 23:22:33.876798 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:22:33.876771    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve/llmisvc-controller-manager-5cbcfb6fbf-n7654"]
Sep 17 23:22:33.883516 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:22:33.883492    2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kserve/seaweedfs-64568bcd49-xx74p"]
Sep 17 23:22:33.886435 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:22:33.886412    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve/seaweedfs-64568bcd49-xx74p"
Sep 17 23:22:33.888781 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:22:33.888756    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kserve\"/\"mlpipeline-s3-artifact\""
Sep 17 23:22:33.888908 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:22:33.888847    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kserve\"/\"default-dockercfg-jw2ph\""
Sep 17 23:22:33.896150 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:22:33.896126    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve/seaweedfs-64568bcd49-xx74p"]
Sep 17 23:22:33.932350 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:22:33.932316    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zmpkx\" (UniqueName: \"kubernetes.io/projected/ba27e9c4-76d1-474b-b78d-2e9c1531026b-kube-api-access-zmpkx\") pod \"kserve-controller-manager-5c5fcd7899-7tdw7\" (UID: \"ba27e9c4-76d1-474b-b78d-2e9c1531026b\") " pod="kserve/kserve-controller-manager-5c5fcd7899-7tdw7"
Sep 17 23:22:33.932532 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:22:33.932397    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-kpr7s\" (UniqueName: \"kubernetes.io/projected/aa8896c1-9f88-4d89-a8a0-fcebd0ac028e-kube-api-access-kpr7s\") pod \"llmisvc-controller-manager-5cbcfb6fbf-n7654\" (UID: \"aa8896c1-9f88-4d89-a8a0-fcebd0ac028e\") " pod="kserve/llmisvc-controller-manager-5cbcfb6fbf-n7654"
Sep 17 23:22:33.932532 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:22:33.932454    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/aa8896c1-9f88-4d89-a8a0-fcebd0ac028e-cert\") pod \"llmisvc-controller-manager-5cbcfb6fbf-n7654\" (UID: \"aa8896c1-9f88-4d89-a8a0-fcebd0ac028e\") " pod="kserve/llmisvc-controller-manager-5cbcfb6fbf-n7654"
Sep 17 23:22:33.932532 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:22:33.932526    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/ba27e9c4-76d1-474b-b78d-2e9c1531026b-cert\") pod \"kserve-controller-manager-5c5fcd7899-7tdw7\" (UID: \"ba27e9c4-76d1-474b-b78d-2e9c1531026b\") " pod="kserve/kserve-controller-manager-5c5fcd7899-7tdw7"
Sep 17 23:22:33.945786 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:22:33.945752    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fwwc5_7e452bbb-7dc6-4c9d-a9b1-76400942f5b4/konnectivity-agent/0.log" limit=1024
Sep 17 23:22:34.033242 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:22:34.033139    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-z6rf4\" (UniqueName: \"kubernetes.io/projected/d1bccb49-bf6c-4f3b-aa32-0673a7ce5651-kube-api-access-z6rf4\") pod \"seaweedfs-64568bcd49-xx74p\" (UID: \"d1bccb49-bf6c-4f3b-aa32-0673a7ce5651\") " pod="kserve/seaweedfs-64568bcd49-xx74p"
Sep 17 23:22:34.033242 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:22:34.033223    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/ba27e9c4-76d1-474b-b78d-2e9c1531026b-cert\") pod \"kserve-controller-manager-5c5fcd7899-7tdw7\" (UID: \"ba27e9c4-76d1-474b-b78d-2e9c1531026b\") " pod="kserve/kserve-controller-manager-5c5fcd7899-7tdw7"
Sep 17 23:22:34.033434 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:22:34.033248    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zmpkx\" (UniqueName: \"kubernetes.io/projected/ba27e9c4-76d1-474b-b78d-2e9c1531026b-kube-api-access-zmpkx\") pod \"kserve-controller-manager-5c5fcd7899-7tdw7\" (UID: \"ba27e9c4-76d1-474b-b78d-2e9c1531026b\") " pod="kserve/kserve-controller-manager-5c5fcd7899-7tdw7"
Sep 17 23:22:34.033434 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:22:34.033271    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kpr7s\" (UniqueName: \"kubernetes.io/projected/aa8896c1-9f88-4d89-a8a0-fcebd0ac028e-kube-api-access-kpr7s\") pod \"llmisvc-controller-manager-5cbcfb6fbf-n7654\" (UID: \"aa8896c1-9f88-4d89-a8a0-fcebd0ac028e\") " pod="kserve/llmisvc-controller-manager-5cbcfb6fbf-n7654"
Sep 17 23:22:34.033434 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:22:34.033321    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"data\" (UniqueName: \"kubernetes.io/empty-dir/d1bccb49-bf6c-4f3b-aa32-0673a7ce5651-data\") pod \"seaweedfs-64568bcd49-xx74p\" (UID: \"d1bccb49-bf6c-4f3b-aa32-0673a7ce5651\") " pod="kserve/seaweedfs-64568bcd49-xx74p"
Sep 17 23:22:34.033434 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:22:34.033337    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/aa8896c1-9f88-4d89-a8a0-fcebd0ac028e-cert\") pod \"llmisvc-controller-manager-5cbcfb6fbf-n7654\" (UID: \"aa8896c1-9f88-4d89-a8a0-fcebd0ac028e\") " pod="kserve/llmisvc-controller-manager-5cbcfb6fbf-n7654"
Sep 17 23:22:34.035735 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:22:34.035710    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/aa8896c1-9f88-4d89-a8a0-fcebd0ac028e-cert\") pod \"llmisvc-controller-manager-5cbcfb6fbf-n7654\" (UID: \"aa8896c1-9f88-4d89-a8a0-fcebd0ac028e\") " pod="kserve/llmisvc-controller-manager-5cbcfb6fbf-n7654"
Sep 17 23:22:34.035858 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:22:34.035826    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/ba27e9c4-76d1-474b-b78d-2e9c1531026b-cert\") pod \"kserve-controller-manager-5c5fcd7899-7tdw7\" (UID: \"ba27e9c4-76d1-474b-b78d-2e9c1531026b\") " pod="kserve/kserve-controller-manager-5c5fcd7899-7tdw7"
Sep 17 23:22:34.041265 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:22:34.041229    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-zmpkx\" (UniqueName: \"kubernetes.io/projected/ba27e9c4-76d1-474b-b78d-2e9c1531026b-kube-api-access-zmpkx\") pod \"kserve-controller-manager-5c5fcd7899-7tdw7\" (UID: \"ba27e9c4-76d1-474b-b78d-2e9c1531026b\") " pod="kserve/kserve-controller-manager-5c5fcd7899-7tdw7"
Sep 17 23:22:34.041389 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:22:34.041272    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-kpr7s\" (UniqueName: \"kubernetes.io/projected/aa8896c1-9f88-4d89-a8a0-fcebd0ac028e-kube-api-access-kpr7s\") pod \"llmisvc-controller-manager-5cbcfb6fbf-n7654\" (UID: \"aa8896c1-9f88-4d89-a8a0-fcebd0ac028e\") " pod="kserve/llmisvc-controller-manager-5cbcfb6fbf-n7654"
Sep 17 23:22:34.046573 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:22:34.046540    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fwwc5_7e452bbb-7dc6-4c9d-a9b1-76400942f5b4/konnectivity-agent/0.log" limit=1024
Sep 17 23:22:34.134484 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:22:34.134450    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"data\" (UniqueName: \"kubernetes.io/empty-dir/d1bccb49-bf6c-4f3b-aa32-0673a7ce5651-data\") pod \"seaweedfs-64568bcd49-xx74p\" (UID: \"d1bccb49-bf6c-4f3b-aa32-0673a7ce5651\") " pod="kserve/seaweedfs-64568bcd49-xx74p"
Sep 17 23:22:34.134653 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:22:34.134497    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-z6rf4\" (UniqueName: \"kubernetes.io/projected/d1bccb49-bf6c-4f3b-aa32-0673a7ce5651-kube-api-access-z6rf4\") pod \"seaweedfs-64568bcd49-xx74p\" (UID: \"d1bccb49-bf6c-4f3b-aa32-0673a7ce5651\") " pod="kserve/seaweedfs-64568bcd49-xx74p"
Sep 17 23:22:34.134869 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:22:34.134847    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"data\" (UniqueName: \"kubernetes.io/empty-dir/d1bccb49-bf6c-4f3b-aa32-0673a7ce5651-data\") pod \"seaweedfs-64568bcd49-xx74p\" (UID: \"d1bccb49-bf6c-4f3b-aa32-0673a7ce5651\") " pod="kserve/seaweedfs-64568bcd49-xx74p"
Sep 17 23:22:34.142827 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:22:34.142796    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-z6rf4\" (UniqueName: \"kubernetes.io/projected/d1bccb49-bf6c-4f3b-aa32-0673a7ce5651-kube-api-access-z6rf4\") pod \"seaweedfs-64568bcd49-xx74p\" (UID: \"d1bccb49-bf6c-4f3b-aa32-0673a7ce5651\") " pod="kserve/seaweedfs-64568bcd49-xx74p"
Sep 17 23:22:34.147548 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:22:34.147514    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fwwc5_7e452bbb-7dc6-4c9d-a9b1-76400942f5b4/konnectivity-agent/0.log" limit=1024
Sep 17 23:22:34.158343 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:22:34.158312    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve/kserve-controller-manager-5c5fcd7899-7tdw7"
Sep 17 23:22:34.178248 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:22:34.178221    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve/llmisvc-controller-manager-5cbcfb6fbf-n7654"
Sep 17 23:22:34.197736 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:22:34.197701    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve/seaweedfs-64568bcd49-xx74p"
Sep 17 23:22:34.360971 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:22:34.360938    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fwwc5_7e452bbb-7dc6-4c9d-a9b1-76400942f5b4/konnectivity-agent/0.log" limit=1024
Sep 17 23:22:34.434629 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:22:34.434596    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve/kserve-controller-manager-5c5fcd7899-7tdw7"]
Sep 17 23:22:34.437074 ip-10-0-132-188 kubenswrapper[2562]: W0917 23:22:34.437040    2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podba27e9c4_76d1_474b_b78d_2e9c1531026b.slice/crio-c95607e015eb4257eb3807c1b406e04a002b29bb9085d08b91a1c91cab42a987 WatchSource:0}: Error finding container c95607e015eb4257eb3807c1b406e04a002b29bb9085d08b91a1c91cab42a987: Status 404 returned error can't find the container with id c95607e015eb4257eb3807c1b406e04a002b29bb9085d08b91a1c91cab42a987
Sep 17 23:22:34.437286 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:22:34.437262    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fwwc5_7e452bbb-7dc6-4c9d-a9b1-76400942f5b4/konnectivity-agent/0.log" limit=1024
Sep 17 23:22:34.460315 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:22:34.460282    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve/llmisvc-controller-manager-5cbcfb6fbf-n7654"]
Sep 17 23:22:34.462917 ip-10-0-132-188 kubenswrapper[2562]: W0917 23:22:34.462890    2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-podaa8896c1_9f88_4d89_a8a0_fcebd0ac028e.slice/crio-59a66dff9a589a989ce341388c4ced7ec6a5635deca1bd943a58ef53bd2bbd43 WatchSource:0}: Error finding container 59a66dff9a589a989ce341388c4ced7ec6a5635deca1bd943a58ef53bd2bbd43: Status 404 returned error can't find the container with id 59a66dff9a589a989ce341388c4ced7ec6a5635deca1bd943a58ef53bd2bbd43
Sep 17 23:22:34.488537 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:22:34.488504    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve/seaweedfs-64568bcd49-xx74p"]
Sep 17 23:22:34.491502 ip-10-0-132-188 kubenswrapper[2562]: W0917 23:22:34.491473    2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd1bccb49_bf6c_4f3b_aa32_0673a7ce5651.slice/crio-e11d3a31170e5e25790fdbe5c96a2ce8cd9bc91c09165a518397f615047baec0 WatchSource:0}: Error finding container e11d3a31170e5e25790fdbe5c96a2ce8cd9bc91c09165a518397f615047baec0: Status 404 returned error can't find the container with id e11d3a31170e5e25790fdbe5c96a2ce8cd9bc91c09165a518397f615047baec0
Sep 17 23:22:34.507510 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:22:34.507479    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fwwc5_7e452bbb-7dc6-4c9d-a9b1-76400942f5b4/konnectivity-agent/0.log" limit=1024
Sep 17 23:22:35.430273 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:22:35.430201    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve/kserve-controller-manager-5c5fcd7899-7tdw7" event={"ID":"ba27e9c4-76d1-474b-b78d-2e9c1531026b","Type":"ContainerStarted","Data":"c95607e015eb4257eb3807c1b406e04a002b29bb9085d08b91a1c91cab42a987"}
Sep 17 23:22:35.432196 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:22:35.432146    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve/seaweedfs-64568bcd49-xx74p" event={"ID":"d1bccb49-bf6c-4f3b-aa32-0673a7ce5651","Type":"ContainerStarted","Data":"e11d3a31170e5e25790fdbe5c96a2ce8cd9bc91c09165a518397f615047baec0"}
Sep 17 23:22:35.434992 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:22:35.434960    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve/llmisvc-controller-manager-5cbcfb6fbf-n7654" event={"ID":"aa8896c1-9f88-4d89-a8a0-fcebd0ac028e","Type":"ContainerStarted","Data":"59a66dff9a589a989ce341388c4ced7ec6a5635deca1bd943a58ef53bd2bbd43"}
Sep 17 23:22:39.458507 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:22:39.458470    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve/kserve-controller-manager-5c5fcd7899-7tdw7" event={"ID":"ba27e9c4-76d1-474b-b78d-2e9c1531026b","Type":"ContainerStarted","Data":"7773a594d377f8516f2b2444c1ec425e10f81acd167ddf587553a034fdb0a44a"}
Sep 17 23:22:39.459038 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:22:39.458638    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve/kserve-controller-manager-5c5fcd7899-7tdw7"
Sep 17 23:22:39.460271 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:22:39.460240    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve/seaweedfs-64568bcd49-xx74p" event={"ID":"d1bccb49-bf6c-4f3b-aa32-0673a7ce5651","Type":"ContainerStarted","Data":"accc2f755cf06f3c32c353296e8608d3e902ffc2b9d64707c677cd1fb851af53"}
Sep 17 23:22:39.460400 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:22:39.460322    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve/seaweedfs-64568bcd49-xx74p"
Sep 17 23:22:39.461926 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:22:39.461900    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve/llmisvc-controller-manager-5cbcfb6fbf-n7654" event={"ID":"aa8896c1-9f88-4d89-a8a0-fcebd0ac028e","Type":"ContainerStarted","Data":"c8bbb86f4c6e19ca35c36dcc2c85d37dc9bfe0eb3e0a37e7eeac4506aafdf58b"}
Sep 17 23:22:39.462060 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:22:39.462047    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve/llmisvc-controller-manager-5cbcfb6fbf-n7654"
Sep 17 23:22:39.475510 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:22:39.475462    2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kserve/kserve-controller-manager-5c5fcd7899-7tdw7" podStartSLOduration=1.993396573 podStartE2EDuration="6.475445932s" totalImagesPullingTime="4.482049359s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-17 23:22:33 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-17 23:22:39.473407517 +0000 UTC m=+711.875304285" watchObservedRunningTime="2026-09-17 23:22:39.475445932 +0000 UTC m=+711.877342733"
Sep 17 23:22:39.492992 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:22:39.492932    2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kserve/llmisvc-controller-manager-5cbcfb6fbf-n7654" podStartSLOduration=1.994190752 podStartE2EDuration="6.492913914s" totalImagesPullingTime="4.498723162s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-17 23:22:33 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-17 23:22:39.491104455 +0000 UTC m=+711.893001220" watchObservedRunningTime="2026-09-17 23:22:39.492913914 +0000 UTC m=+711.894810736"
Sep 17 23:22:39.506756 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:22:39.506687    2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kserve/seaweedfs-64568bcd49-xx74p" podStartSLOduration=1.942062542 podStartE2EDuration="6.506668683s" totalImagesPullingTime="4.564606141s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-17 23:22:33 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-17 23:22:39.505850949 +0000 UTC m=+711.907747718" watchObservedRunningTime="2026-09-17 23:22:39.506668683 +0000 UTC m=+711.908565491"
Sep 17 23:22:45.469887 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:22:45.469852    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kserve/seaweedfs-64568bcd49-xx74p"
Sep 17 23:23:02.516899 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:23:02.516857    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fwwc5_7e452bbb-7dc6-4c9d-a9b1-76400942f5b4/konnectivity-agent/0.log" limit=1024
Sep 17 23:23:10.469737 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:23:10.469703    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kserve/llmisvc-controller-manager-5cbcfb6fbf-n7654"
Sep 17 23:23:10.472704 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:23:10.472685    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kserve/kserve-controller-manager-5c5fcd7899-7tdw7"
Sep 17 23:23:17.460453 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:23:17.460409    2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kserve/kserve-controller-manager-5c5fcd7899-7tdw7"]
Sep 17 23:23:17.461344 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:23:17.461306    2562 kuberuntime_container.go:873] "Killing container with a grace period" pod="kserve/kserve-controller-manager-5c5fcd7899-7tdw7" podUID="ba27e9c4-76d1-474b-b78d-2e9c1531026b" containerName="manager" containerID="cri-o://7773a594d377f8516f2b2444c1ec425e10f81acd167ddf587553a034fdb0a44a" gracePeriod=10
Sep 17 23:23:17.482835 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:23:17.482803    2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kserve/kserve-controller-manager-5c5fcd7899-nv2hv"]
Sep 17 23:23:17.484947 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:23:17.484926    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve/kserve-controller-manager-5c5fcd7899-nv2hv"
Sep 17 23:23:17.492866 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:23:17.492836    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve/kserve-controller-manager-5c5fcd7899-nv2hv"]
Sep 17 23:23:17.625515 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:23:17.625480    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/180a5b21-9dea-4a9d-9961-ff92c5aaba82-cert\") pod \"kserve-controller-manager-5c5fcd7899-nv2hv\" (UID: \"180a5b21-9dea-4a9d-9961-ff92c5aaba82\") " pod="kserve/kserve-controller-manager-5c5fcd7899-nv2hv"
Sep 17 23:23:17.625686 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:23:17.625520    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-kpbx9\" (UniqueName: \"kubernetes.io/projected/180a5b21-9dea-4a9d-9961-ff92c5aaba82-kube-api-access-kpbx9\") pod \"kserve-controller-manager-5c5fcd7899-nv2hv\" (UID: \"180a5b21-9dea-4a9d-9961-ff92c5aaba82\") " pod="kserve/kserve-controller-manager-5c5fcd7899-nv2hv"
Sep 17 23:23:17.637556 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:23:17.637255    2562 generic.go:356] "Generic (PLEG): container finished" podID="ba27e9c4-76d1-474b-b78d-2e9c1531026b" containerID="7773a594d377f8516f2b2444c1ec425e10f81acd167ddf587553a034fdb0a44a" exitCode=0
Sep 17 23:23:17.637556 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:23:17.637400    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve/kserve-controller-manager-5c5fcd7899-7tdw7" event={"ID":"ba27e9c4-76d1-474b-b78d-2e9c1531026b","Type":"ContainerDied","Data":"7773a594d377f8516f2b2444c1ec425e10f81acd167ddf587553a034fdb0a44a"}
Sep 17 23:23:17.709504 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:23:17.709472    2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve/kserve-controller-manager-5c5fcd7899-7tdw7"
Sep 17 23:23:17.726517 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:23:17.726390    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/180a5b21-9dea-4a9d-9961-ff92c5aaba82-cert\") pod \"kserve-controller-manager-5c5fcd7899-nv2hv\" (UID: \"180a5b21-9dea-4a9d-9961-ff92c5aaba82\") " pod="kserve/kserve-controller-manager-5c5fcd7899-nv2hv"
Sep 17 23:23:17.726517 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:23:17.726440    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kpbx9\" (UniqueName: \"kubernetes.io/projected/180a5b21-9dea-4a9d-9961-ff92c5aaba82-kube-api-access-kpbx9\") pod \"kserve-controller-manager-5c5fcd7899-nv2hv\" (UID: \"180a5b21-9dea-4a9d-9961-ff92c5aaba82\") " pod="kserve/kserve-controller-manager-5c5fcd7899-nv2hv"
Sep 17 23:23:17.729185 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:23:17.729137    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/180a5b21-9dea-4a9d-9961-ff92c5aaba82-cert\") pod \"kserve-controller-manager-5c5fcd7899-nv2hv\" (UID: \"180a5b21-9dea-4a9d-9961-ff92c5aaba82\") " pod="kserve/kserve-controller-manager-5c5fcd7899-nv2hv"
Sep 17 23:23:17.738555 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:23:17.738474    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-kpbx9\" (UniqueName: \"kubernetes.io/projected/180a5b21-9dea-4a9d-9961-ff92c5aaba82-kube-api-access-kpbx9\") pod \"kserve-controller-manager-5c5fcd7899-nv2hv\" (UID: \"180a5b21-9dea-4a9d-9961-ff92c5aaba82\") " pod="kserve/kserve-controller-manager-5c5fcd7899-nv2hv"
Sep 17 23:23:17.827556 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:23:17.827514    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/ba27e9c4-76d1-474b-b78d-2e9c1531026b-cert\") pod \"ba27e9c4-76d1-474b-b78d-2e9c1531026b\" (UID: \"ba27e9c4-76d1-474b-b78d-2e9c1531026b\") "
Sep 17 23:23:17.827758 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:23:17.827626    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-zmpkx\" (UniqueName: \"kubernetes.io/projected/ba27e9c4-76d1-474b-b78d-2e9c1531026b-kube-api-access-zmpkx\") pod \"ba27e9c4-76d1-474b-b78d-2e9c1531026b\" (UID: \"ba27e9c4-76d1-474b-b78d-2e9c1531026b\") "
Sep 17 23:23:17.829985 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:23:17.829950    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/ba27e9c4-76d1-474b-b78d-2e9c1531026b-cert" (OuterVolumeSpecName: "cert") pod "ba27e9c4-76d1-474b-b78d-2e9c1531026b" (UID: "ba27e9c4-76d1-474b-b78d-2e9c1531026b"). InnerVolumeSpecName "cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 17 23:23:17.830109 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:23:17.829981    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/ba27e9c4-76d1-474b-b78d-2e9c1531026b-kube-api-access-zmpkx" (OuterVolumeSpecName: "kube-api-access-zmpkx") pod "ba27e9c4-76d1-474b-b78d-2e9c1531026b" (UID: "ba27e9c4-76d1-474b-b78d-2e9c1531026b"). InnerVolumeSpecName "kube-api-access-zmpkx". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 17 23:23:17.837206 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:23:17.837163    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve/kserve-controller-manager-5c5fcd7899-nv2hv"
Sep 17 23:23:17.928646 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:23:17.928601    2562 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-zmpkx\" (UniqueName: \"kubernetes.io/projected/ba27e9c4-76d1-474b-b78d-2e9c1531026b-kube-api-access-zmpkx\") on node \"ip-10-0-132-188.ec2.internal\" DevicePath \"\""
Sep 17 23:23:17.928646 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:23:17.928634    2562 reconciler_common.go:299] "Volume detached for volume \"cert\" (UniqueName: \"kubernetes.io/secret/ba27e9c4-76d1-474b-b78d-2e9c1531026b-cert\") on node \"ip-10-0-132-188.ec2.internal\" DevicePath \"\""
Sep 17 23:23:18.110291 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:23:18.110255    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve/kserve-controller-manager-5c5fcd7899-nv2hv"]
Sep 17 23:23:18.113765 ip-10-0-132-188 kubenswrapper[2562]: W0917 23:23:18.113730    2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod180a5b21_9dea_4a9d_9961_ff92c5aaba82.slice/crio-00edb6212faf23ff9e9e0aa872ccf7b9f2988fe9c89b1b829214696da3086da2 WatchSource:0}: Error finding container 00edb6212faf23ff9e9e0aa872ccf7b9f2988fe9c89b1b829214696da3086da2: Status 404 returned error can't find the container with id 00edb6212faf23ff9e9e0aa872ccf7b9f2988fe9c89b1b829214696da3086da2
Sep 17 23:23:18.115064 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:23:18.115046    2562 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Sep 17 23:23:18.643526 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:23:18.643489    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve/kserve-controller-manager-5c5fcd7899-nv2hv" event={"ID":"180a5b21-9dea-4a9d-9961-ff92c5aaba82","Type":"ContainerStarted","Data":"c0d82594900d68c21c82b51c42a514d1b52ea820fe4c4125b66c1103b24eb61e"}
Sep 17 23:23:18.644022 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:23:18.643551    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve/kserve-controller-manager-5c5fcd7899-nv2hv" event={"ID":"180a5b21-9dea-4a9d-9961-ff92c5aaba82","Type":"ContainerStarted","Data":"00edb6212faf23ff9e9e0aa872ccf7b9f2988fe9c89b1b829214696da3086da2"}
Sep 17 23:23:18.644022 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:23:18.643660    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve/kserve-controller-manager-5c5fcd7899-nv2hv"
Sep 17 23:23:18.644970 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:23:18.644940    2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve/kserve-controller-manager-5c5fcd7899-7tdw7"
Sep 17 23:23:18.644970 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:23:18.644951    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve/kserve-controller-manager-5c5fcd7899-7tdw7" event={"ID":"ba27e9c4-76d1-474b-b78d-2e9c1531026b","Type":"ContainerDied","Data":"c95607e015eb4257eb3807c1b406e04a002b29bb9085d08b91a1c91cab42a987"}
Sep 17 23:23:18.645143 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:23:18.645002    2562 scope.go:117] "RemoveContainer" containerID="7773a594d377f8516f2b2444c1ec425e10f81acd167ddf587553a034fdb0a44a"
Sep 17 23:23:18.660373 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:23:18.660321    2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kserve/kserve-controller-manager-5c5fcd7899-nv2hv" podStartSLOduration=1.378364735 podStartE2EDuration="1.660302272s" totalImagesPullingTime="281.937537ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-17 23:23:17 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-17 23:23:18.658921611 +0000 UTC m=+751.060818376" watchObservedRunningTime="2026-09-17 23:23:18.660302272 +0000 UTC m=+751.062199114"
Sep 17 23:23:18.675033 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:23:18.674996    2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kserve/kserve-controller-manager-5c5fcd7899-7tdw7"]
Sep 17 23:23:18.676753 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:23:18.676728    2562 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kserve/kserve-controller-manager-5c5fcd7899-7tdw7"]
Sep 17 23:23:19.666246 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:23:19.666215    2562 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="ba27e9c4-76d1-474b-b78d-2e9c1531026b" path="/var/lib/kubelet/pods/ba27e9c4-76d1-474b-b78d-2e9c1531026b/volumes"
Sep 17 23:23:49.655591 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:23:49.655558    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kserve/kserve-controller-manager-5c5fcd7899-nv2hv"
Sep 17 23:23:50.842141 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:23:50.842087    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fwwc5_7e452bbb-7dc6-4c9d-a9b1-76400942f5b4/konnectivity-agent/0.log" limit=1024
Sep 17 23:23:50.946042 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:23:50.946000    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fwwc5_7e452bbb-7dc6-4c9d-a9b1-76400942f5b4/konnectivity-agent/0.log" limit=1024
Sep 17 23:23:51.073829 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:23:51.073790    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fwwc5_7e452bbb-7dc6-4c9d-a9b1-76400942f5b4/konnectivity-agent/0.log" limit=1024
Sep 17 23:23:51.166156 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:23:51.166051    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fwwc5_7e452bbb-7dc6-4c9d-a9b1-76400942f5b4/konnectivity-agent/0.log" limit=1024
Sep 17 23:23:51.866664 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:23:51.866628    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fwwc5_7e452bbb-7dc6-4c9d-a9b1-76400942f5b4/konnectivity-agent/0.log" limit=1024
Sep 17 23:23:51.930292 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:23:51.930259    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fwwc5_7e452bbb-7dc6-4c9d-a9b1-76400942f5b4/konnectivity-agent/0.log" limit=1024
Sep 17 23:24:29.290546 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:24:29.290507    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fwwc5_7e452bbb-7dc6-4c9d-a9b1-76400942f5b4/konnectivity-agent/0.log" limit=1024
Sep 17 23:24:29.390820 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:24:29.390782    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fwwc5_7e452bbb-7dc6-4c9d-a9b1-76400942f5b4/konnectivity-agent/0.log" limit=1024
Sep 17 23:24:50.505792 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:24:50.505743    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fwwc5_7e452bbb-7dc6-4c9d-a9b1-76400942f5b4/konnectivity-agent/0.log" limit=1024
Sep 17 23:24:52.082241 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:24:52.082192    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fwwc5_7e452bbb-7dc6-4c9d-a9b1-76400942f5b4/konnectivity-agent/0.log" limit=1024
Sep 17 23:25:25.789152 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:25:25.789111    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fwwc5_7e452bbb-7dc6-4c9d-a9b1-76400942f5b4/konnectivity-agent/0.log" limit=1024
Sep 17 23:25:25.889492 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:25:25.889459    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fwwc5_7e452bbb-7dc6-4c9d-a9b1-76400942f5b4/konnectivity-agent/0.log" limit=1024
Sep 17 23:25:37.864803 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:25:37.864711    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fwwc5_7e452bbb-7dc6-4c9d-a9b1-76400942f5b4/konnectivity-agent/0.log" limit=1024
Sep 17 23:25:37.964516 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:25:37.964486    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fwwc5_7e452bbb-7dc6-4c9d-a9b1-76400942f5b4/konnectivity-agent/0.log" limit=1024
Sep 17 23:25:40.807422 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:25:40.807384    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fwwc5_7e452bbb-7dc6-4c9d-a9b1-76400942f5b4/konnectivity-agent/0.log" limit=1024
Sep 17 23:25:41.140520 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:25:41.140489    2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["e2e-test-llm-inference-service-2541cd65/scheduler-ha-replicas-test-kserve-router-scheduler-557bbc5d597s"]
Sep 17 23:25:41.140812 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:25:41.140801    2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ba27e9c4-76d1-474b-b78d-2e9c1531026b" containerName="manager"
Sep 17 23:25:41.140867 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:25:41.140813    2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="ba27e9c4-76d1-474b-b78d-2e9c1531026b" containerName="manager"
Sep 17 23:25:41.140902 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:25:41.140872    2562 memory_manager.go:368] "RemoveStaleState removing state" podUID="ba27e9c4-76d1-474b-b78d-2e9c1531026b" containerName="manager"
Sep 17 23:25:41.143120 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:25:41.143098    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-test-llm-inference-service-2541cd65/scheduler-ha-replicas-test-kserve-router-scheduler-557bbc5d597s"
Sep 17 23:25:41.145708 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:25:41.145685    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-test-llm-inference-service-2541cd65\"/\"kube-root-ca.crt\""
Sep 17 23:25:41.145850 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:25:41.145684    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-test-llm-inference-service-2541cd65\"/\"scheduler-ha-replicas-test-epp-sa-dockercfg-tz5d6\""
Sep 17 23:25:41.146384 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:25:41.146369    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-test-llm-inference-service-2541cd65\"/\"openshift-service-ca.crt\""
Sep 17 23:25:41.146626 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:25:41.146606    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-test-llm-inference-service-2541cd65\"/\"scheduler-ha-replicas-test-kserve-self-signed-certs\""
Sep 17 23:25:41.146728 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:25:41.146678    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-test-llm-inference-service-2541cd65\"/\"default-dockercfg-vsxfw\""
Sep 17 23:25:41.153946 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:25:41.153918    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["e2e-test-llm-inference-service-2541cd65/scheduler-ha-replicas-test-kserve-router-scheduler-557bbc5d597s"]
Sep 17 23:25:41.268300 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:25:41.268266    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-v6x5w\" (UniqueName: \"kubernetes.io/projected/f3a2fa5e-4b4e-45b4-9529-a0e271d719e6-kube-api-access-v6x5w\") pod \"scheduler-ha-replicas-test-kserve-router-scheduler-557bbc5d597s\" (UID: \"f3a2fa5e-4b4e-45b4-9529-a0e271d719e6\") " pod="e2e-test-llm-inference-service-2541cd65/scheduler-ha-replicas-test-kserve-router-scheduler-557bbc5d597s"
Sep 17 23:25:41.268469 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:25:41.268377    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/f3a2fa5e-4b4e-45b4-9529-a0e271d719e6-tls-certs\") pod \"scheduler-ha-replicas-test-kserve-router-scheduler-557bbc5d597s\" (UID: \"f3a2fa5e-4b4e-45b4-9529-a0e271d719e6\") " pod="e2e-test-llm-inference-service-2541cd65/scheduler-ha-replicas-test-kserve-router-scheduler-557bbc5d597s"
Sep 17 23:25:41.283504 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:25:41.283470    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fwwc5_7e452bbb-7dc6-4c9d-a9b1-76400942f5b4/konnectivity-agent/0.log" limit=1024
Sep 17 23:25:41.369734 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:25:41.369697    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-v6x5w\" (UniqueName: \"kubernetes.io/projected/f3a2fa5e-4b4e-45b4-9529-a0e271d719e6-kube-api-access-v6x5w\") pod \"scheduler-ha-replicas-test-kserve-router-scheduler-557bbc5d597s\" (UID: \"f3a2fa5e-4b4e-45b4-9529-a0e271d719e6\") " pod="e2e-test-llm-inference-service-2541cd65/scheduler-ha-replicas-test-kserve-router-scheduler-557bbc5d597s"
Sep 17 23:25:41.369897 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:25:41.369764    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/f3a2fa5e-4b4e-45b4-9529-a0e271d719e6-tls-certs\") pod \"scheduler-ha-replicas-test-kserve-router-scheduler-557bbc5d597s\" (UID: \"f3a2fa5e-4b4e-45b4-9529-a0e271d719e6\") " pod="e2e-test-llm-inference-service-2541cd65/scheduler-ha-replicas-test-kserve-router-scheduler-557bbc5d597s"
Sep 17 23:25:41.372164 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:25:41.372142    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/f3a2fa5e-4b4e-45b4-9529-a0e271d719e6-tls-certs\") pod \"scheduler-ha-replicas-test-kserve-router-scheduler-557bbc5d597s\" (UID: \"f3a2fa5e-4b4e-45b4-9529-a0e271d719e6\") " pod="e2e-test-llm-inference-service-2541cd65/scheduler-ha-replicas-test-kserve-router-scheduler-557bbc5d597s"
Sep 17 23:25:41.378402 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:25:41.378373    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-v6x5w\" (UniqueName: \"kubernetes.io/projected/f3a2fa5e-4b4e-45b4-9529-a0e271d719e6-kube-api-access-v6x5w\") pod \"scheduler-ha-replicas-test-kserve-router-scheduler-557bbc5d597s\" (UID: \"f3a2fa5e-4b4e-45b4-9529-a0e271d719e6\") " pod="e2e-test-llm-inference-service-2541cd65/scheduler-ha-replicas-test-kserve-router-scheduler-557bbc5d597s"
Sep 17 23:25:41.454228 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:25:41.454123    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-test-llm-inference-service-2541cd65/scheduler-ha-replicas-test-kserve-router-scheduler-557bbc5d597s"
Sep 17 23:25:41.726442 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:25:41.726415    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["e2e-test-llm-inference-service-2541cd65/scheduler-ha-replicas-test-kserve-router-scheduler-557bbc5d597s"]
Sep 17 23:25:41.729247 ip-10-0-132-188 kubenswrapper[2562]: W0917 23:25:41.729209    2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf3a2fa5e_4b4e_45b4_9529_a0e271d719e6.slice/crio-111dddff17a12666c3a68c4aa41e70b15b64b21c613d98c007cc2f5c417e172c WatchSource:0}: Error finding container 111dddff17a12666c3a68c4aa41e70b15b64b21c613d98c007cc2f5c417e172c: Status 404 returned error can't find the container with id 111dddff17a12666c3a68c4aa41e70b15b64b21c613d98c007cc2f5c417e172c
Sep 17 23:25:42.287326 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:25:42.287290    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-test-llm-inference-service-2541cd65/scheduler-ha-replicas-test-kserve-router-scheduler-557bbc5d597s" event={"ID":"f3a2fa5e-4b4e-45b4-9529-a0e271d719e6","Type":"ContainerStarted","Data":"111dddff17a12666c3a68c4aa41e70b15b64b21c613d98c007cc2f5c417e172c"}
Sep 17 23:25:45.307636 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:25:45.307601    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-test-llm-inference-service-2541cd65/scheduler-ha-replicas-test-kserve-router-scheduler-557bbc5d597s" event={"ID":"f3a2fa5e-4b4e-45b4-9529-a0e271d719e6","Type":"ContainerStarted","Data":"39ddbf7b154c4b5a48d444ebbfdee5c4d8bd61260a9d5aa4adbd5a6da560d4e1"}
Sep 17 23:25:45.308041 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:25:45.307733    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-test-llm-inference-service-2541cd65/scheduler-ha-replicas-test-kserve-router-scheduler-557bbc5d597s"
Sep 17 23:25:45.325232 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:25:45.325161    2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="e2e-test-llm-inference-service-2541cd65/scheduler-ha-replicas-test-kserve-router-scheduler-557bbc5d597s" podStartSLOduration=1.331013251 podStartE2EDuration="4.325143665s" totalImagesPullingTime="2.994130414s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-17 23:25:41 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-17 23:25:45.323550173 +0000 UTC m=+897.725446951" watchObservedRunningTime="2026-09-17 23:25:45.325143665 +0000 UTC m=+897.727040429"
Sep 17 23:25:51.339135 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:25:51.339098    2562 generic.go:356] "Generic (PLEG): container finished" podID="bafca983-9e23-438d-b13c-ff56f8b81b6b" containerID="7301d5024fc3878457bd3445e772e22c7e740b1d2d1aeef66db3c53552d0af76" exitCode=0
Sep 17 23:25:51.339637 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:25:51.339163    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/openshift-ai-inference-openshift-default-58c5cd85c9-9wjvx" event={"ID":"bafca983-9e23-438d-b13c-ff56f8b81b6b","Type":"ContainerDied","Data":"7301d5024fc3878457bd3445e772e22c7e740b1d2d1aeef66db3c53552d0af76"}
Sep 17 23:25:51.339637 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:25:51.339551    2562 scope.go:117] "RemoveContainer" containerID="7301d5024fc3878457bd3445e772e22c7e740b1d2d1aeef66db3c53552d0af76"
Sep 17 23:25:51.340653 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:25:51.340615    2562 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"}
Sep 17 23:25:51.340759 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:25:51.340690    2562 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"}
Sep 17 23:25:51.340759 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:25:51.340727    2562 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"}
Sep 17 23:25:52.349414 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:25:52.349359    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/openshift-ai-inference-openshift-default-58c5cd85c9-9wjvx" event={"ID":"bafca983-9e23-438d-b13c-ff56f8b81b6b","Type":"ContainerStarted","Data":"2565006514f9e5b57a6d3581cb9d9205b1b84459567edcfbd275826fdfa9d522"}
Sep 17 23:25:52.493027 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:25:52.492986    2562 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/openshift-ai-inference-openshift-default-58c5cd85c9-9wjvx"
Sep 17 23:25:52.494392 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:25:52.494360    2562 patch_prober.go:28] interesting pod/openshift-ai-inference-openshift-default-58c5cd85c9-9wjvx container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.20:15021/healthz/ready\": dial tcp 10.134.0.20:15021: connect: connection refused" start-of-body=
Sep 17 23:25:52.494523 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:25:52.494419    2562 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/openshift-ai-inference-openshift-default-58c5cd85c9-9wjvx" podUID="bafca983-9e23-438d-b13c-ff56f8b81b6b" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.20:15021/healthz/ready\": dial tcp 10.134.0.20:15021: connect: connection refused"
Sep 17 23:25:53.492879 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:25:53.492836    2562 patch_prober.go:28] interesting pod/openshift-ai-inference-openshift-default-58c5cd85c9-9wjvx container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.20:15021/healthz/ready\": dial tcp 10.134.0.20:15021: connect: connection refused" start-of-body=
Sep 17 23:25:53.493308 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:25:53.492903    2562 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/openshift-ai-inference-openshift-default-58c5cd85c9-9wjvx" podUID="bafca983-9e23-438d-b13c-ff56f8b81b6b" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.20:15021/healthz/ready\": dial tcp 10.134.0.20:15021: connect: connection refused"
Sep 17 23:25:53.755260 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:25:53.749873    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fwwc5_7e452bbb-7dc6-4c9d-a9b1-76400942f5b4/konnectivity-agent/0.log" limit=1024
Sep 17 23:25:53.772500 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:25:53.772454    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/openshift-ai-inference-openshift-default-58c5cd85c9-9wjvx"
Sep 17 23:25:53.906080 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:25:53.906044    2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["e2e-test-flow-control-smoke-4f8df83c/fc-smoke-test-kserve-router-scheduler-76d5658f88-zznwt"]
Sep 17 23:25:53.910577 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:25:53.910550    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-test-flow-control-smoke-4f8df83c/fc-smoke-test-kserve-router-scheduler-76d5658f88-zznwt"
Sep 17 23:25:53.914446 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:25:53.914419    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-test-flow-control-smoke-4f8df83c\"/\"kube-root-ca.crt\""
Sep 17 23:25:53.914446 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:25:53.914438    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-test-flow-control-smoke-4f8df83c\"/\"fc-smoke-test-epp-sa-dockercfg-dsqb9\""
Sep 17 23:25:53.914658 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:25:53.914474    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-test-flow-control-smoke-4f8df83c\"/\"openshift-service-ca.crt\""
Sep 17 23:25:53.914658 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:25:53.914494    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-test-flow-control-smoke-4f8df83c\"/\"fc-smoke-test-kserve-self-signed-certs\""
Sep 17 23:25:53.914658 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:25:53.914581    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-test-flow-control-smoke-4f8df83c\"/\"default-dockercfg-w4pws\""
Sep 17 23:25:53.918596 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:25:53.918571    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["e2e-test-flow-control-smoke-4f8df83c/fc-smoke-test-kserve-router-scheduler-76d5658f88-zznwt"]
Sep 17 23:25:53.925810 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:25:53.925777    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fwwc5_7e452bbb-7dc6-4c9d-a9b1-76400942f5b4/konnectivity-agent/0.log" limit=1024
Sep 17 23:25:53.987308 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:25:53.987266    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/24cb7800-f21d-4613-807a-3b031aa4219b-tls-certs\") pod \"fc-smoke-test-kserve-router-scheduler-76d5658f88-zznwt\" (UID: \"24cb7800-f21d-4613-807a-3b031aa4219b\") " pod="e2e-test-flow-control-smoke-4f8df83c/fc-smoke-test-kserve-router-scheduler-76d5658f88-zznwt"
Sep 17 23:25:53.987489 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:25:53.987318    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qgm49\" (UniqueName: \"kubernetes.io/projected/24cb7800-f21d-4613-807a-3b031aa4219b-kube-api-access-qgm49\") pod \"fc-smoke-test-kserve-router-scheduler-76d5658f88-zznwt\" (UID: \"24cb7800-f21d-4613-807a-3b031aa4219b\") " pod="e2e-test-flow-control-smoke-4f8df83c/fc-smoke-test-kserve-router-scheduler-76d5658f88-zznwt"
Sep 17 23:25:54.088360 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:25:54.088320    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qgm49\" (UniqueName: \"kubernetes.io/projected/24cb7800-f21d-4613-807a-3b031aa4219b-kube-api-access-qgm49\") pod \"fc-smoke-test-kserve-router-scheduler-76d5658f88-zznwt\" (UID: \"24cb7800-f21d-4613-807a-3b031aa4219b\") " pod="e2e-test-flow-control-smoke-4f8df83c/fc-smoke-test-kserve-router-scheduler-76d5658f88-zznwt"
Sep 17 23:25:54.088547 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:25:54.088432    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/24cb7800-f21d-4613-807a-3b031aa4219b-tls-certs\") pod \"fc-smoke-test-kserve-router-scheduler-76d5658f88-zznwt\" (UID: \"24cb7800-f21d-4613-807a-3b031aa4219b\") " pod="e2e-test-flow-control-smoke-4f8df83c/fc-smoke-test-kserve-router-scheduler-76d5658f88-zznwt"
Sep 17 23:25:54.090943 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:25:54.090918    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/24cb7800-f21d-4613-807a-3b031aa4219b-tls-certs\") pod \"fc-smoke-test-kserve-router-scheduler-76d5658f88-zznwt\" (UID: \"24cb7800-f21d-4613-807a-3b031aa4219b\") " pod="e2e-test-flow-control-smoke-4f8df83c/fc-smoke-test-kserve-router-scheduler-76d5658f88-zznwt"
Sep 17 23:25:54.095827 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:25:54.095800    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-qgm49\" (UniqueName: \"kubernetes.io/projected/24cb7800-f21d-4613-807a-3b031aa4219b-kube-api-access-qgm49\") pod \"fc-smoke-test-kserve-router-scheduler-76d5658f88-zznwt\" (UID: \"24cb7800-f21d-4613-807a-3b031aa4219b\") " pod="e2e-test-flow-control-smoke-4f8df83c/fc-smoke-test-kserve-router-scheduler-76d5658f88-zznwt"
Sep 17 23:25:54.221922 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:25:54.221881    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-test-flow-control-smoke-4f8df83c/fc-smoke-test-kserve-router-scheduler-76d5658f88-zznwt"
Sep 17 23:25:54.492358 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:25:54.492273    2562 patch_prober.go:28] interesting pod/openshift-ai-inference-openshift-default-58c5cd85c9-9wjvx container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.20:15021/healthz/ready\": dial tcp 10.134.0.20:15021: connect: connection refused" start-of-body=
Sep 17 23:25:54.492358 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:25:54.492328    2562 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/openshift-ai-inference-openshift-default-58c5cd85c9-9wjvx" podUID="bafca983-9e23-438d-b13c-ff56f8b81b6b" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.20:15021/healthz/ready\": dial tcp 10.134.0.20:15021: connect: connection refused"
Sep 17 23:25:54.519640 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:25:54.519608    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["e2e-test-flow-control-smoke-4f8df83c/fc-smoke-test-kserve-router-scheduler-76d5658f88-zznwt"]
Sep 17 23:25:54.522716 ip-10-0-132-188 kubenswrapper[2562]: W0917 23:25:54.522680    2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod24cb7800_f21d_4613_807a_3b031aa4219b.slice/crio-225b4f6e639b001a8a5cdc8894d322861087d868b542ea1308d24202a9aec123 WatchSource:0}: Error finding container 225b4f6e639b001a8a5cdc8894d322861087d868b542ea1308d24202a9aec123: Status 404 returned error can't find the container with id 225b4f6e639b001a8a5cdc8894d322861087d868b542ea1308d24202a9aec123
Sep 17 23:25:55.371899 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:25:55.371864    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-test-flow-control-smoke-4f8df83c/fc-smoke-test-kserve-router-scheduler-76d5658f88-zznwt" event={"ID":"24cb7800-f21d-4613-807a-3b031aa4219b","Type":"ContainerStarted","Data":"48099fc54a64e6741a6e5fda77cab19983076988c5e2e5e817d6af4c252f0858"}
Sep 17 23:25:55.372101 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:25:55.371917    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-test-flow-control-smoke-4f8df83c/fc-smoke-test-kserve-router-scheduler-76d5658f88-zznwt" event={"ID":"24cb7800-f21d-4613-807a-3b031aa4219b","Type":"ContainerStarted","Data":"225b4f6e639b001a8a5cdc8894d322861087d868b542ea1308d24202a9aec123"}
Sep 17 23:25:55.372101 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:25:55.371971    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-test-flow-control-smoke-4f8df83c/fc-smoke-test-kserve-router-scheduler-76d5658f88-zznwt"
Sep 17 23:25:55.388551 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:25:55.388484    2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="e2e-test-flow-control-smoke-4f8df83c/fc-smoke-test-kserve-router-scheduler-76d5658f88-zznwt" podStartSLOduration=2.388464497 podStartE2EDuration="2.388464497s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-17 23:25:53 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-17 23:25:55.387720738 +0000 UTC m=+907.789617502" watchObservedRunningTime="2026-09-17 23:25:55.388464497 +0000 UTC m=+907.790361261"
Sep 17 23:25:55.492490 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:25:55.492452    2562 patch_prober.go:28] interesting pod/openshift-ai-inference-openshift-default-58c5cd85c9-9wjvx container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.20:15021/healthz/ready\": dial tcp 10.134.0.20:15021: connect: connection refused" start-of-body=
Sep 17 23:25:55.492673 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:25:55.492510    2562 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/openshift-ai-inference-openshift-default-58c5cd85c9-9wjvx" podUID="bafca983-9e23-438d-b13c-ff56f8b81b6b" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.20:15021/healthz/ready\": dial tcp 10.134.0.20:15021: connect: connection refused"
Sep 17 23:25:56.492926 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:25:56.492882    2562 patch_prober.go:28] interesting pod/openshift-ai-inference-openshift-default-58c5cd85c9-9wjvx container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.20:15021/healthz/ready\": dial tcp 10.134.0.20:15021: connect: connection refused" start-of-body=
Sep 17 23:25:56.493394 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:25:56.492943    2562 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/openshift-ai-inference-openshift-default-58c5cd85c9-9wjvx" podUID="bafca983-9e23-438d-b13c-ff56f8b81b6b" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.20:15021/healthz/ready\": dial tcp 10.134.0.20:15021: connect: connection refused"
Sep 17 23:25:57.883218 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:25:57.883164    2562 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/openshift-ai-inference-openshift-default-58c5cd85c9-9wjvx"
Sep 17 23:25:57.884972 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:25:57.884877    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/openshift-ai-inference-openshift-default-58c5cd85c9-9wjvx"
Sep 17 23:25:59.664143 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:25:59.664110    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fwwc5_7e452bbb-7dc6-4c9d-a9b1-76400942f5b4/konnectivity-agent/0.log" limit=1024
Sep 17 23:26:03.199728 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:26:03.199690    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fwwc5_7e452bbb-7dc6-4c9d-a9b1-76400942f5b4/konnectivity-agent/0.log" limit=1024
Sep 17 23:26:12.557420 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:26:12.557385    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fwwc5_7e452bbb-7dc6-4c9d-a9b1-76400942f5b4/konnectivity-agent/0.log" limit=1024
Sep 17 23:26:12.606835 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:26:12.606798    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fwwc5_7e452bbb-7dc6-4c9d-a9b1-76400942f5b4/konnectivity-agent/0.log" limit=1024
Sep 17 23:26:14.575998 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:26:14.575962    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fwwc5_7e452bbb-7dc6-4c9d-a9b1-76400942f5b4/konnectivity-agent/0.log" limit=1024
Sep 17 23:26:16.314909 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:26:16.314868    2562 prober.go:120] "Probe failed" probeType="Readiness" pod="e2e-test-llm-inference-service-2541cd65/scheduler-ha-replicas-test-kserve-router-scheduler-557bbc5d597s" podUID="f3a2fa5e-4b4e-45b4-9529-a0e271d719e6" containerName="main" probeResult="failure" output="service unhealthy (responded with \"NOT_SERVING\")"
Sep 17 23:26:17.340322 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:26:17.340281    2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["e2e-test-llm-inference-service-2541cd65/scheduler-ha-replicas-test-kserve-router-scheduler-557bbc5d597s"]
Sep 17 23:26:17.343283 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:26:17.343206    2562 prober.go:120] "Probe failed" probeType="Readiness" pod="e2e-test-llm-inference-service-2541cd65/scheduler-ha-replicas-test-kserve-router-scheduler-557bbc5d597s" podUID="f3a2fa5e-4b4e-45b4-9529-a0e271d719e6" containerName="main" probeResult="failure" output="service unhealthy (responded with \"NOT_SERVING\")"
Sep 17 23:26:17.353137 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:26:17.353099    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fwwc5_7e452bbb-7dc6-4c9d-a9b1-76400942f5b4/konnectivity-agent/0.log" limit=1024
Sep 17 23:26:17.412664 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:26:17.412625    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fwwc5_7e452bbb-7dc6-4c9d-a9b1-76400942f5b4/konnectivity-agent/0.log" limit=1024
Sep 17 23:26:26.378346 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:26:26.378313    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-test-flow-control-smoke-4f8df83c/fc-smoke-test-kserve-router-scheduler-76d5658f88-zznwt"
Sep 17 23:26:27.343340 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:26:27.343307    2562 prober.go:120] "Probe failed" probeType="Readiness" pod="e2e-test-llm-inference-service-2541cd65/scheduler-ha-replicas-test-kserve-router-scheduler-557bbc5d597s" podUID="f3a2fa5e-4b4e-45b4-9529-a0e271d719e6" containerName="main" probeResult="failure" output="service unhealthy (responded with \"NOT_SERVING\")"
Sep 17 23:26:32.362555 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:26:32.362486    2562 kuberuntime_container.go:873] "Killing container with a grace period" pod="e2e-test-llm-inference-service-2541cd65/scheduler-ha-replicas-test-kserve-router-scheduler-557bbc5d597s" podUID="f3a2fa5e-4b4e-45b4-9529-a0e271d719e6" containerName="main" containerID="cri-o://39ddbf7b154c4b5a48d444ebbfdee5c4d8bd61260a9d5aa4adbd5a6da560d4e1" gracePeriod=45
Sep 17 23:26:33.171627 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:26:33.171591    2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["e2e-test-flow-control-smoke-4f8df83c/fc-smoke-test-kserve-router-scheduler-76d5658f88-zznwt"]
Sep 17 23:26:33.190358 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:26:33.190319    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fwwc5_7e452bbb-7dc6-4c9d-a9b1-76400942f5b4/konnectivity-agent/0.log" limit=1024
Sep 17 23:26:33.284479 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:26:33.284417    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fwwc5_7e452bbb-7dc6-4c9d-a9b1-76400942f5b4/konnectivity-agent/0.log" limit=1024
Sep 17 23:26:36.378608 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:26:36.378569    2562 prober.go:120] "Probe failed" probeType="Readiness" pod="e2e-test-flow-control-smoke-4f8df83c/fc-smoke-test-kserve-router-scheduler-76d5658f88-zznwt" podUID="24cb7800-f21d-4613-807a-3b031aa4219b" containerName="main" probeResult="failure" output="service unhealthy (responded with \"NOT_SERVING\")"
Sep 17 23:26:37.343803 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:26:37.343767    2562 prober.go:120] "Probe failed" probeType="Readiness" pod="e2e-test-llm-inference-service-2541cd65/scheduler-ha-replicas-test-kserve-router-scheduler-557bbc5d597s" podUID="f3a2fa5e-4b4e-45b4-9529-a0e271d719e6" containerName="main" probeResult="failure" output="service unhealthy (responded with \"NOT_SERVING\")"
Sep 17 23:26:46.377641 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:26:46.377600    2562 prober.go:120] "Probe failed" probeType="Readiness" pod="e2e-test-flow-control-smoke-4f8df83c/fc-smoke-test-kserve-router-scheduler-76d5658f88-zznwt" podUID="24cb7800-f21d-4613-807a-3b031aa4219b" containerName="main" probeResult="failure" output="service unhealthy (responded with \"NOT_SERVING\")"
Sep 17 23:26:47.343094 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:26:47.343052    2562 prober.go:120] "Probe failed" probeType="Readiness" pod="e2e-test-llm-inference-service-2541cd65/scheduler-ha-replicas-test-kserve-router-scheduler-557bbc5d597s" podUID="f3a2fa5e-4b4e-45b4-9529-a0e271d719e6" containerName="main" probeResult="failure" output="service unhealthy (responded with \"NOT_SERVING\")"
Sep 17 23:26:48.192688 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:26:48.192643    2562 kuberuntime_container.go:873] "Killing container with a grace period" pod="e2e-test-flow-control-smoke-4f8df83c/fc-smoke-test-kserve-router-scheduler-76d5658f88-zznwt" podUID="24cb7800-f21d-4613-807a-3b031aa4219b" containerName="main" containerID="cri-o://48099fc54a64e6741a6e5fda77cab19983076988c5e2e5e817d6af4c252f0858" gracePeriod=45
Sep 17 23:26:56.378299 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:26:56.378252    2562 prober.go:120] "Probe failed" probeType="Readiness" pod="e2e-test-flow-control-smoke-4f8df83c/fc-smoke-test-kserve-router-scheduler-76d5658f88-zznwt" podUID="24cb7800-f21d-4613-807a-3b031aa4219b" containerName="main" probeResult="failure" output="service unhealthy (responded with \"NOT_SERVING\")"
Sep 17 23:26:56.378678 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:26:56.378374    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-test-flow-control-smoke-4f8df83c/fc-smoke-test-kserve-router-scheduler-76d5658f88-zznwt"
Sep 17 23:26:57.343912 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:26:57.343868    2562 prober.go:120] "Probe failed" probeType="Readiness" pod="e2e-test-llm-inference-service-2541cd65/scheduler-ha-replicas-test-kserve-router-scheduler-557bbc5d597s" podUID="f3a2fa5e-4b4e-45b4-9529-a0e271d719e6" containerName="main" probeResult="failure" output="service unhealthy (responded with \"NOT_SERVING\")"
Sep 17 23:27:02.607498 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:27:02.607471    2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-test-llm-inference-service-2541cd65/scheduler-ha-replicas-test-kserve-router-scheduler-557bbc5d597s"
Sep 17 23:27:02.681806 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:27:02.681714    2562 generic.go:356] "Generic (PLEG): container finished" podID="f3a2fa5e-4b4e-45b4-9529-a0e271d719e6" containerID="39ddbf7b154c4b5a48d444ebbfdee5c4d8bd61260a9d5aa4adbd5a6da560d4e1" exitCode=0
Sep 17 23:27:02.681806 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:27:02.681765    2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-test-llm-inference-service-2541cd65/scheduler-ha-replicas-test-kserve-router-scheduler-557bbc5d597s"
Sep 17 23:27:02.681988 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:27:02.681801    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-test-llm-inference-service-2541cd65/scheduler-ha-replicas-test-kserve-router-scheduler-557bbc5d597s" event={"ID":"f3a2fa5e-4b4e-45b4-9529-a0e271d719e6","Type":"ContainerDied","Data":"39ddbf7b154c4b5a48d444ebbfdee5c4d8bd61260a9d5aa4adbd5a6da560d4e1"}
Sep 17 23:27:02.681988 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:27:02.681866    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-test-llm-inference-service-2541cd65/scheduler-ha-replicas-test-kserve-router-scheduler-557bbc5d597s" event={"ID":"f3a2fa5e-4b4e-45b4-9529-a0e271d719e6","Type":"ContainerDied","Data":"111dddff17a12666c3a68c4aa41e70b15b64b21c613d98c007cc2f5c417e172c"}
Sep 17 23:27:02.681988 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:27:02.681890    2562 scope.go:117] "RemoveContainer" containerID="39ddbf7b154c4b5a48d444ebbfdee5c4d8bd61260a9d5aa4adbd5a6da560d4e1"
Sep 17 23:27:02.690350 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:27:02.690329    2562 scope.go:117] "RemoveContainer" containerID="39ddbf7b154c4b5a48d444ebbfdee5c4d8bd61260a9d5aa4adbd5a6da560d4e1"
Sep 17 23:27:02.690628 ip-10-0-132-188 kubenswrapper[2562]: E0917 23:27:02.690608    2562 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"39ddbf7b154c4b5a48d444ebbfdee5c4d8bd61260a9d5aa4adbd5a6da560d4e1\": container with ID starting with 39ddbf7b154c4b5a48d444ebbfdee5c4d8bd61260a9d5aa4adbd5a6da560d4e1 not found: ID does not exist" containerID="39ddbf7b154c4b5a48d444ebbfdee5c4d8bd61260a9d5aa4adbd5a6da560d4e1"
Sep 17 23:27:02.690675 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:27:02.690638    2562 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"39ddbf7b154c4b5a48d444ebbfdee5c4d8bd61260a9d5aa4adbd5a6da560d4e1"} err="failed to get container status \"39ddbf7b154c4b5a48d444ebbfdee5c4d8bd61260a9d5aa4adbd5a6da560d4e1\": rpc error: code = NotFound desc = could not find container \"39ddbf7b154c4b5a48d444ebbfdee5c4d8bd61260a9d5aa4adbd5a6da560d4e1\": container with ID starting with 39ddbf7b154c4b5a48d444ebbfdee5c4d8bd61260a9d5aa4adbd5a6da560d4e1 not found: ID does not exist"
Sep 17 23:27:02.711011 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:27:02.710971    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/f3a2fa5e-4b4e-45b4-9529-a0e271d719e6-tls-certs\") pod \"f3a2fa5e-4b4e-45b4-9529-a0e271d719e6\" (UID: \"f3a2fa5e-4b4e-45b4-9529-a0e271d719e6\") "
Sep 17 23:27:02.711159 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:27:02.711099    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-v6x5w\" (UniqueName: \"kubernetes.io/projected/f3a2fa5e-4b4e-45b4-9529-a0e271d719e6-kube-api-access-v6x5w\") pod \"f3a2fa5e-4b4e-45b4-9529-a0e271d719e6\" (UID: \"f3a2fa5e-4b4e-45b4-9529-a0e271d719e6\") "
Sep 17 23:27:02.713376 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:27:02.713337    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/f3a2fa5e-4b4e-45b4-9529-a0e271d719e6-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "f3a2fa5e-4b4e-45b4-9529-a0e271d719e6" (UID: "f3a2fa5e-4b4e-45b4-9529-a0e271d719e6"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 17 23:27:02.713495 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:27:02.713401    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f3a2fa5e-4b4e-45b4-9529-a0e271d719e6-kube-api-access-v6x5w" (OuterVolumeSpecName: "kube-api-access-v6x5w") pod "f3a2fa5e-4b4e-45b4-9529-a0e271d719e6" (UID: "f3a2fa5e-4b4e-45b4-9529-a0e271d719e6"). InnerVolumeSpecName "kube-api-access-v6x5w". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 17 23:27:02.812690 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:27:02.812647    2562 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-v6x5w\" (UniqueName: \"kubernetes.io/projected/f3a2fa5e-4b4e-45b4-9529-a0e271d719e6-kube-api-access-v6x5w\") on node \"ip-10-0-132-188.ec2.internal\" DevicePath \"\""
Sep 17 23:27:02.812690 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:27:02.812684    2562 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/f3a2fa5e-4b4e-45b4-9529-a0e271d719e6-tls-certs\") on node \"ip-10-0-132-188.ec2.internal\" DevicePath \"\""
Sep 17 23:27:03.003566 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:27:03.003530    2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["e2e-test-llm-inference-service-2541cd65/scheduler-ha-replicas-test-kserve-router-scheduler-557bbc5d597s"]
Sep 17 23:27:03.007257 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:27:03.007229    2562 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["e2e-test-llm-inference-service-2541cd65/scheduler-ha-replicas-test-kserve-router-scheduler-557bbc5d597s"]
Sep 17 23:27:03.666418 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:27:03.666385    2562 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="f3a2fa5e-4b4e-45b4-9529-a0e271d719e6" path="/var/lib/kubelet/pods/f3a2fa5e-4b4e-45b4-9529-a0e271d719e6/volumes"
Sep 17 23:27:05.581799 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:27:05.581761    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fwwc5_7e452bbb-7dc6-4c9d-a9b1-76400942f5b4/konnectivity-agent/0.log" limit=1024
Sep 17 23:27:06.378236 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:27:06.378199    2562 prober.go:120] "Probe failed" probeType="Readiness" pod="e2e-test-flow-control-smoke-4f8df83c/fc-smoke-test-kserve-router-scheduler-76d5658f88-zznwt" podUID="24cb7800-f21d-4613-807a-3b031aa4219b" containerName="main" probeResult="failure" output="service unhealthy (responded with \"NOT_SERVING\")"
Sep 17 23:27:07.300915 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:27:07.300871    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fwwc5_7e452bbb-7dc6-4c9d-a9b1-76400942f5b4/konnectivity-agent/0.log" limit=1024
Sep 17 23:27:07.400068 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:27:07.400035    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fwwc5_7e452bbb-7dc6-4c9d-a9b1-76400942f5b4/konnectivity-agent/0.log" limit=1024
Sep 17 23:27:10.428137 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:27:10.428101    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fwwc5_7e452bbb-7dc6-4c9d-a9b1-76400942f5b4/konnectivity-agent/0.log" limit=1024
Sep 17 23:27:10.625284 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:27:10.625250    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fwwc5_7e452bbb-7dc6-4c9d-a9b1-76400942f5b4/konnectivity-agent/0.log" limit=1024
Sep 17 23:27:13.681623 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:27:13.681583    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fwwc5_7e452bbb-7dc6-4c9d-a9b1-76400942f5b4/konnectivity-agent/0.log" limit=1024
Sep 17 23:27:16.377801 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:27:16.377756    2562 prober.go:120] "Probe failed" probeType="Readiness" pod="e2e-test-flow-control-smoke-4f8df83c/fc-smoke-test-kserve-router-scheduler-76d5658f88-zznwt" podUID="24cb7800-f21d-4613-807a-3b031aa4219b" containerName="main" probeResult="failure" output="service unhealthy (responded with \"NOT_SERVING\")"
Sep 17 23:27:18.443324 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:27:18.443294    2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-test-flow-control-smoke-4f8df83c/fc-smoke-test-kserve-router-scheduler-76d5658f88-zznwt"
Sep 17 23:27:18.451582 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:27:18.451553    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-qgm49\" (UniqueName: \"kubernetes.io/projected/24cb7800-f21d-4613-807a-3b031aa4219b-kube-api-access-qgm49\") pod \"24cb7800-f21d-4613-807a-3b031aa4219b\" (UID: \"24cb7800-f21d-4613-807a-3b031aa4219b\") "
Sep 17 23:27:18.451703 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:27:18.451600    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/24cb7800-f21d-4613-807a-3b031aa4219b-tls-certs\") pod \"24cb7800-f21d-4613-807a-3b031aa4219b\" (UID: \"24cb7800-f21d-4613-807a-3b031aa4219b\") "
Sep 17 23:27:18.453873 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:27:18.453799    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/24cb7800-f21d-4613-807a-3b031aa4219b-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "24cb7800-f21d-4613-807a-3b031aa4219b" (UID: "24cb7800-f21d-4613-807a-3b031aa4219b"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 17 23:27:18.453873 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:27:18.453831    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/24cb7800-f21d-4613-807a-3b031aa4219b-kube-api-access-qgm49" (OuterVolumeSpecName: "kube-api-access-qgm49") pod "24cb7800-f21d-4613-807a-3b031aa4219b" (UID: "24cb7800-f21d-4613-807a-3b031aa4219b"). InnerVolumeSpecName "kube-api-access-qgm49". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 17 23:27:18.552280 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:27:18.552245    2562 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-qgm49\" (UniqueName: \"kubernetes.io/projected/24cb7800-f21d-4613-807a-3b031aa4219b-kube-api-access-qgm49\") on node \"ip-10-0-132-188.ec2.internal\" DevicePath \"\""
Sep 17 23:27:18.552280 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:27:18.552275    2562 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/24cb7800-f21d-4613-807a-3b031aa4219b-tls-certs\") on node \"ip-10-0-132-188.ec2.internal\" DevicePath \"\""
Sep 17 23:27:18.761065 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:27:18.760964    2562 generic.go:356] "Generic (PLEG): container finished" podID="24cb7800-f21d-4613-807a-3b031aa4219b" containerID="48099fc54a64e6741a6e5fda77cab19983076988c5e2e5e817d6af4c252f0858" exitCode=0
Sep 17 23:27:18.761065 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:27:18.761002    2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-test-flow-control-smoke-4f8df83c/fc-smoke-test-kserve-router-scheduler-76d5658f88-zznwt"
Sep 17 23:27:18.761065 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:27:18.761042    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-test-flow-control-smoke-4f8df83c/fc-smoke-test-kserve-router-scheduler-76d5658f88-zznwt" event={"ID":"24cb7800-f21d-4613-807a-3b031aa4219b","Type":"ContainerDied","Data":"48099fc54a64e6741a6e5fda77cab19983076988c5e2e5e817d6af4c252f0858"}
Sep 17 23:27:18.761352 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:27:18.761094    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-test-flow-control-smoke-4f8df83c/fc-smoke-test-kserve-router-scheduler-76d5658f88-zznwt" event={"ID":"24cb7800-f21d-4613-807a-3b031aa4219b","Type":"ContainerDied","Data":"225b4f6e639b001a8a5cdc8894d322861087d868b542ea1308d24202a9aec123"}
Sep 17 23:27:18.761352 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:27:18.761118    2562 scope.go:117] "RemoveContainer" containerID="48099fc54a64e6741a6e5fda77cab19983076988c5e2e5e817d6af4c252f0858"
Sep 17 23:27:18.770360 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:27:18.770304    2562 scope.go:117] "RemoveContainer" containerID="48099fc54a64e6741a6e5fda77cab19983076988c5e2e5e817d6af4c252f0858"
Sep 17 23:27:18.770598 ip-10-0-132-188 kubenswrapper[2562]: E0917 23:27:18.770578    2562 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"48099fc54a64e6741a6e5fda77cab19983076988c5e2e5e817d6af4c252f0858\": container with ID starting with 48099fc54a64e6741a6e5fda77cab19983076988c5e2e5e817d6af4c252f0858 not found: ID does not exist" containerID="48099fc54a64e6741a6e5fda77cab19983076988c5e2e5e817d6af4c252f0858"
Sep 17 23:27:18.770644 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:27:18.770604    2562 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"48099fc54a64e6741a6e5fda77cab19983076988c5e2e5e817d6af4c252f0858"} err="failed to get container status \"48099fc54a64e6741a6e5fda77cab19983076988c5e2e5e817d6af4c252f0858\": rpc error: code = NotFound desc = could not find container \"48099fc54a64e6741a6e5fda77cab19983076988c5e2e5e817d6af4c252f0858\": container with ID starting with 48099fc54a64e6741a6e5fda77cab19983076988c5e2e5e817d6af4c252f0858 not found: ID does not exist"
Sep 17 23:27:18.781011 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:27:18.780981    2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["e2e-test-flow-control-smoke-4f8df83c/fc-smoke-test-kserve-router-scheduler-76d5658f88-zznwt"]
Sep 17 23:27:18.784686 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:27:18.784660    2562 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["e2e-test-flow-control-smoke-4f8df83c/fc-smoke-test-kserve-router-scheduler-76d5658f88-zznwt"]
Sep 17 23:27:19.129705 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:27:19.129666    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fwwc5_7e452bbb-7dc6-4c9d-a9b1-76400942f5b4/konnectivity-agent/0.log" limit=1024
Sep 17 23:27:19.229186 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:27:19.229137    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fwwc5_7e452bbb-7dc6-4c9d-a9b1-76400942f5b4/konnectivity-agent/0.log" limit=1024
Sep 17 23:27:19.666764 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:27:19.666723    2562 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="24cb7800-f21d-4613-807a-3b031aa4219b" path="/var/lib/kubelet/pods/24cb7800-f21d-4613-807a-3b031aa4219b/volumes"
Sep 17 23:27:21.376985 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:27:21.376944    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fwwc5_7e452bbb-7dc6-4c9d-a9b1-76400942f5b4/konnectivity-agent/0.log" limit=1024
Sep 17 23:27:23.309669 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:27:23.309621    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fwwc5_7e452bbb-7dc6-4c9d-a9b1-76400942f5b4/konnectivity-agent/0.log" limit=1024
Sep 17 23:27:23.403983 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:27:23.403943    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fwwc5_7e452bbb-7dc6-4c9d-a9b1-76400942f5b4/konnectivity-agent/0.log" limit=1024
Sep 17 23:27:28.792483 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:27:28.792445    2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["e2e-test-flow-control-smoke-1e98f46d/fc-concurrency-test-kserve-b5d44467c-l9b27"]
Sep 17 23:27:28.792918 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:27:28.792861    2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f3a2fa5e-4b4e-45b4-9529-a0e271d719e6" containerName="main"
Sep 17 23:27:28.792918 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:27:28.792880    2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="f3a2fa5e-4b4e-45b4-9529-a0e271d719e6" containerName="main"
Sep 17 23:27:28.792918 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:27:28.792904    2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="24cb7800-f21d-4613-807a-3b031aa4219b" containerName="main"
Sep 17 23:27:28.792918 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:27:28.792913    2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="24cb7800-f21d-4613-807a-3b031aa4219b" containerName="main"
Sep 17 23:27:28.793064 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:27:28.792978    2562 memory_manager.go:368] "RemoveStaleState removing state" podUID="f3a2fa5e-4b4e-45b4-9529-a0e271d719e6" containerName="main"
Sep 17 23:27:28.793064 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:27:28.792987    2562 memory_manager.go:368] "RemoveStaleState removing state" podUID="24cb7800-f21d-4613-807a-3b031aa4219b" containerName="main"
Sep 17 23:27:28.797631 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:27:28.797607    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-test-flow-control-smoke-1e98f46d/fc-concurrency-test-kserve-b5d44467c-l9b27"
Sep 17 23:27:28.800951 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:27:28.800921    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-test-flow-control-smoke-1e98f46d\"/\"default-dockercfg-qnjt9\""
Sep 17 23:27:28.801102 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:27:28.801080    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-test-flow-control-smoke-1e98f46d\"/\"fc-concurrency-test-kserve-self-signed-certs\""
Sep 17 23:27:28.801159 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:27:28.801145    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-test-flow-control-smoke-1e98f46d\"/\"openshift-service-ca.crt\""
Sep 17 23:27:28.801231 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:27:28.801192    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-test-flow-control-smoke-1e98f46d\"/\"kube-root-ca.crt\""
Sep 17 23:27:28.806860 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:27:28.806819    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["e2e-test-flow-control-smoke-1e98f46d/fc-concurrency-test-kserve-b5d44467c-l9b27"]
Sep 17 23:27:28.822441 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:27:28.822396    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fwwc5_7e452bbb-7dc6-4c9d-a9b1-76400942f5b4/konnectivity-agent/0.log" limit=1024
Sep 17 23:27:28.841426 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:27:28.841390    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/efe002a2-2b44-4596-a3ff-f59f3aa16386-tmp-dir\") pod \"fc-concurrency-test-kserve-b5d44467c-l9b27\" (UID: \"efe002a2-2b44-4596-a3ff-f59f3aa16386\") " pod="e2e-test-flow-control-smoke-1e98f46d/fc-concurrency-test-kserve-b5d44467c-l9b27"
Sep 17 23:27:28.841426 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:27:28.841428    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/efe002a2-2b44-4596-a3ff-f59f3aa16386-tls-certs\") pod \"fc-concurrency-test-kserve-b5d44467c-l9b27\" (UID: \"efe002a2-2b44-4596-a3ff-f59f3aa16386\") " pod="e2e-test-flow-control-smoke-1e98f46d/fc-concurrency-test-kserve-b5d44467c-l9b27"
Sep 17 23:27:28.841650 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:27:28.841454    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/efe002a2-2b44-4596-a3ff-f59f3aa16386-model-cache\") pod \"fc-concurrency-test-kserve-b5d44467c-l9b27\" (UID: \"efe002a2-2b44-4596-a3ff-f59f3aa16386\") " pod="e2e-test-flow-control-smoke-1e98f46d/fc-concurrency-test-kserve-b5d44467c-l9b27"
Sep 17 23:27:28.841650 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:27:28.841548    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-d4d65\" (UniqueName: \"kubernetes.io/projected/efe002a2-2b44-4596-a3ff-f59f3aa16386-kube-api-access-d4d65\") pod \"fc-concurrency-test-kserve-b5d44467c-l9b27\" (UID: \"efe002a2-2b44-4596-a3ff-f59f3aa16386\") " pod="e2e-test-flow-control-smoke-1e98f46d/fc-concurrency-test-kserve-b5d44467c-l9b27"
Sep 17 23:27:28.841650 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:27:28.841599    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/efe002a2-2b44-4596-a3ff-f59f3aa16386-dshm\") pod \"fc-concurrency-test-kserve-b5d44467c-l9b27\" (UID: \"efe002a2-2b44-4596-a3ff-f59f3aa16386\") " pod="e2e-test-flow-control-smoke-1e98f46d/fc-concurrency-test-kserve-b5d44467c-l9b27"
Sep 17 23:27:28.841650 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:27:28.841622    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/efe002a2-2b44-4596-a3ff-f59f3aa16386-home\") pod \"fc-concurrency-test-kserve-b5d44467c-l9b27\" (UID: \"efe002a2-2b44-4596-a3ff-f59f3aa16386\") " pod="e2e-test-flow-control-smoke-1e98f46d/fc-concurrency-test-kserve-b5d44467c-l9b27"
Sep 17 23:27:28.942299 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:27:28.942253    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/efe002a2-2b44-4596-a3ff-f59f3aa16386-dshm\") pod \"fc-concurrency-test-kserve-b5d44467c-l9b27\" (UID: \"efe002a2-2b44-4596-a3ff-f59f3aa16386\") " pod="e2e-test-flow-control-smoke-1e98f46d/fc-concurrency-test-kserve-b5d44467c-l9b27"
Sep 17 23:27:28.942299 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:27:28.942304    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/efe002a2-2b44-4596-a3ff-f59f3aa16386-home\") pod \"fc-concurrency-test-kserve-b5d44467c-l9b27\" (UID: \"efe002a2-2b44-4596-a3ff-f59f3aa16386\") " pod="e2e-test-flow-control-smoke-1e98f46d/fc-concurrency-test-kserve-b5d44467c-l9b27"
Sep 17 23:27:28.942551 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:27:28.942339    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/efe002a2-2b44-4596-a3ff-f59f3aa16386-tmp-dir\") pod \"fc-concurrency-test-kserve-b5d44467c-l9b27\" (UID: \"efe002a2-2b44-4596-a3ff-f59f3aa16386\") " pod="e2e-test-flow-control-smoke-1e98f46d/fc-concurrency-test-kserve-b5d44467c-l9b27"
Sep 17 23:27:28.942551 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:27:28.942358    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/efe002a2-2b44-4596-a3ff-f59f3aa16386-tls-certs\") pod \"fc-concurrency-test-kserve-b5d44467c-l9b27\" (UID: \"efe002a2-2b44-4596-a3ff-f59f3aa16386\") " pod="e2e-test-flow-control-smoke-1e98f46d/fc-concurrency-test-kserve-b5d44467c-l9b27"
Sep 17 23:27:28.942551 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:27:28.942382    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/efe002a2-2b44-4596-a3ff-f59f3aa16386-model-cache\") pod \"fc-concurrency-test-kserve-b5d44467c-l9b27\" (UID: \"efe002a2-2b44-4596-a3ff-f59f3aa16386\") " pod="e2e-test-flow-control-smoke-1e98f46d/fc-concurrency-test-kserve-b5d44467c-l9b27"
Sep 17 23:27:28.942551 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:27:28.942438    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-d4d65\" (UniqueName: \"kubernetes.io/projected/efe002a2-2b44-4596-a3ff-f59f3aa16386-kube-api-access-d4d65\") pod \"fc-concurrency-test-kserve-b5d44467c-l9b27\" (UID: \"efe002a2-2b44-4596-a3ff-f59f3aa16386\") " pod="e2e-test-flow-control-smoke-1e98f46d/fc-concurrency-test-kserve-b5d44467c-l9b27"
Sep 17 23:27:28.942859 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:27:28.942826    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/efe002a2-2b44-4596-a3ff-f59f3aa16386-home\") pod \"fc-concurrency-test-kserve-b5d44467c-l9b27\" (UID: \"efe002a2-2b44-4596-a3ff-f59f3aa16386\") " pod="e2e-test-flow-control-smoke-1e98f46d/fc-concurrency-test-kserve-b5d44467c-l9b27"
Sep 17 23:27:28.942981 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:27:28.942862    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/efe002a2-2b44-4596-a3ff-f59f3aa16386-tmp-dir\") pod \"fc-concurrency-test-kserve-b5d44467c-l9b27\" (UID: \"efe002a2-2b44-4596-a3ff-f59f3aa16386\") " pod="e2e-test-flow-control-smoke-1e98f46d/fc-concurrency-test-kserve-b5d44467c-l9b27"
Sep 17 23:27:28.942981 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:27:28.942904    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/efe002a2-2b44-4596-a3ff-f59f3aa16386-model-cache\") pod \"fc-concurrency-test-kserve-b5d44467c-l9b27\" (UID: \"efe002a2-2b44-4596-a3ff-f59f3aa16386\") " pod="e2e-test-flow-control-smoke-1e98f46d/fc-concurrency-test-kserve-b5d44467c-l9b27"
Sep 17 23:27:28.944718 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:27:28.944696    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/efe002a2-2b44-4596-a3ff-f59f3aa16386-dshm\") pod \"fc-concurrency-test-kserve-b5d44467c-l9b27\" (UID: \"efe002a2-2b44-4596-a3ff-f59f3aa16386\") " pod="e2e-test-flow-control-smoke-1e98f46d/fc-concurrency-test-kserve-b5d44467c-l9b27"
Sep 17 23:27:28.944908 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:27:28.944887    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/efe002a2-2b44-4596-a3ff-f59f3aa16386-tls-certs\") pod \"fc-concurrency-test-kserve-b5d44467c-l9b27\" (UID: \"efe002a2-2b44-4596-a3ff-f59f3aa16386\") " pod="e2e-test-flow-control-smoke-1e98f46d/fc-concurrency-test-kserve-b5d44467c-l9b27"
Sep 17 23:27:28.950502 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:27:28.950474    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-d4d65\" (UniqueName: \"kubernetes.io/projected/efe002a2-2b44-4596-a3ff-f59f3aa16386-kube-api-access-d4d65\") pod \"fc-concurrency-test-kserve-b5d44467c-l9b27\" (UID: \"efe002a2-2b44-4596-a3ff-f59f3aa16386\") " pod="e2e-test-flow-control-smoke-1e98f46d/fc-concurrency-test-kserve-b5d44467c-l9b27"
Sep 17 23:27:29.081701 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:27:29.081666    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fwwc5_7e452bbb-7dc6-4c9d-a9b1-76400942f5b4/konnectivity-agent/0.log" limit=1024
Sep 17 23:27:29.110583 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:27:29.110541    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-test-flow-control-smoke-1e98f46d/fc-concurrency-test-kserve-b5d44467c-l9b27"
Sep 17 23:27:29.313330 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:27:29.313299    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fwwc5_7e452bbb-7dc6-4c9d-a9b1-76400942f5b4/konnectivity-agent/0.log" limit=1024
Sep 17 23:27:29.392505 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:27:29.392404    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["e2e-test-flow-control-smoke-1e98f46d/fc-concurrency-test-kserve-b5d44467c-l9b27"]
Sep 17 23:27:29.395441 ip-10-0-132-188 kubenswrapper[2562]: W0917 23:27:29.395403    2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podefe002a2_2b44_4596_a3ff_f59f3aa16386.slice/crio-313fc22a7c264b489bc56188bbdd7fd68d508771b184129d65472eb09d7aae23 WatchSource:0}: Error finding container 313fc22a7c264b489bc56188bbdd7fd68d508771b184129d65472eb09d7aae23: Status 404 returned error can't find the container with id 313fc22a7c264b489bc56188bbdd7fd68d508771b184129d65472eb09d7aae23
Sep 17 23:27:29.817830 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:27:29.817737    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-test-flow-control-smoke-1e98f46d/fc-concurrency-test-kserve-b5d44467c-l9b27" event={"ID":"efe002a2-2b44-4596-a3ff-f59f3aa16386","Type":"ContainerStarted","Data":"313fc22a7c264b489bc56188bbdd7fd68d508771b184129d65472eb09d7aae23"}
Sep 17 23:27:31.829036 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:27:31.828989    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-test-flow-control-smoke-1e98f46d/fc-concurrency-test-kserve-b5d44467c-l9b27" event={"ID":"efe002a2-2b44-4596-a3ff-f59f3aa16386","Type":"ContainerStarted","Data":"fc38084da9e7849e74aeee1a6179b168b16d1ccfbbb41087c2dce9c9570555d2"}
Sep 17 23:27:31.844509 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:27:31.844425    2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="e2e-test-flow-control-smoke-1e98f46d/fc-concurrency-test-kserve-b5d44467c-l9b27" podStartSLOduration=2.360821931 podStartE2EDuration="3.844403446s" totalImagesPullingTime="1.483581515s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-17 23:27:28 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-17 23:27:31.844003912 +0000 UTC m=+1004.245900672" watchObservedRunningTime="2026-09-17 23:27:31.844403446 +0000 UTC m=+1004.246300209"
Sep 17 23:27:32.111205 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:27:32.111092    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-test-flow-control-smoke-1e98f46d/fc-concurrency-test-kserve-b5d44467c-l9b27"
Sep 17 23:27:34.813791 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:27:34.813748    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fwwc5_7e452bbb-7dc6-4c9d-a9b1-76400942f5b4/konnectivity-agent/0.log" limit=1024
Sep 17 23:27:34.913921 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:27:34.913875    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fwwc5_7e452bbb-7dc6-4c9d-a9b1-76400942f5b4/konnectivity-agent/0.log" limit=1024
Sep 17 23:27:36.616010 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:27:36.615978    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fwwc5_7e452bbb-7dc6-4c9d-a9b1-76400942f5b4/konnectivity-agent/0.log" limit=1024
Sep 17 23:27:37.894009 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:27:37.893966    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fwwc5_7e452bbb-7dc6-4c9d-a9b1-76400942f5b4/konnectivity-agent/0.log" limit=1024
Sep 17 23:27:39.111378 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:27:39.111335    2562 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="e2e-test-flow-control-smoke-1e98f46d/fc-concurrency-test-kserve-b5d44467c-l9b27"
Sep 17 23:27:39.124008 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:27:39.123976    2562 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="e2e-test-flow-control-smoke-1e98f46d/fc-concurrency-test-kserve-b5d44467c-l9b27"
Sep 17 23:27:39.881596 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:27:39.881562    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-test-flow-control-smoke-1e98f46d/fc-concurrency-test-kserve-b5d44467c-l9b27"
Sep 17 23:27:49.416285 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:27:49.416241    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fwwc5_7e452bbb-7dc6-4c9d-a9b1-76400942f5b4/konnectivity-agent/0.log" limit=1024
Sep 17 23:27:49.517094 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:27:49.517059    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fwwc5_7e452bbb-7dc6-4c9d-a9b1-76400942f5b4/konnectivity-agent/0.log" limit=1024
Sep 17 23:28:08.323520 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:28:08.323482    2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["e2e-test-flow-control-smoke-1e98f46d/fc-concurrency-test-kserve-b5d44467c-l9b27"]
Sep 17 23:28:08.332987 ip-10-0-132-188 kubenswrapper[2562]: E0917 23:28:08.332948    2562 log.go:32] "ExecSync cmd from runtime service failed" err=<
Sep 17 23:28:08.332987 ip-10-0-132-188 kubenswrapper[2562]:         rpc error: code = Unknown desc = command error: executable file `/bin/sleep` not found: No such file or directory
Sep 17 23:28:08.332987 ip-10-0-132-188 kubenswrapper[2562]:         , stdout: , stderr: , exit code -1
Sep 17 23:28:08.332987 ip-10-0-132-188 kubenswrapper[2562]:  > containerID="fc38084da9e7849e74aeee1a6179b168b16d1ccfbbb41087c2dce9c9570555d2" cmd=["/bin/sleep","15"]
Sep 17 23:28:08.333292 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:28:08.333009    2562 handlers.go:82] "Exec lifecycle hook for Container in Pod failed" execCommand=["/bin/sleep","15"] containerName="main" pod="e2e-test-flow-control-smoke-1e98f46d/fc-concurrency-test-kserve-b5d44467c-l9b27" message="" err=<
Sep 17 23:28:08.333292 ip-10-0-132-188 kubenswrapper[2562]:         rpc error: code = Unknown desc = command error: executable file `/bin/sleep` not found: No such file or directory
Sep 17 23:28:08.333292 ip-10-0-132-188 kubenswrapper[2562]:         , stdout: , stderr: , exit code -1
Sep 17 23:28:08.333292 ip-10-0-132-188 kubenswrapper[2562]:  >
Sep 17 23:28:08.333292 ip-10-0-132-188 kubenswrapper[2562]: E0917 23:28:08.333024    2562 kuberuntime_container.go:755] "PreStop hook failed" err=<
Sep 17 23:28:08.333292 ip-10-0-132-188 kubenswrapper[2562]:         rpc error: code = Unknown desc = command error: executable file `/bin/sleep` not found: No such file or directory
Sep 17 23:28:08.333292 ip-10-0-132-188 kubenswrapper[2562]:         , stdout: , stderr: , exit code -1
Sep 17 23:28:08.333292 ip-10-0-132-188 kubenswrapper[2562]:  > pod="e2e-test-flow-control-smoke-1e98f46d/fc-concurrency-test-kserve-b5d44467c-l9b27" podUID="efe002a2-2b44-4596-a3ff-f59f3aa16386" containerName="main" containerID="cri-o://fc38084da9e7849e74aeee1a6179b168b16d1ccfbbb41087c2dce9c9570555d2"
Sep 17 23:28:08.333292 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:28:08.333063    2562 kuberuntime_container.go:873] "Killing container with a grace period" pod="e2e-test-flow-control-smoke-1e98f46d/fc-concurrency-test-kserve-b5d44467c-l9b27" podUID="efe002a2-2b44-4596-a3ff-f59f3aa16386" containerName="main" containerID="cri-o://fc38084da9e7849e74aeee1a6179b168b16d1ccfbbb41087c2dce9c9570555d2" gracePeriod=60
Sep 17 23:28:08.341894 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:28:08.341858    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fwwc5_7e452bbb-7dc6-4c9d-a9b1-76400942f5b4/konnectivity-agent/0.log" limit=1024
Sep 17 23:28:08.441477 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:28:08.441443    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fwwc5_7e452bbb-7dc6-4c9d-a9b1-76400942f5b4/konnectivity-agent/0.log" limit=1024
Sep 17 23:28:08.870866 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:28:08.870825    2562 prober.go:120] "Probe failed" probeType="Readiness" pod="e2e-test-flow-control-smoke-1e98f46d/fc-concurrency-test-kserve-b5d44467c-l9b27" podUID="efe002a2-2b44-4596-a3ff-f59f3aa16386" containerName="main" probeResult="failure" output="Get \"https://10.134.0.32:8000/health\": dial tcp 10.134.0.32:8000: connect: connection refused"
Sep 17 23:28:09.011602 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:28:09.011567    2562 generic.go:356] "Generic (PLEG): container finished" podID="efe002a2-2b44-4596-a3ff-f59f3aa16386" containerID="fc38084da9e7849e74aeee1a6179b168b16d1ccfbbb41087c2dce9c9570555d2" exitCode=0
Sep 17 23:28:09.011769 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:28:09.011635    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-test-flow-control-smoke-1e98f46d/fc-concurrency-test-kserve-b5d44467c-l9b27" event={"ID":"efe002a2-2b44-4596-a3ff-f59f3aa16386","Type":"ContainerDied","Data":"fc38084da9e7849e74aeee1a6179b168b16d1ccfbbb41087c2dce9c9570555d2"}
Sep 17 23:28:09.085387 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:28:09.085359    2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-test-flow-control-smoke-1e98f46d/fc-concurrency-test-kserve-b5d44467c-l9b27"
Sep 17 23:28:09.098606 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:28:09.098578    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/efe002a2-2b44-4596-a3ff-f59f3aa16386-tmp-dir\") pod \"efe002a2-2b44-4596-a3ff-f59f3aa16386\" (UID: \"efe002a2-2b44-4596-a3ff-f59f3aa16386\") "
Sep 17 23:28:09.098786 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:28:09.098637    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/efe002a2-2b44-4596-a3ff-f59f3aa16386-dshm\") pod \"efe002a2-2b44-4596-a3ff-f59f3aa16386\" (UID: \"efe002a2-2b44-4596-a3ff-f59f3aa16386\") "
Sep 17 23:28:09.098786 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:28:09.098685    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/efe002a2-2b44-4596-a3ff-f59f3aa16386-home\") pod \"efe002a2-2b44-4596-a3ff-f59f3aa16386\" (UID: \"efe002a2-2b44-4596-a3ff-f59f3aa16386\") "
Sep 17 23:28:09.098786 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:28:09.098711    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/efe002a2-2b44-4596-a3ff-f59f3aa16386-model-cache\") pod \"efe002a2-2b44-4596-a3ff-f59f3aa16386\" (UID: \"efe002a2-2b44-4596-a3ff-f59f3aa16386\") "
Sep 17 23:28:09.098786 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:28:09.098744    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-d4d65\" (UniqueName: \"kubernetes.io/projected/efe002a2-2b44-4596-a3ff-f59f3aa16386-kube-api-access-d4d65\") pod \"efe002a2-2b44-4596-a3ff-f59f3aa16386\" (UID: \"efe002a2-2b44-4596-a3ff-f59f3aa16386\") "
Sep 17 23:28:09.098786 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:28:09.098770    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/efe002a2-2b44-4596-a3ff-f59f3aa16386-tls-certs\") pod \"efe002a2-2b44-4596-a3ff-f59f3aa16386\" (UID: \"efe002a2-2b44-4596-a3ff-f59f3aa16386\") "
Sep 17 23:28:09.099042 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:28:09.098901    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/efe002a2-2b44-4596-a3ff-f59f3aa16386-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "efe002a2-2b44-4596-a3ff-f59f3aa16386" (UID: "efe002a2-2b44-4596-a3ff-f59f3aa16386"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 17 23:28:09.099042 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:28:09.098930    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/efe002a2-2b44-4596-a3ff-f59f3aa16386-home" (OuterVolumeSpecName: "home") pod "efe002a2-2b44-4596-a3ff-f59f3aa16386" (UID: "efe002a2-2b44-4596-a3ff-f59f3aa16386"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 17 23:28:09.099042 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:28:09.098989    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/efe002a2-2b44-4596-a3ff-f59f3aa16386-model-cache" (OuterVolumeSpecName: "model-cache") pod "efe002a2-2b44-4596-a3ff-f59f3aa16386" (UID: "efe002a2-2b44-4596-a3ff-f59f3aa16386"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 17 23:28:09.099042 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:28:09.099013    2562 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/efe002a2-2b44-4596-a3ff-f59f3aa16386-tmp-dir\") on node \"ip-10-0-132-188.ec2.internal\" DevicePath \"\""
Sep 17 23:28:09.101387 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:28:09.101355    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/efe002a2-2b44-4596-a3ff-f59f3aa16386-dshm" (OuterVolumeSpecName: "dshm") pod "efe002a2-2b44-4596-a3ff-f59f3aa16386" (UID: "efe002a2-2b44-4596-a3ff-f59f3aa16386"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 17 23:28:09.101646 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:28:09.101619    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/efe002a2-2b44-4596-a3ff-f59f3aa16386-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "efe002a2-2b44-4596-a3ff-f59f3aa16386" (UID: "efe002a2-2b44-4596-a3ff-f59f3aa16386"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 17 23:28:09.101981 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:28:09.101950    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/efe002a2-2b44-4596-a3ff-f59f3aa16386-kube-api-access-d4d65" (OuterVolumeSpecName: "kube-api-access-d4d65") pod "efe002a2-2b44-4596-a3ff-f59f3aa16386" (UID: "efe002a2-2b44-4596-a3ff-f59f3aa16386"). InnerVolumeSpecName "kube-api-access-d4d65". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 17 23:28:09.200055 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:28:09.199953    2562 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-d4d65\" (UniqueName: \"kubernetes.io/projected/efe002a2-2b44-4596-a3ff-f59f3aa16386-kube-api-access-d4d65\") on node \"ip-10-0-132-188.ec2.internal\" DevicePath \"\""
Sep 17 23:28:09.200055 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:28:09.199988    2562 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/efe002a2-2b44-4596-a3ff-f59f3aa16386-tls-certs\") on node \"ip-10-0-132-188.ec2.internal\" DevicePath \"\""
Sep 17 23:28:09.200055 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:28:09.199998    2562 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/efe002a2-2b44-4596-a3ff-f59f3aa16386-dshm\") on node \"ip-10-0-132-188.ec2.internal\" DevicePath \"\""
Sep 17 23:28:09.200055 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:28:09.200006    2562 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/efe002a2-2b44-4596-a3ff-f59f3aa16386-home\") on node \"ip-10-0-132-188.ec2.internal\" DevicePath \"\""
Sep 17 23:28:09.200055 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:28:09.200014    2562 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/efe002a2-2b44-4596-a3ff-f59f3aa16386-model-cache\") on node \"ip-10-0-132-188.ec2.internal\" DevicePath \"\""
Sep 17 23:28:10.017362 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:28:10.017274    2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-test-flow-control-smoke-1e98f46d/fc-concurrency-test-kserve-b5d44467c-l9b27"
Sep 17 23:28:10.017362 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:28:10.017308    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-test-flow-control-smoke-1e98f46d/fc-concurrency-test-kserve-b5d44467c-l9b27" event={"ID":"efe002a2-2b44-4596-a3ff-f59f3aa16386","Type":"ContainerDied","Data":"313fc22a7c264b489bc56188bbdd7fd68d508771b184129d65472eb09d7aae23"}
Sep 17 23:28:10.017884 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:28:10.017368    2562 scope.go:117] "RemoveContainer" containerID="fc38084da9e7849e74aeee1a6179b168b16d1ccfbbb41087c2dce9c9570555d2"
Sep 17 23:28:10.036289 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:28:10.034657    2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["e2e-test-flow-control-smoke-1e98f46d/fc-concurrency-test-kserve-b5d44467c-l9b27"]
Sep 17 23:28:10.039665 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:28:10.039636    2562 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["e2e-test-flow-control-smoke-1e98f46d/fc-concurrency-test-kserve-b5d44467c-l9b27"]
Sep 17 23:28:11.667182 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:28:11.667146    2562 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="efe002a2-2b44-4596-a3ff-f59f3aa16386" path="/var/lib/kubelet/pods/efe002a2-2b44-4596-a3ff-f59f3aa16386/volumes"
Sep 17 23:28:39.125704 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:28:39.125613    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fwwc5_7e452bbb-7dc6-4c9d-a9b1-76400942f5b4/konnectivity-agent/0.log" limit=1024
Sep 17 23:28:39.218513 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:28:39.218477    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fwwc5_7e452bbb-7dc6-4c9d-a9b1-76400942f5b4/konnectivity-agent/0.log" limit=1024
Sep 17 23:28:43.050540 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:28:43.050505    2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["e2e-test-llm-inference-service-9c386aa9/scheduler-v06-pd-migration-test-kserve-prefill-8665687897-rpvrt"]
Sep 17 23:28:43.050954 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:28:43.050843    2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="efe002a2-2b44-4596-a3ff-f59f3aa16386" containerName="main"
Sep 17 23:28:43.050954 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:28:43.050855    2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="efe002a2-2b44-4596-a3ff-f59f3aa16386" containerName="main"
Sep 17 23:28:43.050954 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:28:43.050913    2562 memory_manager.go:368] "RemoveStaleState removing state" podUID="efe002a2-2b44-4596-a3ff-f59f3aa16386" containerName="main"
Sep 17 23:28:43.054145 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:28:43.054119    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-test-llm-inference-service-9c386aa9/scheduler-v06-pd-migration-test-kserve-prefill-8665687897-rpvrt"
Sep 17 23:28:43.057516 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:28:43.057491    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-test-llm-inference-service-9c386aa9\"/\"kube-root-ca.crt\""
Sep 17 23:28:43.057655 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:28:43.057492    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-test-llm-inference-service-9c386aa9\"/\"default-dockercfg-jgn68\""
Sep 17 23:28:43.057655 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:28:43.057492    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-test-llm-inference-service-9c386aa9\"/\"scheduler-v06-pd-migration-test-kserve-self-signed-certs\""
Sep 17 23:28:43.057655 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:28:43.057495    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-test-llm-inference-service-9c386aa9\"/\"openshift-service-ca.crt\""
Sep 17 23:28:43.061889 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:28:43.061787    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["e2e-test-llm-inference-service-9c386aa9/scheduler-v06-pd-migration-test-kserve-prefill-8665687897-rpvrt"]
Sep 17 23:28:43.077643 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:28:43.077604    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fwwc5_7e452bbb-7dc6-4c9d-a9b1-76400942f5b4/konnectivity-agent/0.log" limit=1024
Sep 17 23:28:43.199312 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:28:43.199270    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/9dd4e65c-5e0a-4175-8f86-1bd8d2a0e984-home\") pod \"scheduler-v06-pd-migration-test-kserve-prefill-8665687897-rpvrt\" (UID: \"9dd4e65c-5e0a-4175-8f86-1bd8d2a0e984\") " pod="e2e-test-llm-inference-service-9c386aa9/scheduler-v06-pd-migration-test-kserve-prefill-8665687897-rpvrt"
Sep 17 23:28:43.199518 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:28:43.199317    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vc5mh\" (UniqueName: \"kubernetes.io/projected/9dd4e65c-5e0a-4175-8f86-1bd8d2a0e984-kube-api-access-vc5mh\") pod \"scheduler-v06-pd-migration-test-kserve-prefill-8665687897-rpvrt\" (UID: \"9dd4e65c-5e0a-4175-8f86-1bd8d2a0e984\") " pod="e2e-test-llm-inference-service-9c386aa9/scheduler-v06-pd-migration-test-kserve-prefill-8665687897-rpvrt"
Sep 17 23:28:43.199518 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:28:43.199417    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/9dd4e65c-5e0a-4175-8f86-1bd8d2a0e984-dshm\") pod \"scheduler-v06-pd-migration-test-kserve-prefill-8665687897-rpvrt\" (UID: \"9dd4e65c-5e0a-4175-8f86-1bd8d2a0e984\") " pod="e2e-test-llm-inference-service-9c386aa9/scheduler-v06-pd-migration-test-kserve-prefill-8665687897-rpvrt"
Sep 17 23:28:43.199518 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:28:43.199461    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/9dd4e65c-5e0a-4175-8f86-1bd8d2a0e984-model-cache\") pod \"scheduler-v06-pd-migration-test-kserve-prefill-8665687897-rpvrt\" (UID: \"9dd4e65c-5e0a-4175-8f86-1bd8d2a0e984\") " pod="e2e-test-llm-inference-service-9c386aa9/scheduler-v06-pd-migration-test-kserve-prefill-8665687897-rpvrt"
Sep 17 23:28:43.199667 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:28:43.199522    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/9dd4e65c-5e0a-4175-8f86-1bd8d2a0e984-tmp-dir\") pod \"scheduler-v06-pd-migration-test-kserve-prefill-8665687897-rpvrt\" (UID: \"9dd4e65c-5e0a-4175-8f86-1bd8d2a0e984\") " pod="e2e-test-llm-inference-service-9c386aa9/scheduler-v06-pd-migration-test-kserve-prefill-8665687897-rpvrt"
Sep 17 23:28:43.199667 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:28:43.199557    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/9dd4e65c-5e0a-4175-8f86-1bd8d2a0e984-tls-certs\") pod \"scheduler-v06-pd-migration-test-kserve-prefill-8665687897-rpvrt\" (UID: \"9dd4e65c-5e0a-4175-8f86-1bd8d2a0e984\") " pod="e2e-test-llm-inference-service-9c386aa9/scheduler-v06-pd-migration-test-kserve-prefill-8665687897-rpvrt"
Sep 17 23:28:43.300791 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:28:43.300690    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/9dd4e65c-5e0a-4175-8f86-1bd8d2a0e984-model-cache\") pod \"scheduler-v06-pd-migration-test-kserve-prefill-8665687897-rpvrt\" (UID: \"9dd4e65c-5e0a-4175-8f86-1bd8d2a0e984\") " pod="e2e-test-llm-inference-service-9c386aa9/scheduler-v06-pd-migration-test-kserve-prefill-8665687897-rpvrt"
Sep 17 23:28:43.300791 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:28:43.300749    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/9dd4e65c-5e0a-4175-8f86-1bd8d2a0e984-tmp-dir\") pod \"scheduler-v06-pd-migration-test-kserve-prefill-8665687897-rpvrt\" (UID: \"9dd4e65c-5e0a-4175-8f86-1bd8d2a0e984\") " pod="e2e-test-llm-inference-service-9c386aa9/scheduler-v06-pd-migration-test-kserve-prefill-8665687897-rpvrt"
Sep 17 23:28:43.300791 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:28:43.300777    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/9dd4e65c-5e0a-4175-8f86-1bd8d2a0e984-tls-certs\") pod \"scheduler-v06-pd-migration-test-kserve-prefill-8665687897-rpvrt\" (UID: \"9dd4e65c-5e0a-4175-8f86-1bd8d2a0e984\") " pod="e2e-test-llm-inference-service-9c386aa9/scheduler-v06-pd-migration-test-kserve-prefill-8665687897-rpvrt"
Sep 17 23:28:43.301079 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:28:43.300806    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/9dd4e65c-5e0a-4175-8f86-1bd8d2a0e984-home\") pod \"scheduler-v06-pd-migration-test-kserve-prefill-8665687897-rpvrt\" (UID: \"9dd4e65c-5e0a-4175-8f86-1bd8d2a0e984\") " pod="e2e-test-llm-inference-service-9c386aa9/scheduler-v06-pd-migration-test-kserve-prefill-8665687897-rpvrt"
Sep 17 23:28:43.301079 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:28:43.300839    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vc5mh\" (UniqueName: \"kubernetes.io/projected/9dd4e65c-5e0a-4175-8f86-1bd8d2a0e984-kube-api-access-vc5mh\") pod \"scheduler-v06-pd-migration-test-kserve-prefill-8665687897-rpvrt\" (UID: \"9dd4e65c-5e0a-4175-8f86-1bd8d2a0e984\") " pod="e2e-test-llm-inference-service-9c386aa9/scheduler-v06-pd-migration-test-kserve-prefill-8665687897-rpvrt"
Sep 17 23:28:43.301079 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:28:43.300903    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/9dd4e65c-5e0a-4175-8f86-1bd8d2a0e984-dshm\") pod \"scheduler-v06-pd-migration-test-kserve-prefill-8665687897-rpvrt\" (UID: \"9dd4e65c-5e0a-4175-8f86-1bd8d2a0e984\") " pod="e2e-test-llm-inference-service-9c386aa9/scheduler-v06-pd-migration-test-kserve-prefill-8665687897-rpvrt"
Sep 17 23:28:43.301255 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:28:43.301148    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/9dd4e65c-5e0a-4175-8f86-1bd8d2a0e984-model-cache\") pod \"scheduler-v06-pd-migration-test-kserve-prefill-8665687897-rpvrt\" (UID: \"9dd4e65c-5e0a-4175-8f86-1bd8d2a0e984\") " pod="e2e-test-llm-inference-service-9c386aa9/scheduler-v06-pd-migration-test-kserve-prefill-8665687897-rpvrt"
Sep 17 23:28:43.301311 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:28:43.301276    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/9dd4e65c-5e0a-4175-8f86-1bd8d2a0e984-tmp-dir\") pod \"scheduler-v06-pd-migration-test-kserve-prefill-8665687897-rpvrt\" (UID: \"9dd4e65c-5e0a-4175-8f86-1bd8d2a0e984\") " pod="e2e-test-llm-inference-service-9c386aa9/scheduler-v06-pd-migration-test-kserve-prefill-8665687897-rpvrt"
Sep 17 23:28:43.301450 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:28:43.301430    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/9dd4e65c-5e0a-4175-8f86-1bd8d2a0e984-home\") pod \"scheduler-v06-pd-migration-test-kserve-prefill-8665687897-rpvrt\" (UID: \"9dd4e65c-5e0a-4175-8f86-1bd8d2a0e984\") " pod="e2e-test-llm-inference-service-9c386aa9/scheduler-v06-pd-migration-test-kserve-prefill-8665687897-rpvrt"
Sep 17 23:28:43.303517 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:28:43.303495    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/9dd4e65c-5e0a-4175-8f86-1bd8d2a0e984-dshm\") pod \"scheduler-v06-pd-migration-test-kserve-prefill-8665687897-rpvrt\" (UID: \"9dd4e65c-5e0a-4175-8f86-1bd8d2a0e984\") " pod="e2e-test-llm-inference-service-9c386aa9/scheduler-v06-pd-migration-test-kserve-prefill-8665687897-rpvrt"
Sep 17 23:28:43.303782 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:28:43.303761    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/9dd4e65c-5e0a-4175-8f86-1bd8d2a0e984-tls-certs\") pod \"scheduler-v06-pd-migration-test-kserve-prefill-8665687897-rpvrt\" (UID: \"9dd4e65c-5e0a-4175-8f86-1bd8d2a0e984\") " pod="e2e-test-llm-inference-service-9c386aa9/scheduler-v06-pd-migration-test-kserve-prefill-8665687897-rpvrt"
Sep 17 23:28:43.312809 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:28:43.312781    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-vc5mh\" (UniqueName: \"kubernetes.io/projected/9dd4e65c-5e0a-4175-8f86-1bd8d2a0e984-kube-api-access-vc5mh\") pod \"scheduler-v06-pd-migration-test-kserve-prefill-8665687897-rpvrt\" (UID: \"9dd4e65c-5e0a-4175-8f86-1bd8d2a0e984\") " pod="e2e-test-llm-inference-service-9c386aa9/scheduler-v06-pd-migration-test-kserve-prefill-8665687897-rpvrt"
Sep 17 23:28:43.347729 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:28:43.347675    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fwwc5_7e452bbb-7dc6-4c9d-a9b1-76400942f5b4/konnectivity-agent/0.log" limit=1024
Sep 17 23:28:43.365542 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:28:43.365506    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-test-llm-inference-service-9c386aa9/scheduler-v06-pd-migration-test-kserve-prefill-8665687897-rpvrt"
Sep 17 23:28:43.639934 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:28:43.639897    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["e2e-test-llm-inference-service-9c386aa9/scheduler-v06-pd-migration-test-kserve-prefill-8665687897-rpvrt"]
Sep 17 23:28:43.643777 ip-10-0-132-188 kubenswrapper[2562]: W0917 23:28:43.643735    2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9dd4e65c_5e0a_4175_8f86_1bd8d2a0e984.slice/crio-5a4cb3e93f49a0f0d41385fe89c8e7b5d43e576ebe62d9a09924a8c871ec24fa WatchSource:0}: Error finding container 5a4cb3e93f49a0f0d41385fe89c8e7b5d43e576ebe62d9a09924a8c871ec24fa: Status 404 returned error can't find the container with id 5a4cb3e93f49a0f0d41385fe89c8e7b5d43e576ebe62d9a09924a8c871ec24fa
Sep 17 23:28:44.178551 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:28:44.178508    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-test-llm-inference-service-9c386aa9/scheduler-v06-pd-migration-test-kserve-prefill-8665687897-rpvrt" event={"ID":"9dd4e65c-5e0a-4175-8f86-1bd8d2a0e984","Type":"ContainerStarted","Data":"05023b8ffa449da3c06d75d0802fb9d95aedd51d6841308ea06e94cd7db4dbfc"}
Sep 17 23:28:44.179042 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:28:44.178564    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-test-llm-inference-service-9c386aa9/scheduler-v06-pd-migration-test-kserve-prefill-8665687897-rpvrt" event={"ID":"9dd4e65c-5e0a-4175-8f86-1bd8d2a0e984","Type":"ContainerStarted","Data":"5a4cb3e93f49a0f0d41385fe89c8e7b5d43e576ebe62d9a09924a8c871ec24fa"}
Sep 17 23:28:44.193629 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:28:44.193565    2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="e2e-test-llm-inference-service-9c386aa9/scheduler-v06-pd-migration-test-kserve-prefill-8665687897-rpvrt" podStartSLOduration=1.193544271 podStartE2EDuration="1.193544271s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-17 23:28:43 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-17 23:28:44.193375562 +0000 UTC m=+1076.595272327" watchObservedRunningTime="2026-09-17 23:28:44.193544271 +0000 UTC m=+1076.595441036"
Sep 17 23:28:44.366319 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:28:44.366280    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-test-llm-inference-service-9c386aa9/scheduler-v06-pd-migration-test-kserve-prefill-8665687897-rpvrt"
Sep 17 23:28:45.623887 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:28:45.623851    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fwwc5_7e452bbb-7dc6-4c9d-a9b1-76400942f5b4/konnectivity-agent/0.log" limit=1024
Sep 17 23:28:49.434661 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:28:49.434622    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fwwc5_7e452bbb-7dc6-4c9d-a9b1-76400942f5b4/konnectivity-agent/0.log" limit=1024
Sep 17 23:28:51.108070 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:28:51.108029    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fwwc5_7e452bbb-7dc6-4c9d-a9b1-76400942f5b4/konnectivity-agent/0.log" limit=1024
Sep 17 23:28:51.170259 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:28:51.170224    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fwwc5_7e452bbb-7dc6-4c9d-a9b1-76400942f5b4/konnectivity-agent/0.log" limit=1024
Sep 17 23:28:53.365916 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:28:53.365870    2562 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="e2e-test-llm-inference-service-9c386aa9/scheduler-v06-pd-migration-test-kserve-prefill-8665687897-rpvrt"
Sep 17 23:28:53.378446 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:28:53.378414    2562 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="e2e-test-llm-inference-service-9c386aa9/scheduler-v06-pd-migration-test-kserve-prefill-8665687897-rpvrt"
Sep 17 23:28:54.235455 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:28:54.235422    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-test-llm-inference-service-9c386aa9/scheduler-v06-pd-migration-test-kserve-prefill-8665687897-rpvrt"
Sep 17 23:28:57.974585 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:28:57.974542    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fwwc5_7e452bbb-7dc6-4c9d-a9b1-76400942f5b4/konnectivity-agent/0.log" limit=1024
Sep 17 23:28:58.076541 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:28:58.076498    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fwwc5_7e452bbb-7dc6-4c9d-a9b1-76400942f5b4/konnectivity-agent/0.log" limit=1024
Sep 17 23:28:58.290415 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:28:58.290316    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fwwc5_7e452bbb-7dc6-4c9d-a9b1-76400942f5b4/konnectivity-agent/0.log" limit=1024
Sep 17 23:28:58.390524 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:28:58.390491    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fwwc5_7e452bbb-7dc6-4c9d-a9b1-76400942f5b4/konnectivity-agent/0.log" limit=1024
Sep 17 23:28:58.494085 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:28:58.494044    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fwwc5_7e452bbb-7dc6-4c9d-a9b1-76400942f5b4/konnectivity-agent/0.log" limit=1024
Sep 17 23:28:58.502914 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:28:58.502880    2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["e2e-test-gateway-section-name-propagation-49b5edd9/router-gateway-1-openshift-default-6b46477c5b-l8vp4"]
Sep 17 23:28:58.508226 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:28:58.508197    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-test-gateway-section-name-propagation-49b5edd9/router-gateway-1-openshift-default-6b46477c5b-l8vp4"
Sep 17 23:28:58.510560 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:28:58.510529    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-test-gateway-section-name-propagation-49b5edd9\"/\"openshift-gw-ca-root-cert\""
Sep 17 23:28:58.511019 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:28:58.510997    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-test-gateway-section-name-propagation-49b5edd9\"/\"openshift-service-ca.crt\""
Sep 17 23:28:58.511518 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:28:58.511287    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-test-gateway-section-name-propagation-49b5edd9\"/\"kube-root-ca.crt\""
Sep 17 23:28:58.511518 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:28:58.511373    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-test-gateway-section-name-propagation-49b5edd9\"/\"router-gateway-1-openshift-default-dockercfg-px9zq\""
Sep 17 23:28:58.519229 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:28:58.519201    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["e2e-test-gateway-section-name-propagation-49b5edd9/router-gateway-1-openshift-default-6b46477c5b-l8vp4"]
Sep 17 23:28:58.562247 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:28:58.562111    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fwwc5_7e452bbb-7dc6-4c9d-a9b1-76400942f5b4/konnectivity-agent/0.log" limit=1024
Sep 17 23:28:58.646226 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:28:58.646164    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/a0fd1118-d360-4f54-b3e9-f62977cea0aa-credential-socket\") pod \"router-gateway-1-openshift-default-6b46477c5b-l8vp4\" (UID: \"a0fd1118-d360-4f54-b3e9-f62977cea0aa\") " pod="e2e-test-gateway-section-name-propagation-49b5edd9/router-gateway-1-openshift-default-6b46477c5b-l8vp4"
Sep 17 23:28:58.646435 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:28:58.646236    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/a0fd1118-d360-4f54-b3e9-f62977cea0aa-istio-data\") pod \"router-gateway-1-openshift-default-6b46477c5b-l8vp4\" (UID: \"a0fd1118-d360-4f54-b3e9-f62977cea0aa\") " pod="e2e-test-gateway-section-name-propagation-49b5edd9/router-gateway-1-openshift-default-6b46477c5b-l8vp4"
Sep 17 23:28:58.646435 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:28:58.646273    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/a0fd1118-d360-4f54-b3e9-f62977cea0aa-istio-podinfo\") pod \"router-gateway-1-openshift-default-6b46477c5b-l8vp4\" (UID: \"a0fd1118-d360-4f54-b3e9-f62977cea0aa\") " pod="e2e-test-gateway-section-name-propagation-49b5edd9/router-gateway-1-openshift-default-6b46477c5b-l8vp4"
Sep 17 23:28:58.646435 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:28:58.646302    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/a0fd1118-d360-4f54-b3e9-f62977cea0aa-workload-certs\") pod \"router-gateway-1-openshift-default-6b46477c5b-l8vp4\" (UID: \"a0fd1118-d360-4f54-b3e9-f62977cea0aa\") " pod="e2e-test-gateway-section-name-propagation-49b5edd9/router-gateway-1-openshift-default-6b46477c5b-l8vp4"
Sep 17 23:28:58.646435 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:28:58.646351    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/a0fd1118-d360-4f54-b3e9-f62977cea0aa-istio-envoy\") pod \"router-gateway-1-openshift-default-6b46477c5b-l8vp4\" (UID: \"a0fd1118-d360-4f54-b3e9-f62977cea0aa\") " pod="e2e-test-gateway-section-name-propagation-49b5edd9/router-gateway-1-openshift-default-6b46477c5b-l8vp4"
Sep 17 23:28:58.646435 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:28:58.646375    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9n7rt\" (UniqueName: \"kubernetes.io/projected/a0fd1118-d360-4f54-b3e9-f62977cea0aa-kube-api-access-9n7rt\") pod \"router-gateway-1-openshift-default-6b46477c5b-l8vp4\" (UID: \"a0fd1118-d360-4f54-b3e9-f62977cea0aa\") " pod="e2e-test-gateway-section-name-propagation-49b5edd9/router-gateway-1-openshift-default-6b46477c5b-l8vp4"
Sep 17 23:28:58.646435 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:28:58.646410    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/a0fd1118-d360-4f54-b3e9-f62977cea0aa-workload-socket\") pod \"router-gateway-1-openshift-default-6b46477c5b-l8vp4\" (UID: \"a0fd1118-d360-4f54-b3e9-f62977cea0aa\") " pod="e2e-test-gateway-section-name-propagation-49b5edd9/router-gateway-1-openshift-default-6b46477c5b-l8vp4"
Sep 17 23:28:58.646673 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:28:58.646466    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/a0fd1118-d360-4f54-b3e9-f62977cea0aa-istio-token\") pod \"router-gateway-1-openshift-default-6b46477c5b-l8vp4\" (UID: \"a0fd1118-d360-4f54-b3e9-f62977cea0aa\") " pod="e2e-test-gateway-section-name-propagation-49b5edd9/router-gateway-1-openshift-default-6b46477c5b-l8vp4"
Sep 17 23:28:58.646673 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:28:58.646515    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/a0fd1118-d360-4f54-b3e9-f62977cea0aa-istiod-ca-cert\") pod \"router-gateway-1-openshift-default-6b46477c5b-l8vp4\" (UID: \"a0fd1118-d360-4f54-b3e9-f62977cea0aa\") " pod="e2e-test-gateway-section-name-propagation-49b5edd9/router-gateway-1-openshift-default-6b46477c5b-l8vp4"
Sep 17 23:28:58.747340 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:28:58.747297    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/a0fd1118-d360-4f54-b3e9-f62977cea0aa-istio-token\") pod \"router-gateway-1-openshift-default-6b46477c5b-l8vp4\" (UID: \"a0fd1118-d360-4f54-b3e9-f62977cea0aa\") " pod="e2e-test-gateway-section-name-propagation-49b5edd9/router-gateway-1-openshift-default-6b46477c5b-l8vp4"
Sep 17 23:28:58.747551 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:28:58.747352    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/a0fd1118-d360-4f54-b3e9-f62977cea0aa-istiod-ca-cert\") pod \"router-gateway-1-openshift-default-6b46477c5b-l8vp4\" (UID: \"a0fd1118-d360-4f54-b3e9-f62977cea0aa\") " pod="e2e-test-gateway-section-name-propagation-49b5edd9/router-gateway-1-openshift-default-6b46477c5b-l8vp4"
Sep 17 23:28:58.747551 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:28:58.747400    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/a0fd1118-d360-4f54-b3e9-f62977cea0aa-credential-socket\") pod \"router-gateway-1-openshift-default-6b46477c5b-l8vp4\" (UID: \"a0fd1118-d360-4f54-b3e9-f62977cea0aa\") " pod="e2e-test-gateway-section-name-propagation-49b5edd9/router-gateway-1-openshift-default-6b46477c5b-l8vp4"
Sep 17 23:28:58.747551 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:28:58.747428    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/a0fd1118-d360-4f54-b3e9-f62977cea0aa-istio-data\") pod \"router-gateway-1-openshift-default-6b46477c5b-l8vp4\" (UID: \"a0fd1118-d360-4f54-b3e9-f62977cea0aa\") " pod="e2e-test-gateway-section-name-propagation-49b5edd9/router-gateway-1-openshift-default-6b46477c5b-l8vp4"
Sep 17 23:28:58.747551 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:28:58.747459    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/a0fd1118-d360-4f54-b3e9-f62977cea0aa-istio-podinfo\") pod \"router-gateway-1-openshift-default-6b46477c5b-l8vp4\" (UID: \"a0fd1118-d360-4f54-b3e9-f62977cea0aa\") " pod="e2e-test-gateway-section-name-propagation-49b5edd9/router-gateway-1-openshift-default-6b46477c5b-l8vp4"
Sep 17 23:28:58.747551 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:28:58.747489    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/a0fd1118-d360-4f54-b3e9-f62977cea0aa-workload-certs\") pod \"router-gateway-1-openshift-default-6b46477c5b-l8vp4\" (UID: \"a0fd1118-d360-4f54-b3e9-f62977cea0aa\") " pod="e2e-test-gateway-section-name-propagation-49b5edd9/router-gateway-1-openshift-default-6b46477c5b-l8vp4"
Sep 17 23:28:58.747551 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:28:58.747544    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/a0fd1118-d360-4f54-b3e9-f62977cea0aa-istio-envoy\") pod \"router-gateway-1-openshift-default-6b46477c5b-l8vp4\" (UID: \"a0fd1118-d360-4f54-b3e9-f62977cea0aa\") " pod="e2e-test-gateway-section-name-propagation-49b5edd9/router-gateway-1-openshift-default-6b46477c5b-l8vp4"
Sep 17 23:28:58.747859 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:28:58.747567    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9n7rt\" (UniqueName: \"kubernetes.io/projected/a0fd1118-d360-4f54-b3e9-f62977cea0aa-kube-api-access-9n7rt\") pod \"router-gateway-1-openshift-default-6b46477c5b-l8vp4\" (UID: \"a0fd1118-d360-4f54-b3e9-f62977cea0aa\") " pod="e2e-test-gateway-section-name-propagation-49b5edd9/router-gateway-1-openshift-default-6b46477c5b-l8vp4"
Sep 17 23:28:58.747859 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:28:58.747607    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/a0fd1118-d360-4f54-b3e9-f62977cea0aa-workload-socket\") pod \"router-gateway-1-openshift-default-6b46477c5b-l8vp4\" (UID: \"a0fd1118-d360-4f54-b3e9-f62977cea0aa\") " pod="e2e-test-gateway-section-name-propagation-49b5edd9/router-gateway-1-openshift-default-6b46477c5b-l8vp4"
Sep 17 23:28:58.748001 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:28:58.747970    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/a0fd1118-d360-4f54-b3e9-f62977cea0aa-credential-socket\") pod \"router-gateway-1-openshift-default-6b46477c5b-l8vp4\" (UID: \"a0fd1118-d360-4f54-b3e9-f62977cea0aa\") " pod="e2e-test-gateway-section-name-propagation-49b5edd9/router-gateway-1-openshift-default-6b46477c5b-l8vp4"
Sep 17 23:28:58.748054 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:28:58.747985    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/a0fd1118-d360-4f54-b3e9-f62977cea0aa-istio-data\") pod \"router-gateway-1-openshift-default-6b46477c5b-l8vp4\" (UID: \"a0fd1118-d360-4f54-b3e9-f62977cea0aa\") " pod="e2e-test-gateway-section-name-propagation-49b5edd9/router-gateway-1-openshift-default-6b46477c5b-l8vp4"
Sep 17 23:28:58.748100 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:28:58.748033    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/a0fd1118-d360-4f54-b3e9-f62977cea0aa-workload-certs\") pod \"router-gateway-1-openshift-default-6b46477c5b-l8vp4\" (UID: \"a0fd1118-d360-4f54-b3e9-f62977cea0aa\") " pod="e2e-test-gateway-section-name-propagation-49b5edd9/router-gateway-1-openshift-default-6b46477c5b-l8vp4"
Sep 17 23:28:58.748217 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:28:58.748156    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/a0fd1118-d360-4f54-b3e9-f62977cea0aa-workload-socket\") pod \"router-gateway-1-openshift-default-6b46477c5b-l8vp4\" (UID: \"a0fd1118-d360-4f54-b3e9-f62977cea0aa\") " pod="e2e-test-gateway-section-name-propagation-49b5edd9/router-gateway-1-openshift-default-6b46477c5b-l8vp4"
Sep 17 23:28:58.748217 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:28:58.748205    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/a0fd1118-d360-4f54-b3e9-f62977cea0aa-istiod-ca-cert\") pod \"router-gateway-1-openshift-default-6b46477c5b-l8vp4\" (UID: \"a0fd1118-d360-4f54-b3e9-f62977cea0aa\") " pod="e2e-test-gateway-section-name-propagation-49b5edd9/router-gateway-1-openshift-default-6b46477c5b-l8vp4"
Sep 17 23:28:58.749804 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:28:58.749784    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/a0fd1118-d360-4f54-b3e9-f62977cea0aa-istio-envoy\") pod \"router-gateway-1-openshift-default-6b46477c5b-l8vp4\" (UID: \"a0fd1118-d360-4f54-b3e9-f62977cea0aa\") " pod="e2e-test-gateway-section-name-propagation-49b5edd9/router-gateway-1-openshift-default-6b46477c5b-l8vp4"
Sep 17 23:28:58.750049 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:28:58.750034    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/a0fd1118-d360-4f54-b3e9-f62977cea0aa-istio-podinfo\") pod \"router-gateway-1-openshift-default-6b46477c5b-l8vp4\" (UID: \"a0fd1118-d360-4f54-b3e9-f62977cea0aa\") " pod="e2e-test-gateway-section-name-propagation-49b5edd9/router-gateway-1-openshift-default-6b46477c5b-l8vp4"
Sep 17 23:28:58.760005 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:28:58.759978    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/a0fd1118-d360-4f54-b3e9-f62977cea0aa-istio-token\") pod \"router-gateway-1-openshift-default-6b46477c5b-l8vp4\" (UID: \"a0fd1118-d360-4f54-b3e9-f62977cea0aa\") " pod="e2e-test-gateway-section-name-propagation-49b5edd9/router-gateway-1-openshift-default-6b46477c5b-l8vp4"
Sep 17 23:28:58.760522 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:28:58.760499    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-9n7rt\" (UniqueName: \"kubernetes.io/projected/a0fd1118-d360-4f54-b3e9-f62977cea0aa-kube-api-access-9n7rt\") pod \"router-gateway-1-openshift-default-6b46477c5b-l8vp4\" (UID: \"a0fd1118-d360-4f54-b3e9-f62977cea0aa\") " pod="e2e-test-gateway-section-name-propagation-49b5edd9/router-gateway-1-openshift-default-6b46477c5b-l8vp4"
Sep 17 23:28:58.823328 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:28:58.823226    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-test-gateway-section-name-propagation-49b5edd9/router-gateway-1-openshift-default-6b46477c5b-l8vp4"
Sep 17 23:28:59.102286 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:28:59.102258    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["e2e-test-gateway-section-name-propagation-49b5edd9/router-gateway-1-openshift-default-6b46477c5b-l8vp4"]
Sep 17 23:28:59.104417 ip-10-0-132-188 kubenswrapper[2562]: W0917 23:28:59.104385    2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda0fd1118_d360_4f54_b3e9_f62977cea0aa.slice/crio-d376d2a94ce019b131fc869ec8d0e51c2d40047a1dd5bea52cb9c45a2af6c9c3 WatchSource:0}: Error finding container d376d2a94ce019b131fc869ec8d0e51c2d40047a1dd5bea52cb9c45a2af6c9c3: Status 404 returned error can't find the container with id d376d2a94ce019b131fc869ec8d0e51c2d40047a1dd5bea52cb9c45a2af6c9c3
Sep 17 23:28:59.106586 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:28:59.106546    2562 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"}
Sep 17 23:28:59.106685 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:28:59.106625    2562 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"}
Sep 17 23:28:59.106685 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:28:59.106658    2562 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"}
Sep 17 23:28:59.249629 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:28:59.249593    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-test-gateway-section-name-propagation-49b5edd9/router-gateway-1-openshift-default-6b46477c5b-l8vp4" event={"ID":"a0fd1118-d360-4f54-b3e9-f62977cea0aa","Type":"ContainerStarted","Data":"e85b07ef6e81e265435b6968f0eb9d1834ba69f314f7a57429cfa1bdff5acdb0"}
Sep 17 23:28:59.249805 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:28:59.249649    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-test-gateway-section-name-propagation-49b5edd9/router-gateway-1-openshift-default-6b46477c5b-l8vp4" event={"ID":"a0fd1118-d360-4f54-b3e9-f62977cea0aa","Type":"ContainerStarted","Data":"d376d2a94ce019b131fc869ec8d0e51c2d40047a1dd5bea52cb9c45a2af6c9c3"}
Sep 17 23:28:59.272313 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:28:59.272259    2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="e2e-test-gateway-section-name-propagation-49b5edd9/router-gateway-1-openshift-default-6b46477c5b-l8vp4" podStartSLOduration=1.272239654 podStartE2EDuration="1.272239654s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-17 23:28:58 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-17 23:28:59.270332206 +0000 UTC m=+1091.672228983" watchObservedRunningTime="2026-09-17 23:28:59.272239654 +0000 UTC m=+1091.674136417"
Sep 17 23:28:59.824060 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:28:59.823998    2562 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="e2e-test-gateway-section-name-propagation-49b5edd9/router-gateway-1-openshift-default-6b46477c5b-l8vp4"
Sep 17 23:29:00.829085 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:29:00.829055    2562 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="e2e-test-gateway-section-name-propagation-49b5edd9/router-gateway-1-openshift-default-6b46477c5b-l8vp4"
Sep 17 23:29:01.259968 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:29:01.259880    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-test-gateway-section-name-propagation-49b5edd9/router-gateway-1-openshift-default-6b46477c5b-l8vp4"
Sep 17 23:29:01.260904 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:29:01.260875    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-test-gateway-section-name-propagation-49b5edd9/router-gateway-1-openshift-default-6b46477c5b-l8vp4"
Sep 17 23:29:01.755196 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:29:01.755142    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fwwc5_7e452bbb-7dc6-4c9d-a9b1-76400942f5b4/konnectivity-agent/0.log" limit=1024
Sep 17 23:29:04.190139 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:29:04.190097    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fwwc5_7e452bbb-7dc6-4c9d-a9b1-76400942f5b4/konnectivity-agent/0.log" limit=1024
Sep 17 23:29:07.017761 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:29:07.017724    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fwwc5_7e452bbb-7dc6-4c9d-a9b1-76400942f5b4/konnectivity-agent/0.log" limit=1024
Sep 17 23:29:09.588212 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:29:09.588155    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fwwc5_7e452bbb-7dc6-4c9d-a9b1-76400942f5b4/konnectivity-agent/0.log" limit=1024
Sep 17 23:29:09.685807 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:29:09.685762    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fwwc5_7e452bbb-7dc6-4c9d-a9b1-76400942f5b4/konnectivity-agent/0.log" limit=1024
Sep 17 23:29:11.214310 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:29:11.214257    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fwwc5_7e452bbb-7dc6-4c9d-a9b1-76400942f5b4/konnectivity-agent/0.log" limit=1024
Sep 17 23:29:13.417074 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:29:13.417031    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fwwc5_7e452bbb-7dc6-4c9d-a9b1-76400942f5b4/konnectivity-agent/0.log" limit=1024
Sep 17 23:29:13.518125 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:29:13.518083    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fwwc5_7e452bbb-7dc6-4c9d-a9b1-76400942f5b4/konnectivity-agent/0.log" limit=1024
Sep 17 23:29:13.628543 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:29:13.628498    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fwwc5_7e452bbb-7dc6-4c9d-a9b1-76400942f5b4/konnectivity-agent/0.log" limit=1024
Sep 17 23:29:13.682106 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:29:13.682011    2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["e2e-test-gateway-section-name-propagation-94799d44/router-gateway-1-openshift-default-7dcbd489bd-ncsm7"]
Sep 17 23:29:13.685558 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:29:13.685495    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-test-gateway-section-name-propagation-94799d44/router-gateway-1-openshift-default-7dcbd489bd-ncsm7"
Sep 17 23:29:13.688652 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:29:13.688608    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-test-gateway-section-name-propagation-94799d44\"/\"kube-root-ca.crt\""
Sep 17 23:29:13.688652 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:29:13.688646    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-test-gateway-section-name-propagation-94799d44\"/\"openshift-service-ca.crt\""
Sep 17 23:29:13.688878 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:29:13.688677    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-test-gateway-section-name-propagation-94799d44\"/\"router-gateway-1-openshift-default-dockercfg-dxr4w\""
Sep 17 23:29:13.688878 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:29:13.688775    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-test-gateway-section-name-propagation-94799d44\"/\"openshift-gw-ca-root-cert\""
Sep 17 23:29:13.698397 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:29:13.698367    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["e2e-test-gateway-section-name-propagation-94799d44/router-gateway-1-openshift-default-7dcbd489bd-ncsm7"]
Sep 17 23:29:13.720818 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:29:13.720779    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fwwc5_7e452bbb-7dc6-4c9d-a9b1-76400942f5b4/konnectivity-agent/0.log" limit=1024
Sep 17 23:29:13.767754 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:29:13.767716    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/27ae677b-9eab-404a-a9aa-a06767e259f9-credential-socket\") pod \"router-gateway-1-openshift-default-7dcbd489bd-ncsm7\" (UID: \"27ae677b-9eab-404a-a9aa-a06767e259f9\") " pod="e2e-test-gateway-section-name-propagation-94799d44/router-gateway-1-openshift-default-7dcbd489bd-ncsm7"
Sep 17 23:29:13.767754 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:29:13.767756    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2dk4d\" (UniqueName: \"kubernetes.io/projected/27ae677b-9eab-404a-a9aa-a06767e259f9-kube-api-access-2dk4d\") pod \"router-gateway-1-openshift-default-7dcbd489bd-ncsm7\" (UID: \"27ae677b-9eab-404a-a9aa-a06767e259f9\") " pod="e2e-test-gateway-section-name-propagation-94799d44/router-gateway-1-openshift-default-7dcbd489bd-ncsm7"
Sep 17 23:29:13.768393 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:29:13.767787    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/27ae677b-9eab-404a-a9aa-a06767e259f9-istio-podinfo\") pod \"router-gateway-1-openshift-default-7dcbd489bd-ncsm7\" (UID: \"27ae677b-9eab-404a-a9aa-a06767e259f9\") " pod="e2e-test-gateway-section-name-propagation-94799d44/router-gateway-1-openshift-default-7dcbd489bd-ncsm7"
Sep 17 23:29:13.768393 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:29:13.767870    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/27ae677b-9eab-404a-a9aa-a06767e259f9-istio-envoy\") pod \"router-gateway-1-openshift-default-7dcbd489bd-ncsm7\" (UID: \"27ae677b-9eab-404a-a9aa-a06767e259f9\") " pod="e2e-test-gateway-section-name-propagation-94799d44/router-gateway-1-openshift-default-7dcbd489bd-ncsm7"
Sep 17 23:29:13.768393 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:29:13.767920    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/27ae677b-9eab-404a-a9aa-a06767e259f9-istio-token\") pod \"router-gateway-1-openshift-default-7dcbd489bd-ncsm7\" (UID: \"27ae677b-9eab-404a-a9aa-a06767e259f9\") " pod="e2e-test-gateway-section-name-propagation-94799d44/router-gateway-1-openshift-default-7dcbd489bd-ncsm7"
Sep 17 23:29:13.768393 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:29:13.767944    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/27ae677b-9eab-404a-a9aa-a06767e259f9-workload-socket\") pod \"router-gateway-1-openshift-default-7dcbd489bd-ncsm7\" (UID: \"27ae677b-9eab-404a-a9aa-a06767e259f9\") " pod="e2e-test-gateway-section-name-propagation-94799d44/router-gateway-1-openshift-default-7dcbd489bd-ncsm7"
Sep 17 23:29:13.768393 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:29:13.767990    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/27ae677b-9eab-404a-a9aa-a06767e259f9-istiod-ca-cert\") pod \"router-gateway-1-openshift-default-7dcbd489bd-ncsm7\" (UID: \"27ae677b-9eab-404a-a9aa-a06767e259f9\") " pod="e2e-test-gateway-section-name-propagation-94799d44/router-gateway-1-openshift-default-7dcbd489bd-ncsm7"
Sep 17 23:29:13.768393 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:29:13.768029    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/27ae677b-9eab-404a-a9aa-a06767e259f9-workload-certs\") pod \"router-gateway-1-openshift-default-7dcbd489bd-ncsm7\" (UID: \"27ae677b-9eab-404a-a9aa-a06767e259f9\") " pod="e2e-test-gateway-section-name-propagation-94799d44/router-gateway-1-openshift-default-7dcbd489bd-ncsm7"
Sep 17 23:29:13.768393 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:29:13.768048    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/27ae677b-9eab-404a-a9aa-a06767e259f9-istio-data\") pod \"router-gateway-1-openshift-default-7dcbd489bd-ncsm7\" (UID: \"27ae677b-9eab-404a-a9aa-a06767e259f9\") " pod="e2e-test-gateway-section-name-propagation-94799d44/router-gateway-1-openshift-default-7dcbd489bd-ncsm7"
Sep 17 23:29:13.818116 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:29:13.818077    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fwwc5_7e452bbb-7dc6-4c9d-a9b1-76400942f5b4/konnectivity-agent/0.log" limit=1024
Sep 17 23:29:13.869520 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:29:13.869481    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/27ae677b-9eab-404a-a9aa-a06767e259f9-workload-certs\") pod \"router-gateway-1-openshift-default-7dcbd489bd-ncsm7\" (UID: \"27ae677b-9eab-404a-a9aa-a06767e259f9\") " pod="e2e-test-gateway-section-name-propagation-94799d44/router-gateway-1-openshift-default-7dcbd489bd-ncsm7"
Sep 17 23:29:13.869520 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:29:13.869523    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/27ae677b-9eab-404a-a9aa-a06767e259f9-istio-data\") pod \"router-gateway-1-openshift-default-7dcbd489bd-ncsm7\" (UID: \"27ae677b-9eab-404a-a9aa-a06767e259f9\") " pod="e2e-test-gateway-section-name-propagation-94799d44/router-gateway-1-openshift-default-7dcbd489bd-ncsm7"
Sep 17 23:29:13.869812 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:29:13.869574    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/27ae677b-9eab-404a-a9aa-a06767e259f9-credential-socket\") pod \"router-gateway-1-openshift-default-7dcbd489bd-ncsm7\" (UID: \"27ae677b-9eab-404a-a9aa-a06767e259f9\") " pod="e2e-test-gateway-section-name-propagation-94799d44/router-gateway-1-openshift-default-7dcbd489bd-ncsm7"
Sep 17 23:29:13.869812 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:29:13.869591    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2dk4d\" (UniqueName: \"kubernetes.io/projected/27ae677b-9eab-404a-a9aa-a06767e259f9-kube-api-access-2dk4d\") pod \"router-gateway-1-openshift-default-7dcbd489bd-ncsm7\" (UID: \"27ae677b-9eab-404a-a9aa-a06767e259f9\") " pod="e2e-test-gateway-section-name-propagation-94799d44/router-gateway-1-openshift-default-7dcbd489bd-ncsm7"
Sep 17 23:29:13.869812 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:29:13.869720    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/27ae677b-9eab-404a-a9aa-a06767e259f9-istio-podinfo\") pod \"router-gateway-1-openshift-default-7dcbd489bd-ncsm7\" (UID: \"27ae677b-9eab-404a-a9aa-a06767e259f9\") " pod="e2e-test-gateway-section-name-propagation-94799d44/router-gateway-1-openshift-default-7dcbd489bd-ncsm7"
Sep 17 23:29:13.869812 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:29:13.869781    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/27ae677b-9eab-404a-a9aa-a06767e259f9-istio-envoy\") pod \"router-gateway-1-openshift-default-7dcbd489bd-ncsm7\" (UID: \"27ae677b-9eab-404a-a9aa-a06767e259f9\") " pod="e2e-test-gateway-section-name-propagation-94799d44/router-gateway-1-openshift-default-7dcbd489bd-ncsm7"
Sep 17 23:29:13.870033 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:29:13.869813    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/27ae677b-9eab-404a-a9aa-a06767e259f9-istio-token\") pod \"router-gateway-1-openshift-default-7dcbd489bd-ncsm7\" (UID: \"27ae677b-9eab-404a-a9aa-a06767e259f9\") " pod="e2e-test-gateway-section-name-propagation-94799d44/router-gateway-1-openshift-default-7dcbd489bd-ncsm7"
Sep 17 23:29:13.870033 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:29:13.869844    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/27ae677b-9eab-404a-a9aa-a06767e259f9-workload-socket\") pod \"router-gateway-1-openshift-default-7dcbd489bd-ncsm7\" (UID: \"27ae677b-9eab-404a-a9aa-a06767e259f9\") " pod="e2e-test-gateway-section-name-propagation-94799d44/router-gateway-1-openshift-default-7dcbd489bd-ncsm7"
Sep 17 23:29:13.870033 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:29:13.869906    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/27ae677b-9eab-404a-a9aa-a06767e259f9-istiod-ca-cert\") pod \"router-gateway-1-openshift-default-7dcbd489bd-ncsm7\" (UID: \"27ae677b-9eab-404a-a9aa-a06767e259f9\") " pod="e2e-test-gateway-section-name-propagation-94799d44/router-gateway-1-openshift-default-7dcbd489bd-ncsm7"
Sep 17 23:29:13.870033 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:29:13.869968    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/27ae677b-9eab-404a-a9aa-a06767e259f9-workload-certs\") pod \"router-gateway-1-openshift-default-7dcbd489bd-ncsm7\" (UID: \"27ae677b-9eab-404a-a9aa-a06767e259f9\") " pod="e2e-test-gateway-section-name-propagation-94799d44/router-gateway-1-openshift-default-7dcbd489bd-ncsm7"
Sep 17 23:29:13.870289 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:29:13.870070    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/27ae677b-9eab-404a-a9aa-a06767e259f9-istio-data\") pod \"router-gateway-1-openshift-default-7dcbd489bd-ncsm7\" (UID: \"27ae677b-9eab-404a-a9aa-a06767e259f9\") " pod="e2e-test-gateway-section-name-propagation-94799d44/router-gateway-1-openshift-default-7dcbd489bd-ncsm7"
Sep 17 23:29:13.870349 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:29:13.870320    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/27ae677b-9eab-404a-a9aa-a06767e259f9-credential-socket\") pod \"router-gateway-1-openshift-default-7dcbd489bd-ncsm7\" (UID: \"27ae677b-9eab-404a-a9aa-a06767e259f9\") " pod="e2e-test-gateway-section-name-propagation-94799d44/router-gateway-1-openshift-default-7dcbd489bd-ncsm7"
Sep 17 23:29:13.870397 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:29:13.870320    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/27ae677b-9eab-404a-a9aa-a06767e259f9-workload-socket\") pod \"router-gateway-1-openshift-default-7dcbd489bd-ncsm7\" (UID: \"27ae677b-9eab-404a-a9aa-a06767e259f9\") " pod="e2e-test-gateway-section-name-propagation-94799d44/router-gateway-1-openshift-default-7dcbd489bd-ncsm7"
Sep 17 23:29:13.870667 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:29:13.870648    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/27ae677b-9eab-404a-a9aa-a06767e259f9-istiod-ca-cert\") pod \"router-gateway-1-openshift-default-7dcbd489bd-ncsm7\" (UID: \"27ae677b-9eab-404a-a9aa-a06767e259f9\") " pod="e2e-test-gateway-section-name-propagation-94799d44/router-gateway-1-openshift-default-7dcbd489bd-ncsm7"
Sep 17 23:29:13.872159 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:29:13.872138    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/27ae677b-9eab-404a-a9aa-a06767e259f9-istio-envoy\") pod \"router-gateway-1-openshift-default-7dcbd489bd-ncsm7\" (UID: \"27ae677b-9eab-404a-a9aa-a06767e259f9\") " pod="e2e-test-gateway-section-name-propagation-94799d44/router-gateway-1-openshift-default-7dcbd489bd-ncsm7"
Sep 17 23:29:13.872403 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:29:13.872383    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/27ae677b-9eab-404a-a9aa-a06767e259f9-istio-podinfo\") pod \"router-gateway-1-openshift-default-7dcbd489bd-ncsm7\" (UID: \"27ae677b-9eab-404a-a9aa-a06767e259f9\") " pod="e2e-test-gateway-section-name-propagation-94799d44/router-gateway-1-openshift-default-7dcbd489bd-ncsm7"
Sep 17 23:29:13.877511 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:29:13.877478    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/27ae677b-9eab-404a-a9aa-a06767e259f9-istio-token\") pod \"router-gateway-1-openshift-default-7dcbd489bd-ncsm7\" (UID: \"27ae677b-9eab-404a-a9aa-a06767e259f9\") " pod="e2e-test-gateway-section-name-propagation-94799d44/router-gateway-1-openshift-default-7dcbd489bd-ncsm7"
Sep 17 23:29:13.877667 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:29:13.877635    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-2dk4d\" (UniqueName: \"kubernetes.io/projected/27ae677b-9eab-404a-a9aa-a06767e259f9-kube-api-access-2dk4d\") pod \"router-gateway-1-openshift-default-7dcbd489bd-ncsm7\" (UID: \"27ae677b-9eab-404a-a9aa-a06767e259f9\") " pod="e2e-test-gateway-section-name-propagation-94799d44/router-gateway-1-openshift-default-7dcbd489bd-ncsm7"
Sep 17 23:29:13.999126 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:29:13.999016    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-test-gateway-section-name-propagation-94799d44/router-gateway-1-openshift-default-7dcbd489bd-ncsm7"
Sep 17 23:29:14.302072 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:29:14.302041    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["e2e-test-gateway-section-name-propagation-94799d44/router-gateway-1-openshift-default-7dcbd489bd-ncsm7"]
Sep 17 23:29:14.304809 ip-10-0-132-188 kubenswrapper[2562]: W0917 23:29:14.304767    2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod27ae677b_9eab_404a_a9aa_a06767e259f9.slice/crio-67318fbc523ba3d6017711db3513c4ea61b50ffc5542a7f9e8cff6f9b4bd6c33 WatchSource:0}: Error finding container 67318fbc523ba3d6017711db3513c4ea61b50ffc5542a7f9e8cff6f9b4bd6c33: Status 404 returned error can't find the container with id 67318fbc523ba3d6017711db3513c4ea61b50ffc5542a7f9e8cff6f9b4bd6c33
Sep 17 23:29:14.306998 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:29:14.306964    2562 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"}
Sep 17 23:29:14.307066 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:29:14.307038    2562 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"}
Sep 17 23:29:14.307100 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:29:14.307088    2562 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"}
Sep 17 23:29:14.317329 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:29:14.317298    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-test-gateway-section-name-propagation-94799d44/router-gateway-1-openshift-default-7dcbd489bd-ncsm7" event={"ID":"27ae677b-9eab-404a-a9aa-a06767e259f9","Type":"ContainerStarted","Data":"67318fbc523ba3d6017711db3513c4ea61b50ffc5542a7f9e8cff6f9b4bd6c33"}
Sep 17 23:29:14.727233 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:29:14.727197    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fwwc5_7e452bbb-7dc6-4c9d-a9b1-76400942f5b4/konnectivity-agent/0.log" limit=1024
Sep 17 23:29:15.323367 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:29:15.323321    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-test-gateway-section-name-propagation-94799d44/router-gateway-1-openshift-default-7dcbd489bd-ncsm7" event={"ID":"27ae677b-9eab-404a-a9aa-a06767e259f9","Type":"ContainerStarted","Data":"7940d387d519414771302c325f5bdf144fcad0172eed97539363a4d8b6e81021"}
Sep 17 23:29:15.347777 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:29:15.347699    2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="e2e-test-gateway-section-name-propagation-94799d44/router-gateway-1-openshift-default-7dcbd489bd-ncsm7" podStartSLOduration=2.347662753 podStartE2EDuration="2.347662753s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-17 23:29:13 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-17 23:29:15.344823561 +0000 UTC m=+1107.746720366" watchObservedRunningTime="2026-09-17 23:29:15.347662753 +0000 UTC m=+1107.749559519"
Sep 17 23:29:15.999413 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:29:15.999375    2562 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="e2e-test-gateway-section-name-propagation-94799d44/router-gateway-1-openshift-default-7dcbd489bd-ncsm7"
Sep 17 23:29:16.004418 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:29:16.004389    2562 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="e2e-test-gateway-section-name-propagation-94799d44/router-gateway-1-openshift-default-7dcbd489bd-ncsm7"
Sep 17 23:29:16.328475 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:29:16.328270    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-test-gateway-section-name-propagation-94799d44/router-gateway-1-openshift-default-7dcbd489bd-ncsm7"
Sep 17 23:29:16.329222 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:29:16.329163    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-test-gateway-section-name-propagation-94799d44/router-gateway-1-openshift-default-7dcbd489bd-ncsm7"
Sep 17 23:29:16.568056 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:29:16.567482    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fwwc5_7e452bbb-7dc6-4c9d-a9b1-76400942f5b4/konnectivity-agent/0.log" limit=1024
Sep 17 23:29:18.396439 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:29:18.396387    2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["e2e-test-gateway-section-name-propagation-49b5edd9/router-gateway-1-openshift-default-6b46477c5b-l8vp4"]
Sep 17 23:29:18.396914 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:29:18.396703    2562 kuberuntime_container.go:873] "Killing container with a grace period" pod="e2e-test-gateway-section-name-propagation-49b5edd9/router-gateway-1-openshift-default-6b46477c5b-l8vp4" podUID="a0fd1118-d360-4f54-b3e9-f62977cea0aa" containerName="istio-proxy" containerID="cri-o://e85b07ef6e81e265435b6968f0eb9d1834ba69f314f7a57429cfa1bdff5acdb0" gracePeriod=30
Sep 17 23:29:18.489378 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:29:18.489337    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fwwc5_7e452bbb-7dc6-4c9d-a9b1-76400942f5b4/konnectivity-agent/0.log" limit=1024
Sep 17 23:29:20.943902 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:29:20.943876    2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-test-gateway-section-name-propagation-49b5edd9/router-gateway-1-openshift-default-6b46477c5b-l8vp4"
Sep 17 23:29:21.040720 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:29:21.040691    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/a0fd1118-d360-4f54-b3e9-f62977cea0aa-workload-socket\") pod \"a0fd1118-d360-4f54-b3e9-f62977cea0aa\" (UID: \"a0fd1118-d360-4f54-b3e9-f62977cea0aa\") "
Sep 17 23:29:21.040720 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:29:21.040723    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/a0fd1118-d360-4f54-b3e9-f62977cea0aa-istio-token\") pod \"a0fd1118-d360-4f54-b3e9-f62977cea0aa\" (UID: \"a0fd1118-d360-4f54-b3e9-f62977cea0aa\") "
Sep 17 23:29:21.040938 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:29:21.040749    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-9n7rt\" (UniqueName: \"kubernetes.io/projected/a0fd1118-d360-4f54-b3e9-f62977cea0aa-kube-api-access-9n7rt\") pod \"a0fd1118-d360-4f54-b3e9-f62977cea0aa\" (UID: \"a0fd1118-d360-4f54-b3e9-f62977cea0aa\") "
Sep 17 23:29:21.040938 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:29:21.040806    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/a0fd1118-d360-4f54-b3e9-f62977cea0aa-istiod-ca-cert\") pod \"a0fd1118-d360-4f54-b3e9-f62977cea0aa\" (UID: \"a0fd1118-d360-4f54-b3e9-f62977cea0aa\") "
Sep 17 23:29:21.040938 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:29:21.040845    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/a0fd1118-d360-4f54-b3e9-f62977cea0aa-workload-certs\") pod \"a0fd1118-d360-4f54-b3e9-f62977cea0aa\" (UID: \"a0fd1118-d360-4f54-b3e9-f62977cea0aa\") "
Sep 17 23:29:21.040938 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:29:21.040880    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/a0fd1118-d360-4f54-b3e9-f62977cea0aa-istio-data\") pod \"a0fd1118-d360-4f54-b3e9-f62977cea0aa\" (UID: \"a0fd1118-d360-4f54-b3e9-f62977cea0aa\") "
Sep 17 23:29:21.040938 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:29:21.040905    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/a0fd1118-d360-4f54-b3e9-f62977cea0aa-istio-podinfo\") pod \"a0fd1118-d360-4f54-b3e9-f62977cea0aa\" (UID: \"a0fd1118-d360-4f54-b3e9-f62977cea0aa\") "
Sep 17 23:29:21.041222 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:29:21.040941    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/a0fd1118-d360-4f54-b3e9-f62977cea0aa-istio-envoy\") pod \"a0fd1118-d360-4f54-b3e9-f62977cea0aa\" (UID: \"a0fd1118-d360-4f54-b3e9-f62977cea0aa\") "
Sep 17 23:29:21.041222 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:29:21.040979    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a0fd1118-d360-4f54-b3e9-f62977cea0aa-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "a0fd1118-d360-4f54-b3e9-f62977cea0aa" (UID: "a0fd1118-d360-4f54-b3e9-f62977cea0aa"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 17 23:29:21.041222 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:29:21.040997    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/a0fd1118-d360-4f54-b3e9-f62977cea0aa-credential-socket\") pod \"a0fd1118-d360-4f54-b3e9-f62977cea0aa\" (UID: \"a0fd1118-d360-4f54-b3e9-f62977cea0aa\") "
Sep 17 23:29:21.041403 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:29:21.041254    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a0fd1118-d360-4f54-b3e9-f62977cea0aa-istio-data" (OuterVolumeSpecName: "istio-data") pod "a0fd1118-d360-4f54-b3e9-f62977cea0aa" (UID: "a0fd1118-d360-4f54-b3e9-f62977cea0aa"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 17 23:29:21.041403 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:29:21.041308    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/a0fd1118-d360-4f54-b3e9-f62977cea0aa-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "a0fd1118-d360-4f54-b3e9-f62977cea0aa" (UID: "a0fd1118-d360-4f54-b3e9-f62977cea0aa"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 17 23:29:21.041500 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:29:21.041475    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a0fd1118-d360-4f54-b3e9-f62977cea0aa-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "a0fd1118-d360-4f54-b3e9-f62977cea0aa" (UID: "a0fd1118-d360-4f54-b3e9-f62977cea0aa"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 17 23:29:21.041558 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:29:21.041516    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a0fd1118-d360-4f54-b3e9-f62977cea0aa-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "a0fd1118-d360-4f54-b3e9-f62977cea0aa" (UID: "a0fd1118-d360-4f54-b3e9-f62977cea0aa"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 17 23:29:21.041632 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:29:21.041578    2562 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/a0fd1118-d360-4f54-b3e9-f62977cea0aa-istiod-ca-cert\") on node \"ip-10-0-132-188.ec2.internal\" DevicePath \"\""
Sep 17 23:29:21.041632 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:29:21.041592    2562 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/a0fd1118-d360-4f54-b3e9-f62977cea0aa-workload-certs\") on node \"ip-10-0-132-188.ec2.internal\" DevicePath \"\""
Sep 17 23:29:21.041632 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:29:21.041606    2562 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/a0fd1118-d360-4f54-b3e9-f62977cea0aa-istio-data\") on node \"ip-10-0-132-188.ec2.internal\" DevicePath \"\""
Sep 17 23:29:21.041632 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:29:21.041617    2562 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/a0fd1118-d360-4f54-b3e9-f62977cea0aa-credential-socket\") on node \"ip-10-0-132-188.ec2.internal\" DevicePath \"\""
Sep 17 23:29:21.041632 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:29:21.041631    2562 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/a0fd1118-d360-4f54-b3e9-f62977cea0aa-workload-socket\") on node \"ip-10-0-132-188.ec2.internal\" DevicePath \"\""
Sep 17 23:29:21.043373 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:29:21.043341    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a0fd1118-d360-4f54-b3e9-f62977cea0aa-istio-token" (OuterVolumeSpecName: "istio-token") pod "a0fd1118-d360-4f54-b3e9-f62977cea0aa" (UID: "a0fd1118-d360-4f54-b3e9-f62977cea0aa"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 17 23:29:21.043506 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:29:21.043386    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a0fd1118-d360-4f54-b3e9-f62977cea0aa-kube-api-access-9n7rt" (OuterVolumeSpecName: "kube-api-access-9n7rt") pod "a0fd1118-d360-4f54-b3e9-f62977cea0aa" (UID: "a0fd1118-d360-4f54-b3e9-f62977cea0aa"). InnerVolumeSpecName "kube-api-access-9n7rt". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 17 23:29:21.043506 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:29:21.043433    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/a0fd1118-d360-4f54-b3e9-f62977cea0aa-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "a0fd1118-d360-4f54-b3e9-f62977cea0aa" (UID: "a0fd1118-d360-4f54-b3e9-f62977cea0aa"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Sep 17 23:29:21.043630 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:29:21.043611    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a0fd1118-d360-4f54-b3e9-f62977cea0aa-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "a0fd1118-d360-4f54-b3e9-f62977cea0aa" (UID: "a0fd1118-d360-4f54-b3e9-f62977cea0aa"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 17 23:29:21.142822 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:29:21.142785    2562 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-9n7rt\" (UniqueName: \"kubernetes.io/projected/a0fd1118-d360-4f54-b3e9-f62977cea0aa-kube-api-access-9n7rt\") on node \"ip-10-0-132-188.ec2.internal\" DevicePath \"\""
Sep 17 23:29:21.142822 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:29:21.142814    2562 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/a0fd1118-d360-4f54-b3e9-f62977cea0aa-istio-podinfo\") on node \"ip-10-0-132-188.ec2.internal\" DevicePath \"\""
Sep 17 23:29:21.142822 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:29:21.142823    2562 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/a0fd1118-d360-4f54-b3e9-f62977cea0aa-istio-envoy\") on node \"ip-10-0-132-188.ec2.internal\" DevicePath \"\""
Sep 17 23:29:21.142822 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:29:21.142831    2562 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/a0fd1118-d360-4f54-b3e9-f62977cea0aa-istio-token\") on node \"ip-10-0-132-188.ec2.internal\" DevicePath \"\""
Sep 17 23:29:21.355329 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:29:21.355292    2562 generic.go:356] "Generic (PLEG): container finished" podID="a0fd1118-d360-4f54-b3e9-f62977cea0aa" containerID="e85b07ef6e81e265435b6968f0eb9d1834ba69f314f7a57429cfa1bdff5acdb0" exitCode=0
Sep 17 23:29:21.355523 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:29:21.355345    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-test-gateway-section-name-propagation-49b5edd9/router-gateway-1-openshift-default-6b46477c5b-l8vp4" event={"ID":"a0fd1118-d360-4f54-b3e9-f62977cea0aa","Type":"ContainerDied","Data":"e85b07ef6e81e265435b6968f0eb9d1834ba69f314f7a57429cfa1bdff5acdb0"}
Sep 17 23:29:21.355523 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:29:21.355388    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-test-gateway-section-name-propagation-49b5edd9/router-gateway-1-openshift-default-6b46477c5b-l8vp4" event={"ID":"a0fd1118-d360-4f54-b3e9-f62977cea0aa","Type":"ContainerDied","Data":"d376d2a94ce019b131fc869ec8d0e51c2d40047a1dd5bea52cb9c45a2af6c9c3"}
Sep 17 23:29:21.355523 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:29:21.355391    2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-test-gateway-section-name-propagation-49b5edd9/router-gateway-1-openshift-default-6b46477c5b-l8vp4"
Sep 17 23:29:21.355523 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:29:21.355417    2562 scope.go:117] "RemoveContainer" containerID="e85b07ef6e81e265435b6968f0eb9d1834ba69f314f7a57429cfa1bdff5acdb0"
Sep 17 23:29:21.365053 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:29:21.365030    2562 scope.go:117] "RemoveContainer" containerID="e85b07ef6e81e265435b6968f0eb9d1834ba69f314f7a57429cfa1bdff5acdb0"
Sep 17 23:29:21.365396 ip-10-0-132-188 kubenswrapper[2562]: E0917 23:29:21.365370    2562 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"e85b07ef6e81e265435b6968f0eb9d1834ba69f314f7a57429cfa1bdff5acdb0\": container with ID starting with e85b07ef6e81e265435b6968f0eb9d1834ba69f314f7a57429cfa1bdff5acdb0 not found: ID does not exist" containerID="e85b07ef6e81e265435b6968f0eb9d1834ba69f314f7a57429cfa1bdff5acdb0"
Sep 17 23:29:21.365502 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:29:21.365407    2562 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e85b07ef6e81e265435b6968f0eb9d1834ba69f314f7a57429cfa1bdff5acdb0"} err="failed to get container status \"e85b07ef6e81e265435b6968f0eb9d1834ba69f314f7a57429cfa1bdff5acdb0\": rpc error: code = NotFound desc = could not find container \"e85b07ef6e81e265435b6968f0eb9d1834ba69f314f7a57429cfa1bdff5acdb0\": container with ID starting with e85b07ef6e81e265435b6968f0eb9d1834ba69f314f7a57429cfa1bdff5acdb0 not found: ID does not exist"
Sep 17 23:29:21.376225 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:29:21.376191    2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["e2e-test-gateway-section-name-propagation-49b5edd9/router-gateway-1-openshift-default-6b46477c5b-l8vp4"]
Sep 17 23:29:21.379661 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:29:21.379630    2562 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["e2e-test-gateway-section-name-propagation-49b5edd9/router-gateway-1-openshift-default-6b46477c5b-l8vp4"]
Sep 17 23:29:21.672442 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:29:21.672355    2562 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="a0fd1118-d360-4f54-b3e9-f62977cea0aa" path="/var/lib/kubelet/pods/a0fd1118-d360-4f54-b3e9-f62977cea0aa/volumes"
Sep 17 23:29:23.773578 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:29:23.773534    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fwwc5_7e452bbb-7dc6-4c9d-a9b1-76400942f5b4/konnectivity-agent/0.log" limit=1024
Sep 17 23:29:24.293271 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:29:24.293226    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fwwc5_7e452bbb-7dc6-4c9d-a9b1-76400942f5b4/konnectivity-agent/0.log" limit=1024
Sep 17 23:29:29.238124 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:29:29.238082    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fwwc5_7e452bbb-7dc6-4c9d-a9b1-76400942f5b4/konnectivity-agent/0.log" limit=1024
Sep 17 23:29:30.151444 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:29:30.151402    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fwwc5_7e452bbb-7dc6-4c9d-a9b1-76400942f5b4/konnectivity-agent/0.log" limit=1024
Sep 17 23:29:30.291690 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:29:30.291653    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fwwc5_7e452bbb-7dc6-4c9d-a9b1-76400942f5b4/konnectivity-agent/0.log" limit=1024
Sep 17 23:29:30.393310 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:29:30.393201    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fwwc5_7e452bbb-7dc6-4c9d-a9b1-76400942f5b4/konnectivity-agent/0.log" limit=1024
Sep 17 23:29:31.530290 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:29:31.530251    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fwwc5_7e452bbb-7dc6-4c9d-a9b1-76400942f5b4/konnectivity-agent/0.log" limit=1024
Sep 17 23:29:31.627057 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:29:31.627022    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fwwc5_7e452bbb-7dc6-4c9d-a9b1-76400942f5b4/konnectivity-agent/0.log" limit=1024
Sep 17 23:29:32.062196 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:29:32.061246    2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["e2e-test-llm-inference-service-9c386aa9/scheduler-v06-pd-migration-test-kserve-prefill-8665687897-rpvrt"]
Sep 17 23:29:32.070197 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:29:32.069947    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fwwc5_7e452bbb-7dc6-4c9d-a9b1-76400942f5b4/konnectivity-agent/0.log" limit=1024
Sep 17 23:29:32.078801 ip-10-0-132-188 kubenswrapper[2562]: E0917 23:29:32.078749    2562 log.go:32] "ExecSync cmd from runtime service failed" err=<
Sep 17 23:29:32.078801 ip-10-0-132-188 kubenswrapper[2562]:         rpc error: code = Unknown desc = command error: executable file `/bin/sleep` not found: No such file or directory
Sep 17 23:29:32.078801 ip-10-0-132-188 kubenswrapper[2562]:         , stdout: , stderr: , exit code -1
Sep 17 23:29:32.078801 ip-10-0-132-188 kubenswrapper[2562]:  > containerID="05023b8ffa449da3c06d75d0802fb9d95aedd51d6841308ea06e94cd7db4dbfc" cmd=["/bin/sleep","15"]
Sep 17 23:29:32.079059 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:29:32.078812    2562 handlers.go:82] "Exec lifecycle hook for Container in Pod failed" execCommand=["/bin/sleep","15"] containerName="main" pod="e2e-test-llm-inference-service-9c386aa9/scheduler-v06-pd-migration-test-kserve-prefill-8665687897-rpvrt" message="" err=<
Sep 17 23:29:32.079059 ip-10-0-132-188 kubenswrapper[2562]:         rpc error: code = Unknown desc = command error: executable file `/bin/sleep` not found: No such file or directory
Sep 17 23:29:32.079059 ip-10-0-132-188 kubenswrapper[2562]:         , stdout: , stderr: , exit code -1
Sep 17 23:29:32.079059 ip-10-0-132-188 kubenswrapper[2562]:  >
Sep 17 23:29:32.079059 ip-10-0-132-188 kubenswrapper[2562]: E0917 23:29:32.078825    2562 kuberuntime_container.go:755] "PreStop hook failed" err=<
Sep 17 23:29:32.079059 ip-10-0-132-188 kubenswrapper[2562]:         rpc error: code = Unknown desc = command error: executable file `/bin/sleep` not found: No such file or directory
Sep 17 23:29:32.079059 ip-10-0-132-188 kubenswrapper[2562]:         , stdout: , stderr: , exit code -1
Sep 17 23:29:32.079059 ip-10-0-132-188 kubenswrapper[2562]:  > pod="e2e-test-llm-inference-service-9c386aa9/scheduler-v06-pd-migration-test-kserve-prefill-8665687897-rpvrt" podUID="9dd4e65c-5e0a-4175-8f86-1bd8d2a0e984" containerName="main" containerID="cri-o://05023b8ffa449da3c06d75d0802fb9d95aedd51d6841308ea06e94cd7db4dbfc"
Sep 17 23:29:32.079059 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:29:32.078871    2562 kuberuntime_container.go:873] "Killing container with a grace period" pod="e2e-test-llm-inference-service-9c386aa9/scheduler-v06-pd-migration-test-kserve-prefill-8665687897-rpvrt" podUID="9dd4e65c-5e0a-4175-8f86-1bd8d2a0e984" containerName="main" containerID="cri-o://05023b8ffa449da3c06d75d0802fb9d95aedd51d6841308ea06e94cd7db4dbfc" gracePeriod=60
Sep 17 23:29:32.143782 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:29:32.143722    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fwwc5_7e452bbb-7dc6-4c9d-a9b1-76400942f5b4/konnectivity-agent/0.log" limit=1024
Sep 17 23:29:32.225095 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:29:32.225052    2562 prober.go:120] "Probe failed" probeType="Readiness" pod="e2e-test-llm-inference-service-9c386aa9/scheduler-v06-pd-migration-test-kserve-prefill-8665687897-rpvrt" podUID="9dd4e65c-5e0a-4175-8f86-1bd8d2a0e984" containerName="main" probeResult="failure" output="Get \"https://10.134.0.33:8000/health\": dial tcp 10.134.0.33:8000: connect: connection refused"
Sep 17 23:29:32.409563 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:29:32.409531    2562 generic.go:356] "Generic (PLEG): container finished" podID="9dd4e65c-5e0a-4175-8f86-1bd8d2a0e984" containerID="05023b8ffa449da3c06d75d0802fb9d95aedd51d6841308ea06e94cd7db4dbfc" exitCode=0
Sep 17 23:29:32.409750 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:29:32.409605    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-test-llm-inference-service-9c386aa9/scheduler-v06-pd-migration-test-kserve-prefill-8665687897-rpvrt" event={"ID":"9dd4e65c-5e0a-4175-8f86-1bd8d2a0e984","Type":"ContainerDied","Data":"05023b8ffa449da3c06d75d0802fb9d95aedd51d6841308ea06e94cd7db4dbfc"}
Sep 17 23:29:32.867233 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:29:32.867204    2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-test-llm-inference-service-9c386aa9/scheduler-v06-pd-migration-test-kserve-prefill-8665687897-rpvrt"
Sep 17 23:29:32.971854 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:29:32.971753    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/9dd4e65c-5e0a-4175-8f86-1bd8d2a0e984-home\") pod \"9dd4e65c-5e0a-4175-8f86-1bd8d2a0e984\" (UID: \"9dd4e65c-5e0a-4175-8f86-1bd8d2a0e984\") "
Sep 17 23:29:32.971854 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:29:32.971803    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/9dd4e65c-5e0a-4175-8f86-1bd8d2a0e984-tmp-dir\") pod \"9dd4e65c-5e0a-4175-8f86-1bd8d2a0e984\" (UID: \"9dd4e65c-5e0a-4175-8f86-1bd8d2a0e984\") "
Sep 17 23:29:32.971854 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:29:32.971837    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/9dd4e65c-5e0a-4175-8f86-1bd8d2a0e984-dshm\") pod \"9dd4e65c-5e0a-4175-8f86-1bd8d2a0e984\" (UID: \"9dd4e65c-5e0a-4175-8f86-1bd8d2a0e984\") "
Sep 17 23:29:32.972138 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:29:32.971874    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-vc5mh\" (UniqueName: \"kubernetes.io/projected/9dd4e65c-5e0a-4175-8f86-1bd8d2a0e984-kube-api-access-vc5mh\") pod \"9dd4e65c-5e0a-4175-8f86-1bd8d2a0e984\" (UID: \"9dd4e65c-5e0a-4175-8f86-1bd8d2a0e984\") "
Sep 17 23:29:32.972138 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:29:32.971909    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/9dd4e65c-5e0a-4175-8f86-1bd8d2a0e984-model-cache\") pod \"9dd4e65c-5e0a-4175-8f86-1bd8d2a0e984\" (UID: \"9dd4e65c-5e0a-4175-8f86-1bd8d2a0e984\") "
Sep 17 23:29:32.972138 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:29:32.971949    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/9dd4e65c-5e0a-4175-8f86-1bd8d2a0e984-tls-certs\") pod \"9dd4e65c-5e0a-4175-8f86-1bd8d2a0e984\" (UID: \"9dd4e65c-5e0a-4175-8f86-1bd8d2a0e984\") "
Sep 17 23:29:32.972138 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:29:32.972045    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/9dd4e65c-5e0a-4175-8f86-1bd8d2a0e984-home" (OuterVolumeSpecName: "home") pod "9dd4e65c-5e0a-4175-8f86-1bd8d2a0e984" (UID: "9dd4e65c-5e0a-4175-8f86-1bd8d2a0e984"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 17 23:29:32.972138 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:29:32.972058    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/9dd4e65c-5e0a-4175-8f86-1bd8d2a0e984-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "9dd4e65c-5e0a-4175-8f86-1bd8d2a0e984" (UID: "9dd4e65c-5e0a-4175-8f86-1bd8d2a0e984"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 17 23:29:32.972372 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:29:32.972254    2562 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/9dd4e65c-5e0a-4175-8f86-1bd8d2a0e984-home\") on node \"ip-10-0-132-188.ec2.internal\" DevicePath \"\""
Sep 17 23:29:32.972372 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:29:32.972275    2562 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/9dd4e65c-5e0a-4175-8f86-1bd8d2a0e984-tmp-dir\") on node \"ip-10-0-132-188.ec2.internal\" DevicePath \"\""
Sep 17 23:29:32.972489 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:29:32.972461    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/9dd4e65c-5e0a-4175-8f86-1bd8d2a0e984-model-cache" (OuterVolumeSpecName: "model-cache") pod "9dd4e65c-5e0a-4175-8f86-1bd8d2a0e984" (UID: "9dd4e65c-5e0a-4175-8f86-1bd8d2a0e984"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 17 23:29:32.974403 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:29:32.974375    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/9dd4e65c-5e0a-4175-8f86-1bd8d2a0e984-dshm" (OuterVolumeSpecName: "dshm") pod "9dd4e65c-5e0a-4175-8f86-1bd8d2a0e984" (UID: "9dd4e65c-5e0a-4175-8f86-1bd8d2a0e984"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 17 23:29:32.974543 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:29:32.974447    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/9dd4e65c-5e0a-4175-8f86-1bd8d2a0e984-kube-api-access-vc5mh" (OuterVolumeSpecName: "kube-api-access-vc5mh") pod "9dd4e65c-5e0a-4175-8f86-1bd8d2a0e984" (UID: "9dd4e65c-5e0a-4175-8f86-1bd8d2a0e984"). InnerVolumeSpecName "kube-api-access-vc5mh". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 17 23:29:32.974543 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:29:32.974499    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/9dd4e65c-5e0a-4175-8f86-1bd8d2a0e984-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "9dd4e65c-5e0a-4175-8f86-1bd8d2a0e984" (UID: "9dd4e65c-5e0a-4175-8f86-1bd8d2a0e984"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 17 23:29:33.073645 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:29:33.073601    2562 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/9dd4e65c-5e0a-4175-8f86-1bd8d2a0e984-dshm\") on node \"ip-10-0-132-188.ec2.internal\" DevicePath \"\""
Sep 17 23:29:33.073645 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:29:33.073646    2562 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-vc5mh\" (UniqueName: \"kubernetes.io/projected/9dd4e65c-5e0a-4175-8f86-1bd8d2a0e984-kube-api-access-vc5mh\") on node \"ip-10-0-132-188.ec2.internal\" DevicePath \"\""
Sep 17 23:29:33.073866 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:29:33.073665    2562 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/9dd4e65c-5e0a-4175-8f86-1bd8d2a0e984-model-cache\") on node \"ip-10-0-132-188.ec2.internal\" DevicePath \"\""
Sep 17 23:29:33.073866 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:29:33.073679    2562 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/9dd4e65c-5e0a-4175-8f86-1bd8d2a0e984-tls-certs\") on node \"ip-10-0-132-188.ec2.internal\" DevicePath \"\""
Sep 17 23:29:33.129200 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:29:33.129151    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fwwc5_7e452bbb-7dc6-4c9d-a9b1-76400942f5b4/konnectivity-agent/0.log" limit=1024
Sep 17 23:29:33.415891 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:29:33.415854    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-test-llm-inference-service-9c386aa9/scheduler-v06-pd-migration-test-kserve-prefill-8665687897-rpvrt" event={"ID":"9dd4e65c-5e0a-4175-8f86-1bd8d2a0e984","Type":"ContainerDied","Data":"5a4cb3e93f49a0f0d41385fe89c8e7b5d43e576ebe62d9a09924a8c871ec24fa"}
Sep 17 23:29:33.415891 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:29:33.415880    2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-test-llm-inference-service-9c386aa9/scheduler-v06-pd-migration-test-kserve-prefill-8665687897-rpvrt"
Sep 17 23:29:33.416152 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:29:33.415918    2562 scope.go:117] "RemoveContainer" containerID="05023b8ffa449da3c06d75d0802fb9d95aedd51d6841308ea06e94cd7db4dbfc"
Sep 17 23:29:33.438090 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:29:33.438046    2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["e2e-test-llm-inference-service-9c386aa9/scheduler-v06-pd-migration-test-kserve-prefill-8665687897-rpvrt"]
Sep 17 23:29:33.441997 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:29:33.441953    2562 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["e2e-test-llm-inference-service-9c386aa9/scheduler-v06-pd-migration-test-kserve-prefill-8665687897-rpvrt"]
Sep 17 23:29:33.668417 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:29:33.668332    2562 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="9dd4e65c-5e0a-4175-8f86-1bd8d2a0e984" path="/var/lib/kubelet/pods/9dd4e65c-5e0a-4175-8f86-1bd8d2a0e984/volumes"
Sep 17 23:29:36.502962 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:29:36.502923    2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["e2e-test-gateway-section-name-propagation-94799d44/router-gateway-1-openshift-default-7dcbd489bd-ncsm7"]
Sep 17 23:29:36.503407 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:29:36.503133    2562 kuberuntime_container.go:873] "Killing container with a grace period" pod="e2e-test-gateway-section-name-propagation-94799d44/router-gateway-1-openshift-default-7dcbd489bd-ncsm7" podUID="27ae677b-9eab-404a-a9aa-a06767e259f9" containerName="istio-proxy" containerID="cri-o://7940d387d519414771302c325f5bdf144fcad0172eed97539363a4d8b6e81021" gracePeriod=30
Sep 17 23:29:36.598318 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:29:36.598283    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fwwc5_7e452bbb-7dc6-4c9d-a9b1-76400942f5b4/konnectivity-agent/0.log" limit=1024
Sep 17 23:29:39.449923 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:29:39.449889    2562 generic.go:356] "Generic (PLEG): container finished" podID="27ae677b-9eab-404a-a9aa-a06767e259f9" containerID="7940d387d519414771302c325f5bdf144fcad0172eed97539363a4d8b6e81021" exitCode=0
Sep 17 23:29:39.450345 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:29:39.449945    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-test-gateway-section-name-propagation-94799d44/router-gateway-1-openshift-default-7dcbd489bd-ncsm7" event={"ID":"27ae677b-9eab-404a-a9aa-a06767e259f9","Type":"ContainerDied","Data":"7940d387d519414771302c325f5bdf144fcad0172eed97539363a4d8b6e81021"}
Sep 17 23:29:39.548761 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:29:39.548733    2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-test-gateway-section-name-propagation-94799d44/router-gateway-1-openshift-default-7dcbd489bd-ncsm7"
Sep 17 23:29:39.733150 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:29:39.733118    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/27ae677b-9eab-404a-a9aa-a06767e259f9-istio-podinfo\") pod \"27ae677b-9eab-404a-a9aa-a06767e259f9\" (UID: \"27ae677b-9eab-404a-a9aa-a06767e259f9\") "
Sep 17 23:29:39.733362 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:29:39.733221    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/27ae677b-9eab-404a-a9aa-a06767e259f9-istio-token\") pod \"27ae677b-9eab-404a-a9aa-a06767e259f9\" (UID: \"27ae677b-9eab-404a-a9aa-a06767e259f9\") "
Sep 17 23:29:39.733362 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:29:39.733259    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/27ae677b-9eab-404a-a9aa-a06767e259f9-workload-socket\") pod \"27ae677b-9eab-404a-a9aa-a06767e259f9\" (UID: \"27ae677b-9eab-404a-a9aa-a06767e259f9\") "
Sep 17 23:29:39.733362 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:29:39.733282    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-2dk4d\" (UniqueName: \"kubernetes.io/projected/27ae677b-9eab-404a-a9aa-a06767e259f9-kube-api-access-2dk4d\") pod \"27ae677b-9eab-404a-a9aa-a06767e259f9\" (UID: \"27ae677b-9eab-404a-a9aa-a06767e259f9\") "
Sep 17 23:29:39.733362 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:29:39.733323    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/27ae677b-9eab-404a-a9aa-a06767e259f9-workload-certs\") pod \"27ae677b-9eab-404a-a9aa-a06767e259f9\" (UID: \"27ae677b-9eab-404a-a9aa-a06767e259f9\") "
Sep 17 23:29:39.733362 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:29:39.733345    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/27ae677b-9eab-404a-a9aa-a06767e259f9-credential-socket\") pod \"27ae677b-9eab-404a-a9aa-a06767e259f9\" (UID: \"27ae677b-9eab-404a-a9aa-a06767e259f9\") "
Sep 17 23:29:39.733635 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:29:39.733365    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/27ae677b-9eab-404a-a9aa-a06767e259f9-istiod-ca-cert\") pod \"27ae677b-9eab-404a-a9aa-a06767e259f9\" (UID: \"27ae677b-9eab-404a-a9aa-a06767e259f9\") "
Sep 17 23:29:39.733635 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:29:39.733409    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/27ae677b-9eab-404a-a9aa-a06767e259f9-istio-data\") pod \"27ae677b-9eab-404a-a9aa-a06767e259f9\" (UID: \"27ae677b-9eab-404a-a9aa-a06767e259f9\") "
Sep 17 23:29:39.733635 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:29:39.733440    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/27ae677b-9eab-404a-a9aa-a06767e259f9-istio-envoy\") pod \"27ae677b-9eab-404a-a9aa-a06767e259f9\" (UID: \"27ae677b-9eab-404a-a9aa-a06767e259f9\") "
Sep 17 23:29:39.733808 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:29:39.733689    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/27ae677b-9eab-404a-a9aa-a06767e259f9-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "27ae677b-9eab-404a-a9aa-a06767e259f9" (UID: "27ae677b-9eab-404a-a9aa-a06767e259f9"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 17 23:29:39.733808 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:29:39.733697    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/27ae677b-9eab-404a-a9aa-a06767e259f9-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "27ae677b-9eab-404a-a9aa-a06767e259f9" (UID: "27ae677b-9eab-404a-a9aa-a06767e259f9"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 17 23:29:39.733808 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:29:39.733799    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/27ae677b-9eab-404a-a9aa-a06767e259f9-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "27ae677b-9eab-404a-a9aa-a06767e259f9" (UID: "27ae677b-9eab-404a-a9aa-a06767e259f9"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 17 23:29:39.733980 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:29:39.733957    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/27ae677b-9eab-404a-a9aa-a06767e259f9-istio-data" (OuterVolumeSpecName: "istio-data") pod "27ae677b-9eab-404a-a9aa-a06767e259f9" (UID: "27ae677b-9eab-404a-a9aa-a06767e259f9"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 17 23:29:39.734042 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:29:39.733978    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/27ae677b-9eab-404a-a9aa-a06767e259f9-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "27ae677b-9eab-404a-a9aa-a06767e259f9" (UID: "27ae677b-9eab-404a-a9aa-a06767e259f9"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 17 23:29:39.735799 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:29:39.735763    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/27ae677b-9eab-404a-a9aa-a06767e259f9-istio-token" (OuterVolumeSpecName: "istio-token") pod "27ae677b-9eab-404a-a9aa-a06767e259f9" (UID: "27ae677b-9eab-404a-a9aa-a06767e259f9"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 17 23:29:39.735799 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:29:39.735778    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/27ae677b-9eab-404a-a9aa-a06767e259f9-kube-api-access-2dk4d" (OuterVolumeSpecName: "kube-api-access-2dk4d") pod "27ae677b-9eab-404a-a9aa-a06767e259f9" (UID: "27ae677b-9eab-404a-a9aa-a06767e259f9"). InnerVolumeSpecName "kube-api-access-2dk4d". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 17 23:29:39.735799 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:29:39.735771    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/27ae677b-9eab-404a-a9aa-a06767e259f9-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "27ae677b-9eab-404a-a9aa-a06767e259f9" (UID: "27ae677b-9eab-404a-a9aa-a06767e259f9"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Sep 17 23:29:39.736049 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:29:39.735848    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/27ae677b-9eab-404a-a9aa-a06767e259f9-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "27ae677b-9eab-404a-a9aa-a06767e259f9" (UID: "27ae677b-9eab-404a-a9aa-a06767e259f9"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 17 23:29:39.835035 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:29:39.834995    2562 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/27ae677b-9eab-404a-a9aa-a06767e259f9-istio-token\") on node \"ip-10-0-132-188.ec2.internal\" DevicePath \"\""
Sep 17 23:29:39.835035 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:29:39.835027    2562 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/27ae677b-9eab-404a-a9aa-a06767e259f9-workload-socket\") on node \"ip-10-0-132-188.ec2.internal\" DevicePath \"\""
Sep 17 23:29:39.835035 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:29:39.835037    2562 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-2dk4d\" (UniqueName: \"kubernetes.io/projected/27ae677b-9eab-404a-a9aa-a06767e259f9-kube-api-access-2dk4d\") on node \"ip-10-0-132-188.ec2.internal\" DevicePath \"\""
Sep 17 23:29:39.835035 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:29:39.835046    2562 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/27ae677b-9eab-404a-a9aa-a06767e259f9-workload-certs\") on node \"ip-10-0-132-188.ec2.internal\" DevicePath \"\""
Sep 17 23:29:39.835445 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:29:39.835056    2562 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/27ae677b-9eab-404a-a9aa-a06767e259f9-credential-socket\") on node \"ip-10-0-132-188.ec2.internal\" DevicePath \"\""
Sep 17 23:29:39.835445 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:29:39.835065    2562 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/27ae677b-9eab-404a-a9aa-a06767e259f9-istiod-ca-cert\") on node \"ip-10-0-132-188.ec2.internal\" DevicePath \"\""
Sep 17 23:29:39.835445 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:29:39.835074    2562 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/27ae677b-9eab-404a-a9aa-a06767e259f9-istio-data\") on node \"ip-10-0-132-188.ec2.internal\" DevicePath \"\""
Sep 17 23:29:39.835445 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:29:39.835082    2562 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/27ae677b-9eab-404a-a9aa-a06767e259f9-istio-envoy\") on node \"ip-10-0-132-188.ec2.internal\" DevicePath \"\""
Sep 17 23:29:39.835445 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:29:39.835090    2562 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/27ae677b-9eab-404a-a9aa-a06767e259f9-istio-podinfo\") on node \"ip-10-0-132-188.ec2.internal\" DevicePath \"\""
Sep 17 23:29:40.455990 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:29:40.455955    2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-test-gateway-section-name-propagation-94799d44/router-gateway-1-openshift-default-7dcbd489bd-ncsm7"
Sep 17 23:29:40.455990 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:29:40.455982    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-test-gateway-section-name-propagation-94799d44/router-gateway-1-openshift-default-7dcbd489bd-ncsm7" event={"ID":"27ae677b-9eab-404a-a9aa-a06767e259f9","Type":"ContainerDied","Data":"67318fbc523ba3d6017711db3513c4ea61b50ffc5542a7f9e8cff6f9b4bd6c33"}
Sep 17 23:29:40.456522 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:29:40.456044    2562 scope.go:117] "RemoveContainer" containerID="7940d387d519414771302c325f5bdf144fcad0172eed97539363a4d8b6e81021"
Sep 17 23:29:40.476955 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:29:40.476926    2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["e2e-test-gateway-section-name-propagation-94799d44/router-gateway-1-openshift-default-7dcbd489bd-ncsm7"]
Sep 17 23:29:40.482615 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:29:40.482588    2562 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["e2e-test-gateway-section-name-propagation-94799d44/router-gateway-1-openshift-default-7dcbd489bd-ncsm7"]
Sep 17 23:29:41.667335 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:29:41.667303    2562 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="27ae677b-9eab-404a-a9aa-a06767e259f9" path="/var/lib/kubelet/pods/27ae677b-9eab-404a-a9aa-a06767e259f9/volumes"
Sep 17 23:29:42.056476 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:29:42.056445    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fwwc5_7e452bbb-7dc6-4c9d-a9b1-76400942f5b4/konnectivity-agent/0.log" limit=1024
Sep 17 23:29:42.244343 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:29:42.244307    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fwwc5_7e452bbb-7dc6-4c9d-a9b1-76400942f5b4/konnectivity-agent/0.log" limit=1024
Sep 17 23:29:48.715385 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:29:48.715354    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fwwc5_7e452bbb-7dc6-4c9d-a9b1-76400942f5b4/konnectivity-agent/0.log" limit=1024
Sep 17 23:29:56.320409 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:29:56.320368    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fwwc5_7e452bbb-7dc6-4c9d-a9b1-76400942f5b4/konnectivity-agent/0.log" limit=1024
Sep 17 23:29:56.616481 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:29:56.616437    2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["e2e-test-llm-auth-enabled-requires-token-d4ed2c08/auth-enabled-test-kserve-router-scheduler-547fb7d6bb-rvcrg"]
Sep 17 23:29:56.616853 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:29:56.616834    2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="9dd4e65c-5e0a-4175-8f86-1bd8d2a0e984" containerName="main"
Sep 17 23:29:56.616950 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:29:56.616857    2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="9dd4e65c-5e0a-4175-8f86-1bd8d2a0e984" containerName="main"
Sep 17 23:29:56.616950 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:29:56.616876    2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a0fd1118-d360-4f54-b3e9-f62977cea0aa" containerName="istio-proxy"
Sep 17 23:29:56.616950 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:29:56.616884    2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="a0fd1118-d360-4f54-b3e9-f62977cea0aa" containerName="istio-proxy"
Sep 17 23:29:56.616950 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:29:56.616900    2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="27ae677b-9eab-404a-a9aa-a06767e259f9" containerName="istio-proxy"
Sep 17 23:29:56.616950 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:29:56.616909    2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="27ae677b-9eab-404a-a9aa-a06767e259f9" containerName="istio-proxy"
Sep 17 23:29:56.617217 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:29:56.617001    2562 memory_manager.go:368] "RemoveStaleState removing state" podUID="27ae677b-9eab-404a-a9aa-a06767e259f9" containerName="istio-proxy"
Sep 17 23:29:56.617217 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:29:56.617012    2562 memory_manager.go:368] "RemoveStaleState removing state" podUID="a0fd1118-d360-4f54-b3e9-f62977cea0aa" containerName="istio-proxy"
Sep 17 23:29:56.617217 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:29:56.617021    2562 memory_manager.go:368] "RemoveStaleState removing state" podUID="9dd4e65c-5e0a-4175-8f86-1bd8d2a0e984" containerName="main"
Sep 17 23:29:56.620464 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:29:56.620428    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-test-llm-auth-enabled-requires-token-d4ed2c08/auth-enabled-test-kserve-router-scheduler-547fb7d6bb-rvcrg"
Sep 17 23:29:56.624272 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:29:56.624242    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-test-llm-auth-enabled-requires-token-d4ed2c08\"/\"default-dockercfg-b2xz5\""
Sep 17 23:29:56.624521 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:29:56.624261    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-test-llm-auth-enabled-requires-token-d4ed2c08\"/\"openshift-service-ca.crt\""
Sep 17 23:29:56.624657 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:29:56.624553    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-test-llm-auth-enabled-requires-token-d4ed2c08\"/\"auth-enabled-test-kserve-self-signed-certs\""
Sep 17 23:29:56.624657 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:29:56.624283    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-test-llm-auth-enabled-requires-token-d4ed2c08\"/\"auth-enabled-test-epp-sa-dockercfg-rnxpv\""
Sep 17 23:29:56.624816 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:29:56.624801    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-test-llm-auth-enabled-requires-token-d4ed2c08\"/\"kube-root-ca.crt\""
Sep 17 23:29:56.630528 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:29:56.630504    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["e2e-test-llm-auth-enabled-requires-token-d4ed2c08/auth-enabled-test-kserve-router-scheduler-547fb7d6bb-rvcrg"]
Sep 17 23:29:56.676677 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:29:56.676634    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/b68cebe3-9f55-4358-8cf9-b02b2e975bb6-tls-certs\") pod \"auth-enabled-test-kserve-router-scheduler-547fb7d6bb-rvcrg\" (UID: \"b68cebe3-9f55-4358-8cf9-b02b2e975bb6\") " pod="e2e-test-llm-auth-enabled-requires-token-d4ed2c08/auth-enabled-test-kserve-router-scheduler-547fb7d6bb-rvcrg"
Sep 17 23:29:56.676979 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:29:56.676940    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-r4vxf\" (UniqueName: \"kubernetes.io/projected/b68cebe3-9f55-4358-8cf9-b02b2e975bb6-kube-api-access-r4vxf\") pod \"auth-enabled-test-kserve-router-scheduler-547fb7d6bb-rvcrg\" (UID: \"b68cebe3-9f55-4358-8cf9-b02b2e975bb6\") " pod="e2e-test-llm-auth-enabled-requires-token-d4ed2c08/auth-enabled-test-kserve-router-scheduler-547fb7d6bb-rvcrg"
Sep 17 23:29:56.778066 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:29:56.778021    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-r4vxf\" (UniqueName: \"kubernetes.io/projected/b68cebe3-9f55-4358-8cf9-b02b2e975bb6-kube-api-access-r4vxf\") pod \"auth-enabled-test-kserve-router-scheduler-547fb7d6bb-rvcrg\" (UID: \"b68cebe3-9f55-4358-8cf9-b02b2e975bb6\") " pod="e2e-test-llm-auth-enabled-requires-token-d4ed2c08/auth-enabled-test-kserve-router-scheduler-547fb7d6bb-rvcrg"
Sep 17 23:29:56.778320 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:29:56.778080    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/b68cebe3-9f55-4358-8cf9-b02b2e975bb6-tls-certs\") pod \"auth-enabled-test-kserve-router-scheduler-547fb7d6bb-rvcrg\" (UID: \"b68cebe3-9f55-4358-8cf9-b02b2e975bb6\") " pod="e2e-test-llm-auth-enabled-requires-token-d4ed2c08/auth-enabled-test-kserve-router-scheduler-547fb7d6bb-rvcrg"
Sep 17 23:29:56.780839 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:29:56.780806    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/b68cebe3-9f55-4358-8cf9-b02b2e975bb6-tls-certs\") pod \"auth-enabled-test-kserve-router-scheduler-547fb7d6bb-rvcrg\" (UID: \"b68cebe3-9f55-4358-8cf9-b02b2e975bb6\") " pod="e2e-test-llm-auth-enabled-requires-token-d4ed2c08/auth-enabled-test-kserve-router-scheduler-547fb7d6bb-rvcrg"
Sep 17 23:29:56.789560 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:29:56.789530    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-r4vxf\" (UniqueName: \"kubernetes.io/projected/b68cebe3-9f55-4358-8cf9-b02b2e975bb6-kube-api-access-r4vxf\") pod \"auth-enabled-test-kserve-router-scheduler-547fb7d6bb-rvcrg\" (UID: \"b68cebe3-9f55-4358-8cf9-b02b2e975bb6\") " pod="e2e-test-llm-auth-enabled-requires-token-d4ed2c08/auth-enabled-test-kserve-router-scheduler-547fb7d6bb-rvcrg"
Sep 17 23:29:56.933545 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:29:56.933455    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-test-llm-auth-enabled-requires-token-d4ed2c08/auth-enabled-test-kserve-router-scheduler-547fb7d6bb-rvcrg"
Sep 17 23:29:57.207055 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:29:57.206843    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["e2e-test-llm-auth-enabled-requires-token-d4ed2c08/auth-enabled-test-kserve-router-scheduler-547fb7d6bb-rvcrg"]
Sep 17 23:29:57.209913 ip-10-0-132-188 kubenswrapper[2562]: W0917 23:29:57.209879    2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb68cebe3_9f55_4358_8cf9_b02b2e975bb6.slice/crio-7dd222a2ca5bc7606eef8cb2917c963e789466f91d4abac43aa7e564a7c1f0ab WatchSource:0}: Error finding container 7dd222a2ca5bc7606eef8cb2917c963e789466f91d4abac43aa7e564a7c1f0ab: Status 404 returned error can't find the container with id 7dd222a2ca5bc7606eef8cb2917c963e789466f91d4abac43aa7e564a7c1f0ab
Sep 17 23:29:57.448098 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:29:57.448041    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fwwc5_7e452bbb-7dc6-4c9d-a9b1-76400942f5b4/konnectivity-agent/0.log" limit=1024
Sep 17 23:29:57.537309 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:29:57.537202    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-test-llm-auth-enabled-requires-token-d4ed2c08/auth-enabled-test-kserve-router-scheduler-547fb7d6bb-rvcrg" event={"ID":"b68cebe3-9f55-4358-8cf9-b02b2e975bb6","Type":"ContainerStarted","Data":"5211dda4a7c0d54a7dd0470650a8ee45848482bc9fdc7f0d803e139aa34ab329"}
Sep 17 23:29:57.537497 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:29:57.537311    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-test-llm-auth-enabled-requires-token-d4ed2c08/auth-enabled-test-kserve-router-scheduler-547fb7d6bb-rvcrg" event={"ID":"b68cebe3-9f55-4358-8cf9-b02b2e975bb6","Type":"ContainerStarted","Data":"7dd222a2ca5bc7606eef8cb2917c963e789466f91d4abac43aa7e564a7c1f0ab"}
Sep 17 23:29:57.537581 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:29:57.537546    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-test-llm-auth-enabled-requires-token-d4ed2c08/auth-enabled-test-kserve-router-scheduler-547fb7d6bb-rvcrg"
Sep 17 23:29:57.556730 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:29:57.556675    2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="e2e-test-llm-auth-enabled-requires-token-d4ed2c08/auth-enabled-test-kserve-router-scheduler-547fb7d6bb-rvcrg" podStartSLOduration=1.5566579539999998 podStartE2EDuration="1.556657954s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-17 23:29:56 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-17 23:29:57.555433264 +0000 UTC m=+1149.957330029" watchObservedRunningTime="2026-09-17 23:29:57.556657954 +0000 UTC m=+1149.958554717"
Sep 17 23:30:04.633450 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:30:04.633351    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fwwc5_7e452bbb-7dc6-4c9d-a9b1-76400942f5b4/konnectivity-agent/0.log" limit=1024
Sep 17 23:30:21.943992 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:30:21.943951    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fwwc5_7e452bbb-7dc6-4c9d-a9b1-76400942f5b4/konnectivity-agent/0.log" limit=1024
Sep 17 23:30:22.036705 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:30:22.036660    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fwwc5_7e452bbb-7dc6-4c9d-a9b1-76400942f5b4/konnectivity-agent/0.log" limit=1024
Sep 17 23:30:27.797016 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:30:27.796975    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fwwc5_7e452bbb-7dc6-4c9d-a9b1-76400942f5b4/konnectivity-agent/0.log" limit=1024
Sep 17 23:30:28.544362 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:30:28.544323    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-test-llm-auth-enabled-requires-token-d4ed2c08/auth-enabled-test-kserve-router-scheduler-547fb7d6bb-rvcrg"
Sep 17 23:30:33.519531 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:30:33.519493    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fwwc5_7e452bbb-7dc6-4c9d-a9b1-76400942f5b4/konnectivity-agent/0.log" limit=1024
Sep 17 23:30:33.576154 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:30:33.576118    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fwwc5_7e452bbb-7dc6-4c9d-a9b1-76400942f5b4/konnectivity-agent/0.log" limit=1024
Sep 17 23:30:38.543471 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:30:38.543437    2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["e2e-test-llm-inference-service-40b85f95/scheduler-v06-threshold-migration-test-kserve-prefill-6444tpqlc"]
Sep 17 23:30:38.546743 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:30:38.546719    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-test-llm-inference-service-40b85f95/scheduler-v06-threshold-migration-test-kserve-prefill-6444tpqlc"
Sep 17 23:30:38.549286 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:30:38.549258    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-test-llm-inference-service-40b85f95\"/\"openshift-service-ca.crt\""
Sep 17 23:30:38.550299 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:30:38.550276    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-test-llm-inference-service-40b85f95\"/\"kube-root-ca.crt\""
Sep 17 23:30:38.550434 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:30:38.550320    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-test-llm-inference-service-40b85f95\"/\"default-dockercfg-8frb9\""
Sep 17 23:30:38.550434 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:30:38.550362    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-test-llm-inference-service-40b85f95\"/\"scheduler-v06-threshold-migration-test-kserve-self-signed-certs\""
Sep 17 23:30:38.555988 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:30:38.555963    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["e2e-test-llm-inference-service-40b85f95/scheduler-v06-threshold-migration-test-kserve-prefill-6444tpqlc"]
Sep 17 23:30:38.577568 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:30:38.577533    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fwwc5_7e452bbb-7dc6-4c9d-a9b1-76400942f5b4/konnectivity-agent/0.log" limit=1024
Sep 17 23:30:38.637620 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:30:38.637581    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/eda1df77-551e-46ca-afb3-8da2bb765d32-tmp-dir\") pod \"scheduler-v06-threshold-migration-test-kserve-prefill-6444tpqlc\" (UID: \"eda1df77-551e-46ca-afb3-8da2bb765d32\") " pod="e2e-test-llm-inference-service-40b85f95/scheduler-v06-threshold-migration-test-kserve-prefill-6444tpqlc"
Sep 17 23:30:38.637814 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:30:38.637628    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/eda1df77-551e-46ca-afb3-8da2bb765d32-tls-certs\") pod \"scheduler-v06-threshold-migration-test-kserve-prefill-6444tpqlc\" (UID: \"eda1df77-551e-46ca-afb3-8da2bb765d32\") " pod="e2e-test-llm-inference-service-40b85f95/scheduler-v06-threshold-migration-test-kserve-prefill-6444tpqlc"
Sep 17 23:30:38.637814 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:30:38.637673    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/eda1df77-551e-46ca-afb3-8da2bb765d32-model-cache\") pod \"scheduler-v06-threshold-migration-test-kserve-prefill-6444tpqlc\" (UID: \"eda1df77-551e-46ca-afb3-8da2bb765d32\") " pod="e2e-test-llm-inference-service-40b85f95/scheduler-v06-threshold-migration-test-kserve-prefill-6444tpqlc"
Sep 17 23:30:38.637814 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:30:38.637783    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/eda1df77-551e-46ca-afb3-8da2bb765d32-dshm\") pod \"scheduler-v06-threshold-migration-test-kserve-prefill-6444tpqlc\" (UID: \"eda1df77-551e-46ca-afb3-8da2bb765d32\") " pod="e2e-test-llm-inference-service-40b85f95/scheduler-v06-threshold-migration-test-kserve-prefill-6444tpqlc"
Sep 17 23:30:38.637933 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:30:38.637820    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-n2dg8\" (UniqueName: \"kubernetes.io/projected/eda1df77-551e-46ca-afb3-8da2bb765d32-kube-api-access-n2dg8\") pod \"scheduler-v06-threshold-migration-test-kserve-prefill-6444tpqlc\" (UID: \"eda1df77-551e-46ca-afb3-8da2bb765d32\") " pod="e2e-test-llm-inference-service-40b85f95/scheduler-v06-threshold-migration-test-kserve-prefill-6444tpqlc"
Sep 17 23:30:38.637933 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:30:38.637885    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/eda1df77-551e-46ca-afb3-8da2bb765d32-home\") pod \"scheduler-v06-threshold-migration-test-kserve-prefill-6444tpqlc\" (UID: \"eda1df77-551e-46ca-afb3-8da2bb765d32\") " pod="e2e-test-llm-inference-service-40b85f95/scheduler-v06-threshold-migration-test-kserve-prefill-6444tpqlc"
Sep 17 23:30:38.738636 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:30:38.738599    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/eda1df77-551e-46ca-afb3-8da2bb765d32-model-cache\") pod \"scheduler-v06-threshold-migration-test-kserve-prefill-6444tpqlc\" (UID: \"eda1df77-551e-46ca-afb3-8da2bb765d32\") " pod="e2e-test-llm-inference-service-40b85f95/scheduler-v06-threshold-migration-test-kserve-prefill-6444tpqlc"
Sep 17 23:30:38.738830 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:30:38.738665    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/eda1df77-551e-46ca-afb3-8da2bb765d32-dshm\") pod \"scheduler-v06-threshold-migration-test-kserve-prefill-6444tpqlc\" (UID: \"eda1df77-551e-46ca-afb3-8da2bb765d32\") " pod="e2e-test-llm-inference-service-40b85f95/scheduler-v06-threshold-migration-test-kserve-prefill-6444tpqlc"
Sep 17 23:30:38.738830 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:30:38.738699    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-n2dg8\" (UniqueName: \"kubernetes.io/projected/eda1df77-551e-46ca-afb3-8da2bb765d32-kube-api-access-n2dg8\") pod \"scheduler-v06-threshold-migration-test-kserve-prefill-6444tpqlc\" (UID: \"eda1df77-551e-46ca-afb3-8da2bb765d32\") " pod="e2e-test-llm-inference-service-40b85f95/scheduler-v06-threshold-migration-test-kserve-prefill-6444tpqlc"
Sep 17 23:30:38.738830 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:30:38.738733    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/eda1df77-551e-46ca-afb3-8da2bb765d32-home\") pod \"scheduler-v06-threshold-migration-test-kserve-prefill-6444tpqlc\" (UID: \"eda1df77-551e-46ca-afb3-8da2bb765d32\") " pod="e2e-test-llm-inference-service-40b85f95/scheduler-v06-threshold-migration-test-kserve-prefill-6444tpqlc"
Sep 17 23:30:38.738830 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:30:38.738779    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/eda1df77-551e-46ca-afb3-8da2bb765d32-tmp-dir\") pod \"scheduler-v06-threshold-migration-test-kserve-prefill-6444tpqlc\" (UID: \"eda1df77-551e-46ca-afb3-8da2bb765d32\") " pod="e2e-test-llm-inference-service-40b85f95/scheduler-v06-threshold-migration-test-kserve-prefill-6444tpqlc"
Sep 17 23:30:38.738830 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:30:38.738815    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/eda1df77-551e-46ca-afb3-8da2bb765d32-tls-certs\") pod \"scheduler-v06-threshold-migration-test-kserve-prefill-6444tpqlc\" (UID: \"eda1df77-551e-46ca-afb3-8da2bb765d32\") " pod="e2e-test-llm-inference-service-40b85f95/scheduler-v06-threshold-migration-test-kserve-prefill-6444tpqlc"
Sep 17 23:30:38.739146 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:30:38.739125    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/eda1df77-551e-46ca-afb3-8da2bb765d32-model-cache\") pod \"scheduler-v06-threshold-migration-test-kserve-prefill-6444tpqlc\" (UID: \"eda1df77-551e-46ca-afb3-8da2bb765d32\") " pod="e2e-test-llm-inference-service-40b85f95/scheduler-v06-threshold-migration-test-kserve-prefill-6444tpqlc"
Sep 17 23:30:38.739223 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:30:38.739203    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/eda1df77-551e-46ca-afb3-8da2bb765d32-tmp-dir\") pod \"scheduler-v06-threshold-migration-test-kserve-prefill-6444tpqlc\" (UID: \"eda1df77-551e-46ca-afb3-8da2bb765d32\") " pod="e2e-test-llm-inference-service-40b85f95/scheduler-v06-threshold-migration-test-kserve-prefill-6444tpqlc"
Sep 17 23:30:38.739271 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:30:38.739199    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/eda1df77-551e-46ca-afb3-8da2bb765d32-home\") pod \"scheduler-v06-threshold-migration-test-kserve-prefill-6444tpqlc\" (UID: \"eda1df77-551e-46ca-afb3-8da2bb765d32\") " pod="e2e-test-llm-inference-service-40b85f95/scheduler-v06-threshold-migration-test-kserve-prefill-6444tpqlc"
Sep 17 23:30:38.741257 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:30:38.741164    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/eda1df77-551e-46ca-afb3-8da2bb765d32-dshm\") pod \"scheduler-v06-threshold-migration-test-kserve-prefill-6444tpqlc\" (UID: \"eda1df77-551e-46ca-afb3-8da2bb765d32\") " pod="e2e-test-llm-inference-service-40b85f95/scheduler-v06-threshold-migration-test-kserve-prefill-6444tpqlc"
Sep 17 23:30:38.741257 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:30:38.741694    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/eda1df77-551e-46ca-afb3-8da2bb765d32-tls-certs\") pod \"scheduler-v06-threshold-migration-test-kserve-prefill-6444tpqlc\" (UID: \"eda1df77-551e-46ca-afb3-8da2bb765d32\") " pod="e2e-test-llm-inference-service-40b85f95/scheduler-v06-threshold-migration-test-kserve-prefill-6444tpqlc"
Sep 17 23:30:38.755595 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:30:38.755565    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-n2dg8\" (UniqueName: \"kubernetes.io/projected/eda1df77-551e-46ca-afb3-8da2bb765d32-kube-api-access-n2dg8\") pod \"scheduler-v06-threshold-migration-test-kserve-prefill-6444tpqlc\" (UID: \"eda1df77-551e-46ca-afb3-8da2bb765d32\") " pod="e2e-test-llm-inference-service-40b85f95/scheduler-v06-threshold-migration-test-kserve-prefill-6444tpqlc"
Sep 17 23:30:38.788914 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:30:38.788882    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fwwc5_7e452bbb-7dc6-4c9d-a9b1-76400942f5b4/konnectivity-agent/0.log" limit=1024
Sep 17 23:30:38.859936 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:30:38.859895    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-test-llm-inference-service-40b85f95/scheduler-v06-threshold-migration-test-kserve-prefill-6444tpqlc"
Sep 17 23:30:39.139751 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:30:39.139718    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["e2e-test-llm-inference-service-40b85f95/scheduler-v06-threshold-migration-test-kserve-prefill-6444tpqlc"]
Sep 17 23:30:39.141662 ip-10-0-132-188 kubenswrapper[2562]: W0917 23:30:39.141631    2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podeda1df77_551e_46ca_afb3_8da2bb765d32.slice/crio-e8214c5c79c247b52b49ddac48bff269e6bf90de67953ccbd88650d0f60b8974 WatchSource:0}: Error finding container e8214c5c79c247b52b49ddac48bff269e6bf90de67953ccbd88650d0f60b8974: Status 404 returned error can't find the container with id e8214c5c79c247b52b49ddac48bff269e6bf90de67953ccbd88650d0f60b8974
Sep 17 23:30:39.736646 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:30:39.736606    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-test-llm-inference-service-40b85f95/scheduler-v06-threshold-migration-test-kserve-prefill-6444tpqlc" event={"ID":"eda1df77-551e-46ca-afb3-8da2bb765d32","Type":"ContainerStarted","Data":"71755eaabc1ddff1cfeddb3be6a6b2cf7c1ce0f4a7bfe8f1b4038ce39bbc2e0e"}
Sep 17 23:30:39.737110 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:30:39.736665    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-test-llm-inference-service-40b85f95/scheduler-v06-threshold-migration-test-kserve-prefill-6444tpqlc" event={"ID":"eda1df77-551e-46ca-afb3-8da2bb765d32","Type":"ContainerStarted","Data":"e8214c5c79c247b52b49ddac48bff269e6bf90de67953ccbd88650d0f60b8974"}
Sep 17 23:30:39.755261 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:30:39.755205    2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="e2e-test-llm-inference-service-40b85f95/scheduler-v06-threshold-migration-test-kserve-prefill-6444tpqlc" podStartSLOduration=1.755189892 podStartE2EDuration="1.755189892s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-17 23:30:38 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-17 23:30:39.753648642 +0000 UTC m=+1192.155545405" watchObservedRunningTime="2026-09-17 23:30:39.755189892 +0000 UTC m=+1192.157086683"
Sep 17 23:30:39.860668 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:30:39.860625    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-test-llm-inference-service-40b85f95/scheduler-v06-threshold-migration-test-kserve-prefill-6444tpqlc"
Sep 17 23:30:48.860944 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:30:48.860907    2562 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="e2e-test-llm-inference-service-40b85f95/scheduler-v06-threshold-migration-test-kserve-prefill-6444tpqlc"
Sep 17 23:30:48.873516 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:30:48.873482    2562 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="e2e-test-llm-inference-service-40b85f95/scheduler-v06-threshold-migration-test-kserve-prefill-6444tpqlc"
Sep 17 23:30:49.796037 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:30:49.796001    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-test-llm-inference-service-40b85f95/scheduler-v06-threshold-migration-test-kserve-prefill-6444tpqlc"
Sep 17 23:30:52.493391 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:30:52.493336    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fwwc5_7e452bbb-7dc6-4c9d-a9b1-76400942f5b4/konnectivity-agent/0.log" limit=1024
Sep 17 23:31:14.864244 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:31:14.864210    2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["e2e-test-llm-inference-service-40b85f95/scheduler-v06-threshold-migration-test-kserve-prefill-6444tpqlc"]
Sep 17 23:31:14.864849 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:31:14.864636    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fwwc5_7e452bbb-7dc6-4c9d-a9b1-76400942f5b4/konnectivity-agent/0.log" limit=1024
Sep 17 23:31:14.874097 ip-10-0-132-188 kubenswrapper[2562]: E0917 23:31:14.874057    2562 log.go:32] "ExecSync cmd from runtime service failed" err=<
Sep 17 23:31:14.874097 ip-10-0-132-188 kubenswrapper[2562]:         rpc error: code = Unknown desc = command error: executable file `/bin/sleep` not found: No such file or directory
Sep 17 23:31:14.874097 ip-10-0-132-188 kubenswrapper[2562]:         , stdout: , stderr: , exit code -1
Sep 17 23:31:14.874097 ip-10-0-132-188 kubenswrapper[2562]:  > containerID="71755eaabc1ddff1cfeddb3be6a6b2cf7c1ce0f4a7bfe8f1b4038ce39bbc2e0e" cmd=["/bin/sleep","15"]
Sep 17 23:31:14.874363 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:31:14.874125    2562 handlers.go:82] "Exec lifecycle hook for Container in Pod failed" execCommand=["/bin/sleep","15"] containerName="main" pod="e2e-test-llm-inference-service-40b85f95/scheduler-v06-threshold-migration-test-kserve-prefill-6444tpqlc" message="" err=<
Sep 17 23:31:14.874363 ip-10-0-132-188 kubenswrapper[2562]:         rpc error: code = Unknown desc = command error: executable file `/bin/sleep` not found: No such file or directory
Sep 17 23:31:14.874363 ip-10-0-132-188 kubenswrapper[2562]:         , stdout: , stderr: , exit code -1
Sep 17 23:31:14.874363 ip-10-0-132-188 kubenswrapper[2562]:  >
Sep 17 23:31:14.874363 ip-10-0-132-188 kubenswrapper[2562]: E0917 23:31:14.874144    2562 kuberuntime_container.go:755] "PreStop hook failed" err=<
Sep 17 23:31:14.874363 ip-10-0-132-188 kubenswrapper[2562]:         rpc error: code = Unknown desc = command error: executable file `/bin/sleep` not found: No such file or directory
Sep 17 23:31:14.874363 ip-10-0-132-188 kubenswrapper[2562]:         , stdout: , stderr: , exit code -1
Sep 17 23:31:14.874363 ip-10-0-132-188 kubenswrapper[2562]:  > pod="e2e-test-llm-inference-service-40b85f95/scheduler-v06-threshold-migration-test-kserve-prefill-6444tpqlc" podUID="eda1df77-551e-46ca-afb3-8da2bb765d32" containerName="main" containerID="cri-o://71755eaabc1ddff1cfeddb3be6a6b2cf7c1ce0f4a7bfe8f1b4038ce39bbc2e0e"
Sep 17 23:31:14.874363 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:31:14.874235    2562 kuberuntime_container.go:873] "Killing container with a grace period" pod="e2e-test-llm-inference-service-40b85f95/scheduler-v06-threshold-migration-test-kserve-prefill-6444tpqlc" podUID="eda1df77-551e-46ca-afb3-8da2bb765d32" containerName="main" containerID="cri-o://71755eaabc1ddff1cfeddb3be6a6b2cf7c1ce0f4a7bfe8f1b4038ce39bbc2e0e" gracePeriod=60
Sep 17 23:31:14.926871 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:31:14.926836    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fwwc5_7e452bbb-7dc6-4c9d-a9b1-76400942f5b4/konnectivity-agent/0.log" limit=1024
Sep 17 23:31:15.240148 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:31:15.240121    2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-test-llm-inference-service-40b85f95/scheduler-v06-threshold-migration-test-kserve-prefill-6444tpqlc"
Sep 17 23:31:15.372151 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:31:15.372114    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-n2dg8\" (UniqueName: \"kubernetes.io/projected/eda1df77-551e-46ca-afb3-8da2bb765d32-kube-api-access-n2dg8\") pod \"eda1df77-551e-46ca-afb3-8da2bb765d32\" (UID: \"eda1df77-551e-46ca-afb3-8da2bb765d32\") "
Sep 17 23:31:15.372367 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:31:15.372214    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/eda1df77-551e-46ca-afb3-8da2bb765d32-model-cache\") pod \"eda1df77-551e-46ca-afb3-8da2bb765d32\" (UID: \"eda1df77-551e-46ca-afb3-8da2bb765d32\") "
Sep 17 23:31:15.372367 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:31:15.372262    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/eda1df77-551e-46ca-afb3-8da2bb765d32-home\") pod \"eda1df77-551e-46ca-afb3-8da2bb765d32\" (UID: \"eda1df77-551e-46ca-afb3-8da2bb765d32\") "
Sep 17 23:31:15.372367 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:31:15.372321    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/eda1df77-551e-46ca-afb3-8da2bb765d32-tmp-dir\") pod \"eda1df77-551e-46ca-afb3-8da2bb765d32\" (UID: \"eda1df77-551e-46ca-afb3-8da2bb765d32\") "
Sep 17 23:31:15.372367 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:31:15.372346    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/eda1df77-551e-46ca-afb3-8da2bb765d32-dshm\") pod \"eda1df77-551e-46ca-afb3-8da2bb765d32\" (UID: \"eda1df77-551e-46ca-afb3-8da2bb765d32\") "
Sep 17 23:31:15.372591 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:31:15.372405    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/eda1df77-551e-46ca-afb3-8da2bb765d32-tls-certs\") pod \"eda1df77-551e-46ca-afb3-8da2bb765d32\" (UID: \"eda1df77-551e-46ca-afb3-8da2bb765d32\") "
Sep 17 23:31:15.372591 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:31:15.372526    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/eda1df77-551e-46ca-afb3-8da2bb765d32-model-cache" (OuterVolumeSpecName: "model-cache") pod "eda1df77-551e-46ca-afb3-8da2bb765d32" (UID: "eda1df77-551e-46ca-afb3-8da2bb765d32"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 17 23:31:15.372723 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:31:15.372698    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/eda1df77-551e-46ca-afb3-8da2bb765d32-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "eda1df77-551e-46ca-afb3-8da2bb765d32" (UID: "eda1df77-551e-46ca-afb3-8da2bb765d32"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 17 23:31:15.372785 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:31:15.372711    2562 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/eda1df77-551e-46ca-afb3-8da2bb765d32-model-cache\") on node \"ip-10-0-132-188.ec2.internal\" DevicePath \"\""
Sep 17 23:31:15.372861 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:31:15.372837    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/eda1df77-551e-46ca-afb3-8da2bb765d32-home" (OuterVolumeSpecName: "home") pod "eda1df77-551e-46ca-afb3-8da2bb765d32" (UID: "eda1df77-551e-46ca-afb3-8da2bb765d32"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 17 23:31:15.374792 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:31:15.374755    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/eda1df77-551e-46ca-afb3-8da2bb765d32-dshm" (OuterVolumeSpecName: "dshm") pod "eda1df77-551e-46ca-afb3-8da2bb765d32" (UID: "eda1df77-551e-46ca-afb3-8da2bb765d32"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 17 23:31:15.374910 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:31:15.374848    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/eda1df77-551e-46ca-afb3-8da2bb765d32-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "eda1df77-551e-46ca-afb3-8da2bb765d32" (UID: "eda1df77-551e-46ca-afb3-8da2bb765d32"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 17 23:31:15.374910 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:31:15.374891    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/eda1df77-551e-46ca-afb3-8da2bb765d32-kube-api-access-n2dg8" (OuterVolumeSpecName: "kube-api-access-n2dg8") pod "eda1df77-551e-46ca-afb3-8da2bb765d32" (UID: "eda1df77-551e-46ca-afb3-8da2bb765d32"). InnerVolumeSpecName "kube-api-access-n2dg8". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 17 23:31:15.474272 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:31:15.474160    2562 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-n2dg8\" (UniqueName: \"kubernetes.io/projected/eda1df77-551e-46ca-afb3-8da2bb765d32-kube-api-access-n2dg8\") on node \"ip-10-0-132-188.ec2.internal\" DevicePath \"\""
Sep 17 23:31:15.474272 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:31:15.474213    2562 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/eda1df77-551e-46ca-afb3-8da2bb765d32-home\") on node \"ip-10-0-132-188.ec2.internal\" DevicePath \"\""
Sep 17 23:31:15.474272 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:31:15.474223    2562 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/eda1df77-551e-46ca-afb3-8da2bb765d32-tmp-dir\") on node \"ip-10-0-132-188.ec2.internal\" DevicePath \"\""
Sep 17 23:31:15.474272 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:31:15.474235    2562 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/eda1df77-551e-46ca-afb3-8da2bb765d32-dshm\") on node \"ip-10-0-132-188.ec2.internal\" DevicePath \"\""
Sep 17 23:31:15.474272 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:31:15.474247    2562 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/eda1df77-551e-46ca-afb3-8da2bb765d32-tls-certs\") on node \"ip-10-0-132-188.ec2.internal\" DevicePath \"\""
Sep 17 23:31:15.910991 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:31:15.910954    2562 generic.go:356] "Generic (PLEG): container finished" podID="eda1df77-551e-46ca-afb3-8da2bb765d32" containerID="71755eaabc1ddff1cfeddb3be6a6b2cf7c1ce0f4a7bfe8f1b4038ce39bbc2e0e" exitCode=0
Sep 17 23:31:15.910991 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:31:15.910992    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-test-llm-inference-service-40b85f95/scheduler-v06-threshold-migration-test-kserve-prefill-6444tpqlc" event={"ID":"eda1df77-551e-46ca-afb3-8da2bb765d32","Type":"ContainerDied","Data":"71755eaabc1ddff1cfeddb3be6a6b2cf7c1ce0f4a7bfe8f1b4038ce39bbc2e0e"}
Sep 17 23:31:15.911592 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:31:15.911020    2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-test-llm-inference-service-40b85f95/scheduler-v06-threshold-migration-test-kserve-prefill-6444tpqlc"
Sep 17 23:31:15.911592 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:31:15.911045    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-test-llm-inference-service-40b85f95/scheduler-v06-threshold-migration-test-kserve-prefill-6444tpqlc" event={"ID":"eda1df77-551e-46ca-afb3-8da2bb765d32","Type":"ContainerDied","Data":"e8214c5c79c247b52b49ddac48bff269e6bf90de67953ccbd88650d0f60b8974"}
Sep 17 23:31:15.911592 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:31:15.911069    2562 scope.go:117] "RemoveContainer" containerID="71755eaabc1ddff1cfeddb3be6a6b2cf7c1ce0f4a7bfe8f1b4038ce39bbc2e0e"
Sep 17 23:31:15.919956 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:31:15.919939    2562 scope.go:117] "RemoveContainer" containerID="71755eaabc1ddff1cfeddb3be6a6b2cf7c1ce0f4a7bfe8f1b4038ce39bbc2e0e"
Sep 17 23:31:15.920275 ip-10-0-132-188 kubenswrapper[2562]: E0917 23:31:15.920245    2562 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"71755eaabc1ddff1cfeddb3be6a6b2cf7c1ce0f4a7bfe8f1b4038ce39bbc2e0e\": container with ID starting with 71755eaabc1ddff1cfeddb3be6a6b2cf7c1ce0f4a7bfe8f1b4038ce39bbc2e0e not found: ID does not exist" containerID="71755eaabc1ddff1cfeddb3be6a6b2cf7c1ce0f4a7bfe8f1b4038ce39bbc2e0e"
Sep 17 23:31:15.920330 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:31:15.920282    2562 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"71755eaabc1ddff1cfeddb3be6a6b2cf7c1ce0f4a7bfe8f1b4038ce39bbc2e0e"} err="failed to get container status \"71755eaabc1ddff1cfeddb3be6a6b2cf7c1ce0f4a7bfe8f1b4038ce39bbc2e0e\": rpc error: code = NotFound desc = could not find container \"71755eaabc1ddff1cfeddb3be6a6b2cf7c1ce0f4a7bfe8f1b4038ce39bbc2e0e\": container with ID starting with 71755eaabc1ddff1cfeddb3be6a6b2cf7c1ce0f4a7bfe8f1b4038ce39bbc2e0e not found: ID does not exist"
Sep 17 23:31:15.927328 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:31:15.927298    2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["e2e-test-llm-inference-service-40b85f95/scheduler-v06-threshold-migration-test-kserve-prefill-6444tpqlc"]
Sep 17 23:31:15.928842 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:31:15.928816    2562 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["e2e-test-llm-inference-service-40b85f95/scheduler-v06-threshold-migration-test-kserve-prefill-6444tpqlc"]
Sep 17 23:31:17.666914 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:31:17.666877    2562 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="eda1df77-551e-46ca-afb3-8da2bb765d32" path="/var/lib/kubelet/pods/eda1df77-551e-46ca-afb3-8da2bb765d32/volumes"
Sep 17 23:32:04.924552 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:32:04.924511    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fwwc5_7e452bbb-7dc6-4c9d-a9b1-76400942f5b4/konnectivity-agent/0.log" limit=1024
Sep 17 23:32:05.019410 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:32:05.019373    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fwwc5_7e452bbb-7dc6-4c9d-a9b1-76400942f5b4/konnectivity-agent/0.log" limit=1024
Sep 17 23:32:10.780310 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:32:10.780267    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fwwc5_7e452bbb-7dc6-4c9d-a9b1-76400942f5b4/konnectivity-agent/0.log" limit=1024
Sep 17 23:32:16.172134 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:32:16.172096    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fwwc5_7e452bbb-7dc6-4c9d-a9b1-76400942f5b4/konnectivity-agent/0.log" limit=1024
Sep 17 23:32:16.232475 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:32:16.232434    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fwwc5_7e452bbb-7dc6-4c9d-a9b1-76400942f5b4/konnectivity-agent/0.log" limit=1024
Sep 17 23:32:16.345781 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:32:16.345744    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fwwc5_7e452bbb-7dc6-4c9d-a9b1-76400942f5b4/konnectivity-agent/0.log" limit=1024
Sep 17 23:32:16.612401 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:32:16.612362    2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["e2e-test-llm-inference-service-cf47448b/tokenizer-clean-path-test-kserve-router-scheduler-58494b69snznq"]
Sep 17 23:32:16.612920 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:32:16.612895    2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="eda1df77-551e-46ca-afb3-8da2bb765d32" containerName="main"
Sep 17 23:32:16.612920 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:32:16.612917    2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="eda1df77-551e-46ca-afb3-8da2bb765d32" containerName="main"
Sep 17 23:32:16.613016 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:32:16.613005    2562 memory_manager.go:368] "RemoveStaleState removing state" podUID="eda1df77-551e-46ca-afb3-8da2bb765d32" containerName="main"
Sep 17 23:32:16.616259 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:32:16.616233    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-test-llm-inference-service-cf47448b/tokenizer-clean-path-test-kserve-router-scheduler-58494b69snznq"
Sep 17 23:32:16.618405 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:32:16.618374    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fwwc5_7e452bbb-7dc6-4c9d-a9b1-76400942f5b4/konnectivity-agent/0.log" limit=1024
Sep 17 23:32:16.618860 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:32:16.618830    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-test-llm-inference-service-cf47448b\"/\"openshift-service-ca.crt\""
Sep 17 23:32:16.620296 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:32:16.619951    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-test-llm-inference-service-cf47448b\"/\"kube-root-ca.crt\""
Sep 17 23:32:16.620296 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:32:16.620009    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-test-llm-inference-service-cf47448b\"/\"tokenizer-clean-path-test-epp-sa-dockercfg-s2xnj\""
Sep 17 23:32:16.620296 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:32:16.620028    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-test-llm-inference-service-cf47448b\"/\"tokenizer-clean-path-test-kserve-self-signed-certs\""
Sep 17 23:32:16.620296 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:32:16.620124    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-test-llm-inference-service-cf47448b\"/\"default-dockercfg-rbm66\""
Sep 17 23:32:16.624149 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:32:16.624127    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["e2e-test-llm-inference-service-cf47448b/tokenizer-clean-path-test-kserve-router-scheduler-58494b69snznq"]
Sep 17 23:32:16.684851 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:32:16.684811    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fwwc5_7e452bbb-7dc6-4c9d-a9b1-76400942f5b4/konnectivity-agent/0.log" limit=1024
Sep 17 23:32:16.714637 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:32:16.714593    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qm8lw\" (UniqueName: \"kubernetes.io/projected/5da6c26f-cc45-4ab0-9566-18b13c128053-kube-api-access-qm8lw\") pod \"tokenizer-clean-path-test-kserve-router-scheduler-58494b69snznq\" (UID: \"5da6c26f-cc45-4ab0-9566-18b13c128053\") " pod="e2e-test-llm-inference-service-cf47448b/tokenizer-clean-path-test-kserve-router-scheduler-58494b69snznq"
Sep 17 23:32:16.714836 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:32:16.714746    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/5da6c26f-cc45-4ab0-9566-18b13c128053-tls-certs\") pod \"tokenizer-clean-path-test-kserve-router-scheduler-58494b69snznq\" (UID: \"5da6c26f-cc45-4ab0-9566-18b13c128053\") " pod="e2e-test-llm-inference-service-cf47448b/tokenizer-clean-path-test-kserve-router-scheduler-58494b69snznq"
Sep 17 23:32:16.816105 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:32:16.816057    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/5da6c26f-cc45-4ab0-9566-18b13c128053-tls-certs\") pod \"tokenizer-clean-path-test-kserve-router-scheduler-58494b69snznq\" (UID: \"5da6c26f-cc45-4ab0-9566-18b13c128053\") " pod="e2e-test-llm-inference-service-cf47448b/tokenizer-clean-path-test-kserve-router-scheduler-58494b69snznq"
Sep 17 23:32:16.816340 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:32:16.816148    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qm8lw\" (UniqueName: \"kubernetes.io/projected/5da6c26f-cc45-4ab0-9566-18b13c128053-kube-api-access-qm8lw\") pod \"tokenizer-clean-path-test-kserve-router-scheduler-58494b69snznq\" (UID: \"5da6c26f-cc45-4ab0-9566-18b13c128053\") " pod="e2e-test-llm-inference-service-cf47448b/tokenizer-clean-path-test-kserve-router-scheduler-58494b69snznq"
Sep 17 23:32:16.818731 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:32:16.818698    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/5da6c26f-cc45-4ab0-9566-18b13c128053-tls-certs\") pod \"tokenizer-clean-path-test-kserve-router-scheduler-58494b69snznq\" (UID: \"5da6c26f-cc45-4ab0-9566-18b13c128053\") " pod="e2e-test-llm-inference-service-cf47448b/tokenizer-clean-path-test-kserve-router-scheduler-58494b69snznq"
Sep 17 23:32:16.826883 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:32:16.826860    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-qm8lw\" (UniqueName: \"kubernetes.io/projected/5da6c26f-cc45-4ab0-9566-18b13c128053-kube-api-access-qm8lw\") pod \"tokenizer-clean-path-test-kserve-router-scheduler-58494b69snznq\" (UID: \"5da6c26f-cc45-4ab0-9566-18b13c128053\") " pod="e2e-test-llm-inference-service-cf47448b/tokenizer-clean-path-test-kserve-router-scheduler-58494b69snznq"
Sep 17 23:32:16.929057 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:32:16.928957    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-test-llm-inference-service-cf47448b/tokenizer-clean-path-test-kserve-router-scheduler-58494b69snznq"
Sep 17 23:32:17.203023 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:32:17.202993    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["e2e-test-llm-inference-service-cf47448b/tokenizer-clean-path-test-kserve-router-scheduler-58494b69snznq"]
Sep 17 23:32:17.205471 ip-10-0-132-188 kubenswrapper[2562]: W0917 23:32:17.205443    2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5da6c26f_cc45_4ab0_9566_18b13c128053.slice/crio-722a3ec2a74c4ab3c33e7a9631b1da36d075f03f1a8aa09ad11d22587b605a5b WatchSource:0}: Error finding container 722a3ec2a74c4ab3c33e7a9631b1da36d075f03f1a8aa09ad11d22587b605a5b: Status 404 returned error can't find the container with id 722a3ec2a74c4ab3c33e7a9631b1da36d075f03f1a8aa09ad11d22587b605a5b
Sep 17 23:32:18.202376 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:32:18.202341    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-test-llm-inference-service-cf47448b/tokenizer-clean-path-test-kserve-router-scheduler-58494b69snznq" event={"ID":"5da6c26f-cc45-4ab0-9566-18b13c128053","Type":"ContainerStarted","Data":"0c40d33f71b78d47d05553f8123dee8534357cf754e9f62216b082a1a7e6acd0"}
Sep 17 23:32:18.202569 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:32:18.202395    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-test-llm-inference-service-cf47448b/tokenizer-clean-path-test-kserve-router-scheduler-58494b69snznq" event={"ID":"5da6c26f-cc45-4ab0-9566-18b13c128053","Type":"ContainerStarted","Data":"722a3ec2a74c4ab3c33e7a9631b1da36d075f03f1a8aa09ad11d22587b605a5b"}
Sep 17 23:32:18.202569 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:32:18.202453    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-test-llm-inference-service-cf47448b/tokenizer-clean-path-test-kserve-router-scheduler-58494b69snznq"
Sep 17 23:32:18.218435 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:32:18.218376    2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="e2e-test-llm-inference-service-cf47448b/tokenizer-clean-path-test-kserve-router-scheduler-58494b69snznq" podStartSLOduration=2.218358218 podStartE2EDuration="2.218358218s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-17 23:32:16 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-17 23:32:18.216526663 +0000 UTC m=+1290.618423429" watchObservedRunningTime="2026-09-17 23:32:18.218358218 +0000 UTC m=+1290.620254986"
Sep 17 23:32:30.350957 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:32:30.350915    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fwwc5_7e452bbb-7dc6-4c9d-a9b1-76400942f5b4/konnectivity-agent/0.log" limit=1024
Sep 17 23:32:39.724042 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:32:39.724006    2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["e2e-test-llm-auth-enabled-requires-token-d4ed2c08/auth-enabled-test-kserve-router-scheduler-547fb7d6bb-rvcrg"]
Sep 17 23:32:39.725030 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:32:39.725002    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fwwc5_7e452bbb-7dc6-4c9d-a9b1-76400942f5b4/konnectivity-agent/0.log" limit=1024
Sep 17 23:32:39.778506 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:32:39.778457    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fwwc5_7e452bbb-7dc6-4c9d-a9b1-76400942f5b4/konnectivity-agent/0.log" limit=1024
Sep 17 23:32:48.544196 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:32:48.544129    2562 prober.go:120] "Probe failed" probeType="Readiness" pod="e2e-test-llm-auth-enabled-requires-token-d4ed2c08/auth-enabled-test-kserve-router-scheduler-547fb7d6bb-rvcrg" podUID="b68cebe3-9f55-4358-8cf9-b02b2e975bb6" containerName="main" probeResult="failure" output="service unhealthy (responded with \"NOT_SERVING\")"
Sep 17 23:32:49.208135 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:32:49.208103    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-test-llm-inference-service-cf47448b/tokenizer-clean-path-test-kserve-router-scheduler-58494b69snznq"
Sep 17 23:32:54.745922 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:32:54.745851    2562 kuberuntime_container.go:873] "Killing container with a grace period" pod="e2e-test-llm-auth-enabled-requires-token-d4ed2c08/auth-enabled-test-kserve-router-scheduler-547fb7d6bb-rvcrg" podUID="b68cebe3-9f55-4358-8cf9-b02b2e975bb6" containerName="main" containerID="cri-o://5211dda4a7c0d54a7dd0470650a8ee45848482bc9fdc7f0d803e139aa34ab329" gracePeriod=45
Sep 17 23:32:58.543805 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:32:58.543761    2562 prober.go:120] "Probe failed" probeType="Readiness" pod="e2e-test-llm-auth-enabled-requires-token-d4ed2c08/auth-enabled-test-kserve-router-scheduler-547fb7d6bb-rvcrg" podUID="b68cebe3-9f55-4358-8cf9-b02b2e975bb6" containerName="main" probeResult="failure" output="service unhealthy (responded with \"NOT_SERVING\")"
Sep 17 23:33:08.544426 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:33:08.544307    2562 prober.go:120] "Probe failed" probeType="Readiness" pod="e2e-test-llm-auth-enabled-requires-token-d4ed2c08/auth-enabled-test-kserve-router-scheduler-547fb7d6bb-rvcrg" podUID="b68cebe3-9f55-4358-8cf9-b02b2e975bb6" containerName="main" probeResult="failure" output="service unhealthy (responded with \"NOT_SERVING\")"
Sep 17 23:33:08.544927 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:33:08.544449    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-test-llm-auth-enabled-requires-token-d4ed2c08/auth-enabled-test-kserve-router-scheduler-547fb7d6bb-rvcrg"
Sep 17 23:33:18.544005 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:33:18.543960    2562 prober.go:120] "Probe failed" probeType="Readiness" pod="e2e-test-llm-auth-enabled-requires-token-d4ed2c08/auth-enabled-test-kserve-router-scheduler-547fb7d6bb-rvcrg" podUID="b68cebe3-9f55-4358-8cf9-b02b2e975bb6" containerName="main" probeResult="failure" output="service unhealthy (responded with \"NOT_SERVING\")"
Sep 17 23:33:24.995586 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:33:24.995556    2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-test-llm-auth-enabled-requires-token-d4ed2c08/auth-enabled-test-kserve-router-scheduler-547fb7d6bb-rvcrg"
Sep 17 23:33:25.108055 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:33:25.108022    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/b68cebe3-9f55-4358-8cf9-b02b2e975bb6-tls-certs\") pod \"b68cebe3-9f55-4358-8cf9-b02b2e975bb6\" (UID: \"b68cebe3-9f55-4358-8cf9-b02b2e975bb6\") "
Sep 17 23:33:25.108279 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:33:25.108099    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-r4vxf\" (UniqueName: \"kubernetes.io/projected/b68cebe3-9f55-4358-8cf9-b02b2e975bb6-kube-api-access-r4vxf\") pod \"b68cebe3-9f55-4358-8cf9-b02b2e975bb6\" (UID: \"b68cebe3-9f55-4358-8cf9-b02b2e975bb6\") "
Sep 17 23:33:25.110342 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:33:25.110311    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/b68cebe3-9f55-4358-8cf9-b02b2e975bb6-kube-api-access-r4vxf" (OuterVolumeSpecName: "kube-api-access-r4vxf") pod "b68cebe3-9f55-4358-8cf9-b02b2e975bb6" (UID: "b68cebe3-9f55-4358-8cf9-b02b2e975bb6"). InnerVolumeSpecName "kube-api-access-r4vxf". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 17 23:33:25.110445 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:33:25.110381    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/b68cebe3-9f55-4358-8cf9-b02b2e975bb6-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "b68cebe3-9f55-4358-8cf9-b02b2e975bb6" (UID: "b68cebe3-9f55-4358-8cf9-b02b2e975bb6"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 17 23:33:25.209454 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:33:25.209418    2562 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/b68cebe3-9f55-4358-8cf9-b02b2e975bb6-tls-certs\") on node \"ip-10-0-132-188.ec2.internal\" DevicePath \"\""
Sep 17 23:33:25.209454 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:33:25.209450    2562 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-r4vxf\" (UniqueName: \"kubernetes.io/projected/b68cebe3-9f55-4358-8cf9-b02b2e975bb6-kube-api-access-r4vxf\") on node \"ip-10-0-132-188.ec2.internal\" DevicePath \"\""
Sep 17 23:33:25.514122 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:33:25.514039    2562 generic.go:356] "Generic (PLEG): container finished" podID="b68cebe3-9f55-4358-8cf9-b02b2e975bb6" containerID="5211dda4a7c0d54a7dd0470650a8ee45848482bc9fdc7f0d803e139aa34ab329" exitCode=0
Sep 17 23:33:25.514122 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:33:25.514084    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-test-llm-auth-enabled-requires-token-d4ed2c08/auth-enabled-test-kserve-router-scheduler-547fb7d6bb-rvcrg" event={"ID":"b68cebe3-9f55-4358-8cf9-b02b2e975bb6","Type":"ContainerDied","Data":"5211dda4a7c0d54a7dd0470650a8ee45848482bc9fdc7f0d803e139aa34ab329"}
Sep 17 23:33:25.514122 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:33:25.514083    2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-test-llm-auth-enabled-requires-token-d4ed2c08/auth-enabled-test-kserve-router-scheduler-547fb7d6bb-rvcrg"
Sep 17 23:33:25.514383 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:33:25.514129    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-test-llm-auth-enabled-requires-token-d4ed2c08/auth-enabled-test-kserve-router-scheduler-547fb7d6bb-rvcrg" event={"ID":"b68cebe3-9f55-4358-8cf9-b02b2e975bb6","Type":"ContainerDied","Data":"7dd222a2ca5bc7606eef8cb2917c963e789466f91d4abac43aa7e564a7c1f0ab"}
Sep 17 23:33:25.514383 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:33:25.514152    2562 scope.go:117] "RemoveContainer" containerID="5211dda4a7c0d54a7dd0470650a8ee45848482bc9fdc7f0d803e139aa34ab329"
Sep 17 23:33:25.522885 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:33:25.522864    2562 scope.go:117] "RemoveContainer" containerID="5211dda4a7c0d54a7dd0470650a8ee45848482bc9fdc7f0d803e139aa34ab329"
Sep 17 23:33:25.523136 ip-10-0-132-188 kubenswrapper[2562]: E0917 23:33:25.523115    2562 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"5211dda4a7c0d54a7dd0470650a8ee45848482bc9fdc7f0d803e139aa34ab329\": container with ID starting with 5211dda4a7c0d54a7dd0470650a8ee45848482bc9fdc7f0d803e139aa34ab329 not found: ID does not exist" containerID="5211dda4a7c0d54a7dd0470650a8ee45848482bc9fdc7f0d803e139aa34ab329"
Sep 17 23:33:25.523213 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:33:25.523145    2562 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"5211dda4a7c0d54a7dd0470650a8ee45848482bc9fdc7f0d803e139aa34ab329"} err="failed to get container status \"5211dda4a7c0d54a7dd0470650a8ee45848482bc9fdc7f0d803e139aa34ab329\": rpc error: code = NotFound desc = could not find container \"5211dda4a7c0d54a7dd0470650a8ee45848482bc9fdc7f0d803e139aa34ab329\": container with ID starting with 5211dda4a7c0d54a7dd0470650a8ee45848482bc9fdc7f0d803e139aa34ab329 not found: ID does not exist"
Sep 17 23:33:25.534200 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:33:25.534150    2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["e2e-test-llm-auth-enabled-requires-token-d4ed2c08/auth-enabled-test-kserve-router-scheduler-547fb7d6bb-rvcrg"]
Sep 17 23:33:25.537719 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:33:25.537694    2562 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["e2e-test-llm-auth-enabled-requires-token-d4ed2c08/auth-enabled-test-kserve-router-scheduler-547fb7d6bb-rvcrg"]
Sep 17 23:33:25.666489 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:33:25.666459    2562 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="b68cebe3-9f55-4358-8cf9-b02b2e975bb6" path="/var/lib/kubelet/pods/b68cebe3-9f55-4358-8cf9-b02b2e975bb6/volumes"
Sep 17 23:33:27.204713 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:33:27.204629    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fwwc5_7e452bbb-7dc6-4c9d-a9b1-76400942f5b4/konnectivity-agent/0.log" limit=1024
Sep 17 23:33:27.304522 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:33:27.304489    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fwwc5_7e452bbb-7dc6-4c9d-a9b1-76400942f5b4/konnectivity-agent/0.log" limit=1024
Sep 17 23:33:32.455268 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:33:32.455231    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fwwc5_7e452bbb-7dc6-4c9d-a9b1-76400942f5b4/konnectivity-agent/0.log" limit=1024
Sep 17 23:33:32.632346 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:33:32.632309    2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["e2e-test-llm-auth-invalid-token-rejected-e7cfa2c6/auth-invalid-token-test-kserve-router-scheduler-6dc5d5cf9cr8d89"]
Sep 17 23:33:32.632682 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:33:32.632670    2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b68cebe3-9f55-4358-8cf9-b02b2e975bb6" containerName="main"
Sep 17 23:33:32.632727 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:33:32.632685    2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="b68cebe3-9f55-4358-8cf9-b02b2e975bb6" containerName="main"
Sep 17 23:33:32.632762 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:33:32.632737    2562 memory_manager.go:368] "RemoveStaleState removing state" podUID="b68cebe3-9f55-4358-8cf9-b02b2e975bb6" containerName="main"
Sep 17 23:33:32.637264 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:33:32.637233    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-test-llm-auth-invalid-token-rejected-e7cfa2c6/auth-invalid-token-test-kserve-router-scheduler-6dc5d5cf9cr8d89"
Sep 17 23:33:32.640563 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:33:32.640532    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-test-llm-auth-invalid-token-rejected-e7cfa2c6\"/\"auth-invalid-token-test-kserve-self-signed-certs\""
Sep 17 23:33:32.640563 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:33:32.640553    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-test-llm-auth-invalid-token-rejected-e7cfa2c6\"/\"kube-root-ca.crt\""
Sep 17 23:33:32.640787 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:33:32.640587    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-test-llm-auth-invalid-token-rejected-e7cfa2c6\"/\"auth-invalid-token-test-epp-sa-dockercfg-rcvfm\""
Sep 17 23:33:32.640787 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:33:32.640668    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-test-llm-auth-invalid-token-rejected-e7cfa2c6\"/\"default-dockercfg-fcl7d\""
Sep 17 23:33:32.641036 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:33:32.641009    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-test-llm-auth-invalid-token-rejected-e7cfa2c6\"/\"openshift-service-ca.crt\""
Sep 17 23:33:32.643836 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:33:32.643808    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["e2e-test-llm-auth-invalid-token-rejected-e7cfa2c6/auth-invalid-token-test-kserve-router-scheduler-6dc5d5cf9cr8d89"]
Sep 17 23:33:32.658830 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:33:32.658799    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fwwc5_7e452bbb-7dc6-4c9d-a9b1-76400942f5b4/konnectivity-agent/0.log" limit=1024
Sep 17 23:33:32.780546 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:33:32.780452    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/d7751c60-cefb-4b89-93f5-f6037946e231-tls-certs\") pod \"auth-invalid-token-test-kserve-router-scheduler-6dc5d5cf9cr8d89\" (UID: \"d7751c60-cefb-4b89-93f5-f6037946e231\") " pod="e2e-test-llm-auth-invalid-token-rejected-e7cfa2c6/auth-invalid-token-test-kserve-router-scheduler-6dc5d5cf9cr8d89"
Sep 17 23:33:32.780730 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:33:32.780597    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-kjwkb\" (UniqueName: \"kubernetes.io/projected/d7751c60-cefb-4b89-93f5-f6037946e231-kube-api-access-kjwkb\") pod \"auth-invalid-token-test-kserve-router-scheduler-6dc5d5cf9cr8d89\" (UID: \"d7751c60-cefb-4b89-93f5-f6037946e231\") " pod="e2e-test-llm-auth-invalid-token-rejected-e7cfa2c6/auth-invalid-token-test-kserve-router-scheduler-6dc5d5cf9cr8d89"
Sep 17 23:33:32.881355 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:33:32.881316    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/d7751c60-cefb-4b89-93f5-f6037946e231-tls-certs\") pod \"auth-invalid-token-test-kserve-router-scheduler-6dc5d5cf9cr8d89\" (UID: \"d7751c60-cefb-4b89-93f5-f6037946e231\") " pod="e2e-test-llm-auth-invalid-token-rejected-e7cfa2c6/auth-invalid-token-test-kserve-router-scheduler-6dc5d5cf9cr8d89"
Sep 17 23:33:32.881355 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:33:32.881356    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kjwkb\" (UniqueName: \"kubernetes.io/projected/d7751c60-cefb-4b89-93f5-f6037946e231-kube-api-access-kjwkb\") pod \"auth-invalid-token-test-kserve-router-scheduler-6dc5d5cf9cr8d89\" (UID: \"d7751c60-cefb-4b89-93f5-f6037946e231\") " pod="e2e-test-llm-auth-invalid-token-rejected-e7cfa2c6/auth-invalid-token-test-kserve-router-scheduler-6dc5d5cf9cr8d89"
Sep 17 23:33:32.883855 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:33:32.883832    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/d7751c60-cefb-4b89-93f5-f6037946e231-tls-certs\") pod \"auth-invalid-token-test-kserve-router-scheduler-6dc5d5cf9cr8d89\" (UID: \"d7751c60-cefb-4b89-93f5-f6037946e231\") " pod="e2e-test-llm-auth-invalid-token-rejected-e7cfa2c6/auth-invalid-token-test-kserve-router-scheduler-6dc5d5cf9cr8d89"
Sep 17 23:33:32.888453 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:33:32.888423    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-kjwkb\" (UniqueName: \"kubernetes.io/projected/d7751c60-cefb-4b89-93f5-f6037946e231-kube-api-access-kjwkb\") pod \"auth-invalid-token-test-kserve-router-scheduler-6dc5d5cf9cr8d89\" (UID: \"d7751c60-cefb-4b89-93f5-f6037946e231\") " pod="e2e-test-llm-auth-invalid-token-rejected-e7cfa2c6/auth-invalid-token-test-kserve-router-scheduler-6dc5d5cf9cr8d89"
Sep 17 23:33:32.949129 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:33:32.949095    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-test-llm-auth-invalid-token-rejected-e7cfa2c6/auth-invalid-token-test-kserve-router-scheduler-6dc5d5cf9cr8d89"
Sep 17 23:33:33.220235 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:33:33.220205    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["e2e-test-llm-auth-invalid-token-rejected-e7cfa2c6/auth-invalid-token-test-kserve-router-scheduler-6dc5d5cf9cr8d89"]
Sep 17 23:33:33.223108 ip-10-0-132-188 kubenswrapper[2562]: W0917 23:33:33.223079    2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd7751c60_cefb_4b89_93f5_f6037946e231.slice/crio-5ff74e5ddc25024ebeeea075e9de4001c940444c4be7755aac1e9f08e24d9700 WatchSource:0}: Error finding container 5ff74e5ddc25024ebeeea075e9de4001c940444c4be7755aac1e9f08e24d9700: Status 404 returned error can't find the container with id 5ff74e5ddc25024ebeeea075e9de4001c940444c4be7755aac1e9f08e24d9700
Sep 17 23:33:33.555239 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:33:33.555116    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-test-llm-auth-invalid-token-rejected-e7cfa2c6/auth-invalid-token-test-kserve-router-scheduler-6dc5d5cf9cr8d89" event={"ID":"d7751c60-cefb-4b89-93f5-f6037946e231","Type":"ContainerStarted","Data":"891b395790c7bbbbf342b280d386da475471d68b452ffcae9d0345687d8584d6"}
Sep 17 23:33:33.555239 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:33:33.555201    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-test-llm-auth-invalid-token-rejected-e7cfa2c6/auth-invalid-token-test-kserve-router-scheduler-6dc5d5cf9cr8d89" event={"ID":"d7751c60-cefb-4b89-93f5-f6037946e231","Type":"ContainerStarted","Data":"5ff74e5ddc25024ebeeea075e9de4001c940444c4be7755aac1e9f08e24d9700"}
Sep 17 23:33:33.555239 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:33:33.555231    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-test-llm-auth-invalid-token-rejected-e7cfa2c6/auth-invalid-token-test-kserve-router-scheduler-6dc5d5cf9cr8d89"
Sep 17 23:33:33.570887 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:33:33.570833    2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="e2e-test-llm-auth-invalid-token-rejected-e7cfa2c6/auth-invalid-token-test-kserve-router-scheduler-6dc5d5cf9cr8d89" podStartSLOduration=1.570815291 podStartE2EDuration="1.570815291s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-17 23:33:32 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-17 23:33:33.569315043 +0000 UTC m=+1365.971211810" watchObservedRunningTime="2026-09-17 23:33:33.570815291 +0000 UTC m=+1365.972712055"
Sep 17 23:33:34.323563 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:33:34.323524    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fwwc5_7e452bbb-7dc6-4c9d-a9b1-76400942f5b4/konnectivity-agent/0.log" limit=1024
Sep 17 23:33:39.710988 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:33:39.710946    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fwwc5_7e452bbb-7dc6-4c9d-a9b1-76400942f5b4/konnectivity-agent/0.log" limit=1024
Sep 17 23:33:39.774253 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:33:39.774207    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fwwc5_7e452bbb-7dc6-4c9d-a9b1-76400942f5b4/konnectivity-agent/0.log" limit=1024
Sep 17 23:33:41.978522 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:33:41.978476    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fwwc5_7e452bbb-7dc6-4c9d-a9b1-76400942f5b4/konnectivity-agent/0.log" limit=1024
Sep 17 23:33:45.158865 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:33:45.158809    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fwwc5_7e452bbb-7dc6-4c9d-a9b1-76400942f5b4/konnectivity-agent/0.log" limit=1024
Sep 17 23:33:45.175190 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:33:45.175139    2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["e2e-test-llm-inference-service-cf47448b/tokenizer-clean-path-test-kserve-router-scheduler-58494b69snznq"]
Sep 17 23:33:45.228349 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:33:45.228310    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fwwc5_7e452bbb-7dc6-4c9d-a9b1-76400942f5b4/konnectivity-agent/0.log" limit=1024
Sep 17 23:33:49.208600 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:33:49.208557    2562 prober.go:120] "Probe failed" probeType="Readiness" pod="e2e-test-llm-inference-service-cf47448b/tokenizer-clean-path-test-kserve-router-scheduler-58494b69snznq" podUID="5da6c26f-cc45-4ab0-9566-18b13c128053" containerName="main" probeResult="failure" output="service unhealthy (responded with \"NOT_SERVING\")"
Sep 17 23:33:59.208774 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:33:59.208729    2562 prober.go:120] "Probe failed" probeType="Readiness" pod="e2e-test-llm-inference-service-cf47448b/tokenizer-clean-path-test-kserve-router-scheduler-58494b69snznq" podUID="5da6c26f-cc45-4ab0-9566-18b13c128053" containerName="main" probeResult="failure" output="service unhealthy (responded with \"NOT_SERVING\")"
Sep 17 23:34:00.197272 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:34:00.197227    2562 kuberuntime_container.go:873] "Killing container with a grace period" pod="e2e-test-llm-inference-service-cf47448b/tokenizer-clean-path-test-kserve-router-scheduler-58494b69snznq" podUID="5da6c26f-cc45-4ab0-9566-18b13c128053" containerName="main" containerID="cri-o://0c40d33f71b78d47d05553f8123dee8534357cf754e9f62216b082a1a7e6acd0" gracePeriod=45
Sep 17 23:34:04.562678 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:34:04.562646    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-test-llm-auth-invalid-token-rejected-e7cfa2c6/auth-invalid-token-test-kserve-router-scheduler-6dc5d5cf9cr8d89"
Sep 17 23:34:09.209076 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:34:09.209036    2562 prober.go:120] "Probe failed" probeType="Readiness" pod="e2e-test-llm-inference-service-cf47448b/tokenizer-clean-path-test-kserve-router-scheduler-58494b69snznq" podUID="5da6c26f-cc45-4ab0-9566-18b13c128053" containerName="main" probeResult="failure" output="service unhealthy (responded with \"NOT_SERVING\")"
Sep 17 23:34:09.209509 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:34:09.209144    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-test-llm-inference-service-cf47448b/tokenizer-clean-path-test-kserve-router-scheduler-58494b69snznq"
Sep 17 23:34:19.209055 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:34:19.209009    2562 prober.go:120] "Probe failed" probeType="Readiness" pod="e2e-test-llm-inference-service-cf47448b/tokenizer-clean-path-test-kserve-router-scheduler-58494b69snznq" podUID="5da6c26f-cc45-4ab0-9566-18b13c128053" containerName="main" probeResult="failure" output="service unhealthy (responded with \"NOT_SERVING\")"
Sep 17 23:34:24.969797 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:34:24.969751    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fwwc5_7e452bbb-7dc6-4c9d-a9b1-76400942f5b4/konnectivity-agent/0.log" limit=1024
Sep 17 23:34:25.035345 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:34:25.035304    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fwwc5_7e452bbb-7dc6-4c9d-a9b1-76400942f5b4/konnectivity-agent/0.log" limit=1024
Sep 17 23:34:29.208431 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:34:29.208391    2562 prober.go:120] "Probe failed" probeType="Readiness" pod="e2e-test-llm-inference-service-cf47448b/tokenizer-clean-path-test-kserve-router-scheduler-58494b69snznq" podUID="5da6c26f-cc45-4ab0-9566-18b13c128053" containerName="main" probeResult="failure" output="service unhealthy (responded with \"NOT_SERVING\")"
Sep 17 23:34:30.445716 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:34:30.445685    2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-test-llm-inference-service-cf47448b/tokenizer-clean-path-test-kserve-router-scheduler-58494b69snznq"
Sep 17 23:34:30.592905 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:34:30.592864    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-qm8lw\" (UniqueName: \"kubernetes.io/projected/5da6c26f-cc45-4ab0-9566-18b13c128053-kube-api-access-qm8lw\") pod \"5da6c26f-cc45-4ab0-9566-18b13c128053\" (UID: \"5da6c26f-cc45-4ab0-9566-18b13c128053\") "
Sep 17 23:34:30.593095 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:34:30.592955    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/5da6c26f-cc45-4ab0-9566-18b13c128053-tls-certs\") pod \"5da6c26f-cc45-4ab0-9566-18b13c128053\" (UID: \"5da6c26f-cc45-4ab0-9566-18b13c128053\") "
Sep 17 23:34:30.595150 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:34:30.595124    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/5da6c26f-cc45-4ab0-9566-18b13c128053-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "5da6c26f-cc45-4ab0-9566-18b13c128053" (UID: "5da6c26f-cc45-4ab0-9566-18b13c128053"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 17 23:34:30.595287 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:34:30.595269    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/5da6c26f-cc45-4ab0-9566-18b13c128053-kube-api-access-qm8lw" (OuterVolumeSpecName: "kube-api-access-qm8lw") pod "5da6c26f-cc45-4ab0-9566-18b13c128053" (UID: "5da6c26f-cc45-4ab0-9566-18b13c128053"). InnerVolumeSpecName "kube-api-access-qm8lw". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 17 23:34:30.693596 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:34:30.693559    2562 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-qm8lw\" (UniqueName: \"kubernetes.io/projected/5da6c26f-cc45-4ab0-9566-18b13c128053-kube-api-access-qm8lw\") on node \"ip-10-0-132-188.ec2.internal\" DevicePath \"\""
Sep 17 23:34:30.693596 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:34:30.693588    2562 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/5da6c26f-cc45-4ab0-9566-18b13c128053-tls-certs\") on node \"ip-10-0-132-188.ec2.internal\" DevicePath \"\""
Sep 17 23:34:30.836887 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:34:30.836854    2562 generic.go:356] "Generic (PLEG): container finished" podID="5da6c26f-cc45-4ab0-9566-18b13c128053" containerID="0c40d33f71b78d47d05553f8123dee8534357cf754e9f62216b082a1a7e6acd0" exitCode=0
Sep 17 23:34:30.837083 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:34:30.836917    2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-test-llm-inference-service-cf47448b/tokenizer-clean-path-test-kserve-router-scheduler-58494b69snznq"
Sep 17 23:34:30.837083 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:34:30.836942    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-test-llm-inference-service-cf47448b/tokenizer-clean-path-test-kserve-router-scheduler-58494b69snznq" event={"ID":"5da6c26f-cc45-4ab0-9566-18b13c128053","Type":"ContainerDied","Data":"0c40d33f71b78d47d05553f8123dee8534357cf754e9f62216b082a1a7e6acd0"}
Sep 17 23:34:30.837083 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:34:30.837008    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-test-llm-inference-service-cf47448b/tokenizer-clean-path-test-kserve-router-scheduler-58494b69snznq" event={"ID":"5da6c26f-cc45-4ab0-9566-18b13c128053","Type":"ContainerDied","Data":"722a3ec2a74c4ab3c33e7a9631b1da36d075f03f1a8aa09ad11d22587b605a5b"}
Sep 17 23:34:30.837083 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:34:30.837037    2562 scope.go:117] "RemoveContainer" containerID="0c40d33f71b78d47d05553f8123dee8534357cf754e9f62216b082a1a7e6acd0"
Sep 17 23:34:30.845675 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:34:30.845658    2562 scope.go:117] "RemoveContainer" containerID="0c40d33f71b78d47d05553f8123dee8534357cf754e9f62216b082a1a7e6acd0"
Sep 17 23:34:30.845943 ip-10-0-132-188 kubenswrapper[2562]: E0917 23:34:30.845922    2562 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"0c40d33f71b78d47d05553f8123dee8534357cf754e9f62216b082a1a7e6acd0\": container with ID starting with 0c40d33f71b78d47d05553f8123dee8534357cf754e9f62216b082a1a7e6acd0 not found: ID does not exist" containerID="0c40d33f71b78d47d05553f8123dee8534357cf754e9f62216b082a1a7e6acd0"
Sep 17 23:34:30.845991 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:34:30.845952    2562 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"0c40d33f71b78d47d05553f8123dee8534357cf754e9f62216b082a1a7e6acd0"} err="failed to get container status \"0c40d33f71b78d47d05553f8123dee8534357cf754e9f62216b082a1a7e6acd0\": rpc error: code = NotFound desc = could not find container \"0c40d33f71b78d47d05553f8123dee8534357cf754e9f62216b082a1a7e6acd0\": container with ID starting with 0c40d33f71b78d47d05553f8123dee8534357cf754e9f62216b082a1a7e6acd0 not found: ID does not exist"
Sep 17 23:34:30.857183 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:34:30.857139    2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["e2e-test-llm-inference-service-cf47448b/tokenizer-clean-path-test-kserve-router-scheduler-58494b69snznq"]
Sep 17 23:34:30.860357 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:34:30.860327    2562 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["e2e-test-llm-inference-service-cf47448b/tokenizer-clean-path-test-kserve-router-scheduler-58494b69snznq"]
Sep 17 23:34:31.666777 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:34:31.666743    2562 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="5da6c26f-cc45-4ab0-9566-18b13c128053" path="/var/lib/kubelet/pods/5da6c26f-cc45-4ab0-9566-18b13c128053/volumes"
Sep 17 23:34:35.150470 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:34:35.150440    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fwwc5_7e452bbb-7dc6-4c9d-a9b1-76400942f5b4/konnectivity-agent/0.log" limit=1024
Sep 17 23:34:35.245068 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:34:35.245033    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fwwc5_7e452bbb-7dc6-4c9d-a9b1-76400942f5b4/konnectivity-agent/0.log" limit=1024
Sep 17 23:34:41.125369 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:34:41.125331    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fwwc5_7e452bbb-7dc6-4c9d-a9b1-76400942f5b4/konnectivity-agent/0.log" limit=1024
Sep 17 23:34:41.329441 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:34:41.329399    2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["e2e-test-llm-inference-service-312caf2d/tokenizer-migration-path-test-kserve-router-scheduler-5785sxtvr"]
Sep 17 23:34:41.329943 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:34:41.329917    2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="5da6c26f-cc45-4ab0-9566-18b13c128053" containerName="main"
Sep 17 23:34:41.330089 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:34:41.329978    2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="5da6c26f-cc45-4ab0-9566-18b13c128053" containerName="main"
Sep 17 23:34:41.330089 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:34:41.330085    2562 memory_manager.go:368] "RemoveStaleState removing state" podUID="5da6c26f-cc45-4ab0-9566-18b13c128053" containerName="main"
Sep 17 23:34:41.333112 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:34:41.333088    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-test-llm-inference-service-312caf2d/tokenizer-migration-path-test-kserve-router-scheduler-5785sxtvr"
Sep 17 23:34:41.335475 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:34:41.335437    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fwwc5_7e452bbb-7dc6-4c9d-a9b1-76400942f5b4/konnectivity-agent/0.log" limit=1024
Sep 17 23:34:41.335617 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:34:41.335578    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-test-llm-inference-service-312caf2d\"/\"kube-root-ca.crt\""
Sep 17 23:34:41.336706 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:34:41.336675    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-test-llm-inference-service-312caf2d\"/\"tokenizer-migration-path-test-epp-sa-dockercfg-8dfg7\""
Sep 17 23:34:41.336826 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:34:41.336709    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-test-llm-inference-service-312caf2d\"/\"default-dockercfg-2qblg\""
Sep 17 23:34:41.336826 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:34:41.336675    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-test-llm-inference-service-312caf2d\"/\"tokenizer-migration-path-test-kserve-self-signed-certs\""
Sep 17 23:34:41.336826 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:34:41.336764    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-test-llm-inference-service-312caf2d\"/\"openshift-service-ca.crt\""
Sep 17 23:34:41.342292 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:34:41.342264    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["e2e-test-llm-inference-service-312caf2d/tokenizer-migration-path-test-kserve-router-scheduler-5785sxtvr"]
Sep 17 23:34:41.390298 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:34:41.390202    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/79fa0e66-4e36-4c58-8031-58515d1978fd-tls-certs\") pod \"tokenizer-migration-path-test-kserve-router-scheduler-5785sxtvr\" (UID: \"79fa0e66-4e36-4c58-8031-58515d1978fd\") " pod="e2e-test-llm-inference-service-312caf2d/tokenizer-migration-path-test-kserve-router-scheduler-5785sxtvr"
Sep 17 23:34:41.390458 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:34:41.390309    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wj47w\" (UniqueName: \"kubernetes.io/projected/79fa0e66-4e36-4c58-8031-58515d1978fd-kube-api-access-wj47w\") pod \"tokenizer-migration-path-test-kserve-router-scheduler-5785sxtvr\" (UID: \"79fa0e66-4e36-4c58-8031-58515d1978fd\") " pod="e2e-test-llm-inference-service-312caf2d/tokenizer-migration-path-test-kserve-router-scheduler-5785sxtvr"
Sep 17 23:34:41.431342 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:34:41.431286    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fwwc5_7e452bbb-7dc6-4c9d-a9b1-76400942f5b4/konnectivity-agent/0.log" limit=1024
Sep 17 23:34:41.490956 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:34:41.490912    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wj47w\" (UniqueName: \"kubernetes.io/projected/79fa0e66-4e36-4c58-8031-58515d1978fd-kube-api-access-wj47w\") pod \"tokenizer-migration-path-test-kserve-router-scheduler-5785sxtvr\" (UID: \"79fa0e66-4e36-4c58-8031-58515d1978fd\") " pod="e2e-test-llm-inference-service-312caf2d/tokenizer-migration-path-test-kserve-router-scheduler-5785sxtvr"
Sep 17 23:34:41.491143 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:34:41.491003    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/79fa0e66-4e36-4c58-8031-58515d1978fd-tls-certs\") pod \"tokenizer-migration-path-test-kserve-router-scheduler-5785sxtvr\" (UID: \"79fa0e66-4e36-4c58-8031-58515d1978fd\") " pod="e2e-test-llm-inference-service-312caf2d/tokenizer-migration-path-test-kserve-router-scheduler-5785sxtvr"
Sep 17 23:34:41.493531 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:34:41.493499    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/79fa0e66-4e36-4c58-8031-58515d1978fd-tls-certs\") pod \"tokenizer-migration-path-test-kserve-router-scheduler-5785sxtvr\" (UID: \"79fa0e66-4e36-4c58-8031-58515d1978fd\") " pod="e2e-test-llm-inference-service-312caf2d/tokenizer-migration-path-test-kserve-router-scheduler-5785sxtvr"
Sep 17 23:34:41.499000 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:34:41.498966    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-wj47w\" (UniqueName: \"kubernetes.io/projected/79fa0e66-4e36-4c58-8031-58515d1978fd-kube-api-access-wj47w\") pod \"tokenizer-migration-path-test-kserve-router-scheduler-5785sxtvr\" (UID: \"79fa0e66-4e36-4c58-8031-58515d1978fd\") " pod="e2e-test-llm-inference-service-312caf2d/tokenizer-migration-path-test-kserve-router-scheduler-5785sxtvr"
Sep 17 23:34:41.531136 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:34:41.531098    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fwwc5_7e452bbb-7dc6-4c9d-a9b1-76400942f5b4/konnectivity-agent/0.log" limit=1024
Sep 17 23:34:41.648584 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:34:41.648482    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-test-llm-inference-service-312caf2d/tokenizer-migration-path-test-kserve-router-scheduler-5785sxtvr"
Sep 17 23:34:41.923654 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:34:41.923514    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["e2e-test-llm-inference-service-312caf2d/tokenizer-migration-path-test-kserve-router-scheduler-5785sxtvr"]
Sep 17 23:34:41.926051 ip-10-0-132-188 kubenswrapper[2562]: W0917 23:34:41.926019    2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod79fa0e66_4e36_4c58_8031_58515d1978fd.slice/crio-98ff4d1c5ff45af9566866256c22b5d389fea9da5ffb46a3f3dd016c3f007cb7 WatchSource:0}: Error finding container 98ff4d1c5ff45af9566866256c22b5d389fea9da5ffb46a3f3dd016c3f007cb7: Status 404 returned error can't find the container with id 98ff4d1c5ff45af9566866256c22b5d389fea9da5ffb46a3f3dd016c3f007cb7
Sep 17 23:34:42.894737 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:34:42.894690    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-test-llm-inference-service-312caf2d/tokenizer-migration-path-test-kserve-router-scheduler-5785sxtvr" event={"ID":"79fa0e66-4e36-4c58-8031-58515d1978fd","Type":"ContainerStarted","Data":"adea9337c68b956e61ef471921d10068dd22bd21bca29e8e0438a58fe61c4405"}
Sep 17 23:34:42.895244 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:34:42.894785    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-test-llm-inference-service-312caf2d/tokenizer-migration-path-test-kserve-router-scheduler-5785sxtvr" event={"ID":"79fa0e66-4e36-4c58-8031-58515d1978fd","Type":"ContainerStarted","Data":"98ff4d1c5ff45af9566866256c22b5d389fea9da5ffb46a3f3dd016c3f007cb7"}
Sep 17 23:34:42.895244 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:34:42.894875    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-test-llm-inference-service-312caf2d/tokenizer-migration-path-test-kserve-router-scheduler-5785sxtvr"
Sep 17 23:34:42.910475 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:34:42.910420    2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="e2e-test-llm-inference-service-312caf2d/tokenizer-migration-path-test-kserve-router-scheduler-5785sxtvr" podStartSLOduration=1.910404662 podStartE2EDuration="1.910404662s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-17 23:34:41 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-17 23:34:42.909353195 +0000 UTC m=+1435.311249960" watchObservedRunningTime="2026-09-17 23:34:42.910404662 +0000 UTC m=+1435.312301425"
Sep 17 23:34:46.924300 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:34:46.924262    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fwwc5_7e452bbb-7dc6-4c9d-a9b1-76400942f5b4/konnectivity-agent/0.log" limit=1024
Sep 17 23:34:46.987058 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:34:46.987026    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fwwc5_7e452bbb-7dc6-4c9d-a9b1-76400942f5b4/konnectivity-agent/0.log" limit=1024
Sep 17 23:34:50.450437 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:34:50.450392    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fwwc5_7e452bbb-7dc6-4c9d-a9b1-76400942f5b4/konnectivity-agent/0.log" limit=1024
Sep 17 23:35:01.909474 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:35:01.909432    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fwwc5_7e452bbb-7dc6-4c9d-a9b1-76400942f5b4/konnectivity-agent/0.log" limit=1024
Sep 17 23:35:09.241036 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:35:09.240995    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fwwc5_7e452bbb-7dc6-4c9d-a9b1-76400942f5b4/konnectivity-agent/0.log" limit=1024
Sep 17 23:35:12.323245 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:35:12.323207    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fwwc5_7e452bbb-7dc6-4c9d-a9b1-76400942f5b4/konnectivity-agent/0.log" limit=1024
Sep 17 23:35:13.903860 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:35:13.903829    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-test-llm-inference-service-312caf2d/tokenizer-migration-path-test-kserve-router-scheduler-5785sxtvr"
Sep 17 23:35:14.688870 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:35:14.688835    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fwwc5_7e452bbb-7dc6-4c9d-a9b1-76400942f5b4/konnectivity-agent/0.log" limit=1024
Sep 17 23:35:18.949376 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:35:18.949338    2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["e2e-test-llm-inference-service-312caf2d/tokenizer-migration-path-test-kserve-router-scheduler-5785sxtvr"]
Sep 17 23:35:18.951850 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:35:18.951810    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fwwc5_7e452bbb-7dc6-4c9d-a9b1-76400942f5b4/konnectivity-agent/0.log" limit=1024
Sep 17 23:35:19.007157 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:35:19.007117    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fwwc5_7e452bbb-7dc6-4c9d-a9b1-76400942f5b4/konnectivity-agent/0.log" limit=1024
Sep 17 23:35:23.904523 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:35:23.904483    2562 prober.go:120] "Probe failed" probeType="Readiness" pod="e2e-test-llm-inference-service-312caf2d/tokenizer-migration-path-test-kserve-router-scheduler-5785sxtvr" podUID="79fa0e66-4e36-4c58-8031-58515d1978fd" containerName="main" probeResult="failure" output="service unhealthy (responded with \"NOT_SERVING\")"
Sep 17 23:35:33.904685 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:35:33.904647    2562 prober.go:120] "Probe failed" probeType="Readiness" pod="e2e-test-llm-inference-service-312caf2d/tokenizer-migration-path-test-kserve-router-scheduler-5785sxtvr" podUID="79fa0e66-4e36-4c58-8031-58515d1978fd" containerName="main" probeResult="failure" output="service unhealthy (responded with \"NOT_SERVING\")"
Sep 17 23:35:33.970091 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:35:33.970033    2562 kuberuntime_container.go:873] "Killing container with a grace period" pod="e2e-test-llm-inference-service-312caf2d/tokenizer-migration-path-test-kserve-router-scheduler-5785sxtvr" podUID="79fa0e66-4e36-4c58-8031-58515d1978fd" containerName="main" containerID="cri-o://adea9337c68b956e61ef471921d10068dd22bd21bca29e8e0438a58fe61c4405" gracePeriod=45
Sep 17 23:35:34.331856 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:35:34.331824    2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["e2e-test-llm-auth-invalid-token-rejected-e7cfa2c6/auth-invalid-token-test-kserve-router-scheduler-6dc5d5cf9cr8d89"]
Sep 17 23:35:34.336900 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:35:34.336870    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fwwc5_7e452bbb-7dc6-4c9d-a9b1-76400942f5b4/konnectivity-agent/0.log" limit=1024
Sep 17 23:35:34.401532 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:35:34.401493    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fwwc5_7e452bbb-7dc6-4c9d-a9b1-76400942f5b4/konnectivity-agent/0.log" limit=1024
Sep 17 23:35:34.562589 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:35:34.562552    2562 prober.go:120] "Probe failed" probeType="Readiness" pod="e2e-test-llm-auth-invalid-token-rejected-e7cfa2c6/auth-invalid-token-test-kserve-router-scheduler-6dc5d5cf9cr8d89" podUID="d7751c60-cefb-4b89-93f5-f6037946e231" containerName="main" probeResult="failure" output="service unhealthy (responded with \"NOT_SERVING\")"
Sep 17 23:35:43.904228 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:35:43.904138    2562 prober.go:120] "Probe failed" probeType="Readiness" pod="e2e-test-llm-inference-service-312caf2d/tokenizer-migration-path-test-kserve-router-scheduler-5785sxtvr" podUID="79fa0e66-4e36-4c58-8031-58515d1978fd" containerName="main" probeResult="failure" output="service unhealthy (responded with \"NOT_SERVING\")"
Sep 17 23:35:43.904604 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:35:43.904254    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-test-llm-inference-service-312caf2d/tokenizer-migration-path-test-kserve-router-scheduler-5785sxtvr"
Sep 17 23:35:44.568713 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:35:44.568666    2562 prober.go:120] "Probe failed" probeType="Readiness" pod="e2e-test-llm-auth-invalid-token-rejected-e7cfa2c6/auth-invalid-token-test-kserve-router-scheduler-6dc5d5cf9cr8d89" podUID="d7751c60-cefb-4b89-93f5-f6037946e231" containerName="main" probeResult="failure" output="service unhealthy (responded with \"NOT_SERVING\")"
Sep 17 23:35:46.003429 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:35:46.003376    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fwwc5_7e452bbb-7dc6-4c9d-a9b1-76400942f5b4/konnectivity-agent/0.log" limit=1024
Sep 17 23:35:49.353080 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:35:49.353034    2562 kuberuntime_container.go:873] "Killing container with a grace period" pod="e2e-test-llm-auth-invalid-token-rejected-e7cfa2c6/auth-invalid-token-test-kserve-router-scheduler-6dc5d5cf9cr8d89" podUID="d7751c60-cefb-4b89-93f5-f6037946e231" containerName="main" containerID="cri-o://891b395790c7bbbbf342b280d386da475471d68b452ffcae9d0345687d8584d6" gracePeriod=45
Sep 17 23:35:53.904549 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:35:53.904512    2562 prober.go:120] "Probe failed" probeType="Readiness" pod="e2e-test-llm-inference-service-312caf2d/tokenizer-migration-path-test-kserve-router-scheduler-5785sxtvr" podUID="79fa0e66-4e36-4c58-8031-58515d1978fd" containerName="main" probeResult="failure" output="service unhealthy (responded with \"NOT_SERVING\")"
Sep 17 23:35:54.563164 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:35:54.563126    2562 prober.go:120] "Probe failed" probeType="Readiness" pod="e2e-test-llm-auth-invalid-token-rejected-e7cfa2c6/auth-invalid-token-test-kserve-router-scheduler-6dc5d5cf9cr8d89" podUID="d7751c60-cefb-4b89-93f5-f6037946e231" containerName="main" probeResult="failure" output="service unhealthy (responded with \"NOT_SERVING\")"
Sep 17 23:35:54.563352 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:35:54.563250    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-test-llm-auth-invalid-token-rejected-e7cfa2c6/auth-invalid-token-test-kserve-router-scheduler-6dc5d5cf9cr8d89"
Sep 17 23:35:57.051681 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:35:57.051641    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fwwc5_7e452bbb-7dc6-4c9d-a9b1-76400942f5b4/konnectivity-agent/0.log" limit=1024
Sep 17 23:36:03.904711 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:36:03.904626    2562 prober.go:120] "Probe failed" probeType="Readiness" pod="e2e-test-llm-inference-service-312caf2d/tokenizer-migration-path-test-kserve-router-scheduler-5785sxtvr" podUID="79fa0e66-4e36-4c58-8031-58515d1978fd" containerName="main" probeResult="failure" output="service unhealthy (responded with \"NOT_SERVING\")"
Sep 17 23:36:04.211895 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:36:04.211870    2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-test-llm-inference-service-312caf2d/tokenizer-migration-path-test-kserve-router-scheduler-5785sxtvr"
Sep 17 23:36:04.269723 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:36:04.269689    2562 generic.go:356] "Generic (PLEG): container finished" podID="79fa0e66-4e36-4c58-8031-58515d1978fd" containerID="adea9337c68b956e61ef471921d10068dd22bd21bca29e8e0438a58fe61c4405" exitCode=0
Sep 17 23:36:04.269873 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:36:04.269738    2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-test-llm-inference-service-312caf2d/tokenizer-migration-path-test-kserve-router-scheduler-5785sxtvr"
Sep 17 23:36:04.269873 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:36:04.269751    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-test-llm-inference-service-312caf2d/tokenizer-migration-path-test-kserve-router-scheduler-5785sxtvr" event={"ID":"79fa0e66-4e36-4c58-8031-58515d1978fd","Type":"ContainerDied","Data":"adea9337c68b956e61ef471921d10068dd22bd21bca29e8e0438a58fe61c4405"}
Sep 17 23:36:04.269873 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:36:04.269801    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-test-llm-inference-service-312caf2d/tokenizer-migration-path-test-kserve-router-scheduler-5785sxtvr" event={"ID":"79fa0e66-4e36-4c58-8031-58515d1978fd","Type":"ContainerDied","Data":"98ff4d1c5ff45af9566866256c22b5d389fea9da5ffb46a3f3dd016c3f007cb7"}
Sep 17 23:36:04.269873 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:36:04.269832    2562 scope.go:117] "RemoveContainer" containerID="adea9337c68b956e61ef471921d10068dd22bd21bca29e8e0438a58fe61c4405"
Sep 17 23:36:04.277822 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:36:04.277803    2562 scope.go:117] "RemoveContainer" containerID="adea9337c68b956e61ef471921d10068dd22bd21bca29e8e0438a58fe61c4405"
Sep 17 23:36:04.278064 ip-10-0-132-188 kubenswrapper[2562]: E0917 23:36:04.278043    2562 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"adea9337c68b956e61ef471921d10068dd22bd21bca29e8e0438a58fe61c4405\": container with ID starting with adea9337c68b956e61ef471921d10068dd22bd21bca29e8e0438a58fe61c4405 not found: ID does not exist" containerID="adea9337c68b956e61ef471921d10068dd22bd21bca29e8e0438a58fe61c4405"
Sep 17 23:36:04.278128 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:36:04.278075    2562 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"adea9337c68b956e61ef471921d10068dd22bd21bca29e8e0438a58fe61c4405"} err="failed to get container status \"adea9337c68b956e61ef471921d10068dd22bd21bca29e8e0438a58fe61c4405\": rpc error: code = NotFound desc = could not find container \"adea9337c68b956e61ef471921d10068dd22bd21bca29e8e0438a58fe61c4405\": container with ID starting with adea9337c68b956e61ef471921d10068dd22bd21bca29e8e0438a58fe61c4405 not found: ID does not exist"
Sep 17 23:36:04.309390 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:36:04.309367    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-wj47w\" (UniqueName: \"kubernetes.io/projected/79fa0e66-4e36-4c58-8031-58515d1978fd-kube-api-access-wj47w\") pod \"79fa0e66-4e36-4c58-8031-58515d1978fd\" (UID: \"79fa0e66-4e36-4c58-8031-58515d1978fd\") "
Sep 17 23:36:04.309485 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:36:04.309476    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/79fa0e66-4e36-4c58-8031-58515d1978fd-tls-certs\") pod \"79fa0e66-4e36-4c58-8031-58515d1978fd\" (UID: \"79fa0e66-4e36-4c58-8031-58515d1978fd\") "
Sep 17 23:36:04.311452 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:36:04.311423    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/79fa0e66-4e36-4c58-8031-58515d1978fd-kube-api-access-wj47w" (OuterVolumeSpecName: "kube-api-access-wj47w") pod "79fa0e66-4e36-4c58-8031-58515d1978fd" (UID: "79fa0e66-4e36-4c58-8031-58515d1978fd"). InnerVolumeSpecName "kube-api-access-wj47w". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 17 23:36:04.311535 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:36:04.311462    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/79fa0e66-4e36-4c58-8031-58515d1978fd-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "79fa0e66-4e36-4c58-8031-58515d1978fd" (UID: "79fa0e66-4e36-4c58-8031-58515d1978fd"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 17 23:36:04.410860 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:36:04.410824    2562 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/79fa0e66-4e36-4c58-8031-58515d1978fd-tls-certs\") on node \"ip-10-0-132-188.ec2.internal\" DevicePath \"\""
Sep 17 23:36:04.410860 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:36:04.410855    2562 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-wj47w\" (UniqueName: \"kubernetes.io/projected/79fa0e66-4e36-4c58-8031-58515d1978fd-kube-api-access-wj47w\") on node \"ip-10-0-132-188.ec2.internal\" DevicePath \"\""
Sep 17 23:36:04.562611 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:36:04.562533    2562 prober.go:120] "Probe failed" probeType="Readiness" pod="e2e-test-llm-auth-invalid-token-rejected-e7cfa2c6/auth-invalid-token-test-kserve-router-scheduler-6dc5d5cf9cr8d89" podUID="d7751c60-cefb-4b89-93f5-f6037946e231" containerName="main" probeResult="failure" output="service unhealthy (responded with \"NOT_SERVING\")"
Sep 17 23:36:04.590587 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:36:04.590556    2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["e2e-test-llm-inference-service-312caf2d/tokenizer-migration-path-test-kserve-router-scheduler-5785sxtvr"]
Sep 17 23:36:04.594508 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:36:04.594483    2562 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["e2e-test-llm-inference-service-312caf2d/tokenizer-migration-path-test-kserve-router-scheduler-5785sxtvr"]
Sep 17 23:36:05.667577 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:36:05.667546    2562 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="79fa0e66-4e36-4c58-8031-58515d1978fd" path="/var/lib/kubelet/pods/79fa0e66-4e36-4c58-8031-58515d1978fd/volumes"
Sep 17 23:36:09.073081 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:36:09.073041    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fwwc5_7e452bbb-7dc6-4c9d-a9b1-76400942f5b4/konnectivity-agent/0.log" limit=1024
Sep 17 23:36:09.169915 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:36:09.169878    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fwwc5_7e452bbb-7dc6-4c9d-a9b1-76400942f5b4/konnectivity-agent/0.log" limit=1024
Sep 17 23:36:14.562642 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:36:14.562608    2562 prober.go:120] "Probe failed" probeType="Readiness" pod="e2e-test-llm-auth-invalid-token-rejected-e7cfa2c6/auth-invalid-token-test-kserve-router-scheduler-6dc5d5cf9cr8d89" podUID="d7751c60-cefb-4b89-93f5-f6037946e231" containerName="main" probeResult="failure" output="service unhealthy (responded with \"NOT_SERVING\")"
Sep 17 23:36:15.106528 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:36:15.106492    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fwwc5_7e452bbb-7dc6-4c9d-a9b1-76400942f5b4/konnectivity-agent/0.log" limit=1024
Sep 17 23:36:19.597617 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:36:19.597591    2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-test-llm-auth-invalid-token-rejected-e7cfa2c6/auth-invalid-token-test-kserve-router-scheduler-6dc5d5cf9cr8d89"
Sep 17 23:36:19.745781 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:36:19.745692    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/d7751c60-cefb-4b89-93f5-f6037946e231-tls-certs\") pod \"d7751c60-cefb-4b89-93f5-f6037946e231\" (UID: \"d7751c60-cefb-4b89-93f5-f6037946e231\") "
Sep 17 23:36:19.745781 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:36:19.745775    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-kjwkb\" (UniqueName: \"kubernetes.io/projected/d7751c60-cefb-4b89-93f5-f6037946e231-kube-api-access-kjwkb\") pod \"d7751c60-cefb-4b89-93f5-f6037946e231\" (UID: \"d7751c60-cefb-4b89-93f5-f6037946e231\") "
Sep 17 23:36:19.747996 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:36:19.747961    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/d7751c60-cefb-4b89-93f5-f6037946e231-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "d7751c60-cefb-4b89-93f5-f6037946e231" (UID: "d7751c60-cefb-4b89-93f5-f6037946e231"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 17 23:36:19.748122 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:36:19.748029    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d7751c60-cefb-4b89-93f5-f6037946e231-kube-api-access-kjwkb" (OuterVolumeSpecName: "kube-api-access-kjwkb") pod "d7751c60-cefb-4b89-93f5-f6037946e231" (UID: "d7751c60-cefb-4b89-93f5-f6037946e231"). InnerVolumeSpecName "kube-api-access-kjwkb". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 17 23:36:19.846618 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:36:19.846581    2562 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/d7751c60-cefb-4b89-93f5-f6037946e231-tls-certs\") on node \"ip-10-0-132-188.ec2.internal\" DevicePath \"\""
Sep 17 23:36:19.846618 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:36:19.846614    2562 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-kjwkb\" (UniqueName: \"kubernetes.io/projected/d7751c60-cefb-4b89-93f5-f6037946e231-kube-api-access-kjwkb\") on node \"ip-10-0-132-188.ec2.internal\" DevicePath \"\""
Sep 17 23:36:20.347055 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:36:20.347019    2562 generic.go:356] "Generic (PLEG): container finished" podID="d7751c60-cefb-4b89-93f5-f6037946e231" containerID="891b395790c7bbbbf342b280d386da475471d68b452ffcae9d0345687d8584d6" exitCode=0
Sep 17 23:36:20.347257 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:36:20.347060    2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-test-llm-auth-invalid-token-rejected-e7cfa2c6/auth-invalid-token-test-kserve-router-scheduler-6dc5d5cf9cr8d89"
Sep 17 23:36:20.347257 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:36:20.347098    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-test-llm-auth-invalid-token-rejected-e7cfa2c6/auth-invalid-token-test-kserve-router-scheduler-6dc5d5cf9cr8d89" event={"ID":"d7751c60-cefb-4b89-93f5-f6037946e231","Type":"ContainerDied","Data":"891b395790c7bbbbf342b280d386da475471d68b452ffcae9d0345687d8584d6"}
Sep 17 23:36:20.347257 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:36:20.347152    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-test-llm-auth-invalid-token-rejected-e7cfa2c6/auth-invalid-token-test-kserve-router-scheduler-6dc5d5cf9cr8d89" event={"ID":"d7751c60-cefb-4b89-93f5-f6037946e231","Type":"ContainerDied","Data":"5ff74e5ddc25024ebeeea075e9de4001c940444c4be7755aac1e9f08e24d9700"}
Sep 17 23:36:20.347257 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:36:20.347191    2562 scope.go:117] "RemoveContainer" containerID="891b395790c7bbbbf342b280d386da475471d68b452ffcae9d0345687d8584d6"
Sep 17 23:36:20.356021 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:36:20.356000    2562 scope.go:117] "RemoveContainer" containerID="891b395790c7bbbbf342b280d386da475471d68b452ffcae9d0345687d8584d6"
Sep 17 23:36:20.356326 ip-10-0-132-188 kubenswrapper[2562]: E0917 23:36:20.356307    2562 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"891b395790c7bbbbf342b280d386da475471d68b452ffcae9d0345687d8584d6\": container with ID starting with 891b395790c7bbbbf342b280d386da475471d68b452ffcae9d0345687d8584d6 not found: ID does not exist" containerID="891b395790c7bbbbf342b280d386da475471d68b452ffcae9d0345687d8584d6"
Sep 17 23:36:20.356375 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:36:20.356334    2562 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"891b395790c7bbbbf342b280d386da475471d68b452ffcae9d0345687d8584d6"} err="failed to get container status \"891b395790c7bbbbf342b280d386da475471d68b452ffcae9d0345687d8584d6\": rpc error: code = NotFound desc = could not find container \"891b395790c7bbbbf342b280d386da475471d68b452ffcae9d0345687d8584d6\": container with ID starting with 891b395790c7bbbbf342b280d386da475471d68b452ffcae9d0345687d8584d6 not found: ID does not exist"
Sep 17 23:36:20.367117 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:36:20.367091    2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["e2e-test-llm-auth-invalid-token-rejected-e7cfa2c6/auth-invalid-token-test-kserve-router-scheduler-6dc5d5cf9cr8d89"]
Sep 17 23:36:20.370546 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:36:20.370523    2562 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["e2e-test-llm-auth-invalid-token-rejected-e7cfa2c6/auth-invalid-token-test-kserve-router-scheduler-6dc5d5cf9cr8d89"]
Sep 17 23:36:20.566930 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:36:20.566896    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fwwc5_7e452bbb-7dc6-4c9d-a9b1-76400942f5b4/konnectivity-agent/0.log" limit=1024
Sep 17 23:36:20.667718 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:36:20.667632    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fwwc5_7e452bbb-7dc6-4c9d-a9b1-76400942f5b4/konnectivity-agent/0.log" limit=1024
Sep 17 23:36:21.667267 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:36:21.667236    2562 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="d7751c60-cefb-4b89-93f5-f6037946e231" path="/var/lib/kubelet/pods/d7751c60-cefb-4b89-93f5-f6037946e231/volumes"
Sep 17 23:36:23.341844 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:36:23.341804    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fwwc5_7e452bbb-7dc6-4c9d-a9b1-76400942f5b4/konnectivity-agent/0.log" limit=1024
Sep 17 23:36:23.563499 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:36:23.563459    2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["e2e-test-llm-inference-service-0eb24714/tokenizer-template-args-test-kserve-router-scheduler-8cd7dpsvrh"]
Sep 17 23:36:23.563889 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:36:23.563871    2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="79fa0e66-4e36-4c58-8031-58515d1978fd" containerName="main"
Sep 17 23:36:23.564007 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:36:23.563891    2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="79fa0e66-4e36-4c58-8031-58515d1978fd" containerName="main"
Sep 17 23:36:23.564007 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:36:23.563934    2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d7751c60-cefb-4b89-93f5-f6037946e231" containerName="main"
Sep 17 23:36:23.564007 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:36:23.563944    2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="d7751c60-cefb-4b89-93f5-f6037946e231" containerName="main"
Sep 17 23:36:23.564162 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:36:23.564017    2562 memory_manager.go:368] "RemoveStaleState removing state" podUID="d7751c60-cefb-4b89-93f5-f6037946e231" containerName="main"
Sep 17 23:36:23.564162 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:36:23.564032    2562 memory_manager.go:368] "RemoveStaleState removing state" podUID="79fa0e66-4e36-4c58-8031-58515d1978fd" containerName="main"
Sep 17 23:36:23.568588 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:36:23.568560    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-test-llm-inference-service-0eb24714/tokenizer-template-args-test-kserve-router-scheduler-8cd7dpsvrh"
Sep 17 23:36:23.570528 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:36:23.570491    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fwwc5_7e452bbb-7dc6-4c9d-a9b1-76400942f5b4/konnectivity-agent/0.log" limit=1024
Sep 17 23:36:23.572195 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:36:23.572158    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-test-llm-inference-service-0eb24714\"/\"openshift-service-ca.crt\""
Sep 17 23:36:23.572287 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:36:23.572198    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-test-llm-inference-service-0eb24714\"/\"kube-root-ca.crt\""
Sep 17 23:36:23.572287 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:36:23.572230    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-test-llm-inference-service-0eb24714\"/\"tokenizer-template-args-test-epp-sa-dockercfg-gm2tn\""
Sep 17 23:36:23.572427 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:36:23.572158    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-test-llm-inference-service-0eb24714\"/\"tokenizer-template-args-test-kserve-self-signed-certs\""
Sep 17 23:36:23.572427 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:36:23.572372    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-test-llm-inference-service-0eb24714\"/\"default-dockercfg-cz6v9\""
Sep 17 23:36:23.577401 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:36:23.577380    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["e2e-test-llm-inference-service-0eb24714/tokenizer-template-args-test-kserve-router-scheduler-8cd7dpsvrh"]
Sep 17 23:36:23.579401 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:36:23.579374    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/a1269ff0-e6da-461d-af5b-958337b7d39c-tls-certs\") pod \"tokenizer-template-args-test-kserve-router-scheduler-8cd7dpsvrh\" (UID: \"a1269ff0-e6da-461d-af5b-958337b7d39c\") " pod="e2e-test-llm-inference-service-0eb24714/tokenizer-template-args-test-kserve-router-scheduler-8cd7dpsvrh"
Sep 17 23:36:23.579521 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:36:23.579421    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-l2kmd\" (UniqueName: \"kubernetes.io/projected/a1269ff0-e6da-461d-af5b-958337b7d39c-kube-api-access-l2kmd\") pod \"tokenizer-template-args-test-kserve-router-scheduler-8cd7dpsvrh\" (UID: \"a1269ff0-e6da-461d-af5b-958337b7d39c\") " pod="e2e-test-llm-inference-service-0eb24714/tokenizer-template-args-test-kserve-router-scheduler-8cd7dpsvrh"
Sep 17 23:36:23.669673 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:36:23.669589    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fwwc5_7e452bbb-7dc6-4c9d-a9b1-76400942f5b4/konnectivity-agent/0.log" limit=1024
Sep 17 23:36:23.680397 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:36:23.680371    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/a1269ff0-e6da-461d-af5b-958337b7d39c-tls-certs\") pod \"tokenizer-template-args-test-kserve-router-scheduler-8cd7dpsvrh\" (UID: \"a1269ff0-e6da-461d-af5b-958337b7d39c\") " pod="e2e-test-llm-inference-service-0eb24714/tokenizer-template-args-test-kserve-router-scheduler-8cd7dpsvrh"
Sep 17 23:36:23.680509 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:36:23.680414    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-l2kmd\" (UniqueName: \"kubernetes.io/projected/a1269ff0-e6da-461d-af5b-958337b7d39c-kube-api-access-l2kmd\") pod \"tokenizer-template-args-test-kserve-router-scheduler-8cd7dpsvrh\" (UID: \"a1269ff0-e6da-461d-af5b-958337b7d39c\") " pod="e2e-test-llm-inference-service-0eb24714/tokenizer-template-args-test-kserve-router-scheduler-8cd7dpsvrh"
Sep 17 23:36:23.682952 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:36:23.682929    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/a1269ff0-e6da-461d-af5b-958337b7d39c-tls-certs\") pod \"tokenizer-template-args-test-kserve-router-scheduler-8cd7dpsvrh\" (UID: \"a1269ff0-e6da-461d-af5b-958337b7d39c\") " pod="e2e-test-llm-inference-service-0eb24714/tokenizer-template-args-test-kserve-router-scheduler-8cd7dpsvrh"
Sep 17 23:36:23.687706 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:36:23.687683    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-l2kmd\" (UniqueName: \"kubernetes.io/projected/a1269ff0-e6da-461d-af5b-958337b7d39c-kube-api-access-l2kmd\") pod \"tokenizer-template-args-test-kserve-router-scheduler-8cd7dpsvrh\" (UID: \"a1269ff0-e6da-461d-af5b-958337b7d39c\") " pod="e2e-test-llm-inference-service-0eb24714/tokenizer-template-args-test-kserve-router-scheduler-8cd7dpsvrh"
Sep 17 23:36:23.881386 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:36:23.881343    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-test-llm-inference-service-0eb24714/tokenizer-template-args-test-kserve-router-scheduler-8cd7dpsvrh"
Sep 17 23:36:24.095123 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:36:24.095082    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fwwc5_7e452bbb-7dc6-4c9d-a9b1-76400942f5b4/konnectivity-agent/0.log" limit=1024
Sep 17 23:36:24.214828 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:36:24.214798    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["e2e-test-llm-inference-service-0eb24714/tokenizer-template-args-test-kserve-router-scheduler-8cd7dpsvrh"]
Sep 17 23:36:24.217521 ip-10-0-132-188 kubenswrapper[2562]: W0917 23:36:24.217495    2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda1269ff0_e6da_461d_af5b_958337b7d39c.slice/crio-9c9784fd728dd52595bdb9bdb6e4f1a39d3d3b34148fba45d78ffd579bfb677a WatchSource:0}: Error finding container 9c9784fd728dd52595bdb9bdb6e4f1a39d3d3b34148fba45d78ffd579bfb677a: Status 404 returned error can't find the container with id 9c9784fd728dd52595bdb9bdb6e4f1a39d3d3b34148fba45d78ffd579bfb677a
Sep 17 23:36:24.229854 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:36:24.229824    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fwwc5_7e452bbb-7dc6-4c9d-a9b1-76400942f5b4/konnectivity-agent/0.log" limit=1024
Sep 17 23:36:24.362856 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:36:24.362816    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fwwc5_7e452bbb-7dc6-4c9d-a9b1-76400942f5b4/konnectivity-agent/0.log" limit=1024
Sep 17 23:36:24.372031 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:36:24.371898    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-test-llm-inference-service-0eb24714/tokenizer-template-args-test-kserve-router-scheduler-8cd7dpsvrh" event={"ID":"a1269ff0-e6da-461d-af5b-958337b7d39c","Type":"ContainerStarted","Data":"8aa2447cd2bb54fc286b94620eb632c43f297535e1d4917bba99940757908427"}
Sep 17 23:36:24.372241 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:36:24.372069    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-test-llm-inference-service-0eb24714/tokenizer-template-args-test-kserve-router-scheduler-8cd7dpsvrh"
Sep 17 23:36:24.372241 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:36:24.372117    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-test-llm-inference-service-0eb24714/tokenizer-template-args-test-kserve-router-scheduler-8cd7dpsvrh" event={"ID":"a1269ff0-e6da-461d-af5b-958337b7d39c","Type":"ContainerStarted","Data":"9c9784fd728dd52595bdb9bdb6e4f1a39d3d3b34148fba45d78ffd579bfb677a"}
Sep 17 23:36:24.391350 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:36:24.391298    2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="e2e-test-llm-inference-service-0eb24714/tokenizer-template-args-test-kserve-router-scheduler-8cd7dpsvrh" podStartSLOduration=1.391259694 podStartE2EDuration="1.391259694s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-17 23:36:23 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-17 23:36:24.389589533 +0000 UTC m=+1536.791486295" watchObservedRunningTime="2026-09-17 23:36:24.391259694 +0000 UTC m=+1536.793156486"
Sep 17 23:36:29.916384 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:36:29.916346    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fwwc5_7e452bbb-7dc6-4c9d-a9b1-76400942f5b4/konnectivity-agent/0.log" limit=1024
Sep 17 23:36:35.376961 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:36:35.376919    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fwwc5_7e452bbb-7dc6-4c9d-a9b1-76400942f5b4/konnectivity-agent/0.log" limit=1024
Sep 17 23:36:35.443653 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:36:35.443615    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fwwc5_7e452bbb-7dc6-4c9d-a9b1-76400942f5b4/konnectivity-agent/0.log" limit=1024
Sep 17 23:36:39.520402 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:36:39.520368    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fwwc5_7e452bbb-7dc6-4c9d-a9b1-76400942f5b4/konnectivity-agent/0.log" limit=1024
Sep 17 23:36:55.378197 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:36:55.378151    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-test-llm-inference-service-0eb24714/tokenizer-template-args-test-kserve-router-scheduler-8cd7dpsvrh"
Sep 17 23:36:58.548628 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:36:58.548577    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fwwc5_7e452bbb-7dc6-4c9d-a9b1-76400942f5b4/konnectivity-agent/0.log" limit=1024
Sep 17 23:36:58.716100 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:36:58.716063    2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["e2e-test-llm-auth-disabled-no-token-required-396e4821/auth-disabled-test-kserve-router-scheduler-596d9bb85f-7vh22"]
Sep 17 23:36:58.721304 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:36:58.721275    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-test-llm-auth-disabled-no-token-required-396e4821/auth-disabled-test-kserve-router-scheduler-596d9bb85f-7vh22"
Sep 17 23:36:58.724594 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:36:58.724568    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-test-llm-auth-disabled-no-token-required-396e4821\"/\"auth-disabled-test-kserve-self-signed-certs\""
Sep 17 23:36:58.724847 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:36:58.724820    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-test-llm-auth-disabled-no-token-required-396e4821\"/\"default-dockercfg-ctv8h\""
Sep 17 23:36:58.724978 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:36:58.724937    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-test-llm-auth-disabled-no-token-required-396e4821\"/\"openshift-service-ca.crt\""
Sep 17 23:36:58.725063 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:36:58.725051    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-test-llm-auth-disabled-no-token-required-396e4821\"/\"auth-disabled-test-epp-sa-dockercfg-j6g6k\""
Sep 17 23:36:58.725210 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:36:58.725161    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-test-llm-auth-disabled-no-token-required-396e4821\"/\"kube-root-ca.crt\""
Sep 17 23:36:58.727093 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:36:58.727072    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["e2e-test-llm-auth-disabled-no-token-required-396e4821/auth-disabled-test-kserve-router-scheduler-596d9bb85f-7vh22"]
Sep 17 23:36:58.740577 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:36:58.740549    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fwwc5_7e452bbb-7dc6-4c9d-a9b1-76400942f5b4/konnectivity-agent/0.log" limit=1024
Sep 17 23:36:58.889231 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:36:58.889194    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-h9cbg\" (UniqueName: \"kubernetes.io/projected/1f97e64b-189a-4a2f-898b-5f156bc0c0ab-kube-api-access-h9cbg\") pod \"auth-disabled-test-kserve-router-scheduler-596d9bb85f-7vh22\" (UID: \"1f97e64b-189a-4a2f-898b-5f156bc0c0ab\") " pod="e2e-test-llm-auth-disabled-no-token-required-396e4821/auth-disabled-test-kserve-router-scheduler-596d9bb85f-7vh22"
Sep 17 23:36:58.889231 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:36:58.889233    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/1f97e64b-189a-4a2f-898b-5f156bc0c0ab-tls-certs\") pod \"auth-disabled-test-kserve-router-scheduler-596d9bb85f-7vh22\" (UID: \"1f97e64b-189a-4a2f-898b-5f156bc0c0ab\") " pod="e2e-test-llm-auth-disabled-no-token-required-396e4821/auth-disabled-test-kserve-router-scheduler-596d9bb85f-7vh22"
Sep 17 23:36:58.990195 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:36:58.990130    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-h9cbg\" (UniqueName: \"kubernetes.io/projected/1f97e64b-189a-4a2f-898b-5f156bc0c0ab-kube-api-access-h9cbg\") pod \"auth-disabled-test-kserve-router-scheduler-596d9bb85f-7vh22\" (UID: \"1f97e64b-189a-4a2f-898b-5f156bc0c0ab\") " pod="e2e-test-llm-auth-disabled-no-token-required-396e4821/auth-disabled-test-kserve-router-scheduler-596d9bb85f-7vh22"
Sep 17 23:36:58.990195 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:36:58.990194    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/1f97e64b-189a-4a2f-898b-5f156bc0c0ab-tls-certs\") pod \"auth-disabled-test-kserve-router-scheduler-596d9bb85f-7vh22\" (UID: \"1f97e64b-189a-4a2f-898b-5f156bc0c0ab\") " pod="e2e-test-llm-auth-disabled-no-token-required-396e4821/auth-disabled-test-kserve-router-scheduler-596d9bb85f-7vh22"
Sep 17 23:36:58.992698 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:36:58.992667    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/1f97e64b-189a-4a2f-898b-5f156bc0c0ab-tls-certs\") pod \"auth-disabled-test-kserve-router-scheduler-596d9bb85f-7vh22\" (UID: \"1f97e64b-189a-4a2f-898b-5f156bc0c0ab\") " pod="e2e-test-llm-auth-disabled-no-token-required-396e4821/auth-disabled-test-kserve-router-scheduler-596d9bb85f-7vh22"
Sep 17 23:36:58.997924 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:36:58.997900    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-h9cbg\" (UniqueName: \"kubernetes.io/projected/1f97e64b-189a-4a2f-898b-5f156bc0c0ab-kube-api-access-h9cbg\") pod \"auth-disabled-test-kserve-router-scheduler-596d9bb85f-7vh22\" (UID: \"1f97e64b-189a-4a2f-898b-5f156bc0c0ab\") " pod="e2e-test-llm-auth-disabled-no-token-required-396e4821/auth-disabled-test-kserve-router-scheduler-596d9bb85f-7vh22"
Sep 17 23:36:59.033097 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:36:59.033037    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-test-llm-auth-disabled-no-token-required-396e4821/auth-disabled-test-kserve-router-scheduler-596d9bb85f-7vh22"
Sep 17 23:36:59.305467 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:36:59.305444    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["e2e-test-llm-auth-disabled-no-token-required-396e4821/auth-disabled-test-kserve-router-scheduler-596d9bb85f-7vh22"]
Sep 17 23:36:59.308126 ip-10-0-132-188 kubenswrapper[2562]: W0917 23:36:59.308092    2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1f97e64b_189a_4a2f_898b_5f156bc0c0ab.slice/crio-10a0c3a7f16ac666f3e896c112f5987b854aeaadaddcc41c3e5bd26efb545191 WatchSource:0}: Error finding container 10a0c3a7f16ac666f3e896c112f5987b854aeaadaddcc41c3e5bd26efb545191: Status 404 returned error can't find the container with id 10a0c3a7f16ac666f3e896c112f5987b854aeaadaddcc41c3e5bd26efb545191
Sep 17 23:36:59.542007 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:36:59.541974    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-test-llm-auth-disabled-no-token-required-396e4821/auth-disabled-test-kserve-router-scheduler-596d9bb85f-7vh22" event={"ID":"1f97e64b-189a-4a2f-898b-5f156bc0c0ab","Type":"ContainerStarted","Data":"48fb04eacabae233d5e64328aec2cce08f5e83401c7b19e7e24440c9faefb92c"}
Sep 17 23:36:59.542270 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:36:59.542024    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-test-llm-auth-disabled-no-token-required-396e4821/auth-disabled-test-kserve-router-scheduler-596d9bb85f-7vh22" event={"ID":"1f97e64b-189a-4a2f-898b-5f156bc0c0ab","Type":"ContainerStarted","Data":"10a0c3a7f16ac666f3e896c112f5987b854aeaadaddcc41c3e5bd26efb545191"}
Sep 17 23:36:59.542270 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:36:59.542146    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-test-llm-auth-disabled-no-token-required-396e4821/auth-disabled-test-kserve-router-scheduler-596d9bb85f-7vh22"
Sep 17 23:36:59.561830 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:36:59.561704    2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="e2e-test-llm-auth-disabled-no-token-required-396e4821/auth-disabled-test-kserve-router-scheduler-596d9bb85f-7vh22" podStartSLOduration=1.561685762 podStartE2EDuration="1.561685762s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-17 23:36:58 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-17 23:36:59.560278456 +0000 UTC m=+1571.962175231" watchObservedRunningTime="2026-09-17 23:36:59.561685762 +0000 UTC m=+1571.963582526"
Sep 17 23:37:20.584518 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:37:20.584478    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fwwc5_7e452bbb-7dc6-4c9d-a9b1-76400942f5b4/konnectivity-agent/0.log" limit=1024
Sep 17 23:37:22.241519 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:37:22.241482    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fwwc5_7e452bbb-7dc6-4c9d-a9b1-76400942f5b4/konnectivity-agent/0.log" limit=1024
Sep 17 23:37:22.252752 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:37:22.252721    2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["e2e-test-llm-inference-service-0eb24714/tokenizer-template-args-test-kserve-router-scheduler-8cd7dpsvrh"]
Sep 17 23:37:22.307161 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:37:22.307111    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fwwc5_7e452bbb-7dc6-4c9d-a9b1-76400942f5b4/konnectivity-agent/0.log" limit=1024
Sep 17 23:37:25.378209 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:37:25.378150    2562 prober.go:120] "Probe failed" probeType="Readiness" pod="e2e-test-llm-inference-service-0eb24714/tokenizer-template-args-test-kserve-router-scheduler-8cd7dpsvrh" podUID="a1269ff0-e6da-461d-af5b-958337b7d39c" containerName="main" probeResult="failure" output="service unhealthy (responded with \"NOT_SERVING\")"
Sep 17 23:37:28.353080 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:37:28.353039    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fwwc5_7e452bbb-7dc6-4c9d-a9b1-76400942f5b4/konnectivity-agent/0.log" limit=1024
Sep 17 23:37:28.453606 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:37:28.453573    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fwwc5_7e452bbb-7dc6-4c9d-a9b1-76400942f5b4/konnectivity-agent/0.log" limit=1024
Sep 17 23:37:30.548601 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:37:30.548570    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-test-llm-auth-disabled-no-token-required-396e4821/auth-disabled-test-kserve-router-scheduler-596d9bb85f-7vh22"
Sep 17 23:37:35.378578 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:37:35.378499    2562 prober.go:120] "Probe failed" probeType="Readiness" pod="e2e-test-llm-inference-service-0eb24714/tokenizer-template-args-test-kserve-router-scheduler-8cd7dpsvrh" podUID="a1269ff0-e6da-461d-af5b-958337b7d39c" containerName="main" probeResult="failure" output="service unhealthy (responded with \"NOT_SERVING\")"
Sep 17 23:37:37.273075 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:37:37.273039    2562 kuberuntime_container.go:873] "Killing container with a grace period" pod="e2e-test-llm-inference-service-0eb24714/tokenizer-template-args-test-kserve-router-scheduler-8cd7dpsvrh" podUID="a1269ff0-e6da-461d-af5b-958337b7d39c" containerName="main" containerID="cri-o://8aa2447cd2bb54fc286b94620eb632c43f297535e1d4917bba99940757908427" gracePeriod=45
Sep 17 23:37:45.378929 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:37:45.378889    2562 prober.go:120] "Probe failed" probeType="Readiness" pod="e2e-test-llm-inference-service-0eb24714/tokenizer-template-args-test-kserve-router-scheduler-8cd7dpsvrh" podUID="a1269ff0-e6da-461d-af5b-958337b7d39c" containerName="main" probeResult="failure" output="service unhealthy (responded with \"NOT_SERVING\")"
Sep 17 23:37:45.379330 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:37:45.378996    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-test-llm-inference-service-0eb24714/tokenizer-template-args-test-kserve-router-scheduler-8cd7dpsvrh"
Sep 17 23:37:55.379066 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:37:55.379024    2562 prober.go:120] "Probe failed" probeType="Readiness" pod="e2e-test-llm-inference-service-0eb24714/tokenizer-template-args-test-kserve-router-scheduler-8cd7dpsvrh" podUID="a1269ff0-e6da-461d-af5b-958337b7d39c" containerName="main" probeResult="failure" output="service unhealthy (responded with \"NOT_SERVING\")"
Sep 17 23:38:05.378861 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:38:05.378819    2562 prober.go:120] "Probe failed" probeType="Readiness" pod="e2e-test-llm-inference-service-0eb24714/tokenizer-template-args-test-kserve-router-scheduler-8cd7dpsvrh" podUID="a1269ff0-e6da-461d-af5b-958337b7d39c" containerName="main" probeResult="failure" output="service unhealthy (responded with \"NOT_SERVING\")"
Sep 17 23:38:07.513835 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:38:07.513810    2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-test-llm-inference-service-0eb24714/tokenizer-template-args-test-kserve-router-scheduler-8cd7dpsvrh"
Sep 17 23:38:07.681417 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:38:07.681385    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-l2kmd\" (UniqueName: \"kubernetes.io/projected/a1269ff0-e6da-461d-af5b-958337b7d39c-kube-api-access-l2kmd\") pod \"a1269ff0-e6da-461d-af5b-958337b7d39c\" (UID: \"a1269ff0-e6da-461d-af5b-958337b7d39c\") "
Sep 17 23:38:07.681581 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:38:07.681456    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/a1269ff0-e6da-461d-af5b-958337b7d39c-tls-certs\") pod \"a1269ff0-e6da-461d-af5b-958337b7d39c\" (UID: \"a1269ff0-e6da-461d-af5b-958337b7d39c\") "
Sep 17 23:38:07.683602 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:38:07.683572    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/a1269ff0-e6da-461d-af5b-958337b7d39c-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "a1269ff0-e6da-461d-af5b-958337b7d39c" (UID: "a1269ff0-e6da-461d-af5b-958337b7d39c"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 17 23:38:07.683747 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:38:07.683705    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a1269ff0-e6da-461d-af5b-958337b7d39c-kube-api-access-l2kmd" (OuterVolumeSpecName: "kube-api-access-l2kmd") pod "a1269ff0-e6da-461d-af5b-958337b7d39c" (UID: "a1269ff0-e6da-461d-af5b-958337b7d39c"). InnerVolumeSpecName "kube-api-access-l2kmd". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 17 23:38:07.782374 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:38:07.782335    2562 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/a1269ff0-e6da-461d-af5b-958337b7d39c-tls-certs\") on node \"ip-10-0-132-188.ec2.internal\" DevicePath \"\""
Sep 17 23:38:07.782374 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:38:07.782368    2562 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-l2kmd\" (UniqueName: \"kubernetes.io/projected/a1269ff0-e6da-461d-af5b-958337b7d39c-kube-api-access-l2kmd\") on node \"ip-10-0-132-188.ec2.internal\" DevicePath \"\""
Sep 17 23:38:07.854566 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:38:07.854529    2562 generic.go:356] "Generic (PLEG): container finished" podID="a1269ff0-e6da-461d-af5b-958337b7d39c" containerID="8aa2447cd2bb54fc286b94620eb632c43f297535e1d4917bba99940757908427" exitCode=0
Sep 17 23:38:07.854566 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:38:07.854565    2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-test-llm-inference-service-0eb24714/tokenizer-template-args-test-kserve-router-scheduler-8cd7dpsvrh"
Sep 17 23:38:07.854792 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:38:07.854600    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-test-llm-inference-service-0eb24714/tokenizer-template-args-test-kserve-router-scheduler-8cd7dpsvrh" event={"ID":"a1269ff0-e6da-461d-af5b-958337b7d39c","Type":"ContainerDied","Data":"8aa2447cd2bb54fc286b94620eb632c43f297535e1d4917bba99940757908427"}
Sep 17 23:38:07.854792 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:38:07.854651    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-test-llm-inference-service-0eb24714/tokenizer-template-args-test-kserve-router-scheduler-8cd7dpsvrh" event={"ID":"a1269ff0-e6da-461d-af5b-958337b7d39c","Type":"ContainerDied","Data":"9c9784fd728dd52595bdb9bdb6e4f1a39d3d3b34148fba45d78ffd579bfb677a"}
Sep 17 23:38:07.854792 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:38:07.854672    2562 scope.go:117] "RemoveContainer" containerID="8aa2447cd2bb54fc286b94620eb632c43f297535e1d4917bba99940757908427"
Sep 17 23:38:07.864104 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:38:07.864087    2562 scope.go:117] "RemoveContainer" containerID="8aa2447cd2bb54fc286b94620eb632c43f297535e1d4917bba99940757908427"
Sep 17 23:38:07.864366 ip-10-0-132-188 kubenswrapper[2562]: E0917 23:38:07.864347    2562 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"8aa2447cd2bb54fc286b94620eb632c43f297535e1d4917bba99940757908427\": container with ID starting with 8aa2447cd2bb54fc286b94620eb632c43f297535e1d4917bba99940757908427 not found: ID does not exist" containerID="8aa2447cd2bb54fc286b94620eb632c43f297535e1d4917bba99940757908427"
Sep 17 23:38:07.864419 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:38:07.864374    2562 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"8aa2447cd2bb54fc286b94620eb632c43f297535e1d4917bba99940757908427"} err="failed to get container status \"8aa2447cd2bb54fc286b94620eb632c43f297535e1d4917bba99940757908427\": rpc error: code = NotFound desc = could not find container \"8aa2447cd2bb54fc286b94620eb632c43f297535e1d4917bba99940757908427\": container with ID starting with 8aa2447cd2bb54fc286b94620eb632c43f297535e1d4917bba99940757908427 not found: ID does not exist"
Sep 17 23:38:07.876027 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:38:07.876006    2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["e2e-test-llm-inference-service-0eb24714/tokenizer-template-args-test-kserve-router-scheduler-8cd7dpsvrh"]
Sep 17 23:38:07.880116 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:38:07.880093    2562 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["e2e-test-llm-inference-service-0eb24714/tokenizer-template-args-test-kserve-router-scheduler-8cd7dpsvrh"]
Sep 17 23:38:09.667029 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:38:09.666998    2562 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="a1269ff0-e6da-461d-af5b-958337b7d39c" path="/var/lib/kubelet/pods/a1269ff0-e6da-461d-af5b-958337b7d39c/volumes"
Sep 17 23:38:12.722035 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:38:12.721995    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fwwc5_7e452bbb-7dc6-4c9d-a9b1-76400942f5b4/konnectivity-agent/0.log" limit=1024
Sep 17 23:38:12.816297 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:38:12.816254    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fwwc5_7e452bbb-7dc6-4c9d-a9b1-76400942f5b4/konnectivity-agent/0.log" limit=1024
Sep 17 23:38:18.770087 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:38:18.770030    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fwwc5_7e452bbb-7dc6-4c9d-a9b1-76400942f5b4/konnectivity-agent/0.log" limit=1024
Sep 17 23:38:21.693643 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:38:21.693610    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fwwc5_7e452bbb-7dc6-4c9d-a9b1-76400942f5b4/konnectivity-agent/0.log" limit=1024
Sep 17 23:38:21.874549 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:38:21.874516    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fwwc5_7e452bbb-7dc6-4c9d-a9b1-76400942f5b4/konnectivity-agent/0.log" limit=1024
Sep 17 23:38:24.420738 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:38:24.420695    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fwwc5_7e452bbb-7dc6-4c9d-a9b1-76400942f5b4/konnectivity-agent/0.log" limit=1024
Sep 17 23:38:24.521344 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:38:24.521296    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fwwc5_7e452bbb-7dc6-4c9d-a9b1-76400942f5b4/konnectivity-agent/0.log" limit=1024
Sep 17 23:38:24.820204 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:38:24.820091    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fwwc5_7e452bbb-7dc6-4c9d-a9b1-76400942f5b4/konnectivity-agent/0.log" limit=1024
Sep 17 23:38:44.622986 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:38:44.622950    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fwwc5_7e452bbb-7dc6-4c9d-a9b1-76400942f5b4/konnectivity-agent/0.log" limit=1024
Sep 17 23:38:54.845768 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:38:54.845721    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fwwc5_7e452bbb-7dc6-4c9d-a9b1-76400942f5b4/konnectivity-agent/0.log" limit=1024
Sep 17 23:38:54.936459 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:38:54.936422    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fwwc5_7e452bbb-7dc6-4c9d-a9b1-76400942f5b4/konnectivity-agent/0.log" limit=1024
Sep 17 23:39:03.041185 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:39:03.041079    2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["e2e-test-llm-auth-disabled-no-token-required-396e4821/auth-disabled-test-kserve-router-scheduler-596d9bb85f-7vh22"]
Sep 17 23:39:03.045731 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:39:03.045687    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fwwc5_7e452bbb-7dc6-4c9d-a9b1-76400942f5b4/konnectivity-agent/0.log" limit=1024
Sep 17 23:39:03.102835 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:39:03.102798    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fwwc5_7e452bbb-7dc6-4c9d-a9b1-76400942f5b4/konnectivity-agent/0.log" limit=1024
Sep 17 23:39:10.548819 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:39:10.548783    2562 prober.go:120] "Probe failed" probeType="Readiness" pod="e2e-test-llm-auth-disabled-no-token-required-396e4821/auth-disabled-test-kserve-router-scheduler-596d9bb85f-7vh22" podUID="1f97e64b-189a-4a2f-898b-5f156bc0c0ab" containerName="main" probeResult="failure" output="service unhealthy (responded with \"NOT_SERVING\")"
Sep 17 23:39:18.062453 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:39:18.062385    2562 kuberuntime_container.go:873] "Killing container with a grace period" pod="e2e-test-llm-auth-disabled-no-token-required-396e4821/auth-disabled-test-kserve-router-scheduler-596d9bb85f-7vh22" podUID="1f97e64b-189a-4a2f-898b-5f156bc0c0ab" containerName="main" containerID="cri-o://48fb04eacabae233d5e64328aec2cce08f5e83401c7b19e7e24440c9faefb92c" gracePeriod=45
Sep 17 23:39:20.549050 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:39:20.549010    2562 prober.go:120] "Probe failed" probeType="Readiness" pod="e2e-test-llm-auth-disabled-no-token-required-396e4821/auth-disabled-test-kserve-router-scheduler-596d9bb85f-7vh22" podUID="1f97e64b-189a-4a2f-898b-5f156bc0c0ab" containerName="main" probeResult="failure" output="service unhealthy (responded with \"NOT_SERVING\")"
Sep 17 23:39:30.548532 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:39:30.548485    2562 prober.go:120] "Probe failed" probeType="Readiness" pod="e2e-test-llm-auth-disabled-no-token-required-396e4821/auth-disabled-test-kserve-router-scheduler-596d9bb85f-7vh22" podUID="1f97e64b-189a-4a2f-898b-5f156bc0c0ab" containerName="main" probeResult="failure" output="service unhealthy (responded with \"NOT_SERVING\")"
Sep 17 23:39:30.548921 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:39:30.548593    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-test-llm-auth-disabled-no-token-required-396e4821/auth-disabled-test-kserve-router-scheduler-596d9bb85f-7vh22"
Sep 17 23:39:40.548333 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:39:40.548290    2562 prober.go:120] "Probe failed" probeType="Readiness" pod="e2e-test-llm-auth-disabled-no-token-required-396e4821/auth-disabled-test-kserve-router-scheduler-596d9bb85f-7vh22" podUID="1f97e64b-189a-4a2f-898b-5f156bc0c0ab" containerName="main" probeResult="failure" output="service unhealthy (responded with \"NOT_SERVING\")"
Sep 17 23:39:44.741042 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:39:44.741006    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fwwc5_7e452bbb-7dc6-4c9d-a9b1-76400942f5b4/konnectivity-agent/0.log" limit=1024
Sep 17 23:39:44.837250 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:39:44.837213    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fwwc5_7e452bbb-7dc6-4c9d-a9b1-76400942f5b4/konnectivity-agent/0.log" limit=1024
Sep 17 23:39:48.305156 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:39:48.305124    2562 generic.go:356] "Generic (PLEG): container finished" podID="1f97e64b-189a-4a2f-898b-5f156bc0c0ab" containerID="48fb04eacabae233d5e64328aec2cce08f5e83401c7b19e7e24440c9faefb92c" exitCode=0
Sep 17 23:39:48.305500 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:39:48.305204    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-test-llm-auth-disabled-no-token-required-396e4821/auth-disabled-test-kserve-router-scheduler-596d9bb85f-7vh22" event={"ID":"1f97e64b-189a-4a2f-898b-5f156bc0c0ab","Type":"ContainerDied","Data":"48fb04eacabae233d5e64328aec2cce08f5e83401c7b19e7e24440c9faefb92c"}
Sep 17 23:39:48.322719 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:39:48.322658    2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-test-llm-auth-disabled-no-token-required-396e4821/auth-disabled-test-kserve-router-scheduler-596d9bb85f-7vh22"
Sep 17 23:39:48.408375 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:39:48.408337    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-h9cbg\" (UniqueName: \"kubernetes.io/projected/1f97e64b-189a-4a2f-898b-5f156bc0c0ab-kube-api-access-h9cbg\") pod \"1f97e64b-189a-4a2f-898b-5f156bc0c0ab\" (UID: \"1f97e64b-189a-4a2f-898b-5f156bc0c0ab\") "
Sep 17 23:39:48.408548 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:39:48.408396    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/1f97e64b-189a-4a2f-898b-5f156bc0c0ab-tls-certs\") pod \"1f97e64b-189a-4a2f-898b-5f156bc0c0ab\" (UID: \"1f97e64b-189a-4a2f-898b-5f156bc0c0ab\") "
Sep 17 23:39:48.410677 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:39:48.410647    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/1f97e64b-189a-4a2f-898b-5f156bc0c0ab-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "1f97e64b-189a-4a2f-898b-5f156bc0c0ab" (UID: "1f97e64b-189a-4a2f-898b-5f156bc0c0ab"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 17 23:39:48.410797 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:39:48.410682    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/1f97e64b-189a-4a2f-898b-5f156bc0c0ab-kube-api-access-h9cbg" (OuterVolumeSpecName: "kube-api-access-h9cbg") pod "1f97e64b-189a-4a2f-898b-5f156bc0c0ab" (UID: "1f97e64b-189a-4a2f-898b-5f156bc0c0ab"). InnerVolumeSpecName "kube-api-access-h9cbg". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 17 23:39:48.509679 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:39:48.509639    2562 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-h9cbg\" (UniqueName: \"kubernetes.io/projected/1f97e64b-189a-4a2f-898b-5f156bc0c0ab-kube-api-access-h9cbg\") on node \"ip-10-0-132-188.ec2.internal\" DevicePath \"\""
Sep 17 23:39:48.509679 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:39:48.509672    2562 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/1f97e64b-189a-4a2f-898b-5f156bc0c0ab-tls-certs\") on node \"ip-10-0-132-188.ec2.internal\" DevicePath \"\""
Sep 17 23:39:49.311702 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:39:49.311670    2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-test-llm-auth-disabled-no-token-required-396e4821/auth-disabled-test-kserve-router-scheduler-596d9bb85f-7vh22"
Sep 17 23:39:49.311702 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:39:49.311680    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-test-llm-auth-disabled-no-token-required-396e4821/auth-disabled-test-kserve-router-scheduler-596d9bb85f-7vh22" event={"ID":"1f97e64b-189a-4a2f-898b-5f156bc0c0ab","Type":"ContainerDied","Data":"10a0c3a7f16ac666f3e896c112f5987b854aeaadaddcc41c3e5bd26efb545191"}
Sep 17 23:39:49.312270 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:39:49.311747    2562 scope.go:117] "RemoveContainer" containerID="48fb04eacabae233d5e64328aec2cce08f5e83401c7b19e7e24440c9faefb92c"
Sep 17 23:39:49.331695 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:39:49.331666    2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["e2e-test-llm-auth-disabled-no-token-required-396e4821/auth-disabled-test-kserve-router-scheduler-596d9bb85f-7vh22"]
Sep 17 23:39:49.335902 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:39:49.335866    2562 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["e2e-test-llm-auth-disabled-no-token-required-396e4821/auth-disabled-test-kserve-router-scheduler-596d9bb85f-7vh22"]
Sep 17 23:39:49.666394 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:39:49.666363    2562 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="1f97e64b-189a-4a2f-898b-5f156bc0c0ab" path="/var/lib/kubelet/pods/1f97e64b-189a-4a2f-898b-5f156bc0c0ab/volumes"
Sep 17 23:39:50.486594 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:39:50.486558    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fwwc5_7e452bbb-7dc6-4c9d-a9b1-76400942f5b4/konnectivity-agent/0.log" limit=1024
Sep 17 23:39:53.162723 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:39:53.162688    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fwwc5_7e452bbb-7dc6-4c9d-a9b1-76400942f5b4/konnectivity-agent/0.log" limit=1024
Sep 17 23:39:53.259711 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:39:53.259675    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fwwc5_7e452bbb-7dc6-4c9d-a9b1-76400942f5b4/konnectivity-agent/0.log" limit=1024
Sep 17 23:39:53.334646 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:39:53.334613    2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["e2e-test-llm-inference-service-6f0aa991/router-gateway-1-openshift-default-8555f5d76c-pqrgz"]
Sep 17 23:39:53.335113 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:39:53.335092    2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a1269ff0-e6da-461d-af5b-958337b7d39c" containerName="main"
Sep 17 23:39:53.335212 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:39:53.335119    2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="a1269ff0-e6da-461d-af5b-958337b7d39c" containerName="main"
Sep 17 23:39:53.335212 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:39:53.335133    2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="1f97e64b-189a-4a2f-898b-5f156bc0c0ab" containerName="main"
Sep 17 23:39:53.335212 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:39:53.335144    2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="1f97e64b-189a-4a2f-898b-5f156bc0c0ab" containerName="main"
Sep 17 23:39:53.335385 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:39:53.335302    2562 memory_manager.go:368] "RemoveStaleState removing state" podUID="1f97e64b-189a-4a2f-898b-5f156bc0c0ab" containerName="main"
Sep 17 23:39:53.335385 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:39:53.335317    2562 memory_manager.go:368] "RemoveStaleState removing state" podUID="a1269ff0-e6da-461d-af5b-958337b7d39c" containerName="main"
Sep 17 23:39:53.339725 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:39:53.339703    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-test-llm-inference-service-6f0aa991/router-gateway-1-openshift-default-8555f5d76c-pqrgz"
Sep 17 23:39:53.342471 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:39:53.342448    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-test-llm-inference-service-6f0aa991\"/\"router-gateway-1-openshift-default-dockercfg-9qhjd\""
Sep 17 23:39:53.342618 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:39:53.342506    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-test-llm-inference-service-6f0aa991\"/\"kube-root-ca.crt\""
Sep 17 23:39:53.342618 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:39:53.342546    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-test-llm-inference-service-6f0aa991\"/\"openshift-service-ca.crt\""
Sep 17 23:39:53.342618 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:39:53.342454    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-test-llm-inference-service-6f0aa991\"/\"openshift-gw-ca-root-cert\""
Sep 17 23:39:53.350403 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:39:53.350378    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["e2e-test-llm-inference-service-6f0aa991/router-gateway-1-openshift-default-8555f5d76c-pqrgz"]
Sep 17 23:39:53.369949 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:39:53.369914    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fwwc5_7e452bbb-7dc6-4c9d-a9b1-76400942f5b4/konnectivity-agent/0.log" limit=1024
Sep 17 23:39:53.455349 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:39:53.455253    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/f95f475e-903b-4e31-b5da-c4a7612fe54a-credential-socket\") pod \"router-gateway-1-openshift-default-8555f5d76c-pqrgz\" (UID: \"f95f475e-903b-4e31-b5da-c4a7612fe54a\") " pod="e2e-test-llm-inference-service-6f0aa991/router-gateway-1-openshift-default-8555f5d76c-pqrgz"
Sep 17 23:39:53.455349 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:39:53.455295    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-lf7xq\" (UniqueName: \"kubernetes.io/projected/f95f475e-903b-4e31-b5da-c4a7612fe54a-kube-api-access-lf7xq\") pod \"router-gateway-1-openshift-default-8555f5d76c-pqrgz\" (UID: \"f95f475e-903b-4e31-b5da-c4a7612fe54a\") " pod="e2e-test-llm-inference-service-6f0aa991/router-gateway-1-openshift-default-8555f5d76c-pqrgz"
Sep 17 23:39:53.455349 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:39:53.455322    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/f95f475e-903b-4e31-b5da-c4a7612fe54a-istiod-ca-cert\") pod \"router-gateway-1-openshift-default-8555f5d76c-pqrgz\" (UID: \"f95f475e-903b-4e31-b5da-c4a7612fe54a\") " pod="e2e-test-llm-inference-service-6f0aa991/router-gateway-1-openshift-default-8555f5d76c-pqrgz"
Sep 17 23:39:53.455620 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:39:53.455372    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/f95f475e-903b-4e31-b5da-c4a7612fe54a-istio-token\") pod \"router-gateway-1-openshift-default-8555f5d76c-pqrgz\" (UID: \"f95f475e-903b-4e31-b5da-c4a7612fe54a\") " pod="e2e-test-llm-inference-service-6f0aa991/router-gateway-1-openshift-default-8555f5d76c-pqrgz"
Sep 17 23:39:53.455620 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:39:53.455440    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/f95f475e-903b-4e31-b5da-c4a7612fe54a-workload-socket\") pod \"router-gateway-1-openshift-default-8555f5d76c-pqrgz\" (UID: \"f95f475e-903b-4e31-b5da-c4a7612fe54a\") " pod="e2e-test-llm-inference-service-6f0aa991/router-gateway-1-openshift-default-8555f5d76c-pqrgz"
Sep 17 23:39:53.455620 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:39:53.455472    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/f95f475e-903b-4e31-b5da-c4a7612fe54a-istio-podinfo\") pod \"router-gateway-1-openshift-default-8555f5d76c-pqrgz\" (UID: \"f95f475e-903b-4e31-b5da-c4a7612fe54a\") " pod="e2e-test-llm-inference-service-6f0aa991/router-gateway-1-openshift-default-8555f5d76c-pqrgz"
Sep 17 23:39:53.455620 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:39:53.455542    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/f95f475e-903b-4e31-b5da-c4a7612fe54a-istio-envoy\") pod \"router-gateway-1-openshift-default-8555f5d76c-pqrgz\" (UID: \"f95f475e-903b-4e31-b5da-c4a7612fe54a\") " pod="e2e-test-llm-inference-service-6f0aa991/router-gateway-1-openshift-default-8555f5d76c-pqrgz"
Sep 17 23:39:53.455769 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:39:53.455621    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/f95f475e-903b-4e31-b5da-c4a7612fe54a-workload-certs\") pod \"router-gateway-1-openshift-default-8555f5d76c-pqrgz\" (UID: \"f95f475e-903b-4e31-b5da-c4a7612fe54a\") " pod="e2e-test-llm-inference-service-6f0aa991/router-gateway-1-openshift-default-8555f5d76c-pqrgz"
Sep 17 23:39:53.455769 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:39:53.455669    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/f95f475e-903b-4e31-b5da-c4a7612fe54a-istio-data\") pod \"router-gateway-1-openshift-default-8555f5d76c-pqrgz\" (UID: \"f95f475e-903b-4e31-b5da-c4a7612fe54a\") " pod="e2e-test-llm-inference-service-6f0aa991/router-gateway-1-openshift-default-8555f5d76c-pqrgz"
Sep 17 23:39:53.466997 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:39:53.466967    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fwwc5_7e452bbb-7dc6-4c9d-a9b1-76400942f5b4/konnectivity-agent/0.log" limit=1024
Sep 17 23:39:53.556641 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:39:53.556589    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/f95f475e-903b-4e31-b5da-c4a7612fe54a-workload-socket\") pod \"router-gateway-1-openshift-default-8555f5d76c-pqrgz\" (UID: \"f95f475e-903b-4e31-b5da-c4a7612fe54a\") " pod="e2e-test-llm-inference-service-6f0aa991/router-gateway-1-openshift-default-8555f5d76c-pqrgz"
Sep 17 23:39:53.556641 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:39:53.556649    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/f95f475e-903b-4e31-b5da-c4a7612fe54a-istio-podinfo\") pod \"router-gateway-1-openshift-default-8555f5d76c-pqrgz\" (UID: \"f95f475e-903b-4e31-b5da-c4a7612fe54a\") " pod="e2e-test-llm-inference-service-6f0aa991/router-gateway-1-openshift-default-8555f5d76c-pqrgz"
Sep 17 23:39:53.556886 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:39:53.556693    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/f95f475e-903b-4e31-b5da-c4a7612fe54a-istio-envoy\") pod \"router-gateway-1-openshift-default-8555f5d76c-pqrgz\" (UID: \"f95f475e-903b-4e31-b5da-c4a7612fe54a\") " pod="e2e-test-llm-inference-service-6f0aa991/router-gateway-1-openshift-default-8555f5d76c-pqrgz"
Sep 17 23:39:53.556886 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:39:53.556713    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/f95f475e-903b-4e31-b5da-c4a7612fe54a-workload-certs\") pod \"router-gateway-1-openshift-default-8555f5d76c-pqrgz\" (UID: \"f95f475e-903b-4e31-b5da-c4a7612fe54a\") " pod="e2e-test-llm-inference-service-6f0aa991/router-gateway-1-openshift-default-8555f5d76c-pqrgz"
Sep 17 23:39:53.556886 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:39:53.556731    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/f95f475e-903b-4e31-b5da-c4a7612fe54a-istio-data\") pod \"router-gateway-1-openshift-default-8555f5d76c-pqrgz\" (UID: \"f95f475e-903b-4e31-b5da-c4a7612fe54a\") " pod="e2e-test-llm-inference-service-6f0aa991/router-gateway-1-openshift-default-8555f5d76c-pqrgz"
Sep 17 23:39:53.556886 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:39:53.556760    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/f95f475e-903b-4e31-b5da-c4a7612fe54a-credential-socket\") pod \"router-gateway-1-openshift-default-8555f5d76c-pqrgz\" (UID: \"f95f475e-903b-4e31-b5da-c4a7612fe54a\") " pod="e2e-test-llm-inference-service-6f0aa991/router-gateway-1-openshift-default-8555f5d76c-pqrgz"
Sep 17 23:39:53.556886 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:39:53.556779    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lf7xq\" (UniqueName: \"kubernetes.io/projected/f95f475e-903b-4e31-b5da-c4a7612fe54a-kube-api-access-lf7xq\") pod \"router-gateway-1-openshift-default-8555f5d76c-pqrgz\" (UID: \"f95f475e-903b-4e31-b5da-c4a7612fe54a\") " pod="e2e-test-llm-inference-service-6f0aa991/router-gateway-1-openshift-default-8555f5d76c-pqrgz"
Sep 17 23:39:53.556886 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:39:53.556810    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/f95f475e-903b-4e31-b5da-c4a7612fe54a-istiod-ca-cert\") pod \"router-gateway-1-openshift-default-8555f5d76c-pqrgz\" (UID: \"f95f475e-903b-4e31-b5da-c4a7612fe54a\") " pod="e2e-test-llm-inference-service-6f0aa991/router-gateway-1-openshift-default-8555f5d76c-pqrgz"
Sep 17 23:39:53.556886 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:39:53.556835    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/f95f475e-903b-4e31-b5da-c4a7612fe54a-istio-token\") pod \"router-gateway-1-openshift-default-8555f5d76c-pqrgz\" (UID: \"f95f475e-903b-4e31-b5da-c4a7612fe54a\") " pod="e2e-test-llm-inference-service-6f0aa991/router-gateway-1-openshift-default-8555f5d76c-pqrgz"
Sep 17 23:39:53.557349 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:39:53.557099    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/f95f475e-903b-4e31-b5da-c4a7612fe54a-workload-socket\") pod \"router-gateway-1-openshift-default-8555f5d76c-pqrgz\" (UID: \"f95f475e-903b-4e31-b5da-c4a7612fe54a\") " pod="e2e-test-llm-inference-service-6f0aa991/router-gateway-1-openshift-default-8555f5d76c-pqrgz"
Sep 17 23:39:53.557349 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:39:53.557313    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/f95f475e-903b-4e31-b5da-c4a7612fe54a-istio-data\") pod \"router-gateway-1-openshift-default-8555f5d76c-pqrgz\" (UID: \"f95f475e-903b-4e31-b5da-c4a7612fe54a\") " pod="e2e-test-llm-inference-service-6f0aa991/router-gateway-1-openshift-default-8555f5d76c-pqrgz"
Sep 17 23:39:53.557435 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:39:53.557345    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/f95f475e-903b-4e31-b5da-c4a7612fe54a-credential-socket\") pod \"router-gateway-1-openshift-default-8555f5d76c-pqrgz\" (UID: \"f95f475e-903b-4e31-b5da-c4a7612fe54a\") " pod="e2e-test-llm-inference-service-6f0aa991/router-gateway-1-openshift-default-8555f5d76c-pqrgz"
Sep 17 23:39:53.557848 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:39:53.557411    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/f95f475e-903b-4e31-b5da-c4a7612fe54a-workload-certs\") pod \"router-gateway-1-openshift-default-8555f5d76c-pqrgz\" (UID: \"f95f475e-903b-4e31-b5da-c4a7612fe54a\") " pod="e2e-test-llm-inference-service-6f0aa991/router-gateway-1-openshift-default-8555f5d76c-pqrgz"
Sep 17 23:39:53.558330 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:39:53.558305    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/f95f475e-903b-4e31-b5da-c4a7612fe54a-istiod-ca-cert\") pod \"router-gateway-1-openshift-default-8555f5d76c-pqrgz\" (UID: \"f95f475e-903b-4e31-b5da-c4a7612fe54a\") " pod="e2e-test-llm-inference-service-6f0aa991/router-gateway-1-openshift-default-8555f5d76c-pqrgz"
Sep 17 23:39:53.564438 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:39:53.564414    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/f95f475e-903b-4e31-b5da-c4a7612fe54a-istio-envoy\") pod \"router-gateway-1-openshift-default-8555f5d76c-pqrgz\" (UID: \"f95f475e-903b-4e31-b5da-c4a7612fe54a\") " pod="e2e-test-llm-inference-service-6f0aa991/router-gateway-1-openshift-default-8555f5d76c-pqrgz"
Sep 17 23:39:53.564817 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:39:53.564798    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/f95f475e-903b-4e31-b5da-c4a7612fe54a-istio-podinfo\") pod \"router-gateway-1-openshift-default-8555f5d76c-pqrgz\" (UID: \"f95f475e-903b-4e31-b5da-c4a7612fe54a\") " pod="e2e-test-llm-inference-service-6f0aa991/router-gateway-1-openshift-default-8555f5d76c-pqrgz"
Sep 17 23:39:53.573361 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:39:53.573341    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-lf7xq\" (UniqueName: \"kubernetes.io/projected/f95f475e-903b-4e31-b5da-c4a7612fe54a-kube-api-access-lf7xq\") pod \"router-gateway-1-openshift-default-8555f5d76c-pqrgz\" (UID: \"f95f475e-903b-4e31-b5da-c4a7612fe54a\") " pod="e2e-test-llm-inference-service-6f0aa991/router-gateway-1-openshift-default-8555f5d76c-pqrgz"
Sep 17 23:39:53.573466 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:39:53.573446    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/f95f475e-903b-4e31-b5da-c4a7612fe54a-istio-token\") pod \"router-gateway-1-openshift-default-8555f5d76c-pqrgz\" (UID: \"f95f475e-903b-4e31-b5da-c4a7612fe54a\") " pod="e2e-test-llm-inference-service-6f0aa991/router-gateway-1-openshift-default-8555f5d76c-pqrgz"
Sep 17 23:39:53.651917 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:39:53.651882    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-test-llm-inference-service-6f0aa991/router-gateway-1-openshift-default-8555f5d76c-pqrgz"
Sep 17 23:39:53.927989 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:39:53.927962    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["e2e-test-llm-inference-service-6f0aa991/router-gateway-1-openshift-default-8555f5d76c-pqrgz"]
Sep 17 23:39:53.930605 ip-10-0-132-188 kubenswrapper[2562]: W0917 23:39:53.930578    2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf95f475e_903b_4e31_b5da_c4a7612fe54a.slice/crio-d0b4d01eba58dbd77a6762a35caf6405082bd0d655df9fd19e6c826beb6d59b8 WatchSource:0}: Error finding container d0b4d01eba58dbd77a6762a35caf6405082bd0d655df9fd19e6c826beb6d59b8: Status 404 returned error can't find the container with id d0b4d01eba58dbd77a6762a35caf6405082bd0d655df9fd19e6c826beb6d59b8
Sep 17 23:39:53.932631 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:39:53.932595    2562 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"}
Sep 17 23:39:53.932727 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:39:53.932672    2562 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"}
Sep 17 23:39:53.932727 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:39:53.932704    2562 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"}
Sep 17 23:39:54.282603 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:39:54.282508    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fwwc5_7e452bbb-7dc6-4c9d-a9b1-76400942f5b4/konnectivity-agent/0.log" limit=1024
Sep 17 23:39:54.338919 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:39:54.338877    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-test-llm-inference-service-6f0aa991/router-gateway-1-openshift-default-8555f5d76c-pqrgz" event={"ID":"f95f475e-903b-4e31-b5da-c4a7612fe54a","Type":"ContainerStarted","Data":"b371fca776b137da2990b92271d927bbeed714b557061c129922b1af717cb0c9"}
Sep 17 23:39:54.339113 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:39:54.338947    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-test-llm-inference-service-6f0aa991/router-gateway-1-openshift-default-8555f5d76c-pqrgz" event={"ID":"f95f475e-903b-4e31-b5da-c4a7612fe54a","Type":"ContainerStarted","Data":"d0b4d01eba58dbd77a6762a35caf6405082bd0d655df9fd19e6c826beb6d59b8"}
Sep 17 23:39:54.360561 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:39:54.360500    2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="e2e-test-llm-inference-service-6f0aa991/router-gateway-1-openshift-default-8555f5d76c-pqrgz" podStartSLOduration=1.360483087 podStartE2EDuration="1.360483087s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-17 23:39:53 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-17 23:39:54.358479513 +0000 UTC m=+1746.760376293" watchObservedRunningTime="2026-09-17 23:39:54.360483087 +0000 UTC m=+1746.762379850"
Sep 17 23:39:54.652920 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:39:54.652886    2562 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="e2e-test-llm-inference-service-6f0aa991/router-gateway-1-openshift-default-8555f5d76c-pqrgz"
Sep 17 23:39:54.658516 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:39:54.658489    2562 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="e2e-test-llm-inference-service-6f0aa991/router-gateway-1-openshift-default-8555f5d76c-pqrgz"
Sep 17 23:39:54.884526 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:39:54.884492    2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["e2e-test-llm-inference-service-84370a53/llmisvce55ae740357a3a31a27cdb8b66ffe20f-kserve-router-schedcv75"]
Sep 17 23:39:54.888048 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:39:54.888023    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-test-llm-inference-service-84370a53/llmisvce55ae740357a3a31a27cdb8b66ffe20f-kserve-router-schedcv75"
Sep 17 23:39:54.890561 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:39:54.890535    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-test-llm-inference-service-84370a53\"/\"openshift-service-ca.crt\""
Sep 17 23:39:54.890729 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:39:54.890561    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-test-llm-inference-service-84370a53\"/\"default-dockercfg-nb7xx\""
Sep 17 23:39:54.890729 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:39:54.890560    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-test-llm-inference-service-84370a53\"/\"kube-root-ca.crt\""
Sep 17 23:39:54.890729 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:39:54.890600    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-test-llm-inference-service-84370a53\"/\"llmisve55ae740357a3a31a27cdb8b66ffe20f-kserve-self-signed-certs\""
Sep 17 23:39:54.890729 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:39:54.890544    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-test-llm-inference-service-84370a53\"/\"llmisvc-router-managed-test-llm-5b1e8f15-epp-sa-dockercfg-ppqrs\""
Sep 17 23:39:54.898434 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:39:54.898413    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["e2e-test-llm-inference-service-84370a53/llmisvce55ae740357a3a31a27cdb8b66ffe20f-kserve-router-schedcv75"]
Sep 17 23:39:54.916615 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:39:54.916545    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fwwc5_7e452bbb-7dc6-4c9d-a9b1-76400942f5b4/konnectivity-agent/0.log" limit=1024
Sep 17 23:39:54.970344 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:39:54.970310    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/1c219d63-86d1-47f9-a114-a2635bd799f8-tls-certs\") pod \"llmisvce55ae740357a3a31a27cdb8b66ffe20f-kserve-router-schedcv75\" (UID: \"1c219d63-86d1-47f9-a114-a2635bd799f8\") " pod="e2e-test-llm-inference-service-84370a53/llmisvce55ae740357a3a31a27cdb8b66ffe20f-kserve-router-schedcv75"
Sep 17 23:39:54.970494 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:39:54.970370    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9gvbn\" (UniqueName: \"kubernetes.io/projected/1c219d63-86d1-47f9-a114-a2635bd799f8-kube-api-access-9gvbn\") pod \"llmisvce55ae740357a3a31a27cdb8b66ffe20f-kserve-router-schedcv75\" (UID: \"1c219d63-86d1-47f9-a114-a2635bd799f8\") " pod="e2e-test-llm-inference-service-84370a53/llmisvce55ae740357a3a31a27cdb8b66ffe20f-kserve-router-schedcv75"
Sep 17 23:39:55.071856 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:39:55.071813    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/1c219d63-86d1-47f9-a114-a2635bd799f8-tls-certs\") pod \"llmisvce55ae740357a3a31a27cdb8b66ffe20f-kserve-router-schedcv75\" (UID: \"1c219d63-86d1-47f9-a114-a2635bd799f8\") " pod="e2e-test-llm-inference-service-84370a53/llmisvce55ae740357a3a31a27cdb8b66ffe20f-kserve-router-schedcv75"
Sep 17 23:39:55.072047 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:39:55.071891    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9gvbn\" (UniqueName: \"kubernetes.io/projected/1c219d63-86d1-47f9-a114-a2635bd799f8-kube-api-access-9gvbn\") pod \"llmisvce55ae740357a3a31a27cdb8b66ffe20f-kserve-router-schedcv75\" (UID: \"1c219d63-86d1-47f9-a114-a2635bd799f8\") " pod="e2e-test-llm-inference-service-84370a53/llmisvce55ae740357a3a31a27cdb8b66ffe20f-kserve-router-schedcv75"
Sep 17 23:39:55.074618 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:39:55.074587    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/1c219d63-86d1-47f9-a114-a2635bd799f8-tls-certs\") pod \"llmisvce55ae740357a3a31a27cdb8b66ffe20f-kserve-router-schedcv75\" (UID: \"1c219d63-86d1-47f9-a114-a2635bd799f8\") " pod="e2e-test-llm-inference-service-84370a53/llmisvce55ae740357a3a31a27cdb8b66ffe20f-kserve-router-schedcv75"
Sep 17 23:39:55.079774 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:39:55.079741    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-9gvbn\" (UniqueName: \"kubernetes.io/projected/1c219d63-86d1-47f9-a114-a2635bd799f8-kube-api-access-9gvbn\") pod \"llmisvce55ae740357a3a31a27cdb8b66ffe20f-kserve-router-schedcv75\" (UID: \"1c219d63-86d1-47f9-a114-a2635bd799f8\") " pod="e2e-test-llm-inference-service-84370a53/llmisvce55ae740357a3a31a27cdb8b66ffe20f-kserve-router-schedcv75"
Sep 17 23:39:55.200505 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:39:55.200413    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-test-llm-inference-service-84370a53/llmisvce55ae740357a3a31a27cdb8b66ffe20f-kserve-router-schedcv75"
Sep 17 23:39:55.343894 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:39:55.343858    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-test-llm-inference-service-6f0aa991/router-gateway-1-openshift-default-8555f5d76c-pqrgz"
Sep 17 23:39:55.344731 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:39:55.344713    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-test-llm-inference-service-6f0aa991/router-gateway-1-openshift-default-8555f5d76c-pqrgz"
Sep 17 23:39:55.470861 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:39:55.470784    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["e2e-test-llm-inference-service-84370a53/llmisvce55ae740357a3a31a27cdb8b66ffe20f-kserve-router-schedcv75"]
Sep 17 23:39:55.474764 ip-10-0-132-188 kubenswrapper[2562]: W0917 23:39:55.474729    2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1c219d63_86d1_47f9_a114_a2635bd799f8.slice/crio-cf4d0033fcdedc47f9451606f575f15fc33870553bcfab04bff3ff335b5dace4 WatchSource:0}: Error finding container cf4d0033fcdedc47f9451606f575f15fc33870553bcfab04bff3ff335b5dace4: Status 404 returned error can't find the container with id cf4d0033fcdedc47f9451606f575f15fc33870553bcfab04bff3ff335b5dace4
Sep 17 23:39:56.353829 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:39:56.353784    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-test-llm-inference-service-84370a53/llmisvce55ae740357a3a31a27cdb8b66ffe20f-kserve-router-schedcv75" event={"ID":"1c219d63-86d1-47f9-a114-a2635bd799f8","Type":"ContainerStarted","Data":"02f79bb5d043be083f7044ac3b928a6d5ac3ab310cd2b5c758d984801d096ad4"}
Sep 17 23:39:56.354364 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:39:56.353852    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-test-llm-inference-service-84370a53/llmisvce55ae740357a3a31a27cdb8b66ffe20f-kserve-router-schedcv75" event={"ID":"1c219d63-86d1-47f9-a114-a2635bd799f8","Type":"ContainerStarted","Data":"cf4d0033fcdedc47f9451606f575f15fc33870553bcfab04bff3ff335b5dace4"}
Sep 17 23:39:56.354364 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:39:56.353898    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-test-llm-inference-service-84370a53/llmisvce55ae740357a3a31a27cdb8b66ffe20f-kserve-router-schedcv75"
Sep 17 23:39:56.372032 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:39:56.371973    2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="e2e-test-llm-inference-service-84370a53/llmisvce55ae740357a3a31a27cdb8b66ffe20f-kserve-router-schedcv75" podStartSLOduration=2.371954381 podStartE2EDuration="2.371954381s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-17 23:39:54 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-17 23:39:56.370200268 +0000 UTC m=+1748.772097033" watchObservedRunningTime="2026-09-17 23:39:56.371954381 +0000 UTC m=+1748.773851159"
Sep 17 23:39:56.651827 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:39:56.651788    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fwwc5_7e452bbb-7dc6-4c9d-a9b1-76400942f5b4/konnectivity-agent/0.log" limit=1024
Sep 17 23:39:56.792960 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:39:56.792930    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fwwc5_7e452bbb-7dc6-4c9d-a9b1-76400942f5b4/konnectivity-agent/0.log" limit=1024
Sep 17 23:39:56.893350 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:39:56.893264    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fwwc5_7e452bbb-7dc6-4c9d-a9b1-76400942f5b4/konnectivity-agent/0.log" limit=1024
Sep 17 23:39:59.522232 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:39:59.522204    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fwwc5_7e452bbb-7dc6-4c9d-a9b1-76400942f5b4/konnectivity-agent/0.log" limit=1024
Sep 17 23:40:05.058590 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:40:05.058551    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fwwc5_7e452bbb-7dc6-4c9d-a9b1-76400942f5b4/konnectivity-agent/0.log" limit=1024
Sep 17 23:40:06.467743 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:40:06.467705    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fwwc5_7e452bbb-7dc6-4c9d-a9b1-76400942f5b4/konnectivity-agent/0.log" limit=1024
Sep 17 23:40:09.772483 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:40:09.772438    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fwwc5_7e452bbb-7dc6-4c9d-a9b1-76400942f5b4/konnectivity-agent/0.log" limit=1024
Sep 17 23:40:09.829686 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:40:09.829650    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fwwc5_7e452bbb-7dc6-4c9d-a9b1-76400942f5b4/konnectivity-agent/0.log" limit=1024
Sep 17 23:40:15.196465 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:40:15.196425    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fwwc5_7e452bbb-7dc6-4c9d-a9b1-76400942f5b4/konnectivity-agent/0.log" limit=1024
Sep 17 23:40:23.321760 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:40:23.321723    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fwwc5_7e452bbb-7dc6-4c9d-a9b1-76400942f5b4/konnectivity-agent/0.log" limit=1024
Sep 17 23:40:28.367150 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:40:28.367113    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-test-llm-inference-service-84370a53/llmisvce55ae740357a3a31a27cdb8b66ffe20f-kserve-router-schedcv75"
Sep 17 23:40:33.524671 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:40:33.524583    2562 generic.go:356] "Generic (PLEG): container finished" podID="f95f475e-903b-4e31-b5da-c4a7612fe54a" containerID="b371fca776b137da2990b92271d927bbeed714b557061c129922b1af717cb0c9" exitCode=0
Sep 17 23:40:33.524671 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:40:33.524654    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-test-llm-inference-service-6f0aa991/router-gateway-1-openshift-default-8555f5d76c-pqrgz" event={"ID":"f95f475e-903b-4e31-b5da-c4a7612fe54a","Type":"ContainerDied","Data":"b371fca776b137da2990b92271d927bbeed714b557061c129922b1af717cb0c9"}
Sep 17 23:40:33.525111 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:40:33.525049    2562 scope.go:117] "RemoveContainer" containerID="b371fca776b137da2990b92271d927bbeed714b557061c129922b1af717cb0c9"
Sep 17 23:40:33.525977 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:40:33.525947    2562 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"}
Sep 17 23:40:33.526077 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:40:33.526017    2562 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"}
Sep 17 23:40:33.526077 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:40:33.526053    2562 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"}
Sep 17 23:40:34.531855 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:40:34.531820    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-test-llm-inference-service-6f0aa991/router-gateway-1-openshift-default-8555f5d76c-pqrgz" event={"ID":"f95f475e-903b-4e31-b5da-c4a7612fe54a","Type":"ContainerStarted","Data":"3a5aa5a7ee68fd27748412aa07997365cadda686d30618fadb477ed341990392"}
Sep 17 23:40:34.652463 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:40:34.652331    2562 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="e2e-test-llm-inference-service-6f0aa991/router-gateway-1-openshift-default-8555f5d76c-pqrgz"
Sep 17 23:40:34.652788 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:40:34.652757    2562 prober.go:120] "Probe failed" probeType="Startup" pod="e2e-test-llm-inference-service-6f0aa991/router-gateway-1-openshift-default-8555f5d76c-pqrgz" podUID="f95f475e-903b-4e31-b5da-c4a7612fe54a" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.43:15021/healthz/ready\": dial tcp 10.134.0.43:15021: connect: connection refused"
Sep 17 23:40:35.652674 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:40:35.652626    2562 prober.go:120] "Probe failed" probeType="Startup" pod="e2e-test-llm-inference-service-6f0aa991/router-gateway-1-openshift-default-8555f5d76c-pqrgz" podUID="f95f475e-903b-4e31-b5da-c4a7612fe54a" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.43:15021/healthz/ready\": dial tcp 10.134.0.43:15021: connect: connection refused"
Sep 17 23:40:36.652916 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:40:36.652863    2562 prober.go:120] "Probe failed" probeType="Startup" pod="e2e-test-llm-inference-service-6f0aa991/router-gateway-1-openshift-default-8555f5d76c-pqrgz" podUID="f95f475e-903b-4e31-b5da-c4a7612fe54a" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.43:15021/healthz/ready\": dial tcp 10.134.0.43:15021: connect: connection refused"
Sep 17 23:40:37.652919 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:40:37.652860    2562 prober.go:120] "Probe failed" probeType="Startup" pod="e2e-test-llm-inference-service-6f0aa991/router-gateway-1-openshift-default-8555f5d76c-pqrgz" podUID="f95f475e-903b-4e31-b5da-c4a7612fe54a" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.43:15021/healthz/ready\": dial tcp 10.134.0.43:15021: connect: connection refused"
Sep 17 23:40:38.652930 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:40:38.652880    2562 prober.go:120] "Probe failed" probeType="Startup" pod="e2e-test-llm-inference-service-6f0aa991/router-gateway-1-openshift-default-8555f5d76c-pqrgz" podUID="f95f475e-903b-4e31-b5da-c4a7612fe54a" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.43:15021/healthz/ready\": dial tcp 10.134.0.43:15021: connect: connection refused"
Sep 17 23:40:39.656260 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:40:39.656226    2562 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="e2e-test-llm-inference-service-6f0aa991/router-gateway-1-openshift-default-8555f5d76c-pqrgz"
Sep 17 23:40:39.656654 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:40:39.656480    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-test-llm-inference-service-6f0aa991/router-gateway-1-openshift-default-8555f5d76c-pqrgz"
Sep 17 23:40:39.657124 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:40:39.657105    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-test-llm-inference-service-6f0aa991/router-gateway-1-openshift-default-8555f5d76c-pqrgz"
Sep 17 23:41:20.932979 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:41:20.932924    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fwwc5_7e452bbb-7dc6-4c9d-a9b1-76400942f5b4/konnectivity-agent/0.log" limit=1024
Sep 17 23:41:21.130868 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:41:21.130825    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fwwc5_7e452bbb-7dc6-4c9d-a9b1-76400942f5b4/konnectivity-agent/0.log" limit=1024
Sep 17 23:41:31.107113 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:41:31.107076    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fwwc5_7e452bbb-7dc6-4c9d-a9b1-76400942f5b4/konnectivity-agent/0.log" limit=1024
Sep 17 23:41:32.898228 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:41:32.898190    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fwwc5_7e452bbb-7dc6-4c9d-a9b1-76400942f5b4/konnectivity-agent/0.log" limit=1024
Sep 17 23:41:37.991205 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:41:37.991146    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fwwc5_7e452bbb-7dc6-4c9d-a9b1-76400942f5b4/konnectivity-agent/0.log" limit=1024
Sep 17 23:41:38.088518 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:41:38.088458    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fwwc5_7e452bbb-7dc6-4c9d-a9b1-76400942f5b4/konnectivity-agent/0.log" limit=1024
Sep 17 23:41:42.942904 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:41:42.942869    2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["e2e-test-llm-inference-service-6f0aa991/router-gateway-1-openshift-default-8555f5d76c-pqrgz"]
Sep 17 23:41:42.943342 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:41:42.943059    2562 kuberuntime_container.go:873] "Killing container with a grace period" pod="e2e-test-llm-inference-service-6f0aa991/router-gateway-1-openshift-default-8555f5d76c-pqrgz" podUID="f95f475e-903b-4e31-b5da-c4a7612fe54a" containerName="istio-proxy" containerID="cri-o://3a5aa5a7ee68fd27748412aa07997365cadda686d30618fadb477ed341990392" gracePeriod=30
Sep 17 23:41:43.038696 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:41:43.038655    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fwwc5_7e452bbb-7dc6-4c9d-a9b1-76400942f5b4/konnectivity-agent/0.log" limit=1024
Sep 17 23:41:48.194761 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:41:48.194727    2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-test-llm-inference-service-6f0aa991/router-gateway-1-openshift-default-8555f5d76c-pqrgz"
Sep 17 23:41:48.335311 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:41:48.335276    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/f95f475e-903b-4e31-b5da-c4a7612fe54a-istio-envoy\") pod \"f95f475e-903b-4e31-b5da-c4a7612fe54a\" (UID: \"f95f475e-903b-4e31-b5da-c4a7612fe54a\") "
Sep 17 23:41:48.335475 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:41:48.335342    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/f95f475e-903b-4e31-b5da-c4a7612fe54a-credential-socket\") pod \"f95f475e-903b-4e31-b5da-c4a7612fe54a\" (UID: \"f95f475e-903b-4e31-b5da-c4a7612fe54a\") "
Sep 17 23:41:48.335475 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:41:48.335370    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/f95f475e-903b-4e31-b5da-c4a7612fe54a-istio-data\") pod \"f95f475e-903b-4e31-b5da-c4a7612fe54a\" (UID: \"f95f475e-903b-4e31-b5da-c4a7612fe54a\") "
Sep 17 23:41:48.335475 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:41:48.335405    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/f95f475e-903b-4e31-b5da-c4a7612fe54a-istio-podinfo\") pod \"f95f475e-903b-4e31-b5da-c4a7612fe54a\" (UID: \"f95f475e-903b-4e31-b5da-c4a7612fe54a\") "
Sep 17 23:41:48.335475 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:41:48.335453    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/f95f475e-903b-4e31-b5da-c4a7612fe54a-workload-socket\") pod \"f95f475e-903b-4e31-b5da-c4a7612fe54a\" (UID: \"f95f475e-903b-4e31-b5da-c4a7612fe54a\") "
Sep 17 23:41:48.335670 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:41:48.335484    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/f95f475e-903b-4e31-b5da-c4a7612fe54a-istio-token\") pod \"f95f475e-903b-4e31-b5da-c4a7612fe54a\" (UID: \"f95f475e-903b-4e31-b5da-c4a7612fe54a\") "
Sep 17 23:41:48.335670 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:41:48.335520    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/f95f475e-903b-4e31-b5da-c4a7612fe54a-istiod-ca-cert\") pod \"f95f475e-903b-4e31-b5da-c4a7612fe54a\" (UID: \"f95f475e-903b-4e31-b5da-c4a7612fe54a\") "
Sep 17 23:41:48.335670 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:41:48.335566    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/f95f475e-903b-4e31-b5da-c4a7612fe54a-workload-certs\") pod \"f95f475e-903b-4e31-b5da-c4a7612fe54a\" (UID: \"f95f475e-903b-4e31-b5da-c4a7612fe54a\") "
Sep 17 23:41:48.335670 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:41:48.335586    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f95f475e-903b-4e31-b5da-c4a7612fe54a-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "f95f475e-903b-4e31-b5da-c4a7612fe54a" (UID: "f95f475e-903b-4e31-b5da-c4a7612fe54a"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 17 23:41:48.335670 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:41:48.335601    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-lf7xq\" (UniqueName: \"kubernetes.io/projected/f95f475e-903b-4e31-b5da-c4a7612fe54a-kube-api-access-lf7xq\") pod \"f95f475e-903b-4e31-b5da-c4a7612fe54a\" (UID: \"f95f475e-903b-4e31-b5da-c4a7612fe54a\") "
Sep 17 23:41:48.335670 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:41:48.335639    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f95f475e-903b-4e31-b5da-c4a7612fe54a-istio-data" (OuterVolumeSpecName: "istio-data") pod "f95f475e-903b-4e31-b5da-c4a7612fe54a" (UID: "f95f475e-903b-4e31-b5da-c4a7612fe54a"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 17 23:41:48.335937 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:41:48.335798    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f95f475e-903b-4e31-b5da-c4a7612fe54a-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "f95f475e-903b-4e31-b5da-c4a7612fe54a" (UID: "f95f475e-903b-4e31-b5da-c4a7612fe54a"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 17 23:41:48.335937 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:41:48.335854    2562 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/f95f475e-903b-4e31-b5da-c4a7612fe54a-credential-socket\") on node \"ip-10-0-132-188.ec2.internal\" DevicePath \"\""
Sep 17 23:41:48.335937 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:41:48.335871    2562 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/f95f475e-903b-4e31-b5da-c4a7612fe54a-istio-data\") on node \"ip-10-0-132-188.ec2.internal\" DevicePath \"\""
Sep 17 23:41:48.336033 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:41:48.335948    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f95f475e-903b-4e31-b5da-c4a7612fe54a-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "f95f475e-903b-4e31-b5da-c4a7612fe54a" (UID: "f95f475e-903b-4e31-b5da-c4a7612fe54a"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 17 23:41:48.336266 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:41:48.336239    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/f95f475e-903b-4e31-b5da-c4a7612fe54a-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "f95f475e-903b-4e31-b5da-c4a7612fe54a" (UID: "f95f475e-903b-4e31-b5da-c4a7612fe54a"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 17 23:41:48.337696 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:41:48.337671    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f95f475e-903b-4e31-b5da-c4a7612fe54a-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "f95f475e-903b-4e31-b5da-c4a7612fe54a" (UID: "f95f475e-903b-4e31-b5da-c4a7612fe54a"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 17 23:41:48.337825 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:41:48.337781    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f95f475e-903b-4e31-b5da-c4a7612fe54a-istio-token" (OuterVolumeSpecName: "istio-token") pod "f95f475e-903b-4e31-b5da-c4a7612fe54a" (UID: "f95f475e-903b-4e31-b5da-c4a7612fe54a"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 17 23:41:48.338115 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:41:48.338095    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/f95f475e-903b-4e31-b5da-c4a7612fe54a-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "f95f475e-903b-4e31-b5da-c4a7612fe54a" (UID: "f95f475e-903b-4e31-b5da-c4a7612fe54a"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Sep 17 23:41:48.338245 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:41:48.338230    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f95f475e-903b-4e31-b5da-c4a7612fe54a-kube-api-access-lf7xq" (OuterVolumeSpecName: "kube-api-access-lf7xq") pod "f95f475e-903b-4e31-b5da-c4a7612fe54a" (UID: "f95f475e-903b-4e31-b5da-c4a7612fe54a"). InnerVolumeSpecName "kube-api-access-lf7xq". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 17 23:41:48.436917 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:41:48.436879    2562 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/f95f475e-903b-4e31-b5da-c4a7612fe54a-workload-certs\") on node \"ip-10-0-132-188.ec2.internal\" DevicePath \"\""
Sep 17 23:41:48.436917 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:41:48.436913    2562 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-lf7xq\" (UniqueName: \"kubernetes.io/projected/f95f475e-903b-4e31-b5da-c4a7612fe54a-kube-api-access-lf7xq\") on node \"ip-10-0-132-188.ec2.internal\" DevicePath \"\""
Sep 17 23:41:48.437109 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:41:48.436928    2562 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/f95f475e-903b-4e31-b5da-c4a7612fe54a-istio-envoy\") on node \"ip-10-0-132-188.ec2.internal\" DevicePath \"\""
Sep 17 23:41:48.437109 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:41:48.436943    2562 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/f95f475e-903b-4e31-b5da-c4a7612fe54a-istio-podinfo\") on node \"ip-10-0-132-188.ec2.internal\" DevicePath \"\""
Sep 17 23:41:48.437109 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:41:48.436956    2562 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/f95f475e-903b-4e31-b5da-c4a7612fe54a-workload-socket\") on node \"ip-10-0-132-188.ec2.internal\" DevicePath \"\""
Sep 17 23:41:48.437109 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:41:48.436970    2562 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/f95f475e-903b-4e31-b5da-c4a7612fe54a-istio-token\") on node \"ip-10-0-132-188.ec2.internal\" DevicePath \"\""
Sep 17 23:41:48.437109 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:41:48.436982    2562 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/f95f475e-903b-4e31-b5da-c4a7612fe54a-istiod-ca-cert\") on node \"ip-10-0-132-188.ec2.internal\" DevicePath \"\""
Sep 17 23:41:48.897462 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:41:48.897426    2562 generic.go:356] "Generic (PLEG): container finished" podID="f95f475e-903b-4e31-b5da-c4a7612fe54a" containerID="3a5aa5a7ee68fd27748412aa07997365cadda686d30618fadb477ed341990392" exitCode=0
Sep 17 23:41:48.897662 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:41:48.897483    2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-test-llm-inference-service-6f0aa991/router-gateway-1-openshift-default-8555f5d76c-pqrgz"
Sep 17 23:41:48.897662 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:41:48.897516    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-test-llm-inference-service-6f0aa991/router-gateway-1-openshift-default-8555f5d76c-pqrgz" event={"ID":"f95f475e-903b-4e31-b5da-c4a7612fe54a","Type":"ContainerDied","Data":"3a5aa5a7ee68fd27748412aa07997365cadda686d30618fadb477ed341990392"}
Sep 17 23:41:48.897662 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:41:48.897584    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-test-llm-inference-service-6f0aa991/router-gateway-1-openshift-default-8555f5d76c-pqrgz" event={"ID":"f95f475e-903b-4e31-b5da-c4a7612fe54a","Type":"ContainerDied","Data":"d0b4d01eba58dbd77a6762a35caf6405082bd0d655df9fd19e6c826beb6d59b8"}
Sep 17 23:41:48.897662 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:41:48.897616    2562 scope.go:117] "RemoveContainer" containerID="3a5aa5a7ee68fd27748412aa07997365cadda686d30618fadb477ed341990392"
Sep 17 23:41:48.906646 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:41:48.906628    2562 scope.go:117] "RemoveContainer" containerID="b371fca776b137da2990b92271d927bbeed714b557061c129922b1af717cb0c9"
Sep 17 23:41:48.914346 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:41:48.914328    2562 scope.go:117] "RemoveContainer" containerID="3a5aa5a7ee68fd27748412aa07997365cadda686d30618fadb477ed341990392"
Sep 17 23:41:48.914613 ip-10-0-132-188 kubenswrapper[2562]: E0917 23:41:48.914594    2562 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"3a5aa5a7ee68fd27748412aa07997365cadda686d30618fadb477ed341990392\": container with ID starting with 3a5aa5a7ee68fd27748412aa07997365cadda686d30618fadb477ed341990392 not found: ID does not exist" containerID="3a5aa5a7ee68fd27748412aa07997365cadda686d30618fadb477ed341990392"
Sep 17 23:41:48.914670 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:41:48.914621    2562 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"3a5aa5a7ee68fd27748412aa07997365cadda686d30618fadb477ed341990392"} err="failed to get container status \"3a5aa5a7ee68fd27748412aa07997365cadda686d30618fadb477ed341990392\": rpc error: code = NotFound desc = could not find container \"3a5aa5a7ee68fd27748412aa07997365cadda686d30618fadb477ed341990392\": container with ID starting with 3a5aa5a7ee68fd27748412aa07997365cadda686d30618fadb477ed341990392 not found: ID does not exist"
Sep 17 23:41:48.914670 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:41:48.914638    2562 scope.go:117] "RemoveContainer" containerID="b371fca776b137da2990b92271d927bbeed714b557061c129922b1af717cb0c9"
Sep 17 23:41:48.914889 ip-10-0-132-188 kubenswrapper[2562]: E0917 23:41:48.914870    2562 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"b371fca776b137da2990b92271d927bbeed714b557061c129922b1af717cb0c9\": container with ID starting with b371fca776b137da2990b92271d927bbeed714b557061c129922b1af717cb0c9 not found: ID does not exist" containerID="b371fca776b137da2990b92271d927bbeed714b557061c129922b1af717cb0c9"
Sep 17 23:41:48.914934 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:41:48.914894    2562 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"b371fca776b137da2990b92271d927bbeed714b557061c129922b1af717cb0c9"} err="failed to get container status \"b371fca776b137da2990b92271d927bbeed714b557061c129922b1af717cb0c9\": rpc error: code = NotFound desc = could not find container \"b371fca776b137da2990b92271d927bbeed714b557061c129922b1af717cb0c9\": container with ID starting with b371fca776b137da2990b92271d927bbeed714b557061c129922b1af717cb0c9 not found: ID does not exist"
Sep 17 23:41:48.917629 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:41:48.917604    2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["e2e-test-llm-inference-service-6f0aa991/router-gateway-1-openshift-default-8555f5d76c-pqrgz"]
Sep 17 23:41:48.923496 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:41:48.923475    2562 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["e2e-test-llm-inference-service-6f0aa991/router-gateway-1-openshift-default-8555f5d76c-pqrgz"]
Sep 17 23:41:49.236091 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:41:49.236016    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fwwc5_7e452bbb-7dc6-4c9d-a9b1-76400942f5b4/konnectivity-agent/0.log" limit=1024
Sep 17 23:41:49.425667 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:41:49.425634    2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["e2e-test-llm-inference-service-079cb970/llmisvca690bbc929faec8bc98c767f16c003c1-kserve-router-scheb68sc"]
Sep 17 23:41:49.425958 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:41:49.425946    2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f95f475e-903b-4e31-b5da-c4a7612fe54a" containerName="istio-proxy"
Sep 17 23:41:49.426004 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:41:49.425959    2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="f95f475e-903b-4e31-b5da-c4a7612fe54a" containerName="istio-proxy"
Sep 17 23:41:49.426004 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:41:49.425970    2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f95f475e-903b-4e31-b5da-c4a7612fe54a" containerName="istio-proxy"
Sep 17 23:41:49.426004 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:41:49.425976    2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="f95f475e-903b-4e31-b5da-c4a7612fe54a" containerName="istio-proxy"
Sep 17 23:41:49.426103 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:41:49.426028    2562 memory_manager.go:368] "RemoveStaleState removing state" podUID="f95f475e-903b-4e31-b5da-c4a7612fe54a" containerName="istio-proxy"
Sep 17 23:41:49.426103 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:41:49.426036    2562 memory_manager.go:368] "RemoveStaleState removing state" podUID="f95f475e-903b-4e31-b5da-c4a7612fe54a" containerName="istio-proxy"
Sep 17 23:41:49.428955 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:41:49.428931    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-test-llm-inference-service-079cb970/llmisvca690bbc929faec8bc98c767f16c003c1-kserve-router-scheb68sc"
Sep 17 23:41:49.432087 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:41:49.432064    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-test-llm-inference-service-079cb970\"/\"openshift-service-ca.crt\""
Sep 17 23:41:49.432087 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:41:49.432075    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-test-llm-inference-service-079cb970\"/\"default-dockercfg-4gwh7\""
Sep 17 23:41:49.432310 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:41:49.432075    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-test-llm-inference-service-079cb970\"/\"kube-root-ca.crt\""
Sep 17 23:41:49.432310 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:41:49.432150    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-test-llm-inference-service-079cb970\"/\"llmisva690bbc929faec8bc98c767f16c003c1-kserve-self-signed-certs\""
Sep 17 23:41:49.432310 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:41:49.432166    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-test-llm-inference-service-079cb970\"/\"llmisvc-model-fb-opt-125m-route-87882a8e-epp-sa-dockercfg-97rh2\""
Sep 17 23:41:49.441810 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:41:49.441786    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["e2e-test-llm-inference-service-079cb970/llmisvca690bbc929faec8bc98c767f16c003c1-kserve-router-scheb68sc"]
Sep 17 23:41:49.494638 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:41:49.494546    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fwwc5_7e452bbb-7dc6-4c9d-a9b1-76400942f5b4/konnectivity-agent/0.log" limit=1024
Sep 17 23:41:49.545355 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:41:49.545325    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/c6fcdcef-f170-49ac-968c-f2cacceb70f1-tls-certs\") pod \"llmisvca690bbc929faec8bc98c767f16c003c1-kserve-router-scheb68sc\" (UID: \"c6fcdcef-f170-49ac-968c-f2cacceb70f1\") " pod="e2e-test-llm-inference-service-079cb970/llmisvca690bbc929faec8bc98c767f16c003c1-kserve-router-scheb68sc"
Sep 17 23:41:49.545520 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:41:49.545373    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-z57qm\" (UniqueName: \"kubernetes.io/projected/c6fcdcef-f170-49ac-968c-f2cacceb70f1-kube-api-access-z57qm\") pod \"llmisvca690bbc929faec8bc98c767f16c003c1-kserve-router-scheb68sc\" (UID: \"c6fcdcef-f170-49ac-968c-f2cacceb70f1\") " pod="e2e-test-llm-inference-service-079cb970/llmisvca690bbc929faec8bc98c767f16c003c1-kserve-router-scheb68sc"
Sep 17 23:41:49.646793 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:41:49.646754    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/c6fcdcef-f170-49ac-968c-f2cacceb70f1-tls-certs\") pod \"llmisvca690bbc929faec8bc98c767f16c003c1-kserve-router-scheb68sc\" (UID: \"c6fcdcef-f170-49ac-968c-f2cacceb70f1\") " pod="e2e-test-llm-inference-service-079cb970/llmisvca690bbc929faec8bc98c767f16c003c1-kserve-router-scheb68sc"
Sep 17 23:41:49.646976 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:41:49.646822    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-z57qm\" (UniqueName: \"kubernetes.io/projected/c6fcdcef-f170-49ac-968c-f2cacceb70f1-kube-api-access-z57qm\") pod \"llmisvca690bbc929faec8bc98c767f16c003c1-kserve-router-scheb68sc\" (UID: \"c6fcdcef-f170-49ac-968c-f2cacceb70f1\") " pod="e2e-test-llm-inference-service-079cb970/llmisvca690bbc929faec8bc98c767f16c003c1-kserve-router-scheb68sc"
Sep 17 23:41:49.649218 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:41:49.649193    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-test-llm-inference-service-079cb970\"/\"llmisva690bbc929faec8bc98c767f16c003c1-kserve-self-signed-certs\""
Sep 17 23:41:49.653991 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:41:49.653963    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-test-llm-inference-service-079cb970\"/\"kube-root-ca.crt\""
Sep 17 23:41:49.659982 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:41:49.659958    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/c6fcdcef-f170-49ac-968c-f2cacceb70f1-tls-certs\") pod \"llmisvca690bbc929faec8bc98c767f16c003c1-kserve-router-scheb68sc\" (UID: \"c6fcdcef-f170-49ac-968c-f2cacceb70f1\") " pod="e2e-test-llm-inference-service-079cb970/llmisvca690bbc929faec8bc98c767f16c003c1-kserve-router-scheb68sc"
Sep 17 23:41:49.664815 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:41:49.664796    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-test-llm-inference-service-079cb970\"/\"openshift-service-ca.crt\""
Sep 17 23:41:49.668950 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:41:49.668928    2562 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="f95f475e-903b-4e31-b5da-c4a7612fe54a" path="/var/lib/kubelet/pods/f95f475e-903b-4e31-b5da-c4a7612fe54a/volumes"
Sep 17 23:41:49.676074 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:41:49.676047    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-z57qm\" (UniqueName: \"kubernetes.io/projected/c6fcdcef-f170-49ac-968c-f2cacceb70f1-kube-api-access-z57qm\") pod \"llmisvca690bbc929faec8bc98c767f16c003c1-kserve-router-scheb68sc\" (UID: \"c6fcdcef-f170-49ac-968c-f2cacceb70f1\") " pod="e2e-test-llm-inference-service-079cb970/llmisvca690bbc929faec8bc98c767f16c003c1-kserve-router-scheb68sc"
Sep 17 23:41:49.746916 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:41:49.746849    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-test-llm-inference-service-079cb970\"/\"default-dockercfg-4gwh7\""
Sep 17 23:41:49.757341 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:41:49.757317    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-test-llm-inference-service-079cb970\"/\"llmisvc-model-fb-opt-125m-route-87882a8e-epp-sa-dockercfg-97rh2\""
Sep 17 23:41:49.766119 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:41:49.766101    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-test-llm-inference-service-079cb970/llmisvca690bbc929faec8bc98c767f16c003c1-kserve-router-scheb68sc"
Sep 17 23:41:50.037600 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:41:50.037506    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["e2e-test-llm-inference-service-079cb970/llmisvca690bbc929faec8bc98c767f16c003c1-kserve-router-scheb68sc"]
Sep 17 23:41:50.041108 ip-10-0-132-188 kubenswrapper[2562]: W0917 23:41:50.041072    2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc6fcdcef_f170_49ac_968c_f2cacceb70f1.slice/crio-733bc3d2decd5d09142387c8cfce28d68d3cdcd842a4877d726d96e45776b2bc WatchSource:0}: Error finding container 733bc3d2decd5d09142387c8cfce28d68d3cdcd842a4877d726d96e45776b2bc: Status 404 returned error can't find the container with id 733bc3d2decd5d09142387c8cfce28d68d3cdcd842a4877d726d96e45776b2bc
Sep 17 23:41:50.909946 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:41:50.909902    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-test-llm-inference-service-079cb970/llmisvca690bbc929faec8bc98c767f16c003c1-kserve-router-scheb68sc" event={"ID":"c6fcdcef-f170-49ac-968c-f2cacceb70f1","Type":"ContainerStarted","Data":"8e6041e1e024e292942e4711b5c5196b77b3cf21c92b009f17e89b0c95bc0063"}
Sep 17 23:41:50.910424 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:41:50.909963    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-test-llm-inference-service-079cb970/llmisvca690bbc929faec8bc98c767f16c003c1-kserve-router-scheb68sc" event={"ID":"c6fcdcef-f170-49ac-968c-f2cacceb70f1","Type":"ContainerStarted","Data":"733bc3d2decd5d09142387c8cfce28d68d3cdcd842a4877d726d96e45776b2bc"}
Sep 17 23:41:50.910424 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:41:50.910087    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-test-llm-inference-service-079cb970/llmisvca690bbc929faec8bc98c767f16c003c1-kserve-router-scheb68sc"
Sep 17 23:41:50.926865 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:41:50.926818    2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="e2e-test-llm-inference-service-079cb970/llmisvca690bbc929faec8bc98c767f16c003c1-kserve-router-scheb68sc" podStartSLOduration=1.926805291 podStartE2EDuration="1.926805291s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-17 23:41:49 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-17 23:41:50.924982164 +0000 UTC m=+1863.326878932" watchObservedRunningTime="2026-09-17 23:41:50.926805291 +0000 UTC m=+1863.328702056"
Sep 17 23:41:59.067591 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:41:59.067548    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fwwc5_7e452bbb-7dc6-4c9d-a9b1-76400942f5b4/konnectivity-agent/0.log" limit=1024
Sep 17 23:42:04.076994 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:42:04.076897    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fwwc5_7e452bbb-7dc6-4c9d-a9b1-76400942f5b4/konnectivity-agent/0.log" limit=1024
Sep 17 23:42:04.319496 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:42:04.319460    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fwwc5_7e452bbb-7dc6-4c9d-a9b1-76400942f5b4/konnectivity-agent/0.log" limit=1024
Sep 17 23:42:04.413975 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:42:04.413876    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fwwc5_7e452bbb-7dc6-4c9d-a9b1-76400942f5b4/konnectivity-agent/0.log" limit=1024
Sep 17 23:42:05.819956 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:42:05.819919    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fwwc5_7e452bbb-7dc6-4c9d-a9b1-76400942f5b4/konnectivity-agent/0.log" limit=1024
Sep 17 23:42:11.266157 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:42:11.266115    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fwwc5_7e452bbb-7dc6-4c9d-a9b1-76400942f5b4/konnectivity-agent/0.log" limit=1024
Sep 17 23:42:20.911211 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:42:20.911164    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fwwc5_7e452bbb-7dc6-4c9d-a9b1-76400942f5b4/konnectivity-agent/0.log" limit=1024
Sep 17 23:42:21.915648 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:42:21.915613    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-test-llm-inference-service-079cb970/llmisvca690bbc929faec8bc98c767f16c003c1-kserve-router-scheb68sc"
Sep 17 23:42:25.232382 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:42:25.232341    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fwwc5_7e452bbb-7dc6-4c9d-a9b1-76400942f5b4/konnectivity-agent/0.log" limit=1024
Sep 17 23:42:25.333302 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:42:25.333260    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fwwc5_7e452bbb-7dc6-4c9d-a9b1-76400942f5b4/konnectivity-agent/0.log" limit=1024
Sep 17 23:42:53.622626 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:42:53.622590    2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["e2e-test-llm-inference-service-84370a53/llmisvce55ae740357a3a31a27cdb8b66ffe20f-kserve-router-schedcv75"]
Sep 17 23:42:53.647150 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:42:53.647104    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fwwc5_7e452bbb-7dc6-4c9d-a9b1-76400942f5b4/konnectivity-agent/0.log" limit=1024
Sep 17 23:42:53.742968 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:42:53.742935    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fwwc5_7e452bbb-7dc6-4c9d-a9b1-76400942f5b4/konnectivity-agent/0.log" limit=1024
Sep 17 23:42:58.367050 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:42:58.367000    2562 prober.go:120] "Probe failed" probeType="Readiness" pod="e2e-test-llm-inference-service-84370a53/llmisvce55ae740357a3a31a27cdb8b66ffe20f-kserve-router-schedcv75" podUID="1c219d63-86d1-47f9-a114-a2635bd799f8" containerName="main" probeResult="failure" output="service unhealthy (responded with \"NOT_SERVING\")"
Sep 17 23:43:08.367158 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:43:08.367116    2562 prober.go:120] "Probe failed" probeType="Readiness" pod="e2e-test-llm-inference-service-84370a53/llmisvce55ae740357a3a31a27cdb8b66ffe20f-kserve-router-schedcv75" podUID="1c219d63-86d1-47f9-a114-a2635bd799f8" containerName="main" probeResult="failure" output="service unhealthy (responded with \"NOT_SERVING\")"
Sep 17 23:43:08.643027 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:43:08.642926    2562 kuberuntime_container.go:873] "Killing container with a grace period" pod="e2e-test-llm-inference-service-84370a53/llmisvce55ae740357a3a31a27cdb8b66ffe20f-kserve-router-schedcv75" podUID="1c219d63-86d1-47f9-a114-a2635bd799f8" containerName="main" containerID="cri-o://02f79bb5d043be083f7044ac3b928a6d5ac3ab310cd2b5c758d984801d096ad4" gracePeriod=45
Sep 17 23:43:18.012401 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:43:18.012360    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fwwc5_7e452bbb-7dc6-4c9d-a9b1-76400942f5b4/konnectivity-agent/0.log" limit=1024
Sep 17 23:43:18.366974 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:43:18.366937    2562 prober.go:120] "Probe failed" probeType="Readiness" pod="e2e-test-llm-inference-service-84370a53/llmisvce55ae740357a3a31a27cdb8b66ffe20f-kserve-router-schedcv75" podUID="1c219d63-86d1-47f9-a114-a2635bd799f8" containerName="main" probeResult="failure" output="service unhealthy (responded with \"NOT_SERVING\")"
Sep 17 23:43:18.367163 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:43:18.367038    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-test-llm-inference-service-84370a53/llmisvce55ae740357a3a31a27cdb8b66ffe20f-kserve-router-schedcv75"
Sep 17 23:43:28.367047 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:43:28.367009    2562 prober.go:120] "Probe failed" probeType="Readiness" pod="e2e-test-llm-inference-service-84370a53/llmisvce55ae740357a3a31a27cdb8b66ffe20f-kserve-router-schedcv75" podUID="1c219d63-86d1-47f9-a114-a2635bd799f8" containerName="main" probeResult="failure" output="service unhealthy (responded with \"NOT_SERVING\")"
Sep 17 23:43:38.366955 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:43:38.366871    2562 prober.go:120] "Probe failed" probeType="Readiness" pod="e2e-test-llm-inference-service-84370a53/llmisvce55ae740357a3a31a27cdb8b66ffe20f-kserve-router-schedcv75" podUID="1c219d63-86d1-47f9-a114-a2635bd799f8" containerName="main" probeResult="failure" output="service unhealthy (responded with \"NOT_SERVING\")"
Sep 17 23:43:38.899236 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:43:38.899196    2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-test-llm-inference-service-84370a53/llmisvce55ae740357a3a31a27cdb8b66ffe20f-kserve-router-schedcv75"
Sep 17 23:43:38.970197 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:43:38.970069    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/1c219d63-86d1-47f9-a114-a2635bd799f8-tls-certs\") pod \"1c219d63-86d1-47f9-a114-a2635bd799f8\" (UID: \"1c219d63-86d1-47f9-a114-a2635bd799f8\") "
Sep 17 23:43:38.970197 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:43:38.970193    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-9gvbn\" (UniqueName: \"kubernetes.io/projected/1c219d63-86d1-47f9-a114-a2635bd799f8-kube-api-access-9gvbn\") pod \"1c219d63-86d1-47f9-a114-a2635bd799f8\" (UID: \"1c219d63-86d1-47f9-a114-a2635bd799f8\") "
Sep 17 23:43:38.972460 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:43:38.972422    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/1c219d63-86d1-47f9-a114-a2635bd799f8-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "1c219d63-86d1-47f9-a114-a2635bd799f8" (UID: "1c219d63-86d1-47f9-a114-a2635bd799f8"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 17 23:43:38.972574 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:43:38.972456    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/1c219d63-86d1-47f9-a114-a2635bd799f8-kube-api-access-9gvbn" (OuterVolumeSpecName: "kube-api-access-9gvbn") pod "1c219d63-86d1-47f9-a114-a2635bd799f8" (UID: "1c219d63-86d1-47f9-a114-a2635bd799f8"). InnerVolumeSpecName "kube-api-access-9gvbn". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 17 23:43:39.070801 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:43:39.070763    2562 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/1c219d63-86d1-47f9-a114-a2635bd799f8-tls-certs\") on node \"ip-10-0-132-188.ec2.internal\" DevicePath \"\""
Sep 17 23:43:39.070801 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:43:39.070795    2562 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-9gvbn\" (UniqueName: \"kubernetes.io/projected/1c219d63-86d1-47f9-a114-a2635bd799f8-kube-api-access-9gvbn\") on node \"ip-10-0-132-188.ec2.internal\" DevicePath \"\""
Sep 17 23:43:39.413268 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:43:39.413234    2562 generic.go:356] "Generic (PLEG): container finished" podID="1c219d63-86d1-47f9-a114-a2635bd799f8" containerID="02f79bb5d043be083f7044ac3b928a6d5ac3ab310cd2b5c758d984801d096ad4" exitCode=0
Sep 17 23:43:39.413657 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:43:39.413286    2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-test-llm-inference-service-84370a53/llmisvce55ae740357a3a31a27cdb8b66ffe20f-kserve-router-schedcv75"
Sep 17 23:43:39.413657 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:43:39.413323    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-test-llm-inference-service-84370a53/llmisvce55ae740357a3a31a27cdb8b66ffe20f-kserve-router-schedcv75" event={"ID":"1c219d63-86d1-47f9-a114-a2635bd799f8","Type":"ContainerDied","Data":"02f79bb5d043be083f7044ac3b928a6d5ac3ab310cd2b5c758d984801d096ad4"}
Sep 17 23:43:39.413657 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:43:39.413380    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-test-llm-inference-service-84370a53/llmisvce55ae740357a3a31a27cdb8b66ffe20f-kserve-router-schedcv75" event={"ID":"1c219d63-86d1-47f9-a114-a2635bd799f8","Type":"ContainerDied","Data":"cf4d0033fcdedc47f9451606f575f15fc33870553bcfab04bff3ff335b5dace4"}
Sep 17 23:43:39.413657 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:43:39.413409    2562 scope.go:117] "RemoveContainer" containerID="02f79bb5d043be083f7044ac3b928a6d5ac3ab310cd2b5c758d984801d096ad4"
Sep 17 23:43:39.422377 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:43:39.422359    2562 scope.go:117] "RemoveContainer" containerID="02f79bb5d043be083f7044ac3b928a6d5ac3ab310cd2b5c758d984801d096ad4"
Sep 17 23:43:39.422675 ip-10-0-132-188 kubenswrapper[2562]: E0917 23:43:39.422653    2562 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"02f79bb5d043be083f7044ac3b928a6d5ac3ab310cd2b5c758d984801d096ad4\": container with ID starting with 02f79bb5d043be083f7044ac3b928a6d5ac3ab310cd2b5c758d984801d096ad4 not found: ID does not exist" containerID="02f79bb5d043be083f7044ac3b928a6d5ac3ab310cd2b5c758d984801d096ad4"
Sep 17 23:43:39.422757 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:43:39.422683    2562 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"02f79bb5d043be083f7044ac3b928a6d5ac3ab310cd2b5c758d984801d096ad4"} err="failed to get container status \"02f79bb5d043be083f7044ac3b928a6d5ac3ab310cd2b5c758d984801d096ad4\": rpc error: code = NotFound desc = could not find container \"02f79bb5d043be083f7044ac3b928a6d5ac3ab310cd2b5c758d984801d096ad4\": container with ID starting with 02f79bb5d043be083f7044ac3b928a6d5ac3ab310cd2b5c758d984801d096ad4 not found: ID does not exist"
Sep 17 23:43:39.433400 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:43:39.433375    2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["e2e-test-llm-inference-service-84370a53/llmisvce55ae740357a3a31a27cdb8b66ffe20f-kserve-router-schedcv75"]
Sep 17 23:43:39.437394 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:43:39.437371    2562 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["e2e-test-llm-inference-service-84370a53/llmisvce55ae740357a3a31a27cdb8b66ffe20f-kserve-router-schedcv75"]
Sep 17 23:43:39.668105 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:43:39.668021    2562 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="1c219d63-86d1-47f9-a114-a2635bd799f8" path="/var/lib/kubelet/pods/1c219d63-86d1-47f9-a114-a2635bd799f8/volumes"
Sep 17 23:43:43.707632 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:43:43.707601    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fwwc5_7e452bbb-7dc6-4c9d-a9b1-76400942f5b4/konnectivity-agent/0.log" limit=1024
Sep 17 23:43:43.803533 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:43:43.803492    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fwwc5_7e452bbb-7dc6-4c9d-a9b1-76400942f5b4/konnectivity-agent/0.log" limit=1024
Sep 17 23:43:49.754328 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:43:49.754286    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fwwc5_7e452bbb-7dc6-4c9d-a9b1-76400942f5b4/konnectivity-agent/0.log" limit=1024
Sep 17 23:43:55.407811 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:43:55.407773    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fwwc5_7e452bbb-7dc6-4c9d-a9b1-76400942f5b4/konnectivity-agent/0.log" limit=1024
Sep 17 23:43:55.507815 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:43:55.507775    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fwwc5_7e452bbb-7dc6-4c9d-a9b1-76400942f5b4/konnectivity-agent/0.log" limit=1024
Sep 17 23:44:01.127702 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:44:01.127661    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fwwc5_7e452bbb-7dc6-4c9d-a9b1-76400942f5b4/konnectivity-agent/0.log" limit=1024
Sep 17 23:44:01.373943 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:44:01.373908    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fwwc5_7e452bbb-7dc6-4c9d-a9b1-76400942f5b4/konnectivity-agent/0.log" limit=1024
Sep 17 23:44:23.480787 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:44:23.480753    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fwwc5_7e452bbb-7dc6-4c9d-a9b1-76400942f5b4/konnectivity-agent/0.log" limit=1024
Sep 17 23:44:29.670931 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:44:29.670889    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fwwc5_7e452bbb-7dc6-4c9d-a9b1-76400942f5b4/konnectivity-agent/0.log" limit=1024
Sep 17 23:44:29.687682 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:44:29.687644    2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["e2e-test-llm-inference-service-079cb970/llmisvca690bbc929faec8bc98c767f16c003c1-kserve-router-scheb68sc"]
Sep 17 23:44:29.793859 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:44:29.793825    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fwwc5_7e452bbb-7dc6-4c9d-a9b1-76400942f5b4/konnectivity-agent/0.log" limit=1024
Sep 17 23:44:31.915699 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:44:31.915652    2562 prober.go:120] "Probe failed" probeType="Readiness" pod="e2e-test-llm-inference-service-079cb970/llmisvca690bbc929faec8bc98c767f16c003c1-kserve-router-scheb68sc" podUID="c6fcdcef-f170-49ac-968c-f2cacceb70f1" containerName="main" probeResult="failure" output="service unhealthy (responded with \"NOT_SERVING\")"
Sep 17 23:44:41.916225 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:44:41.916158    2562 prober.go:120] "Probe failed" probeType="Readiness" pod="e2e-test-llm-inference-service-079cb970/llmisvca690bbc929faec8bc98c767f16c003c1-kserve-router-scheb68sc" podUID="c6fcdcef-f170-49ac-968c-f2cacceb70f1" containerName="main" probeResult="failure" output="service unhealthy (responded with \"NOT_SERVING\")"
Sep 17 23:44:44.706470 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:44:44.706430    2562 kuberuntime_container.go:873] "Killing container with a grace period" pod="e2e-test-llm-inference-service-079cb970/llmisvca690bbc929faec8bc98c767f16c003c1-kserve-router-scheb68sc" podUID="c6fcdcef-f170-49ac-968c-f2cacceb70f1" containerName="main" containerID="cri-o://8e6041e1e024e292942e4711b5c5196b77b3cf21c92b009f17e89b0c95bc0063" gracePeriod=45
Sep 17 23:44:46.628165 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:44:46.628128    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fwwc5_7e452bbb-7dc6-4c9d-a9b1-76400942f5b4/konnectivity-agent/0.log" limit=1024
Sep 17 23:44:46.825401 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:44:46.825362    2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["e2e-test-llm-inference-service-62fead0b/llmisvc08544b88a8d963ffd553cc1f3ed82d16-kserve-router-schezst2m"]
Sep 17 23:44:46.825884 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:44:46.825867    2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="1c219d63-86d1-47f9-a114-a2635bd799f8" containerName="main"
Sep 17 23:44:46.826021 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:44:46.825887    2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="1c219d63-86d1-47f9-a114-a2635bd799f8" containerName="main"
Sep 17 23:44:46.826021 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:44:46.825972    2562 memory_manager.go:368] "RemoveStaleState removing state" podUID="1c219d63-86d1-47f9-a114-a2635bd799f8" containerName="main"
Sep 17 23:44:46.829307 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:44:46.829283    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-test-llm-inference-service-62fead0b/llmisvc08544b88a8d963ffd553cc1f3ed82d16-kserve-router-schezst2m"
Sep 17 23:44:46.832789 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:44:46.832748    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-test-llm-inference-service-62fead0b\"/\"llmisv08544b88a8d963ffd553cc1f3ed82d16-kserve-self-signed-certs\""
Sep 17 23:44:46.832789 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:44:46.832748    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-test-llm-inference-service-62fead0b\"/\"openshift-service-ca.crt\""
Sep 17 23:44:46.832982 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:44:46.832748    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-test-llm-inference-service-62fead0b\"/\"default-dockercfg-6dldf\""
Sep 17 23:44:46.832982 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:44:46.832812    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-test-llm-inference-service-62fead0b\"/\"kube-root-ca.crt\""
Sep 17 23:44:46.832982 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:44:46.832748    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-test-llm-inference-service-62fead0b\"/\"llmisvc-model-qwen2-5-0-5b-rout-4f8c0978-epp-sa-dockercfg-mx5cc\""
Sep 17 23:44:46.836295 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:44:46.836245    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["e2e-test-llm-inference-service-62fead0b/llmisvc08544b88a8d963ffd553cc1f3ed82d16-kserve-router-schezst2m"]
Sep 17 23:44:46.851561 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:44:46.851519    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fwwc5_7e452bbb-7dc6-4c9d-a9b1-76400942f5b4/konnectivity-agent/0.log" limit=1024
Sep 17 23:44:46.868602 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:44:46.868563    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/d252b56e-6b90-4d69-95a5-2886b972b04e-tls-certs\") pod \"llmisvc08544b88a8d963ffd553cc1f3ed82d16-kserve-router-schezst2m\" (UID: \"d252b56e-6b90-4d69-95a5-2886b972b04e\") " pod="e2e-test-llm-inference-service-62fead0b/llmisvc08544b88a8d963ffd553cc1f3ed82d16-kserve-router-schezst2m"
Sep 17 23:44:46.868772 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:44:46.868629    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5g99g\" (UniqueName: \"kubernetes.io/projected/d252b56e-6b90-4d69-95a5-2886b972b04e-kube-api-access-5g99g\") pod \"llmisvc08544b88a8d963ffd553cc1f3ed82d16-kserve-router-schezst2m\" (UID: \"d252b56e-6b90-4d69-95a5-2886b972b04e\") " pod="e2e-test-llm-inference-service-62fead0b/llmisvc08544b88a8d963ffd553cc1f3ed82d16-kserve-router-schezst2m"
Sep 17 23:44:46.969918 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:44:46.969814    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5g99g\" (UniqueName: \"kubernetes.io/projected/d252b56e-6b90-4d69-95a5-2886b972b04e-kube-api-access-5g99g\") pod \"llmisvc08544b88a8d963ffd553cc1f3ed82d16-kserve-router-schezst2m\" (UID: \"d252b56e-6b90-4d69-95a5-2886b972b04e\") " pod="e2e-test-llm-inference-service-62fead0b/llmisvc08544b88a8d963ffd553cc1f3ed82d16-kserve-router-schezst2m"
Sep 17 23:44:46.970119 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:44:46.969954    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/d252b56e-6b90-4d69-95a5-2886b972b04e-tls-certs\") pod \"llmisvc08544b88a8d963ffd553cc1f3ed82d16-kserve-router-schezst2m\" (UID: \"d252b56e-6b90-4d69-95a5-2886b972b04e\") " pod="e2e-test-llm-inference-service-62fead0b/llmisvc08544b88a8d963ffd553cc1f3ed82d16-kserve-router-schezst2m"
Sep 17 23:44:46.972745 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:44:46.972717    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/d252b56e-6b90-4d69-95a5-2886b972b04e-tls-certs\") pod \"llmisvc08544b88a8d963ffd553cc1f3ed82d16-kserve-router-schezst2m\" (UID: \"d252b56e-6b90-4d69-95a5-2886b972b04e\") " pod="e2e-test-llm-inference-service-62fead0b/llmisvc08544b88a8d963ffd553cc1f3ed82d16-kserve-router-schezst2m"
Sep 17 23:44:46.978141 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:44:46.978109    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-5g99g\" (UniqueName: \"kubernetes.io/projected/d252b56e-6b90-4d69-95a5-2886b972b04e-kube-api-access-5g99g\") pod \"llmisvc08544b88a8d963ffd553cc1f3ed82d16-kserve-router-schezst2m\" (UID: \"d252b56e-6b90-4d69-95a5-2886b972b04e\") " pod="e2e-test-llm-inference-service-62fead0b/llmisvc08544b88a8d963ffd553cc1f3ed82d16-kserve-router-schezst2m"
Sep 17 23:44:47.142150 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:44:47.142117    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-test-llm-inference-service-62fead0b/llmisvc08544b88a8d963ffd553cc1f3ed82d16-kserve-router-schezst2m"
Sep 17 23:44:47.416537 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:44:47.416489    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["e2e-test-llm-inference-service-62fead0b/llmisvc08544b88a8d963ffd553cc1f3ed82d16-kserve-router-schezst2m"]
Sep 17 23:44:47.419523 ip-10-0-132-188 kubenswrapper[2562]: W0917 23:44:47.419490    2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd252b56e_6b90_4d69_95a5_2886b972b04e.slice/crio-8c6b533917cf223d5ea5fa14befdddfd1285d6632a943010837552fb6dece3b8 WatchSource:0}: Error finding container 8c6b533917cf223d5ea5fa14befdddfd1285d6632a943010837552fb6dece3b8: Status 404 returned error can't find the container with id 8c6b533917cf223d5ea5fa14befdddfd1285d6632a943010837552fb6dece3b8
Sep 17 23:44:47.725715 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:44:47.725606    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-test-llm-inference-service-62fead0b/llmisvc08544b88a8d963ffd553cc1f3ed82d16-kserve-router-schezst2m" event={"ID":"d252b56e-6b90-4d69-95a5-2886b972b04e","Type":"ContainerStarted","Data":"9c5b3f69f2806e839e4351e15b6edc3d9ccd7787c946afaeb25ddef3ccb5af87"}
Sep 17 23:44:47.725715 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:44:47.725670    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-test-llm-inference-service-62fead0b/llmisvc08544b88a8d963ffd553cc1f3ed82d16-kserve-router-schezst2m" event={"ID":"d252b56e-6b90-4d69-95a5-2886b972b04e","Type":"ContainerStarted","Data":"8c6b533917cf223d5ea5fa14befdddfd1285d6632a943010837552fb6dece3b8"}
Sep 17 23:44:47.726121 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:44:47.725800    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-test-llm-inference-service-62fead0b/llmisvc08544b88a8d963ffd553cc1f3ed82d16-kserve-router-schezst2m"
Sep 17 23:44:47.741923 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:44:47.741870    2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="e2e-test-llm-inference-service-62fead0b/llmisvc08544b88a8d963ffd553cc1f3ed82d16-kserve-router-schezst2m" podStartSLOduration=1.741854935 podStartE2EDuration="1.741854935s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-17 23:44:46 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-17 23:44:47.740142064 +0000 UTC m=+2040.142038832" watchObservedRunningTime="2026-09-17 23:44:47.741854935 +0000 UTC m=+2040.143751699"
Sep 17 23:44:51.916286 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:44:51.916242    2562 prober.go:120] "Probe failed" probeType="Readiness" pod="e2e-test-llm-inference-service-079cb970/llmisvca690bbc929faec8bc98c767f16c003c1-kserve-router-scheb68sc" podUID="c6fcdcef-f170-49ac-968c-f2cacceb70f1" containerName="main" probeResult="failure" output="service unhealthy (responded with \"NOT_SERVING\")"
Sep 17 23:44:51.916677 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:44:51.916364    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-test-llm-inference-service-079cb970/llmisvca690bbc929faec8bc98c767f16c003c1-kserve-router-scheb68sc"
Sep 17 23:44:53.373241 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:44:53.373202    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fwwc5_7e452bbb-7dc6-4c9d-a9b1-76400942f5b4/konnectivity-agent/0.log" limit=1024
Sep 17 23:44:53.427444 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:44:53.427406    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fwwc5_7e452bbb-7dc6-4c9d-a9b1-76400942f5b4/konnectivity-agent/0.log" limit=1024
Sep 17 23:45:01.915877 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:45:01.915777    2562 prober.go:120] "Probe failed" probeType="Readiness" pod="e2e-test-llm-inference-service-079cb970/llmisvca690bbc929faec8bc98c767f16c003c1-kserve-router-scheb68sc" podUID="c6fcdcef-f170-49ac-968c-f2cacceb70f1" containerName="main" probeResult="failure" output="service unhealthy (responded with \"NOT_SERVING\")"
Sep 17 23:45:09.775745 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:45:09.775690    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fwwc5_7e452bbb-7dc6-4c9d-a9b1-76400942f5b4/konnectivity-agent/0.log" limit=1024
Sep 17 23:45:11.915379 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:45:11.915335    2562 prober.go:120] "Probe failed" probeType="Readiness" pod="e2e-test-llm-inference-service-079cb970/llmisvca690bbc929faec8bc98c767f16c003c1-kserve-router-scheb68sc" podUID="c6fcdcef-f170-49ac-968c-f2cacceb70f1" containerName="main" probeResult="failure" output="service unhealthy (responded with \"NOT_SERVING\")"
Sep 17 23:45:14.866058 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:45:14.866022    2562 generic.go:356] "Generic (PLEG): container finished" podID="c6fcdcef-f170-49ac-968c-f2cacceb70f1" containerID="8e6041e1e024e292942e4711b5c5196b77b3cf21c92b009f17e89b0c95bc0063" exitCode=0
Sep 17 23:45:14.866490 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:45:14.866089    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-test-llm-inference-service-079cb970/llmisvca690bbc929faec8bc98c767f16c003c1-kserve-router-scheb68sc" event={"ID":"c6fcdcef-f170-49ac-968c-f2cacceb70f1","Type":"ContainerDied","Data":"8e6041e1e024e292942e4711b5c5196b77b3cf21c92b009f17e89b0c95bc0063"}
Sep 17 23:45:14.965463 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:45:14.965390    2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-test-llm-inference-service-079cb970/llmisvca690bbc929faec8bc98c767f16c003c1-kserve-router-scheb68sc"
Sep 17 23:45:15.122284 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:45:15.122165    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/c6fcdcef-f170-49ac-968c-f2cacceb70f1-tls-certs\") pod \"c6fcdcef-f170-49ac-968c-f2cacceb70f1\" (UID: \"c6fcdcef-f170-49ac-968c-f2cacceb70f1\") "
Sep 17 23:45:15.122484 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:45:15.122340    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-z57qm\" (UniqueName: \"kubernetes.io/projected/c6fcdcef-f170-49ac-968c-f2cacceb70f1-kube-api-access-z57qm\") pod \"c6fcdcef-f170-49ac-968c-f2cacceb70f1\" (UID: \"c6fcdcef-f170-49ac-968c-f2cacceb70f1\") "
Sep 17 23:45:15.124589 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:45:15.124552    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/c6fcdcef-f170-49ac-968c-f2cacceb70f1-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "c6fcdcef-f170-49ac-968c-f2cacceb70f1" (UID: "c6fcdcef-f170-49ac-968c-f2cacceb70f1"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 17 23:45:15.124688 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:45:15.124566    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/c6fcdcef-f170-49ac-968c-f2cacceb70f1-kube-api-access-z57qm" (OuterVolumeSpecName: "kube-api-access-z57qm") pod "c6fcdcef-f170-49ac-968c-f2cacceb70f1" (UID: "c6fcdcef-f170-49ac-968c-f2cacceb70f1"). InnerVolumeSpecName "kube-api-access-z57qm". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 17 23:45:15.223776 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:45:15.223669    2562 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/c6fcdcef-f170-49ac-968c-f2cacceb70f1-tls-certs\") on node \"ip-10-0-132-188.ec2.internal\" DevicePath \"\""
Sep 17 23:45:15.223776 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:45:15.223709    2562 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-z57qm\" (UniqueName: \"kubernetes.io/projected/c6fcdcef-f170-49ac-968c-f2cacceb70f1-kube-api-access-z57qm\") on node \"ip-10-0-132-188.ec2.internal\" DevicePath \"\""
Sep 17 23:45:15.872292 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:45:15.872259    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-test-llm-inference-service-079cb970/llmisvca690bbc929faec8bc98c767f16c003c1-kserve-router-scheb68sc" event={"ID":"c6fcdcef-f170-49ac-968c-f2cacceb70f1","Type":"ContainerDied","Data":"733bc3d2decd5d09142387c8cfce28d68d3cdcd842a4877d726d96e45776b2bc"}
Sep 17 23:45:15.872292 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:45:15.872291    2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-test-llm-inference-service-079cb970/llmisvca690bbc929faec8bc98c767f16c003c1-kserve-router-scheb68sc"
Sep 17 23:45:15.872778 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:45:15.872327    2562 scope.go:117] "RemoveContainer" containerID="8e6041e1e024e292942e4711b5c5196b77b3cf21c92b009f17e89b0c95bc0063"
Sep 17 23:45:15.887994 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:45:15.887958    2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["e2e-test-llm-inference-service-079cb970/llmisvca690bbc929faec8bc98c767f16c003c1-kserve-router-scheb68sc"]
Sep 17 23:45:15.891480 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:45:15.891449    2562 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["e2e-test-llm-inference-service-079cb970/llmisvca690bbc929faec8bc98c767f16c003c1-kserve-router-scheb68sc"]
Sep 17 23:45:17.667094 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:45:17.667059    2562 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="c6fcdcef-f170-49ac-968c-f2cacceb70f1" path="/var/lib/kubelet/pods/c6fcdcef-f170-49ac-968c-f2cacceb70f1/volumes"
Sep 17 23:45:18.731471 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:45:18.731436    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-test-llm-inference-service-62fead0b/llmisvc08544b88a8d963ffd553cc1f3ed82d16-kserve-router-schezst2m"
Sep 17 23:45:19.784019 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:45:19.783982    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fwwc5_7e452bbb-7dc6-4c9d-a9b1-76400942f5b4/konnectivity-agent/0.log" limit=1024
Sep 17 23:45:19.883437 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:45:19.883399    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fwwc5_7e452bbb-7dc6-4c9d-a9b1-76400942f5b4/konnectivity-agent/0.log" limit=1024
Sep 17 23:45:20.813051 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:45:20.813018    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fwwc5_7e452bbb-7dc6-4c9d-a9b1-76400942f5b4/konnectivity-agent/0.log" limit=1024
Sep 17 23:45:20.817467 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:45:20.817435    2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["e2e-test-llm-inference-service-62fead0b/llmisvc08544b88a8d963ffd553cc1f3ed82d16-kserve-router-schezst2m"]
Sep 17 23:45:20.912326 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:45:20.912285    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fwwc5_7e452bbb-7dc6-4c9d-a9b1-76400942f5b4/konnectivity-agent/0.log" limit=1024
Sep 17 23:45:25.701632 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:45:25.701590    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fwwc5_7e452bbb-7dc6-4c9d-a9b1-76400942f5b4/konnectivity-agent/0.log" limit=1024
Sep 17 23:45:28.731710 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:45:28.731672    2562 prober.go:120] "Probe failed" probeType="Readiness" pod="e2e-test-llm-inference-service-62fead0b/llmisvc08544b88a8d963ffd553cc1f3ed82d16-kserve-router-schezst2m" podUID="d252b56e-6b90-4d69-95a5-2886b972b04e" containerName="main" probeResult="failure" output="service unhealthy (responded with \"NOT_SERVING\")"
Sep 17 23:45:31.097249 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:45:31.097208    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fwwc5_7e452bbb-7dc6-4c9d-a9b1-76400942f5b4/konnectivity-agent/0.log" limit=1024
Sep 17 23:45:31.155130 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:45:31.155098    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fwwc5_7e452bbb-7dc6-4c9d-a9b1-76400942f5b4/konnectivity-agent/0.log" limit=1024
Sep 17 23:45:35.838007 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:45:35.837960    2562 kuberuntime_container.go:873] "Killing container with a grace period" pod="e2e-test-llm-inference-service-62fead0b/llmisvc08544b88a8d963ffd553cc1f3ed82d16-kserve-router-schezst2m" podUID="d252b56e-6b90-4d69-95a5-2886b972b04e" containerName="main" containerID="cri-o://9c5b3f69f2806e839e4351e15b6edc3d9ccd7787c946afaeb25ddef3ccb5af87" gracePeriod=45
Sep 17 23:45:37.694701 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:45:37.694662    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fwwc5_7e452bbb-7dc6-4c9d-a9b1-76400942f5b4/konnectivity-agent/0.log" limit=1024
Sep 17 23:45:37.852065 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:45:37.852022    2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["e2e-test-llm-inference-service-7bc4bd10/llmisvc20d1e8ef181cac7effb0fab7b8590b15-kserve-router-schecq7mc"]
Sep 17 23:45:37.852384 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:45:37.852370    2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c6fcdcef-f170-49ac-968c-f2cacceb70f1" containerName="main"
Sep 17 23:45:37.852432 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:45:37.852386    2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="c6fcdcef-f170-49ac-968c-f2cacceb70f1" containerName="main"
Sep 17 23:45:37.852469 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:45:37.852451    2562 memory_manager.go:368] "RemoveStaleState removing state" podUID="c6fcdcef-f170-49ac-968c-f2cacceb70f1" containerName="main"
Sep 17 23:45:37.855343 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:45:37.855313    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-test-llm-inference-service-7bc4bd10/llmisvc20d1e8ef181cac7effb0fab7b8590b15-kserve-router-schecq7mc"
Sep 17 23:45:37.858608 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:45:37.858551    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-test-llm-inference-service-7bc4bd10\"/\"llmisv20d1e8ef181cac7effb0fab7b8590b15-kserve-self-signed-certs\""
Sep 17 23:45:37.858608 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:45:37.858564    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-test-llm-inference-service-7bc4bd10\"/\"llmisvc-model-fb-opt-125m-oci-r-0fba19de-epp-sa-dockercfg-6q2f6\""
Sep 17 23:45:37.858608 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:45:37.858594    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-test-llm-inference-service-7bc4bd10\"/\"kube-root-ca.crt\""
Sep 17 23:45:37.858923 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:45:37.858878    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-test-llm-inference-service-7bc4bd10\"/\"default-dockercfg-6lp8v\""
Sep 17 23:45:37.858985 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:45:37.858964    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-test-llm-inference-service-7bc4bd10\"/\"openshift-service-ca.crt\""
Sep 17 23:45:37.863127 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:45:37.863090    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["e2e-test-llm-inference-service-7bc4bd10/llmisvc20d1e8ef181cac7effb0fab7b8590b15-kserve-router-schecq7mc"]
Sep 17 23:45:37.878456 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:45:37.878421    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fwwc5_7e452bbb-7dc6-4c9d-a9b1-76400942f5b4/konnectivity-agent/0.log" limit=1024
Sep 17 23:45:37.907319 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:45:37.907283    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8qn2m\" (UniqueName: \"kubernetes.io/projected/e762f179-624c-44c3-a75b-d67f3f63bb08-kube-api-access-8qn2m\") pod \"llmisvc20d1e8ef181cac7effb0fab7b8590b15-kserve-router-schecq7mc\" (UID: \"e762f179-624c-44c3-a75b-d67f3f63bb08\") " pod="e2e-test-llm-inference-service-7bc4bd10/llmisvc20d1e8ef181cac7effb0fab7b8590b15-kserve-router-schecq7mc"
Sep 17 23:45:37.907319 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:45:37.907324    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/e762f179-624c-44c3-a75b-d67f3f63bb08-tls-certs\") pod \"llmisvc20d1e8ef181cac7effb0fab7b8590b15-kserve-router-schecq7mc\" (UID: \"e762f179-624c-44c3-a75b-d67f3f63bb08\") " pod="e2e-test-llm-inference-service-7bc4bd10/llmisvc20d1e8ef181cac7effb0fab7b8590b15-kserve-router-schecq7mc"
Sep 17 23:45:38.008707 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:45:38.008617    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8qn2m\" (UniqueName: \"kubernetes.io/projected/e762f179-624c-44c3-a75b-d67f3f63bb08-kube-api-access-8qn2m\") pod \"llmisvc20d1e8ef181cac7effb0fab7b8590b15-kserve-router-schecq7mc\" (UID: \"e762f179-624c-44c3-a75b-d67f3f63bb08\") " pod="e2e-test-llm-inference-service-7bc4bd10/llmisvc20d1e8ef181cac7effb0fab7b8590b15-kserve-router-schecq7mc"
Sep 17 23:45:38.008707 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:45:38.008657    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/e762f179-624c-44c3-a75b-d67f3f63bb08-tls-certs\") pod \"llmisvc20d1e8ef181cac7effb0fab7b8590b15-kserve-router-schecq7mc\" (UID: \"e762f179-624c-44c3-a75b-d67f3f63bb08\") " pod="e2e-test-llm-inference-service-7bc4bd10/llmisvc20d1e8ef181cac7effb0fab7b8590b15-kserve-router-schecq7mc"
Sep 17 23:45:38.011200 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:45:38.011158    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/e762f179-624c-44c3-a75b-d67f3f63bb08-tls-certs\") pod \"llmisvc20d1e8ef181cac7effb0fab7b8590b15-kserve-router-schecq7mc\" (UID: \"e762f179-624c-44c3-a75b-d67f3f63bb08\") " pod="e2e-test-llm-inference-service-7bc4bd10/llmisvc20d1e8ef181cac7effb0fab7b8590b15-kserve-router-schecq7mc"
Sep 17 23:45:38.016358 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:45:38.016331    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-8qn2m\" (UniqueName: \"kubernetes.io/projected/e762f179-624c-44c3-a75b-d67f3f63bb08-kube-api-access-8qn2m\") pod \"llmisvc20d1e8ef181cac7effb0fab7b8590b15-kserve-router-schecq7mc\" (UID: \"e762f179-624c-44c3-a75b-d67f3f63bb08\") " pod="e2e-test-llm-inference-service-7bc4bd10/llmisvc20d1e8ef181cac7effb0fab7b8590b15-kserve-router-schecq7mc"
Sep 17 23:45:38.167760 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:45:38.167719    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-test-llm-inference-service-7bc4bd10/llmisvc20d1e8ef181cac7effb0fab7b8590b15-kserve-router-schecq7mc"
Sep 17 23:45:38.441253 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:45:38.441225    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["e2e-test-llm-inference-service-7bc4bd10/llmisvc20d1e8ef181cac7effb0fab7b8590b15-kserve-router-schecq7mc"]
Sep 17 23:45:38.443701 ip-10-0-132-188 kubenswrapper[2562]: W0917 23:45:38.443663    2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode762f179_624c_44c3_a75b_d67f3f63bb08.slice/crio-6f6033eff9ceff5c07e9e0a1dfd1733c6152a929382019c7680c804503bd4918 WatchSource:0}: Error finding container 6f6033eff9ceff5c07e9e0a1dfd1733c6152a929382019c7680c804503bd4918: Status 404 returned error can't find the container with id 6f6033eff9ceff5c07e9e0a1dfd1733c6152a929382019c7680c804503bd4918
Sep 17 23:45:38.732086 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:45:38.731988    2562 prober.go:120] "Probe failed" probeType="Readiness" pod="e2e-test-llm-inference-service-62fead0b/llmisvc08544b88a8d963ffd553cc1f3ed82d16-kserve-router-schezst2m" podUID="d252b56e-6b90-4d69-95a5-2886b972b04e" containerName="main" probeResult="failure" output="service unhealthy (responded with \"NOT_SERVING\")"
Sep 17 23:45:38.982451 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:45:38.982356    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-test-llm-inference-service-7bc4bd10/llmisvc20d1e8ef181cac7effb0fab7b8590b15-kserve-router-schecq7mc" event={"ID":"e762f179-624c-44c3-a75b-d67f3f63bb08","Type":"ContainerStarted","Data":"6c4e6bbc6ea75aa8742dad801ca8728995c7b05e49b422dc7fb05f69dbbecfc5"}
Sep 17 23:45:38.982451 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:45:38.982410    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-test-llm-inference-service-7bc4bd10/llmisvc20d1e8ef181cac7effb0fab7b8590b15-kserve-router-schecq7mc" event={"ID":"e762f179-624c-44c3-a75b-d67f3f63bb08","Type":"ContainerStarted","Data":"6f6033eff9ceff5c07e9e0a1dfd1733c6152a929382019c7680c804503bd4918"}
Sep 17 23:45:38.982655 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:45:38.982467    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-test-llm-inference-service-7bc4bd10/llmisvc20d1e8ef181cac7effb0fab7b8590b15-kserve-router-schecq7mc"
Sep 17 23:45:39.000125 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:45:39.000067    2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="e2e-test-llm-inference-service-7bc4bd10/llmisvc20d1e8ef181cac7effb0fab7b8590b15-kserve-router-schecq7mc" podStartSLOduration=2.000051383 podStartE2EDuration="2.000051383s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-17 23:45:37 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-17 23:45:38.998833183 +0000 UTC m=+2091.400729948" watchObservedRunningTime="2026-09-17 23:45:39.000051383 +0000 UTC m=+2091.401948168"
Sep 17 23:45:48.403095 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:45:48.403055    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fwwc5_7e452bbb-7dc6-4c9d-a9b1-76400942f5b4/konnectivity-agent/0.log" limit=1024
Sep 17 23:45:48.731688 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:45:48.731648    2562 prober.go:120] "Probe failed" probeType="Readiness" pod="e2e-test-llm-inference-service-62fead0b/llmisvc08544b88a8d963ffd553cc1f3ed82d16-kserve-router-schezst2m" podUID="d252b56e-6b90-4d69-95a5-2886b972b04e" containerName="main" probeResult="failure" output="service unhealthy (responded with \"NOT_SERVING\")"
Sep 17 23:45:48.731858 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:45:48.731784    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-test-llm-inference-service-62fead0b/llmisvc08544b88a8d963ffd553cc1f3ed82d16-kserve-router-schezst2m"
Sep 17 23:45:58.731626 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:45:58.731588    2562 prober.go:120] "Probe failed" probeType="Readiness" pod="e2e-test-llm-inference-service-62fead0b/llmisvc08544b88a8d963ffd553cc1f3ed82d16-kserve-router-schezst2m" podUID="d252b56e-6b90-4d69-95a5-2886b972b04e" containerName="main" probeResult="failure" output="service unhealthy (responded with \"NOT_SERVING\")"
Sep 17 23:46:06.085458 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:46:06.085432    2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-test-llm-inference-service-62fead0b/llmisvc08544b88a8d963ffd553cc1f3ed82d16-kserve-router-schezst2m"
Sep 17 23:46:06.104544 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:46:06.104464    2562 generic.go:356] "Generic (PLEG): container finished" podID="d252b56e-6b90-4d69-95a5-2886b972b04e" containerID="9c5b3f69f2806e839e4351e15b6edc3d9ccd7787c946afaeb25ddef3ccb5af87" exitCode=0
Sep 17 23:46:06.104544 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:46:06.104507    2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-test-llm-inference-service-62fead0b/llmisvc08544b88a8d963ffd553cc1f3ed82d16-kserve-router-schezst2m"
Sep 17 23:46:06.104729 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:46:06.104543    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-test-llm-inference-service-62fead0b/llmisvc08544b88a8d963ffd553cc1f3ed82d16-kserve-router-schezst2m" event={"ID":"d252b56e-6b90-4d69-95a5-2886b972b04e","Type":"ContainerDied","Data":"9c5b3f69f2806e839e4351e15b6edc3d9ccd7787c946afaeb25ddef3ccb5af87"}
Sep 17 23:46:06.104729 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:46:06.104601    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-test-llm-inference-service-62fead0b/llmisvc08544b88a8d963ffd553cc1f3ed82d16-kserve-router-schezst2m" event={"ID":"d252b56e-6b90-4d69-95a5-2886b972b04e","Type":"ContainerDied","Data":"8c6b533917cf223d5ea5fa14befdddfd1285d6632a943010837552fb6dece3b8"}
Sep 17 23:46:06.104729 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:46:06.104629    2562 scope.go:117] "RemoveContainer" containerID="9c5b3f69f2806e839e4351e15b6edc3d9ccd7787c946afaeb25ddef3ccb5af87"
Sep 17 23:46:06.113546 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:46:06.113524    2562 scope.go:117] "RemoveContainer" containerID="9c5b3f69f2806e839e4351e15b6edc3d9ccd7787c946afaeb25ddef3ccb5af87"
Sep 17 23:46:06.113887 ip-10-0-132-188 kubenswrapper[2562]: E0917 23:46:06.113859    2562 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"9c5b3f69f2806e839e4351e15b6edc3d9ccd7787c946afaeb25ddef3ccb5af87\": container with ID starting with 9c5b3f69f2806e839e4351e15b6edc3d9ccd7787c946afaeb25ddef3ccb5af87 not found: ID does not exist" containerID="9c5b3f69f2806e839e4351e15b6edc3d9ccd7787c946afaeb25ddef3ccb5af87"
Sep 17 23:46:06.113957 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:46:06.113902    2562 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"9c5b3f69f2806e839e4351e15b6edc3d9ccd7787c946afaeb25ddef3ccb5af87"} err="failed to get container status \"9c5b3f69f2806e839e4351e15b6edc3d9ccd7787c946afaeb25ddef3ccb5af87\": rpc error: code = NotFound desc = could not find container \"9c5b3f69f2806e839e4351e15b6edc3d9ccd7787c946afaeb25ddef3ccb5af87\": container with ID starting with 9c5b3f69f2806e839e4351e15b6edc3d9ccd7787c946afaeb25ddef3ccb5af87 not found: ID does not exist"
Sep 17 23:46:06.158336 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:46:06.158304    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/d252b56e-6b90-4d69-95a5-2886b972b04e-tls-certs\") pod \"d252b56e-6b90-4d69-95a5-2886b972b04e\" (UID: \"d252b56e-6b90-4d69-95a5-2886b972b04e\") "
Sep 17 23:46:06.158515 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:46:06.158365    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-5g99g\" (UniqueName: \"kubernetes.io/projected/d252b56e-6b90-4d69-95a5-2886b972b04e-kube-api-access-5g99g\") pod \"d252b56e-6b90-4d69-95a5-2886b972b04e\" (UID: \"d252b56e-6b90-4d69-95a5-2886b972b04e\") "
Sep 17 23:46:06.160662 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:46:06.160629    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/d252b56e-6b90-4d69-95a5-2886b972b04e-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "d252b56e-6b90-4d69-95a5-2886b972b04e" (UID: "d252b56e-6b90-4d69-95a5-2886b972b04e"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 17 23:46:06.160662 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:46:06.160633    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d252b56e-6b90-4d69-95a5-2886b972b04e-kube-api-access-5g99g" (OuterVolumeSpecName: "kube-api-access-5g99g") pod "d252b56e-6b90-4d69-95a5-2886b972b04e" (UID: "d252b56e-6b90-4d69-95a5-2886b972b04e"). InnerVolumeSpecName "kube-api-access-5g99g". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 17 23:46:06.259804 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:46:06.259761    2562 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-5g99g\" (UniqueName: \"kubernetes.io/projected/d252b56e-6b90-4d69-95a5-2886b972b04e-kube-api-access-5g99g\") on node \"ip-10-0-132-188.ec2.internal\" DevicePath \"\""
Sep 17 23:46:06.259804 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:46:06.259796    2562 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/d252b56e-6b90-4d69-95a5-2886b972b04e-tls-certs\") on node \"ip-10-0-132-188.ec2.internal\" DevicePath \"\""
Sep 17 23:46:06.430134 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:46:06.430099    2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["e2e-test-llm-inference-service-62fead0b/llmisvc08544b88a8d963ffd553cc1f3ed82d16-kserve-router-schezst2m"]
Sep 17 23:46:06.434976 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:46:06.434942    2562 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["e2e-test-llm-inference-service-62fead0b/llmisvc08544b88a8d963ffd553cc1f3ed82d16-kserve-router-schezst2m"]
Sep 17 23:46:07.666372 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:46:07.666335    2562 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="d252b56e-6b90-4d69-95a5-2886b972b04e" path="/var/lib/kubelet/pods/d252b56e-6b90-4d69-95a5-2886b972b04e/volumes"
Sep 17 23:46:09.849194 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:46:09.849138    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fwwc5_7e452bbb-7dc6-4c9d-a9b1-76400942f5b4/konnectivity-agent/0.log" limit=1024
Sep 17 23:46:09.904143 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:46:09.904101    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fwwc5_7e452bbb-7dc6-4c9d-a9b1-76400942f5b4/konnectivity-agent/0.log" limit=1024
Sep 17 23:46:09.989160 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:46:09.989126    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-test-llm-inference-service-7bc4bd10/llmisvc20d1e8ef181cac7effb0fab7b8590b15-kserve-router-schecq7mc"
Sep 17 23:46:51.154043 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:46:51.154004    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fwwc5_7e452bbb-7dc6-4c9d-a9b1-76400942f5b4/konnectivity-agent/0.log" limit=1024
Sep 17 23:46:59.898663 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:46:59.898620    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fwwc5_7e452bbb-7dc6-4c9d-a9b1-76400942f5b4/konnectivity-agent/0.log" limit=1024
Sep 17 23:46:59.994553 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:46:59.994519    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fwwc5_7e452bbb-7dc6-4c9d-a9b1-76400942f5b4/konnectivity-agent/0.log" limit=1024
Sep 17 23:47:06.210442 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:47:06.210399    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fwwc5_7e452bbb-7dc6-4c9d-a9b1-76400942f5b4/konnectivity-agent/0.log" limit=1024
Sep 17 23:47:11.632208 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:47:11.632143    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fwwc5_7e452bbb-7dc6-4c9d-a9b1-76400942f5b4/konnectivity-agent/0.log" limit=1024
Sep 17 23:47:11.697756 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:47:11.697716    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fwwc5_7e452bbb-7dc6-4c9d-a9b1-76400942f5b4/konnectivity-agent/0.log" limit=1024
Sep 17 23:47:14.928721 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:47:14.928681    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fwwc5_7e452bbb-7dc6-4c9d-a9b1-76400942f5b4/konnectivity-agent/0.log" limit=1024
Sep 17 23:47:15.187396 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:47:15.187273    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fwwc5_7e452bbb-7dc6-4c9d-a9b1-76400942f5b4/konnectivity-agent/0.log" limit=1024
Sep 17 23:47:23.377105 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:47:23.377065    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fwwc5_7e452bbb-7dc6-4c9d-a9b1-76400942f5b4/konnectivity-agent/0.log" limit=1024
Sep 17 23:47:52.840491 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:47:52.840446    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fwwc5_7e452bbb-7dc6-4c9d-a9b1-76400942f5b4/konnectivity-agent/0.log" limit=1024
Sep 17 23:47:52.841106 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:47:52.841062    2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["e2e-test-llm-inference-service-7bc4bd10/llmisvc20d1e8ef181cac7effb0fab7b8590b15-kserve-router-schecq7mc"]
Sep 17 23:47:52.896530 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:47:52.896495    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fwwc5_7e452bbb-7dc6-4c9d-a9b1-76400942f5b4/konnectivity-agent/0.log" limit=1024
Sep 17 23:47:59.988803 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:47:59.988763    2562 prober.go:120] "Probe failed" probeType="Readiness" pod="e2e-test-llm-inference-service-7bc4bd10/llmisvc20d1e8ef181cac7effb0fab7b8590b15-kserve-router-schecq7mc" podUID="e762f179-624c-44c3-a75b-d67f3f63bb08" containerName="main" probeResult="failure" output="service unhealthy (responded with \"NOT_SERVING\")"
Sep 17 23:48:07.861316 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:48:07.861199    2562 kuberuntime_container.go:873] "Killing container with a grace period" pod="e2e-test-llm-inference-service-7bc4bd10/llmisvc20d1e8ef181cac7effb0fab7b8590b15-kserve-router-schecq7mc" podUID="e762f179-624c-44c3-a75b-d67f3f63bb08" containerName="main" containerID="cri-o://6c4e6bbc6ea75aa8742dad801ca8728995c7b05e49b422dc7fb05f69dbbecfc5" gracePeriod=45
Sep 17 23:48:09.989509 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:48:09.989469    2562 prober.go:120] "Probe failed" probeType="Readiness" pod="e2e-test-llm-inference-service-7bc4bd10/llmisvc20d1e8ef181cac7effb0fab7b8590b15-kserve-router-schecq7mc" podUID="e762f179-624c-44c3-a75b-d67f3f63bb08" containerName="main" probeResult="failure" output="service unhealthy (responded with \"NOT_SERVING\")"
Sep 17 23:48:19.988951 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:48:19.988910    2562 prober.go:120] "Probe failed" probeType="Readiness" pod="e2e-test-llm-inference-service-7bc4bd10/llmisvc20d1e8ef181cac7effb0fab7b8590b15-kserve-router-schecq7mc" podUID="e762f179-624c-44c3-a75b-d67f3f63bb08" containerName="main" probeResult="failure" output="service unhealthy (responded with \"NOT_SERVING\")"
Sep 17 23:48:19.989475 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:48:19.989025    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-test-llm-inference-service-7bc4bd10/llmisvc20d1e8ef181cac7effb0fab7b8590b15-kserve-router-schecq7mc"
Sep 17 23:48:22.064526 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:48:22.064490    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fwwc5_7e452bbb-7dc6-4c9d-a9b1-76400942f5b4/konnectivity-agent/0.log" limit=1024
Sep 17 23:48:29.988870 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:48:29.988827    2562 prober.go:120] "Probe failed" probeType="Readiness" pod="e2e-test-llm-inference-service-7bc4bd10/llmisvc20d1e8ef181cac7effb0fab7b8590b15-kserve-router-schecq7mc" podUID="e762f179-624c-44c3-a75b-d67f3f63bb08" containerName="main" probeResult="failure" output="service unhealthy (responded with \"NOT_SERVING\")"
Sep 17 23:48:38.108365 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:48:38.108328    2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-test-llm-inference-service-7bc4bd10/llmisvc20d1e8ef181cac7effb0fab7b8590b15-kserve-router-schecq7mc"
Sep 17 23:48:38.188517 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:48:38.188409    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-8qn2m\" (UniqueName: \"kubernetes.io/projected/e762f179-624c-44c3-a75b-d67f3f63bb08-kube-api-access-8qn2m\") pod \"e762f179-624c-44c3-a75b-d67f3f63bb08\" (UID: \"e762f179-624c-44c3-a75b-d67f3f63bb08\") "
Sep 17 23:48:38.188517 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:48:38.188454    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/e762f179-624c-44c3-a75b-d67f3f63bb08-tls-certs\") pod \"e762f179-624c-44c3-a75b-d67f3f63bb08\" (UID: \"e762f179-624c-44c3-a75b-d67f3f63bb08\") "
Sep 17 23:48:38.190845 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:48:38.190818    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e762f179-624c-44c3-a75b-d67f3f63bb08-kube-api-access-8qn2m" (OuterVolumeSpecName: "kube-api-access-8qn2m") pod "e762f179-624c-44c3-a75b-d67f3f63bb08" (UID: "e762f179-624c-44c3-a75b-d67f3f63bb08"). InnerVolumeSpecName "kube-api-access-8qn2m". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 17 23:48:38.190955 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:48:38.190820    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/e762f179-624c-44c3-a75b-d67f3f63bb08-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "e762f179-624c-44c3-a75b-d67f3f63bb08" (UID: "e762f179-624c-44c3-a75b-d67f3f63bb08"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 17 23:48:38.290024 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:48:38.289975    2562 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-8qn2m\" (UniqueName: \"kubernetes.io/projected/e762f179-624c-44c3-a75b-d67f3f63bb08-kube-api-access-8qn2m\") on node \"ip-10-0-132-188.ec2.internal\" DevicePath \"\""
Sep 17 23:48:38.290024 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:48:38.290017    2562 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/e762f179-624c-44c3-a75b-d67f3f63bb08-tls-certs\") on node \"ip-10-0-132-188.ec2.internal\" DevicePath \"\""
Sep 17 23:48:38.811992 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:48:38.811948    2562 generic.go:356] "Generic (PLEG): container finished" podID="e762f179-624c-44c3-a75b-d67f3f63bb08" containerID="6c4e6bbc6ea75aa8742dad801ca8728995c7b05e49b422dc7fb05f69dbbecfc5" exitCode=0
Sep 17 23:48:38.812214 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:48:38.812000    2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-test-llm-inference-service-7bc4bd10/llmisvc20d1e8ef181cac7effb0fab7b8590b15-kserve-router-schecq7mc"
Sep 17 23:48:38.812214 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:48:38.812030    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-test-llm-inference-service-7bc4bd10/llmisvc20d1e8ef181cac7effb0fab7b8590b15-kserve-router-schecq7mc" event={"ID":"e762f179-624c-44c3-a75b-d67f3f63bb08","Type":"ContainerDied","Data":"6c4e6bbc6ea75aa8742dad801ca8728995c7b05e49b422dc7fb05f69dbbecfc5"}
Sep 17 23:48:38.812214 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:48:38.812085    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-test-llm-inference-service-7bc4bd10/llmisvc20d1e8ef181cac7effb0fab7b8590b15-kserve-router-schecq7mc" event={"ID":"e762f179-624c-44c3-a75b-d67f3f63bb08","Type":"ContainerDied","Data":"6f6033eff9ceff5c07e9e0a1dfd1733c6152a929382019c7680c804503bd4918"}
Sep 17 23:48:38.812214 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:48:38.812108    2562 scope.go:117] "RemoveContainer" containerID="6c4e6bbc6ea75aa8742dad801ca8728995c7b05e49b422dc7fb05f69dbbecfc5"
Sep 17 23:48:38.820724 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:48:38.820705    2562 scope.go:117] "RemoveContainer" containerID="6c4e6bbc6ea75aa8742dad801ca8728995c7b05e49b422dc7fb05f69dbbecfc5"
Sep 17 23:48:38.820992 ip-10-0-132-188 kubenswrapper[2562]: E0917 23:48:38.820973    2562 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"6c4e6bbc6ea75aa8742dad801ca8728995c7b05e49b422dc7fb05f69dbbecfc5\": container with ID starting with 6c4e6bbc6ea75aa8742dad801ca8728995c7b05e49b422dc7fb05f69dbbecfc5 not found: ID does not exist" containerID="6c4e6bbc6ea75aa8742dad801ca8728995c7b05e49b422dc7fb05f69dbbecfc5"
Sep 17 23:48:38.821040 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:48:38.821001    2562 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"6c4e6bbc6ea75aa8742dad801ca8728995c7b05e49b422dc7fb05f69dbbecfc5"} err="failed to get container status \"6c4e6bbc6ea75aa8742dad801ca8728995c7b05e49b422dc7fb05f69dbbecfc5\": rpc error: code = NotFound desc = could not find container \"6c4e6bbc6ea75aa8742dad801ca8728995c7b05e49b422dc7fb05f69dbbecfc5\": container with ID starting with 6c4e6bbc6ea75aa8742dad801ca8728995c7b05e49b422dc7fb05f69dbbecfc5 not found: ID does not exist"
Sep 17 23:48:38.832812 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:48:38.832781    2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["e2e-test-llm-inference-service-7bc4bd10/llmisvc20d1e8ef181cac7effb0fab7b8590b15-kserve-router-schecq7mc"]
Sep 17 23:48:38.838611 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:48:38.838583    2562 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["e2e-test-llm-inference-service-7bc4bd10/llmisvc20d1e8ef181cac7effb0fab7b8590b15-kserve-router-schecq7mc"]
Sep 17 23:48:39.668915 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:48:39.668874    2562 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="e762f179-624c-44c3-a75b-d67f3f63bb08" path="/var/lib/kubelet/pods/e762f179-624c-44c3-a75b-d67f3f63bb08/volumes"
Sep 17 23:48:42.967925 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:48:42.967892    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fwwc5_7e452bbb-7dc6-4c9d-a9b1-76400942f5b4/konnectivity-agent/0.log" limit=1024
Sep 17 23:48:43.062700 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:48:43.062662    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fwwc5_7e452bbb-7dc6-4c9d-a9b1-76400942f5b4/konnectivity-agent/0.log" limit=1024
Sep 17 23:48:48.994333 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:48:48.994294    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fwwc5_7e452bbb-7dc6-4c9d-a9b1-76400942f5b4/konnectivity-agent/0.log" limit=1024
Sep 17 23:48:54.509901 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:48:54.509860    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fwwc5_7e452bbb-7dc6-4c9d-a9b1-76400942f5b4/konnectivity-agent/0.log" limit=1024
Sep 17 23:48:54.575672 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:48:54.575634    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fwwc5_7e452bbb-7dc6-4c9d-a9b1-76400942f5b4/konnectivity-agent/0.log" limit=1024
Sep 17 23:49:00.621711 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:49:00.621676    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fwwc5_7e452bbb-7dc6-4c9d-a9b1-76400942f5b4/konnectivity-agent/0.log" limit=1024
Sep 17 23:49:00.783845 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:49:00.783806    2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["e2e-test-llm-inference-service-caba0b4a/custom-route-timeout-test-kserve-router-scheduler-54cbcc58rz8lj"]
Sep 17 23:49:00.784438 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:49:00.784410    2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e762f179-624c-44c3-a75b-d67f3f63bb08" containerName="main"
Sep 17 23:49:00.784438 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:49:00.784433    2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="e762f179-624c-44c3-a75b-d67f3f63bb08" containerName="main"
Sep 17 23:49:00.784634 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:49:00.784450    2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d252b56e-6b90-4d69-95a5-2886b972b04e" containerName="main"
Sep 17 23:49:00.784634 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:49:00.784455    2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="d252b56e-6b90-4d69-95a5-2886b972b04e" containerName="main"
Sep 17 23:49:00.784634 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:49:00.784543    2562 memory_manager.go:368] "RemoveStaleState removing state" podUID="d252b56e-6b90-4d69-95a5-2886b972b04e" containerName="main"
Sep 17 23:49:00.784634 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:49:00.784553    2562 memory_manager.go:368] "RemoveStaleState removing state" podUID="e762f179-624c-44c3-a75b-d67f3f63bb08" containerName="main"
Sep 17 23:49:00.788925 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:49:00.788898    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-test-llm-inference-service-caba0b4a/custom-route-timeout-test-kserve-router-scheduler-54cbcc58rz8lj"
Sep 17 23:49:00.793011 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:49:00.792456    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-test-llm-inference-service-caba0b4a\"/\"custom-route-timeout-test-kserve-self-signed-certs\""
Sep 17 23:49:00.793011 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:49:00.792624    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-test-llm-inference-service-caba0b4a\"/\"custom-route-timeout-test-epp-sa-dockercfg-vx5bp\""
Sep 17 23:49:00.793011 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:49:00.792722    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-test-llm-inference-service-caba0b4a\"/\"kube-root-ca.crt\""
Sep 17 23:49:00.793011 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:49:00.792732    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-test-llm-inference-service-caba0b4a\"/\"openshift-service-ca.crt\""
Sep 17 23:49:00.793011 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:49:00.792873    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-test-llm-inference-service-caba0b4a\"/\"default-dockercfg-xlz6l\""
Sep 17 23:49:00.795362 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:49:00.795340    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["e2e-test-llm-inference-service-caba0b4a/custom-route-timeout-test-kserve-router-scheduler-54cbcc58rz8lj"]
Sep 17 23:49:00.819404 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:49:00.819364    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fwwc5_7e452bbb-7dc6-4c9d-a9b1-76400942f5b4/konnectivity-agent/0.log" limit=1024
Sep 17 23:49:00.892473 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:49:00.892376    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/4141f006-949b-4c13-a23a-a18f3810e225-tls-certs\") pod \"custom-route-timeout-test-kserve-router-scheduler-54cbcc58rz8lj\" (UID: \"4141f006-949b-4c13-a23a-a18f3810e225\") " pod="e2e-test-llm-inference-service-caba0b4a/custom-route-timeout-test-kserve-router-scheduler-54cbcc58rz8lj"
Sep 17 23:49:00.892638 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:49:00.892501    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hsgnd\" (UniqueName: \"kubernetes.io/projected/4141f006-949b-4c13-a23a-a18f3810e225-kube-api-access-hsgnd\") pod \"custom-route-timeout-test-kserve-router-scheduler-54cbcc58rz8lj\" (UID: \"4141f006-949b-4c13-a23a-a18f3810e225\") " pod="e2e-test-llm-inference-service-caba0b4a/custom-route-timeout-test-kserve-router-scheduler-54cbcc58rz8lj"
Sep 17 23:49:00.993565 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:49:00.993512    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hsgnd\" (UniqueName: \"kubernetes.io/projected/4141f006-949b-4c13-a23a-a18f3810e225-kube-api-access-hsgnd\") pod \"custom-route-timeout-test-kserve-router-scheduler-54cbcc58rz8lj\" (UID: \"4141f006-949b-4c13-a23a-a18f3810e225\") " pod="e2e-test-llm-inference-service-caba0b4a/custom-route-timeout-test-kserve-router-scheduler-54cbcc58rz8lj"
Sep 17 23:49:00.993777 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:49:00.993601    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/4141f006-949b-4c13-a23a-a18f3810e225-tls-certs\") pod \"custom-route-timeout-test-kserve-router-scheduler-54cbcc58rz8lj\" (UID: \"4141f006-949b-4c13-a23a-a18f3810e225\") " pod="e2e-test-llm-inference-service-caba0b4a/custom-route-timeout-test-kserve-router-scheduler-54cbcc58rz8lj"
Sep 17 23:49:00.996295 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:49:00.996267    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/4141f006-949b-4c13-a23a-a18f3810e225-tls-certs\") pod \"custom-route-timeout-test-kserve-router-scheduler-54cbcc58rz8lj\" (UID: \"4141f006-949b-4c13-a23a-a18f3810e225\") " pod="e2e-test-llm-inference-service-caba0b4a/custom-route-timeout-test-kserve-router-scheduler-54cbcc58rz8lj"
Sep 17 23:49:01.001301 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:49:01.001272    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-hsgnd\" (UniqueName: \"kubernetes.io/projected/4141f006-949b-4c13-a23a-a18f3810e225-kube-api-access-hsgnd\") pod \"custom-route-timeout-test-kserve-router-scheduler-54cbcc58rz8lj\" (UID: \"4141f006-949b-4c13-a23a-a18f3810e225\") " pod="e2e-test-llm-inference-service-caba0b4a/custom-route-timeout-test-kserve-router-scheduler-54cbcc58rz8lj"
Sep 17 23:49:01.101554 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:49:01.101514    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-test-llm-inference-service-caba0b4a/custom-route-timeout-test-kserve-router-scheduler-54cbcc58rz8lj"
Sep 17 23:49:01.375258 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:49:01.375226    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["e2e-test-llm-inference-service-caba0b4a/custom-route-timeout-test-kserve-router-scheduler-54cbcc58rz8lj"]
Sep 17 23:49:01.378009 ip-10-0-132-188 kubenswrapper[2562]: W0917 23:49:01.377972    2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4141f006_949b_4c13_a23a_a18f3810e225.slice/crio-783e587d18ab1e36e064565ee8688961e2463c45024a4afaacd4e475783eaf68 WatchSource:0}: Error finding container 783e587d18ab1e36e064565ee8688961e2463c45024a4afaacd4e475783eaf68: Status 404 returned error can't find the container with id 783e587d18ab1e36e064565ee8688961e2463c45024a4afaacd4e475783eaf68
Sep 17 23:49:01.919556 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:49:01.919521    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-test-llm-inference-service-caba0b4a/custom-route-timeout-test-kserve-router-scheduler-54cbcc58rz8lj" event={"ID":"4141f006-949b-4c13-a23a-a18f3810e225","Type":"ContainerStarted","Data":"7f9db4d4a05aedd145072503a5e79df4bc84beca7977c935510c3feecacc4208"}
Sep 17 23:49:01.919972 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:49:01.919575    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-test-llm-inference-service-caba0b4a/custom-route-timeout-test-kserve-router-scheduler-54cbcc58rz8lj" event={"ID":"4141f006-949b-4c13-a23a-a18f3810e225","Type":"ContainerStarted","Data":"783e587d18ab1e36e064565ee8688961e2463c45024a4afaacd4e475783eaf68"}
Sep 17 23:49:01.919972 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:49:01.919620    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-test-llm-inference-service-caba0b4a/custom-route-timeout-test-kserve-router-scheduler-54cbcc58rz8lj"
Sep 17 23:49:01.934916 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:49:01.934852    2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="e2e-test-llm-inference-service-caba0b4a/custom-route-timeout-test-kserve-router-scheduler-54cbcc58rz8lj" podStartSLOduration=1.934831878 podStartE2EDuration="1.934831878s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-17 23:49:00 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-17 23:49:01.93442793 +0000 UTC m=+2294.336324718" watchObservedRunningTime="2026-09-17 23:49:01.934831878 +0000 UTC m=+2294.336728641"
Sep 17 23:49:22.570958 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:49:22.570917    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fwwc5_7e452bbb-7dc6-4c9d-a9b1-76400942f5b4/konnectivity-agent/0.log" limit=1024
Sep 17 23:49:32.926506 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:49:32.926426    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-test-llm-inference-service-caba0b4a/custom-route-timeout-test-kserve-router-scheduler-54cbcc58rz8lj"
Sep 17 23:49:47.970754 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:49:47.970711    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fwwc5_7e452bbb-7dc6-4c9d-a9b1-76400942f5b4/konnectivity-agent/0.log" limit=1024
Sep 17 23:49:48.026864 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:49:48.026831    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fwwc5_7e452bbb-7dc6-4c9d-a9b1-76400942f5b4/konnectivity-agent/0.log" limit=1024
Sep 17 23:49:59.202947 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:49:59.202911    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fwwc5_7e452bbb-7dc6-4c9d-a9b1-76400942f5b4/konnectivity-agent/0.log" limit=1024
Sep 17 23:50:38.099805 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:50:38.099764    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fwwc5_7e452bbb-7dc6-4c9d-a9b1-76400942f5b4/konnectivity-agent/0.log" limit=1024
Sep 17 23:50:39.115575 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:50:39.115537    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fwwc5_7e452bbb-7dc6-4c9d-a9b1-76400942f5b4/konnectivity-agent/0.log" limit=1024
Sep 17 23:50:43.234340 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:50:43.234305    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fwwc5_7e452bbb-7dc6-4c9d-a9b1-76400942f5b4/konnectivity-agent/0.log" limit=1024
Sep 17 23:50:43.459998 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:50:43.459955    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fwwc5_7e452bbb-7dc6-4c9d-a9b1-76400942f5b4/konnectivity-agent/0.log" limit=1024
Sep 17 23:50:44.919020 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:50:44.918980    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fwwc5_7e452bbb-7dc6-4c9d-a9b1-76400942f5b4/konnectivity-agent/0.log" limit=1024
Sep 17 23:50:50.322668 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:50:50.322626    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fwwc5_7e452bbb-7dc6-4c9d-a9b1-76400942f5b4/konnectivity-agent/0.log" limit=1024
Sep 17 23:50:50.383300 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:50:50.383204    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fwwc5_7e452bbb-7dc6-4c9d-a9b1-76400942f5b4/konnectivity-agent/0.log" limit=1024
Sep 17 23:51:02.479825 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:51:02.479736    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fwwc5_7e452bbb-7dc6-4c9d-a9b1-76400942f5b4/konnectivity-agent/0.log" limit=1024
Sep 17 23:51:08.654098 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:51:08.654060    2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["e2e-test-llm-inference-service-caba0b4a/custom-route-timeout-test-kserve-router-scheduler-54cbcc58rz8lj"]
Sep 17 23:51:08.656455 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:51:08.656421    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fwwc5_7e452bbb-7dc6-4c9d-a9b1-76400942f5b4/konnectivity-agent/0.log" limit=1024
Sep 17 23:51:08.754927 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:51:08.754884    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fwwc5_7e452bbb-7dc6-4c9d-a9b1-76400942f5b4/konnectivity-agent/0.log" limit=1024
Sep 17 23:51:12.927154 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:51:12.927109    2562 prober.go:120] "Probe failed" probeType="Readiness" pod="e2e-test-llm-inference-service-caba0b4a/custom-route-timeout-test-kserve-router-scheduler-54cbcc58rz8lj" podUID="4141f006-949b-4c13-a23a-a18f3810e225" containerName="main" probeResult="failure" output="service unhealthy (responded with \"NOT_SERVING\")"
Sep 17 23:51:22.926306 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:51:22.926259    2562 prober.go:120] "Probe failed" probeType="Readiness" pod="e2e-test-llm-inference-service-caba0b4a/custom-route-timeout-test-kserve-router-scheduler-54cbcc58rz8lj" podUID="4141f006-949b-4c13-a23a-a18f3810e225" containerName="main" probeResult="failure" output="service unhealthy (responded with \"NOT_SERVING\")"
Sep 17 23:51:23.674637 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:51:23.674567    2562 kuberuntime_container.go:873] "Killing container with a grace period" pod="e2e-test-llm-inference-service-caba0b4a/custom-route-timeout-test-kserve-router-scheduler-54cbcc58rz8lj" podUID="4141f006-949b-4c13-a23a-a18f3810e225" containerName="main" containerID="cri-o://7f9db4d4a05aedd145072503a5e79df4bc84beca7977c935510c3feecacc4208" gracePeriod=45
Sep 17 23:51:32.927083 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:51:32.927034    2562 prober.go:120] "Probe failed" probeType="Readiness" pod="e2e-test-llm-inference-service-caba0b4a/custom-route-timeout-test-kserve-router-scheduler-54cbcc58rz8lj" podUID="4141f006-949b-4c13-a23a-a18f3810e225" containerName="main" probeResult="failure" output="service unhealthy (responded with \"NOT_SERVING\")"
Sep 17 23:51:32.927534 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:51:32.927157    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-test-llm-inference-service-caba0b4a/custom-route-timeout-test-kserve-router-scheduler-54cbcc58rz8lj"
Sep 17 23:51:39.315450 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:51:39.315391    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fwwc5_7e452bbb-7dc6-4c9d-a9b1-76400942f5b4/konnectivity-agent/0.log" limit=1024
Sep 17 23:51:39.414062 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:51:39.414028    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fwwc5_7e452bbb-7dc6-4c9d-a9b1-76400942f5b4/konnectivity-agent/0.log" limit=1024
Sep 17 23:51:42.926833 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:51:42.926787    2562 prober.go:120] "Probe failed" probeType="Readiness" pod="e2e-test-llm-inference-service-caba0b4a/custom-route-timeout-test-kserve-router-scheduler-54cbcc58rz8lj" podUID="4141f006-949b-4c13-a23a-a18f3810e225" containerName="main" probeResult="failure" output="service unhealthy (responded with \"NOT_SERVING\")"
Sep 17 23:51:52.926964 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:51:52.926926    2562 prober.go:120] "Probe failed" probeType="Readiness" pod="e2e-test-llm-inference-service-caba0b4a/custom-route-timeout-test-kserve-router-scheduler-54cbcc58rz8lj" podUID="4141f006-949b-4c13-a23a-a18f3810e225" containerName="main" probeResult="failure" output="service unhealthy (responded with \"NOT_SERVING\")"
Sep 17 23:51:53.916778 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:51:53.916750    2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-test-llm-inference-service-caba0b4a/custom-route-timeout-test-kserve-router-scheduler-54cbcc58rz8lj"
Sep 17 23:51:54.032889 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:51:54.032792    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-hsgnd\" (UniqueName: \"kubernetes.io/projected/4141f006-949b-4c13-a23a-a18f3810e225-kube-api-access-hsgnd\") pod \"4141f006-949b-4c13-a23a-a18f3810e225\" (UID: \"4141f006-949b-4c13-a23a-a18f3810e225\") "
Sep 17 23:51:54.032889 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:51:54.032849    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/4141f006-949b-4c13-a23a-a18f3810e225-tls-certs\") pod \"4141f006-949b-4c13-a23a-a18f3810e225\" (UID: \"4141f006-949b-4c13-a23a-a18f3810e225\") "
Sep 17 23:51:54.035124 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:51:54.035089    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/4141f006-949b-4c13-a23a-a18f3810e225-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "4141f006-949b-4c13-a23a-a18f3810e225" (UID: "4141f006-949b-4c13-a23a-a18f3810e225"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 17 23:51:54.035239 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:51:54.035180    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/4141f006-949b-4c13-a23a-a18f3810e225-kube-api-access-hsgnd" (OuterVolumeSpecName: "kube-api-access-hsgnd") pod "4141f006-949b-4c13-a23a-a18f3810e225" (UID: "4141f006-949b-4c13-a23a-a18f3810e225"). InnerVolumeSpecName "kube-api-access-hsgnd". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 17 23:51:54.133873 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:51:54.133830    2562 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-hsgnd\" (UniqueName: \"kubernetes.io/projected/4141f006-949b-4c13-a23a-a18f3810e225-kube-api-access-hsgnd\") on node \"ip-10-0-132-188.ec2.internal\" DevicePath \"\""
Sep 17 23:51:54.133873 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:51:54.133866    2562 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/4141f006-949b-4c13-a23a-a18f3810e225-tls-certs\") on node \"ip-10-0-132-188.ec2.internal\" DevicePath \"\""
Sep 17 23:51:54.717129 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:51:54.717088    2562 generic.go:356] "Generic (PLEG): container finished" podID="4141f006-949b-4c13-a23a-a18f3810e225" containerID="7f9db4d4a05aedd145072503a5e79df4bc84beca7977c935510c3feecacc4208" exitCode=0
Sep 17 23:51:54.717357 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:51:54.717151    2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-test-llm-inference-service-caba0b4a/custom-route-timeout-test-kserve-router-scheduler-54cbcc58rz8lj"
Sep 17 23:51:54.717357 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:51:54.717196    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-test-llm-inference-service-caba0b4a/custom-route-timeout-test-kserve-router-scheduler-54cbcc58rz8lj" event={"ID":"4141f006-949b-4c13-a23a-a18f3810e225","Type":"ContainerDied","Data":"7f9db4d4a05aedd145072503a5e79df4bc84beca7977c935510c3feecacc4208"}
Sep 17 23:51:54.717357 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:51:54.717251    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-test-llm-inference-service-caba0b4a/custom-route-timeout-test-kserve-router-scheduler-54cbcc58rz8lj" event={"ID":"4141f006-949b-4c13-a23a-a18f3810e225","Type":"ContainerDied","Data":"783e587d18ab1e36e064565ee8688961e2463c45024a4afaacd4e475783eaf68"}
Sep 17 23:51:54.717357 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:51:54.717274    2562 scope.go:117] "RemoveContainer" containerID="7f9db4d4a05aedd145072503a5e79df4bc84beca7977c935510c3feecacc4208"
Sep 17 23:51:54.725668 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:51:54.725645    2562 scope.go:117] "RemoveContainer" containerID="7f9db4d4a05aedd145072503a5e79df4bc84beca7977c935510c3feecacc4208"
Sep 17 23:51:54.725939 ip-10-0-132-188 kubenswrapper[2562]: E0917 23:51:54.725920    2562 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"7f9db4d4a05aedd145072503a5e79df4bc84beca7977c935510c3feecacc4208\": container with ID starting with 7f9db4d4a05aedd145072503a5e79df4bc84beca7977c935510c3feecacc4208 not found: ID does not exist" containerID="7f9db4d4a05aedd145072503a5e79df4bc84beca7977c935510c3feecacc4208"
Sep 17 23:51:54.725989 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:51:54.725947    2562 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"7f9db4d4a05aedd145072503a5e79df4bc84beca7977c935510c3feecacc4208"} err="failed to get container status \"7f9db4d4a05aedd145072503a5e79df4bc84beca7977c935510c3feecacc4208\": rpc error: code = NotFound desc = could not find container \"7f9db4d4a05aedd145072503a5e79df4bc84beca7977c935510c3feecacc4208\": container with ID starting with 7f9db4d4a05aedd145072503a5e79df4bc84beca7977c935510c3feecacc4208 not found: ID does not exist"
Sep 17 23:51:54.737202 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:51:54.737159    2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["e2e-test-llm-inference-service-caba0b4a/custom-route-timeout-test-kserve-router-scheduler-54cbcc58rz8lj"]
Sep 17 23:51:54.741769 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:51:54.741743    2562 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["e2e-test-llm-inference-service-caba0b4a/custom-route-timeout-test-kserve-router-scheduler-54cbcc58rz8lj"]
Sep 17 23:51:55.667669 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:51:55.667638    2562 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="4141f006-949b-4c13-a23a-a18f3810e225" path="/var/lib/kubelet/pods/4141f006-949b-4c13-a23a-a18f3810e225/volumes"
Sep 17 23:51:58.813907 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:51:58.813849    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fwwc5_7e452bbb-7dc6-4c9d-a9b1-76400942f5b4/konnectivity-agent/0.log" limit=1024
Sep 17 23:51:58.912159 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:51:58.912117    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fwwc5_7e452bbb-7dc6-4c9d-a9b1-76400942f5b4/konnectivity-agent/0.log" limit=1024
Sep 17 23:51:59.024314 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:51:59.024277    2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["e2e-test-llm-inference-service-028f7809/router-gateway-1-openshift-default-cc7cb45dd-8tcr4"]
Sep 17 23:51:59.024733 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:51:59.024708    2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="4141f006-949b-4c13-a23a-a18f3810e225" containerName="main"
Sep 17 23:51:59.024917 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:51:59.024895    2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="4141f006-949b-4c13-a23a-a18f3810e225" containerName="main"
Sep 17 23:51:59.025051 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:51:59.025037    2562 memory_manager.go:368] "RemoveStaleState removing state" podUID="4141f006-949b-4c13-a23a-a18f3810e225" containerName="main"
Sep 17 23:51:59.029372 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:51:59.029350    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-test-llm-inference-service-028f7809/router-gateway-1-openshift-default-cc7cb45dd-8tcr4"
Sep 17 23:51:59.031825 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:51:59.031768    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-test-llm-inference-service-028f7809\"/\"kube-root-ca.crt\""
Sep 17 23:51:59.031956 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:51:59.031852    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-test-llm-inference-service-028f7809\"/\"openshift-gw-ca-root-cert\""
Sep 17 23:51:59.031956 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:51:59.031895    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-test-llm-inference-service-028f7809\"/\"openshift-service-ca.crt\""
Sep 17 23:51:59.032075 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:51:59.031992    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-test-llm-inference-service-028f7809\"/\"router-gateway-1-openshift-default-dockercfg-lfsxg\""
Sep 17 23:51:59.041048 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:51:59.040963    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["e2e-test-llm-inference-service-028f7809/router-gateway-1-openshift-default-cc7cb45dd-8tcr4"]
Sep 17 23:51:59.058502 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:51:59.058460    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fwwc5_7e452bbb-7dc6-4c9d-a9b1-76400942f5b4/konnectivity-agent/0.log" limit=1024
Sep 17 23:51:59.158521 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:51:59.158485    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fwwc5_7e452bbb-7dc6-4c9d-a9b1-76400942f5b4/konnectivity-agent/0.log" limit=1024
Sep 17 23:51:59.179036 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:51:59.179001    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/f126baf5-b4f4-4593-ae4b-4f9ecab0fb14-istio-token\") pod \"router-gateway-1-openshift-default-cc7cb45dd-8tcr4\" (UID: \"f126baf5-b4f4-4593-ae4b-4f9ecab0fb14\") " pod="e2e-test-llm-inference-service-028f7809/router-gateway-1-openshift-default-cc7cb45dd-8tcr4"
Sep 17 23:51:59.179237 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:51:59.179053    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/f126baf5-b4f4-4593-ae4b-4f9ecab0fb14-workload-certs\") pod \"router-gateway-1-openshift-default-cc7cb45dd-8tcr4\" (UID: \"f126baf5-b4f4-4593-ae4b-4f9ecab0fb14\") " pod="e2e-test-llm-inference-service-028f7809/router-gateway-1-openshift-default-cc7cb45dd-8tcr4"
Sep 17 23:51:59.179237 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:51:59.179101    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6pndd\" (UniqueName: \"kubernetes.io/projected/f126baf5-b4f4-4593-ae4b-4f9ecab0fb14-kube-api-access-6pndd\") pod \"router-gateway-1-openshift-default-cc7cb45dd-8tcr4\" (UID: \"f126baf5-b4f4-4593-ae4b-4f9ecab0fb14\") " pod="e2e-test-llm-inference-service-028f7809/router-gateway-1-openshift-default-cc7cb45dd-8tcr4"
Sep 17 23:51:59.179237 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:51:59.179130    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/f126baf5-b4f4-4593-ae4b-4f9ecab0fb14-istiod-ca-cert\") pod \"router-gateway-1-openshift-default-cc7cb45dd-8tcr4\" (UID: \"f126baf5-b4f4-4593-ae4b-4f9ecab0fb14\") " pod="e2e-test-llm-inference-service-028f7809/router-gateway-1-openshift-default-cc7cb45dd-8tcr4"
Sep 17 23:51:59.179237 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:51:59.179209    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/f126baf5-b4f4-4593-ae4b-4f9ecab0fb14-workload-socket\") pod \"router-gateway-1-openshift-default-cc7cb45dd-8tcr4\" (UID: \"f126baf5-b4f4-4593-ae4b-4f9ecab0fb14\") " pod="e2e-test-llm-inference-service-028f7809/router-gateway-1-openshift-default-cc7cb45dd-8tcr4"
Sep 17 23:51:59.179432 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:51:59.179240    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/f126baf5-b4f4-4593-ae4b-4f9ecab0fb14-credential-socket\") pod \"router-gateway-1-openshift-default-cc7cb45dd-8tcr4\" (UID: \"f126baf5-b4f4-4593-ae4b-4f9ecab0fb14\") " pod="e2e-test-llm-inference-service-028f7809/router-gateway-1-openshift-default-cc7cb45dd-8tcr4"
Sep 17 23:51:59.179432 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:51:59.179281    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/f126baf5-b4f4-4593-ae4b-4f9ecab0fb14-istio-envoy\") pod \"router-gateway-1-openshift-default-cc7cb45dd-8tcr4\" (UID: \"f126baf5-b4f4-4593-ae4b-4f9ecab0fb14\") " pod="e2e-test-llm-inference-service-028f7809/router-gateway-1-openshift-default-cc7cb45dd-8tcr4"
Sep 17 23:51:59.179432 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:51:59.179298    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/f126baf5-b4f4-4593-ae4b-4f9ecab0fb14-istio-data\") pod \"router-gateway-1-openshift-default-cc7cb45dd-8tcr4\" (UID: \"f126baf5-b4f4-4593-ae4b-4f9ecab0fb14\") " pod="e2e-test-llm-inference-service-028f7809/router-gateway-1-openshift-default-cc7cb45dd-8tcr4"
Sep 17 23:51:59.179432 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:51:59.179314    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/f126baf5-b4f4-4593-ae4b-4f9ecab0fb14-istio-podinfo\") pod \"router-gateway-1-openshift-default-cc7cb45dd-8tcr4\" (UID: \"f126baf5-b4f4-4593-ae4b-4f9ecab0fb14\") " pod="e2e-test-llm-inference-service-028f7809/router-gateway-1-openshift-default-cc7cb45dd-8tcr4"
Sep 17 23:51:59.280072 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:51:59.280033    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/f126baf5-b4f4-4593-ae4b-4f9ecab0fb14-istiod-ca-cert\") pod \"router-gateway-1-openshift-default-cc7cb45dd-8tcr4\" (UID: \"f126baf5-b4f4-4593-ae4b-4f9ecab0fb14\") " pod="e2e-test-llm-inference-service-028f7809/router-gateway-1-openshift-default-cc7cb45dd-8tcr4"
Sep 17 23:51:59.280072 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:51:59.280078    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/f126baf5-b4f4-4593-ae4b-4f9ecab0fb14-workload-socket\") pod \"router-gateway-1-openshift-default-cc7cb45dd-8tcr4\" (UID: \"f126baf5-b4f4-4593-ae4b-4f9ecab0fb14\") " pod="e2e-test-llm-inference-service-028f7809/router-gateway-1-openshift-default-cc7cb45dd-8tcr4"
Sep 17 23:51:59.280382 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:51:59.280098    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/f126baf5-b4f4-4593-ae4b-4f9ecab0fb14-credential-socket\") pod \"router-gateway-1-openshift-default-cc7cb45dd-8tcr4\" (UID: \"f126baf5-b4f4-4593-ae4b-4f9ecab0fb14\") " pod="e2e-test-llm-inference-service-028f7809/router-gateway-1-openshift-default-cc7cb45dd-8tcr4"
Sep 17 23:51:59.280382 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:51:59.280129    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/f126baf5-b4f4-4593-ae4b-4f9ecab0fb14-istio-envoy\") pod \"router-gateway-1-openshift-default-cc7cb45dd-8tcr4\" (UID: \"f126baf5-b4f4-4593-ae4b-4f9ecab0fb14\") " pod="e2e-test-llm-inference-service-028f7809/router-gateway-1-openshift-default-cc7cb45dd-8tcr4"
Sep 17 23:51:59.280382 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:51:59.280273    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/f126baf5-b4f4-4593-ae4b-4f9ecab0fb14-istio-data\") pod \"router-gateway-1-openshift-default-cc7cb45dd-8tcr4\" (UID: \"f126baf5-b4f4-4593-ae4b-4f9ecab0fb14\") " pod="e2e-test-llm-inference-service-028f7809/router-gateway-1-openshift-default-cc7cb45dd-8tcr4"
Sep 17 23:51:59.280382 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:51:59.280311    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/f126baf5-b4f4-4593-ae4b-4f9ecab0fb14-istio-podinfo\") pod \"router-gateway-1-openshift-default-cc7cb45dd-8tcr4\" (UID: \"f126baf5-b4f4-4593-ae4b-4f9ecab0fb14\") " pod="e2e-test-llm-inference-service-028f7809/router-gateway-1-openshift-default-cc7cb45dd-8tcr4"
Sep 17 23:51:59.280382 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:51:59.280354    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/f126baf5-b4f4-4593-ae4b-4f9ecab0fb14-istio-token\") pod \"router-gateway-1-openshift-default-cc7cb45dd-8tcr4\" (UID: \"f126baf5-b4f4-4593-ae4b-4f9ecab0fb14\") " pod="e2e-test-llm-inference-service-028f7809/router-gateway-1-openshift-default-cc7cb45dd-8tcr4"
Sep 17 23:51:59.280663 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:51:59.280409    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/f126baf5-b4f4-4593-ae4b-4f9ecab0fb14-workload-certs\") pod \"router-gateway-1-openshift-default-cc7cb45dd-8tcr4\" (UID: \"f126baf5-b4f4-4593-ae4b-4f9ecab0fb14\") " pod="e2e-test-llm-inference-service-028f7809/router-gateway-1-openshift-default-cc7cb45dd-8tcr4"
Sep 17 23:51:59.280663 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:51:59.280469    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6pndd\" (UniqueName: \"kubernetes.io/projected/f126baf5-b4f4-4593-ae4b-4f9ecab0fb14-kube-api-access-6pndd\") pod \"router-gateway-1-openshift-default-cc7cb45dd-8tcr4\" (UID: \"f126baf5-b4f4-4593-ae4b-4f9ecab0fb14\") " pod="e2e-test-llm-inference-service-028f7809/router-gateway-1-openshift-default-cc7cb45dd-8tcr4"
Sep 17 23:51:59.280663 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:51:59.280528    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/f126baf5-b4f4-4593-ae4b-4f9ecab0fb14-workload-socket\") pod \"router-gateway-1-openshift-default-cc7cb45dd-8tcr4\" (UID: \"f126baf5-b4f4-4593-ae4b-4f9ecab0fb14\") " pod="e2e-test-llm-inference-service-028f7809/router-gateway-1-openshift-default-cc7cb45dd-8tcr4"
Sep 17 23:51:59.280663 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:51:59.280577    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/f126baf5-b4f4-4593-ae4b-4f9ecab0fb14-credential-socket\") pod \"router-gateway-1-openshift-default-cc7cb45dd-8tcr4\" (UID: \"f126baf5-b4f4-4593-ae4b-4f9ecab0fb14\") " pod="e2e-test-llm-inference-service-028f7809/router-gateway-1-openshift-default-cc7cb45dd-8tcr4"
Sep 17 23:51:59.280886 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:51:59.280826    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/f126baf5-b4f4-4593-ae4b-4f9ecab0fb14-workload-certs\") pod \"router-gateway-1-openshift-default-cc7cb45dd-8tcr4\" (UID: \"f126baf5-b4f4-4593-ae4b-4f9ecab0fb14\") " pod="e2e-test-llm-inference-service-028f7809/router-gateway-1-openshift-default-cc7cb45dd-8tcr4"
Sep 17 23:51:59.280886 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:51:59.280863    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/f126baf5-b4f4-4593-ae4b-4f9ecab0fb14-istio-data\") pod \"router-gateway-1-openshift-default-cc7cb45dd-8tcr4\" (UID: \"f126baf5-b4f4-4593-ae4b-4f9ecab0fb14\") " pod="e2e-test-llm-inference-service-028f7809/router-gateway-1-openshift-default-cc7cb45dd-8tcr4"
Sep 17 23:51:59.281000 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:51:59.280971    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/f126baf5-b4f4-4593-ae4b-4f9ecab0fb14-istiod-ca-cert\") pod \"router-gateway-1-openshift-default-cc7cb45dd-8tcr4\" (UID: \"f126baf5-b4f4-4593-ae4b-4f9ecab0fb14\") " pod="e2e-test-llm-inference-service-028f7809/router-gateway-1-openshift-default-cc7cb45dd-8tcr4"
Sep 17 23:51:59.283206 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:51:59.283160    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/f126baf5-b4f4-4593-ae4b-4f9ecab0fb14-istio-envoy\") pod \"router-gateway-1-openshift-default-cc7cb45dd-8tcr4\" (UID: \"f126baf5-b4f4-4593-ae4b-4f9ecab0fb14\") " pod="e2e-test-llm-inference-service-028f7809/router-gateway-1-openshift-default-cc7cb45dd-8tcr4"
Sep 17 23:51:59.283340 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:51:59.283322    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/f126baf5-b4f4-4593-ae4b-4f9ecab0fb14-istio-podinfo\") pod \"router-gateway-1-openshift-default-cc7cb45dd-8tcr4\" (UID: \"f126baf5-b4f4-4593-ae4b-4f9ecab0fb14\") " pod="e2e-test-llm-inference-service-028f7809/router-gateway-1-openshift-default-cc7cb45dd-8tcr4"
Sep 17 23:51:59.288198 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:51:59.288142    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/f126baf5-b4f4-4593-ae4b-4f9ecab0fb14-istio-token\") pod \"router-gateway-1-openshift-default-cc7cb45dd-8tcr4\" (UID: \"f126baf5-b4f4-4593-ae4b-4f9ecab0fb14\") " pod="e2e-test-llm-inference-service-028f7809/router-gateway-1-openshift-default-cc7cb45dd-8tcr4"
Sep 17 23:51:59.288198 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:51:59.288162    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-6pndd\" (UniqueName: \"kubernetes.io/projected/f126baf5-b4f4-4593-ae4b-4f9ecab0fb14-kube-api-access-6pndd\") pod \"router-gateway-1-openshift-default-cc7cb45dd-8tcr4\" (UID: \"f126baf5-b4f4-4593-ae4b-4f9ecab0fb14\") " pod="e2e-test-llm-inference-service-028f7809/router-gateway-1-openshift-default-cc7cb45dd-8tcr4"
Sep 17 23:51:59.343254 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:51:59.343210    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-test-llm-inference-service-028f7809/router-gateway-1-openshift-default-cc7cb45dd-8tcr4"
Sep 17 23:51:59.620426 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:51:59.620390    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["e2e-test-llm-inference-service-028f7809/router-gateway-1-openshift-default-cc7cb45dd-8tcr4"]
Sep 17 23:51:59.622713 ip-10-0-132-188 kubenswrapper[2562]: W0917 23:51:59.622674    2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf126baf5_b4f4_4593_ae4b_4f9ecab0fb14.slice/crio-e65e6ad6126e37f75da472681d729c683f75ce41ed04d4f034ed0daea9018a57 WatchSource:0}: Error finding container e65e6ad6126e37f75da472681d729c683f75ce41ed04d4f034ed0daea9018a57: Status 404 returned error can't find the container with id e65e6ad6126e37f75da472681d729c683f75ce41ed04d4f034ed0daea9018a57
Sep 17 23:51:59.624817 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:51:59.624783    2562 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"}
Sep 17 23:51:59.624874 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:51:59.624858    2562 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"}
Sep 17 23:51:59.624910 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:51:59.624888    2562 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"}
Sep 17 23:51:59.743813 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:51:59.743770    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-test-llm-inference-service-028f7809/router-gateway-1-openshift-default-cc7cb45dd-8tcr4" event={"ID":"f126baf5-b4f4-4593-ae4b-4f9ecab0fb14","Type":"ContainerStarted","Data":"e546b8e83c5dab37e2ff0e50db7365293c551fdc57486c9c2da145fbfdae2050"}
Sep 17 23:51:59.744014 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:51:59.743839    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-test-llm-inference-service-028f7809/router-gateway-1-openshift-default-cc7cb45dd-8tcr4" event={"ID":"f126baf5-b4f4-4593-ae4b-4f9ecab0fb14","Type":"ContainerStarted","Data":"e65e6ad6126e37f75da472681d729c683f75ce41ed04d4f034ed0daea9018a57"}
Sep 17 23:51:59.764433 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:51:59.764377    2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="e2e-test-llm-inference-service-028f7809/router-gateway-1-openshift-default-cc7cb45dd-8tcr4" podStartSLOduration=0.764344373 podStartE2EDuration="764.344373ms" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-17 23:51:59 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-17 23:51:59.763196078 +0000 UTC m=+2472.165092837" watchObservedRunningTime="2026-09-17 23:51:59.764344373 +0000 UTC m=+2472.166241136"
Sep 17 23:52:00.344011 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:52:00.343971    2562 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="e2e-test-llm-inference-service-028f7809/router-gateway-1-openshift-default-cc7cb45dd-8tcr4"
Sep 17 23:52:00.348975 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:52:00.348946    2562 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="e2e-test-llm-inference-service-028f7809/router-gateway-1-openshift-default-cc7cb45dd-8tcr4"
Sep 17 23:52:00.748737 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:52:00.748647    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-test-llm-inference-service-028f7809/router-gateway-1-openshift-default-cc7cb45dd-8tcr4"
Sep 17 23:52:00.749527 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:52:00.749508    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-test-llm-inference-service-028f7809/router-gateway-1-openshift-default-cc7cb45dd-8tcr4"
Sep 17 23:52:02.568073 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:52:02.568035    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fwwc5_7e452bbb-7dc6-4c9d-a9b1-76400942f5b4/konnectivity-agent/0.log" limit=1024
Sep 17 23:52:05.059291 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:52:05.059251    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fwwc5_7e452bbb-7dc6-4c9d-a9b1-76400942f5b4/konnectivity-agent/0.log" limit=1024
Sep 17 23:52:10.127083 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:52:10.127043    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fwwc5_7e452bbb-7dc6-4c9d-a9b1-76400942f5b4/konnectivity-agent/0.log" limit=1024
Sep 17 23:52:10.378047 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:52:10.377948    2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["e2e-test-llm-inference-service-028f7809/router-with-refs-test-kserve-router-scheduler-5776bf489f-4cnb5"]
Sep 17 23:52:10.380882 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:52:10.380858    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-test-llm-inference-service-028f7809/router-with-refs-test-kserve-router-scheduler-5776bf489f-4cnb5"
Sep 17 23:52:10.384505 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:52:10.384479    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-test-llm-inference-service-028f7809\"/\"default-dockercfg-48bfg\""
Sep 17 23:52:10.384505 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:52:10.384495    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-test-llm-inference-service-028f7809\"/\"router-with-refs-test-kserve-self-signed-certs\""
Sep 17 23:52:10.384798 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:52:10.384495    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-test-llm-inference-service-028f7809\"/\"router-with-refs-test-epp-sa-dockercfg-krsp5\""
Sep 17 23:52:10.385817 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:52:10.385795    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/611487d3-1111-4db4-b8e6-56578f5c511a-tls-certs\") pod \"router-with-refs-test-kserve-router-scheduler-5776bf489f-4cnb5\" (UID: \"611487d3-1111-4db4-b8e6-56578f5c511a\") " pod="e2e-test-llm-inference-service-028f7809/router-with-refs-test-kserve-router-scheduler-5776bf489f-4cnb5"
Sep 17 23:52:10.385958 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:52:10.385897    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vw8ms\" (UniqueName: \"kubernetes.io/projected/611487d3-1111-4db4-b8e6-56578f5c511a-kube-api-access-vw8ms\") pod \"router-with-refs-test-kserve-router-scheduler-5776bf489f-4cnb5\" (UID: \"611487d3-1111-4db4-b8e6-56578f5c511a\") " pod="e2e-test-llm-inference-service-028f7809/router-with-refs-test-kserve-router-scheduler-5776bf489f-4cnb5"
Sep 17 23:52:10.392253 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:52:10.392210    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["e2e-test-llm-inference-service-028f7809/router-with-refs-test-kserve-router-scheduler-5776bf489f-4cnb5"]
Sep 17 23:52:10.407257 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:52:10.407217    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fwwc5_7e452bbb-7dc6-4c9d-a9b1-76400942f5b4/konnectivity-agent/0.log" limit=1024
Sep 17 23:52:10.486943 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:52:10.486907    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vw8ms\" (UniqueName: \"kubernetes.io/projected/611487d3-1111-4db4-b8e6-56578f5c511a-kube-api-access-vw8ms\") pod \"router-with-refs-test-kserve-router-scheduler-5776bf489f-4cnb5\" (UID: \"611487d3-1111-4db4-b8e6-56578f5c511a\") " pod="e2e-test-llm-inference-service-028f7809/router-with-refs-test-kserve-router-scheduler-5776bf489f-4cnb5"
Sep 17 23:52:10.488332 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:52:10.486993    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/611487d3-1111-4db4-b8e6-56578f5c511a-tls-certs\") pod \"router-with-refs-test-kserve-router-scheduler-5776bf489f-4cnb5\" (UID: \"611487d3-1111-4db4-b8e6-56578f5c511a\") " pod="e2e-test-llm-inference-service-028f7809/router-with-refs-test-kserve-router-scheduler-5776bf489f-4cnb5"
Sep 17 23:52:10.488458 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:52:10.488336    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fwwc5_7e452bbb-7dc6-4c9d-a9b1-76400942f5b4/konnectivity-agent/0.log" limit=1024
Sep 17 23:52:10.490322 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:52:10.490299    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/611487d3-1111-4db4-b8e6-56578f5c511a-tls-certs\") pod \"router-with-refs-test-kserve-router-scheduler-5776bf489f-4cnb5\" (UID: \"611487d3-1111-4db4-b8e6-56578f5c511a\") " pod="e2e-test-llm-inference-service-028f7809/router-with-refs-test-kserve-router-scheduler-5776bf489f-4cnb5"
Sep 17 23:52:10.494816 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:52:10.494791    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-vw8ms\" (UniqueName: \"kubernetes.io/projected/611487d3-1111-4db4-b8e6-56578f5c511a-kube-api-access-vw8ms\") pod \"router-with-refs-test-kserve-router-scheduler-5776bf489f-4cnb5\" (UID: \"611487d3-1111-4db4-b8e6-56578f5c511a\") " pod="e2e-test-llm-inference-service-028f7809/router-with-refs-test-kserve-router-scheduler-5776bf489f-4cnb5"
Sep 17 23:52:10.693643 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:52:10.693530    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-test-llm-inference-service-028f7809/router-with-refs-test-kserve-router-scheduler-5776bf489f-4cnb5"
Sep 17 23:52:10.800985 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:52:10.800955    2562 generic.go:356] "Generic (PLEG): container finished" podID="f126baf5-b4f4-4593-ae4b-4f9ecab0fb14" containerID="e546b8e83c5dab37e2ff0e50db7365293c551fdc57486c9c2da145fbfdae2050" exitCode=0
Sep 17 23:52:10.801199 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:52:10.801041    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-test-llm-inference-service-028f7809/router-gateway-1-openshift-default-cc7cb45dd-8tcr4" event={"ID":"f126baf5-b4f4-4593-ae4b-4f9ecab0fb14","Type":"ContainerDied","Data":"e546b8e83c5dab37e2ff0e50db7365293c551fdc57486c9c2da145fbfdae2050"}
Sep 17 23:52:10.801477 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:52:10.801459    2562 scope.go:117] "RemoveContainer" containerID="e546b8e83c5dab37e2ff0e50db7365293c551fdc57486c9c2da145fbfdae2050"
Sep 17 23:52:10.802407 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:52:10.802373    2562 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"}
Sep 17 23:52:10.802510 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:52:10.802445    2562 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"}
Sep 17 23:52:10.802510 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:52:10.802484    2562 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"}
Sep 17 23:52:10.970643 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:52:10.970493    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["e2e-test-llm-inference-service-028f7809/router-with-refs-test-kserve-router-scheduler-5776bf489f-4cnb5"]
Sep 17 23:52:10.972953 ip-10-0-132-188 kubenswrapper[2562]: W0917 23:52:10.972923    2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod611487d3_1111_4db4_b8e6_56578f5c511a.slice/crio-3e13792ca6a210adc32c93166b8478b49cdd6ccf57b907d0a42708c491c15467 WatchSource:0}: Error finding container 3e13792ca6a210adc32c93166b8478b49cdd6ccf57b907d0a42708c491c15467: Status 404 returned error can't find the container with id 3e13792ca6a210adc32c93166b8478b49cdd6ccf57b907d0a42708c491c15467
Sep 17 23:52:11.288272 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:52:11.288144    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fwwc5_7e452bbb-7dc6-4c9d-a9b1-76400942f5b4/konnectivity-agent/0.log" limit=1024
Sep 17 23:52:11.388192 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:52:11.388148    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fwwc5_7e452bbb-7dc6-4c9d-a9b1-76400942f5b4/konnectivity-agent/0.log" limit=1024
Sep 17 23:52:11.807933 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:52:11.807891    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-test-llm-inference-service-028f7809/router-with-refs-test-kserve-router-scheduler-5776bf489f-4cnb5" event={"ID":"611487d3-1111-4db4-b8e6-56578f5c511a","Type":"ContainerStarted","Data":"61472a5477f1515f7b2528a840a9bb6c7067a4845b00c684861294f5494f045c"}
Sep 17 23:52:11.808138 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:52:11.808058    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-test-llm-inference-service-028f7809/router-with-refs-test-kserve-router-scheduler-5776bf489f-4cnb5"
Sep 17 23:52:11.808138 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:52:11.808106    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-test-llm-inference-service-028f7809/router-with-refs-test-kserve-router-scheduler-5776bf489f-4cnb5" event={"ID":"611487d3-1111-4db4-b8e6-56578f5c511a","Type":"ContainerStarted","Data":"3e13792ca6a210adc32c93166b8478b49cdd6ccf57b907d0a42708c491c15467"}
Sep 17 23:52:11.809800 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:52:11.809775    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-test-llm-inference-service-028f7809/router-gateway-1-openshift-default-cc7cb45dd-8tcr4" event={"ID":"f126baf5-b4f4-4593-ae4b-4f9ecab0fb14","Type":"ContainerStarted","Data":"ff34abc559e1a0d48da85202454b3d713c6d13329764364c860111cbcbd66f36"}
Sep 17 23:52:11.824246 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:52:11.824183    2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="e2e-test-llm-inference-service-028f7809/router-with-refs-test-kserve-router-scheduler-5776bf489f-4cnb5" podStartSLOduration=1.8241512819999999 podStartE2EDuration="1.824151282s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-17 23:52:10 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-17 23:52:11.822746789 +0000 UTC m=+2484.224643554" watchObservedRunningTime="2026-09-17 23:52:11.824151282 +0000 UTC m=+2484.226048095"
Sep 17 23:52:12.344463 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:52:12.344420    2562 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="e2e-test-llm-inference-service-028f7809/router-gateway-1-openshift-default-cc7cb45dd-8tcr4"
Sep 17 23:52:12.345854 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:52:12.345810    2562 prober.go:120] "Probe failed" probeType="Startup" pod="e2e-test-llm-inference-service-028f7809/router-gateway-1-openshift-default-cc7cb45dd-8tcr4" podUID="f126baf5-b4f4-4593-ae4b-4f9ecab0fb14" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.49:15021/healthz/ready\": dial tcp 10.134.0.49:15021: connect: connection refused"
Sep 17 23:52:13.344426 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:52:13.344378    2562 prober.go:120] "Probe failed" probeType="Startup" pod="e2e-test-llm-inference-service-028f7809/router-gateway-1-openshift-default-cc7cb45dd-8tcr4" podUID="f126baf5-b4f4-4593-ae4b-4f9ecab0fb14" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.49:15021/healthz/ready\": dial tcp 10.134.0.49:15021: connect: connection refused"
Sep 17 23:52:14.344663 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:52:14.344611    2562 prober.go:120] "Probe failed" probeType="Startup" pod="e2e-test-llm-inference-service-028f7809/router-gateway-1-openshift-default-cc7cb45dd-8tcr4" podUID="f126baf5-b4f4-4593-ae4b-4f9ecab0fb14" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.49:15021/healthz/ready\": dial tcp 10.134.0.49:15021: connect: connection refused"
Sep 17 23:52:15.343885 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:52:15.343835    2562 prober.go:120] "Probe failed" probeType="Startup" pod="e2e-test-llm-inference-service-028f7809/router-gateway-1-openshift-default-cc7cb45dd-8tcr4" podUID="f126baf5-b4f4-4593-ae4b-4f9ecab0fb14" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.49:15021/healthz/ready\": dial tcp 10.134.0.49:15021: connect: connection refused"
Sep 17 23:52:15.748984 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:52:15.748858    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-test-llm-inference-service-028f7809/router-gateway-1-openshift-default-cc7cb45dd-8tcr4"
Sep 17 23:52:16.344459 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:52:16.344407    2562 prober.go:120] "Probe failed" probeType="Startup" pod="e2e-test-llm-inference-service-028f7809/router-gateway-1-openshift-default-cc7cb45dd-8tcr4" podUID="f126baf5-b4f4-4593-ae4b-4f9ecab0fb14" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.49:15021/healthz/ready\": dial tcp 10.134.0.49:15021: connect: connection refused"
Sep 17 23:52:17.348204 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:52:17.348139    2562 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="e2e-test-llm-inference-service-028f7809/router-gateway-1-openshift-default-cc7cb45dd-8tcr4"
Sep 17 23:52:17.349407 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:52:17.349384    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-test-llm-inference-service-028f7809/router-gateway-1-openshift-default-cc7cb45dd-8tcr4"
Sep 17 23:52:22.279058 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:52:22.279017    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fwwc5_7e452bbb-7dc6-4c9d-a9b1-76400942f5b4/konnectivity-agent/0.log" limit=1024
Sep 17 23:52:22.379690 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:52:22.379650    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fwwc5_7e452bbb-7dc6-4c9d-a9b1-76400942f5b4/konnectivity-agent/0.log" limit=1024
Sep 17 23:52:42.817113 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:52:42.817034    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-test-llm-inference-service-028f7809/router-with-refs-test-kserve-router-scheduler-5776bf489f-4cnb5"
Sep 17 23:53:10.733417 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:53:10.733374    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fwwc5_7e452bbb-7dc6-4c9d-a9b1-76400942f5b4/konnectivity-agent/0.log" limit=1024
Sep 17 23:53:12.038670 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:53:12.038627    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fwwc5_7e452bbb-7dc6-4c9d-a9b1-76400942f5b4/konnectivity-agent/0.log" limit=1024
Sep 17 23:53:12.134599 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:53:12.134564    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fwwc5_7e452bbb-7dc6-4c9d-a9b1-76400942f5b4/konnectivity-agent/0.log" limit=1024
Sep 17 23:53:18.201668 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:53:18.201623    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fwwc5_7e452bbb-7dc6-4c9d-a9b1-76400942f5b4/konnectivity-agent/0.log" limit=1024
Sep 17 23:53:23.741895 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:53:23.741849    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fwwc5_7e452bbb-7dc6-4c9d-a9b1-76400942f5b4/konnectivity-agent/0.log" limit=1024
Sep 17 23:53:23.805529 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:53:23.805491    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fwwc5_7e452bbb-7dc6-4c9d-a9b1-76400942f5b4/konnectivity-agent/0.log" limit=1024
Sep 17 23:53:27.227232 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:53:27.227127    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fwwc5_7e452bbb-7dc6-4c9d-a9b1-76400942f5b4/konnectivity-agent/0.log" limit=1024
Sep 17 23:53:27.484113 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:53:27.484018    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fwwc5_7e452bbb-7dc6-4c9d-a9b1-76400942f5b4/konnectivity-agent/0.log" limit=1024
Sep 17 23:53:37.945404 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:53:37.945367    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fwwc5_7e452bbb-7dc6-4c9d-a9b1-76400942f5b4/konnectivity-agent/0.log" limit=1024
Sep 17 23:53:48.523664 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:53:48.523621    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fwwc5_7e452bbb-7dc6-4c9d-a9b1-76400942f5b4/konnectivity-agent/0.log" limit=1024
Sep 17 23:53:48.571683 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:53:48.571653    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fwwc5_7e452bbb-7dc6-4c9d-a9b1-76400942f5b4/konnectivity-agent/0.log" limit=1024
Sep 17 23:54:14.304382 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:54:14.304294    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fwwc5_7e452bbb-7dc6-4c9d-a9b1-76400942f5b4/konnectivity-agent/0.log" limit=1024
Sep 17 23:55:19.238913 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:55:19.238870    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fwwc5_7e452bbb-7dc6-4c9d-a9b1-76400942f5b4/konnectivity-agent/0.log" limit=1024
Sep 17 23:55:19.582195 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:55:19.582142    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fwwc5_7e452bbb-7dc6-4c9d-a9b1-76400942f5b4/konnectivity-agent/0.log" limit=1024
Sep 17 23:55:19.682407 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:55:19.682361    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fwwc5_7e452bbb-7dc6-4c9d-a9b1-76400942f5b4/konnectivity-agent/0.log" limit=1024
Sep 17 23:55:36.994512 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:55:36.994415    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fwwc5_7e452bbb-7dc6-4c9d-a9b1-76400942f5b4/konnectivity-agent/0.log" limit=1024
Sep 17 23:55:37.163834 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:55:37.163794    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fwwc5_7e452bbb-7dc6-4c9d-a9b1-76400942f5b4/konnectivity-agent/0.log" limit=1024
Sep 17 23:56:03.012327 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:56:03.012288    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fwwc5_7e452bbb-7dc6-4c9d-a9b1-76400942f5b4/konnectivity-agent/0.log" limit=1024
Sep 17 23:56:04.036035 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:56:04.035995    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fwwc5_7e452bbb-7dc6-4c9d-a9b1-76400942f5b4/konnectivity-agent/0.log" limit=1024
Sep 17 23:56:34.671711 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:56:34.671673    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fwwc5_7e452bbb-7dc6-4c9d-a9b1-76400942f5b4/konnectivity-agent/0.log" limit=1024
Sep 17 23:57:38.804130 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:57:38.804088    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fwwc5_7e452bbb-7dc6-4c9d-a9b1-76400942f5b4/konnectivity-agent/0.log" limit=1024
Sep 17 23:57:38.899223 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:57:38.899144    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fwwc5_7e452bbb-7dc6-4c9d-a9b1-76400942f5b4/konnectivity-agent/0.log" limit=1024
Sep 17 23:58:19.837037 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:58:19.836356    2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["e2e-test-llm-inference-service-028f7809/router-with-refs-test-kserve-router-scheduler-5776bf489f-4cnb5"]
Sep 17 23:58:19.843858 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:58:19.843810    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fwwc5_7e452bbb-7dc6-4c9d-a9b1-76400942f5b4/konnectivity-agent/0.log" limit=1024
Sep 17 23:58:19.903559 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:58:19.903520    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fwwc5_7e452bbb-7dc6-4c9d-a9b1-76400942f5b4/konnectivity-agent/0.log" limit=1024
Sep 17 23:58:22.817036 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:58:22.816994    2562 prober.go:120] "Probe failed" probeType="Readiness" pod="e2e-test-llm-inference-service-028f7809/router-with-refs-test-kserve-router-scheduler-5776bf489f-4cnb5" podUID="611487d3-1111-4db4-b8e6-56578f5c511a" containerName="main" probeResult="failure" output="service unhealthy (responded with \"NOT_SERVING\")"
Sep 17 23:58:28.423763 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:58:28.423730    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fwwc5_7e452bbb-7dc6-4c9d-a9b1-76400942f5b4/konnectivity-agent/0.log" limit=1024
Sep 17 23:58:28.523699 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:58:28.523662    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fwwc5_7e452bbb-7dc6-4c9d-a9b1-76400942f5b4/konnectivity-agent/0.log" limit=1024
Sep 17 23:58:28.627061 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:58:28.627025    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fwwc5_7e452bbb-7dc6-4c9d-a9b1-76400942f5b4/konnectivity-agent/0.log" limit=1024
Sep 17 23:58:32.816194 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:58:32.816087    2562 prober.go:120] "Probe failed" probeType="Readiness" pod="e2e-test-llm-inference-service-028f7809/router-with-refs-test-kserve-router-scheduler-5776bf489f-4cnb5" podUID="611487d3-1111-4db4-b8e6-56578f5c511a" containerName="main" probeResult="failure" output="service unhealthy (responded with \"NOT_SERVING\")"
Sep 17 23:58:32.852917 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:58:32.852880    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fwwc5_7e452bbb-7dc6-4c9d-a9b1-76400942f5b4/konnectivity-agent/0.log" limit=1024
Sep 17 23:58:34.858911 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:58:34.858852    2562 kuberuntime_container.go:873] "Killing container with a grace period" pod="e2e-test-llm-inference-service-028f7809/router-with-refs-test-kserve-router-scheduler-5776bf489f-4cnb5" podUID="611487d3-1111-4db4-b8e6-56578f5c511a" containerName="main" containerID="cri-o://61472a5477f1515f7b2528a840a9bb6c7067a4845b00c684861294f5494f045c" gracePeriod=45
Sep 17 23:58:34.871963 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:58:34.871930    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fwwc5_7e452bbb-7dc6-4c9d-a9b1-76400942f5b4/konnectivity-agent/0.log" limit=1024
Sep 17 23:58:40.261001 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:58:40.260961    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fwwc5_7e452bbb-7dc6-4c9d-a9b1-76400942f5b4/konnectivity-agent/0.log" limit=1024
Sep 17 23:58:40.360911 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:58:40.360864    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fwwc5_7e452bbb-7dc6-4c9d-a9b1-76400942f5b4/konnectivity-agent/0.log" limit=1024
Sep 17 23:58:42.816600 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:58:42.816563    2562 prober.go:120] "Probe failed" probeType="Readiness" pod="e2e-test-llm-inference-service-028f7809/router-with-refs-test-kserve-router-scheduler-5776bf489f-4cnb5" podUID="611487d3-1111-4db4-b8e6-56578f5c511a" containerName="main" probeResult="failure" output="service unhealthy (responded with \"NOT_SERVING\")"
Sep 17 23:58:42.817012 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:58:42.816678    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-test-llm-inference-service-028f7809/router-with-refs-test-kserve-router-scheduler-5776bf489f-4cnb5"
Sep 17 23:58:52.816651 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:58:52.816610    2562 prober.go:120] "Probe failed" probeType="Readiness" pod="e2e-test-llm-inference-service-028f7809/router-with-refs-test-kserve-router-scheduler-5776bf489f-4cnb5" podUID="611487d3-1111-4db4-b8e6-56578f5c511a" containerName="main" probeResult="failure" output="service unhealthy (responded with \"NOT_SERVING\")"
Sep 17 23:59:02.816928 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:59:02.816889    2562 prober.go:120] "Probe failed" probeType="Readiness" pod="e2e-test-llm-inference-service-028f7809/router-with-refs-test-kserve-router-scheduler-5776bf489f-4cnb5" podUID="611487d3-1111-4db4-b8e6-56578f5c511a" containerName="main" probeResult="failure" output="service unhealthy (responded with \"NOT_SERVING\")"
Sep 17 23:59:05.108567 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:59:05.108539    2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-test-llm-inference-service-028f7809/router-with-refs-test-kserve-router-scheduler-5776bf489f-4cnb5"
Sep 17 23:59:05.170299 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:59:05.170199    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/611487d3-1111-4db4-b8e6-56578f5c511a-tls-certs\") pod \"611487d3-1111-4db4-b8e6-56578f5c511a\" (UID: \"611487d3-1111-4db4-b8e6-56578f5c511a\") "
Sep 17 23:59:05.170299 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:59:05.170256    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-vw8ms\" (UniqueName: \"kubernetes.io/projected/611487d3-1111-4db4-b8e6-56578f5c511a-kube-api-access-vw8ms\") pod \"611487d3-1111-4db4-b8e6-56578f5c511a\" (UID: \"611487d3-1111-4db4-b8e6-56578f5c511a\") "
Sep 17 23:59:05.172497 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:59:05.172471    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/611487d3-1111-4db4-b8e6-56578f5c511a-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "611487d3-1111-4db4-b8e6-56578f5c511a" (UID: "611487d3-1111-4db4-b8e6-56578f5c511a"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 17 23:59:05.172623 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:59:05.172602    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/611487d3-1111-4db4-b8e6-56578f5c511a-kube-api-access-vw8ms" (OuterVolumeSpecName: "kube-api-access-vw8ms") pod "611487d3-1111-4db4-b8e6-56578f5c511a" (UID: "611487d3-1111-4db4-b8e6-56578f5c511a"). InnerVolumeSpecName "kube-api-access-vw8ms". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 17 23:59:05.271582 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:59:05.271540    2562 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/611487d3-1111-4db4-b8e6-56578f5c511a-tls-certs\") on node \"ip-10-0-132-188.ec2.internal\" DevicePath \"\""
Sep 17 23:59:05.271582 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:59:05.271578    2562 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-vw8ms\" (UniqueName: \"kubernetes.io/projected/611487d3-1111-4db4-b8e6-56578f5c511a-kube-api-access-vw8ms\") on node \"ip-10-0-132-188.ec2.internal\" DevicePath \"\""
Sep 17 23:59:05.750748 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:59:05.750705    2562 generic.go:356] "Generic (PLEG): container finished" podID="611487d3-1111-4db4-b8e6-56578f5c511a" containerID="61472a5477f1515f7b2528a840a9bb6c7067a4845b00c684861294f5494f045c" exitCode=0
Sep 17 23:59:05.750926 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:59:05.750757    2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-test-llm-inference-service-028f7809/router-with-refs-test-kserve-router-scheduler-5776bf489f-4cnb5"
Sep 17 23:59:05.750926 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:59:05.750790    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-test-llm-inference-service-028f7809/router-with-refs-test-kserve-router-scheduler-5776bf489f-4cnb5" event={"ID":"611487d3-1111-4db4-b8e6-56578f5c511a","Type":"ContainerDied","Data":"61472a5477f1515f7b2528a840a9bb6c7067a4845b00c684861294f5494f045c"}
Sep 17 23:59:05.750926 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:59:05.750842    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-test-llm-inference-service-028f7809/router-with-refs-test-kserve-router-scheduler-5776bf489f-4cnb5" event={"ID":"611487d3-1111-4db4-b8e6-56578f5c511a","Type":"ContainerDied","Data":"3e13792ca6a210adc32c93166b8478b49cdd6ccf57b907d0a42708c491c15467"}
Sep 17 23:59:05.750926 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:59:05.750864    2562 scope.go:117] "RemoveContainer" containerID="61472a5477f1515f7b2528a840a9bb6c7067a4845b00c684861294f5494f045c"
Sep 17 23:59:05.761391 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:59:05.761366    2562 scope.go:117] "RemoveContainer" containerID="61472a5477f1515f7b2528a840a9bb6c7067a4845b00c684861294f5494f045c"
Sep 17 23:59:05.761702 ip-10-0-132-188 kubenswrapper[2562]: E0917 23:59:05.761677    2562 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"61472a5477f1515f7b2528a840a9bb6c7067a4845b00c684861294f5494f045c\": container with ID starting with 61472a5477f1515f7b2528a840a9bb6c7067a4845b00c684861294f5494f045c not found: ID does not exist" containerID="61472a5477f1515f7b2528a840a9bb6c7067a4845b00c684861294f5494f045c"
Sep 17 23:59:05.761764 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:59:05.761714    2562 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"61472a5477f1515f7b2528a840a9bb6c7067a4845b00c684861294f5494f045c"} err="failed to get container status \"61472a5477f1515f7b2528a840a9bb6c7067a4845b00c684861294f5494f045c\": rpc error: code = NotFound desc = could not find container \"61472a5477f1515f7b2528a840a9bb6c7067a4845b00c684861294f5494f045c\": container with ID starting with 61472a5477f1515f7b2528a840a9bb6c7067a4845b00c684861294f5494f045c not found: ID does not exist"
Sep 17 23:59:05.766667 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:59:05.766637    2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["e2e-test-llm-inference-service-028f7809/router-with-refs-test-kserve-router-scheduler-5776bf489f-4cnb5"]
Sep 17 23:59:05.770873 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:59:05.770847    2562 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["e2e-test-llm-inference-service-028f7809/router-with-refs-test-kserve-router-scheduler-5776bf489f-4cnb5"]
Sep 17 23:59:07.667125 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:59:07.667091    2562 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="611487d3-1111-4db4-b8e6-56578f5c511a" path="/var/lib/kubelet/pods/611487d3-1111-4db4-b8e6-56578f5c511a/volumes"
Sep 17 23:59:09.911020 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:59:09.910979    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fwwc5_7e452bbb-7dc6-4c9d-a9b1-76400942f5b4/konnectivity-agent/0.log" limit=1024
Sep 17 23:59:10.006985 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:59:10.006945    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fwwc5_7e452bbb-7dc6-4c9d-a9b1-76400942f5b4/konnectivity-agent/0.log" limit=1024
Sep 17 23:59:14.864639 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:59:14.864598    2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["e2e-test-llm-inference-service-028f7809/router-gateway-1-openshift-default-cc7cb45dd-8tcr4"]
Sep 17 23:59:14.865083 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:59:14.864805    2562 kuberuntime_container.go:873] "Killing container with a grace period" pod="e2e-test-llm-inference-service-028f7809/router-gateway-1-openshift-default-cc7cb45dd-8tcr4" podUID="f126baf5-b4f4-4593-ae4b-4f9ecab0fb14" containerName="istio-proxy" containerID="cri-o://ff34abc559e1a0d48da85202454b3d713c6d13329764364c860111cbcbd66f36" gracePeriod=30
Sep 17 23:59:14.915258 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:59:14.915216    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fwwc5_7e452bbb-7dc6-4c9d-a9b1-76400942f5b4/konnectivity-agent/0.log" limit=1024
Sep 17 23:59:17.351547 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:59:17.351502    2562 prober.go:120] "Probe failed" probeType="Readiness" pod="e2e-test-llm-inference-service-028f7809/router-gateway-1-openshift-default-cc7cb45dd-8tcr4" podUID="f126baf5-b4f4-4593-ae4b-4f9ecab0fb14" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Sep 17 23:59:20.111629 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:59:20.111603    2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-test-llm-inference-service-028f7809/router-gateway-1-openshift-default-cc7cb45dd-8tcr4"
Sep 17 23:59:20.201509 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:59:20.201478    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/f126baf5-b4f4-4593-ae4b-4f9ecab0fb14-istiod-ca-cert\") pod \"f126baf5-b4f4-4593-ae4b-4f9ecab0fb14\" (UID: \"f126baf5-b4f4-4593-ae4b-4f9ecab0fb14\") "
Sep 17 23:59:20.201684 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:59:20.201519    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/f126baf5-b4f4-4593-ae4b-4f9ecab0fb14-istio-token\") pod \"f126baf5-b4f4-4593-ae4b-4f9ecab0fb14\" (UID: \"f126baf5-b4f4-4593-ae4b-4f9ecab0fb14\") "
Sep 17 23:59:20.201684 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:59:20.201552    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/f126baf5-b4f4-4593-ae4b-4f9ecab0fb14-workload-socket\") pod \"f126baf5-b4f4-4593-ae4b-4f9ecab0fb14\" (UID: \"f126baf5-b4f4-4593-ae4b-4f9ecab0fb14\") "
Sep 17 23:59:20.201684 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:59:20.201583    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/f126baf5-b4f4-4593-ae4b-4f9ecab0fb14-istio-data\") pod \"f126baf5-b4f4-4593-ae4b-4f9ecab0fb14\" (UID: \"f126baf5-b4f4-4593-ae4b-4f9ecab0fb14\") "
Sep 17 23:59:20.201684 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:59:20.201630    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/f126baf5-b4f4-4593-ae4b-4f9ecab0fb14-istio-envoy\") pod \"f126baf5-b4f4-4593-ae4b-4f9ecab0fb14\" (UID: \"f126baf5-b4f4-4593-ae4b-4f9ecab0fb14\") "
Sep 17 23:59:20.201684 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:59:20.201663    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-6pndd\" (UniqueName: \"kubernetes.io/projected/f126baf5-b4f4-4593-ae4b-4f9ecab0fb14-kube-api-access-6pndd\") pod \"f126baf5-b4f4-4593-ae4b-4f9ecab0fb14\" (UID: \"f126baf5-b4f4-4593-ae4b-4f9ecab0fb14\") "
Sep 17 23:59:20.201948 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:59:20.201706    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/f126baf5-b4f4-4593-ae4b-4f9ecab0fb14-credential-socket\") pod \"f126baf5-b4f4-4593-ae4b-4f9ecab0fb14\" (UID: \"f126baf5-b4f4-4593-ae4b-4f9ecab0fb14\") "
Sep 17 23:59:20.201948 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:59:20.201731    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/f126baf5-b4f4-4593-ae4b-4f9ecab0fb14-workload-certs\") pod \"f126baf5-b4f4-4593-ae4b-4f9ecab0fb14\" (UID: \"f126baf5-b4f4-4593-ae4b-4f9ecab0fb14\") "
Sep 17 23:59:20.201948 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:59:20.201757    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/f126baf5-b4f4-4593-ae4b-4f9ecab0fb14-istio-podinfo\") pod \"f126baf5-b4f4-4593-ae4b-4f9ecab0fb14\" (UID: \"f126baf5-b4f4-4593-ae4b-4f9ecab0fb14\") "
Sep 17 23:59:20.201948 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:59:20.201818    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f126baf5-b4f4-4593-ae4b-4f9ecab0fb14-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "f126baf5-b4f4-4593-ae4b-4f9ecab0fb14" (UID: "f126baf5-b4f4-4593-ae4b-4f9ecab0fb14"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 17 23:59:20.201948 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:59:20.201905    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f126baf5-b4f4-4593-ae4b-4f9ecab0fb14-istio-data" (OuterVolumeSpecName: "istio-data") pod "f126baf5-b4f4-4593-ae4b-4f9ecab0fb14" (UID: "f126baf5-b4f4-4593-ae4b-4f9ecab0fb14"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 17 23:59:20.202235 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:59:20.202011    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f126baf5-b4f4-4593-ae4b-4f9ecab0fb14-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "f126baf5-b4f4-4593-ae4b-4f9ecab0fb14" (UID: "f126baf5-b4f4-4593-ae4b-4f9ecab0fb14"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 17 23:59:20.202235 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:59:20.202009    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/f126baf5-b4f4-4593-ae4b-4f9ecab0fb14-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "f126baf5-b4f4-4593-ae4b-4f9ecab0fb14" (UID: "f126baf5-b4f4-4593-ae4b-4f9ecab0fb14"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 17 23:59:20.202235 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:59:20.202098    2562 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/f126baf5-b4f4-4593-ae4b-4f9ecab0fb14-workload-socket\") on node \"ip-10-0-132-188.ec2.internal\" DevicePath \"\""
Sep 17 23:59:20.202235 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:59:20.202139    2562 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/f126baf5-b4f4-4593-ae4b-4f9ecab0fb14-istio-data\") on node \"ip-10-0-132-188.ec2.internal\" DevicePath \"\""
Sep 17 23:59:20.202235 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:59:20.202155    2562 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/f126baf5-b4f4-4593-ae4b-4f9ecab0fb14-credential-socket\") on node \"ip-10-0-132-188.ec2.internal\" DevicePath \"\""
Sep 17 23:59:20.202235 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:59:20.202186    2562 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/f126baf5-b4f4-4593-ae4b-4f9ecab0fb14-istiod-ca-cert\") on node \"ip-10-0-132-188.ec2.internal\" DevicePath \"\""
Sep 17 23:59:20.202235 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:59:20.202166    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f126baf5-b4f4-4593-ae4b-4f9ecab0fb14-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "f126baf5-b4f4-4593-ae4b-4f9ecab0fb14" (UID: "f126baf5-b4f4-4593-ae4b-4f9ecab0fb14"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 17 23:59:20.203903 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:59:20.203877    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f126baf5-b4f4-4593-ae4b-4f9ecab0fb14-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "f126baf5-b4f4-4593-ae4b-4f9ecab0fb14" (UID: "f126baf5-b4f4-4593-ae4b-4f9ecab0fb14"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 17 23:59:20.204042 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:59:20.204022    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/f126baf5-b4f4-4593-ae4b-4f9ecab0fb14-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "f126baf5-b4f4-4593-ae4b-4f9ecab0fb14" (UID: "f126baf5-b4f4-4593-ae4b-4f9ecab0fb14"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Sep 17 23:59:20.204316 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:59:20.204298    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f126baf5-b4f4-4593-ae4b-4f9ecab0fb14-istio-token" (OuterVolumeSpecName: "istio-token") pod "f126baf5-b4f4-4593-ae4b-4f9ecab0fb14" (UID: "f126baf5-b4f4-4593-ae4b-4f9ecab0fb14"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 17 23:59:20.204386 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:59:20.204315    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f126baf5-b4f4-4593-ae4b-4f9ecab0fb14-kube-api-access-6pndd" (OuterVolumeSpecName: "kube-api-access-6pndd") pod "f126baf5-b4f4-4593-ae4b-4f9ecab0fb14" (UID: "f126baf5-b4f4-4593-ae4b-4f9ecab0fb14"). InnerVolumeSpecName "kube-api-access-6pndd". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 17 23:59:20.303431 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:59:20.303388    2562 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/f126baf5-b4f4-4593-ae4b-4f9ecab0fb14-istio-envoy\") on node \"ip-10-0-132-188.ec2.internal\" DevicePath \"\""
Sep 17 23:59:20.303431 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:59:20.303425    2562 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-6pndd\" (UniqueName: \"kubernetes.io/projected/f126baf5-b4f4-4593-ae4b-4f9ecab0fb14-kube-api-access-6pndd\") on node \"ip-10-0-132-188.ec2.internal\" DevicePath \"\""
Sep 17 23:59:20.303431 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:59:20.303437    2562 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/f126baf5-b4f4-4593-ae4b-4f9ecab0fb14-workload-certs\") on node \"ip-10-0-132-188.ec2.internal\" DevicePath \"\""
Sep 17 23:59:20.303431 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:59:20.303446    2562 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/f126baf5-b4f4-4593-ae4b-4f9ecab0fb14-istio-podinfo\") on node \"ip-10-0-132-188.ec2.internal\" DevicePath \"\""
Sep 17 23:59:20.303724 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:59:20.303454    2562 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/f126baf5-b4f4-4593-ae4b-4f9ecab0fb14-istio-token\") on node \"ip-10-0-132-188.ec2.internal\" DevicePath \"\""
Sep 17 23:59:20.822676 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:59:20.822636    2562 generic.go:356] "Generic (PLEG): container finished" podID="f126baf5-b4f4-4593-ae4b-4f9ecab0fb14" containerID="ff34abc559e1a0d48da85202454b3d713c6d13329764364c860111cbcbd66f36" exitCode=0
Sep 17 23:59:20.822868 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:59:20.822703    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-test-llm-inference-service-028f7809/router-gateway-1-openshift-default-cc7cb45dd-8tcr4" event={"ID":"f126baf5-b4f4-4593-ae4b-4f9ecab0fb14","Type":"ContainerDied","Data":"ff34abc559e1a0d48da85202454b3d713c6d13329764364c860111cbcbd66f36"}
Sep 17 23:59:20.822868 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:59:20.822712    2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-test-llm-inference-service-028f7809/router-gateway-1-openshift-default-cc7cb45dd-8tcr4"
Sep 17 23:59:20.822868 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:59:20.822755    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-test-llm-inference-service-028f7809/router-gateway-1-openshift-default-cc7cb45dd-8tcr4" event={"ID":"f126baf5-b4f4-4593-ae4b-4f9ecab0fb14","Type":"ContainerDied","Data":"e65e6ad6126e37f75da472681d729c683f75ce41ed04d4f034ed0daea9018a57"}
Sep 17 23:59:20.822868 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:59:20.822778    2562 scope.go:117] "RemoveContainer" containerID="ff34abc559e1a0d48da85202454b3d713c6d13329764364c860111cbcbd66f36"
Sep 17 23:59:20.831642 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:59:20.831621    2562 scope.go:117] "RemoveContainer" containerID="e546b8e83c5dab37e2ff0e50db7365293c551fdc57486c9c2da145fbfdae2050"
Sep 17 23:59:20.839773 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:59:20.839753    2562 scope.go:117] "RemoveContainer" containerID="ff34abc559e1a0d48da85202454b3d713c6d13329764364c860111cbcbd66f36"
Sep 17 23:59:20.840061 ip-10-0-132-188 kubenswrapper[2562]: E0917 23:59:20.840033    2562 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"ff34abc559e1a0d48da85202454b3d713c6d13329764364c860111cbcbd66f36\": container with ID starting with ff34abc559e1a0d48da85202454b3d713c6d13329764364c860111cbcbd66f36 not found: ID does not exist" containerID="ff34abc559e1a0d48da85202454b3d713c6d13329764364c860111cbcbd66f36"
Sep 17 23:59:20.840110 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:59:20.840075    2562 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"ff34abc559e1a0d48da85202454b3d713c6d13329764364c860111cbcbd66f36"} err="failed to get container status \"ff34abc559e1a0d48da85202454b3d713c6d13329764364c860111cbcbd66f36\": rpc error: code = NotFound desc = could not find container \"ff34abc559e1a0d48da85202454b3d713c6d13329764364c860111cbcbd66f36\": container with ID starting with ff34abc559e1a0d48da85202454b3d713c6d13329764364c860111cbcbd66f36 not found: ID does not exist"
Sep 17 23:59:20.840110 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:59:20.840094    2562 scope.go:117] "RemoveContainer" containerID="e546b8e83c5dab37e2ff0e50db7365293c551fdc57486c9c2da145fbfdae2050"
Sep 17 23:59:20.840407 ip-10-0-132-188 kubenswrapper[2562]: E0917 23:59:20.840384    2562 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"e546b8e83c5dab37e2ff0e50db7365293c551fdc57486c9c2da145fbfdae2050\": container with ID starting with e546b8e83c5dab37e2ff0e50db7365293c551fdc57486c9c2da145fbfdae2050 not found: ID does not exist" containerID="e546b8e83c5dab37e2ff0e50db7365293c551fdc57486c9c2da145fbfdae2050"
Sep 17 23:59:20.840467 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:59:20.840416    2562 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e546b8e83c5dab37e2ff0e50db7365293c551fdc57486c9c2da145fbfdae2050"} err="failed to get container status \"e546b8e83c5dab37e2ff0e50db7365293c551fdc57486c9c2da145fbfdae2050\": rpc error: code = NotFound desc = could not find container \"e546b8e83c5dab37e2ff0e50db7365293c551fdc57486c9c2da145fbfdae2050\": container with ID starting with e546b8e83c5dab37e2ff0e50db7365293c551fdc57486c9c2da145fbfdae2050 not found: ID does not exist"
Sep 17 23:59:20.843874 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:59:20.843848    2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["e2e-test-llm-inference-service-028f7809/router-gateway-1-openshift-default-cc7cb45dd-8tcr4"]
Sep 17 23:59:20.847945 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:59:20.847916    2562 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["e2e-test-llm-inference-service-028f7809/router-gateway-1-openshift-default-cc7cb45dd-8tcr4"]
Sep 17 23:59:21.666873 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:59:21.666834    2562 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="f126baf5-b4f4-4593-ae4b-4f9ecab0fb14" path="/var/lib/kubelet/pods/f126baf5-b4f4-4593-ae4b-4f9ecab0fb14/volumes"
Sep 17 23:59:36.209992 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:59:36.209935    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fwwc5_7e452bbb-7dc6-4c9d-a9b1-76400942f5b4/konnectivity-agent/0.log" limit=1024
Sep 17 23:59:37.174568 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:59:37.174530    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fwwc5_7e452bbb-7dc6-4c9d-a9b1-76400942f5b4/konnectivity-agent/0.log" limit=1024
Sep 17 23:59:38.596461 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:59:38.596425    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fwwc5_7e452bbb-7dc6-4c9d-a9b1-76400942f5b4/konnectivity-agent/0.log" limit=1024
Sep 17 23:59:38.878972 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:59:38.878876    2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["e2e-test-llm-inference-service-62876e81/llmisvc44d181485fad85e662eb092f3749502f-kserve-router-scheb287j"]
Sep 17 23:59:38.879313 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:59:38.879294    2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f126baf5-b4f4-4593-ae4b-4f9ecab0fb14" containerName="istio-proxy"
Sep 17 23:59:38.879414 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:59:38.879315    2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="f126baf5-b4f4-4593-ae4b-4f9ecab0fb14" containerName="istio-proxy"
Sep 17 23:59:38.879414 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:59:38.879325    2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="611487d3-1111-4db4-b8e6-56578f5c511a" containerName="main"
Sep 17 23:59:38.879414 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:59:38.879333    2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="611487d3-1111-4db4-b8e6-56578f5c511a" containerName="main"
Sep 17 23:59:38.879414 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:59:38.879351    2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f126baf5-b4f4-4593-ae4b-4f9ecab0fb14" containerName="istio-proxy"
Sep 17 23:59:38.879414 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:59:38.879360    2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="f126baf5-b4f4-4593-ae4b-4f9ecab0fb14" containerName="istio-proxy"
Sep 17 23:59:38.879670 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:59:38.879453    2562 memory_manager.go:368] "RemoveStaleState removing state" podUID="f126baf5-b4f4-4593-ae4b-4f9ecab0fb14" containerName="istio-proxy"
Sep 17 23:59:38.879670 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:59:38.879467    2562 memory_manager.go:368] "RemoveStaleState removing state" podUID="f126baf5-b4f4-4593-ae4b-4f9ecab0fb14" containerName="istio-proxy"
Sep 17 23:59:38.879670 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:59:38.879477    2562 memory_manager.go:368] "RemoveStaleState removing state" podUID="611487d3-1111-4db4-b8e6-56578f5c511a" containerName="main"
Sep 17 23:59:38.884213 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:59:38.884186    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-test-llm-inference-service-62876e81/llmisvc44d181485fad85e662eb092f3749502f-kserve-router-scheb287j"
Sep 17 23:59:38.887554 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:59:38.887520    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-test-llm-inference-service-62876e81\"/\"llmisv44d181485fad85e662eb092f3749502f-kserve-self-signed-certs\""
Sep 17 23:59:38.887554 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:59:38.887545    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-test-llm-inference-service-62876e81\"/\"openshift-service-ca.crt\""
Sep 17 23:59:38.887767 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:59:38.887609    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-test-llm-inference-service-62876e81\"/\"default-dockercfg-4wffp\""
Sep 17 23:59:38.887767 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:59:38.887552    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-test-llm-inference-service-62876e81\"/\"llmisvc-model-fb-opt-125m-route-50bc673d-epp-sa-dockercfg-7bzvc\""
Sep 17 23:59:38.887767 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:59:38.887552    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-test-llm-inference-service-62876e81\"/\"kube-root-ca.crt\""
Sep 17 23:59:38.895975 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:59:38.895946    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["e2e-test-llm-inference-service-62876e81/llmisvc44d181485fad85e662eb092f3749502f-kserve-router-scheb287j"]
Sep 17 23:59:38.908949 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:59:38.908907    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fwwc5_7e452bbb-7dc6-4c9d-a9b1-76400942f5b4/konnectivity-agent/0.log" limit=1024
Sep 17 23:59:38.968068 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:59:38.968031    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/fe9ac0c5-3b82-4459-b268-ad4d99996213-tls-certs\") pod \"llmisvc44d181485fad85e662eb092f3749502f-kserve-router-scheb287j\" (UID: \"fe9ac0c5-3b82-4459-b268-ad4d99996213\") " pod="e2e-test-llm-inference-service-62876e81/llmisvc44d181485fad85e662eb092f3749502f-kserve-router-scheb287j"
Sep 17 23:59:38.968324 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:59:38.968152    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-tmlrm\" (UniqueName: \"kubernetes.io/projected/fe9ac0c5-3b82-4459-b268-ad4d99996213-kube-api-access-tmlrm\") pod \"llmisvc44d181485fad85e662eb092f3749502f-kserve-router-scheb287j\" (UID: \"fe9ac0c5-3b82-4459-b268-ad4d99996213\") " pod="e2e-test-llm-inference-service-62876e81/llmisvc44d181485fad85e662eb092f3749502f-kserve-router-scheb287j"
Sep 17 23:59:39.069609 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:59:39.069566    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tmlrm\" (UniqueName: \"kubernetes.io/projected/fe9ac0c5-3b82-4459-b268-ad4d99996213-kube-api-access-tmlrm\") pod \"llmisvc44d181485fad85e662eb092f3749502f-kserve-router-scheb287j\" (UID: \"fe9ac0c5-3b82-4459-b268-ad4d99996213\") " pod="e2e-test-llm-inference-service-62876e81/llmisvc44d181485fad85e662eb092f3749502f-kserve-router-scheb287j"
Sep 17 23:59:39.069799 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:59:39.069646    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/fe9ac0c5-3b82-4459-b268-ad4d99996213-tls-certs\") pod \"llmisvc44d181485fad85e662eb092f3749502f-kserve-router-scheb287j\" (UID: \"fe9ac0c5-3b82-4459-b268-ad4d99996213\") " pod="e2e-test-llm-inference-service-62876e81/llmisvc44d181485fad85e662eb092f3749502f-kserve-router-scheb287j"
Sep 17 23:59:39.072429 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:59:39.072402    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/fe9ac0c5-3b82-4459-b268-ad4d99996213-tls-certs\") pod \"llmisvc44d181485fad85e662eb092f3749502f-kserve-router-scheb287j\" (UID: \"fe9ac0c5-3b82-4459-b268-ad4d99996213\") " pod="e2e-test-llm-inference-service-62876e81/llmisvc44d181485fad85e662eb092f3749502f-kserve-router-scheb287j"
Sep 17 23:59:39.077531 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:59:39.077507    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-tmlrm\" (UniqueName: \"kubernetes.io/projected/fe9ac0c5-3b82-4459-b268-ad4d99996213-kube-api-access-tmlrm\") pod \"llmisvc44d181485fad85e662eb092f3749502f-kserve-router-scheb287j\" (UID: \"fe9ac0c5-3b82-4459-b268-ad4d99996213\") " pod="e2e-test-llm-inference-service-62876e81/llmisvc44d181485fad85e662eb092f3749502f-kserve-router-scheb287j"
Sep 17 23:59:39.195926 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:59:39.195818    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-test-llm-inference-service-62876e81/llmisvc44d181485fad85e662eb092f3749502f-kserve-router-scheb287j"
Sep 17 23:59:39.469948 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:59:39.469920    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["e2e-test-llm-inference-service-62876e81/llmisvc44d181485fad85e662eb092f3749502f-kserve-router-scheb287j"]
Sep 17 23:59:39.472515 ip-10-0-132-188 kubenswrapper[2562]: W0917 23:59:39.472482    2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podfe9ac0c5_3b82_4459_b268_ad4d99996213.slice/crio-3778f8d8a73d101c3e9069eaf0c8311156fe4302cb8c62d7b6b416eed15fd4ef WatchSource:0}: Error finding container 3778f8d8a73d101c3e9069eaf0c8311156fe4302cb8c62d7b6b416eed15fd4ef: Status 404 returned error can't find the container with id 3778f8d8a73d101c3e9069eaf0c8311156fe4302cb8c62d7b6b416eed15fd4ef
Sep 17 23:59:39.914279 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:59:39.914231    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-test-llm-inference-service-62876e81/llmisvc44d181485fad85e662eb092f3749502f-kserve-router-scheb287j" event={"ID":"fe9ac0c5-3b82-4459-b268-ad4d99996213","Type":"ContainerStarted","Data":"759184402c9e698d695658be95775c6ef616839ed8d2951a3b3f6fe9f49d9d77"}
Sep 17 23:59:39.914707 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:59:39.914297    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-test-llm-inference-service-62876e81/llmisvc44d181485fad85e662eb092f3749502f-kserve-router-scheb287j" event={"ID":"fe9ac0c5-3b82-4459-b268-ad4d99996213","Type":"ContainerStarted","Data":"3778f8d8a73d101c3e9069eaf0c8311156fe4302cb8c62d7b6b416eed15fd4ef"}
Sep 17 23:59:39.914707 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:59:39.914326    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-test-llm-inference-service-62876e81/llmisvc44d181485fad85e662eb092f3749502f-kserve-router-scheb287j"
Sep 17 23:59:39.931859 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:59:39.931802    2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="e2e-test-llm-inference-service-62876e81/llmisvc44d181485fad85e662eb092f3749502f-kserve-router-scheb287j" podStartSLOduration=1.9317807230000001 podStartE2EDuration="1.931780723s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-17 23:59:38 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-17 23:59:39.929751788 +0000 UTC m=+2932.331648553" watchObservedRunningTime="2026-09-17 23:59:39.931780723 +0000 UTC m=+2932.333677487"
Sep 17 23:59:42.868604 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:59:42.868571    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fwwc5_7e452bbb-7dc6-4c9d-a9b1-76400942f5b4/konnectivity-agent/0.log" limit=1024
Sep 17 23:59:48.986354 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:59:48.986312    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fwwc5_7e452bbb-7dc6-4c9d-a9b1-76400942f5b4/konnectivity-agent/0.log" limit=1024
Sep 17 23:59:57.351447 ip-10-0-132-188 kubenswrapper[2562]: I0917 23:59:57.351413    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fwwc5_7e452bbb-7dc6-4c9d-a9b1-76400942f5b4/konnectivity-agent/0.log" limit=1024
Sep 18 00:00:00.132340 ip-10-0-132-188 kubenswrapper[2562]: I0918 00:00:00.132300    2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-image-registry/image-pruner-29828160-gvvld"]
Sep 18 00:00:00.136704 ip-10-0-132-188 kubenswrapper[2562]: I0918 00:00:00.136671    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-pruner-29828160-gvvld"
Sep 18 00:00:00.138861 ip-10-0-132-188 kubenswrapper[2562]: I0918 00:00:00.138839    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"serviceca\""
Sep 18 00:00:00.138983 ip-10-0-132-188 kubenswrapper[2562]: I0918 00:00:00.138841    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"pruner-dockercfg-9qwz4\""
Sep 18 00:00:00.142088 ip-10-0-132-188 kubenswrapper[2562]: I0918 00:00:00.142063    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-pruner-29828160-gvvld"]
Sep 18 00:00:00.152880 ip-10-0-132-188 kubenswrapper[2562]: I0918 00:00:00.152832    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xxvdj\" (UniqueName: \"kubernetes.io/projected/7b22e522-10fd-47a8-8c9e-f9f87fee23ca-kube-api-access-xxvdj\") pod \"image-pruner-29828160-gvvld\" (UID: \"7b22e522-10fd-47a8-8c9e-f9f87fee23ca\") " pod="openshift-image-registry/image-pruner-29828160-gvvld"
Sep 18 00:00:00.153341 ip-10-0-132-188 kubenswrapper[2562]: I0918 00:00:00.152975    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/7b22e522-10fd-47a8-8c9e-f9f87fee23ca-serviceca\") pod \"image-pruner-29828160-gvvld\" (UID: \"7b22e522-10fd-47a8-8c9e-f9f87fee23ca\") " pod="openshift-image-registry/image-pruner-29828160-gvvld"
Sep 18 00:00:00.253530 ip-10-0-132-188 kubenswrapper[2562]: I0918 00:00:00.253492    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xxvdj\" (UniqueName: \"kubernetes.io/projected/7b22e522-10fd-47a8-8c9e-f9f87fee23ca-kube-api-access-xxvdj\") pod \"image-pruner-29828160-gvvld\" (UID: \"7b22e522-10fd-47a8-8c9e-f9f87fee23ca\") " pod="openshift-image-registry/image-pruner-29828160-gvvld"
Sep 18 00:00:00.253721 ip-10-0-132-188 kubenswrapper[2562]: I0918 00:00:00.253569    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/7b22e522-10fd-47a8-8c9e-f9f87fee23ca-serviceca\") pod \"image-pruner-29828160-gvvld\" (UID: \"7b22e522-10fd-47a8-8c9e-f9f87fee23ca\") " pod="openshift-image-registry/image-pruner-29828160-gvvld"
Sep 18 00:00:00.254225 ip-10-0-132-188 kubenswrapper[2562]: I0918 00:00:00.254203    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/7b22e522-10fd-47a8-8c9e-f9f87fee23ca-serviceca\") pod \"image-pruner-29828160-gvvld\" (UID: \"7b22e522-10fd-47a8-8c9e-f9f87fee23ca\") " pod="openshift-image-registry/image-pruner-29828160-gvvld"
Sep 18 00:00:00.262434 ip-10-0-132-188 kubenswrapper[2562]: I0918 00:00:00.262401    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-xxvdj\" (UniqueName: \"kubernetes.io/projected/7b22e522-10fd-47a8-8c9e-f9f87fee23ca-kube-api-access-xxvdj\") pod \"image-pruner-29828160-gvvld\" (UID: \"7b22e522-10fd-47a8-8c9e-f9f87fee23ca\") " pod="openshift-image-registry/image-pruner-29828160-gvvld"
Sep 18 00:00:00.464728 ip-10-0-132-188 kubenswrapper[2562]: I0918 00:00:00.464622    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-pruner-29828160-gvvld"
Sep 18 00:00:00.737490 ip-10-0-132-188 kubenswrapper[2562]: I0918 00:00:00.737456    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-pruner-29828160-gvvld"]
Sep 18 00:00:00.740078 ip-10-0-132-188 kubenswrapper[2562]: W0918 00:00:00.740052    2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7b22e522_10fd_47a8_8c9e_f9f87fee23ca.slice/crio-80313a082c7653616bb277982f92e4fe1e98cb567dd52d451066cd38fe429fda WatchSource:0}: Error finding container 80313a082c7653616bb277982f92e4fe1e98cb567dd52d451066cd38fe429fda: Status 404 returned error can't find the container with id 80313a082c7653616bb277982f92e4fe1e98cb567dd52d451066cd38fe429fda
Sep 18 00:00:01.013543 ip-10-0-132-188 kubenswrapper[2562]: I0918 00:00:01.013411    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-pruner-29828160-gvvld" event={"ID":"7b22e522-10fd-47a8-8c9e-f9f87fee23ca","Type":"ContainerStarted","Data":"71cdcec0f1a46db9ddaff4a1db68d1a4f8cce61f359cb93dc8d57647a0e0680d"}
Sep 18 00:00:01.013543 ip-10-0-132-188 kubenswrapper[2562]: I0918 00:00:01.013465    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-pruner-29828160-gvvld" event={"ID":"7b22e522-10fd-47a8-8c9e-f9f87fee23ca","Type":"ContainerStarted","Data":"80313a082c7653616bb277982f92e4fe1e98cb567dd52d451066cd38fe429fda"}
Sep 18 00:00:01.029921 ip-10-0-132-188 kubenswrapper[2562]: I0918 00:00:01.029867    2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/image-pruner-29828160-gvvld" podStartSLOduration=1.02985285 podStartE2EDuration="1.02985285s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-18 00:00:00 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-18 00:00:01.028403355 +0000 UTC m=+2953.430300131" watchObservedRunningTime="2026-09-18 00:00:01.02985285 +0000 UTC m=+2953.431749613"
Sep 18 00:00:02.019217 ip-10-0-132-188 kubenswrapper[2562]: I0918 00:00:02.019184    2562 generic.go:356] "Generic (PLEG): container finished" podID="7b22e522-10fd-47a8-8c9e-f9f87fee23ca" containerID="71cdcec0f1a46db9ddaff4a1db68d1a4f8cce61f359cb93dc8d57647a0e0680d" exitCode=0
Sep 18 00:00:02.019625 ip-10-0-132-188 kubenswrapper[2562]: I0918 00:00:02.019258    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-pruner-29828160-gvvld" event={"ID":"7b22e522-10fd-47a8-8c9e-f9f87fee23ca","Type":"ContainerDied","Data":"71cdcec0f1a46db9ddaff4a1db68d1a4f8cce61f359cb93dc8d57647a0e0680d"}
Sep 18 00:00:03.165932 ip-10-0-132-188 kubenswrapper[2562]: I0918 00:00:03.165903    2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-pruner-29828160-gvvld"
Sep 18 00:00:03.173769 ip-10-0-132-188 kubenswrapper[2562]: I0918 00:00:03.173742    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/7b22e522-10fd-47a8-8c9e-f9f87fee23ca-serviceca\") pod \"7b22e522-10fd-47a8-8c9e-f9f87fee23ca\" (UID: \"7b22e522-10fd-47a8-8c9e-f9f87fee23ca\") "
Sep 18 00:00:03.173880 ip-10-0-132-188 kubenswrapper[2562]: I0918 00:00:03.173797    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-xxvdj\" (UniqueName: \"kubernetes.io/projected/7b22e522-10fd-47a8-8c9e-f9f87fee23ca-kube-api-access-xxvdj\") pod \"7b22e522-10fd-47a8-8c9e-f9f87fee23ca\" (UID: \"7b22e522-10fd-47a8-8c9e-f9f87fee23ca\") "
Sep 18 00:00:03.174118 ip-10-0-132-188 kubenswrapper[2562]: I0918 00:00:03.174098    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/7b22e522-10fd-47a8-8c9e-f9f87fee23ca-serviceca" (OuterVolumeSpecName: "serviceca") pod "7b22e522-10fd-47a8-8c9e-f9f87fee23ca" (UID: "7b22e522-10fd-47a8-8c9e-f9f87fee23ca"). InnerVolumeSpecName "serviceca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 18 00:00:03.176096 ip-10-0-132-188 kubenswrapper[2562]: I0918 00:00:03.176071    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/7b22e522-10fd-47a8-8c9e-f9f87fee23ca-kube-api-access-xxvdj" (OuterVolumeSpecName: "kube-api-access-xxvdj") pod "7b22e522-10fd-47a8-8c9e-f9f87fee23ca" (UID: "7b22e522-10fd-47a8-8c9e-f9f87fee23ca"). InnerVolumeSpecName "kube-api-access-xxvdj". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 18 00:00:03.274528 ip-10-0-132-188 kubenswrapper[2562]: I0918 00:00:03.274436    2562 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-xxvdj\" (UniqueName: \"kubernetes.io/projected/7b22e522-10fd-47a8-8c9e-f9f87fee23ca-kube-api-access-xxvdj\") on node \"ip-10-0-132-188.ec2.internal\" DevicePath \"\""
Sep 18 00:00:03.274528 ip-10-0-132-188 kubenswrapper[2562]: I0918 00:00:03.274471    2562 reconciler_common.go:299] "Volume detached for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/7b22e522-10fd-47a8-8c9e-f9f87fee23ca-serviceca\") on node \"ip-10-0-132-188.ec2.internal\" DevicePath \"\""
Sep 18 00:00:04.030086 ip-10-0-132-188 kubenswrapper[2562]: I0918 00:00:04.029994    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-pruner-29828160-gvvld" event={"ID":"7b22e522-10fd-47a8-8c9e-f9f87fee23ca","Type":"ContainerDied","Data":"80313a082c7653616bb277982f92e4fe1e98cb567dd52d451066cd38fe429fda"}
Sep 18 00:00:04.030086 ip-10-0-132-188 kubenswrapper[2562]: I0918 00:00:04.030042    2562 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="80313a082c7653616bb277982f92e4fe1e98cb567dd52d451066cd38fe429fda"
Sep 18 00:00:04.030086 ip-10-0-132-188 kubenswrapper[2562]: I0918 00:00:04.030011    2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-pruner-29828160-gvvld"
Sep 18 00:00:05.486905 ip-10-0-132-188 kubenswrapper[2562]: I0918 00:00:05.486868    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fwwc5_7e452bbb-7dc6-4c9d-a9b1-76400942f5b4/konnectivity-agent/0.log" limit=1024
Sep 18 00:00:05.756643 ip-10-0-132-188 kubenswrapper[2562]: I0918 00:00:05.756537    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fwwc5_7e452bbb-7dc6-4c9d-a9b1-76400942f5b4/konnectivity-agent/0.log" limit=1024
Sep 18 00:00:10.921098 ip-10-0-132-188 kubenswrapper[2562]: I0918 00:00:10.921067    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-test-llm-inference-service-62876e81/llmisvc44d181485fad85e662eb092f3749502f-kserve-router-scheb287j"
Sep 18 00:00:18.418431 ip-10-0-132-188 kubenswrapper[2562]: I0918 00:00:18.418390    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fwwc5_7e452bbb-7dc6-4c9d-a9b1-76400942f5b4/konnectivity-agent/0.log" limit=1024
Sep 18 00:00:38.513162 ip-10-0-132-188 kubenswrapper[2562]: I0918 00:00:38.513123    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fwwc5_7e452bbb-7dc6-4c9d-a9b1-76400942f5b4/konnectivity-agent/0.log" limit=1024
Sep 18 00:00:38.616250 ip-10-0-132-188 kubenswrapper[2562]: I0918 00:00:38.616203    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fwwc5_7e452bbb-7dc6-4c9d-a9b1-76400942f5b4/konnectivity-agent/0.log" limit=1024
Sep 18 00:00:38.714287 ip-10-0-132-188 kubenswrapper[2562]: I0918 00:00:38.714252    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fwwc5_7e452bbb-7dc6-4c9d-a9b1-76400942f5b4/konnectivity-agent/0.log" limit=1024
Sep 18 00:00:44.371843 ip-10-0-132-188 kubenswrapper[2562]: I0918 00:00:44.371799    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fwwc5_7e452bbb-7dc6-4c9d-a9b1-76400942f5b4/konnectivity-agent/0.log" limit=1024
Sep 18 00:00:44.472256 ip-10-0-132-188 kubenswrapper[2562]: I0918 00:00:44.472214    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fwwc5_7e452bbb-7dc6-4c9d-a9b1-76400942f5b4/konnectivity-agent/0.log" limit=1024
Sep 18 00:00:45.999981 ip-10-0-132-188 kubenswrapper[2562]: I0918 00:00:45.999947    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fwwc5_7e452bbb-7dc6-4c9d-a9b1-76400942f5b4/konnectivity-agent/0.log" limit=1024
Sep 18 00:01:45.423198 ip-10-0-132-188 kubenswrapper[2562]: I0918 00:01:45.423090    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fwwc5_7e452bbb-7dc6-4c9d-a9b1-76400942f5b4/konnectivity-agent/0.log" limit=1024
Sep 18 00:02:33.318721 ip-10-0-132-188 kubenswrapper[2562]: I0918 00:02:33.318657    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fwwc5_7e452bbb-7dc6-4c9d-a9b1-76400942f5b4/konnectivity-agent/0.log" limit=1024
Sep 18 00:02:38.365303 ip-10-0-132-188 kubenswrapper[2562]: I0918 00:02:38.365260    2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["e2e-test-llm-inference-service-62876e81/llmisvc44d181485fad85e662eb092f3749502f-kserve-router-scheb287j"]
Sep 18 00:02:38.371606 ip-10-0-132-188 kubenswrapper[2562]: I0918 00:02:38.371567    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fwwc5_7e452bbb-7dc6-4c9d-a9b1-76400942f5b4/konnectivity-agent/0.log" limit=1024
Sep 18 00:02:38.434840 ip-10-0-132-188 kubenswrapper[2562]: I0918 00:02:38.434801    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fwwc5_7e452bbb-7dc6-4c9d-a9b1-76400942f5b4/konnectivity-agent/0.log" limit=1024
Sep 18 00:02:40.921394 ip-10-0-132-188 kubenswrapper[2562]: I0918 00:02:40.921354    2562 prober.go:120] "Probe failed" probeType="Readiness" pod="e2e-test-llm-inference-service-62876e81/llmisvc44d181485fad85e662eb092f3749502f-kserve-router-scheb287j" podUID="fe9ac0c5-3b82-4459-b268-ad4d99996213" containerName="main" probeResult="failure" output="service unhealthy (responded with \"NOT_SERVING\")"
Sep 18 00:02:45.391746 ip-10-0-132-188 kubenswrapper[2562]: I0918 00:02:45.391718    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fwwc5_7e452bbb-7dc6-4c9d-a9b1-76400942f5b4/konnectivity-agent/0.log" limit=1024
Sep 18 00:02:50.921003 ip-10-0-132-188 kubenswrapper[2562]: I0918 00:02:50.920954    2562 prober.go:120] "Probe failed" probeType="Readiness" pod="e2e-test-llm-inference-service-62876e81/llmisvc44d181485fad85e662eb092f3749502f-kserve-router-scheb287j" podUID="fe9ac0c5-3b82-4459-b268-ad4d99996213" containerName="main" probeResult="failure" output="service unhealthy (responded with \"NOT_SERVING\")"
Sep 18 00:02:53.387094 ip-10-0-132-188 kubenswrapper[2562]: I0918 00:02:53.387049    2562 kuberuntime_container.go:873] "Killing container with a grace period" pod="e2e-test-llm-inference-service-62876e81/llmisvc44d181485fad85e662eb092f3749502f-kserve-router-scheb287j" podUID="fe9ac0c5-3b82-4459-b268-ad4d99996213" containerName="main" containerID="cri-o://759184402c9e698d695658be95775c6ef616839ed8d2951a3b3f6fe9f49d9d77" gracePeriod=45
Sep 18 00:02:53.487234 ip-10-0-132-188 kubenswrapper[2562]: I0918 00:02:53.487195    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fwwc5_7e452bbb-7dc6-4c9d-a9b1-76400942f5b4/konnectivity-agent/0.log" limit=1024
Sep 18 00:02:53.581934 ip-10-0-132-188 kubenswrapper[2562]: I0918 00:02:53.581884    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fwwc5_7e452bbb-7dc6-4c9d-a9b1-76400942f5b4/konnectivity-agent/0.log" limit=1024
Sep 18 00:02:53.682106 ip-10-0-132-188 kubenswrapper[2562]: I0918 00:02:53.681996    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fwwc5_7e452bbb-7dc6-4c9d-a9b1-76400942f5b4/konnectivity-agent/0.log" limit=1024
Sep 18 00:02:53.782251 ip-10-0-132-188 kubenswrapper[2562]: I0918 00:02:53.782208    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fwwc5_7e452bbb-7dc6-4c9d-a9b1-76400942f5b4/konnectivity-agent/0.log" limit=1024
Sep 18 00:02:56.076107 ip-10-0-132-188 kubenswrapper[2562]: I0918 00:02:56.076066    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fwwc5_7e452bbb-7dc6-4c9d-a9b1-76400942f5b4/konnectivity-agent/0.log" limit=1024
Sep 18 00:02:56.143662 ip-10-0-132-188 kubenswrapper[2562]: I0918 00:02:56.143630    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fwwc5_7e452bbb-7dc6-4c9d-a9b1-76400942f5b4/konnectivity-agent/0.log" limit=1024
Sep 18 00:02:56.792991 ip-10-0-132-188 kubenswrapper[2562]: I0918 00:02:56.792960    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fwwc5_7e452bbb-7dc6-4c9d-a9b1-76400942f5b4/konnectivity-agent/0.log" limit=1024
Sep 18 00:02:59.125442 ip-10-0-132-188 kubenswrapper[2562]: I0918 00:02:59.125408    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fwwc5_7e452bbb-7dc6-4c9d-a9b1-76400942f5b4/konnectivity-agent/0.log" limit=1024
Sep 18 00:02:59.233291 ip-10-0-132-188 kubenswrapper[2562]: I0918 00:02:59.233254    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fwwc5_7e452bbb-7dc6-4c9d-a9b1-76400942f5b4/konnectivity-agent/0.log" limit=1024
Sep 18 00:03:00.920816 ip-10-0-132-188 kubenswrapper[2562]: I0918 00:03:00.920774    2562 prober.go:120] "Probe failed" probeType="Readiness" pod="e2e-test-llm-inference-service-62876e81/llmisvc44d181485fad85e662eb092f3749502f-kserve-router-scheb287j" podUID="fe9ac0c5-3b82-4459-b268-ad4d99996213" containerName="main" probeResult="failure" output="service unhealthy (responded with \"NOT_SERVING\")"
Sep 18 00:03:00.921268 ip-10-0-132-188 kubenswrapper[2562]: I0918 00:03:00.920878    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-test-llm-inference-service-62876e81/llmisvc44d181485fad85e662eb092f3749502f-kserve-router-scheb287j"
Sep 18 00:03:10.920443 ip-10-0-132-188 kubenswrapper[2562]: I0918 00:03:10.920405    2562 prober.go:120] "Probe failed" probeType="Readiness" pod="e2e-test-llm-inference-service-62876e81/llmisvc44d181485fad85e662eb092f3749502f-kserve-router-scheb287j" podUID="fe9ac0c5-3b82-4459-b268-ad4d99996213" containerName="main" probeResult="failure" output="service unhealthy (responded with \"NOT_SERVING\")"
Sep 18 00:03:20.921263 ip-10-0-132-188 kubenswrapper[2562]: I0918 00:03:20.921220    2562 prober.go:120] "Probe failed" probeType="Readiness" pod="e2e-test-llm-inference-service-62876e81/llmisvc44d181485fad85e662eb092f3749502f-kserve-router-scheb287j" podUID="fe9ac0c5-3b82-4459-b268-ad4d99996213" containerName="main" probeResult="failure" output="service unhealthy (responded with \"NOT_SERVING\")"
Sep 18 00:03:23.642612 ip-10-0-132-188 kubenswrapper[2562]: I0918 00:03:23.642578    2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-test-llm-inference-service-62876e81/llmisvc44d181485fad85e662eb092f3749502f-kserve-router-scheb287j"
Sep 18 00:03:23.747719 ip-10-0-132-188 kubenswrapper[2562]: I0918 00:03:23.747618    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-tmlrm\" (UniqueName: \"kubernetes.io/projected/fe9ac0c5-3b82-4459-b268-ad4d99996213-kube-api-access-tmlrm\") pod \"fe9ac0c5-3b82-4459-b268-ad4d99996213\" (UID: \"fe9ac0c5-3b82-4459-b268-ad4d99996213\") "
Sep 18 00:03:23.747719 ip-10-0-132-188 kubenswrapper[2562]: I0918 00:03:23.747671    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/fe9ac0c5-3b82-4459-b268-ad4d99996213-tls-certs\") pod \"fe9ac0c5-3b82-4459-b268-ad4d99996213\" (UID: \"fe9ac0c5-3b82-4459-b268-ad4d99996213\") "
Sep 18 00:03:23.750000 ip-10-0-132-188 kubenswrapper[2562]: I0918 00:03:23.749963    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/fe9ac0c5-3b82-4459-b268-ad4d99996213-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "fe9ac0c5-3b82-4459-b268-ad4d99996213" (UID: "fe9ac0c5-3b82-4459-b268-ad4d99996213"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 18 00:03:23.750130 ip-10-0-132-188 kubenswrapper[2562]: I0918 00:03:23.750025    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/fe9ac0c5-3b82-4459-b268-ad4d99996213-kube-api-access-tmlrm" (OuterVolumeSpecName: "kube-api-access-tmlrm") pod "fe9ac0c5-3b82-4459-b268-ad4d99996213" (UID: "fe9ac0c5-3b82-4459-b268-ad4d99996213"). InnerVolumeSpecName "kube-api-access-tmlrm". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 18 00:03:23.848625 ip-10-0-132-188 kubenswrapper[2562]: I0918 00:03:23.848582    2562 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-tmlrm\" (UniqueName: \"kubernetes.io/projected/fe9ac0c5-3b82-4459-b268-ad4d99996213-kube-api-access-tmlrm\") on node \"ip-10-0-132-188.ec2.internal\" DevicePath \"\""
Sep 18 00:03:23.848625 ip-10-0-132-188 kubenswrapper[2562]: I0918 00:03:23.848618    2562 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/fe9ac0c5-3b82-4459-b268-ad4d99996213-tls-certs\") on node \"ip-10-0-132-188.ec2.internal\" DevicePath \"\""
Sep 18 00:03:23.952516 ip-10-0-132-188 kubenswrapper[2562]: I0918 00:03:23.952479    2562 generic.go:356] "Generic (PLEG): container finished" podID="fe9ac0c5-3b82-4459-b268-ad4d99996213" containerID="759184402c9e698d695658be95775c6ef616839ed8d2951a3b3f6fe9f49d9d77" exitCode=0
Sep 18 00:03:23.952688 ip-10-0-132-188 kubenswrapper[2562]: I0918 00:03:23.952526    2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-test-llm-inference-service-62876e81/llmisvc44d181485fad85e662eb092f3749502f-kserve-router-scheb287j"
Sep 18 00:03:23.952688 ip-10-0-132-188 kubenswrapper[2562]: I0918 00:03:23.952567    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-test-llm-inference-service-62876e81/llmisvc44d181485fad85e662eb092f3749502f-kserve-router-scheb287j" event={"ID":"fe9ac0c5-3b82-4459-b268-ad4d99996213","Type":"ContainerDied","Data":"759184402c9e698d695658be95775c6ef616839ed8d2951a3b3f6fe9f49d9d77"}
Sep 18 00:03:23.952688 ip-10-0-132-188 kubenswrapper[2562]: I0918 00:03:23.952613    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-test-llm-inference-service-62876e81/llmisvc44d181485fad85e662eb092f3749502f-kserve-router-scheb287j" event={"ID":"fe9ac0c5-3b82-4459-b268-ad4d99996213","Type":"ContainerDied","Data":"3778f8d8a73d101c3e9069eaf0c8311156fe4302cb8c62d7b6b416eed15fd4ef"}
Sep 18 00:03:23.952688 ip-10-0-132-188 kubenswrapper[2562]: I0918 00:03:23.952636    2562 scope.go:117] "RemoveContainer" containerID="759184402c9e698d695658be95775c6ef616839ed8d2951a3b3f6fe9f49d9d77"
Sep 18 00:03:23.961404 ip-10-0-132-188 kubenswrapper[2562]: I0918 00:03:23.961379    2562 scope.go:117] "RemoveContainer" containerID="759184402c9e698d695658be95775c6ef616839ed8d2951a3b3f6fe9f49d9d77"
Sep 18 00:03:23.961674 ip-10-0-132-188 kubenswrapper[2562]: E0918 00:03:23.961650    2562 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"759184402c9e698d695658be95775c6ef616839ed8d2951a3b3f6fe9f49d9d77\": container with ID starting with 759184402c9e698d695658be95775c6ef616839ed8d2951a3b3f6fe9f49d9d77 not found: ID does not exist" containerID="759184402c9e698d695658be95775c6ef616839ed8d2951a3b3f6fe9f49d9d77"
Sep 18 00:03:23.961751 ip-10-0-132-188 kubenswrapper[2562]: I0918 00:03:23.961681    2562 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"759184402c9e698d695658be95775c6ef616839ed8d2951a3b3f6fe9f49d9d77"} err="failed to get container status \"759184402c9e698d695658be95775c6ef616839ed8d2951a3b3f6fe9f49d9d77\": rpc error: code = NotFound desc = could not find container \"759184402c9e698d695658be95775c6ef616839ed8d2951a3b3f6fe9f49d9d77\": container with ID starting with 759184402c9e698d695658be95775c6ef616839ed8d2951a3b3f6fe9f49d9d77 not found: ID does not exist"
Sep 18 00:03:23.974355 ip-10-0-132-188 kubenswrapper[2562]: I0918 00:03:23.974320    2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["e2e-test-llm-inference-service-62876e81/llmisvc44d181485fad85e662eb092f3749502f-kserve-router-scheb287j"]
Sep 18 00:03:23.977873 ip-10-0-132-188 kubenswrapper[2562]: I0918 00:03:23.977845    2562 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["e2e-test-llm-inference-service-62876e81/llmisvc44d181485fad85e662eb092f3749502f-kserve-router-scheb287j"]
Sep 18 00:03:25.666872 ip-10-0-132-188 kubenswrapper[2562]: I0918 00:03:25.666840    2562 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="fe9ac0c5-3b82-4459-b268-ad4d99996213" path="/var/lib/kubelet/pods/fe9ac0c5-3b82-4459-b268-ad4d99996213/volumes"
Sep 18 00:03:26.535702 ip-10-0-132-188 kubenswrapper[2562]: I0918 00:03:26.535658    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fwwc5_7e452bbb-7dc6-4c9d-a9b1-76400942f5b4/konnectivity-agent/0.log" limit=1024
Sep 18 00:03:28.564782 ip-10-0-132-188 kubenswrapper[2562]: I0918 00:03:28.564733    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fwwc5_7e452bbb-7dc6-4c9d-a9b1-76400942f5b4/konnectivity-agent/0.log" limit=1024
Sep 18 00:03:28.682771 ip-10-0-132-188 kubenswrapper[2562]: I0918 00:03:28.682722    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fwwc5_7e452bbb-7dc6-4c9d-a9b1-76400942f5b4/konnectivity-agent/0.log" limit=1024
Sep 18 00:03:34.362238 ip-10-0-132-188 kubenswrapper[2562]: I0918 00:03:34.362164    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fwwc5_7e452bbb-7dc6-4c9d-a9b1-76400942f5b4/konnectivity-agent/0.log" limit=1024
Sep 18 00:03:39.754448 ip-10-0-132-188 kubenswrapper[2562]: I0918 00:03:39.754404    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fwwc5_7e452bbb-7dc6-4c9d-a9b1-76400942f5b4/konnectivity-agent/0.log" limit=1024
Sep 18 00:03:39.855651 ip-10-0-132-188 kubenswrapper[2562]: I0918 00:03:39.855613    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fwwc5_7e452bbb-7dc6-4c9d-a9b1-76400942f5b4/konnectivity-agent/0.log" limit=1024
Sep 18 00:03:50.908874 ip-10-0-132-188 kubenswrapper[2562]: I0918 00:03:50.908834    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fwwc5_7e452bbb-7dc6-4c9d-a9b1-76400942f5b4/konnectivity-agent/0.log" limit=1024
Sep 18 00:03:51.068213 ip-10-0-132-188 kubenswrapper[2562]: I0918 00:03:51.068155    2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["e2e-test-llm-inference-service-d73d44f4/custom-route-timeout-pd-test-kserve-router-scheduler-84c48mvfch"]
Sep 18 00:03:51.068739 ip-10-0-132-188 kubenswrapper[2562]: I0918 00:03:51.068712    2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="fe9ac0c5-3b82-4459-b268-ad4d99996213" containerName="main"
Sep 18 00:03:51.068952 ip-10-0-132-188 kubenswrapper[2562]: I0918 00:03:51.068861    2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="fe9ac0c5-3b82-4459-b268-ad4d99996213" containerName="main"
Sep 18 00:03:51.068952 ip-10-0-132-188 kubenswrapper[2562]: I0918 00:03:51.068923    2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="7b22e522-10fd-47a8-8c9e-f9f87fee23ca" containerName="image-pruner"
Sep 18 00:03:51.068952 ip-10-0-132-188 kubenswrapper[2562]: I0918 00:03:51.068935    2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="7b22e522-10fd-47a8-8c9e-f9f87fee23ca" containerName="image-pruner"
Sep 18 00:03:51.069230 ip-10-0-132-188 kubenswrapper[2562]: I0918 00:03:51.069058    2562 memory_manager.go:368] "RemoveStaleState removing state" podUID="fe9ac0c5-3b82-4459-b268-ad4d99996213" containerName="main"
Sep 18 00:03:51.069230 ip-10-0-132-188 kubenswrapper[2562]: I0918 00:03:51.069075    2562 memory_manager.go:368] "RemoveStaleState removing state" podUID="7b22e522-10fd-47a8-8c9e-f9f87fee23ca" containerName="image-pruner"
Sep 18 00:03:51.072021 ip-10-0-132-188 kubenswrapper[2562]: I0918 00:03:51.071998    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-test-llm-inference-service-d73d44f4/custom-route-timeout-pd-test-kserve-router-scheduler-84c48mvfch"
Sep 18 00:03:51.075522 ip-10-0-132-188 kubenswrapper[2562]: I0918 00:03:51.075487    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-test-llm-inference-service-d73d44f4\"/\"custom-route-timeout-pd-test-epp-sa-dockercfg-6qn9x\""
Sep 18 00:03:51.075662 ip-10-0-132-188 kubenswrapper[2562]: I0918 00:03:51.075489    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-test-llm-inference-service-d73d44f4\"/\"openshift-service-ca.crt\""
Sep 18 00:03:51.075662 ip-10-0-132-188 kubenswrapper[2562]: I0918 00:03:51.075592    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-test-llm-inference-service-d73d44f4\"/\"kube-root-ca.crt\""
Sep 18 00:03:51.075662 ip-10-0-132-188 kubenswrapper[2562]: I0918 00:03:51.075602    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-test-llm-inference-service-d73d44f4\"/\"custom-route-timeout-pd-test-kserve-self-signed-certs\""
Sep 18 00:03:51.075662 ip-10-0-132-188 kubenswrapper[2562]: I0918 00:03:51.075502    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-test-llm-inference-service-d73d44f4\"/\"default-dockercfg-nxglk\""
Sep 18 00:03:51.079972 ip-10-0-132-188 kubenswrapper[2562]: I0918 00:03:51.079944    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["e2e-test-llm-inference-service-d73d44f4/custom-route-timeout-pd-test-kserve-router-scheduler-84c48mvfch"]
Sep 18 00:03:51.096501 ip-10-0-132-188 kubenswrapper[2562]: I0918 00:03:51.096472    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fwwc5_7e452bbb-7dc6-4c9d-a9b1-76400942f5b4/konnectivity-agent/0.log" limit=1024
Sep 18 00:03:51.189476 ip-10-0-132-188 kubenswrapper[2562]: I0918 00:03:51.189389    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/d4103fa7-df05-46ec-ad32-93f420000988-tls-certs\") pod \"custom-route-timeout-pd-test-kserve-router-scheduler-84c48mvfch\" (UID: \"d4103fa7-df05-46ec-ad32-93f420000988\") " pod="e2e-test-llm-inference-service-d73d44f4/custom-route-timeout-pd-test-kserve-router-scheduler-84c48mvfch"
Sep 18 00:03:51.189476 ip-10-0-132-188 kubenswrapper[2562]: I0918 00:03:51.189462    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-b7lsv\" (UniqueName: \"kubernetes.io/projected/d4103fa7-df05-46ec-ad32-93f420000988-kube-api-access-b7lsv\") pod \"custom-route-timeout-pd-test-kserve-router-scheduler-84c48mvfch\" (UID: \"d4103fa7-df05-46ec-ad32-93f420000988\") " pod="e2e-test-llm-inference-service-d73d44f4/custom-route-timeout-pd-test-kserve-router-scheduler-84c48mvfch"
Sep 18 00:03:51.290558 ip-10-0-132-188 kubenswrapper[2562]: I0918 00:03:51.290517    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/d4103fa7-df05-46ec-ad32-93f420000988-tls-certs\") pod \"custom-route-timeout-pd-test-kserve-router-scheduler-84c48mvfch\" (UID: \"d4103fa7-df05-46ec-ad32-93f420000988\") " pod="e2e-test-llm-inference-service-d73d44f4/custom-route-timeout-pd-test-kserve-router-scheduler-84c48mvfch"
Sep 18 00:03:51.290746 ip-10-0-132-188 kubenswrapper[2562]: I0918 00:03:51.290581    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-b7lsv\" (UniqueName: \"kubernetes.io/projected/d4103fa7-df05-46ec-ad32-93f420000988-kube-api-access-b7lsv\") pod \"custom-route-timeout-pd-test-kserve-router-scheduler-84c48mvfch\" (UID: \"d4103fa7-df05-46ec-ad32-93f420000988\") " pod="e2e-test-llm-inference-service-d73d44f4/custom-route-timeout-pd-test-kserve-router-scheduler-84c48mvfch"
Sep 18 00:03:51.293101 ip-10-0-132-188 kubenswrapper[2562]: I0918 00:03:51.293070    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/d4103fa7-df05-46ec-ad32-93f420000988-tls-certs\") pod \"custom-route-timeout-pd-test-kserve-router-scheduler-84c48mvfch\" (UID: \"d4103fa7-df05-46ec-ad32-93f420000988\") " pod="e2e-test-llm-inference-service-d73d44f4/custom-route-timeout-pd-test-kserve-router-scheduler-84c48mvfch"
Sep 18 00:03:51.298897 ip-10-0-132-188 kubenswrapper[2562]: I0918 00:03:51.298857    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-b7lsv\" (UniqueName: \"kubernetes.io/projected/d4103fa7-df05-46ec-ad32-93f420000988-kube-api-access-b7lsv\") pod \"custom-route-timeout-pd-test-kserve-router-scheduler-84c48mvfch\" (UID: \"d4103fa7-df05-46ec-ad32-93f420000988\") " pod="e2e-test-llm-inference-service-d73d44f4/custom-route-timeout-pd-test-kserve-router-scheduler-84c48mvfch"
Sep 18 00:03:51.384744 ip-10-0-132-188 kubenswrapper[2562]: I0918 00:03:51.384692    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-test-llm-inference-service-d73d44f4/custom-route-timeout-pd-test-kserve-router-scheduler-84c48mvfch"
Sep 18 00:03:51.659473 ip-10-0-132-188 kubenswrapper[2562]: I0918 00:03:51.659432    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["e2e-test-llm-inference-service-d73d44f4/custom-route-timeout-pd-test-kserve-router-scheduler-84c48mvfch"]
Sep 18 00:03:51.662667 ip-10-0-132-188 kubenswrapper[2562]: W0918 00:03:51.662630    2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd4103fa7_df05_46ec_ad32_93f420000988.slice/crio-e84dc2b1342c82bebda382e2b17f7bcdb90c1bd89788d99cdfd9c07f44fc996c WatchSource:0}: Error finding container e84dc2b1342c82bebda382e2b17f7bcdb90c1bd89788d99cdfd9c07f44fc996c: Status 404 returned error can't find the container with id e84dc2b1342c82bebda382e2b17f7bcdb90c1bd89788d99cdfd9c07f44fc996c
Sep 18 00:03:52.084259 ip-10-0-132-188 kubenswrapper[2562]: I0918 00:03:52.084209    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-test-llm-inference-service-d73d44f4/custom-route-timeout-pd-test-kserve-router-scheduler-84c48mvfch" event={"ID":"d4103fa7-df05-46ec-ad32-93f420000988","Type":"ContainerStarted","Data":"d9f97b83d20a62bf5d3735455d3342425862c38754dfc2690fdbabd08a4ed11d"}
Sep 18 00:03:52.084685 ip-10-0-132-188 kubenswrapper[2562]: I0918 00:03:52.084275    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-test-llm-inference-service-d73d44f4/custom-route-timeout-pd-test-kserve-router-scheduler-84c48mvfch" event={"ID":"d4103fa7-df05-46ec-ad32-93f420000988","Type":"ContainerStarted","Data":"e84dc2b1342c82bebda382e2b17f7bcdb90c1bd89788d99cdfd9c07f44fc996c"}
Sep 18 00:03:52.084685 ip-10-0-132-188 kubenswrapper[2562]: I0918 00:03:52.084306    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-test-llm-inference-service-d73d44f4/custom-route-timeout-pd-test-kserve-router-scheduler-84c48mvfch"
Sep 18 00:03:52.101145 ip-10-0-132-188 kubenswrapper[2562]: I0918 00:03:52.101090    2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="e2e-test-llm-inference-service-d73d44f4/custom-route-timeout-pd-test-kserve-router-scheduler-84c48mvfch" podStartSLOduration=1.101072848 podStartE2EDuration="1.101072848s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-18 00:03:51 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-18 00:03:52.099970855 +0000 UTC m=+3184.501867619" watchObservedRunningTime="2026-09-18 00:03:52.101072848 +0000 UTC m=+3184.502969611"
Sep 18 00:04:22.760603 ip-10-0-132-188 kubenswrapper[2562]: I0918 00:04:22.760558    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fwwc5_7e452bbb-7dc6-4c9d-a9b1-76400942f5b4/konnectivity-agent/0.log" limit=1024
Sep 18 00:04:22.861411 ip-10-0-132-188 kubenswrapper[2562]: I0918 00:04:22.861368    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fwwc5_7e452bbb-7dc6-4c9d-a9b1-76400942f5b4/konnectivity-agent/0.log" limit=1024
Sep 18 00:04:23.091868 ip-10-0-132-188 kubenswrapper[2562]: I0918 00:04:23.091837    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-test-llm-inference-service-d73d44f4/custom-route-timeout-pd-test-kserve-router-scheduler-84c48mvfch"
Sep 18 00:04:40.066238 ip-10-0-132-188 kubenswrapper[2562]: I0918 00:04:40.066118    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fwwc5_7e452bbb-7dc6-4c9d-a9b1-76400942f5b4/konnectivity-agent/0.log" limit=1024
Sep 18 00:05:27.302452 ip-10-0-132-188 kubenswrapper[2562]: I0918 00:05:27.302416    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fwwc5_7e452bbb-7dc6-4c9d-a9b1-76400942f5b4/konnectivity-agent/0.log" limit=1024
Sep 18 00:05:29.304221 ip-10-0-132-188 kubenswrapper[2562]: I0918 00:05:29.304144    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fwwc5_7e452bbb-7dc6-4c9d-a9b1-76400942f5b4/konnectivity-agent/0.log" limit=1024
Sep 18 00:05:49.216068 ip-10-0-132-188 kubenswrapper[2562]: I0918 00:05:49.216010    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fwwc5_7e452bbb-7dc6-4c9d-a9b1-76400942f5b4/konnectivity-agent/0.log" limit=1024
Sep 18 00:05:49.319714 ip-10-0-132-188 kubenswrapper[2562]: I0918 00:05:49.319667    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fwwc5_7e452bbb-7dc6-4c9d-a9b1-76400942f5b4/konnectivity-agent/0.log" limit=1024
Sep 18 00:05:49.420264 ip-10-0-132-188 kubenswrapper[2562]: I0918 00:05:49.420227    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fwwc5_7e452bbb-7dc6-4c9d-a9b1-76400942f5b4/konnectivity-agent/0.log" limit=1024
Sep 18 00:05:50.010557 ip-10-0-132-188 kubenswrapper[2562]: I0918 00:05:50.010512    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fwwc5_7e452bbb-7dc6-4c9d-a9b1-76400942f5b4/konnectivity-agent/0.log" limit=1024
Sep 18 00:05:50.068821 ip-10-0-132-188 kubenswrapper[2562]: I0918 00:05:50.068783    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fwwc5_7e452bbb-7dc6-4c9d-a9b1-76400942f5b4/konnectivity-agent/0.log" limit=1024
Sep 18 00:05:51.757781 ip-10-0-132-188 kubenswrapper[2562]: I0918 00:05:51.757743    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fwwc5_7e452bbb-7dc6-4c9d-a9b1-76400942f5b4/konnectivity-agent/0.log" limit=1024
Sep 18 00:05:51.807689 ip-10-0-132-188 kubenswrapper[2562]: I0918 00:05:51.807653    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fwwc5_7e452bbb-7dc6-4c9d-a9b1-76400942f5b4/konnectivity-agent/0.log" limit=1024
Sep 18 00:05:52.698609 ip-10-0-132-188 kubenswrapper[2562]: I0918 00:05:52.698564    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fwwc5_7e452bbb-7dc6-4c9d-a9b1-76400942f5b4/konnectivity-agent/0.log" limit=1024
Sep 18 00:05:55.060342 ip-10-0-132-188 kubenswrapper[2562]: I0918 00:05:55.060294    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fwwc5_7e452bbb-7dc6-4c9d-a9b1-76400942f5b4/konnectivity-agent/0.log" limit=1024
Sep 18 00:05:55.160703 ip-10-0-132-188 kubenswrapper[2562]: I0918 00:05:55.160660    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fwwc5_7e452bbb-7dc6-4c9d-a9b1-76400942f5b4/konnectivity-agent/0.log" limit=1024
Sep 18 00:05:55.261658 ip-10-0-132-188 kubenswrapper[2562]: I0918 00:05:55.261625    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fwwc5_7e452bbb-7dc6-4c9d-a9b1-76400942f5b4/konnectivity-agent/0.log" limit=1024
Sep 18 00:05:55.360747 ip-10-0-132-188 kubenswrapper[2562]: I0918 00:05:55.360708    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fwwc5_7e452bbb-7dc6-4c9d-a9b1-76400942f5b4/konnectivity-agent/0.log" limit=1024
Sep 18 00:06:00.450868 ip-10-0-132-188 kubenswrapper[2562]: I0918 00:06:00.450826    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fwwc5_7e452bbb-7dc6-4c9d-a9b1-76400942f5b4/konnectivity-agent/0.log" limit=1024
Sep 18 00:06:00.510648 ip-10-0-132-188 kubenswrapper[2562]: I0918 00:06:00.510614    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fwwc5_7e452bbb-7dc6-4c9d-a9b1-76400942f5b4/konnectivity-agent/0.log" limit=1024
Sep 18 00:06:18.305593 ip-10-0-132-188 kubenswrapper[2562]: I0918 00:06:18.305552    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fwwc5_7e452bbb-7dc6-4c9d-a9b1-76400942f5b4/konnectivity-agent/0.log" limit=1024
Sep 18 00:06:21.281959 ip-10-0-132-188 kubenswrapper[2562]: I0918 00:06:21.281917    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fwwc5_7e452bbb-7dc6-4c9d-a9b1-76400942f5b4/konnectivity-agent/0.log" limit=1024
Sep 18 00:06:34.651063 ip-10-0-132-188 kubenswrapper[2562]: I0918 00:06:34.650967    2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["e2e-test-llm-inference-service-d73d44f4/custom-route-timeout-pd-test-kserve-router-scheduler-84c48mvfch"]
Sep 18 00:06:34.653268 ip-10-0-132-188 kubenswrapper[2562]: I0918 00:06:34.653220    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fwwc5_7e452bbb-7dc6-4c9d-a9b1-76400942f5b4/konnectivity-agent/0.log" limit=1024
Sep 18 00:06:34.715468 ip-10-0-132-188 kubenswrapper[2562]: I0918 00:06:34.715425    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fwwc5_7e452bbb-7dc6-4c9d-a9b1-76400942f5b4/konnectivity-agent/0.log" limit=1024
Sep 18 00:06:43.092320 ip-10-0-132-188 kubenswrapper[2562]: I0918 00:06:43.092276    2562 prober.go:120] "Probe failed" probeType="Readiness" pod="e2e-test-llm-inference-service-d73d44f4/custom-route-timeout-pd-test-kserve-router-scheduler-84c48mvfch" podUID="d4103fa7-df05-46ec-ad32-93f420000988" containerName="main" probeResult="failure" output="service unhealthy (responded with \"NOT_SERVING\")"
Sep 18 00:06:49.673040 ip-10-0-132-188 kubenswrapper[2562]: I0918 00:06:49.672980    2562 kuberuntime_container.go:873] "Killing container with a grace period" pod="e2e-test-llm-inference-service-d73d44f4/custom-route-timeout-pd-test-kserve-router-scheduler-84c48mvfch" podUID="d4103fa7-df05-46ec-ad32-93f420000988" containerName="main" containerID="cri-o://d9f97b83d20a62bf5d3735455d3342425862c38754dfc2690fdbabd08a4ed11d" gracePeriod=45
Sep 18 00:06:53.091430 ip-10-0-132-188 kubenswrapper[2562]: I0918 00:06:53.091382    2562 prober.go:120] "Probe failed" probeType="Readiness" pod="e2e-test-llm-inference-service-d73d44f4/custom-route-timeout-pd-test-kserve-router-scheduler-84c48mvfch" podUID="d4103fa7-df05-46ec-ad32-93f420000988" containerName="main" probeResult="failure" output="service unhealthy (responded with \"NOT_SERVING\")"
Sep 18 00:07:03.091972 ip-10-0-132-188 kubenswrapper[2562]: I0918 00:07:03.091930    2562 prober.go:120] "Probe failed" probeType="Readiness" pod="e2e-test-llm-inference-service-d73d44f4/custom-route-timeout-pd-test-kserve-router-scheduler-84c48mvfch" podUID="d4103fa7-df05-46ec-ad32-93f420000988" containerName="main" probeResult="failure" output="service unhealthy (responded with \"NOT_SERVING\")"
Sep 18 00:07:03.092402 ip-10-0-132-188 kubenswrapper[2562]: I0918 00:07:03.092056    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-test-llm-inference-service-d73d44f4/custom-route-timeout-pd-test-kserve-router-scheduler-84c48mvfch"
Sep 18 00:07:13.091529 ip-10-0-132-188 kubenswrapper[2562]: I0918 00:07:13.091484    2562 prober.go:120] "Probe failed" probeType="Readiness" pod="e2e-test-llm-inference-service-d73d44f4/custom-route-timeout-pd-test-kserve-router-scheduler-84c48mvfch" podUID="d4103fa7-df05-46ec-ad32-93f420000988" containerName="main" probeResult="failure" output="service unhealthy (responded with \"NOT_SERVING\")"
Sep 18 00:07:19.922349 ip-10-0-132-188 kubenswrapper[2562]: I0918 00:07:19.922324    2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-test-llm-inference-service-d73d44f4/custom-route-timeout-pd-test-kserve-router-scheduler-84c48mvfch"
Sep 18 00:07:20.030879 ip-10-0-132-188 kubenswrapper[2562]: I0918 00:07:20.030771    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/d4103fa7-df05-46ec-ad32-93f420000988-tls-certs\") pod \"d4103fa7-df05-46ec-ad32-93f420000988\" (UID: \"d4103fa7-df05-46ec-ad32-93f420000988\") "
Sep 18 00:07:20.030879 ip-10-0-132-188 kubenswrapper[2562]: I0918 00:07:20.030838    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-b7lsv\" (UniqueName: \"kubernetes.io/projected/d4103fa7-df05-46ec-ad32-93f420000988-kube-api-access-b7lsv\") pod \"d4103fa7-df05-46ec-ad32-93f420000988\" (UID: \"d4103fa7-df05-46ec-ad32-93f420000988\") "
Sep 18 00:07:20.033208 ip-10-0-132-188 kubenswrapper[2562]: I0918 00:07:20.033161    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d4103fa7-df05-46ec-ad32-93f420000988-kube-api-access-b7lsv" (OuterVolumeSpecName: "kube-api-access-b7lsv") pod "d4103fa7-df05-46ec-ad32-93f420000988" (UID: "d4103fa7-df05-46ec-ad32-93f420000988"). InnerVolumeSpecName "kube-api-access-b7lsv". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 18 00:07:20.033287 ip-10-0-132-188 kubenswrapper[2562]: I0918 00:07:20.033185    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/d4103fa7-df05-46ec-ad32-93f420000988-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "d4103fa7-df05-46ec-ad32-93f420000988" (UID: "d4103fa7-df05-46ec-ad32-93f420000988"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 18 00:07:20.071464 ip-10-0-132-188 kubenswrapper[2562]: I0918 00:07:20.071426    2562 generic.go:356] "Generic (PLEG): container finished" podID="d4103fa7-df05-46ec-ad32-93f420000988" containerID="d9f97b83d20a62bf5d3735455d3342425862c38754dfc2690fdbabd08a4ed11d" exitCode=0
Sep 18 00:07:20.071605 ip-10-0-132-188 kubenswrapper[2562]: I0918 00:07:20.071479    2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-test-llm-inference-service-d73d44f4/custom-route-timeout-pd-test-kserve-router-scheduler-84c48mvfch"
Sep 18 00:07:20.071605 ip-10-0-132-188 kubenswrapper[2562]: I0918 00:07:20.071508    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-test-llm-inference-service-d73d44f4/custom-route-timeout-pd-test-kserve-router-scheduler-84c48mvfch" event={"ID":"d4103fa7-df05-46ec-ad32-93f420000988","Type":"ContainerDied","Data":"d9f97b83d20a62bf5d3735455d3342425862c38754dfc2690fdbabd08a4ed11d"}
Sep 18 00:07:20.071605 ip-10-0-132-188 kubenswrapper[2562]: I0918 00:07:20.071563    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-test-llm-inference-service-d73d44f4/custom-route-timeout-pd-test-kserve-router-scheduler-84c48mvfch" event={"ID":"d4103fa7-df05-46ec-ad32-93f420000988","Type":"ContainerDied","Data":"e84dc2b1342c82bebda382e2b17f7bcdb90c1bd89788d99cdfd9c07f44fc996c"}
Sep 18 00:07:20.071605 ip-10-0-132-188 kubenswrapper[2562]: I0918 00:07:20.071587    2562 scope.go:117] "RemoveContainer" containerID="d9f97b83d20a62bf5d3735455d3342425862c38754dfc2690fdbabd08a4ed11d"
Sep 18 00:07:20.080405 ip-10-0-132-188 kubenswrapper[2562]: I0918 00:07:20.080382    2562 scope.go:117] "RemoveContainer" containerID="d9f97b83d20a62bf5d3735455d3342425862c38754dfc2690fdbabd08a4ed11d"
Sep 18 00:07:20.080745 ip-10-0-132-188 kubenswrapper[2562]: E0918 00:07:20.080703    2562 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"d9f97b83d20a62bf5d3735455d3342425862c38754dfc2690fdbabd08a4ed11d\": container with ID starting with d9f97b83d20a62bf5d3735455d3342425862c38754dfc2690fdbabd08a4ed11d not found: ID does not exist" containerID="d9f97b83d20a62bf5d3735455d3342425862c38754dfc2690fdbabd08a4ed11d"
Sep 18 00:07:20.080807 ip-10-0-132-188 kubenswrapper[2562]: I0918 00:07:20.080752    2562 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d9f97b83d20a62bf5d3735455d3342425862c38754dfc2690fdbabd08a4ed11d"} err="failed to get container status \"d9f97b83d20a62bf5d3735455d3342425862c38754dfc2690fdbabd08a4ed11d\": rpc error: code = NotFound desc = could not find container \"d9f97b83d20a62bf5d3735455d3342425862c38754dfc2690fdbabd08a4ed11d\": container with ID starting with d9f97b83d20a62bf5d3735455d3342425862c38754dfc2690fdbabd08a4ed11d not found: ID does not exist"
Sep 18 00:07:20.091707 ip-10-0-132-188 kubenswrapper[2562]: I0918 00:07:20.091676    2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["e2e-test-llm-inference-service-d73d44f4/custom-route-timeout-pd-test-kserve-router-scheduler-84c48mvfch"]
Sep 18 00:07:20.093815 ip-10-0-132-188 kubenswrapper[2562]: I0918 00:07:20.093787    2562 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["e2e-test-llm-inference-service-d73d44f4/custom-route-timeout-pd-test-kserve-router-scheduler-84c48mvfch"]
Sep 18 00:07:20.131801 ip-10-0-132-188 kubenswrapper[2562]: I0918 00:07:20.131760    2562 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/d4103fa7-df05-46ec-ad32-93f420000988-tls-certs\") on node \"ip-10-0-132-188.ec2.internal\" DevicePath \"\""
Sep 18 00:07:20.131801 ip-10-0-132-188 kubenswrapper[2562]: I0918 00:07:20.131794    2562 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-b7lsv\" (UniqueName: \"kubernetes.io/projected/d4103fa7-df05-46ec-ad32-93f420000988-kube-api-access-b7lsv\") on node \"ip-10-0-132-188.ec2.internal\" DevicePath \"\""
Sep 18 00:07:21.671317 ip-10-0-132-188 kubenswrapper[2562]: I0918 00:07:21.671278    2562 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="d4103fa7-df05-46ec-ad32-93f420000988" path="/var/lib/kubelet/pods/d4103fa7-df05-46ec-ad32-93f420000988/volumes"
Sep 18 00:07:24.792456 ip-10-0-132-188 kubenswrapper[2562]: I0918 00:07:24.792420    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fwwc5_7e452bbb-7dc6-4c9d-a9b1-76400942f5b4/konnectivity-agent/0.log" limit=1024
Sep 18 00:07:25.017254 ip-10-0-132-188 kubenswrapper[2562]: I0918 00:07:25.017211    2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["e2e-test-llm-inference-service-b5dd93e6/router-gateway-2-openshift-default-57b79bd49b-2hg85"]
Sep 18 00:07:25.017626 ip-10-0-132-188 kubenswrapper[2562]: I0918 00:07:25.017609    2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d4103fa7-df05-46ec-ad32-93f420000988" containerName="main"
Sep 18 00:07:25.017626 ip-10-0-132-188 kubenswrapper[2562]: I0918 00:07:25.017626    2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="d4103fa7-df05-46ec-ad32-93f420000988" containerName="main"
Sep 18 00:07:25.017782 ip-10-0-132-188 kubenswrapper[2562]: I0918 00:07:25.017681    2562 memory_manager.go:368] "RemoveStaleState removing state" podUID="d4103fa7-df05-46ec-ad32-93f420000988" containerName="main"
Sep 18 00:07:25.020586 ip-10-0-132-188 kubenswrapper[2562]: I0918 00:07:25.020566    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-test-llm-inference-service-b5dd93e6/router-gateway-2-openshift-default-57b79bd49b-2hg85"
Sep 18 00:07:25.024533 ip-10-0-132-188 kubenswrapper[2562]: I0918 00:07:25.024506    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-test-llm-inference-service-b5dd93e6\"/\"router-gateway-2-openshift-default-dockercfg-7xt2v\""
Sep 18 00:07:25.024951 ip-10-0-132-188 kubenswrapper[2562]: I0918 00:07:25.024931    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-test-llm-inference-service-b5dd93e6\"/\"openshift-gw-ca-root-cert\""
Sep 18 00:07:25.025523 ip-10-0-132-188 kubenswrapper[2562]: I0918 00:07:25.025505    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-test-llm-inference-service-b5dd93e6\"/\"openshift-service-ca.crt\""
Sep 18 00:07:25.025846 ip-10-0-132-188 kubenswrapper[2562]: I0918 00:07:25.025832    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-test-llm-inference-service-b5dd93e6\"/\"kube-root-ca.crt\""
Sep 18 00:07:25.052158 ip-10-0-132-188 kubenswrapper[2562]: I0918 00:07:25.052077    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["e2e-test-llm-inference-service-b5dd93e6/router-gateway-2-openshift-default-57b79bd49b-2hg85"]
Sep 18 00:07:25.071628 ip-10-0-132-188 kubenswrapper[2562]: I0918 00:07:25.071592    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-pxsfn\" (UniqueName: \"kubernetes.io/projected/b664114a-d5e2-49bb-b3b6-289d1ebca5ca-kube-api-access-pxsfn\") pod \"router-gateway-2-openshift-default-57b79bd49b-2hg85\" (UID: \"b664114a-d5e2-49bb-b3b6-289d1ebca5ca\") " pod="e2e-test-llm-inference-service-b5dd93e6/router-gateway-2-openshift-default-57b79bd49b-2hg85"
Sep 18 00:07:25.071806 ip-10-0-132-188 kubenswrapper[2562]: I0918 00:07:25.071641    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/b664114a-d5e2-49bb-b3b6-289d1ebca5ca-istio-podinfo\") pod \"router-gateway-2-openshift-default-57b79bd49b-2hg85\" (UID: \"b664114a-d5e2-49bb-b3b6-289d1ebca5ca\") " pod="e2e-test-llm-inference-service-b5dd93e6/router-gateway-2-openshift-default-57b79bd49b-2hg85"
Sep 18 00:07:25.071806 ip-10-0-132-188 kubenswrapper[2562]: I0918 00:07:25.071688    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/b664114a-d5e2-49bb-b3b6-289d1ebca5ca-workload-socket\") pod \"router-gateway-2-openshift-default-57b79bd49b-2hg85\" (UID: \"b664114a-d5e2-49bb-b3b6-289d1ebca5ca\") " pod="e2e-test-llm-inference-service-b5dd93e6/router-gateway-2-openshift-default-57b79bd49b-2hg85"
Sep 18 00:07:25.071806 ip-10-0-132-188 kubenswrapper[2562]: I0918 00:07:25.071724    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/b664114a-d5e2-49bb-b3b6-289d1ebca5ca-istio-token\") pod \"router-gateway-2-openshift-default-57b79bd49b-2hg85\" (UID: \"b664114a-d5e2-49bb-b3b6-289d1ebca5ca\") " pod="e2e-test-llm-inference-service-b5dd93e6/router-gateway-2-openshift-default-57b79bd49b-2hg85"
Sep 18 00:07:25.071806 ip-10-0-132-188 kubenswrapper[2562]: I0918 00:07:25.071764    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/b664114a-d5e2-49bb-b3b6-289d1ebca5ca-istiod-ca-cert\") pod \"router-gateway-2-openshift-default-57b79bd49b-2hg85\" (UID: \"b664114a-d5e2-49bb-b3b6-289d1ebca5ca\") " pod="e2e-test-llm-inference-service-b5dd93e6/router-gateway-2-openshift-default-57b79bd49b-2hg85"
Sep 18 00:07:25.071973 ip-10-0-132-188 kubenswrapper[2562]: I0918 00:07:25.071824    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/b664114a-d5e2-49bb-b3b6-289d1ebca5ca-credential-socket\") pod \"router-gateway-2-openshift-default-57b79bd49b-2hg85\" (UID: \"b664114a-d5e2-49bb-b3b6-289d1ebca5ca\") " pod="e2e-test-llm-inference-service-b5dd93e6/router-gateway-2-openshift-default-57b79bd49b-2hg85"
Sep 18 00:07:25.071973 ip-10-0-132-188 kubenswrapper[2562]: I0918 00:07:25.071859    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/b664114a-d5e2-49bb-b3b6-289d1ebca5ca-workload-certs\") pod \"router-gateway-2-openshift-default-57b79bd49b-2hg85\" (UID: \"b664114a-d5e2-49bb-b3b6-289d1ebca5ca\") " pod="e2e-test-llm-inference-service-b5dd93e6/router-gateway-2-openshift-default-57b79bd49b-2hg85"
Sep 18 00:07:25.071973 ip-10-0-132-188 kubenswrapper[2562]: I0918 00:07:25.071911    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/b664114a-d5e2-49bb-b3b6-289d1ebca5ca-istio-data\") pod \"router-gateway-2-openshift-default-57b79bd49b-2hg85\" (UID: \"b664114a-d5e2-49bb-b3b6-289d1ebca5ca\") " pod="e2e-test-llm-inference-service-b5dd93e6/router-gateway-2-openshift-default-57b79bd49b-2hg85"
Sep 18 00:07:25.071973 ip-10-0-132-188 kubenswrapper[2562]: I0918 00:07:25.071944    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/b664114a-d5e2-49bb-b3b6-289d1ebca5ca-istio-envoy\") pod \"router-gateway-2-openshift-default-57b79bd49b-2hg85\" (UID: \"b664114a-d5e2-49bb-b3b6-289d1ebca5ca\") " pod="e2e-test-llm-inference-service-b5dd93e6/router-gateway-2-openshift-default-57b79bd49b-2hg85"
Sep 18 00:07:25.172856 ip-10-0-132-188 kubenswrapper[2562]: I0918 00:07:25.172819    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-pxsfn\" (UniqueName: \"kubernetes.io/projected/b664114a-d5e2-49bb-b3b6-289d1ebca5ca-kube-api-access-pxsfn\") pod \"router-gateway-2-openshift-default-57b79bd49b-2hg85\" (UID: \"b664114a-d5e2-49bb-b3b6-289d1ebca5ca\") " pod="e2e-test-llm-inference-service-b5dd93e6/router-gateway-2-openshift-default-57b79bd49b-2hg85"
Sep 18 00:07:25.173061 ip-10-0-132-188 kubenswrapper[2562]: I0918 00:07:25.172867    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/b664114a-d5e2-49bb-b3b6-289d1ebca5ca-istio-podinfo\") pod \"router-gateway-2-openshift-default-57b79bd49b-2hg85\" (UID: \"b664114a-d5e2-49bb-b3b6-289d1ebca5ca\") " pod="e2e-test-llm-inference-service-b5dd93e6/router-gateway-2-openshift-default-57b79bd49b-2hg85"
Sep 18 00:07:25.173061 ip-10-0-132-188 kubenswrapper[2562]: I0918 00:07:25.172884    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/b664114a-d5e2-49bb-b3b6-289d1ebca5ca-workload-socket\") pod \"router-gateway-2-openshift-default-57b79bd49b-2hg85\" (UID: \"b664114a-d5e2-49bb-b3b6-289d1ebca5ca\") " pod="e2e-test-llm-inference-service-b5dd93e6/router-gateway-2-openshift-default-57b79bd49b-2hg85"
Sep 18 00:07:25.173061 ip-10-0-132-188 kubenswrapper[2562]: I0918 00:07:25.172906    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/b664114a-d5e2-49bb-b3b6-289d1ebca5ca-istio-token\") pod \"router-gateway-2-openshift-default-57b79bd49b-2hg85\" (UID: \"b664114a-d5e2-49bb-b3b6-289d1ebca5ca\") " pod="e2e-test-llm-inference-service-b5dd93e6/router-gateway-2-openshift-default-57b79bd49b-2hg85"
Sep 18 00:07:25.173061 ip-10-0-132-188 kubenswrapper[2562]: I0918 00:07:25.172933    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/b664114a-d5e2-49bb-b3b6-289d1ebca5ca-istiod-ca-cert\") pod \"router-gateway-2-openshift-default-57b79bd49b-2hg85\" (UID: \"b664114a-d5e2-49bb-b3b6-289d1ebca5ca\") " pod="e2e-test-llm-inference-service-b5dd93e6/router-gateway-2-openshift-default-57b79bd49b-2hg85"
Sep 18 00:07:25.173061 ip-10-0-132-188 kubenswrapper[2562]: I0918 00:07:25.172960    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/b664114a-d5e2-49bb-b3b6-289d1ebca5ca-credential-socket\") pod \"router-gateway-2-openshift-default-57b79bd49b-2hg85\" (UID: \"b664114a-d5e2-49bb-b3b6-289d1ebca5ca\") " pod="e2e-test-llm-inference-service-b5dd93e6/router-gateway-2-openshift-default-57b79bd49b-2hg85"
Sep 18 00:07:25.173061 ip-10-0-132-188 kubenswrapper[2562]: I0918 00:07:25.173016    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/b664114a-d5e2-49bb-b3b6-289d1ebca5ca-workload-certs\") pod \"router-gateway-2-openshift-default-57b79bd49b-2hg85\" (UID: \"b664114a-d5e2-49bb-b3b6-289d1ebca5ca\") " pod="e2e-test-llm-inference-service-b5dd93e6/router-gateway-2-openshift-default-57b79bd49b-2hg85"
Sep 18 00:07:25.173464 ip-10-0-132-188 kubenswrapper[2562]: I0918 00:07:25.173159    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/b664114a-d5e2-49bb-b3b6-289d1ebca5ca-istio-data\") pod \"router-gateway-2-openshift-default-57b79bd49b-2hg85\" (UID: \"b664114a-d5e2-49bb-b3b6-289d1ebca5ca\") " pod="e2e-test-llm-inference-service-b5dd93e6/router-gateway-2-openshift-default-57b79bd49b-2hg85"
Sep 18 00:07:25.173464 ip-10-0-132-188 kubenswrapper[2562]: I0918 00:07:25.173253    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/b664114a-d5e2-49bb-b3b6-289d1ebca5ca-istio-envoy\") pod \"router-gateway-2-openshift-default-57b79bd49b-2hg85\" (UID: \"b664114a-d5e2-49bb-b3b6-289d1ebca5ca\") " pod="e2e-test-llm-inference-service-b5dd93e6/router-gateway-2-openshift-default-57b79bd49b-2hg85"
Sep 18 00:07:25.173464 ip-10-0-132-188 kubenswrapper[2562]: I0918 00:07:25.173381    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/b664114a-d5e2-49bb-b3b6-289d1ebca5ca-workload-socket\") pod \"router-gateway-2-openshift-default-57b79bd49b-2hg85\" (UID: \"b664114a-d5e2-49bb-b3b6-289d1ebca5ca\") " pod="e2e-test-llm-inference-service-b5dd93e6/router-gateway-2-openshift-default-57b79bd49b-2hg85"
Sep 18 00:07:25.173464 ip-10-0-132-188 kubenswrapper[2562]: I0918 00:07:25.173395    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/b664114a-d5e2-49bb-b3b6-289d1ebca5ca-workload-certs\") pod \"router-gateway-2-openshift-default-57b79bd49b-2hg85\" (UID: \"b664114a-d5e2-49bb-b3b6-289d1ebca5ca\") " pod="e2e-test-llm-inference-service-b5dd93e6/router-gateway-2-openshift-default-57b79bd49b-2hg85"
Sep 18 00:07:25.173687 ip-10-0-132-188 kubenswrapper[2562]: I0918 00:07:25.173489    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/b664114a-d5e2-49bb-b3b6-289d1ebca5ca-credential-socket\") pod \"router-gateway-2-openshift-default-57b79bd49b-2hg85\" (UID: \"b664114a-d5e2-49bb-b3b6-289d1ebca5ca\") " pod="e2e-test-llm-inference-service-b5dd93e6/router-gateway-2-openshift-default-57b79bd49b-2hg85"
Sep 18 00:07:25.173687 ip-10-0-132-188 kubenswrapper[2562]: I0918 00:07:25.173614    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/b664114a-d5e2-49bb-b3b6-289d1ebca5ca-istio-data\") pod \"router-gateway-2-openshift-default-57b79bd49b-2hg85\" (UID: \"b664114a-d5e2-49bb-b3b6-289d1ebca5ca\") " pod="e2e-test-llm-inference-service-b5dd93e6/router-gateway-2-openshift-default-57b79bd49b-2hg85"
Sep 18 00:07:25.173834 ip-10-0-132-188 kubenswrapper[2562]: I0918 00:07:25.173812    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/b664114a-d5e2-49bb-b3b6-289d1ebca5ca-istiod-ca-cert\") pod \"router-gateway-2-openshift-default-57b79bd49b-2hg85\" (UID: \"b664114a-d5e2-49bb-b3b6-289d1ebca5ca\") " pod="e2e-test-llm-inference-service-b5dd93e6/router-gateway-2-openshift-default-57b79bd49b-2hg85"
Sep 18 00:07:25.175401 ip-10-0-132-188 kubenswrapper[2562]: I0918 00:07:25.175380    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/b664114a-d5e2-49bb-b3b6-289d1ebca5ca-istio-envoy\") pod \"router-gateway-2-openshift-default-57b79bd49b-2hg85\" (UID: \"b664114a-d5e2-49bb-b3b6-289d1ebca5ca\") " pod="e2e-test-llm-inference-service-b5dd93e6/router-gateway-2-openshift-default-57b79bd49b-2hg85"
Sep 18 00:07:25.175522 ip-10-0-132-188 kubenswrapper[2562]: I0918 00:07:25.175504    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/b664114a-d5e2-49bb-b3b6-289d1ebca5ca-istio-podinfo\") pod \"router-gateway-2-openshift-default-57b79bd49b-2hg85\" (UID: \"b664114a-d5e2-49bb-b3b6-289d1ebca5ca\") " pod="e2e-test-llm-inference-service-b5dd93e6/router-gateway-2-openshift-default-57b79bd49b-2hg85"
Sep 18 00:07:25.186463 ip-10-0-132-188 kubenswrapper[2562]: I0918 00:07:25.186426    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-pxsfn\" (UniqueName: \"kubernetes.io/projected/b664114a-d5e2-49bb-b3b6-289d1ebca5ca-kube-api-access-pxsfn\") pod \"router-gateway-2-openshift-default-57b79bd49b-2hg85\" (UID: \"b664114a-d5e2-49bb-b3b6-289d1ebca5ca\") " pod="e2e-test-llm-inference-service-b5dd93e6/router-gateway-2-openshift-default-57b79bd49b-2hg85"
Sep 18 00:07:25.187354 ip-10-0-132-188 kubenswrapper[2562]: I0918 00:07:25.187334    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/b664114a-d5e2-49bb-b3b6-289d1ebca5ca-istio-token\") pod \"router-gateway-2-openshift-default-57b79bd49b-2hg85\" (UID: \"b664114a-d5e2-49bb-b3b6-289d1ebca5ca\") " pod="e2e-test-llm-inference-service-b5dd93e6/router-gateway-2-openshift-default-57b79bd49b-2hg85"
Sep 18 00:07:25.332055 ip-10-0-132-188 kubenswrapper[2562]: I0918 00:07:25.332015    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-test-llm-inference-service-b5dd93e6/router-gateway-2-openshift-default-57b79bd49b-2hg85"
Sep 18 00:07:25.611779 ip-10-0-132-188 kubenswrapper[2562]: I0918 00:07:25.611749    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["e2e-test-llm-inference-service-b5dd93e6/router-gateway-2-openshift-default-57b79bd49b-2hg85"]
Sep 18 00:07:25.614496 ip-10-0-132-188 kubenswrapper[2562]: W0918 00:07:25.614465    2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb664114a_d5e2_49bb_b3b6_289d1ebca5ca.slice/crio-94c8cb80d0998d98e0f9d125f93a10ff1fb7d7cd327a0b1e058fd9008e5857fe WatchSource:0}: Error finding container 94c8cb80d0998d98e0f9d125f93a10ff1fb7d7cd327a0b1e058fd9008e5857fe: Status 404 returned error can't find the container with id 94c8cb80d0998d98e0f9d125f93a10ff1fb7d7cd327a0b1e058fd9008e5857fe
Sep 18 00:07:25.616676 ip-10-0-132-188 kubenswrapper[2562]: I0918 00:07:25.616637    2562 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"}
Sep 18 00:07:25.616761 ip-10-0-132-188 kubenswrapper[2562]: I0918 00:07:25.616717    2562 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"}
Sep 18 00:07:25.616761 ip-10-0-132-188 kubenswrapper[2562]: I0918 00:07:25.616751    2562 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"}
Sep 18 00:07:26.105787 ip-10-0-132-188 kubenswrapper[2562]: I0918 00:07:26.105747    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-test-llm-inference-service-b5dd93e6/router-gateway-2-openshift-default-57b79bd49b-2hg85" event={"ID":"b664114a-d5e2-49bb-b3b6-289d1ebca5ca","Type":"ContainerStarted","Data":"4fe2749454c9911e4200f54fe3c085a18dc0a2f86eee1ebdaf1c5211535b74cd"}
Sep 18 00:07:26.106273 ip-10-0-132-188 kubenswrapper[2562]: I0918 00:07:26.105812    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-test-llm-inference-service-b5dd93e6/router-gateway-2-openshift-default-57b79bd49b-2hg85" event={"ID":"b664114a-d5e2-49bb-b3b6-289d1ebca5ca","Type":"ContainerStarted","Data":"94c8cb80d0998d98e0f9d125f93a10ff1fb7d7cd327a0b1e058fd9008e5857fe"}
Sep 18 00:07:26.125555 ip-10-0-132-188 kubenswrapper[2562]: I0918 00:07:26.125494    2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="e2e-test-llm-inference-service-b5dd93e6/router-gateway-2-openshift-default-57b79bd49b-2hg85" podStartSLOduration=2.125474618 podStartE2EDuration="2.125474618s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-18 00:07:24 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-18 00:07:26.123760917 +0000 UTC m=+3398.525657684" watchObservedRunningTime="2026-09-18 00:07:26.125474618 +0000 UTC m=+3398.527371382"
Sep 18 00:07:26.332753 ip-10-0-132-188 kubenswrapper[2562]: I0918 00:07:26.332716    2562 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="e2e-test-llm-inference-service-b5dd93e6/router-gateway-2-openshift-default-57b79bd49b-2hg85"
Sep 18 00:07:26.333880 ip-10-0-132-188 kubenswrapper[2562]: I0918 00:07:26.333851    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fwwc5_7e452bbb-7dc6-4c9d-a9b1-76400942f5b4/konnectivity-agent/0.log" limit=1024
Sep 18 00:07:26.338074 ip-10-0-132-188 kubenswrapper[2562]: I0918 00:07:26.338052    2562 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="e2e-test-llm-inference-service-b5dd93e6/router-gateway-2-openshift-default-57b79bd49b-2hg85"
Sep 18 00:07:27.110422 ip-10-0-132-188 kubenswrapper[2562]: I0918 00:07:27.110392    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-test-llm-inference-service-b5dd93e6/router-gateway-2-openshift-default-57b79bd49b-2hg85"
Sep 18 00:07:27.111342 ip-10-0-132-188 kubenswrapper[2562]: I0918 00:07:27.111315    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-test-llm-inference-service-b5dd93e6/router-gateway-2-openshift-default-57b79bd49b-2hg85"
Sep 18 00:07:28.425386 ip-10-0-132-188 kubenswrapper[2562]: I0918 00:07:28.425349    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fwwc5_7e452bbb-7dc6-4c9d-a9b1-76400942f5b4/konnectivity-agent/0.log" limit=1024
Sep 18 00:07:31.389814 ip-10-0-132-188 kubenswrapper[2562]: I0918 00:07:31.389712    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fwwc5_7e452bbb-7dc6-4c9d-a9b1-76400942f5b4/konnectivity-agent/0.log" limit=1024
Sep 18 00:07:36.881461 ip-10-0-132-188 kubenswrapper[2562]: I0918 00:07:36.881416    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fwwc5_7e452bbb-7dc6-4c9d-a9b1-76400942f5b4/konnectivity-agent/0.log" limit=1024
Sep 18 00:07:36.938353 ip-10-0-132-188 kubenswrapper[2562]: I0918 00:07:36.938310    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fwwc5_7e452bbb-7dc6-4c9d-a9b1-76400942f5b4/konnectivity-agent/0.log" limit=1024
Sep 18 00:07:37.158197 ip-10-0-132-188 kubenswrapper[2562]: I0918 00:07:37.158096    2562 generic.go:356] "Generic (PLEG): container finished" podID="b664114a-d5e2-49bb-b3b6-289d1ebca5ca" containerID="4fe2749454c9911e4200f54fe3c085a18dc0a2f86eee1ebdaf1c5211535b74cd" exitCode=0
Sep 18 00:07:37.158197 ip-10-0-132-188 kubenswrapper[2562]: I0918 00:07:37.158179    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-test-llm-inference-service-b5dd93e6/router-gateway-2-openshift-default-57b79bd49b-2hg85" event={"ID":"b664114a-d5e2-49bb-b3b6-289d1ebca5ca","Type":"ContainerDied","Data":"4fe2749454c9911e4200f54fe3c085a18dc0a2f86eee1ebdaf1c5211535b74cd"}
Sep 18 00:07:37.158596 ip-10-0-132-188 kubenswrapper[2562]: I0918 00:07:37.158581    2562 scope.go:117] "RemoveContainer" containerID="4fe2749454c9911e4200f54fe3c085a18dc0a2f86eee1ebdaf1c5211535b74cd"
Sep 18 00:07:37.159536 ip-10-0-132-188 kubenswrapper[2562]: I0918 00:07:37.159506    2562 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"}
Sep 18 00:07:37.159642 ip-10-0-132-188 kubenswrapper[2562]: I0918 00:07:37.159590    2562 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"}
Sep 18 00:07:37.159704 ip-10-0-132-188 kubenswrapper[2562]: I0918 00:07:37.159638    2562 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"}
Sep 18 00:07:37.976115 ip-10-0-132-188 kubenswrapper[2562]: I0918 00:07:37.976071    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fwwc5_7e452bbb-7dc6-4c9d-a9b1-76400942f5b4/konnectivity-agent/0.log" limit=1024
Sep 18 00:07:38.166539 ip-10-0-132-188 kubenswrapper[2562]: I0918 00:07:38.166503    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-test-llm-inference-service-b5dd93e6/router-gateway-2-openshift-default-57b79bd49b-2hg85" event={"ID":"b664114a-d5e2-49bb-b3b6-289d1ebca5ca","Type":"ContainerStarted","Data":"1c0996e40c975d8861a410b78ebd28eb3be78331286023c3bb0960c998712cd7"}
Sep 18 00:07:38.333050 ip-10-0-132-188 kubenswrapper[2562]: I0918 00:07:38.333012    2562 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="e2e-test-llm-inference-service-b5dd93e6/router-gateway-2-openshift-default-57b79bd49b-2hg85"
Sep 18 00:07:38.334435 ip-10-0-132-188 kubenswrapper[2562]: I0918 00:07:38.334403    2562 prober.go:120] "Probe failed" probeType="Startup" pod="e2e-test-llm-inference-service-b5dd93e6/router-gateway-2-openshift-default-57b79bd49b-2hg85" podUID="b664114a-d5e2-49bb-b3b6-289d1ebca5ca" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.54:15021/healthz/ready\": dial tcp 10.134.0.54:15021: connect: connection refused"
Sep 18 00:07:38.361314 ip-10-0-132-188 kubenswrapper[2562]: I0918 00:07:38.353244    2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["e2e-test-llm-inference-service-b5dd93e6/router-with-refs-pd-test-kserve-router-scheduler-99c668b45mdhcc"]
Sep 18 00:07:38.361314 ip-10-0-132-188 kubenswrapper[2562]: I0918 00:07:38.359148    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-test-llm-inference-service-b5dd93e6/router-with-refs-pd-test-kserve-router-scheduler-99c668b45mdhcc"
Sep 18 00:07:38.366787 ip-10-0-132-188 kubenswrapper[2562]: I0918 00:07:38.365723    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-test-llm-inference-service-b5dd93e6\"/\"default-dockercfg-gxpf2\""
Sep 18 00:07:38.366787 ip-10-0-132-188 kubenswrapper[2562]: I0918 00:07:38.366000    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-test-llm-inference-service-b5dd93e6\"/\"router-with-refs-pd-test-epp-sa-dockercfg-h2qqc\""
Sep 18 00:07:38.366787 ip-10-0-132-188 kubenswrapper[2562]: I0918 00:07:38.366228    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-test-llm-inference-service-b5dd93e6\"/\"router-with-refs-pd-test-kserve-self-signed-certs\""
Sep 18 00:07:38.373790 ip-10-0-132-188 kubenswrapper[2562]: I0918 00:07:38.370708    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["e2e-test-llm-inference-service-b5dd93e6/router-with-refs-pd-test-kserve-router-scheduler-99c668b45mdhcc"]
Sep 18 00:07:38.385725 ip-10-0-132-188 kubenswrapper[2562]: I0918 00:07:38.385605    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fwwc5_7e452bbb-7dc6-4c9d-a9b1-76400942f5b4/konnectivity-agent/0.log" limit=1024
Sep 18 00:07:38.458194 ip-10-0-132-188 kubenswrapper[2562]: I0918 00:07:38.458117    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fwwc5_7e452bbb-7dc6-4c9d-a9b1-76400942f5b4/konnectivity-agent/0.log" limit=1024
Sep 18 00:07:38.506272 ip-10-0-132-188 kubenswrapper[2562]: I0918 00:07:38.506226    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dlzzg\" (UniqueName: \"kubernetes.io/projected/37650465-7a5f-49f5-9785-527e96f329e4-kube-api-access-dlzzg\") pod \"router-with-refs-pd-test-kserve-router-scheduler-99c668b45mdhcc\" (UID: \"37650465-7a5f-49f5-9785-527e96f329e4\") " pod="e2e-test-llm-inference-service-b5dd93e6/router-with-refs-pd-test-kserve-router-scheduler-99c668b45mdhcc"
Sep 18 00:07:38.506483 ip-10-0-132-188 kubenswrapper[2562]: I0918 00:07:38.506296    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/37650465-7a5f-49f5-9785-527e96f329e4-tls-certs\") pod \"router-with-refs-pd-test-kserve-router-scheduler-99c668b45mdhcc\" (UID: \"37650465-7a5f-49f5-9785-527e96f329e4\") " pod="e2e-test-llm-inference-service-b5dd93e6/router-with-refs-pd-test-kserve-router-scheduler-99c668b45mdhcc"
Sep 18 00:07:38.607106 ip-10-0-132-188 kubenswrapper[2562]: I0918 00:07:38.607000    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dlzzg\" (UniqueName: \"kubernetes.io/projected/37650465-7a5f-49f5-9785-527e96f329e4-kube-api-access-dlzzg\") pod \"router-with-refs-pd-test-kserve-router-scheduler-99c668b45mdhcc\" (UID: \"37650465-7a5f-49f5-9785-527e96f329e4\") " pod="e2e-test-llm-inference-service-b5dd93e6/router-with-refs-pd-test-kserve-router-scheduler-99c668b45mdhcc"
Sep 18 00:07:38.607106 ip-10-0-132-188 kubenswrapper[2562]: I0918 00:07:38.607064    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/37650465-7a5f-49f5-9785-527e96f329e4-tls-certs\") pod \"router-with-refs-pd-test-kserve-router-scheduler-99c668b45mdhcc\" (UID: \"37650465-7a5f-49f5-9785-527e96f329e4\") " pod="e2e-test-llm-inference-service-b5dd93e6/router-with-refs-pd-test-kserve-router-scheduler-99c668b45mdhcc"
Sep 18 00:07:38.609989 ip-10-0-132-188 kubenswrapper[2562]: I0918 00:07:38.609952    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/37650465-7a5f-49f5-9785-527e96f329e4-tls-certs\") pod \"router-with-refs-pd-test-kserve-router-scheduler-99c668b45mdhcc\" (UID: \"37650465-7a5f-49f5-9785-527e96f329e4\") " pod="e2e-test-llm-inference-service-b5dd93e6/router-with-refs-pd-test-kserve-router-scheduler-99c668b45mdhcc"
Sep 18 00:07:38.616109 ip-10-0-132-188 kubenswrapper[2562]: I0918 00:07:38.616071    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-dlzzg\" (UniqueName: \"kubernetes.io/projected/37650465-7a5f-49f5-9785-527e96f329e4-kube-api-access-dlzzg\") pod \"router-with-refs-pd-test-kserve-router-scheduler-99c668b45mdhcc\" (UID: \"37650465-7a5f-49f5-9785-527e96f329e4\") " pod="e2e-test-llm-inference-service-b5dd93e6/router-with-refs-pd-test-kserve-router-scheduler-99c668b45mdhcc"
Sep 18 00:07:38.685834 ip-10-0-132-188 kubenswrapper[2562]: I0918 00:07:38.685785    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-test-llm-inference-service-b5dd93e6/router-with-refs-pd-test-kserve-router-scheduler-99c668b45mdhcc"
Sep 18 00:07:38.979487 ip-10-0-132-188 kubenswrapper[2562]: I0918 00:07:38.979457    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["e2e-test-llm-inference-service-b5dd93e6/router-with-refs-pd-test-kserve-router-scheduler-99c668b45mdhcc"]
Sep 18 00:07:38.982103 ip-10-0-132-188 kubenswrapper[2562]: W0918 00:07:38.982067    2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod37650465_7a5f_49f5_9785_527e96f329e4.slice/crio-04fabb76375505388092fda8068d0d093df536f5b9504af772fddd3580f6ec7d WatchSource:0}: Error finding container 04fabb76375505388092fda8068d0d093df536f5b9504af772fddd3580f6ec7d: Status 404 returned error can't find the container with id 04fabb76375505388092fda8068d0d093df536f5b9504af772fddd3580f6ec7d
Sep 18 00:07:39.173344 ip-10-0-132-188 kubenswrapper[2562]: I0918 00:07:39.173300    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-test-llm-inference-service-b5dd93e6/router-with-refs-pd-test-kserve-router-scheduler-99c668b45mdhcc" event={"ID":"37650465-7a5f-49f5-9785-527e96f329e4","Type":"ContainerStarted","Data":"8ab4fa5a9a68460cfeb65f0478765a8b7660f7a5d9f93e367af851862ffa481f"}
Sep 18 00:07:39.173520 ip-10-0-132-188 kubenswrapper[2562]: I0918 00:07:39.173363    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-test-llm-inference-service-b5dd93e6/router-with-refs-pd-test-kserve-router-scheduler-99c668b45mdhcc" event={"ID":"37650465-7a5f-49f5-9785-527e96f329e4","Type":"ContainerStarted","Data":"04fabb76375505388092fda8068d0d093df536f5b9504af772fddd3580f6ec7d"}
Sep 18 00:07:39.173610 ip-10-0-132-188 kubenswrapper[2562]: I0918 00:07:39.173533    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-test-llm-inference-service-b5dd93e6/router-with-refs-pd-test-kserve-router-scheduler-99c668b45mdhcc"
Sep 18 00:07:39.190536 ip-10-0-132-188 kubenswrapper[2562]: I0918 00:07:39.190481    2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="e2e-test-llm-inference-service-b5dd93e6/router-with-refs-pd-test-kserve-router-scheduler-99c668b45mdhcc" podStartSLOduration=1.190462213 podStartE2EDuration="1.190462213s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-18 00:07:38 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-18 00:07:39.189397526 +0000 UTC m=+3411.591294292" watchObservedRunningTime="2026-09-18 00:07:39.190462213 +0000 UTC m=+3411.592358978"
Sep 18 00:07:39.332586 ip-10-0-132-188 kubenswrapper[2562]: I0918 00:07:39.332541    2562 prober.go:120] "Probe failed" probeType="Startup" pod="e2e-test-llm-inference-service-b5dd93e6/router-gateway-2-openshift-default-57b79bd49b-2hg85" podUID="b664114a-d5e2-49bb-b3b6-289d1ebca5ca" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.54:15021/healthz/ready\": dial tcp 10.134.0.54:15021: connect: connection refused"
Sep 18 00:07:40.333359 ip-10-0-132-188 kubenswrapper[2562]: I0918 00:07:40.333314    2562 prober.go:120] "Probe failed" probeType="Startup" pod="e2e-test-llm-inference-service-b5dd93e6/router-gateway-2-openshift-default-57b79bd49b-2hg85" podUID="b664114a-d5e2-49bb-b3b6-289d1ebca5ca" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.54:15021/healthz/ready\": dial tcp 10.134.0.54:15021: connect: connection refused"
Sep 18 00:07:41.332666 ip-10-0-132-188 kubenswrapper[2562]: I0918 00:07:41.332618    2562 prober.go:120] "Probe failed" probeType="Startup" pod="e2e-test-llm-inference-service-b5dd93e6/router-gateway-2-openshift-default-57b79bd49b-2hg85" podUID="b664114a-d5e2-49bb-b3b6-289d1ebca5ca" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.54:15021/healthz/ready\": dial tcp 10.134.0.54:15021: connect: connection refused"
Sep 18 00:07:42.110862 ip-10-0-132-188 kubenswrapper[2562]: I0918 00:07:42.110819    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-test-llm-inference-service-b5dd93e6/router-gateway-2-openshift-default-57b79bd49b-2hg85"
Sep 18 00:07:42.332672 ip-10-0-132-188 kubenswrapper[2562]: I0918 00:07:42.332622    2562 prober.go:120] "Probe failed" probeType="Startup" pod="e2e-test-llm-inference-service-b5dd93e6/router-gateway-2-openshift-default-57b79bd49b-2hg85" podUID="b664114a-d5e2-49bb-b3b6-289d1ebca5ca" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.54:15021/healthz/ready\": dial tcp 10.134.0.54:15021: connect: connection refused"
Sep 18 00:07:43.336947 ip-10-0-132-188 kubenswrapper[2562]: I0918 00:07:43.336910    2562 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="e2e-test-llm-inference-service-b5dd93e6/router-gateway-2-openshift-default-57b79bd49b-2hg85"
Sep 18 00:07:43.337916 ip-10-0-132-188 kubenswrapper[2562]: I0918 00:07:43.337886    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-test-llm-inference-service-b5dd93e6/router-gateway-2-openshift-default-57b79bd49b-2hg85"
Sep 18 00:08:10.191835 ip-10-0-132-188 kubenswrapper[2562]: I0918 00:08:10.191799    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-test-llm-inference-service-b5dd93e6/router-with-refs-pd-test-kserve-router-scheduler-99c668b45mdhcc"
Sep 18 00:08:19.617903 ip-10-0-132-188 kubenswrapper[2562]: I0918 00:08:19.617862    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fwwc5_7e452bbb-7dc6-4c9d-a9b1-76400942f5b4/konnectivity-agent/0.log" limit=1024
Sep 18 00:08:39.851140 ip-10-0-132-188 kubenswrapper[2562]: I0918 00:08:39.851108    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fwwc5_7e452bbb-7dc6-4c9d-a9b1-76400942f5b4/konnectivity-agent/0.log" limit=1024
Sep 18 00:08:39.943730 ip-10-0-132-188 kubenswrapper[2562]: I0918 00:08:39.943692    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fwwc5_7e452bbb-7dc6-4c9d-a9b1-76400942f5b4/konnectivity-agent/0.log" limit=1024
Sep 18 00:08:45.139516 ip-10-0-132-188 kubenswrapper[2562]: I0918 00:08:45.139477    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fwwc5_7e452bbb-7dc6-4c9d-a9b1-76400942f5b4/konnectivity-agent/0.log" limit=1024
Sep 18 00:08:45.238201 ip-10-0-132-188 kubenswrapper[2562]: I0918 00:08:45.238146    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fwwc5_7e452bbb-7dc6-4c9d-a9b1-76400942f5b4/konnectivity-agent/0.log" limit=1024
Sep 18 00:08:45.530592 ip-10-0-132-188 kubenswrapper[2562]: I0918 00:08:45.530492    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fwwc5_7e452bbb-7dc6-4c9d-a9b1-76400942f5b4/konnectivity-agent/0.log" limit=1024
Sep 18 00:08:49.235704 ip-10-0-132-188 kubenswrapper[2562]: I0918 00:08:49.235646    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fwwc5_7e452bbb-7dc6-4c9d-a9b1-76400942f5b4/konnectivity-agent/0.log" limit=1024
Sep 18 00:08:49.483493 ip-10-0-132-188 kubenswrapper[2562]: I0918 00:08:49.483458    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fwwc5_7e452bbb-7dc6-4c9d-a9b1-76400942f5b4/konnectivity-agent/0.log" limit=1024
Sep 18 00:08:51.327031 ip-10-0-132-188 kubenswrapper[2562]: I0918 00:08:51.326992    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fwwc5_7e452bbb-7dc6-4c9d-a9b1-76400942f5b4/konnectivity-agent/0.log" limit=1024
Sep 18 00:08:51.426531 ip-10-0-132-188 kubenswrapper[2562]: I0918 00:08:51.426474    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fwwc5_7e452bbb-7dc6-4c9d-a9b1-76400942f5b4/konnectivity-agent/0.log" limit=1024
Sep 18 00:08:52.302741 ip-10-0-132-188 kubenswrapper[2562]: I0918 00:08:52.302687    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fwwc5_7e452bbb-7dc6-4c9d-a9b1-76400942f5b4/konnectivity-agent/0.log" limit=1024
Sep 18 00:08:52.403563 ip-10-0-132-188 kubenswrapper[2562]: I0918 00:08:52.403502    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fwwc5_7e452bbb-7dc6-4c9d-a9b1-76400942f5b4/konnectivity-agent/0.log" limit=1024
Sep 18 00:09:36.627669 ip-10-0-132-188 kubenswrapper[2562]: I0918 00:09:36.627628    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fwwc5_7e452bbb-7dc6-4c9d-a9b1-76400942f5b4/konnectivity-agent/0.log" limit=1024
Sep 18 00:09:47.059907 ip-10-0-132-188 kubenswrapper[2562]: I0918 00:09:47.059868    2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["e2e-test-llm-inference-service-b5dd93e6/router-with-refs-pd-test-kserve-router-scheduler-99c668b45mdhcc"]
Sep 18 00:09:47.070738 ip-10-0-132-188 kubenswrapper[2562]: I0918 00:09:47.070696    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fwwc5_7e452bbb-7dc6-4c9d-a9b1-76400942f5b4/konnectivity-agent/0.log" limit=1024
Sep 18 00:09:47.133685 ip-10-0-132-188 kubenswrapper[2562]: I0918 00:09:47.133648    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fwwc5_7e452bbb-7dc6-4c9d-a9b1-76400942f5b4/konnectivity-agent/0.log" limit=1024
Sep 18 00:09:50.191311 ip-10-0-132-188 kubenswrapper[2562]: I0918 00:09:50.191273    2562 prober.go:120] "Probe failed" probeType="Readiness" pod="e2e-test-llm-inference-service-b5dd93e6/router-with-refs-pd-test-kserve-router-scheduler-99c668b45mdhcc" podUID="37650465-7a5f-49f5-9785-527e96f329e4" containerName="main" probeResult="failure" output="service unhealthy (responded with \"NOT_SERVING\")"
Sep 18 00:09:58.911591 ip-10-0-132-188 kubenswrapper[2562]: I0918 00:09:58.911554    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fwwc5_7e452bbb-7dc6-4c9d-a9b1-76400942f5b4/konnectivity-agent/0.log" limit=1024
Sep 18 00:10:00.191928 ip-10-0-132-188 kubenswrapper[2562]: I0918 00:10:00.191882    2562 prober.go:120] "Probe failed" probeType="Readiness" pod="e2e-test-llm-inference-service-b5dd93e6/router-with-refs-pd-test-kserve-router-scheduler-99c668b45mdhcc" podUID="37650465-7a5f-49f5-9785-527e96f329e4" containerName="main" probeResult="failure" output="service unhealthy (responded with \"NOT_SERVING\")"
Sep 18 00:10:02.089715 ip-10-0-132-188 kubenswrapper[2562]: I0918 00:10:02.089650    2562 kuberuntime_container.go:873] "Killing container with a grace period" pod="e2e-test-llm-inference-service-b5dd93e6/router-with-refs-pd-test-kserve-router-scheduler-99c668b45mdhcc" podUID="37650465-7a5f-49f5-9785-527e96f329e4" containerName="main" containerID="cri-o://8ab4fa5a9a68460cfeb65f0478765a8b7660f7a5d9f93e367af851862ffa481f" gracePeriod=45
Sep 18 00:10:10.191625 ip-10-0-132-188 kubenswrapper[2562]: I0918 00:10:10.191573    2562 prober.go:120] "Probe failed" probeType="Readiness" pod="e2e-test-llm-inference-service-b5dd93e6/router-with-refs-pd-test-kserve-router-scheduler-99c668b45mdhcc" podUID="37650465-7a5f-49f5-9785-527e96f329e4" containerName="main" probeResult="failure" output="service unhealthy (responded with \"NOT_SERVING\")"
Sep 18 00:10:10.192041 ip-10-0-132-188 kubenswrapper[2562]: I0918 00:10:10.191722    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-test-llm-inference-service-b5dd93e6/router-with-refs-pd-test-kserve-router-scheduler-99c668b45mdhcc"
Sep 18 00:10:20.191292 ip-10-0-132-188 kubenswrapper[2562]: I0918 00:10:20.191239    2562 prober.go:120] "Probe failed" probeType="Readiness" pod="e2e-test-llm-inference-service-b5dd93e6/router-with-refs-pd-test-kserve-router-scheduler-99c668b45mdhcc" podUID="37650465-7a5f-49f5-9785-527e96f329e4" containerName="main" probeResult="failure" output="service unhealthy (responded with \"NOT_SERVING\")"
Sep 18 00:10:30.191773 ip-10-0-132-188 kubenswrapper[2562]: I0918 00:10:30.191733    2562 prober.go:120] "Probe failed" probeType="Readiness" pod="e2e-test-llm-inference-service-b5dd93e6/router-with-refs-pd-test-kserve-router-scheduler-99c668b45mdhcc" podUID="37650465-7a5f-49f5-9785-527e96f329e4" containerName="main" probeResult="failure" output="service unhealthy (responded with \"NOT_SERVING\")"
Sep 18 00:10:32.342347 ip-10-0-132-188 kubenswrapper[2562]: I0918 00:10:32.342318    2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-test-llm-inference-service-b5dd93e6/router-with-refs-pd-test-kserve-router-scheduler-99c668b45mdhcc"
Sep 18 00:10:32.438251 ip-10-0-132-188 kubenswrapper[2562]: I0918 00:10:32.438126    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/37650465-7a5f-49f5-9785-527e96f329e4-tls-certs\") pod \"37650465-7a5f-49f5-9785-527e96f329e4\" (UID: \"37650465-7a5f-49f5-9785-527e96f329e4\") "
Sep 18 00:10:32.438251 ip-10-0-132-188 kubenswrapper[2562]: I0918 00:10:32.438212    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-dlzzg\" (UniqueName: \"kubernetes.io/projected/37650465-7a5f-49f5-9785-527e96f329e4-kube-api-access-dlzzg\") pod \"37650465-7a5f-49f5-9785-527e96f329e4\" (UID: \"37650465-7a5f-49f5-9785-527e96f329e4\") "
Sep 18 00:10:32.440465 ip-10-0-132-188 kubenswrapper[2562]: I0918 00:10:32.440438    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/37650465-7a5f-49f5-9785-527e96f329e4-kube-api-access-dlzzg" (OuterVolumeSpecName: "kube-api-access-dlzzg") pod "37650465-7a5f-49f5-9785-527e96f329e4" (UID: "37650465-7a5f-49f5-9785-527e96f329e4"). InnerVolumeSpecName "kube-api-access-dlzzg". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 18 00:10:32.440465 ip-10-0-132-188 kubenswrapper[2562]: I0918 00:10:32.440446    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/37650465-7a5f-49f5-9785-527e96f329e4-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "37650465-7a5f-49f5-9785-527e96f329e4" (UID: "37650465-7a5f-49f5-9785-527e96f329e4"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 18 00:10:32.539559 ip-10-0-132-188 kubenswrapper[2562]: I0918 00:10:32.539518    2562 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/37650465-7a5f-49f5-9785-527e96f329e4-tls-certs\") on node \"ip-10-0-132-188.ec2.internal\" DevicePath \"\""
Sep 18 00:10:32.539559 ip-10-0-132-188 kubenswrapper[2562]: I0918 00:10:32.539550    2562 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-dlzzg\" (UniqueName: \"kubernetes.io/projected/37650465-7a5f-49f5-9785-527e96f329e4-kube-api-access-dlzzg\") on node \"ip-10-0-132-188.ec2.internal\" DevicePath \"\""
Sep 18 00:10:32.999095 ip-10-0-132-188 kubenswrapper[2562]: I0918 00:10:32.999055    2562 generic.go:356] "Generic (PLEG): container finished" podID="37650465-7a5f-49f5-9785-527e96f329e4" containerID="8ab4fa5a9a68460cfeb65f0478765a8b7660f7a5d9f93e367af851862ffa481f" exitCode=0
Sep 18 00:10:32.999295 ip-10-0-132-188 kubenswrapper[2562]: I0918 00:10:32.999122    2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-test-llm-inference-service-b5dd93e6/router-with-refs-pd-test-kserve-router-scheduler-99c668b45mdhcc"
Sep 18 00:10:32.999295 ip-10-0-132-188 kubenswrapper[2562]: I0918 00:10:32.999142    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-test-llm-inference-service-b5dd93e6/router-with-refs-pd-test-kserve-router-scheduler-99c668b45mdhcc" event={"ID":"37650465-7a5f-49f5-9785-527e96f329e4","Type":"ContainerDied","Data":"8ab4fa5a9a68460cfeb65f0478765a8b7660f7a5d9f93e367af851862ffa481f"}
Sep 18 00:10:32.999295 ip-10-0-132-188 kubenswrapper[2562]: I0918 00:10:32.999225    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-test-llm-inference-service-b5dd93e6/router-with-refs-pd-test-kserve-router-scheduler-99c668b45mdhcc" event={"ID":"37650465-7a5f-49f5-9785-527e96f329e4","Type":"ContainerDied","Data":"04fabb76375505388092fda8068d0d093df536f5b9504af772fddd3580f6ec7d"}
Sep 18 00:10:32.999295 ip-10-0-132-188 kubenswrapper[2562]: I0918 00:10:32.999249    2562 scope.go:117] "RemoveContainer" containerID="8ab4fa5a9a68460cfeb65f0478765a8b7660f7a5d9f93e367af851862ffa481f"
Sep 18 00:10:33.008062 ip-10-0-132-188 kubenswrapper[2562]: I0918 00:10:33.008041    2562 scope.go:117] "RemoveContainer" containerID="8ab4fa5a9a68460cfeb65f0478765a8b7660f7a5d9f93e367af851862ffa481f"
Sep 18 00:10:33.008408 ip-10-0-132-188 kubenswrapper[2562]: E0918 00:10:33.008390    2562 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"8ab4fa5a9a68460cfeb65f0478765a8b7660f7a5d9f93e367af851862ffa481f\": container with ID starting with 8ab4fa5a9a68460cfeb65f0478765a8b7660f7a5d9f93e367af851862ffa481f not found: ID does not exist" containerID="8ab4fa5a9a68460cfeb65f0478765a8b7660f7a5d9f93e367af851862ffa481f"
Sep 18 00:10:33.008486 ip-10-0-132-188 kubenswrapper[2562]: I0918 00:10:33.008418    2562 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"8ab4fa5a9a68460cfeb65f0478765a8b7660f7a5d9f93e367af851862ffa481f"} err="failed to get container status \"8ab4fa5a9a68460cfeb65f0478765a8b7660f7a5d9f93e367af851862ffa481f\": rpc error: code = NotFound desc = could not find container \"8ab4fa5a9a68460cfeb65f0478765a8b7660f7a5d9f93e367af851862ffa481f\": container with ID starting with 8ab4fa5a9a68460cfeb65f0478765a8b7660f7a5d9f93e367af851862ffa481f not found: ID does not exist"
Sep 18 00:10:33.019815 ip-10-0-132-188 kubenswrapper[2562]: I0918 00:10:33.019788    2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["e2e-test-llm-inference-service-b5dd93e6/router-with-refs-pd-test-kserve-router-scheduler-99c668b45mdhcc"]
Sep 18 00:10:33.023474 ip-10-0-132-188 kubenswrapper[2562]: I0918 00:10:33.023452    2562 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["e2e-test-llm-inference-service-b5dd93e6/router-with-refs-pd-test-kserve-router-scheduler-99c668b45mdhcc"]
Sep 18 00:10:33.666935 ip-10-0-132-188 kubenswrapper[2562]: I0918 00:10:33.666897    2562 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="37650465-7a5f-49f5-9785-527e96f329e4" path="/var/lib/kubelet/pods/37650465-7a5f-49f5-9785-527e96f329e4/volumes"
Sep 18 00:10:37.389125 ip-10-0-132-188 kubenswrapper[2562]: I0918 00:10:37.389090    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fwwc5_7e452bbb-7dc6-4c9d-a9b1-76400942f5b4/konnectivity-agent/0.log" limit=1024
Sep 18 00:10:37.482340 ip-10-0-132-188 kubenswrapper[2562]: I0918 00:10:37.482299    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fwwc5_7e452bbb-7dc6-4c9d-a9b1-76400942f5b4/konnectivity-agent/0.log" limit=1024
Sep 18 00:10:42.342244 ip-10-0-132-188 kubenswrapper[2562]: I0918 00:10:42.342205    2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["e2e-test-llm-inference-service-b5dd93e6/router-gateway-2-openshift-default-57b79bd49b-2hg85"]
Sep 18 00:10:42.342718 ip-10-0-132-188 kubenswrapper[2562]: I0918 00:10:42.342423    2562 kuberuntime_container.go:873] "Killing container with a grace period" pod="e2e-test-llm-inference-service-b5dd93e6/router-gateway-2-openshift-default-57b79bd49b-2hg85" podUID="b664114a-d5e2-49bb-b3b6-289d1ebca5ca" containerName="istio-proxy" containerID="cri-o://1c0996e40c975d8861a410b78ebd28eb3be78331286023c3bb0960c998712cd7" gracePeriod=30
Sep 18 00:10:42.437231 ip-10-0-132-188 kubenswrapper[2562]: I0918 00:10:42.437199    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fwwc5_7e452bbb-7dc6-4c9d-a9b1-76400942f5b4/konnectivity-agent/0.log" limit=1024
Sep 18 00:10:43.340377 ip-10-0-132-188 kubenswrapper[2562]: I0918 00:10:43.340335    2562 prober.go:120] "Probe failed" probeType="Readiness" pod="e2e-test-llm-inference-service-b5dd93e6/router-gateway-2-openshift-default-57b79bd49b-2hg85" podUID="b664114a-d5e2-49bb-b3b6-289d1ebca5ca" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Sep 18 00:10:47.602924 ip-10-0-132-188 kubenswrapper[2562]: I0918 00:10:47.602895    2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-test-llm-inference-service-b5dd93e6/router-gateway-2-openshift-default-57b79bd49b-2hg85"
Sep 18 00:10:47.680187 ip-10-0-132-188 kubenswrapper[2562]: I0918 00:10:47.680134    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/b664114a-d5e2-49bb-b3b6-289d1ebca5ca-istiod-ca-cert\") pod \"b664114a-d5e2-49bb-b3b6-289d1ebca5ca\" (UID: \"b664114a-d5e2-49bb-b3b6-289d1ebca5ca\") "
Sep 18 00:10:47.680187 ip-10-0-132-188 kubenswrapper[2562]: I0918 00:10:47.680189    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/b664114a-d5e2-49bb-b3b6-289d1ebca5ca-workload-certs\") pod \"b664114a-d5e2-49bb-b3b6-289d1ebca5ca\" (UID: \"b664114a-d5e2-49bb-b3b6-289d1ebca5ca\") "
Sep 18 00:10:47.680437 ip-10-0-132-188 kubenswrapper[2562]: I0918 00:10:47.680229    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/b664114a-d5e2-49bb-b3b6-289d1ebca5ca-workload-socket\") pod \"b664114a-d5e2-49bb-b3b6-289d1ebca5ca\" (UID: \"b664114a-d5e2-49bb-b3b6-289d1ebca5ca\") "
Sep 18 00:10:47.680437 ip-10-0-132-188 kubenswrapper[2562]: I0918 00:10:47.680252    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/b664114a-d5e2-49bb-b3b6-289d1ebca5ca-istio-data\") pod \"b664114a-d5e2-49bb-b3b6-289d1ebca5ca\" (UID: \"b664114a-d5e2-49bb-b3b6-289d1ebca5ca\") "
Sep 18 00:10:47.680437 ip-10-0-132-188 kubenswrapper[2562]: I0918 00:10:47.680277    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/b664114a-d5e2-49bb-b3b6-289d1ebca5ca-istio-podinfo\") pod \"b664114a-d5e2-49bb-b3b6-289d1ebca5ca\" (UID: \"b664114a-d5e2-49bb-b3b6-289d1ebca5ca\") "
Sep 18 00:10:47.680437 ip-10-0-132-188 kubenswrapper[2562]: I0918 00:10:47.680306    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/b664114a-d5e2-49bb-b3b6-289d1ebca5ca-istio-envoy\") pod \"b664114a-d5e2-49bb-b3b6-289d1ebca5ca\" (UID: \"b664114a-d5e2-49bb-b3b6-289d1ebca5ca\") "
Sep 18 00:10:47.680437 ip-10-0-132-188 kubenswrapper[2562]: I0918 00:10:47.680353    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/b664114a-d5e2-49bb-b3b6-289d1ebca5ca-istio-token\") pod \"b664114a-d5e2-49bb-b3b6-289d1ebca5ca\" (UID: \"b664114a-d5e2-49bb-b3b6-289d1ebca5ca\") "
Sep 18 00:10:47.680437 ip-10-0-132-188 kubenswrapper[2562]: I0918 00:10:47.680378    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-pxsfn\" (UniqueName: \"kubernetes.io/projected/b664114a-d5e2-49bb-b3b6-289d1ebca5ca-kube-api-access-pxsfn\") pod \"b664114a-d5e2-49bb-b3b6-289d1ebca5ca\" (UID: \"b664114a-d5e2-49bb-b3b6-289d1ebca5ca\") "
Sep 18 00:10:47.680437 ip-10-0-132-188 kubenswrapper[2562]: I0918 00:10:47.680435    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/b664114a-d5e2-49bb-b3b6-289d1ebca5ca-credential-socket\") pod \"b664114a-d5e2-49bb-b3b6-289d1ebca5ca\" (UID: \"b664114a-d5e2-49bb-b3b6-289d1ebca5ca\") "
Sep 18 00:10:47.680770 ip-10-0-132-188 kubenswrapper[2562]: I0918 00:10:47.680562    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b664114a-d5e2-49bb-b3b6-289d1ebca5ca-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "b664114a-d5e2-49bb-b3b6-289d1ebca5ca" (UID: "b664114a-d5e2-49bb-b3b6-289d1ebca5ca"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 18 00:10:47.680770 ip-10-0-132-188 kubenswrapper[2562]: I0918 00:10:47.680578    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/b664114a-d5e2-49bb-b3b6-289d1ebca5ca-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "b664114a-d5e2-49bb-b3b6-289d1ebca5ca" (UID: "b664114a-d5e2-49bb-b3b6-289d1ebca5ca"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 18 00:10:47.680770 ip-10-0-132-188 kubenswrapper[2562]: I0918 00:10:47.680677    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b664114a-d5e2-49bb-b3b6-289d1ebca5ca-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "b664114a-d5e2-49bb-b3b6-289d1ebca5ca" (UID: "b664114a-d5e2-49bb-b3b6-289d1ebca5ca"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 18 00:10:47.680770 ip-10-0-132-188 kubenswrapper[2562]: I0918 00:10:47.680701    2562 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/b664114a-d5e2-49bb-b3b6-289d1ebca5ca-istiod-ca-cert\") on node \"ip-10-0-132-188.ec2.internal\" DevicePath \"\""
Sep 18 00:10:47.680770 ip-10-0-132-188 kubenswrapper[2562]: I0918 00:10:47.680717    2562 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/b664114a-d5e2-49bb-b3b6-289d1ebca5ca-workload-socket\") on node \"ip-10-0-132-188.ec2.internal\" DevicePath \"\""
Sep 18 00:10:47.680984 ip-10-0-132-188 kubenswrapper[2562]: I0918 00:10:47.680794    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b664114a-d5e2-49bb-b3b6-289d1ebca5ca-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "b664114a-d5e2-49bb-b3b6-289d1ebca5ca" (UID: "b664114a-d5e2-49bb-b3b6-289d1ebca5ca"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 18 00:10:47.681100 ip-10-0-132-188 kubenswrapper[2562]: I0918 00:10:47.681002    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b664114a-d5e2-49bb-b3b6-289d1ebca5ca-istio-data" (OuterVolumeSpecName: "istio-data") pod "b664114a-d5e2-49bb-b3b6-289d1ebca5ca" (UID: "b664114a-d5e2-49bb-b3b6-289d1ebca5ca"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 18 00:10:47.683103 ip-10-0-132-188 kubenswrapper[2562]: I0918 00:10:47.683067    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/b664114a-d5e2-49bb-b3b6-289d1ebca5ca-istio-token" (OuterVolumeSpecName: "istio-token") pod "b664114a-d5e2-49bb-b3b6-289d1ebca5ca" (UID: "b664114a-d5e2-49bb-b3b6-289d1ebca5ca"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 18 00:10:47.683283 ip-10-0-132-188 kubenswrapper[2562]: I0918 00:10:47.683112    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/b664114a-d5e2-49bb-b3b6-289d1ebca5ca-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "b664114a-d5e2-49bb-b3b6-289d1ebca5ca" (UID: "b664114a-d5e2-49bb-b3b6-289d1ebca5ca"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Sep 18 00:10:47.683283 ip-10-0-132-188 kubenswrapper[2562]: I0918 00:10:47.683208    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b664114a-d5e2-49bb-b3b6-289d1ebca5ca-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "b664114a-d5e2-49bb-b3b6-289d1ebca5ca" (UID: "b664114a-d5e2-49bb-b3b6-289d1ebca5ca"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 18 00:10:47.683283 ip-10-0-132-188 kubenswrapper[2562]: I0918 00:10:47.683245    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/b664114a-d5e2-49bb-b3b6-289d1ebca5ca-kube-api-access-pxsfn" (OuterVolumeSpecName: "kube-api-access-pxsfn") pod "b664114a-d5e2-49bb-b3b6-289d1ebca5ca" (UID: "b664114a-d5e2-49bb-b3b6-289d1ebca5ca"). InnerVolumeSpecName "kube-api-access-pxsfn". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 18 00:10:47.781426 ip-10-0-132-188 kubenswrapper[2562]: I0918 00:10:47.781384    2562 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/b664114a-d5e2-49bb-b3b6-289d1ebca5ca-workload-certs\") on node \"ip-10-0-132-188.ec2.internal\" DevicePath \"\""
Sep 18 00:10:47.781426 ip-10-0-132-188 kubenswrapper[2562]: I0918 00:10:47.781418    2562 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/b664114a-d5e2-49bb-b3b6-289d1ebca5ca-istio-data\") on node \"ip-10-0-132-188.ec2.internal\" DevicePath \"\""
Sep 18 00:10:47.781426 ip-10-0-132-188 kubenswrapper[2562]: I0918 00:10:47.781426    2562 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/b664114a-d5e2-49bb-b3b6-289d1ebca5ca-istio-podinfo\") on node \"ip-10-0-132-188.ec2.internal\" DevicePath \"\""
Sep 18 00:10:47.781426 ip-10-0-132-188 kubenswrapper[2562]: I0918 00:10:47.781434    2562 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/b664114a-d5e2-49bb-b3b6-289d1ebca5ca-istio-envoy\") on node \"ip-10-0-132-188.ec2.internal\" DevicePath \"\""
Sep 18 00:10:47.781426 ip-10-0-132-188 kubenswrapper[2562]: I0918 00:10:47.781442    2562 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/b664114a-d5e2-49bb-b3b6-289d1ebca5ca-istio-token\") on node \"ip-10-0-132-188.ec2.internal\" DevicePath \"\""
Sep 18 00:10:47.781747 ip-10-0-132-188 kubenswrapper[2562]: I0918 00:10:47.781451    2562 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-pxsfn\" (UniqueName: \"kubernetes.io/projected/b664114a-d5e2-49bb-b3b6-289d1ebca5ca-kube-api-access-pxsfn\") on node \"ip-10-0-132-188.ec2.internal\" DevicePath \"\""
Sep 18 00:10:47.781747 ip-10-0-132-188 kubenswrapper[2562]: I0918 00:10:47.781461    2562 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/b664114a-d5e2-49bb-b3b6-289d1ebca5ca-credential-socket\") on node \"ip-10-0-132-188.ec2.internal\" DevicePath \"\""
Sep 18 00:10:48.072083 ip-10-0-132-188 kubenswrapper[2562]: I0918 00:10:48.071979    2562 generic.go:356] "Generic (PLEG): container finished" podID="b664114a-d5e2-49bb-b3b6-289d1ebca5ca" containerID="1c0996e40c975d8861a410b78ebd28eb3be78331286023c3bb0960c998712cd7" exitCode=0
Sep 18 00:10:48.072083 ip-10-0-132-188 kubenswrapper[2562]: I0918 00:10:48.072034    2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-test-llm-inference-service-b5dd93e6/router-gateway-2-openshift-default-57b79bd49b-2hg85"
Sep 18 00:10:48.072083 ip-10-0-132-188 kubenswrapper[2562]: I0918 00:10:48.072066    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-test-llm-inference-service-b5dd93e6/router-gateway-2-openshift-default-57b79bd49b-2hg85" event={"ID":"b664114a-d5e2-49bb-b3b6-289d1ebca5ca","Type":"ContainerDied","Data":"1c0996e40c975d8861a410b78ebd28eb3be78331286023c3bb0960c998712cd7"}
Sep 18 00:10:48.072411 ip-10-0-132-188 kubenswrapper[2562]: I0918 00:10:48.072132    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-test-llm-inference-service-b5dd93e6/router-gateway-2-openshift-default-57b79bd49b-2hg85" event={"ID":"b664114a-d5e2-49bb-b3b6-289d1ebca5ca","Type":"ContainerDied","Data":"94c8cb80d0998d98e0f9d125f93a10ff1fb7d7cd327a0b1e058fd9008e5857fe"}
Sep 18 00:10:48.072411 ip-10-0-132-188 kubenswrapper[2562]: I0918 00:10:48.072161    2562 scope.go:117] "RemoveContainer" containerID="1c0996e40c975d8861a410b78ebd28eb3be78331286023c3bb0960c998712cd7"
Sep 18 00:10:48.081933 ip-10-0-132-188 kubenswrapper[2562]: I0918 00:10:48.081914    2562 scope.go:117] "RemoveContainer" containerID="4fe2749454c9911e4200f54fe3c085a18dc0a2f86eee1ebdaf1c5211535b74cd"
Sep 18 00:10:48.089907 ip-10-0-132-188 kubenswrapper[2562]: I0918 00:10:48.089890    2562 scope.go:117] "RemoveContainer" containerID="1c0996e40c975d8861a410b78ebd28eb3be78331286023c3bb0960c998712cd7"
Sep 18 00:10:48.090191 ip-10-0-132-188 kubenswrapper[2562]: E0918 00:10:48.090151    2562 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"1c0996e40c975d8861a410b78ebd28eb3be78331286023c3bb0960c998712cd7\": container with ID starting with 1c0996e40c975d8861a410b78ebd28eb3be78331286023c3bb0960c998712cd7 not found: ID does not exist" containerID="1c0996e40c975d8861a410b78ebd28eb3be78331286023c3bb0960c998712cd7"
Sep 18 00:10:48.090283 ip-10-0-132-188 kubenswrapper[2562]: I0918 00:10:48.090229    2562 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"1c0996e40c975d8861a410b78ebd28eb3be78331286023c3bb0960c998712cd7"} err="failed to get container status \"1c0996e40c975d8861a410b78ebd28eb3be78331286023c3bb0960c998712cd7\": rpc error: code = NotFound desc = could not find container \"1c0996e40c975d8861a410b78ebd28eb3be78331286023c3bb0960c998712cd7\": container with ID starting with 1c0996e40c975d8861a410b78ebd28eb3be78331286023c3bb0960c998712cd7 not found: ID does not exist"
Sep 18 00:10:48.090283 ip-10-0-132-188 kubenswrapper[2562]: I0918 00:10:48.090248    2562 scope.go:117] "RemoveContainer" containerID="4fe2749454c9911e4200f54fe3c085a18dc0a2f86eee1ebdaf1c5211535b74cd"
Sep 18 00:10:48.090473 ip-10-0-132-188 kubenswrapper[2562]: E0918 00:10:48.090459    2562 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"4fe2749454c9911e4200f54fe3c085a18dc0a2f86eee1ebdaf1c5211535b74cd\": container with ID starting with 4fe2749454c9911e4200f54fe3c085a18dc0a2f86eee1ebdaf1c5211535b74cd not found: ID does not exist" containerID="4fe2749454c9911e4200f54fe3c085a18dc0a2f86eee1ebdaf1c5211535b74cd"
Sep 18 00:10:48.090515 ip-10-0-132-188 kubenswrapper[2562]: I0918 00:10:48.090475    2562 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"4fe2749454c9911e4200f54fe3c085a18dc0a2f86eee1ebdaf1c5211535b74cd"} err="failed to get container status \"4fe2749454c9911e4200f54fe3c085a18dc0a2f86eee1ebdaf1c5211535b74cd\": rpc error: code = NotFound desc = could not find container \"4fe2749454c9911e4200f54fe3c085a18dc0a2f86eee1ebdaf1c5211535b74cd\": container with ID starting with 4fe2749454c9911e4200f54fe3c085a18dc0a2f86eee1ebdaf1c5211535b74cd not found: ID does not exist"
Sep 18 00:10:48.094208 ip-10-0-132-188 kubenswrapper[2562]: I0918 00:10:48.094185    2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["e2e-test-llm-inference-service-b5dd93e6/router-gateway-2-openshift-default-57b79bd49b-2hg85"]
Sep 18 00:10:48.099579 ip-10-0-132-188 kubenswrapper[2562]: I0918 00:10:48.099555    2562 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["e2e-test-llm-inference-service-b5dd93e6/router-gateway-2-openshift-default-57b79bd49b-2hg85"]
Sep 18 00:10:49.666850 ip-10-0-132-188 kubenswrapper[2562]: I0918 00:10:49.666812    2562 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="b664114a-d5e2-49bb-b3b6-289d1ebca5ca" path="/var/lib/kubelet/pods/b664114a-d5e2-49bb-b3b6-289d1ebca5ca/volumes"
Sep 18 00:10:50.919048 ip-10-0-132-188 kubenswrapper[2562]: I0918 00:10:50.919009    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fwwc5_7e452bbb-7dc6-4c9d-a9b1-76400942f5b4/konnectivity-agent/0.log" limit=1024
Sep 18 00:11:03.681470 ip-10-0-132-188 kubenswrapper[2562]: I0918 00:11:03.681429    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fwwc5_7e452bbb-7dc6-4c9d-a9b1-76400942f5b4/konnectivity-agent/0.log" limit=1024
Sep 18 00:11:04.772290 ip-10-0-132-188 kubenswrapper[2562]: I0918 00:11:04.772253    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fwwc5_7e452bbb-7dc6-4c9d-a9b1-76400942f5b4/konnectivity-agent/0.log" limit=1024
Sep 18 00:11:06.551505 ip-10-0-132-188 kubenswrapper[2562]: I0918 00:11:06.551469    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fwwc5_7e452bbb-7dc6-4c9d-a9b1-76400942f5b4/konnectivity-agent/0.log" limit=1024
Sep 18 00:11:10.652518 ip-10-0-132-188 kubenswrapper[2562]: I0918 00:11:10.652477    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fwwc5_7e452bbb-7dc6-4c9d-a9b1-76400942f5b4/konnectivity-agent/0.log" limit=1024
Sep 18 00:11:20.525788 ip-10-0-132-188 kubenswrapper[2562]: I0918 00:11:20.525744    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fwwc5_7e452bbb-7dc6-4c9d-a9b1-76400942f5b4/konnectivity-agent/0.log" limit=1024
Sep 18 00:11:20.584248 ip-10-0-132-188 kubenswrapper[2562]: I0918 00:11:20.584211    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fwwc5_7e452bbb-7dc6-4c9d-a9b1-76400942f5b4/konnectivity-agent/0.log" limit=1024
Sep 18 00:11:21.382272 ip-10-0-132-188 kubenswrapper[2562]: I0918 00:11:21.382236    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fwwc5_7e452bbb-7dc6-4c9d-a9b1-76400942f5b4/konnectivity-agent/0.log" limit=1024
Sep 18 00:11:24.867273 ip-10-0-132-188 kubenswrapper[2562]: I0918 00:11:24.867229    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fwwc5_7e452bbb-7dc6-4c9d-a9b1-76400942f5b4/konnectivity-agent/0.log" limit=1024
Sep 18 00:11:33.065485 ip-10-0-132-188 kubenswrapper[2562]: I0918 00:11:33.065448    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fwwc5_7e452bbb-7dc6-4c9d-a9b1-76400942f5b4/konnectivity-agent/0.log" limit=1024
Sep 18 00:11:33.129026 ip-10-0-132-188 kubenswrapper[2562]: I0918 00:11:33.128982    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fwwc5_7e452bbb-7dc6-4c9d-a9b1-76400942f5b4/konnectivity-agent/0.log" limit=1024
Sep 18 00:11:41.134718 ip-10-0-132-188 kubenswrapper[2562]: I0918 00:11:41.134674    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fwwc5_7e452bbb-7dc6-4c9d-a9b1-76400942f5b4/konnectivity-agent/0.log" limit=1024
Sep 18 00:11:41.230965 ip-10-0-132-188 kubenswrapper[2562]: I0918 00:11:41.230925    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fwwc5_7e452bbb-7dc6-4c9d-a9b1-76400942f5b4/konnectivity-agent/0.log" limit=1024
Sep 18 00:12:10.459076 ip-10-0-132-188 kubenswrapper[2562]: I0918 00:12:10.458990    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fwwc5_7e452bbb-7dc6-4c9d-a9b1-76400942f5b4/konnectivity-agent/0.log" limit=1024
Sep 18 00:12:29.798641 ip-10-0-132-188 kubenswrapper[2562]: I0918 00:12:29.798596    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fwwc5_7e452bbb-7dc6-4c9d-a9b1-76400942f5b4/konnectivity-agent/0.log" limit=1024
Sep 18 00:12:30.946737 ip-10-0-132-188 kubenswrapper[2562]: I0918 00:12:30.946691    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fwwc5_7e452bbb-7dc6-4c9d-a9b1-76400942f5b4/konnectivity-agent/0.log" limit=1024
Sep 18 00:12:35.712135 ip-10-0-132-188 kubenswrapper[2562]: I0918 00:12:35.712095    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fwwc5_7e452bbb-7dc6-4c9d-a9b1-76400942f5b4/konnectivity-agent/0.log" limit=1024
Sep 18 00:12:39.133703 ip-10-0-132-188 kubenswrapper[2562]: I0918 00:12:39.133667    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fwwc5_7e452bbb-7dc6-4c9d-a9b1-76400942f5b4/konnectivity-agent/0.log" limit=1024
Sep 18 00:12:39.307241 ip-10-0-132-188 kubenswrapper[2562]: I0918 00:12:39.307207    2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["e2e-test-prestop-hook-22727e61/prestop-hook-test-kserve-router-scheduler-58978fbc99-455nm"]
Sep 18 00:12:39.307536 ip-10-0-132-188 kubenswrapper[2562]: I0918 00:12:39.307522    2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="37650465-7a5f-49f5-9785-527e96f329e4" containerName="main"
Sep 18 00:12:39.307586 ip-10-0-132-188 kubenswrapper[2562]: I0918 00:12:39.307539    2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="37650465-7a5f-49f5-9785-527e96f329e4" containerName="main"
Sep 18 00:12:39.307586 ip-10-0-132-188 kubenswrapper[2562]: I0918 00:12:39.307561    2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b664114a-d5e2-49bb-b3b6-289d1ebca5ca" containerName="istio-proxy"
Sep 18 00:12:39.307586 ip-10-0-132-188 kubenswrapper[2562]: I0918 00:12:39.307571    2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="b664114a-d5e2-49bb-b3b6-289d1ebca5ca" containerName="istio-proxy"
Sep 18 00:12:39.307695 ip-10-0-132-188 kubenswrapper[2562]: I0918 00:12:39.307667    2562 memory_manager.go:368] "RemoveStaleState removing state" podUID="b664114a-d5e2-49bb-b3b6-289d1ebca5ca" containerName="istio-proxy"
Sep 18 00:12:39.307695 ip-10-0-132-188 kubenswrapper[2562]: I0918 00:12:39.307681    2562 memory_manager.go:368] "RemoveStaleState removing state" podUID="b664114a-d5e2-49bb-b3b6-289d1ebca5ca" containerName="istio-proxy"
Sep 18 00:12:39.307695 ip-10-0-132-188 kubenswrapper[2562]: I0918 00:12:39.307692    2562 memory_manager.go:368] "RemoveStaleState removing state" podUID="37650465-7a5f-49f5-9785-527e96f329e4" containerName="main"
Sep 18 00:12:39.310703 ip-10-0-132-188 kubenswrapper[2562]: I0918 00:12:39.310685    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-test-prestop-hook-22727e61/prestop-hook-test-kserve-router-scheduler-58978fbc99-455nm"
Sep 18 00:12:39.313369 ip-10-0-132-188 kubenswrapper[2562]: I0918 00:12:39.313346    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-test-prestop-hook-22727e61\"/\"openshift-service-ca.crt\""
Sep 18 00:12:39.314128 ip-10-0-132-188 kubenswrapper[2562]: I0918 00:12:39.314109    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-test-prestop-hook-22727e61\"/\"prestop-hook-test-epp-sa-dockercfg-njr5m\""
Sep 18 00:12:39.314217 ip-10-0-132-188 kubenswrapper[2562]: I0918 00:12:39.314134    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-test-prestop-hook-22727e61\"/\"kube-root-ca.crt\""
Sep 18 00:12:39.314412 ip-10-0-132-188 kubenswrapper[2562]: I0918 00:12:39.314396    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-test-prestop-hook-22727e61\"/\"default-dockercfg-vn5mx\""
Sep 18 00:12:39.314509 ip-10-0-132-188 kubenswrapper[2562]: I0918 00:12:39.314490    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-test-prestop-hook-22727e61\"/\"prestop-hook-test-kserve-self-signed-certs\""
Sep 18 00:12:39.319422 ip-10-0-132-188 kubenswrapper[2562]: I0918 00:12:39.319397    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["e2e-test-prestop-hook-22727e61/prestop-hook-test-kserve-router-scheduler-58978fbc99-455nm"]
Sep 18 00:12:39.329055 ip-10-0-132-188 kubenswrapper[2562]: I0918 00:12:39.329025    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fwwc5_7e452bbb-7dc6-4c9d-a9b1-76400942f5b4/konnectivity-agent/0.log" limit=1024
Sep 18 00:12:39.456406 ip-10-0-132-188 kubenswrapper[2562]: I0918 00:12:39.456307    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-sgnw5\" (UniqueName: \"kubernetes.io/projected/04ac16c9-6e15-439a-a50a-9df3b274e911-kube-api-access-sgnw5\") pod \"prestop-hook-test-kserve-router-scheduler-58978fbc99-455nm\" (UID: \"04ac16c9-6e15-439a-a50a-9df3b274e911\") " pod="e2e-test-prestop-hook-22727e61/prestop-hook-test-kserve-router-scheduler-58978fbc99-455nm"
Sep 18 00:12:39.456561 ip-10-0-132-188 kubenswrapper[2562]: I0918 00:12:39.456413    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/04ac16c9-6e15-439a-a50a-9df3b274e911-tls-certs\") pod \"prestop-hook-test-kserve-router-scheduler-58978fbc99-455nm\" (UID: \"04ac16c9-6e15-439a-a50a-9df3b274e911\") " pod="e2e-test-prestop-hook-22727e61/prestop-hook-test-kserve-router-scheduler-58978fbc99-455nm"
Sep 18 00:12:39.557759 ip-10-0-132-188 kubenswrapper[2562]: I0918 00:12:39.557722    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-sgnw5\" (UniqueName: \"kubernetes.io/projected/04ac16c9-6e15-439a-a50a-9df3b274e911-kube-api-access-sgnw5\") pod \"prestop-hook-test-kserve-router-scheduler-58978fbc99-455nm\" (UID: \"04ac16c9-6e15-439a-a50a-9df3b274e911\") " pod="e2e-test-prestop-hook-22727e61/prestop-hook-test-kserve-router-scheduler-58978fbc99-455nm"
Sep 18 00:12:39.557964 ip-10-0-132-188 kubenswrapper[2562]: I0918 00:12:39.557801    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/04ac16c9-6e15-439a-a50a-9df3b274e911-tls-certs\") pod \"prestop-hook-test-kserve-router-scheduler-58978fbc99-455nm\" (UID: \"04ac16c9-6e15-439a-a50a-9df3b274e911\") " pod="e2e-test-prestop-hook-22727e61/prestop-hook-test-kserve-router-scheduler-58978fbc99-455nm"
Sep 18 00:12:39.561712 ip-10-0-132-188 kubenswrapper[2562]: I0918 00:12:39.561635    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/04ac16c9-6e15-439a-a50a-9df3b274e911-tls-certs\") pod \"prestop-hook-test-kserve-router-scheduler-58978fbc99-455nm\" (UID: \"04ac16c9-6e15-439a-a50a-9df3b274e911\") " pod="e2e-test-prestop-hook-22727e61/prestop-hook-test-kserve-router-scheduler-58978fbc99-455nm"
Sep 18 00:12:39.567427 ip-10-0-132-188 kubenswrapper[2562]: I0918 00:12:39.567390    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-sgnw5\" (UniqueName: \"kubernetes.io/projected/04ac16c9-6e15-439a-a50a-9df3b274e911-kube-api-access-sgnw5\") pod \"prestop-hook-test-kserve-router-scheduler-58978fbc99-455nm\" (UID: \"04ac16c9-6e15-439a-a50a-9df3b274e911\") " pod="e2e-test-prestop-hook-22727e61/prestop-hook-test-kserve-router-scheduler-58978fbc99-455nm"
Sep 18 00:12:39.622300 ip-10-0-132-188 kubenswrapper[2562]: I0918 00:12:39.622256    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-test-prestop-hook-22727e61/prestop-hook-test-kserve-router-scheduler-58978fbc99-455nm"
Sep 18 00:12:39.897213 ip-10-0-132-188 kubenswrapper[2562]: I0918 00:12:39.897183    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["e2e-test-prestop-hook-22727e61/prestop-hook-test-kserve-router-scheduler-58978fbc99-455nm"]
Sep 18 00:12:39.900024 ip-10-0-132-188 kubenswrapper[2562]: W0918 00:12:39.900000    2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod04ac16c9_6e15_439a_a50a_9df3b274e911.slice/crio-124ab79810da0b9c63ae39d41373936fba35a1ca6652075dd0260ff54bbb0606 WatchSource:0}: Error finding container 124ab79810da0b9c63ae39d41373936fba35a1ca6652075dd0260ff54bbb0606: Status 404 returned error can't find the container with id 124ab79810da0b9c63ae39d41373936fba35a1ca6652075dd0260ff54bbb0606
Sep 18 00:12:40.574468 ip-10-0-132-188 kubenswrapper[2562]: I0918 00:12:40.574434    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-test-prestop-hook-22727e61/prestop-hook-test-kserve-router-scheduler-58978fbc99-455nm" event={"ID":"04ac16c9-6e15-439a-a50a-9df3b274e911","Type":"ContainerStarted","Data":"abaa59dff556eb3fccc363acceacbeaca329047a2cab691c1a6bd60bacc5a1b7"}
Sep 18 00:12:40.575009 ip-10-0-132-188 kubenswrapper[2562]: I0918 00:12:40.574497    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-test-prestop-hook-22727e61/prestop-hook-test-kserve-router-scheduler-58978fbc99-455nm" event={"ID":"04ac16c9-6e15-439a-a50a-9df3b274e911","Type":"ContainerStarted","Data":"124ab79810da0b9c63ae39d41373936fba35a1ca6652075dd0260ff54bbb0606"}
Sep 18 00:12:40.575009 ip-10-0-132-188 kubenswrapper[2562]: I0918 00:12:40.574544    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-test-prestop-hook-22727e61/prestop-hook-test-kserve-router-scheduler-58978fbc99-455nm"
Sep 18 00:12:40.589655 ip-10-0-132-188 kubenswrapper[2562]: I0918 00:12:40.589588    2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="e2e-test-prestop-hook-22727e61/prestop-hook-test-kserve-router-scheduler-58978fbc99-455nm" podStartSLOduration=1.5895682629999999 podStartE2EDuration="1.589568263s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-18 00:12:39 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-18 00:12:40.589523116 +0000 UTC m=+3712.991419877" watchObservedRunningTime="2026-09-18 00:12:40.589568263 +0000 UTC m=+3712.991465028"
Sep 18 00:12:41.851001 ip-10-0-132-188 kubenswrapper[2562]: I0918 00:12:41.850947    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fwwc5_7e452bbb-7dc6-4c9d-a9b1-76400942f5b4/konnectivity-agent/0.log" limit=1024
Sep 18 00:12:41.951399 ip-10-0-132-188 kubenswrapper[2562]: I0918 00:12:41.951300    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fwwc5_7e452bbb-7dc6-4c9d-a9b1-76400942f5b4/konnectivity-agent/0.log" limit=1024
Sep 18 00:12:51.589967 ip-10-0-132-188 kubenswrapper[2562]: I0918 00:12:51.589919    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fwwc5_7e452bbb-7dc6-4c9d-a9b1-76400942f5b4/konnectivity-agent/0.log" limit=1024
Sep 18 00:12:57.431939 ip-10-0-132-188 kubenswrapper[2562]: I0918 00:12:57.431892    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fwwc5_7e452bbb-7dc6-4c9d-a9b1-76400942f5b4/konnectivity-agent/0.log" limit=1024
Sep 18 00:13:11.583373 ip-10-0-132-188 kubenswrapper[2562]: I0918 00:13:11.583336    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-test-prestop-hook-22727e61/prestop-hook-test-kserve-router-scheduler-58978fbc99-455nm"
Sep 18 00:13:17.507690 ip-10-0-132-188 kubenswrapper[2562]: I0918 00:13:17.507651    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fwwc5_7e452bbb-7dc6-4c9d-a9b1-76400942f5b4/konnectivity-agent/0.log" limit=1024
Sep 18 00:13:17.616079 ip-10-0-132-188 kubenswrapper[2562]: I0918 00:13:17.616040    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fwwc5_7e452bbb-7dc6-4c9d-a9b1-76400942f5b4/konnectivity-agent/0.log" limit=1024
Sep 18 00:13:23.546119 ip-10-0-132-188 kubenswrapper[2562]: I0918 00:13:23.546081    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fwwc5_7e452bbb-7dc6-4c9d-a9b1-76400942f5b4/konnectivity-agent/0.log" limit=1024
Sep 18 00:13:26.935194 ip-10-0-132-188 kubenswrapper[2562]: I0918 00:13:26.935136    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fwwc5_7e452bbb-7dc6-4c9d-a9b1-76400942f5b4/konnectivity-agent/0.log" limit=1024
Sep 18 00:13:27.062740 ip-10-0-132-188 kubenswrapper[2562]: I0918 00:13:27.062684    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fwwc5_7e452bbb-7dc6-4c9d-a9b1-76400942f5b4/konnectivity-agent/0.log" limit=1024
Sep 18 00:13:27.241473 ip-10-0-132-188 kubenswrapper[2562]: I0918 00:13:27.241378    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fwwc5_7e452bbb-7dc6-4c9d-a9b1-76400942f5b4/konnectivity-agent/0.log" limit=1024
Sep 18 00:13:27.394613 ip-10-0-132-188 kubenswrapper[2562]: I0918 00:13:27.394575    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fwwc5_7e452bbb-7dc6-4c9d-a9b1-76400942f5b4/konnectivity-agent/0.log" limit=1024
Sep 18 00:13:29.455948 ip-10-0-132-188 kubenswrapper[2562]: I0918 00:13:29.455901    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fwwc5_7e452bbb-7dc6-4c9d-a9b1-76400942f5b4/konnectivity-agent/0.log" limit=1024
Sep 18 00:13:29.553287 ip-10-0-132-188 kubenswrapper[2562]: I0918 00:13:29.553239    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fwwc5_7e452bbb-7dc6-4c9d-a9b1-76400942f5b4/konnectivity-agent/0.log" limit=1024
Sep 18 00:13:50.323928 ip-10-0-132-188 kubenswrapper[2562]: I0918 00:13:50.323824    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fwwc5_7e452bbb-7dc6-4c9d-a9b1-76400942f5b4/konnectivity-agent/0.log" limit=1024
Sep 18 00:15:10.439085 ip-10-0-132-188 kubenswrapper[2562]: I0918 00:15:10.438984    2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["e2e-test-prestop-hook-22727e61/prestop-hook-test-kserve-router-scheduler-58978fbc99-455nm"]
Sep 18 00:15:10.454029 ip-10-0-132-188 kubenswrapper[2562]: I0918 00:15:10.453992    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fwwc5_7e452bbb-7dc6-4c9d-a9b1-76400942f5b4/konnectivity-agent/0.log" limit=1024
Sep 18 00:15:10.522039 ip-10-0-132-188 kubenswrapper[2562]: I0918 00:15:10.522005    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fwwc5_7e452bbb-7dc6-4c9d-a9b1-76400942f5b4/konnectivity-agent/0.log" limit=1024
Sep 18 00:15:11.583846 ip-10-0-132-188 kubenswrapper[2562]: I0918 00:15:11.583799    2562 prober.go:120] "Probe failed" probeType="Readiness" pod="e2e-test-prestop-hook-22727e61/prestop-hook-test-kserve-router-scheduler-58978fbc99-455nm" podUID="04ac16c9-6e15-439a-a50a-9df3b274e911" containerName="main" probeResult="failure" output="service unhealthy (responded with \"NOT_SERVING\")"
Sep 18 00:15:16.434021 ip-10-0-132-188 kubenswrapper[2562]: I0918 00:15:16.433983    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fwwc5_7e452bbb-7dc6-4c9d-a9b1-76400942f5b4/konnectivity-agent/0.log" limit=1024
Sep 18 00:15:16.533585 ip-10-0-132-188 kubenswrapper[2562]: I0918 00:15:16.533554    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fwwc5_7e452bbb-7dc6-4c9d-a9b1-76400942f5b4/konnectivity-agent/0.log" limit=1024
Sep 18 00:15:21.583391 ip-10-0-132-188 kubenswrapper[2562]: I0918 00:15:21.583343    2562 prober.go:120] "Probe failed" probeType="Readiness" pod="e2e-test-prestop-hook-22727e61/prestop-hook-test-kserve-router-scheduler-58978fbc99-455nm" podUID="04ac16c9-6e15-439a-a50a-9df3b274e911" containerName="main" probeResult="failure" output="service unhealthy (responded with \"NOT_SERVING\")"
Sep 18 00:15:25.460270 ip-10-0-132-188 kubenswrapper[2562]: I0918 00:15:25.460204    2562 kuberuntime_container.go:873] "Killing container with a grace period" pod="e2e-test-prestop-hook-22727e61/prestop-hook-test-kserve-router-scheduler-58978fbc99-455nm" podUID="04ac16c9-6e15-439a-a50a-9df3b274e911" containerName="main" containerID="cri-o://abaa59dff556eb3fccc363acceacbeaca329047a2cab691c1a6bd60bacc5a1b7" gracePeriod=45
Sep 18 00:15:31.584114 ip-10-0-132-188 kubenswrapper[2562]: I0918 00:15:31.584065    2562 prober.go:120] "Probe failed" probeType="Readiness" pod="e2e-test-prestop-hook-22727e61/prestop-hook-test-kserve-router-scheduler-58978fbc99-455nm" podUID="04ac16c9-6e15-439a-a50a-9df3b274e911" containerName="main" probeResult="failure" output="service unhealthy (responded with \"NOT_SERVING\")"
Sep 18 00:15:31.584680 ip-10-0-132-188 kubenswrapper[2562]: I0918 00:15:31.584197    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-test-prestop-hook-22727e61/prestop-hook-test-kserve-router-scheduler-58978fbc99-455nm"
Sep 18 00:15:41.584387 ip-10-0-132-188 kubenswrapper[2562]: I0918 00:15:41.584344    2562 prober.go:120] "Probe failed" probeType="Readiness" pod="e2e-test-prestop-hook-22727e61/prestop-hook-test-kserve-router-scheduler-58978fbc99-455nm" podUID="04ac16c9-6e15-439a-a50a-9df3b274e911" containerName="main" probeResult="failure" output="service unhealthy (responded with \"NOT_SERVING\")"
Sep 18 00:15:51.584442 ip-10-0-132-188 kubenswrapper[2562]: I0918 00:15:51.584394    2562 prober.go:120] "Probe failed" probeType="Readiness" pod="e2e-test-prestop-hook-22727e61/prestop-hook-test-kserve-router-scheduler-58978fbc99-455nm" podUID="04ac16c9-6e15-439a-a50a-9df3b274e911" containerName="main" probeResult="failure" output="service unhealthy (responded with \"NOT_SERVING\")"
Sep 18 00:15:55.711318 ip-10-0-132-188 kubenswrapper[2562]: I0918 00:15:55.711286    2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-test-prestop-hook-22727e61/prestop-hook-test-kserve-router-scheduler-58978fbc99-455nm"
Sep 18 00:15:55.769664 ip-10-0-132-188 kubenswrapper[2562]: I0918 00:15:55.769557    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/04ac16c9-6e15-439a-a50a-9df3b274e911-tls-certs\") pod \"04ac16c9-6e15-439a-a50a-9df3b274e911\" (UID: \"04ac16c9-6e15-439a-a50a-9df3b274e911\") "
Sep 18 00:15:55.769664 ip-10-0-132-188 kubenswrapper[2562]: I0918 00:15:55.769653    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-sgnw5\" (UniqueName: \"kubernetes.io/projected/04ac16c9-6e15-439a-a50a-9df3b274e911-kube-api-access-sgnw5\") pod \"04ac16c9-6e15-439a-a50a-9df3b274e911\" (UID: \"04ac16c9-6e15-439a-a50a-9df3b274e911\") "
Sep 18 00:15:55.771952 ip-10-0-132-188 kubenswrapper[2562]: I0918 00:15:55.771920    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/04ac16c9-6e15-439a-a50a-9df3b274e911-kube-api-access-sgnw5" (OuterVolumeSpecName: "kube-api-access-sgnw5") pod "04ac16c9-6e15-439a-a50a-9df3b274e911" (UID: "04ac16c9-6e15-439a-a50a-9df3b274e911"). InnerVolumeSpecName "kube-api-access-sgnw5". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 18 00:15:55.772070 ip-10-0-132-188 kubenswrapper[2562]: I0918 00:15:55.771920    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/04ac16c9-6e15-439a-a50a-9df3b274e911-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "04ac16c9-6e15-439a-a50a-9df3b274e911" (UID: "04ac16c9-6e15-439a-a50a-9df3b274e911"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 18 00:15:55.871245 ip-10-0-132-188 kubenswrapper[2562]: I0918 00:15:55.871211    2562 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/04ac16c9-6e15-439a-a50a-9df3b274e911-tls-certs\") on node \"ip-10-0-132-188.ec2.internal\" DevicePath \"\""
Sep 18 00:15:55.871245 ip-10-0-132-188 kubenswrapper[2562]: I0918 00:15:55.871243    2562 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-sgnw5\" (UniqueName: \"kubernetes.io/projected/04ac16c9-6e15-439a-a50a-9df3b274e911-kube-api-access-sgnw5\") on node \"ip-10-0-132-188.ec2.internal\" DevicePath \"\""
Sep 18 00:15:56.464803 ip-10-0-132-188 kubenswrapper[2562]: I0918 00:15:56.464769    2562 generic.go:356] "Generic (PLEG): container finished" podID="04ac16c9-6e15-439a-a50a-9df3b274e911" containerID="abaa59dff556eb3fccc363acceacbeaca329047a2cab691c1a6bd60bacc5a1b7" exitCode=0
Sep 18 00:15:56.464996 ip-10-0-132-188 kubenswrapper[2562]: I0918 00:15:56.464819    2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-test-prestop-hook-22727e61/prestop-hook-test-kserve-router-scheduler-58978fbc99-455nm"
Sep 18 00:15:56.464996 ip-10-0-132-188 kubenswrapper[2562]: I0918 00:15:56.464839    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-test-prestop-hook-22727e61/prestop-hook-test-kserve-router-scheduler-58978fbc99-455nm" event={"ID":"04ac16c9-6e15-439a-a50a-9df3b274e911","Type":"ContainerDied","Data":"abaa59dff556eb3fccc363acceacbeaca329047a2cab691c1a6bd60bacc5a1b7"}
Sep 18 00:15:56.464996 ip-10-0-132-188 kubenswrapper[2562]: I0918 00:15:56.464892    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-test-prestop-hook-22727e61/prestop-hook-test-kserve-router-scheduler-58978fbc99-455nm" event={"ID":"04ac16c9-6e15-439a-a50a-9df3b274e911","Type":"ContainerDied","Data":"124ab79810da0b9c63ae39d41373936fba35a1ca6652075dd0260ff54bbb0606"}
Sep 18 00:15:56.464996 ip-10-0-132-188 kubenswrapper[2562]: I0918 00:15:56.464921    2562 scope.go:117] "RemoveContainer" containerID="abaa59dff556eb3fccc363acceacbeaca329047a2cab691c1a6bd60bacc5a1b7"
Sep 18 00:15:56.474098 ip-10-0-132-188 kubenswrapper[2562]: I0918 00:15:56.474074    2562 scope.go:117] "RemoveContainer" containerID="abaa59dff556eb3fccc363acceacbeaca329047a2cab691c1a6bd60bacc5a1b7"
Sep 18 00:15:56.474395 ip-10-0-132-188 kubenswrapper[2562]: E0918 00:15:56.474378    2562 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"abaa59dff556eb3fccc363acceacbeaca329047a2cab691c1a6bd60bacc5a1b7\": container with ID starting with abaa59dff556eb3fccc363acceacbeaca329047a2cab691c1a6bd60bacc5a1b7 not found: ID does not exist" containerID="abaa59dff556eb3fccc363acceacbeaca329047a2cab691c1a6bd60bacc5a1b7"
Sep 18 00:15:56.474469 ip-10-0-132-188 kubenswrapper[2562]: I0918 00:15:56.474402    2562 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"abaa59dff556eb3fccc363acceacbeaca329047a2cab691c1a6bd60bacc5a1b7"} err="failed to get container status \"abaa59dff556eb3fccc363acceacbeaca329047a2cab691c1a6bd60bacc5a1b7\": rpc error: code = NotFound desc = could not find container \"abaa59dff556eb3fccc363acceacbeaca329047a2cab691c1a6bd60bacc5a1b7\": container with ID starting with abaa59dff556eb3fccc363acceacbeaca329047a2cab691c1a6bd60bacc5a1b7 not found: ID does not exist"
Sep 18 00:15:56.485322 ip-10-0-132-188 kubenswrapper[2562]: I0918 00:15:56.485293    2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["e2e-test-prestop-hook-22727e61/prestop-hook-test-kserve-router-scheduler-58978fbc99-455nm"]
Sep 18 00:15:56.491472 ip-10-0-132-188 kubenswrapper[2562]: I0918 00:15:56.491433    2562 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["e2e-test-prestop-hook-22727e61/prestop-hook-test-kserve-router-scheduler-58978fbc99-455nm"]
Sep 18 00:15:57.666771 ip-10-0-132-188 kubenswrapper[2562]: I0918 00:15:57.666736    2562 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="04ac16c9-6e15-439a-a50a-9df3b274e911" path="/var/lib/kubelet/pods/04ac16c9-6e15-439a-a50a-9df3b274e911/volumes"
Sep 18 00:16:06.414101 ip-10-0-132-188 kubenswrapper[2562]: I0918 00:16:06.414054    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fwwc5_7e452bbb-7dc6-4c9d-a9b1-76400942f5b4/konnectivity-agent/0.log" limit=1024
Sep 18 00:16:06.513257 ip-10-0-132-188 kubenswrapper[2562]: I0918 00:16:06.513212    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fwwc5_7e452bbb-7dc6-4c9d-a9b1-76400942f5b4/konnectivity-agent/0.log" limit=1024
Sep 18 00:16:12.426993 ip-10-0-132-188 kubenswrapper[2562]: I0918 00:16:12.426955    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fwwc5_7e452bbb-7dc6-4c9d-a9b1-76400942f5b4/konnectivity-agent/0.log" limit=1024
Sep 18 00:16:14.590509 ip-10-0-132-188 kubenswrapper[2562]: I0918 00:16:14.590451    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fwwc5_7e452bbb-7dc6-4c9d-a9b1-76400942f5b4/konnectivity-agent/0.log" limit=1024
Sep 18 00:16:14.689595 ip-10-0-132-188 kubenswrapper[2562]: I0918 00:16:14.689561    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fwwc5_7e452bbb-7dc6-4c9d-a9b1-76400942f5b4/konnectivity-agent/0.log" limit=1024
Sep 18 00:16:17.919449 ip-10-0-132-188 kubenswrapper[2562]: I0918 00:16:17.919406    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fwwc5_7e452bbb-7dc6-4c9d-a9b1-76400942f5b4/konnectivity-agent/0.log" limit=1024
Sep 18 00:16:17.977315 ip-10-0-132-188 kubenswrapper[2562]: I0918 00:16:17.977261    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fwwc5_7e452bbb-7dc6-4c9d-a9b1-76400942f5b4/konnectivity-agent/0.log" limit=1024
Sep 18 00:16:20.450514 ip-10-0-132-188 kubenswrapper[2562]: I0918 00:16:20.450475    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fwwc5_7e452bbb-7dc6-4c9d-a9b1-76400942f5b4/konnectivity-agent/0.log" limit=1024
Sep 18 00:16:25.085045 ip-10-0-132-188 kubenswrapper[2562]: I0918 00:16:25.084963    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fwwc5_7e452bbb-7dc6-4c9d-a9b1-76400942f5b4/konnectivity-agent/0.log" limit=1024
Sep 18 00:16:25.280118 ip-10-0-132-188 kubenswrapper[2562]: I0918 00:16:25.280081    2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["e2e-test-llm-inference-service-47655ff5/scheduler-inline-config-test-kserve-router-scheduler-695cbzct2h"]
Sep 18 00:16:25.280457 ip-10-0-132-188 kubenswrapper[2562]: I0918 00:16:25.280443    2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="04ac16c9-6e15-439a-a50a-9df3b274e911" containerName="main"
Sep 18 00:16:25.280457 ip-10-0-132-188 kubenswrapper[2562]: I0918 00:16:25.280459    2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="04ac16c9-6e15-439a-a50a-9df3b274e911" containerName="main"
Sep 18 00:16:25.280550 ip-10-0-132-188 kubenswrapper[2562]: I0918 00:16:25.280470    2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b664114a-d5e2-49bb-b3b6-289d1ebca5ca" containerName="istio-proxy"
Sep 18 00:16:25.280550 ip-10-0-132-188 kubenswrapper[2562]: I0918 00:16:25.280476    2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="b664114a-d5e2-49bb-b3b6-289d1ebca5ca" containerName="istio-proxy"
Sep 18 00:16:25.280550 ip-10-0-132-188 kubenswrapper[2562]: I0918 00:16:25.280526    2562 memory_manager.go:368] "RemoveStaleState removing state" podUID="04ac16c9-6e15-439a-a50a-9df3b274e911" containerName="main"
Sep 18 00:16:25.284856 ip-10-0-132-188 kubenswrapper[2562]: I0918 00:16:25.284835    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-test-llm-inference-service-47655ff5/scheduler-inline-config-test-kserve-router-scheduler-695cbzct2h"
Sep 18 00:16:25.288407 ip-10-0-132-188 kubenswrapper[2562]: I0918 00:16:25.288384    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-test-llm-inference-service-47655ff5\"/\"default-dockercfg-9kgdb\""
Sep 18 00:16:25.288531 ip-10-0-132-188 kubenswrapper[2562]: I0918 00:16:25.288455    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-test-llm-inference-service-47655ff5\"/\"scheduler-inline-config-test-epp-sa-dockercfg-qwv7d\""
Sep 18 00:16:25.288603 ip-10-0-132-188 kubenswrapper[2562]: I0918 00:16:25.288384    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-test-llm-inference-service-47655ff5\"/\"openshift-service-ca.crt\""
Sep 18 00:16:25.288669 ip-10-0-132-188 kubenswrapper[2562]: I0918 00:16:25.288461    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-test-llm-inference-service-47655ff5\"/\"scheduler-inline-config-test-kserve-self-signed-certs\""
Sep 18 00:16:25.288669 ip-10-0-132-188 kubenswrapper[2562]: I0918 00:16:25.288386    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-test-llm-inference-service-47655ff5\"/\"kube-root-ca.crt\""
Sep 18 00:16:25.292115 ip-10-0-132-188 kubenswrapper[2562]: I0918 00:16:25.292091    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["e2e-test-llm-inference-service-47655ff5/scheduler-inline-config-test-kserve-router-scheduler-695cbzct2h"]
Sep 18 00:16:25.310413 ip-10-0-132-188 kubenswrapper[2562]: I0918 00:16:25.310382    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fwwc5_7e452bbb-7dc6-4c9d-a9b1-76400942f5b4/konnectivity-agent/0.log" limit=1024
Sep 18 00:16:25.324630 ip-10-0-132-188 kubenswrapper[2562]: I0918 00:16:25.324603    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/cc9c7268-4a9c-4315-8b00-6a68afbb4f3e-tls-certs\") pod \"scheduler-inline-config-test-kserve-router-scheduler-695cbzct2h\" (UID: \"cc9c7268-4a9c-4315-8b00-6a68afbb4f3e\") " pod="e2e-test-llm-inference-service-47655ff5/scheduler-inline-config-test-kserve-router-scheduler-695cbzct2h"
Sep 18 00:16:25.324722 ip-10-0-132-188 kubenswrapper[2562]: I0918 00:16:25.324647    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-sjjf2\" (UniqueName: \"kubernetes.io/projected/cc9c7268-4a9c-4315-8b00-6a68afbb4f3e-kube-api-access-sjjf2\") pod \"scheduler-inline-config-test-kserve-router-scheduler-695cbzct2h\" (UID: \"cc9c7268-4a9c-4315-8b00-6a68afbb4f3e\") " pod="e2e-test-llm-inference-service-47655ff5/scheduler-inline-config-test-kserve-router-scheduler-695cbzct2h"
Sep 18 00:16:25.425787 ip-10-0-132-188 kubenswrapper[2562]: I0918 00:16:25.425682    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/cc9c7268-4a9c-4315-8b00-6a68afbb4f3e-tls-certs\") pod \"scheduler-inline-config-test-kserve-router-scheduler-695cbzct2h\" (UID: \"cc9c7268-4a9c-4315-8b00-6a68afbb4f3e\") " pod="e2e-test-llm-inference-service-47655ff5/scheduler-inline-config-test-kserve-router-scheduler-695cbzct2h"
Sep 18 00:16:25.425787 ip-10-0-132-188 kubenswrapper[2562]: I0918 00:16:25.425736    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-sjjf2\" (UniqueName: \"kubernetes.io/projected/cc9c7268-4a9c-4315-8b00-6a68afbb4f3e-kube-api-access-sjjf2\") pod \"scheduler-inline-config-test-kserve-router-scheduler-695cbzct2h\" (UID: \"cc9c7268-4a9c-4315-8b00-6a68afbb4f3e\") " pod="e2e-test-llm-inference-service-47655ff5/scheduler-inline-config-test-kserve-router-scheduler-695cbzct2h"
Sep 18 00:16:25.428264 ip-10-0-132-188 kubenswrapper[2562]: I0918 00:16:25.428233    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/cc9c7268-4a9c-4315-8b00-6a68afbb4f3e-tls-certs\") pod \"scheduler-inline-config-test-kserve-router-scheduler-695cbzct2h\" (UID: \"cc9c7268-4a9c-4315-8b00-6a68afbb4f3e\") " pod="e2e-test-llm-inference-service-47655ff5/scheduler-inline-config-test-kserve-router-scheduler-695cbzct2h"
Sep 18 00:16:25.433223 ip-10-0-132-188 kubenswrapper[2562]: I0918 00:16:25.433198    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-sjjf2\" (UniqueName: \"kubernetes.io/projected/cc9c7268-4a9c-4315-8b00-6a68afbb4f3e-kube-api-access-sjjf2\") pod \"scheduler-inline-config-test-kserve-router-scheduler-695cbzct2h\" (UID: \"cc9c7268-4a9c-4315-8b00-6a68afbb4f3e\") " pod="e2e-test-llm-inference-service-47655ff5/scheduler-inline-config-test-kserve-router-scheduler-695cbzct2h"
Sep 18 00:16:25.597395 ip-10-0-132-188 kubenswrapper[2562]: I0918 00:16:25.597361    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-test-llm-inference-service-47655ff5/scheduler-inline-config-test-kserve-router-scheduler-695cbzct2h"
Sep 18 00:16:25.882126 ip-10-0-132-188 kubenswrapper[2562]: I0918 00:16:25.882095    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["e2e-test-llm-inference-service-47655ff5/scheduler-inline-config-test-kserve-router-scheduler-695cbzct2h"]
Sep 18 00:16:25.884882 ip-10-0-132-188 kubenswrapper[2562]: W0918 00:16:25.884863    2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podcc9c7268_4a9c_4315_8b00_6a68afbb4f3e.slice/crio-74189317cf67e098307cf2ca72ff42079575ed0625ebd1bfe604404e89fca0d7 WatchSource:0}: Error finding container 74189317cf67e098307cf2ca72ff42079575ed0625ebd1bfe604404e89fca0d7: Status 404 returned error can't find the container with id 74189317cf67e098307cf2ca72ff42079575ed0625ebd1bfe604404e89fca0d7
Sep 18 00:16:26.601706 ip-10-0-132-188 kubenswrapper[2562]: I0918 00:16:26.601668    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-test-llm-inference-service-47655ff5/scheduler-inline-config-test-kserve-router-scheduler-695cbzct2h" event={"ID":"cc9c7268-4a9c-4315-8b00-6a68afbb4f3e","Type":"ContainerStarted","Data":"72c1f788dc0f038911d21ce3b81a94afed006a2283426a9654c047201b9a4670"}
Sep 18 00:16:26.602118 ip-10-0-132-188 kubenswrapper[2562]: I0918 00:16:26.601731    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-test-llm-inference-service-47655ff5/scheduler-inline-config-test-kserve-router-scheduler-695cbzct2h"
Sep 18 00:16:26.602118 ip-10-0-132-188 kubenswrapper[2562]: I0918 00:16:26.601753    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-test-llm-inference-service-47655ff5/scheduler-inline-config-test-kserve-router-scheduler-695cbzct2h" event={"ID":"cc9c7268-4a9c-4315-8b00-6a68afbb4f3e","Type":"ContainerStarted","Data":"74189317cf67e098307cf2ca72ff42079575ed0625ebd1bfe604404e89fca0d7"}
Sep 18 00:16:26.618326 ip-10-0-132-188 kubenswrapper[2562]: I0918 00:16:26.618279    2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="e2e-test-llm-inference-service-47655ff5/scheduler-inline-config-test-kserve-router-scheduler-695cbzct2h" podStartSLOduration=1.61826099 podStartE2EDuration="1.61826099s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-18 00:16:25 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-18 00:16:26.616745046 +0000 UTC m=+3939.018641809" watchObservedRunningTime="2026-09-18 00:16:26.61826099 +0000 UTC m=+3939.020157753"
Sep 18 00:16:26.716547 ip-10-0-132-188 kubenswrapper[2562]: I0918 00:16:26.716508    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fwwc5_7e452bbb-7dc6-4c9d-a9b1-76400942f5b4/konnectivity-agent/0.log" limit=1024
Sep 18 00:16:26.825520 ip-10-0-132-188 kubenswrapper[2562]: I0918 00:16:26.825478    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fwwc5_7e452bbb-7dc6-4c9d-a9b1-76400942f5b4/konnectivity-agent/0.log" limit=1024
Sep 18 00:16:31.784324 ip-10-0-132-188 kubenswrapper[2562]: I0918 00:16:31.784219    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fwwc5_7e452bbb-7dc6-4c9d-a9b1-76400942f5b4/konnectivity-agent/0.log" limit=1024
Sep 18 00:16:38.319653 ip-10-0-132-188 kubenswrapper[2562]: I0918 00:16:38.319612    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fwwc5_7e452bbb-7dc6-4c9d-a9b1-76400942f5b4/konnectivity-agent/0.log" limit=1024
Sep 18 00:16:38.532722 ip-10-0-132-188 kubenswrapper[2562]: I0918 00:16:38.532684    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fwwc5_7e452bbb-7dc6-4c9d-a9b1-76400942f5b4/konnectivity-agent/0.log" limit=1024
Sep 18 00:16:52.169690 ip-10-0-132-188 kubenswrapper[2562]: I0918 00:16:52.169646    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fwwc5_7e452bbb-7dc6-4c9d-a9b1-76400942f5b4/konnectivity-agent/0.log" limit=1024
Sep 18 00:16:57.608849 ip-10-0-132-188 kubenswrapper[2562]: I0918 00:16:57.608813    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-test-llm-inference-service-47655ff5/scheduler-inline-config-test-kserve-router-scheduler-695cbzct2h"
Sep 18 00:16:58.928194 ip-10-0-132-188 kubenswrapper[2562]: I0918 00:16:58.928030    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fwwc5_7e452bbb-7dc6-4c9d-a9b1-76400942f5b4/konnectivity-agent/0.log" limit=1024
Sep 18 00:16:58.932411 ip-10-0-132-188 kubenswrapper[2562]: I0918 00:16:58.932380    2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["e2e-test-llm-inference-service-47655ff5/scheduler-inline-config-test-kserve-router-scheduler-695cbzct2h"]
Sep 18 00:16:59.029990 ip-10-0-132-188 kubenswrapper[2562]: I0918 00:16:59.029955    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fwwc5_7e452bbb-7dc6-4c9d-a9b1-76400942f5b4/konnectivity-agent/0.log" limit=1024
Sep 18 00:17:07.608614 ip-10-0-132-188 kubenswrapper[2562]: I0918 00:17:07.608571    2562 prober.go:120] "Probe failed" probeType="Readiness" pod="e2e-test-llm-inference-service-47655ff5/scheduler-inline-config-test-kserve-router-scheduler-695cbzct2h" podUID="cc9c7268-4a9c-4315-8b00-6a68afbb4f3e" containerName="main" probeResult="failure" output="service unhealthy (responded with \"NOT_SERVING\")"
Sep 18 00:17:13.966215 ip-10-0-132-188 kubenswrapper[2562]: I0918 00:17:13.966150    2562 kuberuntime_container.go:873] "Killing container with a grace period" pod="e2e-test-llm-inference-service-47655ff5/scheduler-inline-config-test-kserve-router-scheduler-695cbzct2h" podUID="cc9c7268-4a9c-4315-8b00-6a68afbb4f3e" containerName="main" containerID="cri-o://72c1f788dc0f038911d21ce3b81a94afed006a2283426a9654c047201b9a4670" gracePeriod=45
Sep 18 00:17:17.608283 ip-10-0-132-188 kubenswrapper[2562]: I0918 00:17:17.608236    2562 prober.go:120] "Probe failed" probeType="Readiness" pod="e2e-test-llm-inference-service-47655ff5/scheduler-inline-config-test-kserve-router-scheduler-695cbzct2h" podUID="cc9c7268-4a9c-4315-8b00-6a68afbb4f3e" containerName="main" probeResult="failure" output="service unhealthy (responded with \"NOT_SERVING\")"
Sep 18 00:17:27.608738 ip-10-0-132-188 kubenswrapper[2562]: I0918 00:17:27.608692    2562 prober.go:120] "Probe failed" probeType="Readiness" pod="e2e-test-llm-inference-service-47655ff5/scheduler-inline-config-test-kserve-router-scheduler-695cbzct2h" podUID="cc9c7268-4a9c-4315-8b00-6a68afbb4f3e" containerName="main" probeResult="failure" output="service unhealthy (responded with \"NOT_SERVING\")"
Sep 18 00:17:27.609352 ip-10-0-132-188 kubenswrapper[2562]: I0918 00:17:27.608816    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-test-llm-inference-service-47655ff5/scheduler-inline-config-test-kserve-router-scheduler-695cbzct2h"
Sep 18 00:17:33.470590 ip-10-0-132-188 kubenswrapper[2562]: I0918 00:17:33.470551    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fwwc5_7e452bbb-7dc6-4c9d-a9b1-76400942f5b4/konnectivity-agent/0.log" limit=1024
Sep 18 00:17:34.981381 ip-10-0-132-188 kubenswrapper[2562]: I0918 00:17:34.981341    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fwwc5_7e452bbb-7dc6-4c9d-a9b1-76400942f5b4/konnectivity-agent/0.log" limit=1024
Sep 18 00:17:37.608946 ip-10-0-132-188 kubenswrapper[2562]: I0918 00:17:37.608903    2562 prober.go:120] "Probe failed" probeType="Readiness" pod="e2e-test-llm-inference-service-47655ff5/scheduler-inline-config-test-kserve-router-scheduler-695cbzct2h" podUID="cc9c7268-4a9c-4315-8b00-6a68afbb4f3e" containerName="main" probeResult="failure" output="service unhealthy (responded with \"NOT_SERVING\")"
Sep 18 00:17:44.213578 ip-10-0-132-188 kubenswrapper[2562]: I0918 00:17:44.213546    2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-test-llm-inference-service-47655ff5/scheduler-inline-config-test-kserve-router-scheduler-695cbzct2h"
Sep 18 00:17:44.333429 ip-10-0-132-188 kubenswrapper[2562]: I0918 00:17:44.333400    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-sjjf2\" (UniqueName: \"kubernetes.io/projected/cc9c7268-4a9c-4315-8b00-6a68afbb4f3e-kube-api-access-sjjf2\") pod \"cc9c7268-4a9c-4315-8b00-6a68afbb4f3e\" (UID: \"cc9c7268-4a9c-4315-8b00-6a68afbb4f3e\") "
Sep 18 00:17:44.333596 ip-10-0-132-188 kubenswrapper[2562]: I0918 00:17:44.333457    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/cc9c7268-4a9c-4315-8b00-6a68afbb4f3e-tls-certs\") pod \"cc9c7268-4a9c-4315-8b00-6a68afbb4f3e\" (UID: \"cc9c7268-4a9c-4315-8b00-6a68afbb4f3e\") "
Sep 18 00:17:44.335742 ip-10-0-132-188 kubenswrapper[2562]: I0918 00:17:44.335701    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/cc9c7268-4a9c-4315-8b00-6a68afbb4f3e-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "cc9c7268-4a9c-4315-8b00-6a68afbb4f3e" (UID: "cc9c7268-4a9c-4315-8b00-6a68afbb4f3e"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 18 00:17:44.335866 ip-10-0-132-188 kubenswrapper[2562]: I0918 00:17:44.335787    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/cc9c7268-4a9c-4315-8b00-6a68afbb4f3e-kube-api-access-sjjf2" (OuterVolumeSpecName: "kube-api-access-sjjf2") pod "cc9c7268-4a9c-4315-8b00-6a68afbb4f3e" (UID: "cc9c7268-4a9c-4315-8b00-6a68afbb4f3e"). InnerVolumeSpecName "kube-api-access-sjjf2". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 18 00:17:44.434277 ip-10-0-132-188 kubenswrapper[2562]: I0918 00:17:44.434240    2562 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-sjjf2\" (UniqueName: \"kubernetes.io/projected/cc9c7268-4a9c-4315-8b00-6a68afbb4f3e-kube-api-access-sjjf2\") on node \"ip-10-0-132-188.ec2.internal\" DevicePath \"\""
Sep 18 00:17:44.434277 ip-10-0-132-188 kubenswrapper[2562]: I0918 00:17:44.434273    2562 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/cc9c7268-4a9c-4315-8b00-6a68afbb4f3e-tls-certs\") on node \"ip-10-0-132-188.ec2.internal\" DevicePath \"\""
Sep 18 00:17:44.956263 ip-10-0-132-188 kubenswrapper[2562]: I0918 00:17:44.956224    2562 generic.go:356] "Generic (PLEG): container finished" podID="cc9c7268-4a9c-4315-8b00-6a68afbb4f3e" containerID="72c1f788dc0f038911d21ce3b81a94afed006a2283426a9654c047201b9a4670" exitCode=0
Sep 18 00:17:44.956483 ip-10-0-132-188 kubenswrapper[2562]: I0918 00:17:44.956275    2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-test-llm-inference-service-47655ff5/scheduler-inline-config-test-kserve-router-scheduler-695cbzct2h"
Sep 18 00:17:44.956483 ip-10-0-132-188 kubenswrapper[2562]: I0918 00:17:44.956311    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-test-llm-inference-service-47655ff5/scheduler-inline-config-test-kserve-router-scheduler-695cbzct2h" event={"ID":"cc9c7268-4a9c-4315-8b00-6a68afbb4f3e","Type":"ContainerDied","Data":"72c1f788dc0f038911d21ce3b81a94afed006a2283426a9654c047201b9a4670"}
Sep 18 00:17:44.956483 ip-10-0-132-188 kubenswrapper[2562]: I0918 00:17:44.956374    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-test-llm-inference-service-47655ff5/scheduler-inline-config-test-kserve-router-scheduler-695cbzct2h" event={"ID":"cc9c7268-4a9c-4315-8b00-6a68afbb4f3e","Type":"ContainerDied","Data":"74189317cf67e098307cf2ca72ff42079575ed0625ebd1bfe604404e89fca0d7"}
Sep 18 00:17:44.956483 ip-10-0-132-188 kubenswrapper[2562]: I0918 00:17:44.956403    2562 scope.go:117] "RemoveContainer" containerID="72c1f788dc0f038911d21ce3b81a94afed006a2283426a9654c047201b9a4670"
Sep 18 00:17:44.970913 ip-10-0-132-188 kubenswrapper[2562]: I0918 00:17:44.970885    2562 scope.go:117] "RemoveContainer" containerID="72c1f788dc0f038911d21ce3b81a94afed006a2283426a9654c047201b9a4670"
Sep 18 00:17:44.971270 ip-10-0-132-188 kubenswrapper[2562]: E0918 00:17:44.971243    2562 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"72c1f788dc0f038911d21ce3b81a94afed006a2283426a9654c047201b9a4670\": container with ID starting with 72c1f788dc0f038911d21ce3b81a94afed006a2283426a9654c047201b9a4670 not found: ID does not exist" containerID="72c1f788dc0f038911d21ce3b81a94afed006a2283426a9654c047201b9a4670"
Sep 18 00:17:44.971383 ip-10-0-132-188 kubenswrapper[2562]: I0918 00:17:44.971276    2562 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"72c1f788dc0f038911d21ce3b81a94afed006a2283426a9654c047201b9a4670"} err="failed to get container status \"72c1f788dc0f038911d21ce3b81a94afed006a2283426a9654c047201b9a4670\": rpc error: code = NotFound desc = could not find container \"72c1f788dc0f038911d21ce3b81a94afed006a2283426a9654c047201b9a4670\": container with ID starting with 72c1f788dc0f038911d21ce3b81a94afed006a2283426a9654c047201b9a4670 not found: ID does not exist"
Sep 18 00:17:44.980056 ip-10-0-132-188 kubenswrapper[2562]: I0918 00:17:44.980027    2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["e2e-test-llm-inference-service-47655ff5/scheduler-inline-config-test-kserve-router-scheduler-695cbzct2h"]
Sep 18 00:17:44.983810 ip-10-0-132-188 kubenswrapper[2562]: I0918 00:17:44.983787    2562 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["e2e-test-llm-inference-service-47655ff5/scheduler-inline-config-test-kserve-router-scheduler-695cbzct2h"]
Sep 18 00:17:45.666449 ip-10-0-132-188 kubenswrapper[2562]: I0918 00:17:45.666416    2562 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="cc9c7268-4a9c-4315-8b00-6a68afbb4f3e" path="/var/lib/kubelet/pods/cc9c7268-4a9c-4315-8b00-6a68afbb4f3e/volumes"
Sep 18 00:17:48.597733 ip-10-0-132-188 kubenswrapper[2562]: I0918 00:17:48.597700    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fwwc5_7e452bbb-7dc6-4c9d-a9b1-76400942f5b4/konnectivity-agent/0.log" limit=1024
Sep 18 00:17:48.693231 ip-10-0-132-188 kubenswrapper[2562]: I0918 00:17:48.693196    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fwwc5_7e452bbb-7dc6-4c9d-a9b1-76400942f5b4/konnectivity-agent/0.log" limit=1024
Sep 18 00:17:54.595224 ip-10-0-132-188 kubenswrapper[2562]: I0918 00:17:54.595149    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fwwc5_7e452bbb-7dc6-4c9d-a9b1-76400942f5b4/konnectivity-agent/0.log" limit=1024
Sep 18 00:17:58.878681 ip-10-0-132-188 kubenswrapper[2562]: I0918 00:17:58.878635    2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["e2e-test-llm-inference-service-aae51b37/llmisvc-model-qwen2-5-0-5b-rout-4c1ba212-kserve-668cf8f94fq92kl"]
Sep 18 00:17:58.879377 ip-10-0-132-188 kubenswrapper[2562]: I0918 00:17:58.878969    2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="cc9c7268-4a9c-4315-8b00-6a68afbb4f3e" containerName="main"
Sep 18 00:17:58.879377 ip-10-0-132-188 kubenswrapper[2562]: I0918 00:17:58.878981    2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="cc9c7268-4a9c-4315-8b00-6a68afbb4f3e" containerName="main"
Sep 18 00:17:58.879377 ip-10-0-132-188 kubenswrapper[2562]: I0918 00:17:58.879047    2562 memory_manager.go:368] "RemoveStaleState removing state" podUID="cc9c7268-4a9c-4315-8b00-6a68afbb4f3e" containerName="main"
Sep 18 00:17:58.882097 ip-10-0-132-188 kubenswrapper[2562]: I0918 00:17:58.882068    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-test-llm-inference-service-aae51b37/llmisvc-model-qwen2-5-0-5b-rout-4c1ba212-kserve-668cf8f94fq92kl"
Sep 18 00:17:58.884607 ip-10-0-132-188 kubenswrapper[2562]: I0918 00:17:58.884580    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-test-llm-inference-service-aae51b37\"/\"default-dockercfg-zczm6\""
Sep 18 00:17:58.884607 ip-10-0-132-188 kubenswrapper[2562]: I0918 00:17:58.884599    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-test-llm-inference-service-aae51b37\"/\"openshift-service-ca.crt\""
Sep 18 00:17:58.885282 ip-10-0-132-188 kubenswrapper[2562]: I0918 00:17:58.885259    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-test-llm-inference-service-aae51b37\"/\"kube-root-ca.crt\""
Sep 18 00:17:58.885388 ip-10-0-132-188 kubenswrapper[2562]: I0918 00:17:58.885332    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-test-llm-inference-service-aae51b37\"/\"llmisv431d7e30f13554296364403a5b6365f5-kserve-self-signed-certs\""
Sep 18 00:17:58.890159 ip-10-0-132-188 kubenswrapper[2562]: I0918 00:17:58.889746    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["e2e-test-llm-inference-service-aae51b37/llmisvc-model-qwen2-5-0-5b-rout-4c1ba212-kserve-668cf8f94fq92kl"]
Sep 18 00:17:58.912983 ip-10-0-132-188 kubenswrapper[2562]: I0918 00:17:58.912947    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fwwc5_7e452bbb-7dc6-4c9d-a9b1-76400942f5b4/konnectivity-agent/0.log" limit=1024
Sep 18 00:17:58.955964 ip-10-0-132-188 kubenswrapper[2562]: I0918 00:17:58.955922    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qqhtr\" (UniqueName: \"kubernetes.io/projected/59316f5c-40e8-4069-a8ee-d91bdfb0aad6-kube-api-access-qqhtr\") pod \"llmisvc-model-qwen2-5-0-5b-rout-4c1ba212-kserve-668cf8f94fq92kl\" (UID: \"59316f5c-40e8-4069-a8ee-d91bdfb0aad6\") " pod="e2e-test-llm-inference-service-aae51b37/llmisvc-model-qwen2-5-0-5b-rout-4c1ba212-kserve-668cf8f94fq92kl"
Sep 18 00:17:58.956241 ip-10-0-132-188 kubenswrapper[2562]: I0918 00:17:58.955985    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/59316f5c-40e8-4069-a8ee-d91bdfb0aad6-tls-certs\") pod \"llmisvc-model-qwen2-5-0-5b-rout-4c1ba212-kserve-668cf8f94fq92kl\" (UID: \"59316f5c-40e8-4069-a8ee-d91bdfb0aad6\") " pod="e2e-test-llm-inference-service-aae51b37/llmisvc-model-qwen2-5-0-5b-rout-4c1ba212-kserve-668cf8f94fq92kl"
Sep 18 00:17:58.956241 ip-10-0-132-188 kubenswrapper[2562]: I0918 00:17:58.956007    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/59316f5c-40e8-4069-a8ee-d91bdfb0aad6-dshm\") pod \"llmisvc-model-qwen2-5-0-5b-rout-4c1ba212-kserve-668cf8f94fq92kl\" (UID: \"59316f5c-40e8-4069-a8ee-d91bdfb0aad6\") " pod="e2e-test-llm-inference-service-aae51b37/llmisvc-model-qwen2-5-0-5b-rout-4c1ba212-kserve-668cf8f94fq92kl"
Sep 18 00:17:58.956241 ip-10-0-132-188 kubenswrapper[2562]: I0918 00:17:58.956063    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/59316f5c-40e8-4069-a8ee-d91bdfb0aad6-tmp-dir\") pod \"llmisvc-model-qwen2-5-0-5b-rout-4c1ba212-kserve-668cf8f94fq92kl\" (UID: \"59316f5c-40e8-4069-a8ee-d91bdfb0aad6\") " pod="e2e-test-llm-inference-service-aae51b37/llmisvc-model-qwen2-5-0-5b-rout-4c1ba212-kserve-668cf8f94fq92kl"
Sep 18 00:17:58.956241 ip-10-0-132-188 kubenswrapper[2562]: I0918 00:17:58.956137    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/59316f5c-40e8-4069-a8ee-d91bdfb0aad6-home\") pod \"llmisvc-model-qwen2-5-0-5b-rout-4c1ba212-kserve-668cf8f94fq92kl\" (UID: \"59316f5c-40e8-4069-a8ee-d91bdfb0aad6\") " pod="e2e-test-llm-inference-service-aae51b37/llmisvc-model-qwen2-5-0-5b-rout-4c1ba212-kserve-668cf8f94fq92kl"
Sep 18 00:17:58.956241 ip-10-0-132-188 kubenswrapper[2562]: I0918 00:17:58.956196    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/59316f5c-40e8-4069-a8ee-d91bdfb0aad6-model-cache\") pod \"llmisvc-model-qwen2-5-0-5b-rout-4c1ba212-kserve-668cf8f94fq92kl\" (UID: \"59316f5c-40e8-4069-a8ee-d91bdfb0aad6\") " pod="e2e-test-llm-inference-service-aae51b37/llmisvc-model-qwen2-5-0-5b-rout-4c1ba212-kserve-668cf8f94fq92kl"
Sep 18 00:17:59.056990 ip-10-0-132-188 kubenswrapper[2562]: I0918 00:17:59.056952    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qqhtr\" (UniqueName: \"kubernetes.io/projected/59316f5c-40e8-4069-a8ee-d91bdfb0aad6-kube-api-access-qqhtr\") pod \"llmisvc-model-qwen2-5-0-5b-rout-4c1ba212-kserve-668cf8f94fq92kl\" (UID: \"59316f5c-40e8-4069-a8ee-d91bdfb0aad6\") " pod="e2e-test-llm-inference-service-aae51b37/llmisvc-model-qwen2-5-0-5b-rout-4c1ba212-kserve-668cf8f94fq92kl"
Sep 18 00:17:59.057228 ip-10-0-132-188 kubenswrapper[2562]: I0918 00:17:59.057018    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/59316f5c-40e8-4069-a8ee-d91bdfb0aad6-tls-certs\") pod \"llmisvc-model-qwen2-5-0-5b-rout-4c1ba212-kserve-668cf8f94fq92kl\" (UID: \"59316f5c-40e8-4069-a8ee-d91bdfb0aad6\") " pod="e2e-test-llm-inference-service-aae51b37/llmisvc-model-qwen2-5-0-5b-rout-4c1ba212-kserve-668cf8f94fq92kl"
Sep 18 00:17:59.057228 ip-10-0-132-188 kubenswrapper[2562]: I0918 00:17:59.057040    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/59316f5c-40e8-4069-a8ee-d91bdfb0aad6-dshm\") pod \"llmisvc-model-qwen2-5-0-5b-rout-4c1ba212-kserve-668cf8f94fq92kl\" (UID: \"59316f5c-40e8-4069-a8ee-d91bdfb0aad6\") " pod="e2e-test-llm-inference-service-aae51b37/llmisvc-model-qwen2-5-0-5b-rout-4c1ba212-kserve-668cf8f94fq92kl"
Sep 18 00:17:59.057228 ip-10-0-132-188 kubenswrapper[2562]: I0918 00:17:59.057061    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/59316f5c-40e8-4069-a8ee-d91bdfb0aad6-tmp-dir\") pod \"llmisvc-model-qwen2-5-0-5b-rout-4c1ba212-kserve-668cf8f94fq92kl\" (UID: \"59316f5c-40e8-4069-a8ee-d91bdfb0aad6\") " pod="e2e-test-llm-inference-service-aae51b37/llmisvc-model-qwen2-5-0-5b-rout-4c1ba212-kserve-668cf8f94fq92kl"
Sep 18 00:17:59.057228 ip-10-0-132-188 kubenswrapper[2562]: I0918 00:17:59.057100    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/59316f5c-40e8-4069-a8ee-d91bdfb0aad6-home\") pod \"llmisvc-model-qwen2-5-0-5b-rout-4c1ba212-kserve-668cf8f94fq92kl\" (UID: \"59316f5c-40e8-4069-a8ee-d91bdfb0aad6\") " pod="e2e-test-llm-inference-service-aae51b37/llmisvc-model-qwen2-5-0-5b-rout-4c1ba212-kserve-668cf8f94fq92kl"
Sep 18 00:17:59.057228 ip-10-0-132-188 kubenswrapper[2562]: I0918 00:17:59.057124    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/59316f5c-40e8-4069-a8ee-d91bdfb0aad6-model-cache\") pod \"llmisvc-model-qwen2-5-0-5b-rout-4c1ba212-kserve-668cf8f94fq92kl\" (UID: \"59316f5c-40e8-4069-a8ee-d91bdfb0aad6\") " pod="e2e-test-llm-inference-service-aae51b37/llmisvc-model-qwen2-5-0-5b-rout-4c1ba212-kserve-668cf8f94fq92kl"
Sep 18 00:17:59.057630 ip-10-0-132-188 kubenswrapper[2562]: I0918 00:17:59.057597    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/59316f5c-40e8-4069-a8ee-d91bdfb0aad6-tmp-dir\") pod \"llmisvc-model-qwen2-5-0-5b-rout-4c1ba212-kserve-668cf8f94fq92kl\" (UID: \"59316f5c-40e8-4069-a8ee-d91bdfb0aad6\") " pod="e2e-test-llm-inference-service-aae51b37/llmisvc-model-qwen2-5-0-5b-rout-4c1ba212-kserve-668cf8f94fq92kl"
Sep 18 00:17:59.057752 ip-10-0-132-188 kubenswrapper[2562]: I0918 00:17:59.057631    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/59316f5c-40e8-4069-a8ee-d91bdfb0aad6-model-cache\") pod \"llmisvc-model-qwen2-5-0-5b-rout-4c1ba212-kserve-668cf8f94fq92kl\" (UID: \"59316f5c-40e8-4069-a8ee-d91bdfb0aad6\") " pod="e2e-test-llm-inference-service-aae51b37/llmisvc-model-qwen2-5-0-5b-rout-4c1ba212-kserve-668cf8f94fq92kl"
Sep 18 00:17:59.057752 ip-10-0-132-188 kubenswrapper[2562]: I0918 00:17:59.057600    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/59316f5c-40e8-4069-a8ee-d91bdfb0aad6-home\") pod \"llmisvc-model-qwen2-5-0-5b-rout-4c1ba212-kserve-668cf8f94fq92kl\" (UID: \"59316f5c-40e8-4069-a8ee-d91bdfb0aad6\") " pod="e2e-test-llm-inference-service-aae51b37/llmisvc-model-qwen2-5-0-5b-rout-4c1ba212-kserve-668cf8f94fq92kl"
Sep 18 00:17:59.059525 ip-10-0-132-188 kubenswrapper[2562]: I0918 00:17:59.059500    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/59316f5c-40e8-4069-a8ee-d91bdfb0aad6-dshm\") pod \"llmisvc-model-qwen2-5-0-5b-rout-4c1ba212-kserve-668cf8f94fq92kl\" (UID: \"59316f5c-40e8-4069-a8ee-d91bdfb0aad6\") " pod="e2e-test-llm-inference-service-aae51b37/llmisvc-model-qwen2-5-0-5b-rout-4c1ba212-kserve-668cf8f94fq92kl"
Sep 18 00:17:59.059789 ip-10-0-132-188 kubenswrapper[2562]: I0918 00:17:59.059772    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/59316f5c-40e8-4069-a8ee-d91bdfb0aad6-tls-certs\") pod \"llmisvc-model-qwen2-5-0-5b-rout-4c1ba212-kserve-668cf8f94fq92kl\" (UID: \"59316f5c-40e8-4069-a8ee-d91bdfb0aad6\") " pod="e2e-test-llm-inference-service-aae51b37/llmisvc-model-qwen2-5-0-5b-rout-4c1ba212-kserve-668cf8f94fq92kl"
Sep 18 00:17:59.065020 ip-10-0-132-188 kubenswrapper[2562]: I0918 00:17:59.064996    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-qqhtr\" (UniqueName: \"kubernetes.io/projected/59316f5c-40e8-4069-a8ee-d91bdfb0aad6-kube-api-access-qqhtr\") pod \"llmisvc-model-qwen2-5-0-5b-rout-4c1ba212-kserve-668cf8f94fq92kl\" (UID: \"59316f5c-40e8-4069-a8ee-d91bdfb0aad6\") " pod="e2e-test-llm-inference-service-aae51b37/llmisvc-model-qwen2-5-0-5b-rout-4c1ba212-kserve-668cf8f94fq92kl"
Sep 18 00:17:59.170685 ip-10-0-132-188 kubenswrapper[2562]: I0918 00:17:59.170591    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fwwc5_7e452bbb-7dc6-4c9d-a9b1-76400942f5b4/konnectivity-agent/0.log" limit=1024
Sep 18 00:17:59.195400 ip-10-0-132-188 kubenswrapper[2562]: I0918 00:17:59.195361    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-test-llm-inference-service-aae51b37/llmisvc-model-qwen2-5-0-5b-rout-4c1ba212-kserve-668cf8f94fq92kl"
Sep 18 00:17:59.471651 ip-10-0-132-188 kubenswrapper[2562]: I0918 00:17:59.471620    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["e2e-test-llm-inference-service-aae51b37/llmisvc-model-qwen2-5-0-5b-rout-4c1ba212-kserve-668cf8f94fq92kl"]
Sep 18 00:17:59.473656 ip-10-0-132-188 kubenswrapper[2562]: W0918 00:17:59.473626    2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod59316f5c_40e8_4069_a8ee_d91bdfb0aad6.slice/crio-d6523c791f79354460cc91c9552a303dc10f384eb08e9c9b1edfbcef24842899 WatchSource:0}: Error finding container d6523c791f79354460cc91c9552a303dc10f384eb08e9c9b1edfbcef24842899: Status 404 returned error can't find the container with id d6523c791f79354460cc91c9552a303dc10f384eb08e9c9b1edfbcef24842899
Sep 18 00:18:00.025675 ip-10-0-132-188 kubenswrapper[2562]: I0918 00:18:00.025638    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-test-llm-inference-service-aae51b37/llmisvc-model-qwen2-5-0-5b-rout-4c1ba212-kserve-668cf8f94fq92kl" event={"ID":"59316f5c-40e8-4069-a8ee-d91bdfb0aad6","Type":"ContainerStarted","Data":"f922c8a060193fc49c992528eec5917b8ab4ec70e8d2f1f03dbcb1bd5dbdf574"}
Sep 18 00:18:00.026084 ip-10-0-132-188 kubenswrapper[2562]: I0918 00:18:00.025699    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-test-llm-inference-service-aae51b37/llmisvc-model-qwen2-5-0-5b-rout-4c1ba212-kserve-668cf8f94fq92kl" event={"ID":"59316f5c-40e8-4069-a8ee-d91bdfb0aad6","Type":"ContainerStarted","Data":"d6523c791f79354460cc91c9552a303dc10f384eb08e9c9b1edfbcef24842899"}
Sep 18 00:18:00.040977 ip-10-0-132-188 kubenswrapper[2562]: I0918 00:18:00.040924    2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="e2e-test-llm-inference-service-aae51b37/llmisvc-model-qwen2-5-0-5b-rout-4c1ba212-kserve-668cf8f94fq92kl" podStartSLOduration=2.040906745 podStartE2EDuration="2.040906745s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-18 00:17:58 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-18 00:18:00.040051277 +0000 UTC m=+4032.441948039" watchObservedRunningTime="2026-09-18 00:18:00.040906745 +0000 UTC m=+4032.442803509"
Sep 18 00:18:00.195677 ip-10-0-132-188 kubenswrapper[2562]: I0918 00:18:00.195632    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-test-llm-inference-service-aae51b37/llmisvc-model-qwen2-5-0-5b-rout-4c1ba212-kserve-668cf8f94fq92kl"
Sep 18 00:18:00.746268 ip-10-0-132-188 kubenswrapper[2562]: I0918 00:18:00.746218    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fwwc5_7e452bbb-7dc6-4c9d-a9b1-76400942f5b4/konnectivity-agent/0.log" limit=1024
Sep 18 00:18:00.846948 ip-10-0-132-188 kubenswrapper[2562]: I0918 00:18:00.846907    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fwwc5_7e452bbb-7dc6-4c9d-a9b1-76400942f5b4/konnectivity-agent/0.log" limit=1024
Sep 18 00:18:09.196022 ip-10-0-132-188 kubenswrapper[2562]: I0918 00:18:09.195985    2562 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="e2e-test-llm-inference-service-aae51b37/llmisvc-model-qwen2-5-0-5b-rout-4c1ba212-kserve-668cf8f94fq92kl"
Sep 18 00:18:09.208699 ip-10-0-132-188 kubenswrapper[2562]: I0918 00:18:09.208669    2562 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="e2e-test-llm-inference-service-aae51b37/llmisvc-model-qwen2-5-0-5b-rout-4c1ba212-kserve-668cf8f94fq92kl"
Sep 18 00:18:10.080836 ip-10-0-132-188 kubenswrapper[2562]: I0918 00:18:10.080803    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-test-llm-inference-service-aae51b37/llmisvc-model-qwen2-5-0-5b-rout-4c1ba212-kserve-668cf8f94fq92kl"
Sep 18 00:18:19.411879 ip-10-0-132-188 kubenswrapper[2562]: I0918 00:18:19.411841    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fwwc5_7e452bbb-7dc6-4c9d-a9b1-76400942f5b4/konnectivity-agent/0.log" limit=1024
Sep 18 00:18:20.109602 ip-10-0-132-188 kubenswrapper[2562]: I0918 00:18:20.109568    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fwwc5_7e452bbb-7dc6-4c9d-a9b1-76400942f5b4/konnectivity-agent/0.log" limit=1024
Sep 18 00:18:20.209149 ip-10-0-132-188 kubenswrapper[2562]: I0918 00:18:20.209117    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fwwc5_7e452bbb-7dc6-4c9d-a9b1-76400942f5b4/konnectivity-agent/0.log" limit=1024
Sep 18 00:18:26.089826 ip-10-0-132-188 kubenswrapper[2562]: I0918 00:18:26.089787    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fwwc5_7e452bbb-7dc6-4c9d-a9b1-76400942f5b4/konnectivity-agent/0.log" limit=1024
Sep 18 00:18:28.606714 ip-10-0-132-188 kubenswrapper[2562]: I0918 00:18:28.606669    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fwwc5_7e452bbb-7dc6-4c9d-a9b1-76400942f5b4/konnectivity-agent/0.log" limit=1024
Sep 18 00:18:28.819122 ip-10-0-132-188 kubenswrapper[2562]: I0918 00:18:28.819085    2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["e2e-test-llm-inference-service-15d6126c/scheduler-configmap-ref-test-kserve-router-scheduler-566c9pklzz"]
Sep 18 00:18:28.822394 ip-10-0-132-188 kubenswrapper[2562]: I0918 00:18:28.822366    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-test-llm-inference-service-15d6126c/scheduler-configmap-ref-test-kserve-router-scheduler-566c9pklzz"
Sep 18 00:18:28.825068 ip-10-0-132-188 kubenswrapper[2562]: I0918 00:18:28.825045    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-test-llm-inference-service-15d6126c\"/\"kube-root-ca.crt\""
Sep 18 00:18:28.825794 ip-10-0-132-188 kubenswrapper[2562]: I0918 00:18:28.825773    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-test-llm-inference-service-15d6126c\"/\"openshift-service-ca.crt\""
Sep 18 00:18:28.825956 ip-10-0-132-188 kubenswrapper[2562]: I0918 00:18:28.825935    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-test-llm-inference-service-15d6126c\"/\"scheduler-configmap-ref-test-kserve-self-signed-certs\""
Sep 18 00:18:28.826192 ip-10-0-132-188 kubenswrapper[2562]: I0918 00:18:28.826162    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-test-llm-inference-service-15d6126c\"/\"default-dockercfg-tr5gd\""
Sep 18 00:18:28.826309 ip-10-0-132-188 kubenswrapper[2562]: I0918 00:18:28.826289    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-test-llm-inference-service-15d6126c\"/\"scheduler-configmap-ref-test-epp-sa-dockercfg-mh4p9\""
Sep 18 00:18:28.831772 ip-10-0-132-188 kubenswrapper[2562]: I0918 00:18:28.831751    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["e2e-test-llm-inference-service-15d6126c/scheduler-configmap-ref-test-kserve-router-scheduler-566c9pklzz"]
Sep 18 00:18:28.841723 ip-10-0-132-188 kubenswrapper[2562]: I0918 00:18:28.841697    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fwwc5_7e452bbb-7dc6-4c9d-a9b1-76400942f5b4/konnectivity-agent/0.log" limit=1024
Sep 18 00:18:28.928454 ip-10-0-132-188 kubenswrapper[2562]: I0918 00:18:28.928344    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-f9np7\" (UniqueName: \"kubernetes.io/projected/673b18a9-0d5d-429e-b821-4b0434978f3b-kube-api-access-f9np7\") pod \"scheduler-configmap-ref-test-kserve-router-scheduler-566c9pklzz\" (UID: \"673b18a9-0d5d-429e-b821-4b0434978f3b\") " pod="e2e-test-llm-inference-service-15d6126c/scheduler-configmap-ref-test-kserve-router-scheduler-566c9pklzz"
Sep 18 00:18:28.928454 ip-10-0-132-188 kubenswrapper[2562]: I0918 00:18:28.928406    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/673b18a9-0d5d-429e-b821-4b0434978f3b-tls-certs\") pod \"scheduler-configmap-ref-test-kserve-router-scheduler-566c9pklzz\" (UID: \"673b18a9-0d5d-429e-b821-4b0434978f3b\") " pod="e2e-test-llm-inference-service-15d6126c/scheduler-configmap-ref-test-kserve-router-scheduler-566c9pklzz"
Sep 18 00:18:29.029640 ip-10-0-132-188 kubenswrapper[2562]: I0918 00:18:29.029586    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-f9np7\" (UniqueName: \"kubernetes.io/projected/673b18a9-0d5d-429e-b821-4b0434978f3b-kube-api-access-f9np7\") pod \"scheduler-configmap-ref-test-kserve-router-scheduler-566c9pklzz\" (UID: \"673b18a9-0d5d-429e-b821-4b0434978f3b\") " pod="e2e-test-llm-inference-service-15d6126c/scheduler-configmap-ref-test-kserve-router-scheduler-566c9pklzz"
Sep 18 00:18:29.029640 ip-10-0-132-188 kubenswrapper[2562]: I0918 00:18:29.029648    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/673b18a9-0d5d-429e-b821-4b0434978f3b-tls-certs\") pod \"scheduler-configmap-ref-test-kserve-router-scheduler-566c9pklzz\" (UID: \"673b18a9-0d5d-429e-b821-4b0434978f3b\") " pod="e2e-test-llm-inference-service-15d6126c/scheduler-configmap-ref-test-kserve-router-scheduler-566c9pklzz"
Sep 18 00:18:29.032291 ip-10-0-132-188 kubenswrapper[2562]: I0918 00:18:29.032263    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/673b18a9-0d5d-429e-b821-4b0434978f3b-tls-certs\") pod \"scheduler-configmap-ref-test-kserve-router-scheduler-566c9pklzz\" (UID: \"673b18a9-0d5d-429e-b821-4b0434978f3b\") " pod="e2e-test-llm-inference-service-15d6126c/scheduler-configmap-ref-test-kserve-router-scheduler-566c9pklzz"
Sep 18 00:18:29.038877 ip-10-0-132-188 kubenswrapper[2562]: I0918 00:18:29.038849    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-f9np7\" (UniqueName: \"kubernetes.io/projected/673b18a9-0d5d-429e-b821-4b0434978f3b-kube-api-access-f9np7\") pod \"scheduler-configmap-ref-test-kserve-router-scheduler-566c9pklzz\" (UID: \"673b18a9-0d5d-429e-b821-4b0434978f3b\") " pod="e2e-test-llm-inference-service-15d6126c/scheduler-configmap-ref-test-kserve-router-scheduler-566c9pklzz"
Sep 18 00:18:29.133990 ip-10-0-132-188 kubenswrapper[2562]: I0918 00:18:29.133950    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-test-llm-inference-service-15d6126c/scheduler-configmap-ref-test-kserve-router-scheduler-566c9pklzz"
Sep 18 00:18:29.412904 ip-10-0-132-188 kubenswrapper[2562]: I0918 00:18:29.412868    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["e2e-test-llm-inference-service-15d6126c/scheduler-configmap-ref-test-kserve-router-scheduler-566c9pklzz"]
Sep 18 00:18:29.417491 ip-10-0-132-188 kubenswrapper[2562]: W0918 00:18:29.417454    2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod673b18a9_0d5d_429e_b821_4b0434978f3b.slice/crio-16b2866a19483d194869e41f1c416e99ff99460e086f85b5cccf2147c6f7a274 WatchSource:0}: Error finding container 16b2866a19483d194869e41f1c416e99ff99460e086f85b5cccf2147c6f7a274: Status 404 returned error can't find the container with id 16b2866a19483d194869e41f1c416e99ff99460e086f85b5cccf2147c6f7a274
Sep 18 00:18:30.170188 ip-10-0-132-188 kubenswrapper[2562]: I0918 00:18:30.170133    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-test-llm-inference-service-15d6126c/scheduler-configmap-ref-test-kserve-router-scheduler-566c9pklzz" event={"ID":"673b18a9-0d5d-429e-b821-4b0434978f3b","Type":"ContainerStarted","Data":"620518c74ca975d7c3ede510aaafae183afc4ee4b8fd2182f5ad75551e8d56bc"}
Sep 18 00:18:30.170678 ip-10-0-132-188 kubenswrapper[2562]: I0918 00:18:30.170258    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-test-llm-inference-service-15d6126c/scheduler-configmap-ref-test-kserve-router-scheduler-566c9pklzz" event={"ID":"673b18a9-0d5d-429e-b821-4b0434978f3b","Type":"ContainerStarted","Data":"16b2866a19483d194869e41f1c416e99ff99460e086f85b5cccf2147c6f7a274"}
Sep 18 00:18:30.170678 ip-10-0-132-188 kubenswrapper[2562]: I0918 00:18:30.170337    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-test-llm-inference-service-15d6126c/scheduler-configmap-ref-test-kserve-router-scheduler-566c9pklzz"
Sep 18 00:18:30.186838 ip-10-0-132-188 kubenswrapper[2562]: I0918 00:18:30.186786    2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="e2e-test-llm-inference-service-15d6126c/scheduler-configmap-ref-test-kserve-router-scheduler-566c9pklzz" podStartSLOduration=2.186771572 podStartE2EDuration="2.186771572s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-18 00:18:28 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-18 00:18:30.185447416 +0000 UTC m=+4062.587344180" watchObservedRunningTime="2026-09-18 00:18:30.186771572 +0000 UTC m=+4062.588668336"
Sep 18 00:18:31.885308 ip-10-0-132-188 kubenswrapper[2562]: I0918 00:18:31.885258    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fwwc5_7e452bbb-7dc6-4c9d-a9b1-76400942f5b4/konnectivity-agent/0.log" limit=1024
Sep 18 00:18:31.943224 ip-10-0-132-188 kubenswrapper[2562]: I0918 00:18:31.943186    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fwwc5_7e452bbb-7dc6-4c9d-a9b1-76400942f5b4/konnectivity-agent/0.log" limit=1024
Sep 18 00:18:42.732393 ip-10-0-132-188 kubenswrapper[2562]: I0918 00:18:42.732353    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fwwc5_7e452bbb-7dc6-4c9d-a9b1-76400942f5b4/konnectivity-agent/0.log" limit=1024
Sep 18 00:18:46.803617 ip-10-0-132-188 kubenswrapper[2562]: I0918 00:18:46.803574    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fwwc5_7e452bbb-7dc6-4c9d-a9b1-76400942f5b4/konnectivity-agent/0.log" limit=1024
Sep 18 00:18:46.807492 ip-10-0-132-188 kubenswrapper[2562]: I0918 00:18:46.807459    2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["e2e-test-llm-inference-service-aae51b37/llmisvc-model-qwen2-5-0-5b-rout-4c1ba212-kserve-668cf8f94fq92kl"]
Sep 18 00:18:46.816797 ip-10-0-132-188 kubenswrapper[2562]: E0918 00:18:46.816761    2562 log.go:32] "ExecSync cmd from runtime service failed" err=<
Sep 18 00:18:46.816797 ip-10-0-132-188 kubenswrapper[2562]:         rpc error: code = Unknown desc = command error: executable file `/bin/sleep` not found: No such file or directory
Sep 18 00:18:46.816797 ip-10-0-132-188 kubenswrapper[2562]:         , stdout: , stderr: , exit code -1
Sep 18 00:18:46.816797 ip-10-0-132-188 kubenswrapper[2562]:  > containerID="f922c8a060193fc49c992528eec5917b8ab4ec70e8d2f1f03dbcb1bd5dbdf574" cmd=["/bin/sleep","15"]
Sep 18 00:18:46.817117 ip-10-0-132-188 kubenswrapper[2562]: I0918 00:18:46.816813    2562 handlers.go:82] "Exec lifecycle hook for Container in Pod failed" execCommand=["/bin/sleep","15"] containerName="main" pod="e2e-test-llm-inference-service-aae51b37/llmisvc-model-qwen2-5-0-5b-rout-4c1ba212-kserve-668cf8f94fq92kl" message="" err=<
Sep 18 00:18:46.817117 ip-10-0-132-188 kubenswrapper[2562]:         rpc error: code = Unknown desc = command error: executable file `/bin/sleep` not found: No such file or directory
Sep 18 00:18:46.817117 ip-10-0-132-188 kubenswrapper[2562]:         , stdout: , stderr: , exit code -1
Sep 18 00:18:46.817117 ip-10-0-132-188 kubenswrapper[2562]:  >
Sep 18 00:18:46.817117 ip-10-0-132-188 kubenswrapper[2562]: E0918 00:18:46.816824    2562 kuberuntime_container.go:755] "PreStop hook failed" err=<
Sep 18 00:18:46.817117 ip-10-0-132-188 kubenswrapper[2562]:         rpc error: code = Unknown desc = command error: executable file `/bin/sleep` not found: No such file or directory
Sep 18 00:18:46.817117 ip-10-0-132-188 kubenswrapper[2562]:         , stdout: , stderr: , exit code -1
Sep 18 00:18:46.817117 ip-10-0-132-188 kubenswrapper[2562]:  > pod="e2e-test-llm-inference-service-aae51b37/llmisvc-model-qwen2-5-0-5b-rout-4c1ba212-kserve-668cf8f94fq92kl" podUID="59316f5c-40e8-4069-a8ee-d91bdfb0aad6" containerName="main" containerID="cri-o://f922c8a060193fc49c992528eec5917b8ab4ec70e8d2f1f03dbcb1bd5dbdf574"
Sep 18 00:18:46.817117 ip-10-0-132-188 kubenswrapper[2562]: I0918 00:18:46.816857    2562 kuberuntime_container.go:873] "Killing container with a grace period" pod="e2e-test-llm-inference-service-aae51b37/llmisvc-model-qwen2-5-0-5b-rout-4c1ba212-kserve-668cf8f94fq92kl" podUID="59316f5c-40e8-4069-a8ee-d91bdfb0aad6" containerName="main" containerID="cri-o://f922c8a060193fc49c992528eec5917b8ab4ec70e8d2f1f03dbcb1bd5dbdf574" gracePeriod=60
Sep 18 00:18:46.901009 ip-10-0-132-188 kubenswrapper[2562]: I0918 00:18:46.900970    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fwwc5_7e452bbb-7dc6-4c9d-a9b1-76400942f5b4/konnectivity-agent/0.log" limit=1024
Sep 18 00:18:47.070797 ip-10-0-132-188 kubenswrapper[2562]: I0918 00:18:47.070686    2562 prober.go:120] "Probe failed" probeType="Readiness" pod="e2e-test-llm-inference-service-aae51b37/llmisvc-model-qwen2-5-0-5b-rout-4c1ba212-kserve-668cf8f94fq92kl" podUID="59316f5c-40e8-4069-a8ee-d91bdfb0aad6" containerName="main" probeResult="failure" output="Get \"https://10.134.0.58:8000/health\": dial tcp 10.134.0.58:8000: connect: connection refused"
Sep 18 00:18:47.252186 ip-10-0-132-188 kubenswrapper[2562]: I0918 00:18:47.252143    2562 generic.go:356] "Generic (PLEG): container finished" podID="59316f5c-40e8-4069-a8ee-d91bdfb0aad6" containerID="f922c8a060193fc49c992528eec5917b8ab4ec70e8d2f1f03dbcb1bd5dbdf574" exitCode=0
Sep 18 00:18:47.252368 ip-10-0-132-188 kubenswrapper[2562]: I0918 00:18:47.252201    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-test-llm-inference-service-aae51b37/llmisvc-model-qwen2-5-0-5b-rout-4c1ba212-kserve-668cf8f94fq92kl" event={"ID":"59316f5c-40e8-4069-a8ee-d91bdfb0aad6","Type":"ContainerDied","Data":"f922c8a060193fc49c992528eec5917b8ab4ec70e8d2f1f03dbcb1bd5dbdf574"}
Sep 18 00:18:47.566250 ip-10-0-132-188 kubenswrapper[2562]: I0918 00:18:47.566211    2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-test-llm-inference-service-aae51b37/llmisvc-model-qwen2-5-0-5b-rout-4c1ba212-kserve-668cf8f94fq92kl"
Sep 18 00:18:47.590714 ip-10-0-132-188 kubenswrapper[2562]: I0918 00:18:47.590678    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-qqhtr\" (UniqueName: \"kubernetes.io/projected/59316f5c-40e8-4069-a8ee-d91bdfb0aad6-kube-api-access-qqhtr\") pod \"59316f5c-40e8-4069-a8ee-d91bdfb0aad6\" (UID: \"59316f5c-40e8-4069-a8ee-d91bdfb0aad6\") "
Sep 18 00:18:47.590714 ip-10-0-132-188 kubenswrapper[2562]: I0918 00:18:47.590724    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/59316f5c-40e8-4069-a8ee-d91bdfb0aad6-dshm\") pod \"59316f5c-40e8-4069-a8ee-d91bdfb0aad6\" (UID: \"59316f5c-40e8-4069-a8ee-d91bdfb0aad6\") "
Sep 18 00:18:47.590972 ip-10-0-132-188 kubenswrapper[2562]: I0918 00:18:47.590750    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/59316f5c-40e8-4069-a8ee-d91bdfb0aad6-tls-certs\") pod \"59316f5c-40e8-4069-a8ee-d91bdfb0aad6\" (UID: \"59316f5c-40e8-4069-a8ee-d91bdfb0aad6\") "
Sep 18 00:18:47.590972 ip-10-0-132-188 kubenswrapper[2562]: I0918 00:18:47.590820    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/59316f5c-40e8-4069-a8ee-d91bdfb0aad6-tmp-dir\") pod \"59316f5c-40e8-4069-a8ee-d91bdfb0aad6\" (UID: \"59316f5c-40e8-4069-a8ee-d91bdfb0aad6\") "
Sep 18 00:18:47.590972 ip-10-0-132-188 kubenswrapper[2562]: I0918 00:18:47.590950    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/59316f5c-40e8-4069-a8ee-d91bdfb0aad6-model-cache\") pod \"59316f5c-40e8-4069-a8ee-d91bdfb0aad6\" (UID: \"59316f5c-40e8-4069-a8ee-d91bdfb0aad6\") "
Sep 18 00:18:47.591138 ip-10-0-132-188 kubenswrapper[2562]: I0918 00:18:47.591012    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/59316f5c-40e8-4069-a8ee-d91bdfb0aad6-home\") pod \"59316f5c-40e8-4069-a8ee-d91bdfb0aad6\" (UID: \"59316f5c-40e8-4069-a8ee-d91bdfb0aad6\") "
Sep 18 00:18:47.591138 ip-10-0-132-188 kubenswrapper[2562]: I0918 00:18:47.591079    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/59316f5c-40e8-4069-a8ee-d91bdfb0aad6-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "59316f5c-40e8-4069-a8ee-d91bdfb0aad6" (UID: "59316f5c-40e8-4069-a8ee-d91bdfb0aad6"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 18 00:18:47.591278 ip-10-0-132-188 kubenswrapper[2562]: I0918 00:18:47.591248    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/59316f5c-40e8-4069-a8ee-d91bdfb0aad6-model-cache" (OuterVolumeSpecName: "model-cache") pod "59316f5c-40e8-4069-a8ee-d91bdfb0aad6" (UID: "59316f5c-40e8-4069-a8ee-d91bdfb0aad6"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 18 00:18:47.591350 ip-10-0-132-188 kubenswrapper[2562]: I0918 00:18:47.591324    2562 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/59316f5c-40e8-4069-a8ee-d91bdfb0aad6-tmp-dir\") on node \"ip-10-0-132-188.ec2.internal\" DevicePath \"\""
Sep 18 00:18:47.591407 ip-10-0-132-188 kubenswrapper[2562]: I0918 00:18:47.591367    2562 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/59316f5c-40e8-4069-a8ee-d91bdfb0aad6-model-cache\") on node \"ip-10-0-132-188.ec2.internal\" DevicePath \"\""
Sep 18 00:18:47.591463 ip-10-0-132-188 kubenswrapper[2562]: I0918 00:18:47.591391    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/59316f5c-40e8-4069-a8ee-d91bdfb0aad6-home" (OuterVolumeSpecName: "home") pod "59316f5c-40e8-4069-a8ee-d91bdfb0aad6" (UID: "59316f5c-40e8-4069-a8ee-d91bdfb0aad6"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 18 00:18:47.593247 ip-10-0-132-188 kubenswrapper[2562]: I0918 00:18:47.593215    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/59316f5c-40e8-4069-a8ee-d91bdfb0aad6-kube-api-access-qqhtr" (OuterVolumeSpecName: "kube-api-access-qqhtr") pod "59316f5c-40e8-4069-a8ee-d91bdfb0aad6" (UID: "59316f5c-40e8-4069-a8ee-d91bdfb0aad6"). InnerVolumeSpecName "kube-api-access-qqhtr". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 18 00:18:47.593247 ip-10-0-132-188 kubenswrapper[2562]: I0918 00:18:47.593238    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/59316f5c-40e8-4069-a8ee-d91bdfb0aad6-dshm" (OuterVolumeSpecName: "dshm") pod "59316f5c-40e8-4069-a8ee-d91bdfb0aad6" (UID: "59316f5c-40e8-4069-a8ee-d91bdfb0aad6"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 18 00:18:47.593401 ip-10-0-132-188 kubenswrapper[2562]: I0918 00:18:47.593247    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/59316f5c-40e8-4069-a8ee-d91bdfb0aad6-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "59316f5c-40e8-4069-a8ee-d91bdfb0aad6" (UID: "59316f5c-40e8-4069-a8ee-d91bdfb0aad6"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 18 00:18:47.692074 ip-10-0-132-188 kubenswrapper[2562]: I0918 00:18:47.691989    2562 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/59316f5c-40e8-4069-a8ee-d91bdfb0aad6-home\") on node \"ip-10-0-132-188.ec2.internal\" DevicePath \"\""
Sep 18 00:18:47.692074 ip-10-0-132-188 kubenswrapper[2562]: I0918 00:18:47.692018    2562 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-qqhtr\" (UniqueName: \"kubernetes.io/projected/59316f5c-40e8-4069-a8ee-d91bdfb0aad6-kube-api-access-qqhtr\") on node \"ip-10-0-132-188.ec2.internal\" DevicePath \"\""
Sep 18 00:18:47.692074 ip-10-0-132-188 kubenswrapper[2562]: I0918 00:18:47.692028    2562 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/59316f5c-40e8-4069-a8ee-d91bdfb0aad6-dshm\") on node \"ip-10-0-132-188.ec2.internal\" DevicePath \"\""
Sep 18 00:18:47.692074 ip-10-0-132-188 kubenswrapper[2562]: I0918 00:18:47.692037    2562 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/59316f5c-40e8-4069-a8ee-d91bdfb0aad6-tls-certs\") on node \"ip-10-0-132-188.ec2.internal\" DevicePath \"\""
Sep 18 00:18:48.258524 ip-10-0-132-188 kubenswrapper[2562]: I0918 00:18:48.258495    2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-test-llm-inference-service-aae51b37/llmisvc-model-qwen2-5-0-5b-rout-4c1ba212-kserve-668cf8f94fq92kl"
Sep 18 00:18:48.258524 ip-10-0-132-188 kubenswrapper[2562]: I0918 00:18:48.258523    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-test-llm-inference-service-aae51b37/llmisvc-model-qwen2-5-0-5b-rout-4c1ba212-kserve-668cf8f94fq92kl" event={"ID":"59316f5c-40e8-4069-a8ee-d91bdfb0aad6","Type":"ContainerDied","Data":"d6523c791f79354460cc91c9552a303dc10f384eb08e9c9b1edfbcef24842899"}
Sep 18 00:18:48.259024 ip-10-0-132-188 kubenswrapper[2562]: I0918 00:18:48.258581    2562 scope.go:117] "RemoveContainer" containerID="f922c8a060193fc49c992528eec5917b8ab4ec70e8d2f1f03dbcb1bd5dbdf574"
Sep 18 00:18:48.273218 ip-10-0-132-188 kubenswrapper[2562]: I0918 00:18:48.273185    2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["e2e-test-llm-inference-service-aae51b37/llmisvc-model-qwen2-5-0-5b-rout-4c1ba212-kserve-668cf8f94fq92kl"]
Sep 18 00:18:48.277254 ip-10-0-132-188 kubenswrapper[2562]: I0918 00:18:48.277225    2562 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["e2e-test-llm-inference-service-aae51b37/llmisvc-model-qwen2-5-0-5b-rout-4c1ba212-kserve-668cf8f94fq92kl"]
Sep 18 00:18:49.666825 ip-10-0-132-188 kubenswrapper[2562]: I0918 00:18:49.666791    2562 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="59316f5c-40e8-4069-a8ee-d91bdfb0aad6" path="/var/lib/kubelet/pods/59316f5c-40e8-4069-a8ee-d91bdfb0aad6/volumes"
Sep 18 00:19:01.176047 ip-10-0-132-188 kubenswrapper[2562]: I0918 00:19:01.176010    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-test-llm-inference-service-15d6126c/scheduler-configmap-ref-test-kserve-router-scheduler-566c9pklzz"
Sep 18 00:19:02.832426 ip-10-0-132-188 kubenswrapper[2562]: I0918 00:19:02.832386    2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["e2e-test-llm-inference-service-15d6126c/scheduler-configmap-ref-test-kserve-router-scheduler-566c9pklzz"]
Sep 18 00:19:02.854467 ip-10-0-132-188 kubenswrapper[2562]: I0918 00:19:02.854434    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fwwc5_7e452bbb-7dc6-4c9d-a9b1-76400942f5b4/konnectivity-agent/0.log" limit=1024
Sep 18 00:19:02.942288 ip-10-0-132-188 kubenswrapper[2562]: I0918 00:19:02.942252    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fwwc5_7e452bbb-7dc6-4c9d-a9b1-76400942f5b4/konnectivity-agent/0.log" limit=1024
Sep 18 00:19:11.176295 ip-10-0-132-188 kubenswrapper[2562]: I0918 00:19:11.176249    2562 prober.go:120] "Probe failed" probeType="Readiness" pod="e2e-test-llm-inference-service-15d6126c/scheduler-configmap-ref-test-kserve-router-scheduler-566c9pklzz" podUID="673b18a9-0d5d-429e-b821-4b0434978f3b" containerName="main" probeResult="failure" output="service unhealthy (responded with \"NOT_SERVING\")"
Sep 18 00:19:17.854529 ip-10-0-132-188 kubenswrapper[2562]: I0918 00:19:17.854484    2562 kuberuntime_container.go:873] "Killing container with a grace period" pod="e2e-test-llm-inference-service-15d6126c/scheduler-configmap-ref-test-kserve-router-scheduler-566c9pklzz" podUID="673b18a9-0d5d-429e-b821-4b0434978f3b" containerName="main" containerID="cri-o://620518c74ca975d7c3ede510aaafae183afc4ee4b8fd2182f5ad75551e8d56bc" gracePeriod=45
Sep 18 00:19:21.177060 ip-10-0-132-188 kubenswrapper[2562]: I0918 00:19:21.177021    2562 prober.go:120] "Probe failed" probeType="Readiness" pod="e2e-test-llm-inference-service-15d6126c/scheduler-configmap-ref-test-kserve-router-scheduler-566c9pklzz" podUID="673b18a9-0d5d-429e-b821-4b0434978f3b" containerName="main" probeResult="failure" output="service unhealthy (responded with \"NOT_SERVING\")"
Sep 18 00:19:31.176507 ip-10-0-132-188 kubenswrapper[2562]: I0918 00:19:31.176414    2562 prober.go:120] "Probe failed" probeType="Readiness" pod="e2e-test-llm-inference-service-15d6126c/scheduler-configmap-ref-test-kserve-router-scheduler-566c9pklzz" podUID="673b18a9-0d5d-429e-b821-4b0434978f3b" containerName="main" probeResult="failure" output="service unhealthy (responded with \"NOT_SERVING\")"
Sep 18 00:19:31.177006 ip-10-0-132-188 kubenswrapper[2562]: I0918 00:19:31.176522    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-test-llm-inference-service-15d6126c/scheduler-configmap-ref-test-kserve-router-scheduler-566c9pklzz"
Sep 18 00:19:34.178746 ip-10-0-132-188 kubenswrapper[2562]: I0918 00:19:34.178696    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fwwc5_7e452bbb-7dc6-4c9d-a9b1-76400942f5b4/konnectivity-agent/0.log" limit=1024
Sep 18 00:19:35.576654 ip-10-0-132-188 kubenswrapper[2562]: I0918 00:19:35.576617    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fwwc5_7e452bbb-7dc6-4c9d-a9b1-76400942f5b4/konnectivity-agent/0.log" limit=1024
Sep 18 00:19:41.176247 ip-10-0-132-188 kubenswrapper[2562]: I0918 00:19:41.176201    2562 prober.go:120] "Probe failed" probeType="Readiness" pod="e2e-test-llm-inference-service-15d6126c/scheduler-configmap-ref-test-kserve-router-scheduler-566c9pklzz" podUID="673b18a9-0d5d-429e-b821-4b0434978f3b" containerName="main" probeResult="failure" output="service unhealthy (responded with \"NOT_SERVING\")"
Sep 18 00:19:41.368479 ip-10-0-132-188 kubenswrapper[2562]: I0918 00:19:41.368435    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fwwc5_7e452bbb-7dc6-4c9d-a9b1-76400942f5b4/konnectivity-agent/0.log" limit=1024
Sep 18 00:19:46.843700 ip-10-0-132-188 kubenswrapper[2562]: I0918 00:19:46.843665    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fwwc5_7e452bbb-7dc6-4c9d-a9b1-76400942f5b4/konnectivity-agent/0.log" limit=1024
Sep 18 00:19:46.943691 ip-10-0-132-188 kubenswrapper[2562]: I0918 00:19:46.943648    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fwwc5_7e452bbb-7dc6-4c9d-a9b1-76400942f5b4/konnectivity-agent/0.log" limit=1024
Sep 18 00:19:48.102739 ip-10-0-132-188 kubenswrapper[2562]: I0918 00:19:48.102704    2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-test-llm-inference-service-15d6126c/scheduler-configmap-ref-test-kserve-router-scheduler-566c9pklzz"
Sep 18 00:19:48.220888 ip-10-0-132-188 kubenswrapper[2562]: I0918 00:19:48.220776    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-f9np7\" (UniqueName: \"kubernetes.io/projected/673b18a9-0d5d-429e-b821-4b0434978f3b-kube-api-access-f9np7\") pod \"673b18a9-0d5d-429e-b821-4b0434978f3b\" (UID: \"673b18a9-0d5d-429e-b821-4b0434978f3b\") "
Sep 18 00:19:48.220888 ip-10-0-132-188 kubenswrapper[2562]: I0918 00:19:48.220821    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/673b18a9-0d5d-429e-b821-4b0434978f3b-tls-certs\") pod \"673b18a9-0d5d-429e-b821-4b0434978f3b\" (UID: \"673b18a9-0d5d-429e-b821-4b0434978f3b\") "
Sep 18 00:19:48.223286 ip-10-0-132-188 kubenswrapper[2562]: I0918 00:19:48.223253    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/673b18a9-0d5d-429e-b821-4b0434978f3b-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "673b18a9-0d5d-429e-b821-4b0434978f3b" (UID: "673b18a9-0d5d-429e-b821-4b0434978f3b"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 18 00:19:48.223408 ip-10-0-132-188 kubenswrapper[2562]: I0918 00:19:48.223312    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/673b18a9-0d5d-429e-b821-4b0434978f3b-kube-api-access-f9np7" (OuterVolumeSpecName: "kube-api-access-f9np7") pod "673b18a9-0d5d-429e-b821-4b0434978f3b" (UID: "673b18a9-0d5d-429e-b821-4b0434978f3b"). InnerVolumeSpecName "kube-api-access-f9np7". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 18 00:19:48.321923 ip-10-0-132-188 kubenswrapper[2562]: I0918 00:19:48.321874    2562 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-f9np7\" (UniqueName: \"kubernetes.io/projected/673b18a9-0d5d-429e-b821-4b0434978f3b-kube-api-access-f9np7\") on node \"ip-10-0-132-188.ec2.internal\" DevicePath \"\""
Sep 18 00:19:48.321923 ip-10-0-132-188 kubenswrapper[2562]: I0918 00:19:48.321916    2562 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/673b18a9-0d5d-429e-b821-4b0434978f3b-tls-certs\") on node \"ip-10-0-132-188.ec2.internal\" DevicePath \"\""
Sep 18 00:19:48.536212 ip-10-0-132-188 kubenswrapper[2562]: I0918 00:19:48.536089    2562 generic.go:356] "Generic (PLEG): container finished" podID="673b18a9-0d5d-429e-b821-4b0434978f3b" containerID="620518c74ca975d7c3ede510aaafae183afc4ee4b8fd2182f5ad75551e8d56bc" exitCode=0
Sep 18 00:19:48.536212 ip-10-0-132-188 kubenswrapper[2562]: I0918 00:19:48.536136    2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-test-llm-inference-service-15d6126c/scheduler-configmap-ref-test-kserve-router-scheduler-566c9pklzz"
Sep 18 00:19:48.536212 ip-10-0-132-188 kubenswrapper[2562]: I0918 00:19:48.536181    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-test-llm-inference-service-15d6126c/scheduler-configmap-ref-test-kserve-router-scheduler-566c9pklzz" event={"ID":"673b18a9-0d5d-429e-b821-4b0434978f3b","Type":"ContainerDied","Data":"620518c74ca975d7c3ede510aaafae183afc4ee4b8fd2182f5ad75551e8d56bc"}
Sep 18 00:19:48.536444 ip-10-0-132-188 kubenswrapper[2562]: I0918 00:19:48.536236    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-test-llm-inference-service-15d6126c/scheduler-configmap-ref-test-kserve-router-scheduler-566c9pklzz" event={"ID":"673b18a9-0d5d-429e-b821-4b0434978f3b","Type":"ContainerDied","Data":"16b2866a19483d194869e41f1c416e99ff99460e086f85b5cccf2147c6f7a274"}
Sep 18 00:19:48.536444 ip-10-0-132-188 kubenswrapper[2562]: I0918 00:19:48.536260    2562 scope.go:117] "RemoveContainer" containerID="620518c74ca975d7c3ede510aaafae183afc4ee4b8fd2182f5ad75551e8d56bc"
Sep 18 00:19:48.545627 ip-10-0-132-188 kubenswrapper[2562]: I0918 00:19:48.545603    2562 scope.go:117] "RemoveContainer" containerID="620518c74ca975d7c3ede510aaafae183afc4ee4b8fd2182f5ad75551e8d56bc"
Sep 18 00:19:48.545932 ip-10-0-132-188 kubenswrapper[2562]: E0918 00:19:48.545910    2562 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"620518c74ca975d7c3ede510aaafae183afc4ee4b8fd2182f5ad75551e8d56bc\": container with ID starting with 620518c74ca975d7c3ede510aaafae183afc4ee4b8fd2182f5ad75551e8d56bc not found: ID does not exist" containerID="620518c74ca975d7c3ede510aaafae183afc4ee4b8fd2182f5ad75551e8d56bc"
Sep 18 00:19:48.546040 ip-10-0-132-188 kubenswrapper[2562]: I0918 00:19:48.545938    2562 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"620518c74ca975d7c3ede510aaafae183afc4ee4b8fd2182f5ad75551e8d56bc"} err="failed to get container status \"620518c74ca975d7c3ede510aaafae183afc4ee4b8fd2182f5ad75551e8d56bc\": rpc error: code = NotFound desc = could not find container \"620518c74ca975d7c3ede510aaafae183afc4ee4b8fd2182f5ad75551e8d56bc\": container with ID starting with 620518c74ca975d7c3ede510aaafae183afc4ee4b8fd2182f5ad75551e8d56bc not found: ID does not exist"
Sep 18 00:19:48.557531 ip-10-0-132-188 kubenswrapper[2562]: I0918 00:19:48.557498    2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["e2e-test-llm-inference-service-15d6126c/scheduler-configmap-ref-test-kserve-router-scheduler-566c9pklzz"]
Sep 18 00:19:48.562875 ip-10-0-132-188 kubenswrapper[2562]: I0918 00:19:48.562846    2562 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["e2e-test-llm-inference-service-15d6126c/scheduler-configmap-ref-test-kserve-router-scheduler-566c9pklzz"]
Sep 18 00:19:49.667160 ip-10-0-132-188 kubenswrapper[2562]: I0918 00:19:49.667123    2562 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="673b18a9-0d5d-429e-b821-4b0434978f3b" path="/var/lib/kubelet/pods/673b18a9-0d5d-429e-b821-4b0434978f3b/volumes"
Sep 18 00:19:52.286623 ip-10-0-132-188 kubenswrapper[2562]: I0918 00:19:52.286581    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fwwc5_7e452bbb-7dc6-4c9d-a9b1-76400942f5b4/konnectivity-agent/0.log" limit=1024
Sep 18 00:19:55.944478 ip-10-0-132-188 kubenswrapper[2562]: I0918 00:19:55.944441    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fwwc5_7e452bbb-7dc6-4c9d-a9b1-76400942f5b4/konnectivity-agent/0.log" limit=1024
Sep 18 00:19:56.008461 ip-10-0-132-188 kubenswrapper[2562]: I0918 00:19:56.008423    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fwwc5_7e452bbb-7dc6-4c9d-a9b1-76400942f5b4/konnectivity-agent/0.log" limit=1024
Sep 18 00:19:58.181013 ip-10-0-132-188 kubenswrapper[2562]: I0918 00:19:58.180976    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fwwc5_7e452bbb-7dc6-4c9d-a9b1-76400942f5b4/konnectivity-agent/0.log" limit=1024
Sep 18 00:20:03.588782 ip-10-0-132-188 kubenswrapper[2562]: I0918 00:20:03.588745    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fwwc5_7e452bbb-7dc6-4c9d-a9b1-76400942f5b4/konnectivity-agent/0.log" limit=1024
Sep 18 00:20:03.654416 ip-10-0-132-188 kubenswrapper[2562]: I0918 00:20:03.654379    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fwwc5_7e452bbb-7dc6-4c9d-a9b1-76400942f5b4/konnectivity-agent/0.log" limit=1024
Sep 18 00:20:18.819060 ip-10-0-132-188 kubenswrapper[2562]: I0918 00:20:18.819022    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fwwc5_7e452bbb-7dc6-4c9d-a9b1-76400942f5b4/konnectivity-agent/0.log" limit=1024
Sep 18 00:20:18.867377 ip-10-0-132-188 kubenswrapper[2562]: I0918 00:20:18.867339    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fwwc5_7e452bbb-7dc6-4c9d-a9b1-76400942f5b4/konnectivity-agent/0.log" limit=1024
Sep 18 00:20:19.070840 ip-10-0-132-188 kubenswrapper[2562]: I0918 00:20:19.070756    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_istiod-openshift-gateway-5677ccb49b-np75w_eb73a80f-c670-434f-9ddb-7953608877b2/discovery/0.log"
Sep 18 00:20:19.086374 ip-10-0-132-188 kubenswrapper[2562]: I0918 00:20:19.086344    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_openshift-ai-inference-openshift-default-58c5cd85c9-9wjvx_bafca983-9e23-438d-b13c-ff56f8b81b6b/istio-proxy/0.log"
Sep 18 00:20:19.091184 ip-10-0-132-188 kubenswrapper[2562]: I0918 00:20:19.091141    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_openshift-ai-inference-openshift-default-58c5cd85c9-9wjvx_bafca983-9e23-438d-b13c-ff56f8b81b6b/istio-proxy/1.log"
Sep 18 00:20:19.928461 ip-10-0-132-188 kubenswrapper[2562]: I0918 00:20:19.928424    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_istiod-openshift-gateway-5677ccb49b-np75w_eb73a80f-c670-434f-9ddb-7953608877b2/discovery/0.log"
Sep 18 00:20:19.947294 ip-10-0-132-188 kubenswrapper[2562]: I0918 00:20:19.947262    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_openshift-ai-inference-openshift-default-58c5cd85c9-9wjvx_bafca983-9e23-438d-b13c-ff56f8b81b6b/istio-proxy/0.log"
Sep 18 00:20:19.952430 ip-10-0-132-188 kubenswrapper[2562]: I0918 00:20:19.952397    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_openshift-ai-inference-openshift-default-58c5cd85c9-9wjvx_bafca983-9e23-438d-b13c-ff56f8b81b6b/istio-proxy/1.log"
Sep 18 00:20:20.807137 ip-10-0-132-188 kubenswrapper[2562]: I0918 00:20:20.807105    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-operator-6fc8f864f4-tbvdm_cc89fb3b-161e-46c8-b343-f221ce5fedd1/manager/0.log"
Sep 18 00:20:20.940667 ip-10-0-132-188 kubenswrapper[2562]: I0918 00:20:20.940637    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_limitador-operator-controller-manager-5ffb4b4ccb-l2khb_58552d00-05c8-4a22-b1ae-1446a33743bd/manager/0.log"
Sep 18 00:20:26.515787 ip-10-0-132-188 kubenswrapper[2562]: I0918 00:20:26.515745    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fwwc5_7e452bbb-7dc6-4c9d-a9b1-76400942f5b4/konnectivity-agent/0.log" limit=1024
Sep 18 00:20:26.657285 ip-10-0-132-188 kubenswrapper[2562]: I0918 00:20:26.657244    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fwwc5_7e452bbb-7dc6-4c9d-a9b1-76400942f5b4/konnectivity-agent/0.log" limit=1024
Sep 18 00:20:26.724250 ip-10-0-132-188 kubenswrapper[2562]: I0918 00:20:26.724218    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fwwc5_7e452bbb-7dc6-4c9d-a9b1-76400942f5b4/konnectivity-agent/0.log" limit=1024
Sep 18 00:20:29.438870 ip-10-0-132-188 kubenswrapper[2562]: I0918 00:20:29.438833    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_global-pull-secret-syncer-jt864_79765acc-1a30-4a47-add5-aea2bf5f9efb/global-pull-secret-syncer/0.log"
Sep 18 00:20:29.494932 ip-10-0-132-188 kubenswrapper[2562]: I0918 00:20:29.494897    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_konnectivity-agent-fwwc5_7e452bbb-7dc6-4c9d-a9b1-76400942f5b4/konnectivity-agent/0.log"
Sep 18 00:20:29.617035 ip-10-0-132-188 kubenswrapper[2562]: I0918 00:20:29.616999    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_kube-apiserver-proxy-ip-10-0-132-188.ec2.internal_380b7a798346051e115745fb0a3f8e3c/haproxy/0.log"
Sep 18 00:20:32.361424 ip-10-0-132-188 kubenswrapper[2562]: I0918 00:20:32.361390    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fwwc5_7e452bbb-7dc6-4c9d-a9b1-76400942f5b4/konnectivity-agent/0.log" limit=1024
Sep 18 00:20:32.461679 ip-10-0-132-188 kubenswrapper[2562]: I0918 00:20:32.461642    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fwwc5_7e452bbb-7dc6-4c9d-a9b1-76400942f5b4/konnectivity-agent/0.log" limit=1024
Sep 18 00:20:34.548504 ip-10-0-132-188 kubenswrapper[2562]: I0918 00:20:34.548468    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-operator-6fc8f864f4-tbvdm_cc89fb3b-161e-46c8-b343-f221ce5fedd1/manager/0.log"
Sep 18 00:20:34.735185 ip-10-0-132-188 kubenswrapper[2562]: I0918 00:20:34.735137    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_limitador-operator-controller-manager-5ffb4b4ccb-l2khb_58552d00-05c8-4a22-b1ae-1446a33743bd/manager/0.log"
Sep 18 00:20:36.398390 ip-10-0-132-188 kubenswrapper[2562]: I0918 00:20:36.398353    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_metrics-server-5d89bb6d78-5kftc_8fcaefb7-7432-4b32-95ae-09870e558cdd/metrics-server/0.log"
Sep 18 00:20:36.453300 ip-10-0-132-188 kubenswrapper[2562]: I0918 00:20:36.453269    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-dllnm_34ae65de-7218-4b53-9a3e-b223c5338b11/node-exporter/0.log"
Sep 18 00:20:36.469712 ip-10-0-132-188 kubenswrapper[2562]: I0918 00:20:36.469685    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-dllnm_34ae65de-7218-4b53-9a3e-b223c5338b11/kube-rbac-proxy/0.log"
Sep 18 00:20:36.486902 ip-10-0-132-188 kubenswrapper[2562]: I0918 00:20:36.486879    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-dllnm_34ae65de-7218-4b53-9a3e-b223c5338b11/init-textfile/0.log"
Sep 18 00:20:36.665928 ip-10-0-132-188 kubenswrapper[2562]: I0918 00:20:36.665843    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_openshift-state-metrics-86c5bc79d6-bwj6t_c5c362bf-e3b6-41f3-85c6-4a7d2a3e4b87/kube-rbac-proxy-main/0.log"
Sep 18 00:20:36.683840 ip-10-0-132-188 kubenswrapper[2562]: I0918 00:20:36.683812    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_openshift-state-metrics-86c5bc79d6-bwj6t_c5c362bf-e3b6-41f3-85c6-4a7d2a3e4b87/kube-rbac-proxy-self/0.log"
Sep 18 00:20:36.706438 ip-10-0-132-188 kubenswrapper[2562]: I0918 00:20:36.706390    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_openshift-state-metrics-86c5bc79d6-bwj6t_c5c362bf-e3b6-41f3-85c6-4a7d2a3e4b87/openshift-state-metrics/0.log"
Sep 18 00:20:36.908130 ip-10-0-132-188 kubenswrapper[2562]: I0918 00:20:36.908101    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-operator-7bd5d56d6-x6qkt_0d951367-db80-4aee-9177-dd51602c88ce/prometheus-operator/0.log"
Sep 18 00:20:36.921375 ip-10-0-132-188 kubenswrapper[2562]: I0918 00:20:36.921294    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-operator-7bd5d56d6-x6qkt_0d951367-db80-4aee-9177-dd51602c88ce/kube-rbac-proxy/0.log"
Sep 18 00:20:38.530941 ip-10-0-132-188 kubenswrapper[2562]: I0918 00:20:38.530908    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-network-console_networking-console-plugin-6446c8d75d-9gc6s_68dcc3ba-959d-419d-a77b-31b95d377616/networking-console-plugin/0.log"
Sep 18 00:20:39.470927 ip-10-0-132-188 kubenswrapper[2562]: I0918 00:20:39.470894    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console_console-84794c78d6-brcjh_2a8e168f-81be-41c4-8635-2ba879bdce47/console/0.log"
Sep 18 00:20:40.154114 ip-10-0-132-188 kubenswrapper[2562]: I0918 00:20:40.154073    2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-must-gather-8545l/perf-node-gather-daemonset-jpgjq"]
Sep 18 00:20:40.154521 ip-10-0-132-188 kubenswrapper[2562]: I0918 00:20:40.154419    2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="673b18a9-0d5d-429e-b821-4b0434978f3b" containerName="main"
Sep 18 00:20:40.154521 ip-10-0-132-188 kubenswrapper[2562]: I0918 00:20:40.154431    2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="673b18a9-0d5d-429e-b821-4b0434978f3b" containerName="main"
Sep 18 00:20:40.154521 ip-10-0-132-188 kubenswrapper[2562]: I0918 00:20:40.154451    2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="59316f5c-40e8-4069-a8ee-d91bdfb0aad6" containerName="main"
Sep 18 00:20:40.154521 ip-10-0-132-188 kubenswrapper[2562]: I0918 00:20:40.154458    2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="59316f5c-40e8-4069-a8ee-d91bdfb0aad6" containerName="main"
Sep 18 00:20:40.154521 ip-10-0-132-188 kubenswrapper[2562]: I0918 00:20:40.154509    2562 memory_manager.go:368] "RemoveStaleState removing state" podUID="59316f5c-40e8-4069-a8ee-d91bdfb0aad6" containerName="main"
Sep 18 00:20:40.154521 ip-10-0-132-188 kubenswrapper[2562]: I0918 00:20:40.154518    2562 memory_manager.go:368] "RemoveStaleState removing state" podUID="673b18a9-0d5d-429e-b821-4b0434978f3b" containerName="main"
Sep 18 00:20:40.157515 ip-10-0-132-188 kubenswrapper[2562]: I0918 00:20:40.157492    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-8545l/perf-node-gather-daemonset-jpgjq"
Sep 18 00:20:40.159948 ip-10-0-132-188 kubenswrapper[2562]: I0918 00:20:40.159920    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-must-gather-8545l\"/\"default-dockercfg-vqjqc\""
Sep 18 00:20:40.160117 ip-10-0-132-188 kubenswrapper[2562]: I0918 00:20:40.159920    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-8545l\"/\"openshift-service-ca.crt\""
Sep 18 00:20:40.160769 ip-10-0-132-188 kubenswrapper[2562]: I0918 00:20:40.160748    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-8545l\"/\"kube-root-ca.crt\""
Sep 18 00:20:40.165983 ip-10-0-132-188 kubenswrapper[2562]: I0918 00:20:40.165955    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-8545l/perf-node-gather-daemonset-jpgjq"]
Sep 18 00:20:40.285356 ip-10-0-132-188 kubenswrapper[2562]: I0918 00:20:40.285318    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/76bb6252-a515-47a1-8b01-66b3b2df20f9-sys\") pod \"perf-node-gather-daemonset-jpgjq\" (UID: \"76bb6252-a515-47a1-8b01-66b3b2df20f9\") " pod="openshift-must-gather-8545l/perf-node-gather-daemonset-jpgjq"
Sep 18 00:20:40.285356 ip-10-0-132-188 kubenswrapper[2562]: I0918 00:20:40.285364    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/76bb6252-a515-47a1-8b01-66b3b2df20f9-lib-modules\") pod \"perf-node-gather-daemonset-jpgjq\" (UID: \"76bb6252-a515-47a1-8b01-66b3b2df20f9\") " pod="openshift-must-gather-8545l/perf-node-gather-daemonset-jpgjq"
Sep 18 00:20:40.285597 ip-10-0-132-188 kubenswrapper[2562]: I0918 00:20:40.285472    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vrjl8\" (UniqueName: \"kubernetes.io/projected/76bb6252-a515-47a1-8b01-66b3b2df20f9-kube-api-access-vrjl8\") pod \"perf-node-gather-daemonset-jpgjq\" (UID: \"76bb6252-a515-47a1-8b01-66b3b2df20f9\") " pod="openshift-must-gather-8545l/perf-node-gather-daemonset-jpgjq"
Sep 18 00:20:40.285597 ip-10-0-132-188 kubenswrapper[2562]: I0918 00:20:40.285510    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/76bb6252-a515-47a1-8b01-66b3b2df20f9-proc\") pod \"perf-node-gather-daemonset-jpgjq\" (UID: \"76bb6252-a515-47a1-8b01-66b3b2df20f9\") " pod="openshift-must-gather-8545l/perf-node-gather-daemonset-jpgjq"
Sep 18 00:20:40.285597 ip-10-0-132-188 kubenswrapper[2562]: I0918 00:20:40.285527    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/76bb6252-a515-47a1-8b01-66b3b2df20f9-podres\") pod \"perf-node-gather-daemonset-jpgjq\" (UID: \"76bb6252-a515-47a1-8b01-66b3b2df20f9\") " pod="openshift-must-gather-8545l/perf-node-gather-daemonset-jpgjq"
Sep 18 00:20:40.386533 ip-10-0-132-188 kubenswrapper[2562]: I0918 00:20:40.386494    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/76bb6252-a515-47a1-8b01-66b3b2df20f9-sys\") pod \"perf-node-gather-daemonset-jpgjq\" (UID: \"76bb6252-a515-47a1-8b01-66b3b2df20f9\") " pod="openshift-must-gather-8545l/perf-node-gather-daemonset-jpgjq"
Sep 18 00:20:40.386533 ip-10-0-132-188 kubenswrapper[2562]: I0918 00:20:40.386538    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/76bb6252-a515-47a1-8b01-66b3b2df20f9-lib-modules\") pod \"perf-node-gather-daemonset-jpgjq\" (UID: \"76bb6252-a515-47a1-8b01-66b3b2df20f9\") " pod="openshift-must-gather-8545l/perf-node-gather-daemonset-jpgjq"
Sep 18 00:20:40.386815 ip-10-0-132-188 kubenswrapper[2562]: I0918 00:20:40.386576    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vrjl8\" (UniqueName: \"kubernetes.io/projected/76bb6252-a515-47a1-8b01-66b3b2df20f9-kube-api-access-vrjl8\") pod \"perf-node-gather-daemonset-jpgjq\" (UID: \"76bb6252-a515-47a1-8b01-66b3b2df20f9\") " pod="openshift-must-gather-8545l/perf-node-gather-daemonset-jpgjq"
Sep 18 00:20:40.386815 ip-10-0-132-188 kubenswrapper[2562]: I0918 00:20:40.386605    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/76bb6252-a515-47a1-8b01-66b3b2df20f9-proc\") pod \"perf-node-gather-daemonset-jpgjq\" (UID: \"76bb6252-a515-47a1-8b01-66b3b2df20f9\") " pod="openshift-must-gather-8545l/perf-node-gather-daemonset-jpgjq"
Sep 18 00:20:40.386815 ip-10-0-132-188 kubenswrapper[2562]: I0918 00:20:40.386621    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/76bb6252-a515-47a1-8b01-66b3b2df20f9-podres\") pod \"perf-node-gather-daemonset-jpgjq\" (UID: \"76bb6252-a515-47a1-8b01-66b3b2df20f9\") " pod="openshift-must-gather-8545l/perf-node-gather-daemonset-jpgjq"
Sep 18 00:20:40.386815 ip-10-0-132-188 kubenswrapper[2562]: I0918 00:20:40.386634    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/76bb6252-a515-47a1-8b01-66b3b2df20f9-sys\") pod \"perf-node-gather-daemonset-jpgjq\" (UID: \"76bb6252-a515-47a1-8b01-66b3b2df20f9\") " pod="openshift-must-gather-8545l/perf-node-gather-daemonset-jpgjq"
Sep 18 00:20:40.386815 ip-10-0-132-188 kubenswrapper[2562]: I0918 00:20:40.386727    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/76bb6252-a515-47a1-8b01-66b3b2df20f9-proc\") pod \"perf-node-gather-daemonset-jpgjq\" (UID: \"76bb6252-a515-47a1-8b01-66b3b2df20f9\") " pod="openshift-must-gather-8545l/perf-node-gather-daemonset-jpgjq"
Sep 18 00:20:40.386815 ip-10-0-132-188 kubenswrapper[2562]: I0918 00:20:40.386729    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/76bb6252-a515-47a1-8b01-66b3b2df20f9-lib-modules\") pod \"perf-node-gather-daemonset-jpgjq\" (UID: \"76bb6252-a515-47a1-8b01-66b3b2df20f9\") " pod="openshift-must-gather-8545l/perf-node-gather-daemonset-jpgjq"
Sep 18 00:20:40.386815 ip-10-0-132-188 kubenswrapper[2562]: I0918 00:20:40.386766    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/76bb6252-a515-47a1-8b01-66b3b2df20f9-podres\") pod \"perf-node-gather-daemonset-jpgjq\" (UID: \"76bb6252-a515-47a1-8b01-66b3b2df20f9\") " pod="openshift-must-gather-8545l/perf-node-gather-daemonset-jpgjq"
Sep 18 00:20:40.394113 ip-10-0-132-188 kubenswrapper[2562]: I0918 00:20:40.394080    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-vrjl8\" (UniqueName: \"kubernetes.io/projected/76bb6252-a515-47a1-8b01-66b3b2df20f9-kube-api-access-vrjl8\") pod \"perf-node-gather-daemonset-jpgjq\" (UID: \"76bb6252-a515-47a1-8b01-66b3b2df20f9\") " pod="openshift-must-gather-8545l/perf-node-gather-daemonset-jpgjq"
Sep 18 00:20:40.469156 ip-10-0-132-188 kubenswrapper[2562]: I0918 00:20:40.469067    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-8545l/perf-node-gather-daemonset-jpgjq"
Sep 18 00:20:40.739314 ip-10-0-132-188 kubenswrapper[2562]: I0918 00:20:40.739283    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-8545l/perf-node-gather-daemonset-jpgjq"]
Sep 18 00:20:40.742152 ip-10-0-132-188 kubenswrapper[2562]: W0918 00:20:40.742122    2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-pod76bb6252_a515_47a1_8b01_66b3b2df20f9.slice/crio-7f26c61dea3d0263c802e2ca1edd9fd052775242462c55afee5c73d27f86b2fe WatchSource:0}: Error finding container 7f26c61dea3d0263c802e2ca1edd9fd052775242462c55afee5c73d27f86b2fe: Status 404 returned error can't find the container with id 7f26c61dea3d0263c802e2ca1edd9fd052775242462c55afee5c73d27f86b2fe
Sep 18 00:20:40.776858 ip-10-0-132-188 kubenswrapper[2562]: I0918 00:20:40.776816    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-8545l/perf-node-gather-daemonset-jpgjq" event={"ID":"76bb6252-a515-47a1-8b01-66b3b2df20f9","Type":"ContainerStarted","Data":"7f26c61dea3d0263c802e2ca1edd9fd052775242462c55afee5c73d27f86b2fe"}
Sep 18 00:20:41.017540 ip-10-0-132-188 kubenswrapper[2562]: I0918 00:20:41.017440    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-9z4vw_57e9d0f5-d16a-42d0-ab45-c06fe21e7bbd/dns/0.log"
Sep 18 00:20:41.034683 ip-10-0-132-188 kubenswrapper[2562]: I0918 00:20:41.034652    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-9z4vw_57e9d0f5-d16a-42d0-ab45-c06fe21e7bbd/kube-rbac-proxy/0.log"
Sep 18 00:20:41.151607 ip-10-0-132-188 kubenswrapper[2562]: I0918 00:20:41.151572    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-fzpjd_1d92dc9b-45b6-4912-bf69-f0961e86e8ed/dns-node-resolver/0.log"
Sep 18 00:20:41.650581 ip-10-0-132-188 kubenswrapper[2562]: I0918 00:20:41.650550    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_image-pruner-29828160-gvvld_7b22e522-10fd-47a8-8c9e-f9f87fee23ca/image-pruner/0.log"
Sep 18 00:20:41.675026 ip-10-0-132-188 kubenswrapper[2562]: I0918 00:20:41.674990    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_image-registry-64c75f9db5-qc6hf_51d624a1-203a-4888-be49-4a3c375984a3/registry/0.log"
Sep 18 00:20:41.704078 ip-10-0-132-188 kubenswrapper[2562]: I0918 00:20:41.704039    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_image-registry-64c75f9db5-qc6hf_51d624a1-203a-4888-be49-4a3c375984a3/registry/1.log"
Sep 18 00:20:41.768917 ip-10-0-132-188 kubenswrapper[2562]: I0918 00:20:41.768888    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-vkllk_33db9205-3be0-47dc-ac16-e8f623e43afb/node-ca/0.log"
Sep 18 00:20:41.782409 ip-10-0-132-188 kubenswrapper[2562]: I0918 00:20:41.782368    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-8545l/perf-node-gather-daemonset-jpgjq" event={"ID":"76bb6252-a515-47a1-8b01-66b3b2df20f9","Type":"ContainerStarted","Data":"bbf1df1fb88db7a3c21ddb2501e0d942a3d4eee065ebb4ae4328710cf672c614"}
Sep 18 00:20:41.782610 ip-10-0-132-188 kubenswrapper[2562]: I0918 00:20:41.782436    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-must-gather-8545l/perf-node-gather-daemonset-jpgjq"
Sep 18 00:20:41.798880 ip-10-0-132-188 kubenswrapper[2562]: I0918 00:20:41.798824    2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-must-gather-8545l/perf-node-gather-daemonset-jpgjq" podStartSLOduration=1.798807652 podStartE2EDuration="1.798807652s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-18 00:20:40 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-18 00:20:41.797352454 +0000 UTC m=+4194.199249220" watchObservedRunningTime="2026-09-18 00:20:41.798807652 +0000 UTC m=+4194.200704416"
Sep 18 00:20:42.444114 ip-10-0-132-188 kubenswrapper[2562]: I0918 00:20:42.444077    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fwwc5_7e452bbb-7dc6-4c9d-a9b1-76400942f5b4/konnectivity-agent/0.log" limit=1024
Sep 18 00:20:42.783579 ip-10-0-132-188 kubenswrapper[2562]: I0918 00:20:42.783475    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_istiod-openshift-gateway-5677ccb49b-np75w_eb73a80f-c670-434f-9ddb-7953608877b2/discovery/0.log"
Sep 18 00:20:42.806099 ip-10-0-132-188 kubenswrapper[2562]: I0918 00:20:42.806065    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_openshift-ai-inference-openshift-default-58c5cd85c9-9wjvx_bafca983-9e23-438d-b13c-ff56f8b81b6b/istio-proxy/0.log"
Sep 18 00:20:42.811105 ip-10-0-132-188 kubenswrapper[2562]: I0918 00:20:42.811066    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_openshift-ai-inference-openshift-default-58c5cd85c9-9wjvx_bafca983-9e23-438d-b13c-ff56f8b81b6b/istio-proxy/1.log"
Sep 18 00:20:43.344326 ip-10-0-132-188 kubenswrapper[2562]: I0918 00:20:43.344294    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-hqrgl_5562eaa3-4a89-4ed2-b42b-831c3da5bba3/serve-healthcheck-canary/0.log"
Sep 18 00:20:43.860973 ip-10-0-132-188 kubenswrapper[2562]: I0918 00:20:43.860936    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-59djn_062c04b5-55f2-49ca-aff9-19fe1c0a20b8/kube-rbac-proxy/0.log"
Sep 18 00:20:43.877791 ip-10-0-132-188 kubenswrapper[2562]: I0918 00:20:43.877762    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-59djn_062c04b5-55f2-49ca-aff9-19fe1c0a20b8/exporter/0.log"
Sep 18 00:20:43.895438 ip-10-0-132-188 kubenswrapper[2562]: I0918 00:20:43.895407    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-59djn_062c04b5-55f2-49ca-aff9-19fe1c0a20b8/extractor/0.log"
Sep 18 00:20:47.449739 ip-10-0-132-188 kubenswrapper[2562]: I0918 00:20:47.449686    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve_kserve-controller-manager-5c5fcd7899-nv2hv_180a5b21-9dea-4a9d-9961-ff92c5aaba82/manager/0.log"
Sep 18 00:20:47.798841 ip-10-0-132-188 kubenswrapper[2562]: I0918 00:20:47.798743    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-must-gather-8545l/perf-node-gather-daemonset-jpgjq"
Sep 18 00:20:47.901227 ip-10-0-132-188 kubenswrapper[2562]: I0918 00:20:47.901186    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve_llmisvc-controller-manager-5cbcfb6fbf-n7654_aa8896c1-9f88-4d89-a8a0-fcebd0ac028e/manager/0.log"
Sep 18 00:20:48.504838 ip-10-0-132-188 kubenswrapper[2562]: I0918 00:20:48.504804    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve_seaweedfs-64568bcd49-xx74p_d1bccb49-bf6c-4f3b-aa32-0673a7ce5651/seaweedfs/0.log"
Sep 18 00:20:49.462210 ip-10-0-132-188 kubenswrapper[2562]: I0918 00:20:49.462154    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_istiod-openshift-gateway-5677ccb49b-np75w_eb73a80f-c670-434f-9ddb-7953608877b2/discovery/0.log"
Sep 18 00:20:49.481009 ip-10-0-132-188 kubenswrapper[2562]: I0918 00:20:49.480976    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_openshift-ai-inference-openshift-default-58c5cd85c9-9wjvx_bafca983-9e23-438d-b13c-ff56f8b81b6b/istio-proxy/0.log"
Sep 18 00:20:49.486448 ip-10-0-132-188 kubenswrapper[2562]: I0918 00:20:49.486419    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_openshift-ai-inference-openshift-default-58c5cd85c9-9wjvx_bafca983-9e23-438d-b13c-ff56f8b81b6b/istio-proxy/1.log"
Sep 18 00:20:50.744561 ip-10-0-132-188 kubenswrapper[2562]: I0918 00:20:50.744523    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_istiod-openshift-gateway-5677ccb49b-np75w_eb73a80f-c670-434f-9ddb-7953608877b2/discovery/0.log"
Sep 18 00:20:50.764712 ip-10-0-132-188 kubenswrapper[2562]: I0918 00:20:50.764614    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_openshift-ai-inference-openshift-default-58c5cd85c9-9wjvx_bafca983-9e23-438d-b13c-ff56f8b81b6b/istio-proxy/0.log"
Sep 18 00:20:50.769751 ip-10-0-132-188 kubenswrapper[2562]: I0918 00:20:50.769708    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_openshift-ai-inference-openshift-default-58c5cd85c9-9wjvx_bafca983-9e23-438d-b13c-ff56f8b81b6b/istio-proxy/1.log"
Sep 18 00:20:55.021954 ip-10-0-132-188 kubenswrapper[2562]: I0918 00:20:55.021926    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-srxwj_215c7cbb-4cd1-47be-9a44-26c1801b3fcb/kube-multus-additional-cni-plugins/0.log"
Sep 18 00:20:55.040352 ip-10-0-132-188 kubenswrapper[2562]: I0918 00:20:55.040309    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-srxwj_215c7cbb-4cd1-47be-9a44-26c1801b3fcb/egress-router-binary-copy/0.log"
Sep 18 00:20:55.058782 ip-10-0-132-188 kubenswrapper[2562]: I0918 00:20:55.058748    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-srxwj_215c7cbb-4cd1-47be-9a44-26c1801b3fcb/cni-plugins/0.log"
Sep 18 00:20:55.075271 ip-10-0-132-188 kubenswrapper[2562]: I0918 00:20:55.075237    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-srxwj_215c7cbb-4cd1-47be-9a44-26c1801b3fcb/bond-cni-plugin/0.log"
Sep 18 00:20:55.092835 ip-10-0-132-188 kubenswrapper[2562]: I0918 00:20:55.092799    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-srxwj_215c7cbb-4cd1-47be-9a44-26c1801b3fcb/routeoverride-cni/0.log"
Sep 18 00:20:55.110129 ip-10-0-132-188 kubenswrapper[2562]: I0918 00:20:55.110098    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-srxwj_215c7cbb-4cd1-47be-9a44-26c1801b3fcb/whereabouts-cni-bincopy/0.log"
Sep 18 00:20:55.128252 ip-10-0-132-188 kubenswrapper[2562]: I0918 00:20:55.128217    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-srxwj_215c7cbb-4cd1-47be-9a44-26c1801b3fcb/whereabouts-cni/0.log"
Sep 18 00:20:55.175867 ip-10-0-132-188 kubenswrapper[2562]: I0918 00:20:55.175828    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-s9wtj_914194c1-d316-4f82-9beb-af7827f4a570/kube-multus/0.log"
Sep 18 00:20:55.274244 ip-10-0-132-188 kubenswrapper[2562]: I0918 00:20:55.274130    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_network-metrics-daemon-nbccd_4453ba96-a67f-46af-b61b-9cd8f39b6d36/network-metrics-daemon/0.log"
Sep 18 00:20:55.287526 ip-10-0-132-188 kubenswrapper[2562]: I0918 00:20:55.287492    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_network-metrics-daemon-nbccd_4453ba96-a67f-46af-b61b-9cd8f39b6d36/kube-rbac-proxy/0.log"
Sep 18 00:20:56.100919 ip-10-0-132-188 kubenswrapper[2562]: I0918 00:20:56.100880    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-6xqwg_cae8ed58-1a8a-40af-a329-a3850797cc58/ovn-controller/0.log"
Sep 18 00:20:56.117764 ip-10-0-132-188 kubenswrapper[2562]: I0918 00:20:56.117732    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-6xqwg_cae8ed58-1a8a-40af-a329-a3850797cc58/ovn-acl-logging/0.log"
Sep 18 00:20:56.143730 ip-10-0-132-188 kubenswrapper[2562]: I0918 00:20:56.143689    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-6xqwg_cae8ed58-1a8a-40af-a329-a3850797cc58/kube-rbac-proxy-node/0.log"
Sep 18 00:20:56.166760 ip-10-0-132-188 kubenswrapper[2562]: I0918 00:20:56.166715    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-6xqwg_cae8ed58-1a8a-40af-a329-a3850797cc58/kube-rbac-proxy-ovn-metrics/0.log"
Sep 18 00:20:56.181887 ip-10-0-132-188 kubenswrapper[2562]: I0918 00:20:56.181849    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-6xqwg_cae8ed58-1a8a-40af-a329-a3850797cc58/northd/0.log"
Sep 18 00:20:56.200286 ip-10-0-132-188 kubenswrapper[2562]: I0918 00:20:56.200255    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-6xqwg_cae8ed58-1a8a-40af-a329-a3850797cc58/nbdb/0.log"
Sep 18 00:20:56.217985 ip-10-0-132-188 kubenswrapper[2562]: I0918 00:20:56.217959    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-6xqwg_cae8ed58-1a8a-40af-a329-a3850797cc58/sbdb/0.log"
Sep 18 00:20:56.297927 ip-10-0-132-188 kubenswrapper[2562]: I0918 00:20:56.297879    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-6xqwg_cae8ed58-1a8a-40af-a329-a3850797cc58/ovnkube-controller/0.log"
Sep 18 00:20:57.731226 ip-10-0-132-188 kubenswrapper[2562]: I0918 00:20:57.731188    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-network-diagnostics_network-check-target-t9gqw_440bce33-9c27-46bd-9d1c-0eebcc14fbc3/network-check-target-container/0.log"