Sep 17 17:32:31.710928 ip-10-0-143-74 systemd[1]: kubelet.service: Failed to load environment files: No such file or directory
Sep 17 17:32:31.710939 ip-10-0-143-74 systemd[1]: kubelet.service: Failed to run 'start-pre' task: No such file or directory
Sep 17 17:32:31.710947 ip-10-0-143-74 systemd[1]: kubelet.service: Failed with result 'resources'.
Sep 17 17:32:31.711133 ip-10-0-143-74 systemd[1]: Failed to start Kubernetes Kubelet.
Sep 17 17:32:43.132656 ip-10-0-143-74 systemd[1]: kubelet.service: Failed to schedule restart job: Unit crio.service not found.
Sep 17 17:32:43.132674 ip-10-0-143-74 systemd[1]: kubelet.service: Failed with result 'resources'.
-- Boot 546f2c028f2d488fb67b7a883478d383 --
Sep 17 17:34:40.257738 ip-10-0-143-74 systemd[1]: Starting Kubernetes Kubelet...
Sep 17 17:34:40.644140 ip-10-0-143-74 kubenswrapper[2575]: Flag --container-runtime-endpoint has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Sep 17 17:34:40.644140 ip-10-0-143-74 kubenswrapper[2575]: Flag --minimum-container-ttl-duration has been deprecated, Use --eviction-hard or --eviction-soft instead. Will be removed in a future version.
Sep 17 17:34:40.644140 ip-10-0-143-74 kubenswrapper[2575]: Flag --volume-plugin-dir has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Sep 17 17:34:40.644140 ip-10-0-143-74 kubenswrapper[2575]: Flag --pod-infra-container-image has been deprecated, will be removed in 1.35. Image garbage collector will get sandbox image information from CRI.
Sep 17 17:34:40.644140 ip-10-0-143-74 kubenswrapper[2575]: Flag --system-reserved has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Sep 17 17:34:40.644865 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:34:40.644783    2575 server.go:213] "--pod-infra-container-image will not be pruned by the image garbage collector in kubelet and should also be set in the remote runtime"
Sep 17 17:34:40.649935 ip-10-0-143-74 kubenswrapper[2575]: W0917 17:34:40.649914    2575 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Sep 17 17:34:40.649935 ip-10-0-143-74 kubenswrapper[2575]: W0917 17:34:40.649931    2575 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Sep 17 17:34:40.649935 ip-10-0-143-74 kubenswrapper[2575]: W0917 17:34:40.649936    2575 feature_gate.go:328] unrecognized feature gate: DualReplica
Sep 17 17:34:40.649935 ip-10-0-143-74 kubenswrapper[2575]: W0917 17:34:40.649939    2575 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Sep 17 17:34:40.650106 ip-10-0-143-74 kubenswrapper[2575]: W0917 17:34:40.649943    2575 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Sep 17 17:34:40.650106 ip-10-0-143-74 kubenswrapper[2575]: W0917 17:34:40.649946    2575 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Sep 17 17:34:40.650106 ip-10-0-143-74 kubenswrapper[2575]: W0917 17:34:40.649949    2575 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Sep 17 17:34:40.650106 ip-10-0-143-74 kubenswrapper[2575]: W0917 17:34:40.649952    2575 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Sep 17 17:34:40.650106 ip-10-0-143-74 kubenswrapper[2575]: W0917 17:34:40.649956    2575 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Sep 17 17:34:40.650106 ip-10-0-143-74 kubenswrapper[2575]: W0917 17:34:40.649959    2575 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Sep 17 17:34:40.650106 ip-10-0-143-74 kubenswrapper[2575]: W0917 17:34:40.649962    2575 feature_gate.go:328] unrecognized feature gate: OVNObservability
Sep 17 17:34:40.650106 ip-10-0-143-74 kubenswrapper[2575]: W0917 17:34:40.649964    2575 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Sep 17 17:34:40.650106 ip-10-0-143-74 kubenswrapper[2575]: W0917 17:34:40.649968    2575 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Sep 17 17:34:40.650106 ip-10-0-143-74 kubenswrapper[2575]: W0917 17:34:40.649971    2575 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Sep 17 17:34:40.650106 ip-10-0-143-74 kubenswrapper[2575]: W0917 17:34:40.649973    2575 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Sep 17 17:34:40.650106 ip-10-0-143-74 kubenswrapper[2575]: W0917 17:34:40.649976    2575 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Sep 17 17:34:40.650106 ip-10-0-143-74 kubenswrapper[2575]: W0917 17:34:40.649979    2575 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Sep 17 17:34:40.650106 ip-10-0-143-74 kubenswrapper[2575]: W0917 17:34:40.649982    2575 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Sep 17 17:34:40.650106 ip-10-0-143-74 kubenswrapper[2575]: W0917 17:34:40.649984    2575 feature_gate.go:328] unrecognized feature gate: PinnedImages
Sep 17 17:34:40.650106 ip-10-0-143-74 kubenswrapper[2575]: W0917 17:34:40.649987    2575 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Sep 17 17:34:40.650106 ip-10-0-143-74 kubenswrapper[2575]: W0917 17:34:40.649989    2575 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Sep 17 17:34:40.650106 ip-10-0-143-74 kubenswrapper[2575]: W0917 17:34:40.649992    2575 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Sep 17 17:34:40.650106 ip-10-0-143-74 kubenswrapper[2575]: W0917 17:34:40.649995    2575 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Sep 17 17:34:40.650583 ip-10-0-143-74 kubenswrapper[2575]: W0917 17:34:40.649998    2575 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Sep 17 17:34:40.650583 ip-10-0-143-74 kubenswrapper[2575]: W0917 17:34:40.650000    2575 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Sep 17 17:34:40.650583 ip-10-0-143-74 kubenswrapper[2575]: W0917 17:34:40.650003    2575 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Sep 17 17:34:40.650583 ip-10-0-143-74 kubenswrapper[2575]: W0917 17:34:40.650005    2575 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Sep 17 17:34:40.650583 ip-10-0-143-74 kubenswrapper[2575]: W0917 17:34:40.650008    2575 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Sep 17 17:34:40.650583 ip-10-0-143-74 kubenswrapper[2575]: W0917 17:34:40.650014    2575 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Sep 17 17:34:40.650583 ip-10-0-143-74 kubenswrapper[2575]: W0917 17:34:40.650018    2575 feature_gate.go:328] unrecognized feature gate: NewOLM
Sep 17 17:34:40.650583 ip-10-0-143-74 kubenswrapper[2575]: W0917 17:34:40.650022    2575 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Sep 17 17:34:40.650583 ip-10-0-143-74 kubenswrapper[2575]: W0917 17:34:40.650025    2575 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Sep 17 17:34:40.650583 ip-10-0-143-74 kubenswrapper[2575]: W0917 17:34:40.650029    2575 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Sep 17 17:34:40.650583 ip-10-0-143-74 kubenswrapper[2575]: W0917 17:34:40.650032    2575 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Sep 17 17:34:40.650583 ip-10-0-143-74 kubenswrapper[2575]: W0917 17:34:40.650035    2575 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Sep 17 17:34:40.650583 ip-10-0-143-74 kubenswrapper[2575]: W0917 17:34:40.650038    2575 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Sep 17 17:34:40.650583 ip-10-0-143-74 kubenswrapper[2575]: W0917 17:34:40.650040    2575 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Sep 17 17:34:40.650583 ip-10-0-143-74 kubenswrapper[2575]: W0917 17:34:40.650043    2575 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Sep 17 17:34:40.650583 ip-10-0-143-74 kubenswrapper[2575]: W0917 17:34:40.650046    2575 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Sep 17 17:34:40.650583 ip-10-0-143-74 kubenswrapper[2575]: W0917 17:34:40.650048    2575 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Sep 17 17:34:40.650583 ip-10-0-143-74 kubenswrapper[2575]: W0917 17:34:40.650051    2575 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Sep 17 17:34:40.650583 ip-10-0-143-74 kubenswrapper[2575]: W0917 17:34:40.650054    2575 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Sep 17 17:34:40.651087 ip-10-0-143-74 kubenswrapper[2575]: W0917 17:34:40.650056    2575 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Sep 17 17:34:40.651087 ip-10-0-143-74 kubenswrapper[2575]: W0917 17:34:40.650059    2575 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Sep 17 17:34:40.651087 ip-10-0-143-74 kubenswrapper[2575]: W0917 17:34:40.650062    2575 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Sep 17 17:34:40.651087 ip-10-0-143-74 kubenswrapper[2575]: W0917 17:34:40.650065    2575 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Sep 17 17:34:40.651087 ip-10-0-143-74 kubenswrapper[2575]: W0917 17:34:40.650068    2575 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Sep 17 17:34:40.651087 ip-10-0-143-74 kubenswrapper[2575]: W0917 17:34:40.650070    2575 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Sep 17 17:34:40.651087 ip-10-0-143-74 kubenswrapper[2575]: W0917 17:34:40.650073    2575 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Sep 17 17:34:40.651087 ip-10-0-143-74 kubenswrapper[2575]: W0917 17:34:40.650076    2575 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Sep 17 17:34:40.651087 ip-10-0-143-74 kubenswrapper[2575]: W0917 17:34:40.650079    2575 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Sep 17 17:34:40.651087 ip-10-0-143-74 kubenswrapper[2575]: W0917 17:34:40.650081    2575 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Sep 17 17:34:40.651087 ip-10-0-143-74 kubenswrapper[2575]: W0917 17:34:40.650084    2575 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Sep 17 17:34:40.651087 ip-10-0-143-74 kubenswrapper[2575]: W0917 17:34:40.650086    2575 feature_gate.go:328] unrecognized feature gate: OSStreams
Sep 17 17:34:40.651087 ip-10-0-143-74 kubenswrapper[2575]: W0917 17:34:40.650089    2575 feature_gate.go:328] unrecognized feature gate: SignatureStores
Sep 17 17:34:40.651087 ip-10-0-143-74 kubenswrapper[2575]: W0917 17:34:40.650092    2575 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Sep 17 17:34:40.651087 ip-10-0-143-74 kubenswrapper[2575]: W0917 17:34:40.650095    2575 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Sep 17 17:34:40.651087 ip-10-0-143-74 kubenswrapper[2575]: W0917 17:34:40.650098    2575 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Sep 17 17:34:40.651087 ip-10-0-143-74 kubenswrapper[2575]: W0917 17:34:40.650100    2575 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Sep 17 17:34:40.651087 ip-10-0-143-74 kubenswrapper[2575]: W0917 17:34:40.650103    2575 feature_gate.go:328] unrecognized feature gate: Example2
Sep 17 17:34:40.651087 ip-10-0-143-74 kubenswrapper[2575]: W0917 17:34:40.650106    2575 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Sep 17 17:34:40.651087 ip-10-0-143-74 kubenswrapper[2575]: W0917 17:34:40.650108    2575 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Sep 17 17:34:40.651593 ip-10-0-143-74 kubenswrapper[2575]: W0917 17:34:40.650112    2575 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Sep 17 17:34:40.651593 ip-10-0-143-74 kubenswrapper[2575]: W0917 17:34:40.650117    2575 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Sep 17 17:34:40.651593 ip-10-0-143-74 kubenswrapper[2575]: W0917 17:34:40.650121    2575 feature_gate.go:328] unrecognized feature gate: EventTTL
Sep 17 17:34:40.651593 ip-10-0-143-74 kubenswrapper[2575]: W0917 17:34:40.650125    2575 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Sep 17 17:34:40.651593 ip-10-0-143-74 kubenswrapper[2575]: W0917 17:34:40.650128    2575 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Sep 17 17:34:40.651593 ip-10-0-143-74 kubenswrapper[2575]: W0917 17:34:40.650131    2575 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Sep 17 17:34:40.651593 ip-10-0-143-74 kubenswrapper[2575]: W0917 17:34:40.650134    2575 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Sep 17 17:34:40.651593 ip-10-0-143-74 kubenswrapper[2575]: W0917 17:34:40.650137    2575 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Sep 17 17:34:40.651593 ip-10-0-143-74 kubenswrapper[2575]: W0917 17:34:40.650140    2575 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Sep 17 17:34:40.651593 ip-10-0-143-74 kubenswrapper[2575]: W0917 17:34:40.650142    2575 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Sep 17 17:34:40.651593 ip-10-0-143-74 kubenswrapper[2575]: W0917 17:34:40.650145    2575 feature_gate.go:328] unrecognized feature gate: Example
Sep 17 17:34:40.651593 ip-10-0-143-74 kubenswrapper[2575]: W0917 17:34:40.650148    2575 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Sep 17 17:34:40.651593 ip-10-0-143-74 kubenswrapper[2575]: W0917 17:34:40.650150    2575 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Sep 17 17:34:40.651593 ip-10-0-143-74 kubenswrapper[2575]: W0917 17:34:40.650153    2575 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Sep 17 17:34:40.651593 ip-10-0-143-74 kubenswrapper[2575]: W0917 17:34:40.650156    2575 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Sep 17 17:34:40.651593 ip-10-0-143-74 kubenswrapper[2575]: W0917 17:34:40.650159    2575 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Sep 17 17:34:40.651593 ip-10-0-143-74 kubenswrapper[2575]: W0917 17:34:40.650163    2575 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Sep 17 17:34:40.651593 ip-10-0-143-74 kubenswrapper[2575]: W0917 17:34:40.650165    2575 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Sep 17 17:34:40.651593 ip-10-0-143-74 kubenswrapper[2575]: W0917 17:34:40.650168    2575 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Sep 17 17:34:40.652054 ip-10-0-143-74 kubenswrapper[2575]: W0917 17:34:40.650171    2575 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Sep 17 17:34:40.652054 ip-10-0-143-74 kubenswrapper[2575]: W0917 17:34:40.650174    2575 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Sep 17 17:34:40.652054 ip-10-0-143-74 kubenswrapper[2575]: W0917 17:34:40.650178    2575 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Sep 17 17:34:40.652054 ip-10-0-143-74 kubenswrapper[2575]: W0917 17:34:40.650181    2575 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Sep 17 17:34:40.652054 ip-10-0-143-74 kubenswrapper[2575]: W0917 17:34:40.650184    2575 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Sep 17 17:34:40.652054 ip-10-0-143-74 kubenswrapper[2575]: W0917 17:34:40.650187    2575 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Sep 17 17:34:40.652054 ip-10-0-143-74 kubenswrapper[2575]: W0917 17:34:40.650189    2575 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Sep 17 17:34:40.652054 ip-10-0-143-74 kubenswrapper[2575]: W0917 17:34:40.650192    2575 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Sep 17 17:34:40.652054 ip-10-0-143-74 kubenswrapper[2575]: W0917 17:34:40.650195    2575 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Sep 17 17:34:40.652054 ip-10-0-143-74 kubenswrapper[2575]: W0917 17:34:40.650198    2575 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Sep 17 17:34:40.652054 ip-10-0-143-74 kubenswrapper[2575]: W0917 17:34:40.650200    2575 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Sep 17 17:34:40.652054 ip-10-0-143-74 kubenswrapper[2575]: W0917 17:34:40.650203    2575 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Sep 17 17:34:40.652054 ip-10-0-143-74 kubenswrapper[2575]: W0917 17:34:40.650206    2575 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Sep 17 17:34:40.652054 ip-10-0-143-74 kubenswrapper[2575]: W0917 17:34:40.650208    2575 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Sep 17 17:34:40.652054 ip-10-0-143-74 kubenswrapper[2575]: W0917 17:34:40.650211    2575 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Sep 17 17:34:40.652054 ip-10-0-143-74 kubenswrapper[2575]: W0917 17:34:40.650213    2575 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Sep 17 17:34:40.652054 ip-10-0-143-74 kubenswrapper[2575]: W0917 17:34:40.650217    2575 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Sep 17 17:34:40.652054 ip-10-0-143-74 kubenswrapper[2575]: W0917 17:34:40.650220    2575 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Sep 17 17:34:40.652054 ip-10-0-143-74 kubenswrapper[2575]: W0917 17:34:40.650938    2575 feature_gate.go:328] unrecognized feature gate: OSStreams
Sep 17 17:34:40.652054 ip-10-0-143-74 kubenswrapper[2575]: W0917 17:34:40.650945    2575 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Sep 17 17:34:40.652640 ip-10-0-143-74 kubenswrapper[2575]: W0917 17:34:40.650948    2575 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Sep 17 17:34:40.652640 ip-10-0-143-74 kubenswrapper[2575]: W0917 17:34:40.650951    2575 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Sep 17 17:34:40.652640 ip-10-0-143-74 kubenswrapper[2575]: W0917 17:34:40.650954    2575 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Sep 17 17:34:40.652640 ip-10-0-143-74 kubenswrapper[2575]: W0917 17:34:40.650956    2575 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Sep 17 17:34:40.652640 ip-10-0-143-74 kubenswrapper[2575]: W0917 17:34:40.650959    2575 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Sep 17 17:34:40.652640 ip-10-0-143-74 kubenswrapper[2575]: W0917 17:34:40.650962    2575 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Sep 17 17:34:40.652640 ip-10-0-143-74 kubenswrapper[2575]: W0917 17:34:40.650965    2575 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Sep 17 17:34:40.652640 ip-10-0-143-74 kubenswrapper[2575]: W0917 17:34:40.650967    2575 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Sep 17 17:34:40.652640 ip-10-0-143-74 kubenswrapper[2575]: W0917 17:34:40.650972    2575 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Sep 17 17:34:40.652640 ip-10-0-143-74 kubenswrapper[2575]: W0917 17:34:40.650976    2575 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Sep 17 17:34:40.652640 ip-10-0-143-74 kubenswrapper[2575]: W0917 17:34:40.650978    2575 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Sep 17 17:34:40.652640 ip-10-0-143-74 kubenswrapper[2575]: W0917 17:34:40.650981    2575 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Sep 17 17:34:40.652640 ip-10-0-143-74 kubenswrapper[2575]: W0917 17:34:40.650984    2575 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Sep 17 17:34:40.652640 ip-10-0-143-74 kubenswrapper[2575]: W0917 17:34:40.650986    2575 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Sep 17 17:34:40.652640 ip-10-0-143-74 kubenswrapper[2575]: W0917 17:34:40.650989    2575 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Sep 17 17:34:40.652640 ip-10-0-143-74 kubenswrapper[2575]: W0917 17:34:40.650992    2575 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Sep 17 17:34:40.652640 ip-10-0-143-74 kubenswrapper[2575]: W0917 17:34:40.650995    2575 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Sep 17 17:34:40.652640 ip-10-0-143-74 kubenswrapper[2575]: W0917 17:34:40.650998    2575 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Sep 17 17:34:40.652640 ip-10-0-143-74 kubenswrapper[2575]: W0917 17:34:40.651000    2575 feature_gate.go:328] unrecognized feature gate: NewOLM
Sep 17 17:34:40.652640 ip-10-0-143-74 kubenswrapper[2575]: W0917 17:34:40.651003    2575 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Sep 17 17:34:40.653142 ip-10-0-143-74 kubenswrapper[2575]: W0917 17:34:40.651005    2575 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Sep 17 17:34:40.653142 ip-10-0-143-74 kubenswrapper[2575]: W0917 17:34:40.651008    2575 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Sep 17 17:34:40.653142 ip-10-0-143-74 kubenswrapper[2575]: W0917 17:34:40.651011    2575 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Sep 17 17:34:40.653142 ip-10-0-143-74 kubenswrapper[2575]: W0917 17:34:40.651014    2575 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Sep 17 17:34:40.653142 ip-10-0-143-74 kubenswrapper[2575]: W0917 17:34:40.651016    2575 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Sep 17 17:34:40.653142 ip-10-0-143-74 kubenswrapper[2575]: W0917 17:34:40.651018    2575 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Sep 17 17:34:40.653142 ip-10-0-143-74 kubenswrapper[2575]: W0917 17:34:40.651021    2575 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Sep 17 17:34:40.653142 ip-10-0-143-74 kubenswrapper[2575]: W0917 17:34:40.651024    2575 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Sep 17 17:34:40.653142 ip-10-0-143-74 kubenswrapper[2575]: W0917 17:34:40.651027    2575 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Sep 17 17:34:40.653142 ip-10-0-143-74 kubenswrapper[2575]: W0917 17:34:40.651029    2575 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Sep 17 17:34:40.653142 ip-10-0-143-74 kubenswrapper[2575]: W0917 17:34:40.651032    2575 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Sep 17 17:34:40.653142 ip-10-0-143-74 kubenswrapper[2575]: W0917 17:34:40.651035    2575 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Sep 17 17:34:40.653142 ip-10-0-143-74 kubenswrapper[2575]: W0917 17:34:40.651037    2575 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Sep 17 17:34:40.653142 ip-10-0-143-74 kubenswrapper[2575]: W0917 17:34:40.651040    2575 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Sep 17 17:34:40.653142 ip-10-0-143-74 kubenswrapper[2575]: W0917 17:34:40.651043    2575 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Sep 17 17:34:40.653142 ip-10-0-143-74 kubenswrapper[2575]: W0917 17:34:40.651045    2575 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Sep 17 17:34:40.653142 ip-10-0-143-74 kubenswrapper[2575]: W0917 17:34:40.651049    2575 feature_gate.go:328] unrecognized feature gate: SignatureStores
Sep 17 17:34:40.653142 ip-10-0-143-74 kubenswrapper[2575]: W0917 17:34:40.651051    2575 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Sep 17 17:34:40.653142 ip-10-0-143-74 kubenswrapper[2575]: W0917 17:34:40.651054    2575 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Sep 17 17:34:40.653629 ip-10-0-143-74 kubenswrapper[2575]: W0917 17:34:40.651056    2575 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Sep 17 17:34:40.653629 ip-10-0-143-74 kubenswrapper[2575]: W0917 17:34:40.651060    2575 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Sep 17 17:34:40.653629 ip-10-0-143-74 kubenswrapper[2575]: W0917 17:34:40.651063    2575 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Sep 17 17:34:40.653629 ip-10-0-143-74 kubenswrapper[2575]: W0917 17:34:40.651065    2575 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Sep 17 17:34:40.653629 ip-10-0-143-74 kubenswrapper[2575]: W0917 17:34:40.651068    2575 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Sep 17 17:34:40.653629 ip-10-0-143-74 kubenswrapper[2575]: W0917 17:34:40.651070    2575 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Sep 17 17:34:40.653629 ip-10-0-143-74 kubenswrapper[2575]: W0917 17:34:40.651073    2575 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Sep 17 17:34:40.653629 ip-10-0-143-74 kubenswrapper[2575]: W0917 17:34:40.651076    2575 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Sep 17 17:34:40.653629 ip-10-0-143-74 kubenswrapper[2575]: W0917 17:34:40.651079    2575 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Sep 17 17:34:40.653629 ip-10-0-143-74 kubenswrapper[2575]: W0917 17:34:40.651082    2575 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Sep 17 17:34:40.653629 ip-10-0-143-74 kubenswrapper[2575]: W0917 17:34:40.651085    2575 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Sep 17 17:34:40.653629 ip-10-0-143-74 kubenswrapper[2575]: W0917 17:34:40.651088    2575 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Sep 17 17:34:40.653629 ip-10-0-143-74 kubenswrapper[2575]: W0917 17:34:40.651091    2575 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Sep 17 17:34:40.653629 ip-10-0-143-74 kubenswrapper[2575]: W0917 17:34:40.651094    2575 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Sep 17 17:34:40.653629 ip-10-0-143-74 kubenswrapper[2575]: W0917 17:34:40.651097    2575 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Sep 17 17:34:40.653629 ip-10-0-143-74 kubenswrapper[2575]: W0917 17:34:40.651100    2575 feature_gate.go:328] unrecognized feature gate: EventTTL
Sep 17 17:34:40.653629 ip-10-0-143-74 kubenswrapper[2575]: W0917 17:34:40.651102    2575 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Sep 17 17:34:40.653629 ip-10-0-143-74 kubenswrapper[2575]: W0917 17:34:40.651107    2575 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Sep 17 17:34:40.653629 ip-10-0-143-74 kubenswrapper[2575]: W0917 17:34:40.651110    2575 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Sep 17 17:34:40.654094 ip-10-0-143-74 kubenswrapper[2575]: W0917 17:34:40.651113    2575 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Sep 17 17:34:40.654094 ip-10-0-143-74 kubenswrapper[2575]: W0917 17:34:40.651116    2575 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Sep 17 17:34:40.654094 ip-10-0-143-74 kubenswrapper[2575]: W0917 17:34:40.651119    2575 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Sep 17 17:34:40.654094 ip-10-0-143-74 kubenswrapper[2575]: W0917 17:34:40.651122    2575 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Sep 17 17:34:40.654094 ip-10-0-143-74 kubenswrapper[2575]: W0917 17:34:40.651125    2575 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Sep 17 17:34:40.654094 ip-10-0-143-74 kubenswrapper[2575]: W0917 17:34:40.651128    2575 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Sep 17 17:34:40.654094 ip-10-0-143-74 kubenswrapper[2575]: W0917 17:34:40.651131    2575 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Sep 17 17:34:40.654094 ip-10-0-143-74 kubenswrapper[2575]: W0917 17:34:40.651134    2575 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Sep 17 17:34:40.654094 ip-10-0-143-74 kubenswrapper[2575]: W0917 17:34:40.651136    2575 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Sep 17 17:34:40.654094 ip-10-0-143-74 kubenswrapper[2575]: W0917 17:34:40.651141    2575 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Sep 17 17:34:40.654094 ip-10-0-143-74 kubenswrapper[2575]: W0917 17:34:40.651143    2575 feature_gate.go:328] unrecognized feature gate: PinnedImages
Sep 17 17:34:40.654094 ip-10-0-143-74 kubenswrapper[2575]: W0917 17:34:40.651146    2575 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Sep 17 17:34:40.654094 ip-10-0-143-74 kubenswrapper[2575]: W0917 17:34:40.651148    2575 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Sep 17 17:34:40.654094 ip-10-0-143-74 kubenswrapper[2575]: W0917 17:34:40.651151    2575 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Sep 17 17:34:40.654094 ip-10-0-143-74 kubenswrapper[2575]: W0917 17:34:40.651154    2575 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Sep 17 17:34:40.654094 ip-10-0-143-74 kubenswrapper[2575]: W0917 17:34:40.651157    2575 feature_gate.go:328] unrecognized feature gate: Example2
Sep 17 17:34:40.654094 ip-10-0-143-74 kubenswrapper[2575]: W0917 17:34:40.651160    2575 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Sep 17 17:34:40.654094 ip-10-0-143-74 kubenswrapper[2575]: W0917 17:34:40.651162    2575 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Sep 17 17:34:40.654094 ip-10-0-143-74 kubenswrapper[2575]: W0917 17:34:40.651165    2575 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Sep 17 17:34:40.654094 ip-10-0-143-74 kubenswrapper[2575]: W0917 17:34:40.651168    2575 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Sep 17 17:34:40.654621 ip-10-0-143-74 kubenswrapper[2575]: W0917 17:34:40.651170    2575 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Sep 17 17:34:40.654621 ip-10-0-143-74 kubenswrapper[2575]: W0917 17:34:40.651173    2575 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Sep 17 17:34:40.654621 ip-10-0-143-74 kubenswrapper[2575]: W0917 17:34:40.651175    2575 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Sep 17 17:34:40.654621 ip-10-0-143-74 kubenswrapper[2575]: W0917 17:34:40.651178    2575 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Sep 17 17:34:40.654621 ip-10-0-143-74 kubenswrapper[2575]: W0917 17:34:40.651181    2575 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Sep 17 17:34:40.654621 ip-10-0-143-74 kubenswrapper[2575]: W0917 17:34:40.651183    2575 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Sep 17 17:34:40.654621 ip-10-0-143-74 kubenswrapper[2575]: W0917 17:34:40.651186    2575 feature_gate.go:328] unrecognized feature gate: Example
Sep 17 17:34:40.654621 ip-10-0-143-74 kubenswrapper[2575]: W0917 17:34:40.651189    2575 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Sep 17 17:34:40.654621 ip-10-0-143-74 kubenswrapper[2575]: W0917 17:34:40.651191    2575 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Sep 17 17:34:40.654621 ip-10-0-143-74 kubenswrapper[2575]: W0917 17:34:40.651194    2575 feature_gate.go:328] unrecognized feature gate: DualReplica
Sep 17 17:34:40.654621 ip-10-0-143-74 kubenswrapper[2575]: W0917 17:34:40.651196    2575 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Sep 17 17:34:40.654621 ip-10-0-143-74 kubenswrapper[2575]: W0917 17:34:40.651199    2575 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Sep 17 17:34:40.654621 ip-10-0-143-74 kubenswrapper[2575]: W0917 17:34:40.651202    2575 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Sep 17 17:34:40.654621 ip-10-0-143-74 kubenswrapper[2575]: W0917 17:34:40.651204    2575 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Sep 17 17:34:40.654621 ip-10-0-143-74 kubenswrapper[2575]: W0917 17:34:40.651207    2575 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Sep 17 17:34:40.654621 ip-10-0-143-74 kubenswrapper[2575]: W0917 17:34:40.651210    2575 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Sep 17 17:34:40.654621 ip-10-0-143-74 kubenswrapper[2575]: W0917 17:34:40.651214    2575 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Sep 17 17:34:40.654621 ip-10-0-143-74 kubenswrapper[2575]: W0917 17:34:40.651217    2575 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Sep 17 17:34:40.654621 ip-10-0-143-74 kubenswrapper[2575]: W0917 17:34:40.651220    2575 feature_gate.go:328] unrecognized feature gate: OVNObservability
Sep 17 17:34:40.655084 ip-10-0-143-74 kubenswrapper[2575]: W0917 17:34:40.651708    2575 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Sep 17 17:34:40.655084 ip-10-0-143-74 kubenswrapper[2575]: W0917 17:34:40.651715    2575 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Sep 17 17:34:40.655084 ip-10-0-143-74 kubenswrapper[2575]: W0917 17:34:40.651718    2575 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Sep 17 17:34:40.655084 ip-10-0-143-74 kubenswrapper[2575]: W0917 17:34:40.651722    2575 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Sep 17 17:34:40.655084 ip-10-0-143-74 kubenswrapper[2575]: W0917 17:34:40.651725    2575 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Sep 17 17:34:40.655084 ip-10-0-143-74 kubenswrapper[2575]: W0917 17:34:40.651727    2575 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Sep 17 17:34:40.655084 ip-10-0-143-74 kubenswrapper[2575]: W0917 17:34:40.651730    2575 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Sep 17 17:34:40.655084 ip-10-0-143-74 kubenswrapper[2575]: W0917 17:34:40.651733    2575 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Sep 17 17:34:40.655084 ip-10-0-143-74 kubenswrapper[2575]: W0917 17:34:40.651736    2575 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Sep 17 17:34:40.655084 ip-10-0-143-74 kubenswrapper[2575]: W0917 17:34:40.651738    2575 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Sep 17 17:34:40.655084 ip-10-0-143-74 kubenswrapper[2575]: W0917 17:34:40.651741    2575 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Sep 17 17:34:40.655084 ip-10-0-143-74 kubenswrapper[2575]: W0917 17:34:40.651744    2575 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Sep 17 17:34:40.655084 ip-10-0-143-74 kubenswrapper[2575]: W0917 17:34:40.651747    2575 feature_gate.go:328] unrecognized feature gate: SignatureStores
Sep 17 17:34:40.655084 ip-10-0-143-74 kubenswrapper[2575]: W0917 17:34:40.651749    2575 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Sep 17 17:34:40.655084 ip-10-0-143-74 kubenswrapper[2575]: W0917 17:34:40.651752    2575 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Sep 17 17:34:40.655084 ip-10-0-143-74 kubenswrapper[2575]: W0917 17:34:40.651755    2575 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Sep 17 17:34:40.655084 ip-10-0-143-74 kubenswrapper[2575]: W0917 17:34:40.651758    2575 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Sep 17 17:34:40.655084 ip-10-0-143-74 kubenswrapper[2575]: W0917 17:34:40.651761    2575 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Sep 17 17:34:40.655084 ip-10-0-143-74 kubenswrapper[2575]: W0917 17:34:40.651764    2575 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Sep 17 17:34:40.655084 ip-10-0-143-74 kubenswrapper[2575]: W0917 17:34:40.651766    2575 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Sep 17 17:34:40.655597 ip-10-0-143-74 kubenswrapper[2575]: W0917 17:34:40.651769    2575 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Sep 17 17:34:40.655597 ip-10-0-143-74 kubenswrapper[2575]: W0917 17:34:40.651772    2575 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Sep 17 17:34:40.655597 ip-10-0-143-74 kubenswrapper[2575]: W0917 17:34:40.651774    2575 feature_gate.go:328] unrecognized feature gate: Example2
Sep 17 17:34:40.655597 ip-10-0-143-74 kubenswrapper[2575]: W0917 17:34:40.651779    2575 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Sep 17 17:34:40.655597 ip-10-0-143-74 kubenswrapper[2575]: W0917 17:34:40.651782    2575 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Sep 17 17:34:40.655597 ip-10-0-143-74 kubenswrapper[2575]: W0917 17:34:40.651786    2575 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Sep 17 17:34:40.655597 ip-10-0-143-74 kubenswrapper[2575]: W0917 17:34:40.651789    2575 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Sep 17 17:34:40.655597 ip-10-0-143-74 kubenswrapper[2575]: W0917 17:34:40.651793    2575 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Sep 17 17:34:40.655597 ip-10-0-143-74 kubenswrapper[2575]: W0917 17:34:40.651796    2575 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Sep 17 17:34:40.655597 ip-10-0-143-74 kubenswrapper[2575]: W0917 17:34:40.651799    2575 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Sep 17 17:34:40.655597 ip-10-0-143-74 kubenswrapper[2575]: W0917 17:34:40.651802    2575 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Sep 17 17:34:40.655597 ip-10-0-143-74 kubenswrapper[2575]: W0917 17:34:40.651805    2575 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Sep 17 17:34:40.655597 ip-10-0-143-74 kubenswrapper[2575]: W0917 17:34:40.651808    2575 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Sep 17 17:34:40.655597 ip-10-0-143-74 kubenswrapper[2575]: W0917 17:34:40.651811    2575 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Sep 17 17:34:40.655597 ip-10-0-143-74 kubenswrapper[2575]: W0917 17:34:40.651814    2575 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Sep 17 17:34:40.655597 ip-10-0-143-74 kubenswrapper[2575]: W0917 17:34:40.651817    2575 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Sep 17 17:34:40.655597 ip-10-0-143-74 kubenswrapper[2575]: W0917 17:34:40.651820    2575 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Sep 17 17:34:40.655597 ip-10-0-143-74 kubenswrapper[2575]: W0917 17:34:40.651823    2575 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Sep 17 17:34:40.655597 ip-10-0-143-74 kubenswrapper[2575]: W0917 17:34:40.651826    2575 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Sep 17 17:34:40.656090 ip-10-0-143-74 kubenswrapper[2575]: W0917 17:34:40.651829    2575 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Sep 17 17:34:40.656090 ip-10-0-143-74 kubenswrapper[2575]: W0917 17:34:40.651832    2575 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Sep 17 17:34:40.656090 ip-10-0-143-74 kubenswrapper[2575]: W0917 17:34:40.651835    2575 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Sep 17 17:34:40.656090 ip-10-0-143-74 kubenswrapper[2575]: W0917 17:34:40.651838    2575 feature_gate.go:328] unrecognized feature gate: EventTTL
Sep 17 17:34:40.656090 ip-10-0-143-74 kubenswrapper[2575]: W0917 17:34:40.651841    2575 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Sep 17 17:34:40.656090 ip-10-0-143-74 kubenswrapper[2575]: W0917 17:34:40.651843    2575 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Sep 17 17:34:40.656090 ip-10-0-143-74 kubenswrapper[2575]: W0917 17:34:40.651846    2575 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Sep 17 17:34:40.656090 ip-10-0-143-74 kubenswrapper[2575]: W0917 17:34:40.651849    2575 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Sep 17 17:34:40.656090 ip-10-0-143-74 kubenswrapper[2575]: W0917 17:34:40.651852    2575 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Sep 17 17:34:40.656090 ip-10-0-143-74 kubenswrapper[2575]: W0917 17:34:40.651854    2575 feature_gate.go:328] unrecognized feature gate: PinnedImages
Sep 17 17:34:40.656090 ip-10-0-143-74 kubenswrapper[2575]: W0917 17:34:40.651857    2575 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Sep 17 17:34:40.656090 ip-10-0-143-74 kubenswrapper[2575]: W0917 17:34:40.651860    2575 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Sep 17 17:34:40.656090 ip-10-0-143-74 kubenswrapper[2575]: W0917 17:34:40.651864    2575 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Sep 17 17:34:40.656090 ip-10-0-143-74 kubenswrapper[2575]: W0917 17:34:40.651866    2575 feature_gate.go:328] unrecognized feature gate: OVNObservability
Sep 17 17:34:40.656090 ip-10-0-143-74 kubenswrapper[2575]: W0917 17:34:40.651869    2575 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Sep 17 17:34:40.656090 ip-10-0-143-74 kubenswrapper[2575]: W0917 17:34:40.651871    2575 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Sep 17 17:34:40.656090 ip-10-0-143-74 kubenswrapper[2575]: W0917 17:34:40.651874    2575 feature_gate.go:328] unrecognized feature gate: DualReplica
Sep 17 17:34:40.656090 ip-10-0-143-74 kubenswrapper[2575]: W0917 17:34:40.651876    2575 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Sep 17 17:34:40.656090 ip-10-0-143-74 kubenswrapper[2575]: W0917 17:34:40.651879    2575 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Sep 17 17:34:40.656090 ip-10-0-143-74 kubenswrapper[2575]: W0917 17:34:40.651882    2575 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Sep 17 17:34:40.656589 ip-10-0-143-74 kubenswrapper[2575]: W0917 17:34:40.651884    2575 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Sep 17 17:34:40.656589 ip-10-0-143-74 kubenswrapper[2575]: W0917 17:34:40.651887    2575 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Sep 17 17:34:40.656589 ip-10-0-143-74 kubenswrapper[2575]: W0917 17:34:40.651890    2575 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Sep 17 17:34:40.656589 ip-10-0-143-74 kubenswrapper[2575]: W0917 17:34:40.651892    2575 feature_gate.go:328] unrecognized feature gate: OSStreams
Sep 17 17:34:40.656589 ip-10-0-143-74 kubenswrapper[2575]: W0917 17:34:40.651895    2575 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Sep 17 17:34:40.656589 ip-10-0-143-74 kubenswrapper[2575]: W0917 17:34:40.651898    2575 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Sep 17 17:34:40.656589 ip-10-0-143-74 kubenswrapper[2575]: W0917 17:34:40.651901    2575 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Sep 17 17:34:40.656589 ip-10-0-143-74 kubenswrapper[2575]: W0917 17:34:40.651904    2575 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Sep 17 17:34:40.656589 ip-10-0-143-74 kubenswrapper[2575]: W0917 17:34:40.651906    2575 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Sep 17 17:34:40.656589 ip-10-0-143-74 kubenswrapper[2575]: W0917 17:34:40.651909    2575 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Sep 17 17:34:40.656589 ip-10-0-143-74 kubenswrapper[2575]: W0917 17:34:40.651911    2575 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Sep 17 17:34:40.656589 ip-10-0-143-74 kubenswrapper[2575]: W0917 17:34:40.651914    2575 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Sep 17 17:34:40.656589 ip-10-0-143-74 kubenswrapper[2575]: W0917 17:34:40.651917    2575 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Sep 17 17:34:40.656589 ip-10-0-143-74 kubenswrapper[2575]: W0917 17:34:40.651919    2575 feature_gate.go:328] unrecognized feature gate: Example
Sep 17 17:34:40.656589 ip-10-0-143-74 kubenswrapper[2575]: W0917 17:34:40.651923    2575 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Sep 17 17:34:40.656589 ip-10-0-143-74 kubenswrapper[2575]: W0917 17:34:40.651926    2575 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Sep 17 17:34:40.656589 ip-10-0-143-74 kubenswrapper[2575]: W0917 17:34:40.651929    2575 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Sep 17 17:34:40.656589 ip-10-0-143-74 kubenswrapper[2575]: W0917 17:34:40.651931    2575 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Sep 17 17:34:40.656589 ip-10-0-143-74 kubenswrapper[2575]: W0917 17:34:40.651934    2575 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Sep 17 17:34:40.656589 ip-10-0-143-74 kubenswrapper[2575]: W0917 17:34:40.651937    2575 feature_gate.go:328] unrecognized feature gate: NewOLM
Sep 17 17:34:40.657081 ip-10-0-143-74 kubenswrapper[2575]: W0917 17:34:40.651939    2575 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Sep 17 17:34:40.657081 ip-10-0-143-74 kubenswrapper[2575]: W0917 17:34:40.651942    2575 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Sep 17 17:34:40.657081 ip-10-0-143-74 kubenswrapper[2575]: W0917 17:34:40.651944    2575 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Sep 17 17:34:40.657081 ip-10-0-143-74 kubenswrapper[2575]: W0917 17:34:40.651947    2575 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Sep 17 17:34:40.657081 ip-10-0-143-74 kubenswrapper[2575]: W0917 17:34:40.651951    2575 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Sep 17 17:34:40.657081 ip-10-0-143-74 kubenswrapper[2575]: W0917 17:34:40.651954    2575 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Sep 17 17:34:40.657081 ip-10-0-143-74 kubenswrapper[2575]: W0917 17:34:40.651957    2575 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Sep 17 17:34:40.657081 ip-10-0-143-74 kubenswrapper[2575]: W0917 17:34:40.651960    2575 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Sep 17 17:34:40.657081 ip-10-0-143-74 kubenswrapper[2575]: W0917 17:34:40.651962    2575 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Sep 17 17:34:40.657081 ip-10-0-143-74 kubenswrapper[2575]: W0917 17:34:40.651965    2575 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Sep 17 17:34:40.657081 ip-10-0-143-74 kubenswrapper[2575]: W0917 17:34:40.651968    2575 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Sep 17 17:34:40.657081 ip-10-0-143-74 kubenswrapper[2575]: W0917 17:34:40.651970    2575 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Sep 17 17:34:40.657081 ip-10-0-143-74 kubenswrapper[2575]: W0917 17:34:40.651973    2575 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Sep 17 17:34:40.657081 ip-10-0-143-74 kubenswrapper[2575]: W0917 17:34:40.651975    2575 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Sep 17 17:34:40.657081 ip-10-0-143-74 kubenswrapper[2575]: W0917 17:34:40.651978    2575 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Sep 17 17:34:40.657081 ip-10-0-143-74 kubenswrapper[2575]: W0917 17:34:40.651980    2575 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Sep 17 17:34:40.657081 ip-10-0-143-74 kubenswrapper[2575]: W0917 17:34:40.651985    2575 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Sep 17 17:34:40.657081 ip-10-0-143-74 kubenswrapper[2575]: W0917 17:34:40.651988    2575 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Sep 17 17:34:40.657081 ip-10-0-143-74 kubenswrapper[2575]: W0917 17:34:40.651990    2575 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Sep 17 17:34:40.657081 ip-10-0-143-74 kubenswrapper[2575]: W0917 17:34:40.651993    2575 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Sep 17 17:34:40.657571 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:34:40.652062    2575 flags.go:64] FLAG: --address="0.0.0.0"
Sep 17 17:34:40.657571 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:34:40.652107    2575 flags.go:64] FLAG: --allowed-unsafe-sysctls="[]"
Sep 17 17:34:40.657571 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:34:40.652115    2575 flags.go:64] FLAG: --anonymous-auth="true"
Sep 17 17:34:40.657571 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:34:40.652120    2575 flags.go:64] FLAG: --application-metrics-count-limit="100"
Sep 17 17:34:40.657571 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:34:40.652124    2575 flags.go:64] FLAG: --authentication-token-webhook="false"
Sep 17 17:34:40.657571 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:34:40.652127    2575 flags.go:64] FLAG: --authentication-token-webhook-cache-ttl="2m0s"
Sep 17 17:34:40.657571 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:34:40.652132    2575 flags.go:64] FLAG: --authorization-mode="AlwaysAllow"
Sep 17 17:34:40.657571 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:34:40.652136    2575 flags.go:64] FLAG: --authorization-webhook-cache-authorized-ttl="5m0s"
Sep 17 17:34:40.657571 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:34:40.652139    2575 flags.go:64] FLAG: --authorization-webhook-cache-unauthorized-ttl="30s"
Sep 17 17:34:40.657571 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:34:40.652142    2575 flags.go:64] FLAG: --boot-id-file="/proc/sys/kernel/random/boot_id"
Sep 17 17:34:40.657571 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:34:40.652146    2575 flags.go:64] FLAG: --bootstrap-kubeconfig="/etc/kubernetes/kubeconfig"
Sep 17 17:34:40.657571 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:34:40.652149    2575 flags.go:64] FLAG: --cert-dir="/var/lib/kubelet/pki"
Sep 17 17:34:40.657571 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:34:40.652152    2575 flags.go:64] FLAG: --cgroup-driver="cgroupfs"
Sep 17 17:34:40.657571 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:34:40.652155    2575 flags.go:64] FLAG: --cgroup-root=""
Sep 17 17:34:40.657571 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:34:40.652158    2575 flags.go:64] FLAG: --cgroups-per-qos="true"
Sep 17 17:34:40.657571 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:34:40.652161    2575 flags.go:64] FLAG: --client-ca-file=""
Sep 17 17:34:40.657571 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:34:40.652164    2575 flags.go:64] FLAG: --cloud-provider="external"
Sep 17 17:34:40.657571 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:34:40.652167    2575 flags.go:64] FLAG: --cluster-dns="[]"
Sep 17 17:34:40.657571 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:34:40.652171    2575 flags.go:64] FLAG: --cluster-domain=""
Sep 17 17:34:40.657571 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:34:40.652174    2575 flags.go:64] FLAG: --config="/etc/kubernetes/kubelet.conf"
Sep 17 17:34:40.657571 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:34:40.652177    2575 flags.go:64] FLAG: --config-dir="/etc/openshift/kubelet.conf.d"
Sep 17 17:34:40.657571 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:34:40.652180    2575 flags.go:64] FLAG: --container-hints="/etc/cadvisor/container_hints.json"
Sep 17 17:34:40.657571 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:34:40.652183    2575 flags.go:64] FLAG: --container-log-max-files="5"
Sep 17 17:34:40.657571 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:34:40.652186    2575 flags.go:64] FLAG: --container-log-max-size="10Mi"
Sep 17 17:34:40.658155 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:34:40.652189    2575 flags.go:64] FLAG: --container-runtime-endpoint="/var/run/crio/crio.sock"
Sep 17 17:34:40.658155 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:34:40.652192    2575 flags.go:64] FLAG: --containerd="/run/containerd/containerd.sock"
Sep 17 17:34:40.658155 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:34:40.652195    2575 flags.go:64] FLAG: --containerd-namespace="k8s.io"
Sep 17 17:34:40.658155 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:34:40.652198    2575 flags.go:64] FLAG: --contention-profiling="false"
Sep 17 17:34:40.658155 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:34:40.652203    2575 flags.go:64] FLAG: --cpu-cfs-quota="true"
Sep 17 17:34:40.658155 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:34:40.652206    2575 flags.go:64] FLAG: --cpu-cfs-quota-period="100ms"
Sep 17 17:34:40.658155 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:34:40.652209    2575 flags.go:64] FLAG: --cpu-manager-policy="none"
Sep 17 17:34:40.658155 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:34:40.652213    2575 flags.go:64] FLAG: --cpu-manager-policy-options=""
Sep 17 17:34:40.658155 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:34:40.652216    2575 flags.go:64] FLAG: --cpu-manager-reconcile-period="10s"
Sep 17 17:34:40.658155 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:34:40.652220    2575 flags.go:64] FLAG: --enable-controller-attach-detach="true"
Sep 17 17:34:40.658155 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:34:40.652223    2575 flags.go:64] FLAG: --enable-debugging-handlers="true"
Sep 17 17:34:40.658155 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:34:40.652225    2575 flags.go:64] FLAG: --enable-load-reader="false"
Sep 17 17:34:40.658155 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:34:40.652228    2575 flags.go:64] FLAG: --enable-server="true"
Sep 17 17:34:40.658155 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:34:40.652231    2575 flags.go:64] FLAG: --enforce-node-allocatable="[pods]"
Sep 17 17:34:40.658155 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:34:40.652236    2575 flags.go:64] FLAG: --event-burst="100"
Sep 17 17:34:40.658155 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:34:40.652239    2575 flags.go:64] FLAG: --event-qps="50"
Sep 17 17:34:40.658155 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:34:40.652242    2575 flags.go:64] FLAG: --event-storage-age-limit="default=0"
Sep 17 17:34:40.658155 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:34:40.652245    2575 flags.go:64] FLAG: --event-storage-event-limit="default=0"
Sep 17 17:34:40.658155 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:34:40.652248    2575 flags.go:64] FLAG: --eviction-hard=""
Sep 17 17:34:40.658155 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:34:40.652251    2575 flags.go:64] FLAG: --eviction-max-pod-grace-period="0"
Sep 17 17:34:40.658155 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:34:40.652254    2575 flags.go:64] FLAG: --eviction-minimum-reclaim=""
Sep 17 17:34:40.658155 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:34:40.652257    2575 flags.go:64] FLAG: --eviction-pressure-transition-period="5m0s"
Sep 17 17:34:40.658155 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:34:40.652260    2575 flags.go:64] FLAG: --eviction-soft=""
Sep 17 17:34:40.658155 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:34:40.652263    2575 flags.go:64] FLAG: --eviction-soft-grace-period=""
Sep 17 17:34:40.658155 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:34:40.652265    2575 flags.go:64] FLAG: --exit-on-lock-contention="false"
Sep 17 17:34:40.658783 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:34:40.652283    2575 flags.go:64] FLAG: --experimental-allocatable-ignore-eviction="false"
Sep 17 17:34:40.658783 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:34:40.652286    2575 flags.go:64] FLAG: --experimental-mounter-path=""
Sep 17 17:34:40.658783 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:34:40.652289    2575 flags.go:64] FLAG: --fail-cgroupv1="false"
Sep 17 17:34:40.658783 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:34:40.652292    2575 flags.go:64] FLAG: --fail-swap-on="true"
Sep 17 17:34:40.658783 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:34:40.652295    2575 flags.go:64] FLAG: --feature-gates=""
Sep 17 17:34:40.658783 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:34:40.652303    2575 flags.go:64] FLAG: --file-check-frequency="20s"
Sep 17 17:34:40.658783 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:34:40.652307    2575 flags.go:64] FLAG: --global-housekeeping-interval="1m0s"
Sep 17 17:34:40.658783 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:34:40.652309    2575 flags.go:64] FLAG: --hairpin-mode="promiscuous-bridge"
Sep 17 17:34:40.658783 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:34:40.652313    2575 flags.go:64] FLAG: --healthz-bind-address="127.0.0.1"
Sep 17 17:34:40.658783 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:34:40.652316    2575 flags.go:64] FLAG: --healthz-port="10248"
Sep 17 17:34:40.658783 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:34:40.652319    2575 flags.go:64] FLAG: --help="false"
Sep 17 17:34:40.658783 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:34:40.652323    2575 flags.go:64] FLAG: --hostname-override="ip-10-0-143-74.ec2.internal"
Sep 17 17:34:40.658783 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:34:40.652326    2575 flags.go:64] FLAG: --housekeeping-interval="10s"
Sep 17 17:34:40.658783 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:34:40.652330    2575 flags.go:64] FLAG: --http-check-frequency="20s"
Sep 17 17:34:40.658783 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:34:40.652333    2575 flags.go:64] FLAG: --image-credential-provider-bin-dir="/usr/libexec/kubelet-image-credential-provider-plugins"
Sep 17 17:34:40.658783 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:34:40.652336    2575 flags.go:64] FLAG: --image-credential-provider-config="/etc/kubernetes/credential-providers/ecr-credential-provider.yaml"
Sep 17 17:34:40.658783 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:34:40.652339    2575 flags.go:64] FLAG: --image-gc-high-threshold="85"
Sep 17 17:34:40.658783 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:34:40.652342    2575 flags.go:64] FLAG: --image-gc-low-threshold="80"
Sep 17 17:34:40.658783 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:34:40.652345    2575 flags.go:64] FLAG: --image-service-endpoint=""
Sep 17 17:34:40.658783 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:34:40.652348    2575 flags.go:64] FLAG: --kernel-memcg-notification="false"
Sep 17 17:34:40.658783 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:34:40.652352    2575 flags.go:64] FLAG: --kube-api-burst="100"
Sep 17 17:34:40.658783 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:34:40.652354    2575 flags.go:64] FLAG: --kube-api-content-type="application/vnd.kubernetes.protobuf"
Sep 17 17:34:40.658783 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:34:40.652357    2575 flags.go:64] FLAG: --kube-api-qps="50"
Sep 17 17:34:40.658783 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:34:40.652360    2575 flags.go:64] FLAG: --kube-reserved=""
Sep 17 17:34:40.659452 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:34:40.652363    2575 flags.go:64] FLAG: --kube-reserved-cgroup=""
Sep 17 17:34:40.659452 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:34:40.652366    2575 flags.go:64] FLAG: --kubeconfig="/var/lib/kubelet/kubeconfig"
Sep 17 17:34:40.659452 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:34:40.652369    2575 flags.go:64] FLAG: --kubelet-cgroups=""
Sep 17 17:34:40.659452 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:34:40.652372    2575 flags.go:64] FLAG: --local-storage-capacity-isolation="true"
Sep 17 17:34:40.659452 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:34:40.652374    2575 flags.go:64] FLAG: --lock-file=""
Sep 17 17:34:40.659452 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:34:40.652377    2575 flags.go:64] FLAG: --log-cadvisor-usage="false"
Sep 17 17:34:40.659452 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:34:40.652380    2575 flags.go:64] FLAG: --log-flush-frequency="5s"
Sep 17 17:34:40.659452 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:34:40.652383    2575 flags.go:64] FLAG: --log-json-info-buffer-size="0"
Sep 17 17:34:40.659452 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:34:40.652389    2575 flags.go:64] FLAG: --log-json-split-stream="false"
Sep 17 17:34:40.659452 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:34:40.652398    2575 flags.go:64] FLAG: --log-text-info-buffer-size="0"
Sep 17 17:34:40.659452 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:34:40.652401    2575 flags.go:64] FLAG: --log-text-split-stream="false"
Sep 17 17:34:40.659452 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:34:40.652404    2575 flags.go:64] FLAG: --logging-format="text"
Sep 17 17:34:40.659452 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:34:40.652407    2575 flags.go:64] FLAG: --machine-id-file="/etc/machine-id,/var/lib/dbus/machine-id"
Sep 17 17:34:40.659452 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:34:40.652410    2575 flags.go:64] FLAG: --make-iptables-util-chains="true"
Sep 17 17:34:40.659452 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:34:40.652413    2575 flags.go:64] FLAG: --manifest-url=""
Sep 17 17:34:40.659452 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:34:40.652416    2575 flags.go:64] FLAG: --manifest-url-header=""
Sep 17 17:34:40.659452 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:34:40.652421    2575 flags.go:64] FLAG: --max-housekeeping-interval="15s"
Sep 17 17:34:40.659452 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:34:40.652424    2575 flags.go:64] FLAG: --max-open-files="1000000"
Sep 17 17:34:40.659452 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:34:40.652428    2575 flags.go:64] FLAG: --max-pods="110"
Sep 17 17:34:40.659452 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:34:40.652433    2575 flags.go:64] FLAG: --maximum-dead-containers="-1"
Sep 17 17:34:40.659452 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:34:40.652436    2575 flags.go:64] FLAG: --maximum-dead-containers-per-container="1"
Sep 17 17:34:40.659452 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:34:40.652451    2575 flags.go:64] FLAG: --memory-manager-policy="None"
Sep 17 17:34:40.659452 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:34:40.652454    2575 flags.go:64] FLAG: --minimum-container-ttl-duration="6m0s"
Sep 17 17:34:40.659452 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:34:40.652457    2575 flags.go:64] FLAG: --minimum-image-ttl-duration="2m0s"
Sep 17 17:34:40.659452 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:34:40.652460    2575 flags.go:64] FLAG: --node-ip="0.0.0.0"
Sep 17 17:34:40.660073 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:34:40.652463    2575 flags.go:64] FLAG: --node-labels="node-role.kubernetes.io/worker=,node.openshift.io/os_id=rhel"
Sep 17 17:34:40.660073 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:34:40.652470    2575 flags.go:64] FLAG: --node-status-max-images="50"
Sep 17 17:34:40.660073 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:34:40.652473    2575 flags.go:64] FLAG: --node-status-update-frequency="10s"
Sep 17 17:34:40.660073 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:34:40.652476    2575 flags.go:64] FLAG: --oom-score-adj="-999"
Sep 17 17:34:40.660073 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:34:40.652479    2575 flags.go:64] FLAG: --pod-cidr=""
Sep 17 17:34:40.660073 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:34:40.652482    2575 flags.go:64] FLAG: --pod-infra-container-image="quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:a6abe77af61c5f2e3985a83e7d43538aefc38b1b0444127da14ee0e8cf8f76fc"
Sep 17 17:34:40.660073 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:34:40.652486    2575 flags.go:64] FLAG: --pod-manifest-path=""
Sep 17 17:34:40.660073 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:34:40.652489    2575 flags.go:64] FLAG: --pod-max-pids="-1"
Sep 17 17:34:40.660073 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:34:40.652493    2575 flags.go:64] FLAG: --pods-per-core="0"
Sep 17 17:34:40.660073 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:34:40.652496    2575 flags.go:64] FLAG: --port="10250"
Sep 17 17:34:40.660073 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:34:40.652498    2575 flags.go:64] FLAG: --protect-kernel-defaults="false"
Sep 17 17:34:40.660073 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:34:40.652507    2575 flags.go:64] FLAG: --provider-id="aws:///us-east-1a/i-0330924f5cdbcf95f"
Sep 17 17:34:40.660073 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:34:40.652511    2575 flags.go:64] FLAG: --qos-reserved=""
Sep 17 17:34:40.660073 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:34:40.652513    2575 flags.go:64] FLAG: --read-only-port="10255"
Sep 17 17:34:40.660073 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:34:40.652517    2575 flags.go:64] FLAG: --register-node="true"
Sep 17 17:34:40.660073 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:34:40.652519    2575 flags.go:64] FLAG: --register-with-taints=""
Sep 17 17:34:40.660073 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:34:40.652524    2575 flags.go:64] FLAG: --registry-burst="10"
Sep 17 17:34:40.660073 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:34:40.652533    2575 flags.go:64] FLAG: --registry-qps="5"
Sep 17 17:34:40.660073 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:34:40.652536    2575 flags.go:64] FLAG: --reserved-cpus=""
Sep 17 17:34:40.660073 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:34:40.652539    2575 flags.go:64] FLAG: --reserved-memory=""
Sep 17 17:34:40.660073 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:34:40.652543    2575 flags.go:64] FLAG: --resolv-conf="/etc/resolv.conf"
Sep 17 17:34:40.660073 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:34:40.652546    2575 flags.go:64] FLAG: --root-dir="/var/lib/kubelet"
Sep 17 17:34:40.660073 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:34:40.652549    2575 flags.go:64] FLAG: --rotate-certificates="false"
Sep 17 17:34:40.660073 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:34:40.652551    2575 flags.go:64] FLAG: --rotate-server-certificates="false"
Sep 17 17:34:40.660073 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:34:40.652555    2575 flags.go:64] FLAG: --runonce="false"
Sep 17 17:34:40.660711 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:34:40.652557    2575 flags.go:64] FLAG: --runtime-cgroups="/system.slice/crio.service"
Sep 17 17:34:40.660711 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:34:40.652562    2575 flags.go:64] FLAG: --runtime-request-timeout="2m0s"
Sep 17 17:34:40.660711 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:34:40.652565    2575 flags.go:64] FLAG: --seccomp-default="false"
Sep 17 17:34:40.660711 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:34:40.652567    2575 flags.go:64] FLAG: --serialize-image-pulls="true"
Sep 17 17:34:40.660711 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:34:40.652570    2575 flags.go:64] FLAG: --storage-driver-buffer-duration="1m0s"
Sep 17 17:34:40.660711 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:34:40.652573    2575 flags.go:64] FLAG: --storage-driver-db="cadvisor"
Sep 17 17:34:40.660711 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:34:40.652576    2575 flags.go:64] FLAG: --storage-driver-host="localhost:8086"
Sep 17 17:34:40.660711 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:34:40.652579    2575 flags.go:64] FLAG: --storage-driver-password="root"
Sep 17 17:34:40.660711 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:34:40.652582    2575 flags.go:64] FLAG: --storage-driver-secure="false"
Sep 17 17:34:40.660711 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:34:40.652585    2575 flags.go:64] FLAG: --storage-driver-table="stats"
Sep 17 17:34:40.660711 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:34:40.652588    2575 flags.go:64] FLAG: --storage-driver-user="root"
Sep 17 17:34:40.660711 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:34:40.652591    2575 flags.go:64] FLAG: --streaming-connection-idle-timeout="4h0m0s"
Sep 17 17:34:40.660711 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:34:40.652594    2575 flags.go:64] FLAG: --sync-frequency="1m0s"
Sep 17 17:34:40.660711 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:34:40.652597    2575 flags.go:64] FLAG: --system-cgroups=""
Sep 17 17:34:40.660711 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:34:40.652600    2575 flags.go:64] FLAG: --system-reserved="cpu=500m,ephemeral-storage=1Gi,memory=1Gi"
Sep 17 17:34:40.660711 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:34:40.652605    2575 flags.go:64] FLAG: --system-reserved-cgroup=""
Sep 17 17:34:40.660711 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:34:40.652608    2575 flags.go:64] FLAG: --tls-cert-file=""
Sep 17 17:34:40.660711 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:34:40.652611    2575 flags.go:64] FLAG: --tls-cipher-suites="[]"
Sep 17 17:34:40.660711 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:34:40.652615    2575 flags.go:64] FLAG: --tls-min-version=""
Sep 17 17:34:40.660711 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:34:40.652618    2575 flags.go:64] FLAG: --tls-private-key-file=""
Sep 17 17:34:40.660711 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:34:40.652621    2575 flags.go:64] FLAG: --topology-manager-policy="none"
Sep 17 17:34:40.660711 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:34:40.652624    2575 flags.go:64] FLAG: --topology-manager-policy-options=""
Sep 17 17:34:40.660711 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:34:40.652627    2575 flags.go:64] FLAG: --topology-manager-scope="container"
Sep 17 17:34:40.660711 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:34:40.652631    2575 flags.go:64] FLAG: --v="2"
Sep 17 17:34:40.660711 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:34:40.652635    2575 flags.go:64] FLAG: --version="false"
Sep 17 17:34:40.661320 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:34:40.652639    2575 flags.go:64] FLAG: --vmodule=""
Sep 17 17:34:40.661320 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:34:40.652643    2575 flags.go:64] FLAG: --volume-plugin-dir="/etc/kubernetes/kubelet-plugins/volume/exec"
Sep 17 17:34:40.661320 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:34:40.652646    2575 flags.go:64] FLAG: --volume-stats-agg-period="1m0s"
Sep 17 17:34:40.661320 ip-10-0-143-74 kubenswrapper[2575]: W0917 17:34:40.652759    2575 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Sep 17 17:34:40.661320 ip-10-0-143-74 kubenswrapper[2575]: W0917 17:34:40.652763    2575 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Sep 17 17:34:40.661320 ip-10-0-143-74 kubenswrapper[2575]: W0917 17:34:40.652766    2575 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Sep 17 17:34:40.661320 ip-10-0-143-74 kubenswrapper[2575]: W0917 17:34:40.652784    2575 feature_gate.go:328] unrecognized feature gate: EventTTL
Sep 17 17:34:40.661320 ip-10-0-143-74 kubenswrapper[2575]: W0917 17:34:40.652788    2575 feature_gate.go:328] unrecognized feature gate: Example2
Sep 17 17:34:40.661320 ip-10-0-143-74 kubenswrapper[2575]: W0917 17:34:40.652792    2575 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Sep 17 17:34:40.661320 ip-10-0-143-74 kubenswrapper[2575]: W0917 17:34:40.652795    2575 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Sep 17 17:34:40.661320 ip-10-0-143-74 kubenswrapper[2575]: W0917 17:34:40.652799    2575 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Sep 17 17:34:40.661320 ip-10-0-143-74 kubenswrapper[2575]: W0917 17:34:40.652802    2575 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Sep 17 17:34:40.661320 ip-10-0-143-74 kubenswrapper[2575]: W0917 17:34:40.652805    2575 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Sep 17 17:34:40.661320 ip-10-0-143-74 kubenswrapper[2575]: W0917 17:34:40.652808    2575 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Sep 17 17:34:40.661320 ip-10-0-143-74 kubenswrapper[2575]: W0917 17:34:40.652811    2575 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Sep 17 17:34:40.661320 ip-10-0-143-74 kubenswrapper[2575]: W0917 17:34:40.652813    2575 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Sep 17 17:34:40.661320 ip-10-0-143-74 kubenswrapper[2575]: W0917 17:34:40.652816    2575 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Sep 17 17:34:40.661320 ip-10-0-143-74 kubenswrapper[2575]: W0917 17:34:40.652819    2575 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Sep 17 17:34:40.661320 ip-10-0-143-74 kubenswrapper[2575]: W0917 17:34:40.652821    2575 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Sep 17 17:34:40.661320 ip-10-0-143-74 kubenswrapper[2575]: W0917 17:34:40.652824    2575 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Sep 17 17:34:40.661825 ip-10-0-143-74 kubenswrapper[2575]: W0917 17:34:40.652826    2575 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Sep 17 17:34:40.661825 ip-10-0-143-74 kubenswrapper[2575]: W0917 17:34:40.652829    2575 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Sep 17 17:34:40.661825 ip-10-0-143-74 kubenswrapper[2575]: W0917 17:34:40.652832    2575 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Sep 17 17:34:40.661825 ip-10-0-143-74 kubenswrapper[2575]: W0917 17:34:40.652834    2575 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Sep 17 17:34:40.661825 ip-10-0-143-74 kubenswrapper[2575]: W0917 17:34:40.652837    2575 feature_gate.go:328] unrecognized feature gate: SignatureStores
Sep 17 17:34:40.661825 ip-10-0-143-74 kubenswrapper[2575]: W0917 17:34:40.652839    2575 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Sep 17 17:34:40.661825 ip-10-0-143-74 kubenswrapper[2575]: W0917 17:34:40.652842    2575 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Sep 17 17:34:40.661825 ip-10-0-143-74 kubenswrapper[2575]: W0917 17:34:40.652845    2575 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Sep 17 17:34:40.661825 ip-10-0-143-74 kubenswrapper[2575]: W0917 17:34:40.652848    2575 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Sep 17 17:34:40.661825 ip-10-0-143-74 kubenswrapper[2575]: W0917 17:34:40.652850    2575 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Sep 17 17:34:40.661825 ip-10-0-143-74 kubenswrapper[2575]: W0917 17:34:40.652854    2575 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Sep 17 17:34:40.661825 ip-10-0-143-74 kubenswrapper[2575]: W0917 17:34:40.652857    2575 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Sep 17 17:34:40.661825 ip-10-0-143-74 kubenswrapper[2575]: W0917 17:34:40.652860    2575 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Sep 17 17:34:40.661825 ip-10-0-143-74 kubenswrapper[2575]: W0917 17:34:40.652863    2575 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Sep 17 17:34:40.661825 ip-10-0-143-74 kubenswrapper[2575]: W0917 17:34:40.652866    2575 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Sep 17 17:34:40.661825 ip-10-0-143-74 kubenswrapper[2575]: W0917 17:34:40.652868    2575 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Sep 17 17:34:40.661825 ip-10-0-143-74 kubenswrapper[2575]: W0917 17:34:40.652871    2575 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Sep 17 17:34:40.661825 ip-10-0-143-74 kubenswrapper[2575]: W0917 17:34:40.652873    2575 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Sep 17 17:34:40.661825 ip-10-0-143-74 kubenswrapper[2575]: W0917 17:34:40.652876    2575 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Sep 17 17:34:40.662297 ip-10-0-143-74 kubenswrapper[2575]: W0917 17:34:40.652879    2575 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Sep 17 17:34:40.662297 ip-10-0-143-74 kubenswrapper[2575]: W0917 17:34:40.652882    2575 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Sep 17 17:34:40.662297 ip-10-0-143-74 kubenswrapper[2575]: W0917 17:34:40.652884    2575 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Sep 17 17:34:40.662297 ip-10-0-143-74 kubenswrapper[2575]: W0917 17:34:40.652887    2575 feature_gate.go:328] unrecognized feature gate: NewOLM
Sep 17 17:34:40.662297 ip-10-0-143-74 kubenswrapper[2575]: W0917 17:34:40.652889    2575 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Sep 17 17:34:40.662297 ip-10-0-143-74 kubenswrapper[2575]: W0917 17:34:40.652892    2575 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Sep 17 17:34:40.662297 ip-10-0-143-74 kubenswrapper[2575]: W0917 17:34:40.652894    2575 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Sep 17 17:34:40.662297 ip-10-0-143-74 kubenswrapper[2575]: W0917 17:34:40.652897    2575 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Sep 17 17:34:40.662297 ip-10-0-143-74 kubenswrapper[2575]: W0917 17:34:40.652900    2575 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Sep 17 17:34:40.662297 ip-10-0-143-74 kubenswrapper[2575]: W0917 17:34:40.652902    2575 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Sep 17 17:34:40.662297 ip-10-0-143-74 kubenswrapper[2575]: W0917 17:34:40.652905    2575 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Sep 17 17:34:40.662297 ip-10-0-143-74 kubenswrapper[2575]: W0917 17:34:40.652908    2575 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Sep 17 17:34:40.662297 ip-10-0-143-74 kubenswrapper[2575]: W0917 17:34:40.652910    2575 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Sep 17 17:34:40.662297 ip-10-0-143-74 kubenswrapper[2575]: W0917 17:34:40.652913    2575 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Sep 17 17:34:40.662297 ip-10-0-143-74 kubenswrapper[2575]: W0917 17:34:40.652916    2575 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Sep 17 17:34:40.662297 ip-10-0-143-74 kubenswrapper[2575]: W0917 17:34:40.652918    2575 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Sep 17 17:34:40.662297 ip-10-0-143-74 kubenswrapper[2575]: W0917 17:34:40.652921    2575 feature_gate.go:328] unrecognized feature gate: Example
Sep 17 17:34:40.662297 ip-10-0-143-74 kubenswrapper[2575]: W0917 17:34:40.652923    2575 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Sep 17 17:34:40.662297 ip-10-0-143-74 kubenswrapper[2575]: W0917 17:34:40.652926    2575 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Sep 17 17:34:40.662297 ip-10-0-143-74 kubenswrapper[2575]: W0917 17:34:40.652929    2575 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Sep 17 17:34:40.662797 ip-10-0-143-74 kubenswrapper[2575]: W0917 17:34:40.652931    2575 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Sep 17 17:34:40.662797 ip-10-0-143-74 kubenswrapper[2575]: W0917 17:34:40.652934    2575 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Sep 17 17:34:40.662797 ip-10-0-143-74 kubenswrapper[2575]: W0917 17:34:40.652936    2575 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Sep 17 17:34:40.662797 ip-10-0-143-74 kubenswrapper[2575]: W0917 17:34:40.652940    2575 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Sep 17 17:34:40.662797 ip-10-0-143-74 kubenswrapper[2575]: W0917 17:34:40.652945    2575 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Sep 17 17:34:40.662797 ip-10-0-143-74 kubenswrapper[2575]: W0917 17:34:40.652949    2575 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Sep 17 17:34:40.662797 ip-10-0-143-74 kubenswrapper[2575]: W0917 17:34:40.652951    2575 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Sep 17 17:34:40.662797 ip-10-0-143-74 kubenswrapper[2575]: W0917 17:34:40.652955    2575 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Sep 17 17:34:40.662797 ip-10-0-143-74 kubenswrapper[2575]: W0917 17:34:40.652958    2575 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Sep 17 17:34:40.662797 ip-10-0-143-74 kubenswrapper[2575]: W0917 17:34:40.652960    2575 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Sep 17 17:34:40.662797 ip-10-0-143-74 kubenswrapper[2575]: W0917 17:34:40.652963    2575 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Sep 17 17:34:40.662797 ip-10-0-143-74 kubenswrapper[2575]: W0917 17:34:40.652966    2575 feature_gate.go:328] unrecognized feature gate: PinnedImages
Sep 17 17:34:40.662797 ip-10-0-143-74 kubenswrapper[2575]: W0917 17:34:40.652968    2575 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Sep 17 17:34:40.662797 ip-10-0-143-74 kubenswrapper[2575]: W0917 17:34:40.652971    2575 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Sep 17 17:34:40.662797 ip-10-0-143-74 kubenswrapper[2575]: W0917 17:34:40.652973    2575 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Sep 17 17:34:40.662797 ip-10-0-143-74 kubenswrapper[2575]: W0917 17:34:40.652977    2575 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Sep 17 17:34:40.662797 ip-10-0-143-74 kubenswrapper[2575]: W0917 17:34:40.652981    2575 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Sep 17 17:34:40.662797 ip-10-0-143-74 kubenswrapper[2575]: W0917 17:34:40.652984    2575 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Sep 17 17:34:40.662797 ip-10-0-143-74 kubenswrapper[2575]: W0917 17:34:40.652987    2575 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Sep 17 17:34:40.663305 ip-10-0-143-74 kubenswrapper[2575]: W0917 17:34:40.652990    2575 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Sep 17 17:34:40.663305 ip-10-0-143-74 kubenswrapper[2575]: W0917 17:34:40.652993    2575 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Sep 17 17:34:40.663305 ip-10-0-143-74 kubenswrapper[2575]: W0917 17:34:40.652996    2575 feature_gate.go:328] unrecognized feature gate: OSStreams
Sep 17 17:34:40.663305 ip-10-0-143-74 kubenswrapper[2575]: W0917 17:34:40.652999    2575 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Sep 17 17:34:40.663305 ip-10-0-143-74 kubenswrapper[2575]: W0917 17:34:40.653001    2575 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Sep 17 17:34:40.663305 ip-10-0-143-74 kubenswrapper[2575]: W0917 17:34:40.653004    2575 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Sep 17 17:34:40.663305 ip-10-0-143-74 kubenswrapper[2575]: W0917 17:34:40.653006    2575 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Sep 17 17:34:40.663305 ip-10-0-143-74 kubenswrapper[2575]: W0917 17:34:40.653009    2575 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Sep 17 17:34:40.663305 ip-10-0-143-74 kubenswrapper[2575]: W0917 17:34:40.653012    2575 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Sep 17 17:34:40.663305 ip-10-0-143-74 kubenswrapper[2575]: W0917 17:34:40.653014    2575 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Sep 17 17:34:40.663305 ip-10-0-143-74 kubenswrapper[2575]: W0917 17:34:40.653018    2575 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Sep 17 17:34:40.663305 ip-10-0-143-74 kubenswrapper[2575]: W0917 17:34:40.653022    2575 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Sep 17 17:34:40.663305 ip-10-0-143-74 kubenswrapper[2575]: W0917 17:34:40.653024    2575 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Sep 17 17:34:40.663305 ip-10-0-143-74 kubenswrapper[2575]: W0917 17:34:40.653027    2575 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Sep 17 17:34:40.663305 ip-10-0-143-74 kubenswrapper[2575]: W0917 17:34:40.653030    2575 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Sep 17 17:34:40.663305 ip-10-0-143-74 kubenswrapper[2575]: W0917 17:34:40.653032    2575 feature_gate.go:328] unrecognized feature gate: OVNObservability
Sep 17 17:34:40.663305 ip-10-0-143-74 kubenswrapper[2575]: W0917 17:34:40.653036    2575 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Sep 17 17:34:40.663305 ip-10-0-143-74 kubenswrapper[2575]: W0917 17:34:40.653039    2575 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Sep 17 17:34:40.663305 ip-10-0-143-74 kubenswrapper[2575]: W0917 17:34:40.653042    2575 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Sep 17 17:34:40.663845 ip-10-0-143-74 kubenswrapper[2575]: W0917 17:34:40.653044    2575 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Sep 17 17:34:40.663845 ip-10-0-143-74 kubenswrapper[2575]: W0917 17:34:40.653047    2575 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Sep 17 17:34:40.663845 ip-10-0-143-74 kubenswrapper[2575]: W0917 17:34:40.653050    2575 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Sep 17 17:34:40.663845 ip-10-0-143-74 kubenswrapper[2575]: W0917 17:34:40.653052    2575 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Sep 17 17:34:40.663845 ip-10-0-143-74 kubenswrapper[2575]: W0917 17:34:40.653055    2575 feature_gate.go:328] unrecognized feature gate: DualReplica
Sep 17 17:34:40.663845 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:34:40.653570    2575 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]}
Sep 17 17:34:40.663845 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:34:40.660233    2575 server.go:529] "Kubelet version" kubeletVersion="v1.34.9"
Sep 17 17:34:40.663845 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:34:40.660250    2575 server.go:531] "Golang settings" GOGC="" GOMAXPROCS="" GOTRACEBACK=""
Sep 17 17:34:40.663845 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:34:40.660264    2575 watchdog_linux.go:95] "Systemd watchdog is not enabled"
Sep 17 17:34:40.663845 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:34:40.660269    2575 watchdog_linux.go:137] "Systemd watchdog is not enabled or the interval is invalid, so health checking will not be started."
Sep 17 17:34:40.663845 ip-10-0-143-74 kubenswrapper[2575]: W0917 17:34:40.660326    2575 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Sep 17 17:34:40.663845 ip-10-0-143-74 kubenswrapper[2575]: W0917 17:34:40.660330    2575 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Sep 17 17:34:40.663845 ip-10-0-143-74 kubenswrapper[2575]: W0917 17:34:40.660334    2575 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Sep 17 17:34:40.663845 ip-10-0-143-74 kubenswrapper[2575]: W0917 17:34:40.660337    2575 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Sep 17 17:34:40.663845 ip-10-0-143-74 kubenswrapper[2575]: W0917 17:34:40.660341    2575 feature_gate.go:328] unrecognized feature gate: Example2
Sep 17 17:34:40.664223 ip-10-0-143-74 kubenswrapper[2575]: W0917 17:34:40.660345    2575 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Sep 17 17:34:40.664223 ip-10-0-143-74 kubenswrapper[2575]: W0917 17:34:40.660349    2575 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Sep 17 17:34:40.664223 ip-10-0-143-74 kubenswrapper[2575]: W0917 17:34:40.660352    2575 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Sep 17 17:34:40.664223 ip-10-0-143-74 kubenswrapper[2575]: W0917 17:34:40.660356    2575 feature_gate.go:328] unrecognized feature gate: PinnedImages
Sep 17 17:34:40.664223 ip-10-0-143-74 kubenswrapper[2575]: W0917 17:34:40.660359    2575 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Sep 17 17:34:40.664223 ip-10-0-143-74 kubenswrapper[2575]: W0917 17:34:40.660362    2575 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Sep 17 17:34:40.664223 ip-10-0-143-74 kubenswrapper[2575]: W0917 17:34:40.660365    2575 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Sep 17 17:34:40.664223 ip-10-0-143-74 kubenswrapper[2575]: W0917 17:34:40.660367    2575 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Sep 17 17:34:40.664223 ip-10-0-143-74 kubenswrapper[2575]: W0917 17:34:40.660370    2575 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Sep 17 17:34:40.664223 ip-10-0-143-74 kubenswrapper[2575]: W0917 17:34:40.660373    2575 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Sep 17 17:34:40.664223 ip-10-0-143-74 kubenswrapper[2575]: W0917 17:34:40.660376    2575 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Sep 17 17:34:40.664223 ip-10-0-143-74 kubenswrapper[2575]: W0917 17:34:40.660378    2575 feature_gate.go:328] unrecognized feature gate: DualReplica
Sep 17 17:34:40.664223 ip-10-0-143-74 kubenswrapper[2575]: W0917 17:34:40.660381    2575 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Sep 17 17:34:40.664223 ip-10-0-143-74 kubenswrapper[2575]: W0917 17:34:40.660384    2575 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Sep 17 17:34:40.664223 ip-10-0-143-74 kubenswrapper[2575]: W0917 17:34:40.660386    2575 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Sep 17 17:34:40.664223 ip-10-0-143-74 kubenswrapper[2575]: W0917 17:34:40.660389    2575 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Sep 17 17:34:40.664223 ip-10-0-143-74 kubenswrapper[2575]: W0917 17:34:40.660392    2575 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Sep 17 17:34:40.664223 ip-10-0-143-74 kubenswrapper[2575]: W0917 17:34:40.660395    2575 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Sep 17 17:34:40.664223 ip-10-0-143-74 kubenswrapper[2575]: W0917 17:34:40.660398    2575 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Sep 17 17:34:40.664706 ip-10-0-143-74 kubenswrapper[2575]: W0917 17:34:40.660401    2575 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Sep 17 17:34:40.664706 ip-10-0-143-74 kubenswrapper[2575]: W0917 17:34:40.660404    2575 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Sep 17 17:34:40.664706 ip-10-0-143-74 kubenswrapper[2575]: W0917 17:34:40.660407    2575 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Sep 17 17:34:40.664706 ip-10-0-143-74 kubenswrapper[2575]: W0917 17:34:40.660410    2575 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Sep 17 17:34:40.664706 ip-10-0-143-74 kubenswrapper[2575]: W0917 17:34:40.660412    2575 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Sep 17 17:34:40.664706 ip-10-0-143-74 kubenswrapper[2575]: W0917 17:34:40.660415    2575 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Sep 17 17:34:40.664706 ip-10-0-143-74 kubenswrapper[2575]: W0917 17:34:40.660418    2575 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Sep 17 17:34:40.664706 ip-10-0-143-74 kubenswrapper[2575]: W0917 17:34:40.660421    2575 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Sep 17 17:34:40.664706 ip-10-0-143-74 kubenswrapper[2575]: W0917 17:34:40.660423    2575 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Sep 17 17:34:40.664706 ip-10-0-143-74 kubenswrapper[2575]: W0917 17:34:40.660426    2575 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Sep 17 17:34:40.664706 ip-10-0-143-74 kubenswrapper[2575]: W0917 17:34:40.660429    2575 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Sep 17 17:34:40.664706 ip-10-0-143-74 kubenswrapper[2575]: W0917 17:34:40.660431    2575 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Sep 17 17:34:40.664706 ip-10-0-143-74 kubenswrapper[2575]: W0917 17:34:40.660434    2575 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Sep 17 17:34:40.664706 ip-10-0-143-74 kubenswrapper[2575]: W0917 17:34:40.660437    2575 feature_gate.go:328] unrecognized feature gate: OSStreams
Sep 17 17:34:40.664706 ip-10-0-143-74 kubenswrapper[2575]: W0917 17:34:40.660456    2575 feature_gate.go:328] unrecognized feature gate: Example
Sep 17 17:34:40.664706 ip-10-0-143-74 kubenswrapper[2575]: W0917 17:34:40.660458    2575 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Sep 17 17:34:40.664706 ip-10-0-143-74 kubenswrapper[2575]: W0917 17:34:40.660461    2575 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Sep 17 17:34:40.664706 ip-10-0-143-74 kubenswrapper[2575]: W0917 17:34:40.660464    2575 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Sep 17 17:34:40.664706 ip-10-0-143-74 kubenswrapper[2575]: W0917 17:34:40.660467    2575 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Sep 17 17:34:40.664706 ip-10-0-143-74 kubenswrapper[2575]: W0917 17:34:40.660469    2575 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Sep 17 17:34:40.665192 ip-10-0-143-74 kubenswrapper[2575]: W0917 17:34:40.660472    2575 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Sep 17 17:34:40.665192 ip-10-0-143-74 kubenswrapper[2575]: W0917 17:34:40.660475    2575 feature_gate.go:328] unrecognized feature gate: OVNObservability
Sep 17 17:34:40.665192 ip-10-0-143-74 kubenswrapper[2575]: W0917 17:34:40.660477    2575 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Sep 17 17:34:40.665192 ip-10-0-143-74 kubenswrapper[2575]: W0917 17:34:40.660480    2575 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Sep 17 17:34:40.665192 ip-10-0-143-74 kubenswrapper[2575]: W0917 17:34:40.660483    2575 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Sep 17 17:34:40.665192 ip-10-0-143-74 kubenswrapper[2575]: W0917 17:34:40.660485    2575 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Sep 17 17:34:40.665192 ip-10-0-143-74 kubenswrapper[2575]: W0917 17:34:40.660488    2575 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Sep 17 17:34:40.665192 ip-10-0-143-74 kubenswrapper[2575]: W0917 17:34:40.660490    2575 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Sep 17 17:34:40.665192 ip-10-0-143-74 kubenswrapper[2575]: W0917 17:34:40.660493    2575 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Sep 17 17:34:40.665192 ip-10-0-143-74 kubenswrapper[2575]: W0917 17:34:40.660496    2575 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Sep 17 17:34:40.665192 ip-10-0-143-74 kubenswrapper[2575]: W0917 17:34:40.660499    2575 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Sep 17 17:34:40.665192 ip-10-0-143-74 kubenswrapper[2575]: W0917 17:34:40.660502    2575 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Sep 17 17:34:40.665192 ip-10-0-143-74 kubenswrapper[2575]: W0917 17:34:40.660505    2575 feature_gate.go:328] unrecognized feature gate: SignatureStores
Sep 17 17:34:40.665192 ip-10-0-143-74 kubenswrapper[2575]: W0917 17:34:40.660508    2575 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Sep 17 17:34:40.665192 ip-10-0-143-74 kubenswrapper[2575]: W0917 17:34:40.660511    2575 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Sep 17 17:34:40.665192 ip-10-0-143-74 kubenswrapper[2575]: W0917 17:34:40.660515    2575 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Sep 17 17:34:40.665192 ip-10-0-143-74 kubenswrapper[2575]: W0917 17:34:40.660517    2575 feature_gate.go:328] unrecognized feature gate: NewOLM
Sep 17 17:34:40.665192 ip-10-0-143-74 kubenswrapper[2575]: W0917 17:34:40.660520    2575 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Sep 17 17:34:40.665192 ip-10-0-143-74 kubenswrapper[2575]: W0917 17:34:40.660523    2575 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Sep 17 17:34:40.665192 ip-10-0-143-74 kubenswrapper[2575]: W0917 17:34:40.660525    2575 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Sep 17 17:34:40.665684 ip-10-0-143-74 kubenswrapper[2575]: W0917 17:34:40.660528    2575 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Sep 17 17:34:40.665684 ip-10-0-143-74 kubenswrapper[2575]: W0917 17:34:40.660531    2575 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Sep 17 17:34:40.665684 ip-10-0-143-74 kubenswrapper[2575]: W0917 17:34:40.660533    2575 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Sep 17 17:34:40.665684 ip-10-0-143-74 kubenswrapper[2575]: W0917 17:34:40.660536    2575 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Sep 17 17:34:40.665684 ip-10-0-143-74 kubenswrapper[2575]: W0917 17:34:40.660540    2575 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Sep 17 17:34:40.665684 ip-10-0-143-74 kubenswrapper[2575]: W0917 17:34:40.660543    2575 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Sep 17 17:34:40.665684 ip-10-0-143-74 kubenswrapper[2575]: W0917 17:34:40.660546    2575 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Sep 17 17:34:40.665684 ip-10-0-143-74 kubenswrapper[2575]: W0917 17:34:40.660548    2575 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Sep 17 17:34:40.665684 ip-10-0-143-74 kubenswrapper[2575]: W0917 17:34:40.660551    2575 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Sep 17 17:34:40.665684 ip-10-0-143-74 kubenswrapper[2575]: W0917 17:34:40.660554    2575 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Sep 17 17:34:40.665684 ip-10-0-143-74 kubenswrapper[2575]: W0917 17:34:40.660556    2575 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Sep 17 17:34:40.665684 ip-10-0-143-74 kubenswrapper[2575]: W0917 17:34:40.660559    2575 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Sep 17 17:34:40.665684 ip-10-0-143-74 kubenswrapper[2575]: W0917 17:34:40.660562    2575 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Sep 17 17:34:40.665684 ip-10-0-143-74 kubenswrapper[2575]: W0917 17:34:40.660564    2575 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Sep 17 17:34:40.665684 ip-10-0-143-74 kubenswrapper[2575]: W0917 17:34:40.660568    2575 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Sep 17 17:34:40.665684 ip-10-0-143-74 kubenswrapper[2575]: W0917 17:34:40.660571    2575 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Sep 17 17:34:40.665684 ip-10-0-143-74 kubenswrapper[2575]: W0917 17:34:40.660574    2575 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Sep 17 17:34:40.665684 ip-10-0-143-74 kubenswrapper[2575]: W0917 17:34:40.660576    2575 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Sep 17 17:34:40.665684 ip-10-0-143-74 kubenswrapper[2575]: W0917 17:34:40.660579    2575 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Sep 17 17:34:40.666149 ip-10-0-143-74 kubenswrapper[2575]: W0917 17:34:40.660581    2575 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Sep 17 17:34:40.666149 ip-10-0-143-74 kubenswrapper[2575]: W0917 17:34:40.660584    2575 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Sep 17 17:34:40.666149 ip-10-0-143-74 kubenswrapper[2575]: W0917 17:34:40.660586    2575 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Sep 17 17:34:40.666149 ip-10-0-143-74 kubenswrapper[2575]: W0917 17:34:40.660589    2575 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Sep 17 17:34:40.666149 ip-10-0-143-74 kubenswrapper[2575]: W0917 17:34:40.660594    2575 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Sep 17 17:34:40.666149 ip-10-0-143-74 kubenswrapper[2575]: W0917 17:34:40.660598    2575 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Sep 17 17:34:40.666149 ip-10-0-143-74 kubenswrapper[2575]: W0917 17:34:40.660601    2575 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Sep 17 17:34:40.666149 ip-10-0-143-74 kubenswrapper[2575]: W0917 17:34:40.660604    2575 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Sep 17 17:34:40.666149 ip-10-0-143-74 kubenswrapper[2575]: W0917 17:34:40.660607    2575 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Sep 17 17:34:40.666149 ip-10-0-143-74 kubenswrapper[2575]: W0917 17:34:40.660610    2575 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Sep 17 17:34:40.666149 ip-10-0-143-74 kubenswrapper[2575]: W0917 17:34:40.660613    2575 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Sep 17 17:34:40.666149 ip-10-0-143-74 kubenswrapper[2575]: W0917 17:34:40.660615    2575 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Sep 17 17:34:40.666149 ip-10-0-143-74 kubenswrapper[2575]: W0917 17:34:40.660618    2575 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Sep 17 17:34:40.666149 ip-10-0-143-74 kubenswrapper[2575]: W0917 17:34:40.660620    2575 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Sep 17 17:34:40.666149 ip-10-0-143-74 kubenswrapper[2575]: W0917 17:34:40.660623    2575 feature_gate.go:328] unrecognized feature gate: EventTTL
Sep 17 17:34:40.666149 ip-10-0-143-74 kubenswrapper[2575]: W0917 17:34:40.660626    2575 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Sep 17 17:34:40.666544 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:34:40.660631    2575 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]}
Sep 17 17:34:40.666544 ip-10-0-143-74 kubenswrapper[2575]: W0917 17:34:40.660743    2575 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Sep 17 17:34:40.666544 ip-10-0-143-74 kubenswrapper[2575]: W0917 17:34:40.660748    2575 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Sep 17 17:34:40.666544 ip-10-0-143-74 kubenswrapper[2575]: W0917 17:34:40.660753    2575 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Sep 17 17:34:40.666544 ip-10-0-143-74 kubenswrapper[2575]: W0917 17:34:40.660756    2575 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Sep 17 17:34:40.666544 ip-10-0-143-74 kubenswrapper[2575]: W0917 17:34:40.660759    2575 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Sep 17 17:34:40.666544 ip-10-0-143-74 kubenswrapper[2575]: W0917 17:34:40.660762    2575 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Sep 17 17:34:40.666544 ip-10-0-143-74 kubenswrapper[2575]: W0917 17:34:40.660765    2575 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Sep 17 17:34:40.666544 ip-10-0-143-74 kubenswrapper[2575]: W0917 17:34:40.660768    2575 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Sep 17 17:34:40.666544 ip-10-0-143-74 kubenswrapper[2575]: W0917 17:34:40.660771    2575 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Sep 17 17:34:40.666544 ip-10-0-143-74 kubenswrapper[2575]: W0917 17:34:40.660773    2575 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Sep 17 17:34:40.666544 ip-10-0-143-74 kubenswrapper[2575]: W0917 17:34:40.660776    2575 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Sep 17 17:34:40.666544 ip-10-0-143-74 kubenswrapper[2575]: W0917 17:34:40.660779    2575 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Sep 17 17:34:40.666544 ip-10-0-143-74 kubenswrapper[2575]: W0917 17:34:40.660782    2575 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Sep 17 17:34:40.666544 ip-10-0-143-74 kubenswrapper[2575]: W0917 17:34:40.660785    2575 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Sep 17 17:34:40.666917 ip-10-0-143-74 kubenswrapper[2575]: W0917 17:34:40.660787    2575 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Sep 17 17:34:40.666917 ip-10-0-143-74 kubenswrapper[2575]: W0917 17:34:40.660790    2575 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Sep 17 17:34:40.666917 ip-10-0-143-74 kubenswrapper[2575]: W0917 17:34:40.660792    2575 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Sep 17 17:34:40.666917 ip-10-0-143-74 kubenswrapper[2575]: W0917 17:34:40.660795    2575 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Sep 17 17:34:40.666917 ip-10-0-143-74 kubenswrapper[2575]: W0917 17:34:40.660798    2575 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Sep 17 17:34:40.666917 ip-10-0-143-74 kubenswrapper[2575]: W0917 17:34:40.660800    2575 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Sep 17 17:34:40.666917 ip-10-0-143-74 kubenswrapper[2575]: W0917 17:34:40.660804    2575 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Sep 17 17:34:40.666917 ip-10-0-143-74 kubenswrapper[2575]: W0917 17:34:40.660808    2575 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Sep 17 17:34:40.666917 ip-10-0-143-74 kubenswrapper[2575]: W0917 17:34:40.660811    2575 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Sep 17 17:34:40.666917 ip-10-0-143-74 kubenswrapper[2575]: W0917 17:34:40.660814    2575 feature_gate.go:328] unrecognized feature gate: Example
Sep 17 17:34:40.666917 ip-10-0-143-74 kubenswrapper[2575]: W0917 17:34:40.660817    2575 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Sep 17 17:34:40.666917 ip-10-0-143-74 kubenswrapper[2575]: W0917 17:34:40.660820    2575 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Sep 17 17:34:40.666917 ip-10-0-143-74 kubenswrapper[2575]: W0917 17:34:40.660823    2575 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Sep 17 17:34:40.666917 ip-10-0-143-74 kubenswrapper[2575]: W0917 17:34:40.660825    2575 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Sep 17 17:34:40.666917 ip-10-0-143-74 kubenswrapper[2575]: W0917 17:34:40.660828    2575 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Sep 17 17:34:40.666917 ip-10-0-143-74 kubenswrapper[2575]: W0917 17:34:40.660830    2575 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Sep 17 17:34:40.666917 ip-10-0-143-74 kubenswrapper[2575]: W0917 17:34:40.660833    2575 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Sep 17 17:34:40.666917 ip-10-0-143-74 kubenswrapper[2575]: W0917 17:34:40.660835    2575 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Sep 17 17:34:40.666917 ip-10-0-143-74 kubenswrapper[2575]: W0917 17:34:40.660838    2575 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Sep 17 17:34:40.667388 ip-10-0-143-74 kubenswrapper[2575]: W0917 17:34:40.660840    2575 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Sep 17 17:34:40.667388 ip-10-0-143-74 kubenswrapper[2575]: W0917 17:34:40.660843    2575 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Sep 17 17:34:40.667388 ip-10-0-143-74 kubenswrapper[2575]: W0917 17:34:40.660845    2575 feature_gate.go:328] unrecognized feature gate: PinnedImages
Sep 17 17:34:40.667388 ip-10-0-143-74 kubenswrapper[2575]: W0917 17:34:40.660848    2575 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Sep 17 17:34:40.667388 ip-10-0-143-74 kubenswrapper[2575]: W0917 17:34:40.660850    2575 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Sep 17 17:34:40.667388 ip-10-0-143-74 kubenswrapper[2575]: W0917 17:34:40.660854    2575 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Sep 17 17:34:40.667388 ip-10-0-143-74 kubenswrapper[2575]: W0917 17:34:40.660858    2575 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Sep 17 17:34:40.667388 ip-10-0-143-74 kubenswrapper[2575]: W0917 17:34:40.660860    2575 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Sep 17 17:34:40.667388 ip-10-0-143-74 kubenswrapper[2575]: W0917 17:34:40.660863    2575 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Sep 17 17:34:40.667388 ip-10-0-143-74 kubenswrapper[2575]: W0917 17:34:40.660865    2575 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Sep 17 17:34:40.667388 ip-10-0-143-74 kubenswrapper[2575]: W0917 17:34:40.660868    2575 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Sep 17 17:34:40.667388 ip-10-0-143-74 kubenswrapper[2575]: W0917 17:34:40.660871    2575 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Sep 17 17:34:40.667388 ip-10-0-143-74 kubenswrapper[2575]: W0917 17:34:40.660874    2575 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Sep 17 17:34:40.667388 ip-10-0-143-74 kubenswrapper[2575]: W0917 17:34:40.660876    2575 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Sep 17 17:34:40.667388 ip-10-0-143-74 kubenswrapper[2575]: W0917 17:34:40.660879    2575 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Sep 17 17:34:40.667388 ip-10-0-143-74 kubenswrapper[2575]: W0917 17:34:40.660881    2575 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Sep 17 17:34:40.667388 ip-10-0-143-74 kubenswrapper[2575]: W0917 17:34:40.660884    2575 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Sep 17 17:34:40.667388 ip-10-0-143-74 kubenswrapper[2575]: W0917 17:34:40.660887    2575 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Sep 17 17:34:40.667388 ip-10-0-143-74 kubenswrapper[2575]: W0917 17:34:40.660889    2575 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Sep 17 17:34:40.667866 ip-10-0-143-74 kubenswrapper[2575]: W0917 17:34:40.660892    2575 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Sep 17 17:34:40.667866 ip-10-0-143-74 kubenswrapper[2575]: W0917 17:34:40.660894    2575 feature_gate.go:328] unrecognized feature gate: OVNObservability
Sep 17 17:34:40.667866 ip-10-0-143-74 kubenswrapper[2575]: W0917 17:34:40.660897    2575 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Sep 17 17:34:40.667866 ip-10-0-143-74 kubenswrapper[2575]: W0917 17:34:40.660899    2575 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Sep 17 17:34:40.667866 ip-10-0-143-74 kubenswrapper[2575]: W0917 17:34:40.660904    2575 feature_gate.go:328] unrecognized feature gate: SignatureStores
Sep 17 17:34:40.667866 ip-10-0-143-74 kubenswrapper[2575]: W0917 17:34:40.660906    2575 feature_gate.go:328] unrecognized feature gate: Example2
Sep 17 17:34:40.667866 ip-10-0-143-74 kubenswrapper[2575]: W0917 17:34:40.660909    2575 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Sep 17 17:34:40.667866 ip-10-0-143-74 kubenswrapper[2575]: W0917 17:34:40.660911    2575 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Sep 17 17:34:40.667866 ip-10-0-143-74 kubenswrapper[2575]: W0917 17:34:40.660914    2575 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Sep 17 17:34:40.667866 ip-10-0-143-74 kubenswrapper[2575]: W0917 17:34:40.660916    2575 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Sep 17 17:34:40.667866 ip-10-0-143-74 kubenswrapper[2575]: W0917 17:34:40.660919    2575 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Sep 17 17:34:40.667866 ip-10-0-143-74 kubenswrapper[2575]: W0917 17:34:40.660921    2575 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Sep 17 17:34:40.667866 ip-10-0-143-74 kubenswrapper[2575]: W0917 17:34:40.660924    2575 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Sep 17 17:34:40.667866 ip-10-0-143-74 kubenswrapper[2575]: W0917 17:34:40.660926    2575 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Sep 17 17:34:40.667866 ip-10-0-143-74 kubenswrapper[2575]: W0917 17:34:40.660929    2575 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Sep 17 17:34:40.667866 ip-10-0-143-74 kubenswrapper[2575]: W0917 17:34:40.660931    2575 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Sep 17 17:34:40.667866 ip-10-0-143-74 kubenswrapper[2575]: W0917 17:34:40.660934    2575 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Sep 17 17:34:40.667866 ip-10-0-143-74 kubenswrapper[2575]: W0917 17:34:40.660937    2575 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Sep 17 17:34:40.667866 ip-10-0-143-74 kubenswrapper[2575]: W0917 17:34:40.660939    2575 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Sep 17 17:34:40.668338 ip-10-0-143-74 kubenswrapper[2575]: W0917 17:34:40.660942    2575 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Sep 17 17:34:40.668338 ip-10-0-143-74 kubenswrapper[2575]: W0917 17:34:40.660944    2575 feature_gate.go:328] unrecognized feature gate: EventTTL
Sep 17 17:34:40.668338 ip-10-0-143-74 kubenswrapper[2575]: W0917 17:34:40.660947    2575 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Sep 17 17:34:40.668338 ip-10-0-143-74 kubenswrapper[2575]: W0917 17:34:40.660949    2575 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Sep 17 17:34:40.668338 ip-10-0-143-74 kubenswrapper[2575]: W0917 17:34:40.660952    2575 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Sep 17 17:34:40.668338 ip-10-0-143-74 kubenswrapper[2575]: W0917 17:34:40.660954    2575 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Sep 17 17:34:40.668338 ip-10-0-143-74 kubenswrapper[2575]: W0917 17:34:40.660957    2575 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Sep 17 17:34:40.668338 ip-10-0-143-74 kubenswrapper[2575]: W0917 17:34:40.660960    2575 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Sep 17 17:34:40.668338 ip-10-0-143-74 kubenswrapper[2575]: W0917 17:34:40.660962    2575 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Sep 17 17:34:40.668338 ip-10-0-143-74 kubenswrapper[2575]: W0917 17:34:40.660965    2575 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Sep 17 17:34:40.668338 ip-10-0-143-74 kubenswrapper[2575]: W0917 17:34:40.660968    2575 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Sep 17 17:34:40.668338 ip-10-0-143-74 kubenswrapper[2575]: W0917 17:34:40.660970    2575 feature_gate.go:328] unrecognized feature gate: DualReplica
Sep 17 17:34:40.668338 ip-10-0-143-74 kubenswrapper[2575]: W0917 17:34:40.660973    2575 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Sep 17 17:34:40.668338 ip-10-0-143-74 kubenswrapper[2575]: W0917 17:34:40.660976    2575 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Sep 17 17:34:40.668338 ip-10-0-143-74 kubenswrapper[2575]: W0917 17:34:40.660978    2575 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Sep 17 17:34:40.668338 ip-10-0-143-74 kubenswrapper[2575]: W0917 17:34:40.660981    2575 feature_gate.go:328] unrecognized feature gate: OSStreams
Sep 17 17:34:40.668338 ip-10-0-143-74 kubenswrapper[2575]: W0917 17:34:40.660984    2575 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Sep 17 17:34:40.668338 ip-10-0-143-74 kubenswrapper[2575]: W0917 17:34:40.660987    2575 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Sep 17 17:34:40.668338 ip-10-0-143-74 kubenswrapper[2575]: W0917 17:34:40.660990    2575 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Sep 17 17:34:40.668338 ip-10-0-143-74 kubenswrapper[2575]: W0917 17:34:40.660992    2575 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Sep 17 17:34:40.668864 ip-10-0-143-74 kubenswrapper[2575]: W0917 17:34:40.660995    2575 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Sep 17 17:34:40.668864 ip-10-0-143-74 kubenswrapper[2575]: W0917 17:34:40.660997    2575 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Sep 17 17:34:40.668864 ip-10-0-143-74 kubenswrapper[2575]: W0917 17:34:40.661000    2575 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Sep 17 17:34:40.668864 ip-10-0-143-74 kubenswrapper[2575]: W0917 17:34:40.661003    2575 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Sep 17 17:34:40.668864 ip-10-0-143-74 kubenswrapper[2575]: W0917 17:34:40.661005    2575 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Sep 17 17:34:40.668864 ip-10-0-143-74 kubenswrapper[2575]: W0917 17:34:40.661008    2575 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Sep 17 17:34:40.668864 ip-10-0-143-74 kubenswrapper[2575]: W0917 17:34:40.661010    2575 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Sep 17 17:34:40.668864 ip-10-0-143-74 kubenswrapper[2575]: W0917 17:34:40.661013    2575 feature_gate.go:328] unrecognized feature gate: NewOLM
Sep 17 17:34:40.668864 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:34:40.661018    2575 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]}
Sep 17 17:34:40.668864 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:34:40.661123    2575 server.go:962] "Client rotation is on, will bootstrap in background"
Sep 17 17:34:40.668864 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:34:40.662967    2575 bootstrap.go:101] "Use the bootstrap credentials to request a cert, and set kubeconfig to point to the certificate dir"
Sep 17 17:34:40.668864 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:34:40.663891    2575 server.go:1019] "Starting client certificate rotation"
Sep 17 17:34:40.668864 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:34:40.664000    2575 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kube-apiserver-client-kubelet"
Sep 17 17:34:40.668864 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:34:40.664035    2575 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kube-apiserver-client-kubelet"
Sep 17 17:34:40.669209 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:34:40.666430    2575 dynamic_cafile_content.go:123] "Loaded a new CA Bundle and Verifier" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt"
Sep 17 17:34:40.670483 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:34:40.670465    2575 dynamic_cafile_content.go:161] "Starting controller" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt"
Sep 17 17:34:40.682322 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:34:40.682296    2575 log.go:25] "Validated CRI v1 runtime API"
Sep 17 17:34:40.683964 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:34:40.683949    2575 log.go:25] "Validated CRI v1 image API"
Sep 17 17:34:40.684859 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:34:40.684841    2575 server.go:1429] "Using cgroup driver setting received from the CRI runtime" cgroupDriver="systemd"
Sep 17 17:34:40.689105 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:34:40.689080    2575 fs.go:135] Filesystem UUIDs: map[7B77-95E7:/dev/nvme0n1p2 873d1eea-4b79-4ae6-9779-062962be9f4d:/dev/nvme0n1p4 9610c37c-2fe2-4ab1-8c59-46ffc6733f79:/dev/nvme0n1p3]
Sep 17 17:34:40.689157 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:34:40.689104    2575 fs.go:136] Filesystem partitions: map[/dev/nvme0n1p3:{mountpoint:/boot major:259 minor:3 fsType:ext4 blockSize:0} /dev/nvme0n1p4:{mountpoint:/var major:259 minor:4 fsType:xfs blockSize:0} /dev/shm:{mountpoint:/dev/shm major:0 minor:22 fsType:tmpfs blockSize:0} /run:{mountpoint:/run major:0 minor:24 fsType:tmpfs blockSize:0} /tmp:{mountpoint:/tmp major:0 minor:31 fsType:tmpfs blockSize:0} composefs_0-33:{mountpoint:/ major:0 minor:33 fsType:overlay blockSize:0}]
Sep 17 17:34:40.695021 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:34:40.694912    2575 manager.go:216] Machine: {Timestamp:2026-09-17 17:34:40.693552994 +0000 UTC m=+0.323882960 CPUVendorID:GenuineIntel NumCores:8 NumPhysicalCores:4 NumSockets:1 CpuFrequency:3109156 MemoryCapacity:33164484608 SwapCapacity:0 MemoryByType:map[] NVMInfo:{MemoryModeCapacity:0 AppDirectModeCapacity:0 AvgPowerBudget:0} HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] MachineID:ec2729e84cf3bd90997be7c5e03672b5 SystemUUID:ec2729e8-4cf3-bd90-997b-e7c5e03672b5 BootID:546f2c02-8f2d-488f-b67b-7a883478d383 Filesystems:[{Device:/run DeviceMajor:0 DeviceMinor:24 Capacity:6632898560 Type:vfs Inodes:819200 HasInodes:true} {Device:/dev/nvme0n1p4 DeviceMajor:259 DeviceMinor:4 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:composefs_0-33 DeviceMajor:0 DeviceMinor:33 Capacity:6111232 Type:vfs Inodes:18446744073709551615 HasInodes:true} {Device:/tmp DeviceMajor:0 DeviceMinor:31 Capacity:16582242304 Type:vfs Inodes:1048576 HasInodes:true} {Device:/dev/nvme0n1p3 DeviceMajor:259 DeviceMinor:3 Capacity:366869504 Type:vfs Inodes:98304 HasInodes:true} {Device:/dev/shm DeviceMajor:0 DeviceMinor:22 Capacity:16582242304 Type:vfs Inodes:4048399 HasInodes:true}] DiskMap:map[259:0:{Name:nvme0n1 Major:259 Minor:0 Size:128849018880 Scheduler:none}] NetworkDevices:[{Name:br-ex MacAddress:02:ff:c3:84:7a:c5 Speed:0 Mtu:9001} {Name:ens5 MacAddress:02:ff:c3:84:7a:c5 Speed:0 Mtu:9001} {Name:ovs-system MacAddress:2a:ce:28:42:75:00 Speed:0 Mtu:1500}] Topology:[{Id:0 Memory:33164484608 HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] Cores:[{Id:0 Threads:[0 4] Caches:[{Id:0 Size:32768 Type:Data Level:1} {Id:0 Size:32768 Type:Instruction Level:1} {Id:0 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:1 Threads:[1 5] Caches:[{Id:1 Size:32768 Type:Data Level:1} {Id:1 Size:32768 Type:Instruction Level:1} {Id:1 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:2 Threads:[2 6] Caches:[{Id:2 Size:32768 Type:Data Level:1} {Id:2 Size:32768 Type:Instruction Level:1} {Id:2 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:3 Threads:[3 7] Caches:[{Id:3 Size:32768 Type:Data Level:1} {Id:3 Size:32768 Type:Instruction Level:1} {Id:3 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:}] Caches:[{Id:0 Size:37486592 Type:Unified Level:3}] Distances:[10]}] CloudProvider:Unknown InstanceType:Unknown InstanceID:None}
Sep 17 17:34:40.695021 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:34:40.695016    2575 manager_no_libpfm.go:29] cAdvisor is build without cgo and/or libpfm support. Perf event counters are not available.
Sep 17 17:34:40.695144 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:34:40.695132    2575 manager.go:232] Version: {KernelVersion:5.14.0-570.141.1.el9_6.x86_64 ContainerOsVersion:Red Hat Enterprise Linux CoreOS 9.6.20260914-0 (Plow) DockerVersion: DockerAPIVersion: CadvisorVersion: CadvisorRevision:}
Sep 17 17:34:40.696218 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:34:40.696204    2575 reflector.go:439] "Caches populated" logger="kubernetes.io/kube-apiserver-client-kubelet" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162"
Sep 17 17:34:40.698023 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:34:40.697999    2575 container_manager_linux.go:273] "Container manager verified user specified cgroup-root exists" cgroupRoot=[]
Sep 17 17:34:40.698156 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:34:40.698024    2575 container_manager_linux.go:278] "Creating Container Manager object based on Node Config" nodeConfig={"NodeName":"ip-10-0-143-74.ec2.internal","RuntimeCgroupsName":"/system.slice/crio.service","SystemCgroupsName":"/system.slice","KubeletCgroupsName":"","KubeletOOMScoreAdj":-999,"ContainerRuntime":"","CgroupsPerQOS":true,"CgroupRoot":"/","CgroupDriver":"systemd","KubeletRootDir":"/var/lib/kubelet","ProtectKernelDefaults":true,"KubeReservedCgroupName":"","SystemReservedCgroupName":"","ReservedSystemCPUs":{},"EnforceNodeAllocatable":{"pods":{}},"KubeReserved":null,"SystemReserved":{"cpu":"500m","ephemeral-storage":"1Gi","memory":"1Gi"},"HardEvictionThresholds":[{"Signal":"imagefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.15},"GracePeriod":0,"MinReclaim":null},{"Signal":"imagefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null},{"Signal":"memory.available","Operator":"LessThan","Value":{"Quantity":"100Mi","Percentage":0},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.1},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null}],"QOSReserved":{},"CPUManagerPolicy":"none","CPUManagerPolicyOptions":null,"TopologyManagerScope":"container","CPUManagerReconcilePeriod":10000000000,"MemoryManagerPolicy":"None","MemoryManagerReservedMemory":null,"PodPidsLimit":4096,"EnforceCPULimits":true,"CPUCFSQuotaPeriod":100000000,"TopologyManagerPolicy":"none","TopologyManagerPolicyOptions":null,"CgroupVersion":2}
Sep 17 17:34:40.698625 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:34:40.698614    2575 topology_manager.go:138] "Creating topology manager with none policy"
Sep 17 17:34:40.698658 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:34:40.698626    2575 container_manager_linux.go:309] "Creating device plugin manager"
Sep 17 17:34:40.698658 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:34:40.698634    2575 manager.go:141] "Creating Device Plugin manager" path="/var/lib/kubelet/device-plugins/kubelet.sock"
Sep 17 17:34:40.698658 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:34:40.698649    2575 server.go:74] "Creating device plugin registration server" version="v1beta1" socket="/var/lib/kubelet/device-plugins/kubelet.sock"
Sep 17 17:34:40.698737 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:34:40.698729    2575 container_manager_linux.go:318] "Creating Dynamic Resource Allocation (DRA) manager"
Sep 17 17:34:40.701321 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:34:40.701310    2575 state_mem.go:36] "Initialized new in-memory state store"
Sep 17 17:34:40.701452 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:34:40.701432    2575 server.go:1240] "Using root directory" path="/var/lib/kubelet"
Sep 17 17:34:40.701482 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:34:40.701477    2575 kubelet.go:486] "Attempting to sync node with API server"
Sep 17 17:34:40.701514 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:34:40.701495    2575 kubelet.go:387] "Adding static pod path" path="/etc/kubernetes/manifests"
Sep 17 17:34:40.701514 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:34:40.701510    2575 file.go:69] "Watching path" path="/etc/kubernetes/manifests"
Sep 17 17:34:40.701565 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:34:40.701519    2575 kubelet.go:398] "Adding apiserver pod source"
Sep 17 17:34:40.701565 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:34:40.701528    2575 apiserver.go:42] "Waiting for node sync before watching apiserver pods"
Sep 17 17:34:40.701912 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:34:40.701900    2575 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking"
Sep 17 17:34:40.701952 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:34:40.701925    2575 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking"
Sep 17 17:34:40.704836 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:34:40.704819    2575 kuberuntime_manager.go:301] "Container runtime initialized" containerRuntime="cri-o" version="1.34.13-6.rhaos4.21.git6253162.el9" apiVersion="v1"
Sep 17 17:34:40.706665 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:34:40.706651    2575 kubelet.go:960] "Not starting ClusterTrustBundle informer because we are in static kubelet mode or the ClusterTrustBundleProjection featuregate is disabled"
Sep 17 17:34:40.706718 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:34:40.706682    2575 kubelet.go:984] "Not starting PodCertificateRequest manager because we are in static kubelet mode or the PodCertificateProjection feature gate is disabled"
Sep 17 17:34:40.707518 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:34:40.707508    2575 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/portworx-volume"
Sep 17 17:34:40.707553 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:34:40.707525    2575 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/empty-dir"
Sep 17 17:34:40.707553 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:34:40.707531    2575 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/git-repo"
Sep 17 17:34:40.707553 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:34:40.707540    2575 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/host-path"
Sep 17 17:34:40.707553 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:34:40.707552    2575 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/nfs"
Sep 17 17:34:40.707663 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:34:40.707557    2575 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/secret"
Sep 17 17:34:40.707663 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:34:40.707571    2575 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/iscsi"
Sep 17 17:34:40.707663 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:34:40.707577    2575 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/downward-api"
Sep 17 17:34:40.707663 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:34:40.707583    2575 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/fc"
Sep 17 17:34:40.707663 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:34:40.707589    2575 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/configmap"
Sep 17 17:34:40.707663 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:34:40.707607    2575 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/projected"
Sep 17 17:34:40.707663 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:34:40.707612    2575 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/local-volume"
Sep 17 17:34:40.708345 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:34:40.708335    2575 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/csi"
Sep 17 17:34:40.708345 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:34:40.708346    2575 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/image"
Sep 17 17:34:40.711920 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:34:40.711908    2575 server.go:1268] "Started kubelet"
Sep 17 17:34:40.712037 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:34:40.711956    2575 server.go:180] "Starting to listen" address="0.0.0.0" port=10250
Sep 17 17:34:40.712308 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:34:40.712244    2575 ratelimit.go:56] "Setting rate limiting for endpoint" service="podresources" qps=100 burstTokens=10
Sep 17 17:34:40.712374 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:34:40.712326    2575 server_v1.go:49] "podresources" method="list" useActivePods=true
Sep 17 17:34:40.713359 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:34:40.713330    2575 server.go:249] "Starting to serve the podresources API" endpoint="unix:/var/lib/kubelet/pod-resources/kubelet.sock"
Sep 17 17:34:40.713972 ip-10-0-143-74 systemd[1]: Started Kubernetes Kubelet.
Sep 17 17:34:40.717620 ip-10-0-143-74 kubenswrapper[2575]: E0917 17:34:40.717564    2575 reflector.go:205] "Failed to watch" err="failed to list *v1.Service: services is forbidden: User \"system:anonymous\" cannot list resource \"services\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Service"
Sep 17 17:34:40.717825 ip-10-0-143-74 kubenswrapper[2575]: E0917 17:34:40.717770    2575 reflector.go:205] "Failed to watch" err="failed to list *v1.Node: nodes \"ip-10-0-143-74.ec2.internal\" is forbidden: User \"system:anonymous\" cannot list resource \"nodes\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Node"
Sep 17 17:34:40.718028 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:34:40.718008    2575 csi_plugin.go:990] Failed to contact API server when waiting for CSINode publishing: csinodes.storage.k8s.io "ip-10-0-143-74.ec2.internal" is forbidden: User "system:anonymous" cannot get resource "csinodes" in API group "storage.k8s.io" at the cluster scope
Sep 17 17:34:40.719038 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:34:40.719022    2575 server.go:310] "Adding debug handlers to kubelet server"
Sep 17 17:34:40.719884 ip-10-0-143-74 kubenswrapper[2575]: E0917 17:34:40.719862    2575 kubelet.go:1638] "Image garbage collection failed once. Stats initialization may not have completed yet" err="failed to get imageFs info: unable to find data in memory cache"
Sep 17 17:34:40.721342 ip-10-0-143-74 kubenswrapper[2575]: E0917 17:34:40.720461    2575 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-143-74.ec2.internal.18d62c2ea14158ae  default    0 0001-01-01 00:00:00 +0000 UTC <nil> <nil> map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-143-74.ec2.internal,UID:ip-10-0-143-74.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:Starting,Message:Starting kubelet.,Source:EventSource{Component:kubelet,Host:ip-10-0-143-74.ec2.internal,},FirstTimestamp:2026-09-17 17:34:40.711882926 +0000 UTC m=+0.342212891,LastTimestamp:2026-09-17 17:34:40.711882926 +0000 UTC m=+0.342212891,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-143-74.ec2.internal,}"
Sep 17 17:34:40.722526 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:34:40.722503    2575 fs_resource_analyzer.go:67] "Starting FS ResourceAnalyzer"
Sep 17 17:34:40.722594 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:34:40.722551    2575 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kubelet-serving"
Sep 17 17:34:40.722643 ip-10-0-143-74 kubenswrapper[2575]: E0917 17:34:40.722630    2575 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-143-74.ec2.internal\" not found"
Sep 17 17:34:40.723186 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:34:40.723037    2575 desired_state_of_world_populator.go:146] "Desired state populator starts to run"
Sep 17 17:34:40.724251 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:34:40.724146    2575 volume_manager.go:311] "The desired_state_of_world populator starts"
Sep 17 17:34:40.724251 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:34:40.724166    2575 volume_manager.go:313] "Starting Kubelet Volume Manager"
Sep 17 17:34:40.724388 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:34:40.724265    2575 reconstruct.go:97] "Volume reconstruction finished"
Sep 17 17:34:40.724388 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:34:40.724273    2575 reconciler.go:29] "Reconciler: start to sync state"
Sep 17 17:34:40.725433 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:34:40.725417    2575 factory.go:153] Registering CRI-O factory
Sep 17 17:34:40.725433 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:34:40.725433    2575 factory.go:223] Registration of the crio container factory successfully
Sep 17 17:34:40.725579 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:34:40.725512    2575 factory.go:221] Registration of the containerd container factory failed: unable to create containerd client: containerd: cannot unix dial containerd api service: dial unix /run/containerd/containerd.sock: connect: no such file or directory
Sep 17 17:34:40.725579 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:34:40.725523    2575 factory.go:55] Registering systemd factory
Sep 17 17:34:40.725579 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:34:40.725529    2575 factory.go:223] Registration of the systemd container factory successfully
Sep 17 17:34:40.725579 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:34:40.725543    2575 factory.go:103] Registering Raw factory
Sep 17 17:34:40.725579 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:34:40.725551    2575 manager.go:1196] Started watching for new ooms in manager
Sep 17 17:34:40.726069 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:34:40.726054    2575 manager.go:349] Starting recovery of all containers
Sep 17 17:34:40.728968 ip-10-0-143-74 kubenswrapper[2575]: E0917 17:34:40.728881    2575 controller.go:145] "Failed to ensure lease exists, will retry" err="leases.coordination.k8s.io \"ip-10-0-143-74.ec2.internal\" is forbidden: User \"system:anonymous\" cannot get resource \"leases\" in API group \"coordination.k8s.io\" in the namespace \"kube-node-lease\"" interval="200ms"
Sep 17 17:34:40.729056 ip-10-0-143-74 kubenswrapper[2575]: E0917 17:34:40.728996    2575 reflector.go:205] "Failed to watch" err="failed to list *v1.CSIDriver: csidrivers.storage.k8s.io is forbidden: User \"system:anonymous\" cannot list resource \"csidrivers\" in API group \"storage.k8s.io\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.CSIDriver"
Sep 17 17:34:40.732190 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:34:40.732159    2575 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-p7dff"
Sep 17 17:34:40.736255 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:34:40.736237    2575 manager.go:354] Recovery completed
Sep 17 17:34:40.739788 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:34:40.739739    2575 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-p7dff"
Sep 17 17:34:40.741565 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:34:40.741551    2575 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Sep 17 17:34:40.744123 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:34:40.744100    2575 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-143-74.ec2.internal" event="NodeHasSufficientMemory"
Sep 17 17:34:40.744196 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:34:40.744147    2575 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-143-74.ec2.internal" event="NodeHasNoDiskPressure"
Sep 17 17:34:40.744196 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:34:40.744164    2575 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-143-74.ec2.internal" event="NodeHasSufficientPID"
Sep 17 17:34:40.744892 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:34:40.744877    2575 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Sep 17 17:34:40.744995 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:34:40.744879    2575 cpu_manager.go:222] "Starting CPU manager" policy="none"
Sep 17 17:34:40.744995 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:34:40.744906    2575 cpu_manager.go:223] "Reconciling" reconcilePeriod="10s"
Sep 17 17:34:40.744995 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:34:40.744920    2575 state_mem.go:36] "Initialized new in-memory state store"
Sep 17 17:34:40.745643 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:34:40.745627    2575 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-143-74.ec2.internal" event="NodeHasSufficientMemory"
Sep 17 17:34:40.745714 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:34:40.745657    2575 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-143-74.ec2.internal" event="NodeHasNoDiskPressure"
Sep 17 17:34:40.745714 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:34:40.745670    2575 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-143-74.ec2.internal" event="NodeHasSufficientPID"
Sep 17 17:34:40.747275 ip-10-0-143-74 kubenswrapper[2575]: E0917 17:34:40.747212    2575 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-143-74.ec2.internal.18d62c2ea32d59c8  default    0 0001-01-01 00:00:00 +0000 UTC <nil> <nil> map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-143-74.ec2.internal,UID:ip-10-0-143-74.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:NodeHasSufficientMemory,Message:Node ip-10-0-143-74.ec2.internal status is now: NodeHasSufficientMemory,Source:EventSource{Component:kubelet,Host:ip-10-0-143-74.ec2.internal,},FirstTimestamp:2026-09-17 17:34:40.74412692 +0000 UTC m=+0.374456895,LastTimestamp:2026-09-17 17:34:40.74412692 +0000 UTC m=+0.374456895,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-143-74.ec2.internal,}"
Sep 17 17:34:40.755566 ip-10-0-143-74 kubenswrapper[2575]: E0917 17:34:40.755501    2575 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-143-74.ec2.internal.18d62c2ea32dc7a4  default    0 0001-01-01 00:00:00 +0000 UTC <nil> <nil> map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-143-74.ec2.internal,UID:ip-10-0-143-74.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:NodeHasNoDiskPressure,Message:Node ip-10-0-143-74.ec2.internal status is now: NodeHasNoDiskPressure,Source:EventSource{Component:kubelet,Host:ip-10-0-143-74.ec2.internal,},FirstTimestamp:2026-09-17 17:34:40.744155044 +0000 UTC m=+0.374485012,LastTimestamp:2026-09-17 17:34:40.744155044 +0000 UTC m=+0.374485012,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-143-74.ec2.internal,}"
Sep 17 17:34:40.758835 ip-10-0-143-74 kubenswrapper[2575]: E0917 17:34:40.758763    2575 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-143-74.ec2.internal.18d62c2ea32e00cc  default    0 0001-01-01 00:00:00 +0000 UTC <nil> <nil> map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-143-74.ec2.internal,UID:ip-10-0-143-74.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:NodeHasSufficientPID,Message:Node ip-10-0-143-74.ec2.internal status is now: NodeHasSufficientPID,Source:EventSource{Component:kubelet,Host:ip-10-0-143-74.ec2.internal,},FirstTimestamp:2026-09-17 17:34:40.744169676 +0000 UTC m=+0.374499643,LastTimestamp:2026-09-17 17:34:40.744169676 +0000 UTC m=+0.374499643,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-143-74.ec2.internal,}"
Sep 17 17:34:40.764939 ip-10-0-143-74 kubenswrapper[2575]: E0917 17:34:40.764863    2575 event.go:359] "Server rejected event (will not retry!)" err="events \"ip-10-0-143-74.ec2.internal.18d62c2ea32d59c8\" is forbidden: User \"system:anonymous\" cannot patch resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-143-74.ec2.internal.18d62c2ea32d59c8  default    0 0001-01-01 00:00:00 +0000 UTC <nil> <nil> map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-143-74.ec2.internal,UID:ip-10-0-143-74.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:NodeHasSufficientMemory,Message:Node ip-10-0-143-74.ec2.internal status is now: NodeHasSufficientMemory,Source:EventSource{Component:kubelet,Host:ip-10-0-143-74.ec2.internal,},FirstTimestamp:2026-09-17 17:34:40.74412692 +0000 UTC m=+0.374456895,LastTimestamp:2026-09-17 17:34:40.745642364 +0000 UTC m=+0.375972333,Count:2,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-143-74.ec2.internal,}"
Sep 17 17:34:40.769304 ip-10-0-143-74 kubenswrapper[2575]: E0917 17:34:40.769219    2575 event.go:359] "Server rejected event (will not retry!)" err="events \"ip-10-0-143-74.ec2.internal.18d62c2ea32dc7a4\" is forbidden: User \"system:anonymous\" cannot patch resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-143-74.ec2.internal.18d62c2ea32dc7a4  default    0 0001-01-01 00:00:00 +0000 UTC <nil> <nil> map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-143-74.ec2.internal,UID:ip-10-0-143-74.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:NodeHasNoDiskPressure,Message:Node ip-10-0-143-74.ec2.internal status is now: NodeHasNoDiskPressure,Source:EventSource{Component:kubelet,Host:ip-10-0-143-74.ec2.internal,},FirstTimestamp:2026-09-17 17:34:40.744155044 +0000 UTC m=+0.374485012,LastTimestamp:2026-09-17 17:34:40.74566361 +0000 UTC m=+0.375993578,Count:2,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-143-74.ec2.internal,}"
Sep 17 17:34:40.773492 ip-10-0-143-74 kubenswrapper[2575]: E0917 17:34:40.773392    2575 event.go:359] "Server rejected event (will not retry!)" err="events \"ip-10-0-143-74.ec2.internal.18d62c2ea32e00cc\" is forbidden: User \"system:anonymous\" cannot patch resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-143-74.ec2.internal.18d62c2ea32e00cc  default    0 0001-01-01 00:00:00 +0000 UTC <nil> <nil> map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-143-74.ec2.internal,UID:ip-10-0-143-74.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:NodeHasSufficientPID,Message:Node ip-10-0-143-74.ec2.internal status is now: NodeHasSufficientPID,Source:EventSource{Component:kubelet,Host:ip-10-0-143-74.ec2.internal,},FirstTimestamp:2026-09-17 17:34:40.744169676 +0000 UTC m=+0.374499643,LastTimestamp:2026-09-17 17:34:40.745676262 +0000 UTC m=+0.376006228,Count:2,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-143-74.ec2.internal,}"
Sep 17 17:34:40.893770 ip-10-0-143-74 kubenswrapper[2575]: E0917 17:34:40.823326    2575 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-143-74.ec2.internal\" not found"
Sep 17 17:34:40.893770 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:34:40.892784    2575 policy_none.go:49] "None policy: Start"
Sep 17 17:34:40.893770 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:34:40.892821    2575 memory_manager.go:187] "Starting memorymanager" policy="None"
Sep 17 17:34:40.893770 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:34:40.892835    2575 state_mem.go:36] "Initializing new in-memory state store" logger="Memory Manager state checkpoint"
Sep 17 17:34:40.895014 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:34:40.894961    2575 policy_none.go:47] "Start"
Sep 17 17:34:40.924485 ip-10-0-143-74 kubenswrapper[2575]: E0917 17:34:40.924462    2575 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-143-74.ec2.internal\" not found"
Sep 17 17:34:40.931089 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:34:40.931069    2575 manager.go:337] "Starting Device Plugin manager"
Sep 17 17:34:40.931186 ip-10-0-143-74 kubenswrapper[2575]: E0917 17:34:40.931109    2575 manager.go:513] "Failed to read data from checkpoint" err="checkpoint is not found" checkpoint="kubelet_internal_checkpoint"
Sep 17 17:34:40.931186 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:34:40.931123    2575 server.go:87] "Starting device plugin registration server"
Sep 17 17:34:40.931451 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:34:40.931425    2575 eviction_manager.go:189] "Eviction manager: starting control loop"
Sep 17 17:34:40.931532 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:34:40.931459    2575 container_log_manager.go:146] "Initializing container log rotate workers" workers=1 monitorPeriod="10s"
Sep 17 17:34:40.931585 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:34:40.931545    2575 plugin_watcher.go:51] "Plugin Watcher Start" path="/var/lib/kubelet/plugins_registry"
Sep 17 17:34:40.931637 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:34:40.931626    2575 plugin_manager.go:116] "The desired_state_of_world populator (plugin watcher) starts"
Sep 17 17:34:40.931686 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:34:40.931637    2575 plugin_manager.go:118] "Starting Kubelet Plugin Manager"
Sep 17 17:34:40.932149 ip-10-0-143-74 kubenswrapper[2575]: E0917 17:34:40.932127    2575 controller.go:145] "Failed to ensure lease exists, will retry" err="leases.coordination.k8s.io \"ip-10-0-143-74.ec2.internal\" is forbidden: User \"system:anonymous\" cannot get resource \"leases\" in API group \"coordination.k8s.io\" in the namespace \"kube-node-lease\"" interval="400ms"
Sep 17 17:34:40.932254 ip-10-0-143-74 kubenswrapper[2575]: E0917 17:34:40.932159    2575 eviction_manager.go:267] "eviction manager: failed to check if we have separate container filesystem. Ignoring." err="non-existent label \"crio-containers\""
Sep 17 17:34:40.932254 ip-10-0-143-74 kubenswrapper[2575]: E0917 17:34:40.932209    2575 eviction_manager.go:292] "Eviction manager: failed to get summary stats" err="failed to get node info: node \"ip-10-0-143-74.ec2.internal\" not found"
Sep 17 17:34:40.959268 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:34:40.959235    2575 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv4"
Sep 17 17:34:40.960683 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:34:40.960662    2575 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv6"
Sep 17 17:34:40.960784 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:34:40.960708    2575 status_manager.go:244] "Starting to sync pod status with apiserver"
Sep 17 17:34:40.960784 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:34:40.960732    2575 kubelet.go:2448] "Starting kubelet main sync loop"
Sep 17 17:34:40.960784 ip-10-0-143-74 kubenswrapper[2575]: E0917 17:34:40.960777    2575 kubelet.go:2472] "Skipping pod synchronization" err="PLEG is not healthy: pleg has yet to be successful"
Sep 17 17:34:40.963705 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:34:40.963682    2575 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160"
Sep 17 17:34:41.032616 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:34:41.032590    2575 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Sep 17 17:34:41.033687 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:34:41.033667    2575 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-143-74.ec2.internal" event="NodeHasSufficientMemory"
Sep 17 17:34:41.033815 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:34:41.033697    2575 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-143-74.ec2.internal" event="NodeHasNoDiskPressure"
Sep 17 17:34:41.033815 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:34:41.033710    2575 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-143-74.ec2.internal" event="NodeHasSufficientPID"
Sep 17 17:34:41.033815 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:34:41.033741    2575 kubelet_node_status.go:78] "Attempting to register node" node="ip-10-0-143-74.ec2.internal"
Sep 17 17:34:41.040488 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:34:41.040471    2575 kubelet_node_status.go:81] "Successfully registered node" node="ip-10-0-143-74.ec2.internal"
Sep 17 17:34:41.040579 ip-10-0-143-74 kubenswrapper[2575]: E0917 17:34:41.040493    2575 kubelet_node_status.go:536] "Error updating node status, will retry" err="error getting node \"ip-10-0-143-74.ec2.internal\": node \"ip-10-0-143-74.ec2.internal\" not found"
Sep 17 17:34:41.058986 ip-10-0-143-74 kubenswrapper[2575]: E0917 17:34:41.058957    2575 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-143-74.ec2.internal\" not found"
Sep 17 17:34:41.061015 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:34:41.060998    2575 kubelet.go:2535] "SyncLoop ADD" source="file" pods=["openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-143-74.ec2.internal","kube-system/kube-apiserver-proxy-ip-10-0-143-74.ec2.internal"]
Sep 17 17:34:41.061072 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:34:41.061061    2575 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Sep 17 17:34:41.062454 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:34:41.062430    2575 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-143-74.ec2.internal" event="NodeHasSufficientMemory"
Sep 17 17:34:41.062532 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:34:41.062470    2575 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-143-74.ec2.internal" event="NodeHasNoDiskPressure"
Sep 17 17:34:41.062532 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:34:41.062481    2575 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-143-74.ec2.internal" event="NodeHasSufficientPID"
Sep 17 17:34:41.064829 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:34:41.064815    2575 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Sep 17 17:34:41.064967 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:34:41.064953    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-143-74.ec2.internal"
Sep 17 17:34:41.065024 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:34:41.064987    2575 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Sep 17 17:34:41.065513 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:34:41.065498    2575 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-143-74.ec2.internal" event="NodeHasSufficientMemory"
Sep 17 17:34:41.065513 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:34:41.065508    2575 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-143-74.ec2.internal" event="NodeHasSufficientMemory"
Sep 17 17:34:41.065646 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:34:41.065526    2575 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-143-74.ec2.internal" event="NodeHasNoDiskPressure"
Sep 17 17:34:41.065646 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:34:41.065542    2575 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-143-74.ec2.internal" event="NodeHasSufficientPID"
Sep 17 17:34:41.065646 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:34:41.065529    2575 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-143-74.ec2.internal" event="NodeHasNoDiskPressure"
Sep 17 17:34:41.065777 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:34:41.065680    2575 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-143-74.ec2.internal" event="NodeHasSufficientPID"
Sep 17 17:34:41.067951 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:34:41.067935    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-143-74.ec2.internal"
Sep 17 17:34:41.068051 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:34:41.067965    2575 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Sep 17 17:34:41.068705 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:34:41.068690    2575 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-143-74.ec2.internal" event="NodeHasSufficientMemory"
Sep 17 17:34:41.068776 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:34:41.068718    2575 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-143-74.ec2.internal" event="NodeHasNoDiskPressure"
Sep 17 17:34:41.068776 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:34:41.068727    2575 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-143-74.ec2.internal" event="NodeHasSufficientPID"
Sep 17 17:34:41.083016 ip-10-0-143-74 kubenswrapper[2575]: E0917 17:34:41.082995    2575 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-143-74.ec2.internal\" not found" node="ip-10-0-143-74.ec2.internal"
Sep 17 17:34:41.086806 ip-10-0-143-74 kubenswrapper[2575]: E0917 17:34:41.086791    2575 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-143-74.ec2.internal\" not found" node="ip-10-0-143-74.ec2.internal"
Sep 17 17:34:41.126541 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:34:41.126511    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/50d07cdbe5bdbd290f2ca595730f87af-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-143-74.ec2.internal\" (UID: \"50d07cdbe5bdbd290f2ca595730f87af\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-143-74.ec2.internal"
Sep 17 17:34:41.126663 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:34:41.126560    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/50d07cdbe5bdbd290f2ca595730f87af-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-143-74.ec2.internal\" (UID: \"50d07cdbe5bdbd290f2ca595730f87af\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-143-74.ec2.internal"
Sep 17 17:34:41.126663 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:34:41.126587    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/d937ab292439efa4b2d5e0f9e6f432b8-config\") pod \"kube-apiserver-proxy-ip-10-0-143-74.ec2.internal\" (UID: \"d937ab292439efa4b2d5e0f9e6f432b8\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-143-74.ec2.internal"
Sep 17 17:34:41.159104 ip-10-0-143-74 kubenswrapper[2575]: E0917 17:34:41.159038    2575 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-143-74.ec2.internal\" not found"
Sep 17 17:34:41.227563 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:34:41.227533    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/50d07cdbe5bdbd290f2ca595730f87af-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-143-74.ec2.internal\" (UID: \"50d07cdbe5bdbd290f2ca595730f87af\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-143-74.ec2.internal"
Sep 17 17:34:41.227680 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:34:41.227576    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/50d07cdbe5bdbd290f2ca595730f87af-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-143-74.ec2.internal\" (UID: \"50d07cdbe5bdbd290f2ca595730f87af\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-143-74.ec2.internal"
Sep 17 17:34:41.227680 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:34:41.227616    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/50d07cdbe5bdbd290f2ca595730f87af-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-143-74.ec2.internal\" (UID: \"50d07cdbe5bdbd290f2ca595730f87af\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-143-74.ec2.internal"
Sep 17 17:34:41.227680 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:34:41.227583    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/50d07cdbe5bdbd290f2ca595730f87af-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-143-74.ec2.internal\" (UID: \"50d07cdbe5bdbd290f2ca595730f87af\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-143-74.ec2.internal"
Sep 17 17:34:41.227680 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:34:41.227647    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/d937ab292439efa4b2d5e0f9e6f432b8-config\") pod \"kube-apiserver-proxy-ip-10-0-143-74.ec2.internal\" (UID: \"d937ab292439efa4b2d5e0f9e6f432b8\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-143-74.ec2.internal"
Sep 17 17:34:41.227803 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:34:41.227694    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/host-path/d937ab292439efa4b2d5e0f9e6f432b8-config\") pod \"kube-apiserver-proxy-ip-10-0-143-74.ec2.internal\" (UID: \"d937ab292439efa4b2d5e0f9e6f432b8\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-143-74.ec2.internal"
Sep 17 17:34:41.260068 ip-10-0-143-74 kubenswrapper[2575]: E0917 17:34:41.260031    2575 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-143-74.ec2.internal\" not found"
Sep 17 17:34:41.361108 ip-10-0-143-74 kubenswrapper[2575]: E0917 17:34:41.361077    2575 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-143-74.ec2.internal\" not found"
Sep 17 17:34:41.387216 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:34:41.387176    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-143-74.ec2.internal"
Sep 17 17:34:41.390589 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:34:41.390566    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-143-74.ec2.internal"
Sep 17 17:34:41.462824 ip-10-0-143-74 kubenswrapper[2575]: E0917 17:34:41.462737    2575 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-143-74.ec2.internal\" not found"
Sep 17 17:34:41.565208 ip-10-0-143-74 kubenswrapper[2575]: E0917 17:34:41.565171    2575 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-143-74.ec2.internal\" not found"
Sep 17 17:34:41.664033 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:34:41.663994    2575 transport.go:147] "Certificate rotation detected, shutting down client connections to start using new credentials"
Sep 17 17:34:41.664530 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:34:41.664180    2575 reflector.go:571] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.RuntimeClass" err="very short watch: k8s.io/client-go/informers/factory.go:160: Unexpected watch close - watch lasted less than a second and no items received"
Sep 17 17:34:41.666311 ip-10-0-143-74 kubenswrapper[2575]: E0917 17:34:41.666289    2575 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-143-74.ec2.internal\" not found"
Sep 17 17:34:41.766732 ip-10-0-143-74 kubenswrapper[2575]: E0917 17:34:41.766642    2575 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-143-74.ec2.internal\" not found"
Sep 17 17:34:41.826088 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:34:41.826059    2575 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Sep 17 17:34:41.827200 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:34:41.827185    2575 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-143-74.ec2.internal" event="NodeHasSufficientMemory"
Sep 17 17:34:41.827289 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:34:41.827216    2575 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-143-74.ec2.internal" event="NodeHasNoDiskPressure"
Sep 17 17:34:41.827289 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:34:41.827226    2575 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-143-74.ec2.internal" event="NodeHasSufficientPID"
Sep 17 17:34:41.867984 ip-10-0-143-74 kubenswrapper[2575]: E0917 17:34:41.867948    2575 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-143-74.ec2.internal\" not found"
Sep 17 17:34:41.894053 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:34:41.893987    2575 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kube-apiserver-client-kubelet" expiration="2028-09-16 17:29:41 +0000 UTC" deadline="2028-05-03 10:34:12.924715205 +0000 UTC"
Sep 17 17:34:41.894053 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:34:41.894050    2575 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kube-apiserver-client-kubelet" sleep="14248h59m31.030670634s"
Sep 17 17:34:41.968657 ip-10-0-143-74 kubenswrapper[2575]: E0917 17:34:41.968493    2575 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-143-74.ec2.internal\" not found"
Sep 17 17:34:42.044597 ip-10-0-143-74 kubenswrapper[2575]: W0917 17:34:42.044565    2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd937ab292439efa4b2d5e0f9e6f432b8.slice/crio-f4a05c17ee51b2039bfa0e2368495980ac2acc46d5bf908f9a9c7e10d29266aa WatchSource:0}: Error finding container f4a05c17ee51b2039bfa0e2368495980ac2acc46d5bf908f9a9c7e10d29266aa: Status 404 returned error can't find the container with id f4a05c17ee51b2039bfa0e2368495980ac2acc46d5bf908f9a9c7e10d29266aa
Sep 17 17:34:42.045553 ip-10-0-143-74 kubenswrapper[2575]: W0917 17:34:42.045503    2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod50d07cdbe5bdbd290f2ca595730f87af.slice/crio-51b3b7d86caf7b4ff19a101655b9883b6390735ee3026b700bfe33e179142e8c WatchSource:0}: Error finding container 51b3b7d86caf7b4ff19a101655b9883b6390735ee3026b700bfe33e179142e8c: Status 404 returned error can't find the container with id 51b3b7d86caf7b4ff19a101655b9883b6390735ee3026b700bfe33e179142e8c
Sep 17 17:34:42.052927 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:34:42.052906    2575 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Sep 17 17:34:42.069683 ip-10-0-143-74 kubenswrapper[2575]: E0917 17:34:42.069660    2575 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-143-74.ec2.internal\" not found"
Sep 17 17:34:42.148245 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:34:42.148217    2575 reflector.go:439] "Caches populated" type="*v1.Node" reflector="k8s.io/client-go/informers/factory.go:160"
Sep 17 17:34:42.223689 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:34:42.223645    2575 kubelet.go:3248] "Creating a mirror pod for static pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-143-74.ec2.internal"
Sep 17 17:34:42.237689 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:34:42.237660    2575 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]"
Sep 17 17:34:42.239588 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:34:42.239569    2575 kubelet.go:3248] "Creating a mirror pod for static pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-143-74.ec2.internal"
Sep 17 17:34:42.251165 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:34:42.251142    2575 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]"
Sep 17 17:34:42.260543 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:34:42.260519    2575 reflector.go:439] "Caches populated" type="*v1.Service" reflector="k8s.io/client-go/informers/factory.go:160"
Sep 17 17:34:42.283207 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:34:42.283182    2575 reflector.go:439] "Caches populated" type="*v1.CSIDriver" reflector="k8s.io/client-go/informers/factory.go:160"
Sep 17 17:34:42.704810 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:34:42.704768    2575 apiserver.go:52] "Watching apiserver"
Sep 17 17:34:42.719088 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:34:42.719052    2575 reflector.go:439] "Caches populated" type="*v1.Pod" reflector="pkg/kubelet/config/apiserver.go:66"
Sep 17 17:34:42.720494 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:34:42.720462    2575 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kube-system/konnectivity-agent-djvzt","openshift-cluster-node-tuning-operator/tuned-gtwrb","openshift-dns/node-resolver-5549b","openshift-network-diagnostics/network-check-target-pxt82","openshift-network-operator/iptables-alerter-n92x5","openshift-ovn-kubernetes/ovnkube-node-lkwb5","kube-system/global-pull-secret-syncer-9s5wl","kube-system/kube-apiserver-proxy-ip-10-0-143-74.ec2.internal","openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-zvjts","openshift-image-registry/node-ca-2fp6z","openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-143-74.ec2.internal","openshift-multus/multus-additional-cni-plugins-gtxcq","openshift-multus/multus-srdtd","openshift-multus/network-metrics-daemon-wdp4h"]
Sep 17 17:34:42.723255 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:34:42.723229    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-9s5wl"
Sep 17 17:34:42.723495 ip-10-0-143-74 kubenswrapper[2575]: E0917 17:34:42.723466    2575 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-9s5wl" podUID="7892b408-d2f1-4e39-9bf6-e68a452196b1"
Sep 17 17:34:42.723888 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:34:42.723650    2575 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kubelet-serving"
Sep 17 17:34:42.726509 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:34:42.726118    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-srdtd"
Sep 17 17:34:42.729825 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:34:42.729353    2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"openshift-service-ca.crt\""
Sep 17 17:34:42.729825 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:34:42.729435    2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"kube-root-ca.crt\""
Sep 17 17:34:42.729825 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:34:42.729667    2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"cni-copy-resources\""
Sep 17 17:34:42.729825 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:34:42.729721    2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"multus-daemon-config\""
Sep 17 17:34:42.729825 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:34:42.729733    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"default-dockercfg-zl5dp\""
Sep 17 17:34:42.730309 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:34:42.729881    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-5549b"
Sep 17 17:34:42.731889 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:34:42.731628    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-pxt82"
Sep 17 17:34:42.731889 ip-10-0-143-74 kubenswrapper[2575]: E0917 17:34:42.731697    2575 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-pxt82" podUID="57bf735f-0367-4799-a1a8-889870d65704"
Sep 17 17:34:42.732278 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:34:42.732258    2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"openshift-service-ca.crt\""
Sep 17 17:34:42.732616 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:34:42.732596    2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"kube-root-ca.crt\""
Sep 17 17:34:42.732980 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:34:42.732961    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"node-resolver-dockercfg-g7r8f\""
Sep 17 17:34:42.733927 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:34:42.733905    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-n92x5"
Sep 17 17:34:42.738299 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:34:42.736624    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-lkwb5"
Sep 17 17:34:42.738299 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:34:42.737411    2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"openshift-service-ca.crt\""
Sep 17 17:34:42.738299 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:34:42.737567    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/fda64577-6753-498e-ab9e-128a567424ca-host-var-lib-cni-multus\") pod \"multus-srdtd\" (UID: \"fda64577-6753-498e-ab9e-128a567424ca\") " pod="openshift-multus/multus-srdtd"
Sep 17 17:34:42.738299 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:34:42.737602    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/7892b408-d2f1-4e39-9bf6-e68a452196b1-original-pull-secret\") pod \"global-pull-secret-syncer-9s5wl\" (UID: \"7892b408-d2f1-4e39-9bf6-e68a452196b1\") " pod="kube-system/global-pull-secret-syncer-9s5wl"
Sep 17 17:34:42.738299 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:34:42.737643    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-dockercfg-2rwpn\""
Sep 17 17:34:42.738299 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:34:42.737645    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/fda64577-6753-498e-ab9e-128a567424ca-cni-binary-copy\") pod \"multus-srdtd\" (UID: \"fda64577-6753-498e-ab9e-128a567424ca\") " pod="openshift-multus/multus-srdtd"
Sep 17 17:34:42.738299 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:34:42.737672    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/fda64577-6753-498e-ab9e-128a567424ca-multus-conf-dir\") pod \"multus-srdtd\" (UID: \"fda64577-6753-498e-ab9e-128a567424ca\") " pod="openshift-multus/multus-srdtd"
Sep 17 17:34:42.738299 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:34:42.737703    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/fda64577-6753-498e-ab9e-128a567424ca-host-run-multus-certs\") pod \"multus-srdtd\" (UID: \"fda64577-6753-498e-ab9e-128a567424ca\") " pod="openshift-multus/multus-srdtd"
Sep 17 17:34:42.738299 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:34:42.737716    2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"kube-root-ca.crt\""
Sep 17 17:34:42.738299 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:34:42.737729    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/7892b408-d2f1-4e39-9bf6-e68a452196b1-kubelet-config\") pod \"global-pull-secret-syncer-9s5wl\" (UID: \"7892b408-d2f1-4e39-9bf6-e68a452196b1\") " pod="kube-system/global-pull-secret-syncer-9s5wl"
Sep 17 17:34:42.738299 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:34:42.737766    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/fda64577-6753-498e-ab9e-128a567424ca-system-cni-dir\") pod \"multus-srdtd\" (UID: \"fda64577-6753-498e-ab9e-128a567424ca\") " pod="openshift-multus/multus-srdtd"
Sep 17 17:34:42.738299 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:34:42.737805    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/fda64577-6753-498e-ab9e-128a567424ca-host-run-k8s-cni-cncf-io\") pod \"multus-srdtd\" (UID: \"fda64577-6753-498e-ab9e-128a567424ca\") " pod="openshift-multus/multus-srdtd"
Sep 17 17:34:42.738299 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:34:42.737846    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/fda64577-6753-498e-ab9e-128a567424ca-hostroot\") pod \"multus-srdtd\" (UID: \"fda64577-6753-498e-ab9e-128a567424ca\") " pod="openshift-multus/multus-srdtd"
Sep 17 17:34:42.738299 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:34:42.737884    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/fda64577-6753-498e-ab9e-128a567424ca-multus-daemon-config\") pod \"multus-srdtd\" (UID: \"fda64577-6753-498e-ab9e-128a567424ca\") " pod="openshift-multus/multus-srdtd"
Sep 17 17:34:42.738299 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:34:42.737917    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/fda64577-6753-498e-ab9e-128a567424ca-etc-kubernetes\") pod \"multus-srdtd\" (UID: \"fda64577-6753-498e-ab9e-128a567424ca\") " pod="openshift-multus/multus-srdtd"
Sep 17 17:34:42.738299 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:34:42.737922    2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-script\""
Sep 17 17:34:42.738299 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:34:42.737939    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4s4t5\" (UniqueName: \"kubernetes.io/projected/fda64577-6753-498e-ab9e-128a567424ca-kube-api-access-4s4t5\") pod \"multus-srdtd\" (UID: \"fda64577-6753-498e-ab9e-128a567424ca\") " pod="openshift-multus/multus-srdtd"
Sep 17 17:34:42.738299 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:34:42.737966    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/7892b408-d2f1-4e39-9bf6-e68a452196b1-dbus\") pod \"global-pull-secret-syncer-9s5wl\" (UID: \"7892b408-d2f1-4e39-9bf6-e68a452196b1\") " pod="kube-system/global-pull-secret-syncer-9s5wl"
Sep 17 17:34:42.738299 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:34:42.737991    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/fda64577-6753-498e-ab9e-128a567424ca-host-run-netns\") pod \"multus-srdtd\" (UID: \"fda64577-6753-498e-ab9e-128a567424ca\") " pod="openshift-multus/multus-srdtd"
Sep 17 17:34:42.738299 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:34:42.738017    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/fda64577-6753-498e-ab9e-128a567424ca-host-var-lib-cni-bin\") pod \"multus-srdtd\" (UID: \"fda64577-6753-498e-ab9e-128a567424ca\") " pod="openshift-multus/multus-srdtd"
Sep 17 17:34:42.739421 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:34:42.738041    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/fda64577-6753-498e-ab9e-128a567424ca-host-var-lib-kubelet\") pod \"multus-srdtd\" (UID: \"fda64577-6753-498e-ab9e-128a567424ca\") " pod="openshift-multus/multus-srdtd"
Sep 17 17:34:42.739421 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:34:42.738067    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/fda64577-6753-498e-ab9e-128a567424ca-multus-cni-dir\") pod \"multus-srdtd\" (UID: \"fda64577-6753-498e-ab9e-128a567424ca\") " pod="openshift-multus/multus-srdtd"
Sep 17 17:34:42.739421 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:34:42.738109    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/fda64577-6753-498e-ab9e-128a567424ca-cnibin\") pod \"multus-srdtd\" (UID: \"fda64577-6753-498e-ab9e-128a567424ca\") " pod="openshift-multus/multus-srdtd"
Sep 17 17:34:42.739421 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:34:42.738201    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/fda64577-6753-498e-ab9e-128a567424ca-multus-socket-dir-parent\") pod \"multus-srdtd\" (UID: \"fda64577-6753-498e-ab9e-128a567424ca\") " pod="openshift-multus/multus-srdtd"
Sep 17 17:34:42.740576 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:34:42.739819    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-zvjts"
Sep 17 17:34:42.741851 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:34:42.741146    2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-script-lib\""
Sep 17 17:34:42.741851 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:34:42.741200    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-node-metrics-cert\""
Sep 17 17:34:42.741851 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:34:42.741145    2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"openshift-service-ca.crt\""
Sep 17 17:34:42.741851 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:34:42.741460    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-kubernetes-node-dockercfg-8nck5\""
Sep 17 17:34:42.741851 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:34:42.741647    2575 reflector.go:439] "Caches populated" logger="kubernetes.io/kubelet-serving" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162"
Sep 17 17:34:42.741851 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:34:42.741693    2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"env-overrides\""
Sep 17 17:34:42.742232 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:34:42.741914    2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"kube-root-ca.crt\""
Sep 17 17:34:42.742232 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:34:42.741982    2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-config\""
Sep 17 17:34:42.742232 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:34:42.742180    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-djvzt"
Sep 17 17:34:42.742518 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:34:42.742455    2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"openshift-service-ca.crt\""
Sep 17 17:34:42.742805 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:34:42.742779    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-sa-dockercfg-lpklv\""
Sep 17 17:34:42.742926 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:34:42.742904    2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"kube-root-ca.crt\""
Sep 17 17:34:42.743028 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:34:42.743011    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-metrics-serving-cert\""
Sep 17 17:34:42.744579 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:34:42.744560    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"default-dockercfg-k4dp7\""
Sep 17 17:34:42.745721 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:34:42.745483    2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kube-system\"/\"konnectivity-ca-bundle\""
Sep 17 17:34:42.745721 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:34:42.745538    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"konnectivity-agent\""
Sep 17 17:34:42.745721 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:34:42.745668    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-gtwrb"
Sep 17 17:34:42.747978 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:34:42.747957    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-2fp6z"
Sep 17 17:34:42.749312 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:34:42.749291    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"tuned-dockercfg-4jdj4\""
Sep 17 17:34:42.749417 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:34:42.749364    2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"kube-root-ca.crt\""
Sep 17 17:34:42.749544 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:34:42.749526    2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"openshift-service-ca.crt\""
Sep 17 17:34:42.750349 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:34:42.750327    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-wdp4h"
Sep 17 17:34:42.750440 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:34:42.750414    2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"image-registry-certificates\""
Sep 17 17:34:42.750440 ip-10-0-143-74 kubenswrapper[2575]: E0917 17:34:42.750421    2575 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-wdp4h" podUID="27866e17-c231-49a6-921f-a83a01b8e173"
Sep 17 17:34:42.750684 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:34:42.750660    2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"kube-root-ca.crt\""
Sep 17 17:34:42.750774 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:34:42.750732    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"node-ca-dockercfg-k2ldr\""
Sep 17 17:34:42.750774 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:34:42.750769    2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"openshift-service-ca.crt\""
Sep 17 17:34:42.754009 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:34:42.753986    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-gtxcq"
Sep 17 17:34:42.757103 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:34:42.757080    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"multus-ancillary-tools-dockercfg-gg4fk\""
Sep 17 17:34:42.757389 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:34:42.757366    2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"default-cni-sysctl-allowlist\""
Sep 17 17:34:42.757633 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:34:42.757617    2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"whereabouts-flatfile-config\""
Sep 17 17:34:42.777563 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:34:42.777540    2575 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kubelet-serving" csr="csr-8q6kn"
Sep 17 17:34:42.787491 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:34:42.787440    2575 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kubelet-serving" csr="csr-8q6kn"
Sep 17 17:34:42.824601 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:34:42.824575    2575 desired_state_of_world_populator.go:154] "Finished populating initial desired state of world"
Sep 17 17:34:42.838687 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:34:42.838654    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7rzlr\" (UniqueName: \"kubernetes.io/projected/57bf735f-0367-4799-a1a8-889870d65704-kube-api-access-7rzlr\") pod \"network-check-target-pxt82\" (UID: \"57bf735f-0367-4799-a1a8-889870d65704\") " pod="openshift-network-diagnostics/network-check-target-pxt82"
Sep 17 17:34:42.838853 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:34:42.838701    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/93fd2156-b185-4995-8e49-d6d9c0eda255-tmp-dir\") pod \"node-resolver-5549b\" (UID: \"93fd2156-b185-4995-8e49-d6d9c0eda255\") " pod="openshift-dns/node-resolver-5549b"
Sep 17 17:34:42.838853 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:34:42.838735    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/d0f23713-30dc-4d47-95cb-c083bec7c18c-etc-selinux\") pod \"aws-ebs-csi-driver-node-zvjts\" (UID: \"d0f23713-30dc-4d47-95cb-c083bec7c18c\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-zvjts"
Sep 17 17:34:42.838853 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:34:42.838775    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-f4kqw\" (UniqueName: \"kubernetes.io/projected/173c8902-414c-4212-8d93-ce15853e8ca8-kube-api-access-f4kqw\") pod \"node-ca-2fp6z\" (UID: \"173c8902-414c-4212-8d93-ce15853e8ca8\") " pod="openshift-image-registry/node-ca-2fp6z"
Sep 17 17:34:42.838853 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:34:42.838810    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/fda64577-6753-498e-ab9e-128a567424ca-host-var-lib-cni-bin\") pod \"multus-srdtd\" (UID: \"fda64577-6753-498e-ab9e-128a567424ca\") " pod="openshift-multus/multus-srdtd"
Sep 17 17:34:42.838853 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:34:42.838844    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/83786a21-e36e-4e15-82ec-c28223ef752b-iptables-alerter-script\") pod \"iptables-alerter-n92x5\" (UID: \"83786a21-e36e-4e15-82ec-c28223ef752b\") " pod="openshift-network-operator/iptables-alerter-n92x5"
Sep 17 17:34:42.839116 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:34:42.838869    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/d52f9540-79e9-4df7-b3a9-e8c7cc6d48b0-cnibin\") pod \"multus-additional-cni-plugins-gtxcq\" (UID: \"d52f9540-79e9-4df7-b3a9-e8c7cc6d48b0\") " pod="openshift-multus/multus-additional-cni-plugins-gtxcq"
Sep 17 17:34:42.839116 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:34:42.838892    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/fda64577-6753-498e-ab9e-128a567424ca-host-var-lib-cni-bin\") pod \"multus-srdtd\" (UID: \"fda64577-6753-498e-ab9e-128a567424ca\") " pod="openshift-multus/multus-srdtd"
Sep 17 17:34:42.839116 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:34:42.838893    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/d0f23713-30dc-4d47-95cb-c083bec7c18c-sys-fs\") pod \"aws-ebs-csi-driver-node-zvjts\" (UID: \"d0f23713-30dc-4d47-95cb-c083bec7c18c\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-zvjts"
Sep 17 17:34:42.839116 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:34:42.838936    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/d52f9540-79e9-4df7-b3a9-e8c7cc6d48b0-cni-binary-copy\") pod \"multus-additional-cni-plugins-gtxcq\" (UID: \"d52f9540-79e9-4df7-b3a9-e8c7cc6d48b0\") " pod="openshift-multus/multus-additional-cni-plugins-gtxcq"
Sep 17 17:34:42.839116 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:34:42.838978    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/d52f9540-79e9-4df7-b3a9-e8c7cc6d48b0-tuning-conf-dir\") pod \"multus-additional-cni-plugins-gtxcq\" (UID: \"d52f9540-79e9-4df7-b3a9-e8c7cc6d48b0\") " pod="openshift-multus/multus-additional-cni-plugins-gtxcq"
Sep 17 17:34:42.839116 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:34:42.839015    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/27866e17-c231-49a6-921f-a83a01b8e173-metrics-certs\") pod \"network-metrics-daemon-wdp4h\" (UID: \"27866e17-c231-49a6-921f-a83a01b8e173\") " pod="openshift-multus/network-metrics-daemon-wdp4h"
Sep 17 17:34:42.839116 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:34:42.839039    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/7892b408-d2f1-4e39-9bf6-e68a452196b1-original-pull-secret\") pod \"global-pull-secret-syncer-9s5wl\" (UID: \"7892b408-d2f1-4e39-9bf6-e68a452196b1\") " pod="kube-system/global-pull-secret-syncer-9s5wl"
Sep 17 17:34:42.839116 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:34:42.839055    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/fda64577-6753-498e-ab9e-128a567424ca-host-run-multus-certs\") pod \"multus-srdtd\" (UID: \"fda64577-6753-498e-ab9e-128a567424ca\") " pod="openshift-multus/multus-srdtd"
Sep 17 17:34:42.839116 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:34:42.839075    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/2ccbe31a-b945-423d-a88a-45ef25012451-host-kubelet\") pod \"ovnkube-node-lkwb5\" (UID: \"2ccbe31a-b945-423d-a88a-45ef25012451\") " pod="openshift-ovn-kubernetes/ovnkube-node-lkwb5"
Sep 17 17:34:42.839116 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:34:42.839100    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dksdw\" (UniqueName: \"kubernetes.io/projected/d0f23713-30dc-4d47-95cb-c083bec7c18c-kube-api-access-dksdw\") pod \"aws-ebs-csi-driver-node-zvjts\" (UID: \"d0f23713-30dc-4d47-95cb-c083bec7c18c\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-zvjts"
Sep 17 17:34:42.839625 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:34:42.839131    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/3cb242c8-4f5d-49a3-bb24-85a2170a979e-etc-systemd\") pod \"tuned-gtwrb\" (UID: \"3cb242c8-4f5d-49a3-bb24-85a2170a979e\") " pod="openshift-cluster-node-tuning-operator/tuned-gtwrb"
Sep 17 17:34:42.839625 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:34:42.839163    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/fda64577-6753-498e-ab9e-128a567424ca-host-run-multus-certs\") pod \"multus-srdtd\" (UID: \"fda64577-6753-498e-ab9e-128a567424ca\") " pod="openshift-multus/multus-srdtd"
Sep 17 17:34:42.839625 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:34:42.839168    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/3cb242c8-4f5d-49a3-bb24-85a2170a979e-lib-modules\") pod \"tuned-gtwrb\" (UID: \"3cb242c8-4f5d-49a3-bb24-85a2170a979e\") " pod="openshift-cluster-node-tuning-operator/tuned-gtwrb"
Sep 17 17:34:42.839625 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:34:42.839195    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/3cb242c8-4f5d-49a3-bb24-85a2170a979e-etc-tuned\") pod \"tuned-gtwrb\" (UID: \"3cb242c8-4f5d-49a3-bb24-85a2170a979e\") " pod="openshift-cluster-node-tuning-operator/tuned-gtwrb"
Sep 17 17:34:42.839625 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:34:42.839220    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/3cb242c8-4f5d-49a3-bb24-85a2170a979e-tmp\") pod \"tuned-gtwrb\" (UID: \"3cb242c8-4f5d-49a3-bb24-85a2170a979e\") " pod="openshift-cluster-node-tuning-operator/tuned-gtwrb"
Sep 17 17:34:42.839625 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:34:42.839246    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/fda64577-6753-498e-ab9e-128a567424ca-multus-daemon-config\") pod \"multus-srdtd\" (UID: \"fda64577-6753-498e-ab9e-128a567424ca\") " pod="openshift-multus/multus-srdtd"
Sep 17 17:34:42.839625 ip-10-0-143-74 kubenswrapper[2575]: E0917 17:34:42.839301    2575 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Sep 17 17:34:42.839625 ip-10-0-143-74 kubenswrapper[2575]: E0917 17:34:42.839386    2575 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/7892b408-d2f1-4e39-9bf6-e68a452196b1-original-pull-secret podName:7892b408-d2f1-4e39-9bf6-e68a452196b1 nodeName:}" failed. No retries permitted until 2026-09-17 17:34:43.339350281 +0000 UTC m=+2.969680258 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/7892b408-d2f1-4e39-9bf6-e68a452196b1-original-pull-secret") pod "global-pull-secret-syncer-9s5wl" (UID: "7892b408-d2f1-4e39-9bf6-e68a452196b1") : object "kube-system"/"original-pull-secret" not registered
Sep 17 17:34:42.839625 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:34:42.839286    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/2ccbe31a-b945-423d-a88a-45ef25012451-var-lib-openvswitch\") pod \"ovnkube-node-lkwb5\" (UID: \"2ccbe31a-b945-423d-a88a-45ef25012451\") " pod="openshift-ovn-kubernetes/ovnkube-node-lkwb5"
Sep 17 17:34:42.839625 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:34:42.839519    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/2ccbe31a-b945-423d-a88a-45ef25012451-ovnkube-config\") pod \"ovnkube-node-lkwb5\" (UID: \"2ccbe31a-b945-423d-a88a-45ef25012451\") " pod="openshift-ovn-kubernetes/ovnkube-node-lkwb5"
Sep 17 17:34:42.839625 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:34:42.839551    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/2ccbe31a-b945-423d-a88a-45ef25012451-ovn-node-metrics-cert\") pod \"ovnkube-node-lkwb5\" (UID: \"2ccbe31a-b945-423d-a88a-45ef25012451\") " pod="openshift-ovn-kubernetes/ovnkube-node-lkwb5"
Sep 17 17:34:42.839625 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:34:42.839597    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-d29hd\" (UniqueName: \"kubernetes.io/projected/93fd2156-b185-4995-8e49-d6d9c0eda255-kube-api-access-d29hd\") pod \"node-resolver-5549b\" (UID: \"93fd2156-b185-4995-8e49-d6d9c0eda255\") " pod="openshift-dns/node-resolver-5549b"
Sep 17 17:34:42.839625 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:34:42.839623    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/3cb242c8-4f5d-49a3-bb24-85a2170a979e-etc-modprobe-d\") pod \"tuned-gtwrb\" (UID: \"3cb242c8-4f5d-49a3-bb24-85a2170a979e\") " pod="openshift-cluster-node-tuning-operator/tuned-gtwrb"
Sep 17 17:34:42.840311 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:34:42.839647    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/3cb242c8-4f5d-49a3-bb24-85a2170a979e-run\") pod \"tuned-gtwrb\" (UID: \"3cb242c8-4f5d-49a3-bb24-85a2170a979e\") " pod="openshift-cluster-node-tuning-operator/tuned-gtwrb"
Sep 17 17:34:42.840311 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:34:42.839680    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/fda64577-6753-498e-ab9e-128a567424ca-multus-cni-dir\") pod \"multus-srdtd\" (UID: \"fda64577-6753-498e-ab9e-128a567424ca\") " pod="openshift-multus/multus-srdtd"
Sep 17 17:34:42.840311 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:34:42.839718    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/fda64577-6753-498e-ab9e-128a567424ca-multus-socket-dir-parent\") pod \"multus-srdtd\" (UID: \"fda64577-6753-498e-ab9e-128a567424ca\") " pod="openshift-multus/multus-srdtd"
Sep 17 17:34:42.840311 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:34:42.839793    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zjj2m\" (UniqueName: \"kubernetes.io/projected/83786a21-e36e-4e15-82ec-c28223ef752b-kube-api-access-zjj2m\") pod \"iptables-alerter-n92x5\" (UID: \"83786a21-e36e-4e15-82ec-c28223ef752b\") " pod="openshift-network-operator/iptables-alerter-n92x5"
Sep 17 17:34:42.840311 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:34:42.839824    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/2ccbe31a-b945-423d-a88a-45ef25012451-host-slash\") pod \"ovnkube-node-lkwb5\" (UID: \"2ccbe31a-b945-423d-a88a-45ef25012451\") " pod="openshift-ovn-kubernetes/ovnkube-node-lkwb5"
Sep 17 17:34:42.840311 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:34:42.839828    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/fda64577-6753-498e-ab9e-128a567424ca-multus-cni-dir\") pod \"multus-srdtd\" (UID: \"fda64577-6753-498e-ab9e-128a567424ca\") " pod="openshift-multus/multus-srdtd"
Sep 17 17:34:42.840311 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:34:42.839852    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/d52f9540-79e9-4df7-b3a9-e8c7cc6d48b0-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-gtxcq\" (UID: \"d52f9540-79e9-4df7-b3a9-e8c7cc6d48b0\") " pod="openshift-multus/multus-additional-cni-plugins-gtxcq"
Sep 17 17:34:42.840311 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:34:42.839880    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/83786a21-e36e-4e15-82ec-c28223ef752b-host-slash\") pod \"iptables-alerter-n92x5\" (UID: \"83786a21-e36e-4e15-82ec-c28223ef752b\") " pod="openshift-network-operator/iptables-alerter-n92x5"
Sep 17 17:34:42.840311 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:34:42.839909    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/2ccbe31a-b945-423d-a88a-45ef25012451-run-systemd\") pod \"ovnkube-node-lkwb5\" (UID: \"2ccbe31a-b945-423d-a88a-45ef25012451\") " pod="openshift-ovn-kubernetes/ovnkube-node-lkwb5"
Sep 17 17:34:42.840311 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:34:42.839924    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/fda64577-6753-498e-ab9e-128a567424ca-multus-socket-dir-parent\") pod \"multus-srdtd\" (UID: \"fda64577-6753-498e-ab9e-128a567424ca\") " pod="openshift-multus/multus-srdtd"
Sep 17 17:34:42.840311 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:34:42.839935    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/2ccbe31a-b945-423d-a88a-45ef25012451-run-ovn\") pod \"ovnkube-node-lkwb5\" (UID: \"2ccbe31a-b945-423d-a88a-45ef25012451\") " pod="openshift-ovn-kubernetes/ovnkube-node-lkwb5"
Sep 17 17:34:42.840311 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:34:42.839961    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/d0f23713-30dc-4d47-95cb-c083bec7c18c-kubelet-dir\") pod \"aws-ebs-csi-driver-node-zvjts\" (UID: \"d0f23713-30dc-4d47-95cb-c083bec7c18c\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-zvjts"
Sep 17 17:34:42.840311 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:34:42.839986    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/3cb242c8-4f5d-49a3-bb24-85a2170a979e-etc-sysctl-d\") pod \"tuned-gtwrb\" (UID: \"3cb242c8-4f5d-49a3-bb24-85a2170a979e\") " pod="openshift-cluster-node-tuning-operator/tuned-gtwrb"
Sep 17 17:34:42.840311 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:34:42.839996    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/fda64577-6753-498e-ab9e-128a567424ca-multus-daemon-config\") pod \"multus-srdtd\" (UID: \"fda64577-6753-498e-ab9e-128a567424ca\") " pod="openshift-multus/multus-srdtd"
Sep 17 17:34:42.840311 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:34:42.840011    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/3cb242c8-4f5d-49a3-bb24-85a2170a979e-etc-sysctl-conf\") pod \"tuned-gtwrb\" (UID: \"3cb242c8-4f5d-49a3-bb24-85a2170a979e\") " pod="openshift-cluster-node-tuning-operator/tuned-gtwrb"
Sep 17 17:34:42.840311 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:34:42.840044    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4s4t5\" (UniqueName: \"kubernetes.io/projected/fda64577-6753-498e-ab9e-128a567424ca-kube-api-access-4s4t5\") pod \"multus-srdtd\" (UID: \"fda64577-6753-498e-ab9e-128a567424ca\") " pod="openshift-multus/multus-srdtd"
Sep 17 17:34:42.840311 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:34:42.840096    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/2ccbe31a-b945-423d-a88a-45ef25012451-host-run-netns\") pod \"ovnkube-node-lkwb5\" (UID: \"2ccbe31a-b945-423d-a88a-45ef25012451\") " pod="openshift-ovn-kubernetes/ovnkube-node-lkwb5"
Sep 17 17:34:42.841129 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:34:42.840258    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/2ccbe31a-b945-423d-a88a-45ef25012451-ovnkube-script-lib\") pod \"ovnkube-node-lkwb5\" (UID: \"2ccbe31a-b945-423d-a88a-45ef25012451\") " pod="openshift-ovn-kubernetes/ovnkube-node-lkwb5"
Sep 17 17:34:42.841129 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:34:42.840290    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/fda64577-6753-498e-ab9e-128a567424ca-host-run-netns\") pod \"multus-srdtd\" (UID: \"fda64577-6753-498e-ab9e-128a567424ca\") " pod="openshift-multus/multus-srdtd"
Sep 17 17:34:42.841129 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:34:42.840316    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/fda64577-6753-498e-ab9e-128a567424ca-host-var-lib-kubelet\") pod \"multus-srdtd\" (UID: \"fda64577-6753-498e-ab9e-128a567424ca\") " pod="openshift-multus/multus-srdtd"
Sep 17 17:34:42.841129 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:34:42.840342    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8x7rf\" (UniqueName: \"kubernetes.io/projected/2ccbe31a-b945-423d-a88a-45ef25012451-kube-api-access-8x7rf\") pod \"ovnkube-node-lkwb5\" (UID: \"2ccbe31a-b945-423d-a88a-45ef25012451\") " pod="openshift-ovn-kubernetes/ovnkube-node-lkwb5"
Sep 17 17:34:42.841129 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:34:42.840372    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/d52f9540-79e9-4df7-b3a9-e8c7cc6d48b0-system-cni-dir\") pod \"multus-additional-cni-plugins-gtxcq\" (UID: \"d52f9540-79e9-4df7-b3a9-e8c7cc6d48b0\") " pod="openshift-multus/multus-additional-cni-plugins-gtxcq"
Sep 17 17:34:42.841129 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:34:42.840399    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-fcz2n\" (UniqueName: \"kubernetes.io/projected/3cb242c8-4f5d-49a3-bb24-85a2170a979e-kube-api-access-fcz2n\") pod \"tuned-gtwrb\" (UID: \"3cb242c8-4f5d-49a3-bb24-85a2170a979e\") " pod="openshift-cluster-node-tuning-operator/tuned-gtwrb"
Sep 17 17:34:42.841129 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:34:42.840420    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/fda64577-6753-498e-ab9e-128a567424ca-host-run-netns\") pod \"multus-srdtd\" (UID: \"fda64577-6753-498e-ab9e-128a567424ca\") " pod="openshift-multus/multus-srdtd"
Sep 17 17:34:42.841129 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:34:42.840425    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/fda64577-6753-498e-ab9e-128a567424ca-cnibin\") pod \"multus-srdtd\" (UID: \"fda64577-6753-498e-ab9e-128a567424ca\") " pod="openshift-multus/multus-srdtd"
Sep 17 17:34:42.841129 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:34:42.840488    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/2ccbe31a-b945-423d-a88a-45ef25012451-log-socket\") pod \"ovnkube-node-lkwb5\" (UID: \"2ccbe31a-b945-423d-a88a-45ef25012451\") " pod="openshift-ovn-kubernetes/ovnkube-node-lkwb5"
Sep 17 17:34:42.841129 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:34:42.840516    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/d0f23713-30dc-4d47-95cb-c083bec7c18c-socket-dir\") pod \"aws-ebs-csi-driver-node-zvjts\" (UID: \"d0f23713-30dc-4d47-95cb-c083bec7c18c\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-zvjts"
Sep 17 17:34:42.841129 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:34:42.840543    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/3cb242c8-4f5d-49a3-bb24-85a2170a979e-etc-sysconfig\") pod \"tuned-gtwrb\" (UID: \"3cb242c8-4f5d-49a3-bb24-85a2170a979e\") " pod="openshift-cluster-node-tuning-operator/tuned-gtwrb"
Sep 17 17:34:42.841129 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:34:42.840567    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-69pfg\" (UniqueName: \"kubernetes.io/projected/27866e17-c231-49a6-921f-a83a01b8e173-kube-api-access-69pfg\") pod \"network-metrics-daemon-wdp4h\" (UID: \"27866e17-c231-49a6-921f-a83a01b8e173\") " pod="openshift-multus/network-metrics-daemon-wdp4h"
Sep 17 17:34:42.841129 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:34:42.840545    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/fda64577-6753-498e-ab9e-128a567424ca-cnibin\") pod \"multus-srdtd\" (UID: \"fda64577-6753-498e-ab9e-128a567424ca\") " pod="openshift-multus/multus-srdtd"
Sep 17 17:34:42.841129 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:34:42.840596    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/2ccbe31a-b945-423d-a88a-45ef25012451-host-cni-bin\") pod \"ovnkube-node-lkwb5\" (UID: \"2ccbe31a-b945-423d-a88a-45ef25012451\") " pod="openshift-ovn-kubernetes/ovnkube-node-lkwb5"
Sep 17 17:34:42.841129 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:34:42.840638    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/2ccbe31a-b945-423d-a88a-45ef25012451-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-lkwb5\" (UID: \"2ccbe31a-b945-423d-a88a-45ef25012451\") " pod="openshift-ovn-kubernetes/ovnkube-node-lkwb5"
Sep 17 17:34:42.841129 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:34:42.840647    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/fda64577-6753-498e-ab9e-128a567424ca-host-var-lib-kubelet\") pod \"multus-srdtd\" (UID: \"fda64577-6753-498e-ab9e-128a567424ca\") " pod="openshift-multus/multus-srdtd"
Sep 17 17:34:42.841129 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:34:42.840665    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/1d07e68d-9943-420f-a6a0-5f477792fabd-agent-certs\") pod \"konnectivity-agent-djvzt\" (UID: \"1d07e68d-9943-420f-a6a0-5f477792fabd\") " pod="kube-system/konnectivity-agent-djvzt"
Sep 17 17:34:42.841979 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:34:42.840709    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/3cb242c8-4f5d-49a3-bb24-85a2170a979e-etc-kubernetes\") pod \"tuned-gtwrb\" (UID: \"3cb242c8-4f5d-49a3-bb24-85a2170a979e\") " pod="openshift-cluster-node-tuning-operator/tuned-gtwrb"
Sep 17 17:34:42.841979 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:34:42.840761    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/3cb242c8-4f5d-49a3-bb24-85a2170a979e-sys\") pod \"tuned-gtwrb\" (UID: \"3cb242c8-4f5d-49a3-bb24-85a2170a979e\") " pod="openshift-cluster-node-tuning-operator/tuned-gtwrb"
Sep 17 17:34:42.841979 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:34:42.840814    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/7892b408-d2f1-4e39-9bf6-e68a452196b1-kubelet-config\") pod \"global-pull-secret-syncer-9s5wl\" (UID: \"7892b408-d2f1-4e39-9bf6-e68a452196b1\") " pod="kube-system/global-pull-secret-syncer-9s5wl"
Sep 17 17:34:42.841979 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:34:42.840865    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/2ccbe31a-b945-423d-a88a-45ef25012451-host-cni-netd\") pod \"ovnkube-node-lkwb5\" (UID: \"2ccbe31a-b945-423d-a88a-45ef25012451\") " pod="openshift-ovn-kubernetes/ovnkube-node-lkwb5"
Sep 17 17:34:42.841979 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:34:42.840915    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/7892b408-d2f1-4e39-9bf6-e68a452196b1-kubelet-config\") pod \"global-pull-secret-syncer-9s5wl\" (UID: \"7892b408-d2f1-4e39-9bf6-e68a452196b1\") " pod="kube-system/global-pull-secret-syncer-9s5wl"
Sep 17 17:34:42.841979 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:34:42.840959    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/93fd2156-b185-4995-8e49-d6d9c0eda255-hosts-file\") pod \"node-resolver-5549b\" (UID: \"93fd2156-b185-4995-8e49-d6d9c0eda255\") " pod="openshift-dns/node-resolver-5549b"
Sep 17 17:34:42.841979 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:34:42.840989    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/d0f23713-30dc-4d47-95cb-c083bec7c18c-device-dir\") pod \"aws-ebs-csi-driver-node-zvjts\" (UID: \"d0f23713-30dc-4d47-95cb-c083bec7c18c\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-zvjts"
Sep 17 17:34:42.841979 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:34:42.841012    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/3cb242c8-4f5d-49a3-bb24-85a2170a979e-var-lib-kubelet\") pod \"tuned-gtwrb\" (UID: \"3cb242c8-4f5d-49a3-bb24-85a2170a979e\") " pod="openshift-cluster-node-tuning-operator/tuned-gtwrb"
Sep 17 17:34:42.841979 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:34:42.841034    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/173c8902-414c-4212-8d93-ce15853e8ca8-host\") pod \"node-ca-2fp6z\" (UID: \"173c8902-414c-4212-8d93-ce15853e8ca8\") " pod="openshift-image-registry/node-ca-2fp6z"
Sep 17 17:34:42.841979 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:34:42.841063    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/7892b408-d2f1-4e39-9bf6-e68a452196b1-dbus\") pod \"global-pull-secret-syncer-9s5wl\" (UID: \"7892b408-d2f1-4e39-9bf6-e68a452196b1\") " pod="kube-system/global-pull-secret-syncer-9s5wl"
Sep 17 17:34:42.841979 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:34:42.841088    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/2ccbe31a-b945-423d-a88a-45ef25012451-run-openvswitch\") pod \"ovnkube-node-lkwb5\" (UID: \"2ccbe31a-b945-423d-a88a-45ef25012451\") " pod="openshift-ovn-kubernetes/ovnkube-node-lkwb5"
Sep 17 17:34:42.841979 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:34:42.841114    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/2ccbe31a-b945-423d-a88a-45ef25012451-host-run-ovn-kubernetes\") pod \"ovnkube-node-lkwb5\" (UID: \"2ccbe31a-b945-423d-a88a-45ef25012451\") " pod="openshift-ovn-kubernetes/ovnkube-node-lkwb5"
Sep 17 17:34:42.841979 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:34:42.841138    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/1d07e68d-9943-420f-a6a0-5f477792fabd-konnectivity-ca\") pod \"konnectivity-agent-djvzt\" (UID: \"1d07e68d-9943-420f-a6a0-5f477792fabd\") " pod="kube-system/konnectivity-agent-djvzt"
Sep 17 17:34:42.841979 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:34:42.841176    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/3cb242c8-4f5d-49a3-bb24-85a2170a979e-host\") pod \"tuned-gtwrb\" (UID: \"3cb242c8-4f5d-49a3-bb24-85a2170a979e\") " pod="openshift-cluster-node-tuning-operator/tuned-gtwrb"
Sep 17 17:34:42.841979 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:34:42.841200    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/7892b408-d2f1-4e39-9bf6-e68a452196b1-dbus\") pod \"global-pull-secret-syncer-9s5wl\" (UID: \"7892b408-d2f1-4e39-9bf6-e68a452196b1\") " pod="kube-system/global-pull-secret-syncer-9s5wl"
Sep 17 17:34:42.841979 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:34:42.841203    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/fda64577-6753-498e-ab9e-128a567424ca-host-var-lib-cni-multus\") pod \"multus-srdtd\" (UID: \"fda64577-6753-498e-ab9e-128a567424ca\") " pod="openshift-multus/multus-srdtd"
Sep 17 17:34:42.841979 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:34:42.841245    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/fda64577-6753-498e-ab9e-128a567424ca-host-var-lib-cni-multus\") pod \"multus-srdtd\" (UID: \"fda64577-6753-498e-ab9e-128a567424ca\") " pod="openshift-multus/multus-srdtd"
Sep 17 17:34:42.842679 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:34:42.841257    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/2ccbe31a-b945-423d-a88a-45ef25012451-systemd-units\") pod \"ovnkube-node-lkwb5\" (UID: \"2ccbe31a-b945-423d-a88a-45ef25012451\") " pod="openshift-ovn-kubernetes/ovnkube-node-lkwb5"
Sep 17 17:34:42.842679 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:34:42.841286    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/2ccbe31a-b945-423d-a88a-45ef25012451-node-log\") pod \"ovnkube-node-lkwb5\" (UID: \"2ccbe31a-b945-423d-a88a-45ef25012451\") " pod="openshift-ovn-kubernetes/ovnkube-node-lkwb5"
Sep 17 17:34:42.842679 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:34:42.841308    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/d52f9540-79e9-4df7-b3a9-e8c7cc6d48b0-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-gtxcq\" (UID: \"d52f9540-79e9-4df7-b3a9-e8c7cc6d48b0\") " pod="openshift-multus/multus-additional-cni-plugins-gtxcq"
Sep 17 17:34:42.842679 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:34:42.841328    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/d0f23713-30dc-4d47-95cb-c083bec7c18c-registration-dir\") pod \"aws-ebs-csi-driver-node-zvjts\" (UID: \"d0f23713-30dc-4d47-95cb-c083bec7c18c\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-zvjts"
Sep 17 17:34:42.842679 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:34:42.841363    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/173c8902-414c-4212-8d93-ce15853e8ca8-serviceca\") pod \"node-ca-2fp6z\" (UID: \"173c8902-414c-4212-8d93-ce15853e8ca8\") " pod="openshift-image-registry/node-ca-2fp6z"
Sep 17 17:34:42.842679 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:34:42.841409    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/fda64577-6753-498e-ab9e-128a567424ca-cni-binary-copy\") pod \"multus-srdtd\" (UID: \"fda64577-6753-498e-ab9e-128a567424ca\") " pod="openshift-multus/multus-srdtd"
Sep 17 17:34:42.842679 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:34:42.841498    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/fda64577-6753-498e-ab9e-128a567424ca-multus-conf-dir\") pod \"multus-srdtd\" (UID: \"fda64577-6753-498e-ab9e-128a567424ca\") " pod="openshift-multus/multus-srdtd"
Sep 17 17:34:42.842679 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:34:42.841526    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/2ccbe31a-b945-423d-a88a-45ef25012451-etc-openvswitch\") pod \"ovnkube-node-lkwb5\" (UID: \"2ccbe31a-b945-423d-a88a-45ef25012451\") " pod="openshift-ovn-kubernetes/ovnkube-node-lkwb5"
Sep 17 17:34:42.842679 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:34:42.841559    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/2ccbe31a-b945-423d-a88a-45ef25012451-env-overrides\") pod \"ovnkube-node-lkwb5\" (UID: \"2ccbe31a-b945-423d-a88a-45ef25012451\") " pod="openshift-ovn-kubernetes/ovnkube-node-lkwb5"
Sep 17 17:34:42.842679 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:34:42.841585    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-s42l9\" (UniqueName: \"kubernetes.io/projected/d52f9540-79e9-4df7-b3a9-e8c7cc6d48b0-kube-api-access-s42l9\") pod \"multus-additional-cni-plugins-gtxcq\" (UID: \"d52f9540-79e9-4df7-b3a9-e8c7cc6d48b0\") " pod="openshift-multus/multus-additional-cni-plugins-gtxcq"
Sep 17 17:34:42.842679 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:34:42.841613    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/fda64577-6753-498e-ab9e-128a567424ca-system-cni-dir\") pod \"multus-srdtd\" (UID: \"fda64577-6753-498e-ab9e-128a567424ca\") " pod="openshift-multus/multus-srdtd"
Sep 17 17:34:42.842679 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:34:42.841638    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/fda64577-6753-498e-ab9e-128a567424ca-host-run-k8s-cni-cncf-io\") pod \"multus-srdtd\" (UID: \"fda64577-6753-498e-ab9e-128a567424ca\") " pod="openshift-multus/multus-srdtd"
Sep 17 17:34:42.842679 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:34:42.841676    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/fda64577-6753-498e-ab9e-128a567424ca-hostroot\") pod \"multus-srdtd\" (UID: \"fda64577-6753-498e-ab9e-128a567424ca\") " pod="openshift-multus/multus-srdtd"
Sep 17 17:34:42.842679 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:34:42.841704    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/fda64577-6753-498e-ab9e-128a567424ca-etc-kubernetes\") pod \"multus-srdtd\" (UID: \"fda64577-6753-498e-ab9e-128a567424ca\") " pod="openshift-multus/multus-srdtd"
Sep 17 17:34:42.842679 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:34:42.841798    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/fda64577-6753-498e-ab9e-128a567424ca-etc-kubernetes\") pod \"multus-srdtd\" (UID: \"fda64577-6753-498e-ab9e-128a567424ca\") " pod="openshift-multus/multus-srdtd"
Sep 17 17:34:42.842679 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:34:42.841837    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/fda64577-6753-498e-ab9e-128a567424ca-host-run-k8s-cni-cncf-io\") pod \"multus-srdtd\" (UID: \"fda64577-6753-498e-ab9e-128a567424ca\") " pod="openshift-multus/multus-srdtd"
Sep 17 17:34:42.842679 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:34:42.841875    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/fda64577-6753-498e-ab9e-128a567424ca-multus-conf-dir\") pod \"multus-srdtd\" (UID: \"fda64577-6753-498e-ab9e-128a567424ca\") " pod="openshift-multus/multus-srdtd"
Sep 17 17:34:42.843417 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:34:42.841876    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/fda64577-6753-498e-ab9e-128a567424ca-cni-binary-copy\") pod \"multus-srdtd\" (UID: \"fda64577-6753-498e-ab9e-128a567424ca\") " pod="openshift-multus/multus-srdtd"
Sep 17 17:34:42.843417 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:34:42.841996    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/fda64577-6753-498e-ab9e-128a567424ca-system-cni-dir\") pod \"multus-srdtd\" (UID: \"fda64577-6753-498e-ab9e-128a567424ca\") " pod="openshift-multus/multus-srdtd"
Sep 17 17:34:42.843417 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:34:42.841999    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/fda64577-6753-498e-ab9e-128a567424ca-hostroot\") pod \"multus-srdtd\" (UID: \"fda64577-6753-498e-ab9e-128a567424ca\") " pod="openshift-multus/multus-srdtd"
Sep 17 17:34:42.850805 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:34:42.850779    2575 swap_util.go:74] "error creating dir to test if tmpfs noswap is enabled. Assuming not supported" mount path="" error="stat /var/lib/kubelet/plugins/kubernetes.io/empty-dir: no such file or directory"
Sep 17 17:34:42.855582 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:34:42.855557    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-4s4t5\" (UniqueName: \"kubernetes.io/projected/fda64577-6753-498e-ab9e-128a567424ca-kube-api-access-4s4t5\") pod \"multus-srdtd\" (UID: \"fda64577-6753-498e-ab9e-128a567424ca\") " pod="openshift-multus/multus-srdtd"
Sep 17 17:34:42.942458 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:34:42.942407    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zjj2m\" (UniqueName: \"kubernetes.io/projected/83786a21-e36e-4e15-82ec-c28223ef752b-kube-api-access-zjj2m\") pod \"iptables-alerter-n92x5\" (UID: \"83786a21-e36e-4e15-82ec-c28223ef752b\") " pod="openshift-network-operator/iptables-alerter-n92x5"
Sep 17 17:34:42.942622 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:34:42.942449    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/2ccbe31a-b945-423d-a88a-45ef25012451-host-slash\") pod \"ovnkube-node-lkwb5\" (UID: \"2ccbe31a-b945-423d-a88a-45ef25012451\") " pod="openshift-ovn-kubernetes/ovnkube-node-lkwb5"
Sep 17 17:34:42.942622 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:34:42.942495    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/d52f9540-79e9-4df7-b3a9-e8c7cc6d48b0-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-gtxcq\" (UID: \"d52f9540-79e9-4df7-b3a9-e8c7cc6d48b0\") " pod="openshift-multus/multus-additional-cni-plugins-gtxcq"
Sep 17 17:34:42.942622 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:34:42.942532    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/83786a21-e36e-4e15-82ec-c28223ef752b-host-slash\") pod \"iptables-alerter-n92x5\" (UID: \"83786a21-e36e-4e15-82ec-c28223ef752b\") " pod="openshift-network-operator/iptables-alerter-n92x5"
Sep 17 17:34:42.942622 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:34:42.942548    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/2ccbe31a-b945-423d-a88a-45ef25012451-run-systemd\") pod \"ovnkube-node-lkwb5\" (UID: \"2ccbe31a-b945-423d-a88a-45ef25012451\") " pod="openshift-ovn-kubernetes/ovnkube-node-lkwb5"
Sep 17 17:34:42.942622 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:34:42.942569    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/2ccbe31a-b945-423d-a88a-45ef25012451-run-ovn\") pod \"ovnkube-node-lkwb5\" (UID: \"2ccbe31a-b945-423d-a88a-45ef25012451\") " pod="openshift-ovn-kubernetes/ovnkube-node-lkwb5"
Sep 17 17:34:42.942622 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:34:42.942592    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/d0f23713-30dc-4d47-95cb-c083bec7c18c-kubelet-dir\") pod \"aws-ebs-csi-driver-node-zvjts\" (UID: \"d0f23713-30dc-4d47-95cb-c083bec7c18c\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-zvjts"
Sep 17 17:34:42.942622 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:34:42.942619    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/3cb242c8-4f5d-49a3-bb24-85a2170a979e-etc-sysctl-d\") pod \"tuned-gtwrb\" (UID: \"3cb242c8-4f5d-49a3-bb24-85a2170a979e\") " pod="openshift-cluster-node-tuning-operator/tuned-gtwrb"
Sep 17 17:34:42.943054 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:34:42.942630    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/83786a21-e36e-4e15-82ec-c28223ef752b-host-slash\") pod \"iptables-alerter-n92x5\" (UID: \"83786a21-e36e-4e15-82ec-c28223ef752b\") " pod="openshift-network-operator/iptables-alerter-n92x5"
Sep 17 17:34:42.943054 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:34:42.942645    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/3cb242c8-4f5d-49a3-bb24-85a2170a979e-etc-sysctl-conf\") pod \"tuned-gtwrb\" (UID: \"3cb242c8-4f5d-49a3-bb24-85a2170a979e\") " pod="openshift-cluster-node-tuning-operator/tuned-gtwrb"
Sep 17 17:34:42.943054 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:34:42.942671    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/2ccbe31a-b945-423d-a88a-45ef25012451-host-run-netns\") pod \"ovnkube-node-lkwb5\" (UID: \"2ccbe31a-b945-423d-a88a-45ef25012451\") " pod="openshift-ovn-kubernetes/ovnkube-node-lkwb5"
Sep 17 17:34:42.943054 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:34:42.942689    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/2ccbe31a-b945-423d-a88a-45ef25012451-run-systemd\") pod \"ovnkube-node-lkwb5\" (UID: \"2ccbe31a-b945-423d-a88a-45ef25012451\") " pod="openshift-ovn-kubernetes/ovnkube-node-lkwb5"
Sep 17 17:34:42.943054 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:34:42.942697    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/2ccbe31a-b945-423d-a88a-45ef25012451-ovnkube-script-lib\") pod \"ovnkube-node-lkwb5\" (UID: \"2ccbe31a-b945-423d-a88a-45ef25012451\") " pod="openshift-ovn-kubernetes/ovnkube-node-lkwb5"
Sep 17 17:34:42.943054 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:34:42.942728    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/2ccbe31a-b945-423d-a88a-45ef25012451-host-slash\") pod \"ovnkube-node-lkwb5\" (UID: \"2ccbe31a-b945-423d-a88a-45ef25012451\") " pod="openshift-ovn-kubernetes/ovnkube-node-lkwb5"
Sep 17 17:34:42.943054 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:34:42.942727    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8x7rf\" (UniqueName: \"kubernetes.io/projected/2ccbe31a-b945-423d-a88a-45ef25012451-kube-api-access-8x7rf\") pod \"ovnkube-node-lkwb5\" (UID: \"2ccbe31a-b945-423d-a88a-45ef25012451\") " pod="openshift-ovn-kubernetes/ovnkube-node-lkwb5"
Sep 17 17:34:42.943054 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:34:42.942767    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/d52f9540-79e9-4df7-b3a9-e8c7cc6d48b0-system-cni-dir\") pod \"multus-additional-cni-plugins-gtxcq\" (UID: \"d52f9540-79e9-4df7-b3a9-e8c7cc6d48b0\") " pod="openshift-multus/multus-additional-cni-plugins-gtxcq"
Sep 17 17:34:42.943054 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:34:42.942795    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fcz2n\" (UniqueName: \"kubernetes.io/projected/3cb242c8-4f5d-49a3-bb24-85a2170a979e-kube-api-access-fcz2n\") pod \"tuned-gtwrb\" (UID: \"3cb242c8-4f5d-49a3-bb24-85a2170a979e\") " pod="openshift-cluster-node-tuning-operator/tuned-gtwrb"
Sep 17 17:34:42.943054 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:34:42.942819    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/2ccbe31a-b945-423d-a88a-45ef25012451-log-socket\") pod \"ovnkube-node-lkwb5\" (UID: \"2ccbe31a-b945-423d-a88a-45ef25012451\") " pod="openshift-ovn-kubernetes/ovnkube-node-lkwb5"
Sep 17 17:34:42.943054 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:34:42.942843    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/d0f23713-30dc-4d47-95cb-c083bec7c18c-socket-dir\") pod \"aws-ebs-csi-driver-node-zvjts\" (UID: \"d0f23713-30dc-4d47-95cb-c083bec7c18c\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-zvjts"
Sep 17 17:34:42.943054 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:34:42.942868    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/3cb242c8-4f5d-49a3-bb24-85a2170a979e-etc-sysconfig\") pod \"tuned-gtwrb\" (UID: \"3cb242c8-4f5d-49a3-bb24-85a2170a979e\") " pod="openshift-cluster-node-tuning-operator/tuned-gtwrb"
Sep 17 17:34:42.943054 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:34:42.942893    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-69pfg\" (UniqueName: \"kubernetes.io/projected/27866e17-c231-49a6-921f-a83a01b8e173-kube-api-access-69pfg\") pod \"network-metrics-daemon-wdp4h\" (UID: \"27866e17-c231-49a6-921f-a83a01b8e173\") " pod="openshift-multus/network-metrics-daemon-wdp4h"
Sep 17 17:34:42.943054 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:34:42.942899    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/3cb242c8-4f5d-49a3-bb24-85a2170a979e-etc-sysctl-conf\") pod \"tuned-gtwrb\" (UID: \"3cb242c8-4f5d-49a3-bb24-85a2170a979e\") " pod="openshift-cluster-node-tuning-operator/tuned-gtwrb"
Sep 17 17:34:42.943054 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:34:42.942917    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/2ccbe31a-b945-423d-a88a-45ef25012451-host-cni-bin\") pod \"ovnkube-node-lkwb5\" (UID: \"2ccbe31a-b945-423d-a88a-45ef25012451\") " pod="openshift-ovn-kubernetes/ovnkube-node-lkwb5"
Sep 17 17:34:42.943054 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:34:42.942945    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/2ccbe31a-b945-423d-a88a-45ef25012451-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-lkwb5\" (UID: \"2ccbe31a-b945-423d-a88a-45ef25012451\") " pod="openshift-ovn-kubernetes/ovnkube-node-lkwb5"
Sep 17 17:34:42.943054 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:34:42.942973    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/2ccbe31a-b945-423d-a88a-45ef25012451-run-ovn\") pod \"ovnkube-node-lkwb5\" (UID: \"2ccbe31a-b945-423d-a88a-45ef25012451\") " pod="openshift-ovn-kubernetes/ovnkube-node-lkwb5"
Sep 17 17:34:42.943912 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:34:42.942973    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/1d07e68d-9943-420f-a6a0-5f477792fabd-agent-certs\") pod \"konnectivity-agent-djvzt\" (UID: \"1d07e68d-9943-420f-a6a0-5f477792fabd\") " pod="kube-system/konnectivity-agent-djvzt"
Sep 17 17:34:42.943912 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:34:42.943015    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/3cb242c8-4f5d-49a3-bb24-85a2170a979e-etc-kubernetes\") pod \"tuned-gtwrb\" (UID: \"3cb242c8-4f5d-49a3-bb24-85a2170a979e\") " pod="openshift-cluster-node-tuning-operator/tuned-gtwrb"
Sep 17 17:34:42.943912 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:34:42.943025    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/3cb242c8-4f5d-49a3-bb24-85a2170a979e-etc-sysctl-d\") pod \"tuned-gtwrb\" (UID: \"3cb242c8-4f5d-49a3-bb24-85a2170a979e\") " pod="openshift-cluster-node-tuning-operator/tuned-gtwrb"
Sep 17 17:34:42.943912 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:34:42.943039    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/3cb242c8-4f5d-49a3-bb24-85a2170a979e-sys\") pod \"tuned-gtwrb\" (UID: \"3cb242c8-4f5d-49a3-bb24-85a2170a979e\") " pod="openshift-cluster-node-tuning-operator/tuned-gtwrb"
Sep 17 17:34:42.943912 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:34:42.943066    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/2ccbe31a-b945-423d-a88a-45ef25012451-host-cni-netd\") pod \"ovnkube-node-lkwb5\" (UID: \"2ccbe31a-b945-423d-a88a-45ef25012451\") " pod="openshift-ovn-kubernetes/ovnkube-node-lkwb5"
Sep 17 17:34:42.943912 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:34:42.943072    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/2ccbe31a-b945-423d-a88a-45ef25012451-host-run-netns\") pod \"ovnkube-node-lkwb5\" (UID: \"2ccbe31a-b945-423d-a88a-45ef25012451\") " pod="openshift-ovn-kubernetes/ovnkube-node-lkwb5"
Sep 17 17:34:42.943912 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:34:42.943107    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/3cb242c8-4f5d-49a3-bb24-85a2170a979e-sys\") pod \"tuned-gtwrb\" (UID: \"3cb242c8-4f5d-49a3-bb24-85a2170a979e\") " pod="openshift-cluster-node-tuning-operator/tuned-gtwrb"
Sep 17 17:34:42.943912 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:34:42.943092    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/93fd2156-b185-4995-8e49-d6d9c0eda255-hosts-file\") pod \"node-resolver-5549b\" (UID: \"93fd2156-b185-4995-8e49-d6d9c0eda255\") " pod="openshift-dns/node-resolver-5549b"
Sep 17 17:34:42.943912 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:34:42.943137    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/d0f23713-30dc-4d47-95cb-c083bec7c18c-kubelet-dir\") pod \"aws-ebs-csi-driver-node-zvjts\" (UID: \"d0f23713-30dc-4d47-95cb-c083bec7c18c\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-zvjts"
Sep 17 17:34:42.943912 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:34:42.943140    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/d0f23713-30dc-4d47-95cb-c083bec7c18c-device-dir\") pod \"aws-ebs-csi-driver-node-zvjts\" (UID: \"d0f23713-30dc-4d47-95cb-c083bec7c18c\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-zvjts"
Sep 17 17:34:42.943912 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:34:42.943139    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/d52f9540-79e9-4df7-b3a9-e8c7cc6d48b0-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-gtxcq\" (UID: \"d52f9540-79e9-4df7-b3a9-e8c7cc6d48b0\") " pod="openshift-multus/multus-additional-cni-plugins-gtxcq"
Sep 17 17:34:42.943912 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:34:42.943167    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/3cb242c8-4f5d-49a3-bb24-85a2170a979e-etc-kubernetes\") pod \"tuned-gtwrb\" (UID: \"3cb242c8-4f5d-49a3-bb24-85a2170a979e\") " pod="openshift-cluster-node-tuning-operator/tuned-gtwrb"
Sep 17 17:34:42.943912 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:34:42.943167    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/3cb242c8-4f5d-49a3-bb24-85a2170a979e-var-lib-kubelet\") pod \"tuned-gtwrb\" (UID: \"3cb242c8-4f5d-49a3-bb24-85a2170a979e\") " pod="openshift-cluster-node-tuning-operator/tuned-gtwrb"
Sep 17 17:34:42.943912 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:34:42.943198    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/173c8902-414c-4212-8d93-ce15853e8ca8-host\") pod \"node-ca-2fp6z\" (UID: \"173c8902-414c-4212-8d93-ce15853e8ca8\") " pod="openshift-image-registry/node-ca-2fp6z"
Sep 17 17:34:42.943912 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:34:42.943218    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/2ccbe31a-b945-423d-a88a-45ef25012451-run-openvswitch\") pod \"ovnkube-node-lkwb5\" (UID: \"2ccbe31a-b945-423d-a88a-45ef25012451\") " pod="openshift-ovn-kubernetes/ovnkube-node-lkwb5"
Sep 17 17:34:42.943912 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:34:42.943233    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/d0f23713-30dc-4d47-95cb-c083bec7c18c-device-dir\") pod \"aws-ebs-csi-driver-node-zvjts\" (UID: \"d0f23713-30dc-4d47-95cb-c083bec7c18c\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-zvjts"
Sep 17 17:34:42.943912 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:34:42.943235    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/2ccbe31a-b945-423d-a88a-45ef25012451-host-run-ovn-kubernetes\") pod \"ovnkube-node-lkwb5\" (UID: \"2ccbe31a-b945-423d-a88a-45ef25012451\") " pod="openshift-ovn-kubernetes/ovnkube-node-lkwb5"
Sep 17 17:34:42.944759 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:34:42.943254    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/2ccbe31a-b945-423d-a88a-45ef25012451-host-run-ovn-kubernetes\") pod \"ovnkube-node-lkwb5\" (UID: \"2ccbe31a-b945-423d-a88a-45ef25012451\") " pod="openshift-ovn-kubernetes/ovnkube-node-lkwb5"
Sep 17 17:34:42.944759 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:34:42.943275    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/1d07e68d-9943-420f-a6a0-5f477792fabd-konnectivity-ca\") pod \"konnectivity-agent-djvzt\" (UID: \"1d07e68d-9943-420f-a6a0-5f477792fabd\") " pod="kube-system/konnectivity-agent-djvzt"
Sep 17 17:34:42.944759 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:34:42.943288    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/3cb242c8-4f5d-49a3-bb24-85a2170a979e-var-lib-kubelet\") pod \"tuned-gtwrb\" (UID: \"3cb242c8-4f5d-49a3-bb24-85a2170a979e\") " pod="openshift-cluster-node-tuning-operator/tuned-gtwrb"
Sep 17 17:34:42.944759 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:34:42.943300    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/3cb242c8-4f5d-49a3-bb24-85a2170a979e-host\") pod \"tuned-gtwrb\" (UID: \"3cb242c8-4f5d-49a3-bb24-85a2170a979e\") " pod="openshift-cluster-node-tuning-operator/tuned-gtwrb"
Sep 17 17:34:42.944759 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:34:42.943328    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/2ccbe31a-b945-423d-a88a-45ef25012451-systemd-units\") pod \"ovnkube-node-lkwb5\" (UID: \"2ccbe31a-b945-423d-a88a-45ef25012451\") " pod="openshift-ovn-kubernetes/ovnkube-node-lkwb5"
Sep 17 17:34:42.944759 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:34:42.943336    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/93fd2156-b185-4995-8e49-d6d9c0eda255-hosts-file\") pod \"node-resolver-5549b\" (UID: \"93fd2156-b185-4995-8e49-d6d9c0eda255\") " pod="openshift-dns/node-resolver-5549b"
Sep 17 17:34:42.944759 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:34:42.943376    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/2ccbe31a-b945-423d-a88a-45ef25012451-node-log\") pod \"ovnkube-node-lkwb5\" (UID: \"2ccbe31a-b945-423d-a88a-45ef25012451\") " pod="openshift-ovn-kubernetes/ovnkube-node-lkwb5"
Sep 17 17:34:42.944759 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:34:42.943396    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/173c8902-414c-4212-8d93-ce15853e8ca8-host\") pod \"node-ca-2fp6z\" (UID: \"173c8902-414c-4212-8d93-ce15853e8ca8\") " pod="openshift-image-registry/node-ca-2fp6z"
Sep 17 17:34:42.944759 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:34:42.943404    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/d52f9540-79e9-4df7-b3a9-e8c7cc6d48b0-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-gtxcq\" (UID: \"d52f9540-79e9-4df7-b3a9-e8c7cc6d48b0\") " pod="openshift-multus/multus-additional-cni-plugins-gtxcq"
Sep 17 17:34:42.944759 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:34:42.943434    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/2ccbe31a-b945-423d-a88a-45ef25012451-run-openvswitch\") pod \"ovnkube-node-lkwb5\" (UID: \"2ccbe31a-b945-423d-a88a-45ef25012451\") " pod="openshift-ovn-kubernetes/ovnkube-node-lkwb5"
Sep 17 17:34:42.944759 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:34:42.943435    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/d0f23713-30dc-4d47-95cb-c083bec7c18c-registration-dir\") pod \"aws-ebs-csi-driver-node-zvjts\" (UID: \"d0f23713-30dc-4d47-95cb-c083bec7c18c\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-zvjts"
Sep 17 17:34:42.944759 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:34:42.943460    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/3cb242c8-4f5d-49a3-bb24-85a2170a979e-host\") pod \"tuned-gtwrb\" (UID: \"3cb242c8-4f5d-49a3-bb24-85a2170a979e\") " pod="openshift-cluster-node-tuning-operator/tuned-gtwrb"
Sep 17 17:34:42.944759 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:34:42.943480    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/d0f23713-30dc-4d47-95cb-c083bec7c18c-registration-dir\") pod \"aws-ebs-csi-driver-node-zvjts\" (UID: \"d0f23713-30dc-4d47-95cb-c083bec7c18c\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-zvjts"
Sep 17 17:34:42.944759 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:34:42.943467    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/173c8902-414c-4212-8d93-ce15853e8ca8-serviceca\") pod \"node-ca-2fp6z\" (UID: \"173c8902-414c-4212-8d93-ce15853e8ca8\") " pod="openshift-image-registry/node-ca-2fp6z"
Sep 17 17:34:42.944759 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:34:42.943527    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/2ccbe31a-b945-423d-a88a-45ef25012451-etc-openvswitch\") pod \"ovnkube-node-lkwb5\" (UID: \"2ccbe31a-b945-423d-a88a-45ef25012451\") " pod="openshift-ovn-kubernetes/ovnkube-node-lkwb5"
Sep 17 17:34:42.944759 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:34:42.943543    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/2ccbe31a-b945-423d-a88a-45ef25012451-systemd-units\") pod \"ovnkube-node-lkwb5\" (UID: \"2ccbe31a-b945-423d-a88a-45ef25012451\") " pod="openshift-ovn-kubernetes/ovnkube-node-lkwb5"
Sep 17 17:34:42.944759 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:34:42.943559    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/2ccbe31a-b945-423d-a88a-45ef25012451-env-overrides\") pod \"ovnkube-node-lkwb5\" (UID: \"2ccbe31a-b945-423d-a88a-45ef25012451\") " pod="openshift-ovn-kubernetes/ovnkube-node-lkwb5"
Sep 17 17:34:42.944759 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:34:42.943583    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/2ccbe31a-b945-423d-a88a-45ef25012451-node-log\") pod \"ovnkube-node-lkwb5\" (UID: \"2ccbe31a-b945-423d-a88a-45ef25012451\") " pod="openshift-ovn-kubernetes/ovnkube-node-lkwb5"
Sep 17 17:34:42.945608 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:34:42.943589    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-s42l9\" (UniqueName: \"kubernetes.io/projected/d52f9540-79e9-4df7-b3a9-e8c7cc6d48b0-kube-api-access-s42l9\") pod \"multus-additional-cni-plugins-gtxcq\" (UID: \"d52f9540-79e9-4df7-b3a9-e8c7cc6d48b0\") " pod="openshift-multus/multus-additional-cni-plugins-gtxcq"
Sep 17 17:34:42.945608 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:34:42.943627    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7rzlr\" (UniqueName: \"kubernetes.io/projected/57bf735f-0367-4799-a1a8-889870d65704-kube-api-access-7rzlr\") pod \"network-check-target-pxt82\" (UID: \"57bf735f-0367-4799-a1a8-889870d65704\") " pod="openshift-network-diagnostics/network-check-target-pxt82"
Sep 17 17:34:42.945608 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:34:42.943652    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/93fd2156-b185-4995-8e49-d6d9c0eda255-tmp-dir\") pod \"node-resolver-5549b\" (UID: \"93fd2156-b185-4995-8e49-d6d9c0eda255\") " pod="openshift-dns/node-resolver-5549b"
Sep 17 17:34:42.945608 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:34:42.944015    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/d0f23713-30dc-4d47-95cb-c083bec7c18c-socket-dir\") pod \"aws-ebs-csi-driver-node-zvjts\" (UID: \"d0f23713-30dc-4d47-95cb-c083bec7c18c\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-zvjts"
Sep 17 17:34:42.945608 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:34:42.944530    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/2ccbe31a-b945-423d-a88a-45ef25012451-host-cni-bin\") pod \"ovnkube-node-lkwb5\" (UID: \"2ccbe31a-b945-423d-a88a-45ef25012451\") " pod="openshift-ovn-kubernetes/ovnkube-node-lkwb5"
Sep 17 17:34:42.945608 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:34:42.944621    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/2ccbe31a-b945-423d-a88a-45ef25012451-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-lkwb5\" (UID: \"2ccbe31a-b945-423d-a88a-45ef25012451\") " pod="openshift-ovn-kubernetes/ovnkube-node-lkwb5"
Sep 17 17:34:42.945608 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:34:42.944827    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/2ccbe31a-b945-423d-a88a-45ef25012451-ovnkube-script-lib\") pod \"ovnkube-node-lkwb5\" (UID: \"2ccbe31a-b945-423d-a88a-45ef25012451\") " pod="openshift-ovn-kubernetes/ovnkube-node-lkwb5"
Sep 17 17:34:42.945608 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:34:42.944883    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/d0f23713-30dc-4d47-95cb-c083bec7c18c-etc-selinux\") pod \"aws-ebs-csi-driver-node-zvjts\" (UID: \"d0f23713-30dc-4d47-95cb-c083bec7c18c\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-zvjts"
Sep 17 17:34:42.945608 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:34:42.944930    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-f4kqw\" (UniqueName: \"kubernetes.io/projected/173c8902-414c-4212-8d93-ce15853e8ca8-kube-api-access-f4kqw\") pod \"node-ca-2fp6z\" (UID: \"173c8902-414c-4212-8d93-ce15853e8ca8\") " pod="openshift-image-registry/node-ca-2fp6z"
Sep 17 17:34:42.945608 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:34:42.944971    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/83786a21-e36e-4e15-82ec-c28223ef752b-iptables-alerter-script\") pod \"iptables-alerter-n92x5\" (UID: \"83786a21-e36e-4e15-82ec-c28223ef752b\") " pod="openshift-network-operator/iptables-alerter-n92x5"
Sep 17 17:34:42.945608 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:34:42.945000    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/d52f9540-79e9-4df7-b3a9-e8c7cc6d48b0-cnibin\") pod \"multus-additional-cni-plugins-gtxcq\" (UID: \"d52f9540-79e9-4df7-b3a9-e8c7cc6d48b0\") " pod="openshift-multus/multus-additional-cni-plugins-gtxcq"
Sep 17 17:34:42.945608 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:34:42.945036    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/d0f23713-30dc-4d47-95cb-c083bec7c18c-sys-fs\") pod \"aws-ebs-csi-driver-node-zvjts\" (UID: \"d0f23713-30dc-4d47-95cb-c083bec7c18c\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-zvjts"
Sep 17 17:34:42.945608 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:34:42.945072    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/d52f9540-79e9-4df7-b3a9-e8c7cc6d48b0-cni-binary-copy\") pod \"multus-additional-cni-plugins-gtxcq\" (UID: \"d52f9540-79e9-4df7-b3a9-e8c7cc6d48b0\") " pod="openshift-multus/multus-additional-cni-plugins-gtxcq"
Sep 17 17:34:42.946270 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:34:42.945663    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/d52f9540-79e9-4df7-b3a9-e8c7cc6d48b0-cni-binary-copy\") pod \"multus-additional-cni-plugins-gtxcq\" (UID: \"d52f9540-79e9-4df7-b3a9-e8c7cc6d48b0\") " pod="openshift-multus/multus-additional-cni-plugins-gtxcq"
Sep 17 17:34:42.946414 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:34:42.946383    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/2ccbe31a-b945-423d-a88a-45ef25012451-host-cni-netd\") pod \"ovnkube-node-lkwb5\" (UID: \"2ccbe31a-b945-423d-a88a-45ef25012451\") " pod="openshift-ovn-kubernetes/ovnkube-node-lkwb5"
Sep 17 17:34:42.946414 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:34:42.946412    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/83786a21-e36e-4e15-82ec-c28223ef752b-iptables-alerter-script\") pod \"iptables-alerter-n92x5\" (UID: \"83786a21-e36e-4e15-82ec-c28223ef752b\") " pod="openshift-network-operator/iptables-alerter-n92x5"
Sep 17 17:34:42.946615 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:34:42.946479    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/d52f9540-79e9-4df7-b3a9-e8c7cc6d48b0-cnibin\") pod \"multus-additional-cni-plugins-gtxcq\" (UID: \"d52f9540-79e9-4df7-b3a9-e8c7cc6d48b0\") " pod="openshift-multus/multus-additional-cni-plugins-gtxcq"
Sep 17 17:34:42.946615 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:34:42.946567    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/d0f23713-30dc-4d47-95cb-c083bec7c18c-sys-fs\") pod \"aws-ebs-csi-driver-node-zvjts\" (UID: \"d0f23713-30dc-4d47-95cb-c083bec7c18c\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-zvjts"
Sep 17 17:34:42.946715 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:34:42.946693    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/1d07e68d-9943-420f-a6a0-5f477792fabd-konnectivity-ca\") pod \"konnectivity-agent-djvzt\" (UID: \"1d07e68d-9943-420f-a6a0-5f477792fabd\") " pod="kube-system/konnectivity-agent-djvzt"
Sep 17 17:34:42.947183 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:34:42.946836    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/2ccbe31a-b945-423d-a88a-45ef25012451-log-socket\") pod \"ovnkube-node-lkwb5\" (UID: \"2ccbe31a-b945-423d-a88a-45ef25012451\") " pod="openshift-ovn-kubernetes/ovnkube-node-lkwb5"
Sep 17 17:34:42.947183 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:34:42.946908    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/173c8902-414c-4212-8d93-ce15853e8ca8-serviceca\") pod \"node-ca-2fp6z\" (UID: \"173c8902-414c-4212-8d93-ce15853e8ca8\") " pod="openshift-image-registry/node-ca-2fp6z"
Sep 17 17:34:42.947183 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:34:42.946916    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/d52f9540-79e9-4df7-b3a9-e8c7cc6d48b0-system-cni-dir\") pod \"multus-additional-cni-plugins-gtxcq\" (UID: \"d52f9540-79e9-4df7-b3a9-e8c7cc6d48b0\") " pod="openshift-multus/multus-additional-cni-plugins-gtxcq"
Sep 17 17:34:42.947183 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:34:42.946966    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/d52f9540-79e9-4df7-b3a9-e8c7cc6d48b0-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-gtxcq\" (UID: \"d52f9540-79e9-4df7-b3a9-e8c7cc6d48b0\") " pod="openshift-multus/multus-additional-cni-plugins-gtxcq"
Sep 17 17:34:42.947183 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:34:42.947018    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/2ccbe31a-b945-423d-a88a-45ef25012451-etc-openvswitch\") pod \"ovnkube-node-lkwb5\" (UID: \"2ccbe31a-b945-423d-a88a-45ef25012451\") " pod="openshift-ovn-kubernetes/ovnkube-node-lkwb5"
Sep 17 17:34:42.947183 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:34:42.947086    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/d0f23713-30dc-4d47-95cb-c083bec7c18c-etc-selinux\") pod \"aws-ebs-csi-driver-node-zvjts\" (UID: \"d0f23713-30dc-4d47-95cb-c083bec7c18c\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-zvjts"
Sep 17 17:34:42.947183 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:34:42.947157    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/1d07e68d-9943-420f-a6a0-5f477792fabd-agent-certs\") pod \"konnectivity-agent-djvzt\" (UID: \"1d07e68d-9943-420f-a6a0-5f477792fabd\") " pod="kube-system/konnectivity-agent-djvzt"
Sep 17 17:34:42.947594 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:34:42.947349    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/93fd2156-b185-4995-8e49-d6d9c0eda255-tmp-dir\") pod \"node-resolver-5549b\" (UID: \"93fd2156-b185-4995-8e49-d6d9c0eda255\") " pod="openshift-dns/node-resolver-5549b"
Sep 17 17:34:42.947594 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:34:42.947369    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/2ccbe31a-b945-423d-a88a-45ef25012451-env-overrides\") pod \"ovnkube-node-lkwb5\" (UID: \"2ccbe31a-b945-423d-a88a-45ef25012451\") " pod="openshift-ovn-kubernetes/ovnkube-node-lkwb5"
Sep 17 17:34:42.947594 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:34:42.947429    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/3cb242c8-4f5d-49a3-bb24-85a2170a979e-etc-sysconfig\") pod \"tuned-gtwrb\" (UID: \"3cb242c8-4f5d-49a3-bb24-85a2170a979e\") " pod="openshift-cluster-node-tuning-operator/tuned-gtwrb"
Sep 17 17:34:42.947594 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:34:42.947497    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/d52f9540-79e9-4df7-b3a9-e8c7cc6d48b0-tuning-conf-dir\") pod \"multus-additional-cni-plugins-gtxcq\" (UID: \"d52f9540-79e9-4df7-b3a9-e8c7cc6d48b0\") " pod="openshift-multus/multus-additional-cni-plugins-gtxcq"
Sep 17 17:34:42.947594 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:34:42.947558    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/27866e17-c231-49a6-921f-a83a01b8e173-metrics-certs\") pod \"network-metrics-daemon-wdp4h\" (UID: \"27866e17-c231-49a6-921f-a83a01b8e173\") " pod="openshift-multus/network-metrics-daemon-wdp4h"
Sep 17 17:34:42.947821 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:34:42.947607    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/2ccbe31a-b945-423d-a88a-45ef25012451-host-kubelet\") pod \"ovnkube-node-lkwb5\" (UID: \"2ccbe31a-b945-423d-a88a-45ef25012451\") " pod="openshift-ovn-kubernetes/ovnkube-node-lkwb5"
Sep 17 17:34:42.947821 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:34:42.947640    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dksdw\" (UniqueName: \"kubernetes.io/projected/d0f23713-30dc-4d47-95cb-c083bec7c18c-kube-api-access-dksdw\") pod \"aws-ebs-csi-driver-node-zvjts\" (UID: \"d0f23713-30dc-4d47-95cb-c083bec7c18c\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-zvjts"
Sep 17 17:34:42.947821 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:34:42.947661    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/d52f9540-79e9-4df7-b3a9-e8c7cc6d48b0-tuning-conf-dir\") pod \"multus-additional-cni-plugins-gtxcq\" (UID: \"d52f9540-79e9-4df7-b3a9-e8c7cc6d48b0\") " pod="openshift-multus/multus-additional-cni-plugins-gtxcq"
Sep 17 17:34:42.947821 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:34:42.947671    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/3cb242c8-4f5d-49a3-bb24-85a2170a979e-etc-systemd\") pod \"tuned-gtwrb\" (UID: \"3cb242c8-4f5d-49a3-bb24-85a2170a979e\") " pod="openshift-cluster-node-tuning-operator/tuned-gtwrb"
Sep 17 17:34:42.947821 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:34:42.947701    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/3cb242c8-4f5d-49a3-bb24-85a2170a979e-lib-modules\") pod \"tuned-gtwrb\" (UID: \"3cb242c8-4f5d-49a3-bb24-85a2170a979e\") " pod="openshift-cluster-node-tuning-operator/tuned-gtwrb"
Sep 17 17:34:42.947821 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:34:42.947725    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/3cb242c8-4f5d-49a3-bb24-85a2170a979e-etc-tuned\") pod \"tuned-gtwrb\" (UID: \"3cb242c8-4f5d-49a3-bb24-85a2170a979e\") " pod="openshift-cluster-node-tuning-operator/tuned-gtwrb"
Sep 17 17:34:42.947821 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:34:42.947735    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/2ccbe31a-b945-423d-a88a-45ef25012451-host-kubelet\") pod \"ovnkube-node-lkwb5\" (UID: \"2ccbe31a-b945-423d-a88a-45ef25012451\") " pod="openshift-ovn-kubernetes/ovnkube-node-lkwb5"
Sep 17 17:34:42.947821 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:34:42.947755    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/3cb242c8-4f5d-49a3-bb24-85a2170a979e-tmp\") pod \"tuned-gtwrb\" (UID: \"3cb242c8-4f5d-49a3-bb24-85a2170a979e\") " pod="openshift-cluster-node-tuning-operator/tuned-gtwrb"
Sep 17 17:34:42.948193 ip-10-0-143-74 kubenswrapper[2575]: E0917 17:34:42.947840    2575 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 17 17:34:42.948193 ip-10-0-143-74 kubenswrapper[2575]: E0917 17:34:42.947918    2575 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/27866e17-c231-49a6-921f-a83a01b8e173-metrics-certs podName:27866e17-c231-49a6-921f-a83a01b8e173 nodeName:}" failed. No retries permitted until 2026-09-17 17:34:43.447892626 +0000 UTC m=+3.078222583 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/27866e17-c231-49a6-921f-a83a01b8e173-metrics-certs") pod "network-metrics-daemon-wdp4h" (UID: "27866e17-c231-49a6-921f-a83a01b8e173") : object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 17 17:34:42.948193 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:34:42.948137    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/2ccbe31a-b945-423d-a88a-45ef25012451-var-lib-openvswitch\") pod \"ovnkube-node-lkwb5\" (UID: \"2ccbe31a-b945-423d-a88a-45ef25012451\") " pod="openshift-ovn-kubernetes/ovnkube-node-lkwb5"
Sep 17 17:34:42.948193 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:34:42.948170    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/2ccbe31a-b945-423d-a88a-45ef25012451-ovnkube-config\") pod \"ovnkube-node-lkwb5\" (UID: \"2ccbe31a-b945-423d-a88a-45ef25012451\") " pod="openshift-ovn-kubernetes/ovnkube-node-lkwb5"
Sep 17 17:34:42.948388 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:34:42.948204    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/2ccbe31a-b945-423d-a88a-45ef25012451-ovn-node-metrics-cert\") pod \"ovnkube-node-lkwb5\" (UID: \"2ccbe31a-b945-423d-a88a-45ef25012451\") " pod="openshift-ovn-kubernetes/ovnkube-node-lkwb5"
Sep 17 17:34:42.948388 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:34:42.948239    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-d29hd\" (UniqueName: \"kubernetes.io/projected/93fd2156-b185-4995-8e49-d6d9c0eda255-kube-api-access-d29hd\") pod \"node-resolver-5549b\" (UID: \"93fd2156-b185-4995-8e49-d6d9c0eda255\") " pod="openshift-dns/node-resolver-5549b"
Sep 17 17:34:42.948388 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:34:42.948271    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/3cb242c8-4f5d-49a3-bb24-85a2170a979e-etc-modprobe-d\") pod \"tuned-gtwrb\" (UID: \"3cb242c8-4f5d-49a3-bb24-85a2170a979e\") " pod="openshift-cluster-node-tuning-operator/tuned-gtwrb"
Sep 17 17:34:42.948388 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:34:42.948299    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/3cb242c8-4f5d-49a3-bb24-85a2170a979e-run\") pod \"tuned-gtwrb\" (UID: \"3cb242c8-4f5d-49a3-bb24-85a2170a979e\") " pod="openshift-cluster-node-tuning-operator/tuned-gtwrb"
Sep 17 17:34:42.948602 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:34:42.948406    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run\" (UniqueName: \"kubernetes.io/host-path/3cb242c8-4f5d-49a3-bb24-85a2170a979e-run\") pod \"tuned-gtwrb\" (UID: \"3cb242c8-4f5d-49a3-bb24-85a2170a979e\") " pod="openshift-cluster-node-tuning-operator/tuned-gtwrb"
Sep 17 17:34:42.948602 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:34:42.948464    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/2ccbe31a-b945-423d-a88a-45ef25012451-var-lib-openvswitch\") pod \"ovnkube-node-lkwb5\" (UID: \"2ccbe31a-b945-423d-a88a-45ef25012451\") " pod="openshift-ovn-kubernetes/ovnkube-node-lkwb5"
Sep 17 17:34:42.951674 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:34:42.948776    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/3cb242c8-4f5d-49a3-bb24-85a2170a979e-etc-systemd\") pod \"tuned-gtwrb\" (UID: \"3cb242c8-4f5d-49a3-bb24-85a2170a979e\") " pod="openshift-cluster-node-tuning-operator/tuned-gtwrb"
Sep 17 17:34:42.951674 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:34:42.948972    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/2ccbe31a-b945-423d-a88a-45ef25012451-ovnkube-config\") pod \"ovnkube-node-lkwb5\" (UID: \"2ccbe31a-b945-423d-a88a-45ef25012451\") " pod="openshift-ovn-kubernetes/ovnkube-node-lkwb5"
Sep 17 17:34:42.951674 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:34:42.949137    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/3cb242c8-4f5d-49a3-bb24-85a2170a979e-lib-modules\") pod \"tuned-gtwrb\" (UID: \"3cb242c8-4f5d-49a3-bb24-85a2170a979e\") " pod="openshift-cluster-node-tuning-operator/tuned-gtwrb"
Sep 17 17:34:42.951674 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:34:42.949535    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/3cb242c8-4f5d-49a3-bb24-85a2170a979e-etc-modprobe-d\") pod \"tuned-gtwrb\" (UID: \"3cb242c8-4f5d-49a3-bb24-85a2170a979e\") " pod="openshift-cluster-node-tuning-operator/tuned-gtwrb"
Sep 17 17:34:42.952404 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:34:42.952367    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/3cb242c8-4f5d-49a3-bb24-85a2170a979e-tmp\") pod \"tuned-gtwrb\" (UID: \"3cb242c8-4f5d-49a3-bb24-85a2170a979e\") " pod="openshift-cluster-node-tuning-operator/tuned-gtwrb"
Sep 17 17:34:42.952885 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:34:42.952861    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/2ccbe31a-b945-423d-a88a-45ef25012451-ovn-node-metrics-cert\") pod \"ovnkube-node-lkwb5\" (UID: \"2ccbe31a-b945-423d-a88a-45ef25012451\") " pod="openshift-ovn-kubernetes/ovnkube-node-lkwb5"
Sep 17 17:34:42.953021 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:34:42.952996    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/3cb242c8-4f5d-49a3-bb24-85a2170a979e-etc-tuned\") pod \"tuned-gtwrb\" (UID: \"3cb242c8-4f5d-49a3-bb24-85a2170a979e\") " pod="openshift-cluster-node-tuning-operator/tuned-gtwrb"
Sep 17 17:34:42.956024 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:34:42.955973    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-8x7rf\" (UniqueName: \"kubernetes.io/projected/2ccbe31a-b945-423d-a88a-45ef25012451-kube-api-access-8x7rf\") pod \"ovnkube-node-lkwb5\" (UID: \"2ccbe31a-b945-423d-a88a-45ef25012451\") " pod="openshift-ovn-kubernetes/ovnkube-node-lkwb5"
Sep 17 17:34:42.956569 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:34:42.956528    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-zjj2m\" (UniqueName: \"kubernetes.io/projected/83786a21-e36e-4e15-82ec-c28223ef752b-kube-api-access-zjj2m\") pod \"iptables-alerter-n92x5\" (UID: \"83786a21-e36e-4e15-82ec-c28223ef752b\") " pod="openshift-network-operator/iptables-alerter-n92x5"
Sep 17 17:34:42.959273 ip-10-0-143-74 kubenswrapper[2575]: E0917 17:34:42.959244    2575 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Sep 17 17:34:42.959273 ip-10-0-143-74 kubenswrapper[2575]: E0917 17:34:42.959269    2575 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Sep 17 17:34:42.959398 ip-10-0-143-74 kubenswrapper[2575]: E0917 17:34:42.959285    2575 projected.go:196] Error preparing data for projected volume kube-api-access-7rzlr for pod openshift-network-diagnostics/network-check-target-pxt82: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 17 17:34:42.959398 ip-10-0-143-74 kubenswrapper[2575]: E0917 17:34:42.959351    2575 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/57bf735f-0367-4799-a1a8-889870d65704-kube-api-access-7rzlr podName:57bf735f-0367-4799-a1a8-889870d65704 nodeName:}" failed. No retries permitted until 2026-09-17 17:34:43.459333465 +0000 UTC m=+3.089663435 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-api-access-7rzlr" (UniqueName: "kubernetes.io/projected/57bf735f-0367-4799-a1a8-889870d65704-kube-api-access-7rzlr") pod "network-check-target-pxt82" (UID: "57bf735f-0367-4799-a1a8-889870d65704") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 17 17:34:42.962704 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:34:42.962655    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-f4kqw\" (UniqueName: \"kubernetes.io/projected/173c8902-414c-4212-8d93-ce15853e8ca8-kube-api-access-f4kqw\") pod \"node-ca-2fp6z\" (UID: \"173c8902-414c-4212-8d93-ce15853e8ca8\") " pod="openshift-image-registry/node-ca-2fp6z"
Sep 17 17:34:42.962806 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:34:42.962765    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-dksdw\" (UniqueName: \"kubernetes.io/projected/d0f23713-30dc-4d47-95cb-c083bec7c18c-kube-api-access-dksdw\") pod \"aws-ebs-csi-driver-node-zvjts\" (UID: \"d0f23713-30dc-4d47-95cb-c083bec7c18c\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-zvjts"
Sep 17 17:34:42.962880 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:34:42.962833    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-fcz2n\" (UniqueName: \"kubernetes.io/projected/3cb242c8-4f5d-49a3-bb24-85a2170a979e-kube-api-access-fcz2n\") pod \"tuned-gtwrb\" (UID: \"3cb242c8-4f5d-49a3-bb24-85a2170a979e\") " pod="openshift-cluster-node-tuning-operator/tuned-gtwrb"
Sep 17 17:34:42.963947 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:34:42.963915    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-d29hd\" (UniqueName: \"kubernetes.io/projected/93fd2156-b185-4995-8e49-d6d9c0eda255-kube-api-access-d29hd\") pod \"node-resolver-5549b\" (UID: \"93fd2156-b185-4995-8e49-d6d9c0eda255\") " pod="openshift-dns/node-resolver-5549b"
Sep 17 17:34:42.964357 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:34:42.964330    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-69pfg\" (UniqueName: \"kubernetes.io/projected/27866e17-c231-49a6-921f-a83a01b8e173-kube-api-access-69pfg\") pod \"network-metrics-daemon-wdp4h\" (UID: \"27866e17-c231-49a6-921f-a83a01b8e173\") " pod="openshift-multus/network-metrics-daemon-wdp4h"
Sep 17 17:34:42.964655 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:34:42.964634    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-s42l9\" (UniqueName: \"kubernetes.io/projected/d52f9540-79e9-4df7-b3a9-e8c7cc6d48b0-kube-api-access-s42l9\") pod \"multus-additional-cni-plugins-gtxcq\" (UID: \"d52f9540-79e9-4df7-b3a9-e8c7cc6d48b0\") " pod="openshift-multus/multus-additional-cni-plugins-gtxcq"
Sep 17 17:34:42.966556 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:34:42.966497    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-143-74.ec2.internal" event={"ID":"d937ab292439efa4b2d5e0f9e6f432b8","Type":"ContainerStarted","Data":"f4a05c17ee51b2039bfa0e2368495980ac2acc46d5bf908f9a9c7e10d29266aa"}
Sep 17 17:34:42.967900 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:34:42.967879    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-143-74.ec2.internal" event={"ID":"50d07cdbe5bdbd290f2ca595730f87af","Type":"ContainerStarted","Data":"51b3b7d86caf7b4ff19a101655b9883b6390735ee3026b700bfe33e179142e8c"}
Sep 17 17:34:43.044746 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:34:43.044707    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-srdtd"
Sep 17 17:34:43.059001 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:34:43.058972    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-5549b"
Sep 17 17:34:43.069034 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:34:43.069011    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-n92x5"
Sep 17 17:34:43.075912 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:34:43.075891    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-lkwb5"
Sep 17 17:34:43.083896 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:34:43.083868    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-zvjts"
Sep 17 17:34:43.093644 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:34:43.093626    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-djvzt"
Sep 17 17:34:43.102575 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:34:43.102553    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-gtwrb"
Sep 17 17:34:43.111466 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:34:43.111434    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-2fp6z"
Sep 17 17:34:43.118332 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:34:43.118311    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-gtxcq"
Sep 17 17:34:43.267382 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:34:43.267298    2575 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160"
Sep 17 17:34:43.351643 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:34:43.351604    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/7892b408-d2f1-4e39-9bf6-e68a452196b1-original-pull-secret\") pod \"global-pull-secret-syncer-9s5wl\" (UID: \"7892b408-d2f1-4e39-9bf6-e68a452196b1\") " pod="kube-system/global-pull-secret-syncer-9s5wl"
Sep 17 17:34:43.351803 ip-10-0-143-74 kubenswrapper[2575]: E0917 17:34:43.351763    2575 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Sep 17 17:34:43.351854 ip-10-0-143-74 kubenswrapper[2575]: E0917 17:34:43.351844    2575 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/7892b408-d2f1-4e39-9bf6-e68a452196b1-original-pull-secret podName:7892b408-d2f1-4e39-9bf6-e68a452196b1 nodeName:}" failed. No retries permitted until 2026-09-17 17:34:44.351826499 +0000 UTC m=+3.982156453 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/7892b408-d2f1-4e39-9bf6-e68a452196b1-original-pull-secret") pod "global-pull-secret-syncer-9s5wl" (UID: "7892b408-d2f1-4e39-9bf6-e68a452196b1") : object "kube-system"/"original-pull-secret" not registered
Sep 17 17:34:43.453001 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:34:43.452964    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/27866e17-c231-49a6-921f-a83a01b8e173-metrics-certs\") pod \"network-metrics-daemon-wdp4h\" (UID: \"27866e17-c231-49a6-921f-a83a01b8e173\") " pod="openshift-multus/network-metrics-daemon-wdp4h"
Sep 17 17:34:43.453175 ip-10-0-143-74 kubenswrapper[2575]: E0917 17:34:43.453143    2575 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 17 17:34:43.453220 ip-10-0-143-74 kubenswrapper[2575]: E0917 17:34:43.453215    2575 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/27866e17-c231-49a6-921f-a83a01b8e173-metrics-certs podName:27866e17-c231-49a6-921f-a83a01b8e173 nodeName:}" failed. No retries permitted until 2026-09-17 17:34:44.453197408 +0000 UTC m=+4.083527367 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/27866e17-c231-49a6-921f-a83a01b8e173-metrics-certs") pod "network-metrics-daemon-wdp4h" (UID: "27866e17-c231-49a6-921f-a83a01b8e173") : object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 17 17:34:43.554272 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:34:43.554177    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7rzlr\" (UniqueName: \"kubernetes.io/projected/57bf735f-0367-4799-a1a8-889870d65704-kube-api-access-7rzlr\") pod \"network-check-target-pxt82\" (UID: \"57bf735f-0367-4799-a1a8-889870d65704\") " pod="openshift-network-diagnostics/network-check-target-pxt82"
Sep 17 17:34:43.554441 ip-10-0-143-74 kubenswrapper[2575]: E0917 17:34:43.554359    2575 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Sep 17 17:34:43.554441 ip-10-0-143-74 kubenswrapper[2575]: E0917 17:34:43.554384    2575 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Sep 17 17:34:43.554441 ip-10-0-143-74 kubenswrapper[2575]: E0917 17:34:43.554395    2575 projected.go:196] Error preparing data for projected volume kube-api-access-7rzlr for pod openshift-network-diagnostics/network-check-target-pxt82: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 17 17:34:43.554582 ip-10-0-143-74 kubenswrapper[2575]: E0917 17:34:43.554450    2575 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/57bf735f-0367-4799-a1a8-889870d65704-kube-api-access-7rzlr podName:57bf735f-0367-4799-a1a8-889870d65704 nodeName:}" failed. No retries permitted until 2026-09-17 17:34:44.554435676 +0000 UTC m=+4.184765654 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-api-access-7rzlr" (UniqueName: "kubernetes.io/projected/57bf735f-0367-4799-a1a8-889870d65704-kube-api-access-7rzlr") pod "network-check-target-pxt82" (UID: "57bf735f-0367-4799-a1a8-889870d65704") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 17 17:34:43.791074 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:34:43.791031    2575 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-09-16 17:29:43 +0000 UTC" deadline="2028-03-05 19:06:00.578963137 +0000 UTC"
Sep 17 17:34:43.791074 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:34:43.791066    2575 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="12841h31m16.787900874s"
Sep 17 17:34:43.961061 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:34:43.960972    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-wdp4h"
Sep 17 17:34:43.961231 ip-10-0-143-74 kubenswrapper[2575]: E0917 17:34:43.961112    2575 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-wdp4h" podUID="27866e17-c231-49a6-921f-a83a01b8e173"
Sep 17 17:34:44.017046 ip-10-0-143-74 kubenswrapper[2575]: W0917 17:34:44.017024    2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod93fd2156_b185_4995_8e49_d6d9c0eda255.slice/crio-da87d3ab7ca776cc526236cdeb2019804b7d607707e5cefdc7e935d71e43a4f5 WatchSource:0}: Error finding container da87d3ab7ca776cc526236cdeb2019804b7d607707e5cefdc7e935d71e43a4f5: Status 404 returned error can't find the container with id da87d3ab7ca776cc526236cdeb2019804b7d607707e5cefdc7e935d71e43a4f5
Sep 17 17:34:44.018723 ip-10-0-143-74 kubenswrapper[2575]: W0917 17:34:44.018695    2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3cb242c8_4f5d_49a3_bb24_85a2170a979e.slice/crio-fd21941945dfca4f5f40f3896bc6369d6f227f6dd1607f2c4c269a7c51ba6e7b WatchSource:0}: Error finding container fd21941945dfca4f5f40f3896bc6369d6f227f6dd1607f2c4c269a7c51ba6e7b: Status 404 returned error can't find the container with id fd21941945dfca4f5f40f3896bc6369d6f227f6dd1607f2c4c269a7c51ba6e7b
Sep 17 17:34:44.041919 ip-10-0-143-74 kubenswrapper[2575]: W0917 17:34:44.041884    2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2ccbe31a_b945_423d_a88a_45ef25012451.slice/crio-ce4f411b29db8ebdf3c0135be2df295e4395dd5bd95936718c6f1b1c3d8fd6db WatchSource:0}: Error finding container ce4f411b29db8ebdf3c0135be2df295e4395dd5bd95936718c6f1b1c3d8fd6db: Status 404 returned error can't find the container with id ce4f411b29db8ebdf3c0135be2df295e4395dd5bd95936718c6f1b1c3d8fd6db
Sep 17 17:34:44.042638 ip-10-0-143-74 kubenswrapper[2575]: W0917 17:34:44.042602    2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd52f9540_79e9_4df7_b3a9_e8c7cc6d48b0.slice/crio-b800f47fedc9b3a14910d4c26df858806484d69f3ff163854f39ef913788ef4b WatchSource:0}: Error finding container b800f47fedc9b3a14910d4c26df858806484d69f3ff163854f39ef913788ef4b: Status 404 returned error can't find the container with id b800f47fedc9b3a14910d4c26df858806484d69f3ff163854f39ef913788ef4b
Sep 17 17:34:44.043590 ip-10-0-143-74 kubenswrapper[2575]: W0917 17:34:44.043532    2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1d07e68d_9943_420f_a6a0_5f477792fabd.slice/crio-170c9704728107821d5d560436f7b07ccc5b9234877b98638b1bbb8a92cc58b8 WatchSource:0}: Error finding container 170c9704728107821d5d560436f7b07ccc5b9234877b98638b1bbb8a92cc58b8: Status 404 returned error can't find the container with id 170c9704728107821d5d560436f7b07ccc5b9234877b98638b1bbb8a92cc58b8
Sep 17 17:34:44.044542 ip-10-0-143-74 kubenswrapper[2575]: W0917 17:34:44.044473    2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podfda64577_6753_498e_ab9e_128a567424ca.slice/crio-0dd5d61271f67838fefe132b7e14f701a279314b1b20c799f90c277bfcc61cf1 WatchSource:0}: Error finding container 0dd5d61271f67838fefe132b7e14f701a279314b1b20c799f90c277bfcc61cf1: Status 404 returned error can't find the container with id 0dd5d61271f67838fefe132b7e14f701a279314b1b20c799f90c277bfcc61cf1
Sep 17 17:34:44.045668 ip-10-0-143-74 kubenswrapper[2575]: W0917 17:34:44.045645    2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod83786a21_e36e_4e15_82ec_c28223ef752b.slice/crio-f480f17e1cae6ef28990543a4bbec604a973e2f254a6c72adf027ef8c509bee5 WatchSource:0}: Error finding container f480f17e1cae6ef28990543a4bbec604a973e2f254a6c72adf027ef8c509bee5: Status 404 returned error can't find the container with id f480f17e1cae6ef28990543a4bbec604a973e2f254a6c72adf027ef8c509bee5
Sep 17 17:34:44.360786 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:34:44.360744    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/7892b408-d2f1-4e39-9bf6-e68a452196b1-original-pull-secret\") pod \"global-pull-secret-syncer-9s5wl\" (UID: \"7892b408-d2f1-4e39-9bf6-e68a452196b1\") " pod="kube-system/global-pull-secret-syncer-9s5wl"
Sep 17 17:34:44.360974 ip-10-0-143-74 kubenswrapper[2575]: E0917 17:34:44.360923    2575 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Sep 17 17:34:44.361038 ip-10-0-143-74 kubenswrapper[2575]: E0917 17:34:44.361004    2575 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/7892b408-d2f1-4e39-9bf6-e68a452196b1-original-pull-secret podName:7892b408-d2f1-4e39-9bf6-e68a452196b1 nodeName:}" failed. No retries permitted until 2026-09-17 17:34:46.360984448 +0000 UTC m=+5.991314418 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/7892b408-d2f1-4e39-9bf6-e68a452196b1-original-pull-secret") pod "global-pull-secret-syncer-9s5wl" (UID: "7892b408-d2f1-4e39-9bf6-e68a452196b1") : object "kube-system"/"original-pull-secret" not registered
Sep 17 17:34:44.461734 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:34:44.461693    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/27866e17-c231-49a6-921f-a83a01b8e173-metrics-certs\") pod \"network-metrics-daemon-wdp4h\" (UID: \"27866e17-c231-49a6-921f-a83a01b8e173\") " pod="openshift-multus/network-metrics-daemon-wdp4h"
Sep 17 17:34:44.461922 ip-10-0-143-74 kubenswrapper[2575]: E0917 17:34:44.461910    2575 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 17 17:34:44.461992 ip-10-0-143-74 kubenswrapper[2575]: E0917 17:34:44.461976    2575 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/27866e17-c231-49a6-921f-a83a01b8e173-metrics-certs podName:27866e17-c231-49a6-921f-a83a01b8e173 nodeName:}" failed. No retries permitted until 2026-09-17 17:34:46.461958383 +0000 UTC m=+6.092288342 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/27866e17-c231-49a6-921f-a83a01b8e173-metrics-certs") pod "network-metrics-daemon-wdp4h" (UID: "27866e17-c231-49a6-921f-a83a01b8e173") : object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 17 17:34:44.562887 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:34:44.562799    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7rzlr\" (UniqueName: \"kubernetes.io/projected/57bf735f-0367-4799-a1a8-889870d65704-kube-api-access-7rzlr\") pod \"network-check-target-pxt82\" (UID: \"57bf735f-0367-4799-a1a8-889870d65704\") " pod="openshift-network-diagnostics/network-check-target-pxt82"
Sep 17 17:34:44.563047 ip-10-0-143-74 kubenswrapper[2575]: E0917 17:34:44.563018    2575 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Sep 17 17:34:44.563047 ip-10-0-143-74 kubenswrapper[2575]: E0917 17:34:44.563041    2575 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Sep 17 17:34:44.563178 ip-10-0-143-74 kubenswrapper[2575]: E0917 17:34:44.563054    2575 projected.go:196] Error preparing data for projected volume kube-api-access-7rzlr for pod openshift-network-diagnostics/network-check-target-pxt82: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 17 17:34:44.563178 ip-10-0-143-74 kubenswrapper[2575]: E0917 17:34:44.563115    2575 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/57bf735f-0367-4799-a1a8-889870d65704-kube-api-access-7rzlr podName:57bf735f-0367-4799-a1a8-889870d65704 nodeName:}" failed. No retries permitted until 2026-09-17 17:34:46.563097038 +0000 UTC m=+6.193427015 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-api-access-7rzlr" (UniqueName: "kubernetes.io/projected/57bf735f-0367-4799-a1a8-889870d65704-kube-api-access-7rzlr") pod "network-check-target-pxt82" (UID: "57bf735f-0367-4799-a1a8-889870d65704") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 17 17:34:44.791651 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:34:44.791570    2575 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-09-16 17:29:43 +0000 UTC" deadline="2028-06-16 19:51:07.799548298 +0000 UTC"
Sep 17 17:34:44.791651 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:34:44.791614    2575 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="15314h16m23.00793984s"
Sep 17 17:34:44.964768 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:34:44.964074    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-9s5wl"
Sep 17 17:34:44.964768 ip-10-0-143-74 kubenswrapper[2575]: E0917 17:34:44.964317    2575 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-9s5wl" podUID="7892b408-d2f1-4e39-9bf6-e68a452196b1"
Sep 17 17:34:44.964768 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:34:44.964723    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-pxt82"
Sep 17 17:34:44.965056 ip-10-0-143-74 kubenswrapper[2575]: E0917 17:34:44.964842    2575 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-pxt82" podUID="57bf735f-0367-4799-a1a8-889870d65704"
Sep 17 17:34:44.989424 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:34:44.989386    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-srdtd" event={"ID":"fda64577-6753-498e-ab9e-128a567424ca","Type":"ContainerStarted","Data":"0dd5d61271f67838fefe132b7e14f701a279314b1b20c799f90c277bfcc61cf1"}
Sep 17 17:34:45.005377 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:34:45.005333    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-gtxcq" event={"ID":"d52f9540-79e9-4df7-b3a9-e8c7cc6d48b0","Type":"ContainerStarted","Data":"b800f47fedc9b3a14910d4c26df858806484d69f3ff163854f39ef913788ef4b"}
Sep 17 17:34:45.033482 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:34:45.033421    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-lkwb5" event={"ID":"2ccbe31a-b945-423d-a88a-45ef25012451","Type":"ContainerStarted","Data":"ce4f411b29db8ebdf3c0135be2df295e4395dd5bd95936718c6f1b1c3d8fd6db"}
Sep 17 17:34:45.056225 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:34:45.056184    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-5549b" event={"ID":"93fd2156-b185-4995-8e49-d6d9c0eda255","Type":"ContainerStarted","Data":"da87d3ab7ca776cc526236cdeb2019804b7d607707e5cefdc7e935d71e43a4f5"}
Sep 17 17:34:45.077777 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:34:45.077735    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-143-74.ec2.internal" event={"ID":"d937ab292439efa4b2d5e0f9e6f432b8","Type":"ContainerStarted","Data":"01e46105755fff6af5d80225ad8e06e62964f154b729ad47bbe82578dae2f1ea"}
Sep 17 17:34:45.089695 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:34:45.089660    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-2fp6z" event={"ID":"173c8902-414c-4212-8d93-ce15853e8ca8","Type":"ContainerStarted","Data":"1c8caf83d6c8511e0fbee2ba0c94c3f7ec04850d34ecdad45c3e8425332fdd1d"}
Sep 17 17:34:45.104059 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:34:45.101682    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-n92x5" event={"ID":"83786a21-e36e-4e15-82ec-c28223ef752b","Type":"ContainerStarted","Data":"f480f17e1cae6ef28990543a4bbec604a973e2f254a6c72adf027ef8c509bee5"}
Sep 17 17:34:45.104059 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:34:45.103722    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-djvzt" event={"ID":"1d07e68d-9943-420f-a6a0-5f477792fabd","Type":"ContainerStarted","Data":"170c9704728107821d5d560436f7b07ccc5b9234877b98638b1bbb8a92cc58b8"}
Sep 17 17:34:45.111620 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:34:45.111526    2575 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/kube-apiserver-proxy-ip-10-0-143-74.ec2.internal" podStartSLOduration=3.111506129 podStartE2EDuration="3.111506129s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-17 17:34:42 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-17 17:34:45.099824986 +0000 UTC m=+4.730154957" watchObservedRunningTime="2026-09-17 17:34:45.111506129 +0000 UTC m=+4.741836112"
Sep 17 17:34:45.112642 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:34:45.112613    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-gtwrb" event={"ID":"3cb242c8-4f5d-49a3-bb24-85a2170a979e","Type":"ContainerStarted","Data":"fd21941945dfca4f5f40f3896bc6369d6f227f6dd1607f2c4c269a7c51ba6e7b"}
Sep 17 17:34:45.119362 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:34:45.119333    2575 generic.go:356] "Generic (PLEG): container finished" podID="50d07cdbe5bdbd290f2ca595730f87af" containerID="7d5d24c9d3c413a53db5e9be779114e68d6e2157df0d4d8b6552837e684c9faf" exitCode=0
Sep 17 17:34:45.119485 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:34:45.119413    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-143-74.ec2.internal" event={"ID":"50d07cdbe5bdbd290f2ca595730f87af","Type":"ContainerDied","Data":"7d5d24c9d3c413a53db5e9be779114e68d6e2157df0d4d8b6552837e684c9faf"}
Sep 17 17:34:45.127803 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:34:45.127777    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-zvjts" event={"ID":"d0f23713-30dc-4d47-95cb-c083bec7c18c","Type":"ContainerStarted","Data":"8486e83726bfdc9ca757c7f2bc47acb36bb4b2a7ec61caa3bfe145b942d7b2ea"}
Sep 17 17:34:45.961051 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:34:45.960962    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-wdp4h"
Sep 17 17:34:45.961737 ip-10-0-143-74 kubenswrapper[2575]: E0917 17:34:45.961121    2575 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-wdp4h" podUID="27866e17-c231-49a6-921f-a83a01b8e173"
Sep 17 17:34:46.149973 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:34:46.149888    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-143-74.ec2.internal" event={"ID":"50d07cdbe5bdbd290f2ca595730f87af","Type":"ContainerStarted","Data":"f7722114ee0037578615d01e08683ddeea2cdb88c2e7ac40a25271bec051cedb"}
Sep 17 17:34:46.381050 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:34:46.380948    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/7892b408-d2f1-4e39-9bf6-e68a452196b1-original-pull-secret\") pod \"global-pull-secret-syncer-9s5wl\" (UID: \"7892b408-d2f1-4e39-9bf6-e68a452196b1\") " pod="kube-system/global-pull-secret-syncer-9s5wl"
Sep 17 17:34:46.381235 ip-10-0-143-74 kubenswrapper[2575]: E0917 17:34:46.381100    2575 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Sep 17 17:34:46.381235 ip-10-0-143-74 kubenswrapper[2575]: E0917 17:34:46.381166    2575 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/7892b408-d2f1-4e39-9bf6-e68a452196b1-original-pull-secret podName:7892b408-d2f1-4e39-9bf6-e68a452196b1 nodeName:}" failed. No retries permitted until 2026-09-17 17:34:50.381148181 +0000 UTC m=+10.011478141 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/7892b408-d2f1-4e39-9bf6-e68a452196b1-original-pull-secret") pod "global-pull-secret-syncer-9s5wl" (UID: "7892b408-d2f1-4e39-9bf6-e68a452196b1") : object "kube-system"/"original-pull-secret" not registered
Sep 17 17:34:46.483559 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:34:46.481910    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/27866e17-c231-49a6-921f-a83a01b8e173-metrics-certs\") pod \"network-metrics-daemon-wdp4h\" (UID: \"27866e17-c231-49a6-921f-a83a01b8e173\") " pod="openshift-multus/network-metrics-daemon-wdp4h"
Sep 17 17:34:46.483559 ip-10-0-143-74 kubenswrapper[2575]: E0917 17:34:46.482073    2575 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 17 17:34:46.483559 ip-10-0-143-74 kubenswrapper[2575]: E0917 17:34:46.482137    2575 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/27866e17-c231-49a6-921f-a83a01b8e173-metrics-certs podName:27866e17-c231-49a6-921f-a83a01b8e173 nodeName:}" failed. No retries permitted until 2026-09-17 17:34:50.482118482 +0000 UTC m=+10.112448443 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/27866e17-c231-49a6-921f-a83a01b8e173-metrics-certs") pod "network-metrics-daemon-wdp4h" (UID: "27866e17-c231-49a6-921f-a83a01b8e173") : object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 17 17:34:46.583022 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:34:46.582335    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7rzlr\" (UniqueName: \"kubernetes.io/projected/57bf735f-0367-4799-a1a8-889870d65704-kube-api-access-7rzlr\") pod \"network-check-target-pxt82\" (UID: \"57bf735f-0367-4799-a1a8-889870d65704\") " pod="openshift-network-diagnostics/network-check-target-pxt82"
Sep 17 17:34:46.583022 ip-10-0-143-74 kubenswrapper[2575]: E0917 17:34:46.582542    2575 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Sep 17 17:34:46.583022 ip-10-0-143-74 kubenswrapper[2575]: E0917 17:34:46.582565    2575 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Sep 17 17:34:46.583022 ip-10-0-143-74 kubenswrapper[2575]: E0917 17:34:46.582578    2575 projected.go:196] Error preparing data for projected volume kube-api-access-7rzlr for pod openshift-network-diagnostics/network-check-target-pxt82: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 17 17:34:46.583022 ip-10-0-143-74 kubenswrapper[2575]: E0917 17:34:46.582636    2575 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/57bf735f-0367-4799-a1a8-889870d65704-kube-api-access-7rzlr podName:57bf735f-0367-4799-a1a8-889870d65704 nodeName:}" failed. No retries permitted until 2026-09-17 17:34:50.582618847 +0000 UTC m=+10.212948806 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-api-access-7rzlr" (UniqueName: "kubernetes.io/projected/57bf735f-0367-4799-a1a8-889870d65704-kube-api-access-7rzlr") pod "network-check-target-pxt82" (UID: "57bf735f-0367-4799-a1a8-889870d65704") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 17 17:34:46.963469 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:34:46.963434    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-9s5wl"
Sep 17 17:34:46.963967 ip-10-0-143-74 kubenswrapper[2575]: E0917 17:34:46.963594    2575 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-9s5wl" podUID="7892b408-d2f1-4e39-9bf6-e68a452196b1"
Sep 17 17:34:46.964037 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:34:46.964020    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-pxt82"
Sep 17 17:34:46.964159 ip-10-0-143-74 kubenswrapper[2575]: E0917 17:34:46.964112    2575 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-pxt82" podUID="57bf735f-0367-4799-a1a8-889870d65704"
Sep 17 17:34:47.961453 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:34:47.961408    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-wdp4h"
Sep 17 17:34:47.961668 ip-10-0-143-74 kubenswrapper[2575]: E0917 17:34:47.961603    2575 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-wdp4h" podUID="27866e17-c231-49a6-921f-a83a01b8e173"
Sep 17 17:34:48.961785 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:34:48.961380    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-9s5wl"
Sep 17 17:34:48.961785 ip-10-0-143-74 kubenswrapper[2575]: E0917 17:34:48.961532    2575 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-9s5wl" podUID="7892b408-d2f1-4e39-9bf6-e68a452196b1"
Sep 17 17:34:48.962369 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:34:48.961922    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-pxt82"
Sep 17 17:34:48.962369 ip-10-0-143-74 kubenswrapper[2575]: E0917 17:34:48.962074    2575 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-pxt82" podUID="57bf735f-0367-4799-a1a8-889870d65704"
Sep 17 17:34:49.961481 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:34:49.960871    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-wdp4h"
Sep 17 17:34:49.961481 ip-10-0-143-74 kubenswrapper[2575]: E0917 17:34:49.961016    2575 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-wdp4h" podUID="27866e17-c231-49a6-921f-a83a01b8e173"
Sep 17 17:34:50.417238 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:34:50.417153    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/7892b408-d2f1-4e39-9bf6-e68a452196b1-original-pull-secret\") pod \"global-pull-secret-syncer-9s5wl\" (UID: \"7892b408-d2f1-4e39-9bf6-e68a452196b1\") " pod="kube-system/global-pull-secret-syncer-9s5wl"
Sep 17 17:34:50.417690 ip-10-0-143-74 kubenswrapper[2575]: E0917 17:34:50.417308    2575 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Sep 17 17:34:50.417690 ip-10-0-143-74 kubenswrapper[2575]: E0917 17:34:50.417378    2575 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/7892b408-d2f1-4e39-9bf6-e68a452196b1-original-pull-secret podName:7892b408-d2f1-4e39-9bf6-e68a452196b1 nodeName:}" failed. No retries permitted until 2026-09-17 17:34:58.417356895 +0000 UTC m=+18.047686866 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/7892b408-d2f1-4e39-9bf6-e68a452196b1-original-pull-secret") pod "global-pull-secret-syncer-9s5wl" (UID: "7892b408-d2f1-4e39-9bf6-e68a452196b1") : object "kube-system"/"original-pull-secret" not registered
Sep 17 17:34:50.518707 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:34:50.518380    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/27866e17-c231-49a6-921f-a83a01b8e173-metrics-certs\") pod \"network-metrics-daemon-wdp4h\" (UID: \"27866e17-c231-49a6-921f-a83a01b8e173\") " pod="openshift-multus/network-metrics-daemon-wdp4h"
Sep 17 17:34:50.518707 ip-10-0-143-74 kubenswrapper[2575]: E0917 17:34:50.518539    2575 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 17 17:34:50.518707 ip-10-0-143-74 kubenswrapper[2575]: E0917 17:34:50.518602    2575 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/27866e17-c231-49a6-921f-a83a01b8e173-metrics-certs podName:27866e17-c231-49a6-921f-a83a01b8e173 nodeName:}" failed. No retries permitted until 2026-09-17 17:34:58.518582869 +0000 UTC m=+18.148912827 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/27866e17-c231-49a6-921f-a83a01b8e173-metrics-certs") pod "network-metrics-daemon-wdp4h" (UID: "27866e17-c231-49a6-921f-a83a01b8e173") : object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 17 17:34:50.619034 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:34:50.618996    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7rzlr\" (UniqueName: \"kubernetes.io/projected/57bf735f-0367-4799-a1a8-889870d65704-kube-api-access-7rzlr\") pod \"network-check-target-pxt82\" (UID: \"57bf735f-0367-4799-a1a8-889870d65704\") " pod="openshift-network-diagnostics/network-check-target-pxt82"
Sep 17 17:34:50.619217 ip-10-0-143-74 kubenswrapper[2575]: E0917 17:34:50.619188    2575 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Sep 17 17:34:50.619217 ip-10-0-143-74 kubenswrapper[2575]: E0917 17:34:50.619210    2575 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Sep 17 17:34:50.619314 ip-10-0-143-74 kubenswrapper[2575]: E0917 17:34:50.619221    2575 projected.go:196] Error preparing data for projected volume kube-api-access-7rzlr for pod openshift-network-diagnostics/network-check-target-pxt82: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 17 17:34:50.619314 ip-10-0-143-74 kubenswrapper[2575]: E0917 17:34:50.619278    2575 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/57bf735f-0367-4799-a1a8-889870d65704-kube-api-access-7rzlr podName:57bf735f-0367-4799-a1a8-889870d65704 nodeName:}" failed. No retries permitted until 2026-09-17 17:34:58.6192618 +0000 UTC m=+18.249591773 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "kube-api-access-7rzlr" (UniqueName: "kubernetes.io/projected/57bf735f-0367-4799-a1a8-889870d65704-kube-api-access-7rzlr") pod "network-check-target-pxt82" (UID: "57bf735f-0367-4799-a1a8-889870d65704") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 17 17:34:50.962806 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:34:50.962233    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-9s5wl"
Sep 17 17:34:50.962806 ip-10-0-143-74 kubenswrapper[2575]: E0917 17:34:50.962359    2575 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-9s5wl" podUID="7892b408-d2f1-4e39-9bf6-e68a452196b1"
Sep 17 17:34:50.962806 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:34:50.962405    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-pxt82"
Sep 17 17:34:50.962806 ip-10-0-143-74 kubenswrapper[2575]: E0917 17:34:50.962492    2575 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-pxt82" podUID="57bf735f-0367-4799-a1a8-889870d65704"
Sep 17 17:34:51.961367 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:34:51.961292    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-wdp4h"
Sep 17 17:34:51.961892 ip-10-0-143-74 kubenswrapper[2575]: E0917 17:34:51.961472    2575 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-wdp4h" podUID="27866e17-c231-49a6-921f-a83a01b8e173"
Sep 17 17:34:52.961663 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:34:52.961615    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-9s5wl"
Sep 17 17:34:52.962120 ip-10-0-143-74 kubenswrapper[2575]: E0917 17:34:52.961756    2575 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-9s5wl" podUID="7892b408-d2f1-4e39-9bf6-e68a452196b1"
Sep 17 17:34:52.962120 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:34:52.961886    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-pxt82"
Sep 17 17:34:52.962120 ip-10-0-143-74 kubenswrapper[2575]: E0917 17:34:52.961994    2575 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-pxt82" podUID="57bf735f-0367-4799-a1a8-889870d65704"
Sep 17 17:34:53.961479 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:34:53.961433    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-wdp4h"
Sep 17 17:34:53.961632 ip-10-0-143-74 kubenswrapper[2575]: E0917 17:34:53.961563    2575 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-wdp4h" podUID="27866e17-c231-49a6-921f-a83a01b8e173"
Sep 17 17:34:54.961767 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:34:54.961735    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-pxt82"
Sep 17 17:34:54.962117 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:34:54.961735    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-9s5wl"
Sep 17 17:34:54.962117 ip-10-0-143-74 kubenswrapper[2575]: E0917 17:34:54.961840    2575 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-pxt82" podUID="57bf735f-0367-4799-a1a8-889870d65704"
Sep 17 17:34:54.962117 ip-10-0-143-74 kubenswrapper[2575]: E0917 17:34:54.961914    2575 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-9s5wl" podUID="7892b408-d2f1-4e39-9bf6-e68a452196b1"
Sep 17 17:34:55.961366 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:34:55.961335    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-wdp4h"
Sep 17 17:34:55.961548 ip-10-0-143-74 kubenswrapper[2575]: E0917 17:34:55.961480    2575 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-wdp4h" podUID="27866e17-c231-49a6-921f-a83a01b8e173"
Sep 17 17:34:56.961031 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:34:56.960996    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-pxt82"
Sep 17 17:34:56.961526 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:34:56.960996    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-9s5wl"
Sep 17 17:34:56.961526 ip-10-0-143-74 kubenswrapper[2575]: E0917 17:34:56.961128    2575 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-pxt82" podUID="57bf735f-0367-4799-a1a8-889870d65704"
Sep 17 17:34:56.961526 ip-10-0-143-74 kubenswrapper[2575]: E0917 17:34:56.961225    2575 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-9s5wl" podUID="7892b408-d2f1-4e39-9bf6-e68a452196b1"
Sep 17 17:34:57.961600 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:34:57.961562    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-wdp4h"
Sep 17 17:34:57.962082 ip-10-0-143-74 kubenswrapper[2575]: E0917 17:34:57.961703    2575 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-wdp4h" podUID="27866e17-c231-49a6-921f-a83a01b8e173"
Sep 17 17:34:58.474087 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:34:58.474045    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/7892b408-d2f1-4e39-9bf6-e68a452196b1-original-pull-secret\") pod \"global-pull-secret-syncer-9s5wl\" (UID: \"7892b408-d2f1-4e39-9bf6-e68a452196b1\") " pod="kube-system/global-pull-secret-syncer-9s5wl"
Sep 17 17:34:58.474286 ip-10-0-143-74 kubenswrapper[2575]: E0917 17:34:58.474139    2575 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Sep 17 17:34:58.474286 ip-10-0-143-74 kubenswrapper[2575]: E0917 17:34:58.474207    2575 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/7892b408-d2f1-4e39-9bf6-e68a452196b1-original-pull-secret podName:7892b408-d2f1-4e39-9bf6-e68a452196b1 nodeName:}" failed. No retries permitted until 2026-09-17 17:35:14.474192302 +0000 UTC m=+34.104522255 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/7892b408-d2f1-4e39-9bf6-e68a452196b1-original-pull-secret") pod "global-pull-secret-syncer-9s5wl" (UID: "7892b408-d2f1-4e39-9bf6-e68a452196b1") : object "kube-system"/"original-pull-secret" not registered
Sep 17 17:34:58.575237 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:34:58.575186    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/27866e17-c231-49a6-921f-a83a01b8e173-metrics-certs\") pod \"network-metrics-daemon-wdp4h\" (UID: \"27866e17-c231-49a6-921f-a83a01b8e173\") " pod="openshift-multus/network-metrics-daemon-wdp4h"
Sep 17 17:34:58.575393 ip-10-0-143-74 kubenswrapper[2575]: E0917 17:34:58.575312    2575 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 17 17:34:58.575393 ip-10-0-143-74 kubenswrapper[2575]: E0917 17:34:58.575367    2575 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/27866e17-c231-49a6-921f-a83a01b8e173-metrics-certs podName:27866e17-c231-49a6-921f-a83a01b8e173 nodeName:}" failed. No retries permitted until 2026-09-17 17:35:14.575353172 +0000 UTC m=+34.205683138 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/27866e17-c231-49a6-921f-a83a01b8e173-metrics-certs") pod "network-metrics-daemon-wdp4h" (UID: "27866e17-c231-49a6-921f-a83a01b8e173") : object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 17 17:34:58.676582 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:34:58.676552    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7rzlr\" (UniqueName: \"kubernetes.io/projected/57bf735f-0367-4799-a1a8-889870d65704-kube-api-access-7rzlr\") pod \"network-check-target-pxt82\" (UID: \"57bf735f-0367-4799-a1a8-889870d65704\") " pod="openshift-network-diagnostics/network-check-target-pxt82"
Sep 17 17:34:58.676798 ip-10-0-143-74 kubenswrapper[2575]: E0917 17:34:58.676769    2575 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Sep 17 17:34:58.676798 ip-10-0-143-74 kubenswrapper[2575]: E0917 17:34:58.676801    2575 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Sep 17 17:34:58.676936 ip-10-0-143-74 kubenswrapper[2575]: E0917 17:34:58.676815    2575 projected.go:196] Error preparing data for projected volume kube-api-access-7rzlr for pod openshift-network-diagnostics/network-check-target-pxt82: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 17 17:34:58.676936 ip-10-0-143-74 kubenswrapper[2575]: E0917 17:34:58.676879    2575 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/57bf735f-0367-4799-a1a8-889870d65704-kube-api-access-7rzlr podName:57bf735f-0367-4799-a1a8-889870d65704 nodeName:}" failed. No retries permitted until 2026-09-17 17:35:14.676860312 +0000 UTC m=+34.307190273 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "kube-api-access-7rzlr" (UniqueName: "kubernetes.io/projected/57bf735f-0367-4799-a1a8-889870d65704-kube-api-access-7rzlr") pod "network-check-target-pxt82" (UID: "57bf735f-0367-4799-a1a8-889870d65704") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 17 17:34:58.961315 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:34:58.961271    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-9s5wl"
Sep 17 17:34:58.961496 ip-10-0-143-74 kubenswrapper[2575]: E0917 17:34:58.961413    2575 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-9s5wl" podUID="7892b408-d2f1-4e39-9bf6-e68a452196b1"
Sep 17 17:34:58.961496 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:34:58.961459    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-pxt82"
Sep 17 17:34:58.961602 ip-10-0-143-74 kubenswrapper[2575]: E0917 17:34:58.961561    2575 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-pxt82" podUID="57bf735f-0367-4799-a1a8-889870d65704"
Sep 17 17:34:59.961315 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:34:59.961276    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-wdp4h"
Sep 17 17:34:59.961783 ip-10-0-143-74 kubenswrapper[2575]: E0917 17:34:59.961414    2575 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-wdp4h" podUID="27866e17-c231-49a6-921f-a83a01b8e173"
Sep 17 17:35:00.962238 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:35:00.962205    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-9s5wl"
Sep 17 17:35:00.962696 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:35:00.962307    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-pxt82"
Sep 17 17:35:00.962696 ip-10-0-143-74 kubenswrapper[2575]: E0917 17:35:00.962350    2575 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-9s5wl" podUID="7892b408-d2f1-4e39-9bf6-e68a452196b1"
Sep 17 17:35:00.962696 ip-10-0-143-74 kubenswrapper[2575]: E0917 17:35:00.962380    2575 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-pxt82" podUID="57bf735f-0367-4799-a1a8-889870d65704"
Sep 17 17:35:01.961133 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:35:01.961096    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-wdp4h"
Sep 17 17:35:01.961319 ip-10-0-143-74 kubenswrapper[2575]: E0917 17:35:01.961217    2575 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-wdp4h" podUID="27866e17-c231-49a6-921f-a83a01b8e173"
Sep 17 17:35:02.961923 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:35:02.961755    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-9s5wl"
Sep 17 17:35:02.962826 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:35:02.961824    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-pxt82"
Sep 17 17:35:02.962826 ip-10-0-143-74 kubenswrapper[2575]: E0917 17:35:02.962029    2575 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-9s5wl" podUID="7892b408-d2f1-4e39-9bf6-e68a452196b1"
Sep 17 17:35:02.962826 ip-10-0-143-74 kubenswrapper[2575]: E0917 17:35:02.962092    2575 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-pxt82" podUID="57bf735f-0367-4799-a1a8-889870d65704"
Sep 17 17:35:03.198039 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:35:03.198003    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-2fp6z" event={"ID":"173c8902-414c-4212-8d93-ce15853e8ca8","Type":"ContainerStarted","Data":"29a00a3a5581608162e148200c4a284aa88ed5e89a4d597d1727e57f1355aada"}
Sep 17 17:35:03.199539 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:35:03.199510    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-djvzt" event={"ID":"1d07e68d-9943-420f-a6a0-5f477792fabd","Type":"ContainerStarted","Data":"e3d3630cf155cd04bf02ce9764858a37d1ed793ffc48f9da3d61f73fa2690ac7"}
Sep 17 17:35:03.200884 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:35:03.200864    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-gtwrb" event={"ID":"3cb242c8-4f5d-49a3-bb24-85a2170a979e","Type":"ContainerStarted","Data":"2ae9f626455ec889e0aa86c8e5c2a04a9181634ab75278aa52d611beed9659bb"}
Sep 17 17:35:03.202289 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:35:03.202271    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-zvjts" event={"ID":"d0f23713-30dc-4d47-95cb-c083bec7c18c","Type":"ContainerStarted","Data":"33046ff6655894359a80bfd1b89f2b5621966578e4b8efe980f54b3fc640d2a6"}
Sep 17 17:35:03.203669 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:35:03.203649    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-srdtd" event={"ID":"fda64577-6753-498e-ab9e-128a567424ca","Type":"ContainerStarted","Data":"f9172297b9eb145e8e3cde06e911b1bc054cd16853304e84df79f29a2aa500eb"}
Sep 17 17:35:03.205053 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:35:03.205034    2575 generic.go:356] "Generic (PLEG): container finished" podID="d52f9540-79e9-4df7-b3a9-e8c7cc6d48b0" containerID="5fa5eecb1eb6203fe6605819d717b4871234e35e7f7a70b94eed8792f1036ab5" exitCode=0
Sep 17 17:35:03.205148 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:35:03.205091    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-gtxcq" event={"ID":"d52f9540-79e9-4df7-b3a9-e8c7cc6d48b0","Type":"ContainerDied","Data":"5fa5eecb1eb6203fe6605819d717b4871234e35e7f7a70b94eed8792f1036ab5"}
Sep 17 17:35:03.208247 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:35:03.208229    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-lkwb5" event={"ID":"2ccbe31a-b945-423d-a88a-45ef25012451","Type":"ContainerStarted","Data":"fb4e2c7244086409861cc4e307af23d33913caed28cb5e8557282cbef3fcf7bc"}
Sep 17 17:35:03.208324 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:35:03.208293    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-lkwb5" event={"ID":"2ccbe31a-b945-423d-a88a-45ef25012451","Type":"ContainerStarted","Data":"e09f170d412b10d44ccea97638d25f79b76d6408f3d5a3ae992f0ccc1b152137"}
Sep 17 17:35:03.208375 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:35:03.208345    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-lkwb5" event={"ID":"2ccbe31a-b945-423d-a88a-45ef25012451","Type":"ContainerStarted","Data":"424a0cb104ad5b256afa1da7bd133012d0f294b23be27b655842fe8f6efaf1e7"}
Sep 17 17:35:03.208412 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:35:03.208393    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-lkwb5" event={"ID":"2ccbe31a-b945-423d-a88a-45ef25012451","Type":"ContainerStarted","Data":"fa329cc56192a3dfda524720621522604265e495d3f19c6c046f11daafda698a"}
Sep 17 17:35:03.208513 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:35:03.208488    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-lkwb5" event={"ID":"2ccbe31a-b945-423d-a88a-45ef25012451","Type":"ContainerStarted","Data":"de3273019b455be395839034a48dabb67a5c8e96805ff834bab05d5a47f5734d"}
Sep 17 17:35:03.208631 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:35:03.208544    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-lkwb5" event={"ID":"2ccbe31a-b945-423d-a88a-45ef25012451","Type":"ContainerStarted","Data":"4a385ffe53aa2c0b97601317dac47032a809b8b475e03fc84f31566f8b9f7ad9"}
Sep 17 17:35:03.209694 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:35:03.209679    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-5549b" event={"ID":"93fd2156-b185-4995-8e49-d6d9c0eda255","Type":"ContainerStarted","Data":"9392103c3167b5c22e4d50f3d357d58cfd43f52977e4257e96b1be6374262e70"}
Sep 17 17:35:03.217070 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:35:03.217035    2575 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/node-ca-2fp6z" podStartSLOduration=4.188310774 podStartE2EDuration="22.217023746s" totalImagesPullingTime="18.028712972s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-17 17:34:41 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-17 17:35:03.216793283 +0000 UTC m=+22.847123258" watchObservedRunningTime="2026-09-17 17:35:03.217023746 +0000 UTC m=+22.847353721"
Sep 17 17:35:03.217161 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:35:03.217097    2575 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-143-74.ec2.internal" podStartSLOduration=21.217093218 podStartE2EDuration="21.217093218s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-17 17:34:42 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-17 17:34:46.169574475 +0000 UTC m=+5.799904445" watchObservedRunningTime="2026-09-17 17:35:03.217093218 +0000 UTC m=+22.847423192"
Sep 17 17:35:03.258180 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:35:03.258128    2575 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-srdtd" podStartSLOduration=4.173463089 podStartE2EDuration="22.258114033s" totalImagesPullingTime="18.084650944s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-17 17:34:41 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-17 17:35:03.239723537 +0000 UTC m=+22.870053509" watchObservedRunningTime="2026-09-17 17:35:03.258114033 +0000 UTC m=+22.888444009"
Sep 17 17:35:03.258489 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:35:03.258462    2575 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/node-resolver-5549b" podStartSLOduration=4.220800335 podStartE2EDuration="22.258431859s" totalImagesPullingTime="18.037631524s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-17 17:34:41 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-17 17:35:03.257969387 +0000 UTC m=+22.888299362" watchObservedRunningTime="2026-09-17 17:35:03.258431859 +0000 UTC m=+22.888761835"
Sep 17 17:35:03.275711 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:35:03.275665    2575 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/konnectivity-agent-djvzt" podStartSLOduration=7.569237277 podStartE2EDuration="22.275652439s" totalImagesPullingTime="14.706415162s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-17 17:34:41 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-17 17:35:03.275479072 +0000 UTC m=+22.905809048" watchObservedRunningTime="2026-09-17 17:35:03.275652439 +0000 UTC m=+22.905982414"
Sep 17 17:35:03.295086 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:35:03.295044    2575 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-node-tuning-operator/tuned-gtwrb" podStartSLOduration=4.224168704 podStartE2EDuration="22.295032429s" totalImagesPullingTime="18.070863725s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-17 17:34:41 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-17 17:35:03.29463648 +0000 UTC m=+22.924966455" watchObservedRunningTime="2026-09-17 17:35:03.295032429 +0000 UTC m=+22.925362401"
Sep 17 17:35:03.485676 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:35:03.485651    2575 plugin_watcher.go:194] "Adding socket path or updating timestamp to desired state cache" path="/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock"
Sep 17 17:35:03.948937 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:35:03.948812    2575 reconciler.go:161] "OperationExecutor.RegisterPlugin started" plugin={"SocketPath":"/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock","Timestamp":"2026-09-17T17:35:03.48567392Z","UUID":"14e08092-e080-49ae-aea9-48867ccb7750","Handler":null,"Name":"","Endpoint":""}
Sep 17 17:35:03.950541 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:35:03.950523    2575 csi_plugin.go:106] kubernetes.io/csi: Trying to validate a new CSI Driver with name: ebs.csi.aws.com endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock versions: 1.0.0
Sep 17 17:35:03.950660 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:35:03.950552    2575 csi_plugin.go:119] kubernetes.io/csi: Register new plugin with name: ebs.csi.aws.com at endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock
Sep 17 17:35:03.961606 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:35:03.961579    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-wdp4h"
Sep 17 17:35:03.961725 ip-10-0-143-74 kubenswrapper[2575]: E0917 17:35:03.961706    2575 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-wdp4h" podUID="27866e17-c231-49a6-921f-a83a01b8e173"
Sep 17 17:35:03.998700 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:35:03.998657    2575 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="kube-system/konnectivity-agent-djvzt"
Sep 17 17:35:03.999405 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:35:03.999384    2575 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="kube-system/konnectivity-agent-djvzt"
Sep 17 17:35:04.214322 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:35:04.214237    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-n92x5" event={"ID":"83786a21-e36e-4e15-82ec-c28223ef752b","Type":"ContainerStarted","Data":"89addb882104933fcd00806df9d6885cefe865cd6ab66c8ff8247a6424ac667f"}
Sep 17 17:35:04.216485 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:35:04.216431    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-zvjts" event={"ID":"d0f23713-30dc-4d47-95cb-c083bec7c18c","Type":"ContainerStarted","Data":"150c287f25328194c34a02737f8a8f51330f4b3e4363cdb3606dcaae4130a819"}
Sep 17 17:35:04.217243 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:35:04.217217    2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kube-system/konnectivity-agent-djvzt"
Sep 17 17:35:04.217743 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:35:04.217718    2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kube-system/konnectivity-agent-djvzt"
Sep 17 17:35:04.233250 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:35:04.233193    2575 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-operator/iptables-alerter-n92x5" podStartSLOduration=5.203886299 podStartE2EDuration="23.23317391s" totalImagesPullingTime="18.029287611s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-17 17:34:41 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-17 17:35:04.232251187 +0000 UTC m=+23.862581162" watchObservedRunningTime="2026-09-17 17:35:04.23317391 +0000 UTC m=+23.863503883"
Sep 17 17:35:04.961259 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:35:04.961169    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-9s5wl"
Sep 17 17:35:04.961259 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:35:04.961189    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-pxt82"
Sep 17 17:35:04.961483 ip-10-0-143-74 kubenswrapper[2575]: E0917 17:35:04.961316    2575 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-9s5wl" podUID="7892b408-d2f1-4e39-9bf6-e68a452196b1"
Sep 17 17:35:04.961483 ip-10-0-143-74 kubenswrapper[2575]: E0917 17:35:04.961430    2575 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-pxt82" podUID="57bf735f-0367-4799-a1a8-889870d65704"
Sep 17 17:35:05.221036 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:35:05.220940    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-zvjts" event={"ID":"d0f23713-30dc-4d47-95cb-c083bec7c18c","Type":"ContainerStarted","Data":"6298355666be225637980650df764890e068ac813be9497a5b7c113ec8b3f49a"}
Sep 17 17:35:05.224859 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:35:05.224816    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-lkwb5" event={"ID":"2ccbe31a-b945-423d-a88a-45ef25012451","Type":"ContainerStarted","Data":"f7f1d836527f8df109568e82cd88ecd07a9c6b4ad04654ff875f8a5a983c21f1"}
Sep 17 17:35:05.251703 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:35:05.251652    2575 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-zvjts" podStartSLOduration=3.995182879 podStartE2EDuration="24.25163788s" totalImagesPullingTime="20.256455001s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-17 17:34:41 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-17 17:35:05.251498171 +0000 UTC m=+24.881828147" watchObservedRunningTime="2026-09-17 17:35:05.25163788 +0000 UTC m=+24.881967854"
Sep 17 17:35:05.961451 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:35:05.961404    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-wdp4h"
Sep 17 17:35:05.961637 ip-10-0-143-74 kubenswrapper[2575]: E0917 17:35:05.961552    2575 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-wdp4h" podUID="27866e17-c231-49a6-921f-a83a01b8e173"
Sep 17 17:35:06.961132 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:35:06.961099    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-9s5wl"
Sep 17 17:35:06.961596 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:35:06.961103    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-pxt82"
Sep 17 17:35:06.961596 ip-10-0-143-74 kubenswrapper[2575]: E0917 17:35:06.961240    2575 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-9s5wl" podUID="7892b408-d2f1-4e39-9bf6-e68a452196b1"
Sep 17 17:35:06.961596 ip-10-0-143-74 kubenswrapper[2575]: E0917 17:35:06.961310    2575 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-pxt82" podUID="57bf735f-0367-4799-a1a8-889870d65704"
Sep 17 17:35:07.961029 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:35:07.961003    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-wdp4h"
Sep 17 17:35:07.961160 ip-10-0-143-74 kubenswrapper[2575]: E0917 17:35:07.961109    2575 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-wdp4h" podUID="27866e17-c231-49a6-921f-a83a01b8e173"
Sep 17 17:35:08.233192 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:35:08.233111    2575 generic.go:356] "Generic (PLEG): container finished" podID="d52f9540-79e9-4df7-b3a9-e8c7cc6d48b0" containerID="16005643eac4dfb2904e44bde70d793acc965555d5283df11f41cb159092a6d7" exitCode=0
Sep 17 17:35:08.233334 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:35:08.233192    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-gtxcq" event={"ID":"d52f9540-79e9-4df7-b3a9-e8c7cc6d48b0","Type":"ContainerDied","Data":"16005643eac4dfb2904e44bde70d793acc965555d5283df11f41cb159092a6d7"}
Sep 17 17:35:08.241242 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:35:08.241218    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-lkwb5" event={"ID":"2ccbe31a-b945-423d-a88a-45ef25012451","Type":"ContainerStarted","Data":"19a8fb81057b2148b083e44fd06bc35578da906b27431163a05130de62091aed"}
Sep 17 17:35:08.241479 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:35:08.241457    2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-lkwb5"
Sep 17 17:35:08.257203 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:35:08.257182    2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-lkwb5"
Sep 17 17:35:08.285569 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:35:08.285530    2575 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ovn-kubernetes/ovnkube-node-lkwb5" podStartSLOduration=9.184640867 podStartE2EDuration="27.285517548s" totalImagesPullingTime="18.100876681s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-17 17:34:41 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-17 17:35:08.284504032 +0000 UTC m=+27.914834008" watchObservedRunningTime="2026-09-17 17:35:08.285517548 +0000 UTC m=+27.915847522"
Sep 17 17:35:08.961989 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:35:08.961848    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-pxt82"
Sep 17 17:35:08.962267 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:35:08.961843    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-9s5wl"
Sep 17 17:35:08.962267 ip-10-0-143-74 kubenswrapper[2575]: E0917 17:35:08.962076    2575 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-pxt82" podUID="57bf735f-0367-4799-a1a8-889870d65704"
Sep 17 17:35:08.962267 ip-10-0-143-74 kubenswrapper[2575]: E0917 17:35:08.962121    2575 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-9s5wl" podUID="7892b408-d2f1-4e39-9bf6-e68a452196b1"
Sep 17 17:35:09.136748 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:35:09.136719    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-9s5wl"]
Sep 17 17:35:09.139098 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:35:09.139051    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-wdp4h"]
Sep 17 17:35:09.139333 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:35:09.139272    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-wdp4h"
Sep 17 17:35:09.139525 ip-10-0-143-74 kubenswrapper[2575]: E0917 17:35:09.139475    2575 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-wdp4h" podUID="27866e17-c231-49a6-921f-a83a01b8e173"
Sep 17 17:35:09.139758 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:35:09.139698    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-pxt82"]
Sep 17 17:35:09.245602 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:35:09.245574    2575 generic.go:356] "Generic (PLEG): container finished" podID="d52f9540-79e9-4df7-b3a9-e8c7cc6d48b0" containerID="3a298018042a479d535c7b9bea3d088e8a5f035d79011d7452edbdc26bca176d" exitCode=0
Sep 17 17:35:09.245762 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:35:09.245636    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-gtxcq" event={"ID":"d52f9540-79e9-4df7-b3a9-e8c7cc6d48b0","Type":"ContainerDied","Data":"3a298018042a479d535c7b9bea3d088e8a5f035d79011d7452edbdc26bca176d"}
Sep 17 17:35:09.245762 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:35:09.245656    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-pxt82"
Sep 17 17:35:09.245857 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:35:09.245807    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-9s5wl"
Sep 17 17:35:09.245857 ip-10-0-143-74 kubenswrapper[2575]: E0917 17:35:09.245833    2575 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-pxt82" podUID="57bf735f-0367-4799-a1a8-889870d65704"
Sep 17 17:35:09.246230 ip-10-0-143-74 kubenswrapper[2575]: E0917 17:35:09.246204    2575 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-9s5wl" podUID="7892b408-d2f1-4e39-9bf6-e68a452196b1"
Sep 17 17:35:09.246363 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:35:09.246349    2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-lkwb5"
Sep 17 17:35:09.246435 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:35:09.246423    2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-lkwb5"
Sep 17 17:35:09.260582 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:35:09.260562    2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-lkwb5"
Sep 17 17:35:10.249892 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:35:10.249856    2575 generic.go:356] "Generic (PLEG): container finished" podID="d52f9540-79e9-4df7-b3a9-e8c7cc6d48b0" containerID="25848797154c3cde88ffcd9ed951f88c1e3e6e75141df46a584eba5872e1f3f9" exitCode=0
Sep 17 17:35:10.250291 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:35:10.249942    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-gtxcq" event={"ID":"d52f9540-79e9-4df7-b3a9-e8c7cc6d48b0","Type":"ContainerDied","Data":"25848797154c3cde88ffcd9ed951f88c1e3e6e75141df46a584eba5872e1f3f9"}
Sep 17 17:35:10.396551 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:35:10.396519    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-djvzt_1d07e68d-9943-420f-a6a0-5f477792fabd/konnectivity-agent/0.log" limit=1024
Sep 17 17:35:10.584377 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:35:10.584294    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-djvzt_1d07e68d-9943-420f-a6a0-5f477792fabd/konnectivity-agent/0.log" limit=1024
Sep 17 17:35:10.680981 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:35:10.680949    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-djvzt_1d07e68d-9943-420f-a6a0-5f477792fabd/konnectivity-agent/0.log" limit=1024
Sep 17 17:35:10.962989 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:35:10.962194    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-wdp4h"
Sep 17 17:35:10.962989 ip-10-0-143-74 kubenswrapper[2575]: E0917 17:35:10.962320    2575 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-wdp4h" podUID="27866e17-c231-49a6-921f-a83a01b8e173"
Sep 17 17:35:10.962989 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:35:10.962726    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-9s5wl"
Sep 17 17:35:10.962989 ip-10-0-143-74 kubenswrapper[2575]: E0917 17:35:10.962824    2575 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-9s5wl" podUID="7892b408-d2f1-4e39-9bf6-e68a452196b1"
Sep 17 17:35:10.962989 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:35:10.962877    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-pxt82"
Sep 17 17:35:10.962989 ip-10-0-143-74 kubenswrapper[2575]: E0917 17:35:10.962946    2575 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-pxt82" podUID="57bf735f-0367-4799-a1a8-889870d65704"
Sep 17 17:35:12.351797 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:35:12.351627    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-djvzt_1d07e68d-9943-420f-a6a0-5f477792fabd/konnectivity-agent/0.log" limit=1024
Sep 17 17:35:12.460144 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:35:12.460112    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-djvzt_1d07e68d-9943-420f-a6a0-5f477792fabd/konnectivity-agent/0.log" limit=1024
Sep 17 17:35:12.546157 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:35:12.546121    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-djvzt_1d07e68d-9943-420f-a6a0-5f477792fabd/konnectivity-agent/0.log" limit=1024
Sep 17 17:35:12.961458 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:35:12.961421    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-wdp4h"
Sep 17 17:35:12.961627 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:35:12.961422    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-9s5wl"
Sep 17 17:35:12.961627 ip-10-0-143-74 kubenswrapper[2575]: E0917 17:35:12.961577    2575 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-wdp4h" podUID="27866e17-c231-49a6-921f-a83a01b8e173"
Sep 17 17:35:12.961627 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:35:12.961623    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-pxt82"
Sep 17 17:35:12.961768 ip-10-0-143-74 kubenswrapper[2575]: E0917 17:35:12.961704    2575 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-9s5wl" podUID="7892b408-d2f1-4e39-9bf6-e68a452196b1"
Sep 17 17:35:12.961817 ip-10-0-143-74 kubenswrapper[2575]: E0917 17:35:12.961779    2575 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-pxt82" podUID="57bf735f-0367-4799-a1a8-889870d65704"
Sep 17 17:35:13.292207 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:35:13.292122    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-djvzt_1d07e68d-9943-420f-a6a0-5f477792fabd/konnectivity-agent/0.log" limit=1024
Sep 17 17:35:13.353649 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:35:13.353612    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-djvzt_1d07e68d-9943-420f-a6a0-5f477792fabd/konnectivity-agent/0.log" limit=1024
Sep 17 17:35:13.453505 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:35:13.453471    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-djvzt_1d07e68d-9943-420f-a6a0-5f477792fabd/konnectivity-agent/0.log" limit=1024
Sep 17 17:35:14.458638 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:35:14.458603    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-djvzt_1d07e68d-9943-420f-a6a0-5f477792fabd/konnectivity-agent/0.log" limit=1024
Sep 17 17:35:14.491044 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:35:14.491016    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/7892b408-d2f1-4e39-9bf6-e68a452196b1-original-pull-secret\") pod \"global-pull-secret-syncer-9s5wl\" (UID: \"7892b408-d2f1-4e39-9bf6-e68a452196b1\") " pod="kube-system/global-pull-secret-syncer-9s5wl"
Sep 17 17:35:14.491184 ip-10-0-143-74 kubenswrapper[2575]: E0917 17:35:14.491124    2575 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Sep 17 17:35:14.491184 ip-10-0-143-74 kubenswrapper[2575]: E0917 17:35:14.491177    2575 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/7892b408-d2f1-4e39-9bf6-e68a452196b1-original-pull-secret podName:7892b408-d2f1-4e39-9bf6-e68a452196b1 nodeName:}" failed. No retries permitted until 2026-09-17 17:35:46.491161419 +0000 UTC m=+66.121491372 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/7892b408-d2f1-4e39-9bf6-e68a452196b1-original-pull-secret") pod "global-pull-secret-syncer-9s5wl" (UID: "7892b408-d2f1-4e39-9bf6-e68a452196b1") : object "kube-system"/"original-pull-secret" not registered
Sep 17 17:35:14.559804 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:35:14.559775    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-djvzt_1d07e68d-9943-420f-a6a0-5f477792fabd/konnectivity-agent/0.log" limit=1024
Sep 17 17:35:14.591296 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:35:14.591264    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/27866e17-c231-49a6-921f-a83a01b8e173-metrics-certs\") pod \"network-metrics-daemon-wdp4h\" (UID: \"27866e17-c231-49a6-921f-a83a01b8e173\") " pod="openshift-multus/network-metrics-daemon-wdp4h"
Sep 17 17:35:14.591479 ip-10-0-143-74 kubenswrapper[2575]: E0917 17:35:14.591388    2575 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 17 17:35:14.591550 ip-10-0-143-74 kubenswrapper[2575]: E0917 17:35:14.591480    2575 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/27866e17-c231-49a6-921f-a83a01b8e173-metrics-certs podName:27866e17-c231-49a6-921f-a83a01b8e173 nodeName:}" failed. No retries permitted until 2026-09-17 17:35:46.591460029 +0000 UTC m=+66.221789996 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/27866e17-c231-49a6-921f-a83a01b8e173-metrics-certs") pod "network-metrics-daemon-wdp4h" (UID: "27866e17-c231-49a6-921f-a83a01b8e173") : object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 17 17:35:14.692182 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:35:14.692146    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7rzlr\" (UniqueName: \"kubernetes.io/projected/57bf735f-0367-4799-a1a8-889870d65704-kube-api-access-7rzlr\") pod \"network-check-target-pxt82\" (UID: \"57bf735f-0367-4799-a1a8-889870d65704\") " pod="openshift-network-diagnostics/network-check-target-pxt82"
Sep 17 17:35:14.692339 ip-10-0-143-74 kubenswrapper[2575]: E0917 17:35:14.692297    2575 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Sep 17 17:35:14.692339 ip-10-0-143-74 kubenswrapper[2575]: E0917 17:35:14.692320    2575 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Sep 17 17:35:14.692339 ip-10-0-143-74 kubenswrapper[2575]: E0917 17:35:14.692332    2575 projected.go:196] Error preparing data for projected volume kube-api-access-7rzlr for pod openshift-network-diagnostics/network-check-target-pxt82: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 17 17:35:14.692465 ip-10-0-143-74 kubenswrapper[2575]: E0917 17:35:14.692392    2575 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/57bf735f-0367-4799-a1a8-889870d65704-kube-api-access-7rzlr podName:57bf735f-0367-4799-a1a8-889870d65704 nodeName:}" failed. No retries permitted until 2026-09-17 17:35:46.692374078 +0000 UTC m=+66.322704030 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "kube-api-access-7rzlr" (UniqueName: "kubernetes.io/projected/57bf735f-0367-4799-a1a8-889870d65704-kube-api-access-7rzlr") pod "network-check-target-pxt82" (UID: "57bf735f-0367-4799-a1a8-889870d65704") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 17 17:35:14.961285 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:35:14.961251    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-9s5wl"
Sep 17 17:35:14.961285 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:35:14.961251    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-wdp4h"
Sep 17 17:35:14.961483 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:35:14.961264    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-pxt82"
Sep 17 17:35:14.961483 ip-10-0-143-74 kubenswrapper[2575]: E0917 17:35:14.961387    2575 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-9s5wl" podUID="7892b408-d2f1-4e39-9bf6-e68a452196b1"
Sep 17 17:35:14.961483 ip-10-0-143-74 kubenswrapper[2575]: E0917 17:35:14.961430    2575 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-pxt82" podUID="57bf735f-0367-4799-a1a8-889870d65704"
Sep 17 17:35:14.961588 ip-10-0-143-74 kubenswrapper[2575]: E0917 17:35:14.961558    2575 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-wdp4h" podUID="27866e17-c231-49a6-921f-a83a01b8e173"
Sep 17 17:35:15.264003 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:35:15.263971    2575 generic.go:356] "Generic (PLEG): container finished" podID="d52f9540-79e9-4df7-b3a9-e8c7cc6d48b0" containerID="410688c6299607fd552ec1b6b7f199c17d7c9e473e4a3270f52fc0d576202f67" exitCode=0
Sep 17 17:35:15.264003 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:35:15.264004    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-gtxcq" event={"ID":"d52f9540-79e9-4df7-b3a9-e8c7cc6d48b0","Type":"ContainerDied","Data":"410688c6299607fd552ec1b6b7f199c17d7c9e473e4a3270f52fc0d576202f67"}
Sep 17 17:35:15.627366 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:35:15.627337    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-djvzt_1d07e68d-9943-420f-a6a0-5f477792fabd/konnectivity-agent/0.log" limit=1024
Sep 17 17:35:15.707289 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:35:15.707257    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-djvzt_1d07e68d-9943-420f-a6a0-5f477792fabd/konnectivity-agent/0.log" limit=1024
Sep 17 17:35:16.113015 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:35:16.112984    2575 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-143-74.ec2.internal" event="NodeReady"
Sep 17 17:35:16.113182 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:35:16.113093    2575 kubelet_node_status.go:489] "Fast updating node status as it just became ready"
Sep 17 17:35:16.178163 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:35:16.178132    2575 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-dns/dns-default-tl8ds"]
Sep 17 17:35:16.198670 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:35:16.198637    2575 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress-canary/ingress-canary-4p2w5"]
Sep 17 17:35:16.198819 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:35:16.198768    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-tl8ds"
Sep 17 17:35:16.201511 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:35:16.201490    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-default-metrics-tls\""
Sep 17 17:35:16.201511 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:35:16.201500    2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"dns-default\""
Sep 17 17:35:16.201511 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:35:16.201502    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-dng7x\""
Sep 17 17:35:16.220731 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:35:16.220708    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-tl8ds"]
Sep 17 17:35:16.220844 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:35:16.220742    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-4p2w5"]
Sep 17 17:35:16.220844 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:35:16.220823    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-4p2w5"
Sep 17 17:35:16.223614 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:35:16.223597    2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"openshift-service-ca.crt\""
Sep 17 17:35:16.223747 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:35:16.223639    2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"kube-root-ca.crt\""
Sep 17 17:35:16.223799 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:35:16.223785    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"default-dockercfg-wr4qk\""
Sep 17 17:35:16.223866 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:35:16.223853    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"canary-serving-cert\""
Sep 17 17:35:16.269699 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:35:16.269669    2575 generic.go:356] "Generic (PLEG): container finished" podID="d52f9540-79e9-4df7-b3a9-e8c7cc6d48b0" containerID="eaf3fb98e9b20e201df97b38828361fbf734ca21fe05a7b71f6b6a11365f109e" exitCode=0
Sep 17 17:35:16.269853 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:35:16.269725    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-gtxcq" event={"ID":"d52f9540-79e9-4df7-b3a9-e8c7cc6d48b0","Type":"ContainerDied","Data":"eaf3fb98e9b20e201df97b38828361fbf734ca21fe05a7b71f6b6a11365f109e"}
Sep 17 17:35:16.304632 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:35:16.304599    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-slb9s\" (UniqueName: \"kubernetes.io/projected/cacfc0cf-465f-43c5-bfe6-dd4934eb4a6f-kube-api-access-slb9s\") pod \"ingress-canary-4p2w5\" (UID: \"cacfc0cf-465f-43c5-bfe6-dd4934eb4a6f\") " pod="openshift-ingress-canary/ingress-canary-4p2w5"
Sep 17 17:35:16.304747 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:35:16.304644    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/b8b3d730-8877-45f1-8500-451ee8ac53a3-config-volume\") pod \"dns-default-tl8ds\" (UID: \"b8b3d730-8877-45f1-8500-451ee8ac53a3\") " pod="openshift-dns/dns-default-tl8ds"
Sep 17 17:35:16.304747 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:35:16.304683    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/cacfc0cf-465f-43c5-bfe6-dd4934eb4a6f-cert\") pod \"ingress-canary-4p2w5\" (UID: \"cacfc0cf-465f-43c5-bfe6-dd4934eb4a6f\") " pod="openshift-ingress-canary/ingress-canary-4p2w5"
Sep 17 17:35:16.304747 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:35:16.304714    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/b8b3d730-8877-45f1-8500-451ee8ac53a3-tmp-dir\") pod \"dns-default-tl8ds\" (UID: \"b8b3d730-8877-45f1-8500-451ee8ac53a3\") " pod="openshift-dns/dns-default-tl8ds"
Sep 17 17:35:16.304747 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:35:16.304742    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/b8b3d730-8877-45f1-8500-451ee8ac53a3-metrics-tls\") pod \"dns-default-tl8ds\" (UID: \"b8b3d730-8877-45f1-8500-451ee8ac53a3\") " pod="openshift-dns/dns-default-tl8ds"
Sep 17 17:35:16.304890 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:35:16.304794    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-n67jh\" (UniqueName: \"kubernetes.io/projected/b8b3d730-8877-45f1-8500-451ee8ac53a3-kube-api-access-n67jh\") pod \"dns-default-tl8ds\" (UID: \"b8b3d730-8877-45f1-8500-451ee8ac53a3\") " pod="openshift-dns/dns-default-tl8ds"
Sep 17 17:35:16.365973 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:35:16.365914    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-djvzt_1d07e68d-9943-420f-a6a0-5f477792fabd/konnectivity-agent/0.log" limit=1024
Sep 17 17:35:16.405890 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:35:16.405861    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-slb9s\" (UniqueName: \"kubernetes.io/projected/cacfc0cf-465f-43c5-bfe6-dd4934eb4a6f-kube-api-access-slb9s\") pod \"ingress-canary-4p2w5\" (UID: \"cacfc0cf-465f-43c5-bfe6-dd4934eb4a6f\") " pod="openshift-ingress-canary/ingress-canary-4p2w5"
Sep 17 17:35:16.406006 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:35:16.405895    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/b8b3d730-8877-45f1-8500-451ee8ac53a3-config-volume\") pod \"dns-default-tl8ds\" (UID: \"b8b3d730-8877-45f1-8500-451ee8ac53a3\") " pod="openshift-dns/dns-default-tl8ds"
Sep 17 17:35:16.406006 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:35:16.405962    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/cacfc0cf-465f-43c5-bfe6-dd4934eb4a6f-cert\") pod \"ingress-canary-4p2w5\" (UID: \"cacfc0cf-465f-43c5-bfe6-dd4934eb4a6f\") " pod="openshift-ingress-canary/ingress-canary-4p2w5"
Sep 17 17:35:16.406111 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:35:16.406005    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/b8b3d730-8877-45f1-8500-451ee8ac53a3-tmp-dir\") pod \"dns-default-tl8ds\" (UID: \"b8b3d730-8877-45f1-8500-451ee8ac53a3\") " pod="openshift-dns/dns-default-tl8ds"
Sep 17 17:35:16.406283 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:35:16.406257    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/b8b3d730-8877-45f1-8500-451ee8ac53a3-metrics-tls\") pod \"dns-default-tl8ds\" (UID: \"b8b3d730-8877-45f1-8500-451ee8ac53a3\") " pod="openshift-dns/dns-default-tl8ds"
Sep 17 17:35:16.406385 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:35:16.406307    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-n67jh\" (UniqueName: \"kubernetes.io/projected/b8b3d730-8877-45f1-8500-451ee8ac53a3-kube-api-access-n67jh\") pod \"dns-default-tl8ds\" (UID: \"b8b3d730-8877-45f1-8500-451ee8ac53a3\") " pod="openshift-dns/dns-default-tl8ds"
Sep 17 17:35:16.406385 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:35:16.406369    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/b8b3d730-8877-45f1-8500-451ee8ac53a3-tmp-dir\") pod \"dns-default-tl8ds\" (UID: \"b8b3d730-8877-45f1-8500-451ee8ac53a3\") " pod="openshift-dns/dns-default-tl8ds"
Sep 17 17:35:16.406491 ip-10-0-143-74 kubenswrapper[2575]: E0917 17:35:16.406415    2575 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Sep 17 17:35:16.406535 ip-10-0-143-74 kubenswrapper[2575]: E0917 17:35:16.406490    2575 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b8b3d730-8877-45f1-8500-451ee8ac53a3-metrics-tls podName:b8b3d730-8877-45f1-8500-451ee8ac53a3 nodeName:}" failed. No retries permitted until 2026-09-17 17:35:16.906471435 +0000 UTC m=+36.536801402 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/b8b3d730-8877-45f1-8500-451ee8ac53a3-metrics-tls") pod "dns-default-tl8ds" (UID: "b8b3d730-8877-45f1-8500-451ee8ac53a3") : secret "dns-default-metrics-tls" not found
Sep 17 17:35:16.406627 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:35:16.406612    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/b8b3d730-8877-45f1-8500-451ee8ac53a3-config-volume\") pod \"dns-default-tl8ds\" (UID: \"b8b3d730-8877-45f1-8500-451ee8ac53a3\") " pod="openshift-dns/dns-default-tl8ds"
Sep 17 17:35:16.409913 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:35:16.409892    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/cacfc0cf-465f-43c5-bfe6-dd4934eb4a6f-cert\") pod \"ingress-canary-4p2w5\" (UID: \"cacfc0cf-465f-43c5-bfe6-dd4934eb4a6f\") " pod="openshift-ingress-canary/ingress-canary-4p2w5"
Sep 17 17:35:16.418955 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:35:16.418922    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-n67jh\" (UniqueName: \"kubernetes.io/projected/b8b3d730-8877-45f1-8500-451ee8ac53a3-kube-api-access-n67jh\") pod \"dns-default-tl8ds\" (UID: \"b8b3d730-8877-45f1-8500-451ee8ac53a3\") " pod="openshift-dns/dns-default-tl8ds"
Sep 17 17:35:16.419762 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:35:16.419745    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-slb9s\" (UniqueName: \"kubernetes.io/projected/cacfc0cf-465f-43c5-bfe6-dd4934eb4a6f-kube-api-access-slb9s\") pod \"ingress-canary-4p2w5\" (UID: \"cacfc0cf-465f-43c5-bfe6-dd4934eb4a6f\") " pod="openshift-ingress-canary/ingress-canary-4p2w5"
Sep 17 17:35:16.467333 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:35:16.467298    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-djvzt_1d07e68d-9943-420f-a6a0-5f477792fabd/konnectivity-agent/0.log" limit=1024
Sep 17 17:35:16.529334 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:35:16.529298    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-4p2w5"
Sep 17 17:35:16.853469 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:35:16.853080    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-4p2w5"]
Sep 17 17:35:16.909606 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:35:16.909514    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/b8b3d730-8877-45f1-8500-451ee8ac53a3-metrics-tls\") pod \"dns-default-tl8ds\" (UID: \"b8b3d730-8877-45f1-8500-451ee8ac53a3\") " pod="openshift-dns/dns-default-tl8ds"
Sep 17 17:35:16.909721 ip-10-0-143-74 kubenswrapper[2575]: E0917 17:35:16.909636    2575 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Sep 17 17:35:16.909721 ip-10-0-143-74 kubenswrapper[2575]: E0917 17:35:16.909702    2575 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b8b3d730-8877-45f1-8500-451ee8ac53a3-metrics-tls podName:b8b3d730-8877-45f1-8500-451ee8ac53a3 nodeName:}" failed. No retries permitted until 2026-09-17 17:35:17.909685808 +0000 UTC m=+37.540015768 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/b8b3d730-8877-45f1-8500-451ee8ac53a3-metrics-tls") pod "dns-default-tl8ds" (UID: "b8b3d730-8877-45f1-8500-451ee8ac53a3") : secret "dns-default-metrics-tls" not found
Sep 17 17:35:16.961583 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:35:16.961539    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-wdp4h"
Sep 17 17:35:16.961583 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:35:16.961577    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-9s5wl"
Sep 17 17:35:16.961790 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:35:16.961577    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-pxt82"
Sep 17 17:35:16.964625 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:35:16.964596    2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\""
Sep 17 17:35:16.965812 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:35:16.965781    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\""
Sep 17 17:35:16.965932 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:35:16.965814    2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\""
Sep 17 17:35:16.965932 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:35:16.965832    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\""
Sep 17 17:35:16.965932 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:35:16.965781    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-j84n5\""
Sep 17 17:35:16.966077 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:35:16.965946    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-vwwqh\""
Sep 17 17:35:17.276397 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:35:17.276312    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-gtxcq" event={"ID":"d52f9540-79e9-4df7-b3a9-e8c7cc6d48b0","Type":"ContainerStarted","Data":"bc8bc78c6710a51aeabb5ef11531d5ad97666e38623fcbbb6a1b5c095afe3676"}
Sep 17 17:35:17.277797 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:35:17.277759    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-4p2w5" event={"ID":"cacfc0cf-465f-43c5-bfe6-dd4934eb4a6f","Type":"ContainerStarted","Data":"d3fa62888bcc25763446963275864922eceecaedfb3974650df736d81324aa6f"}
Sep 17 17:35:17.318996 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:35:17.318947    2575 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-additional-cni-plugins-gtxcq" podStartSLOduration=8.526553376 podStartE2EDuration="36.318931987s" totalImagesPullingTime="27.792378611s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-17 17:34:41 +0000 UTC" imagePullSessionsCount=5 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-17 17:35:17.318401911 +0000 UTC m=+36.948731896" watchObservedRunningTime="2026-09-17 17:35:17.318931987 +0000 UTC m=+36.949261959"
Sep 17 17:35:17.497700 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:35:17.497578    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-djvzt_1d07e68d-9943-420f-a6a0-5f477792fabd/konnectivity-agent/0.log" limit=1024
Sep 17 17:35:17.568526 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:35:17.568428    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-djvzt_1d07e68d-9943-420f-a6a0-5f477792fabd/konnectivity-agent/0.log" limit=1024
Sep 17 17:35:17.631375 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:35:17.631335    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-djvzt_1d07e68d-9943-420f-a6a0-5f477792fabd/konnectivity-agent/0.log" limit=1024
Sep 17 17:35:17.918960 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:35:17.918919    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/b8b3d730-8877-45f1-8500-451ee8ac53a3-metrics-tls\") pod \"dns-default-tl8ds\" (UID: \"b8b3d730-8877-45f1-8500-451ee8ac53a3\") " pod="openshift-dns/dns-default-tl8ds"
Sep 17 17:35:17.919512 ip-10-0-143-74 kubenswrapper[2575]: E0917 17:35:17.919087    2575 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Sep 17 17:35:17.919512 ip-10-0-143-74 kubenswrapper[2575]: E0917 17:35:17.919177    2575 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b8b3d730-8877-45f1-8500-451ee8ac53a3-metrics-tls podName:b8b3d730-8877-45f1-8500-451ee8ac53a3 nodeName:}" failed. No retries permitted until 2026-09-17 17:35:19.919158005 +0000 UTC m=+39.549487976 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/b8b3d730-8877-45f1-8500-451ee8ac53a3-metrics-tls") pod "dns-default-tl8ds" (UID: "b8b3d730-8877-45f1-8500-451ee8ac53a3") : secret "dns-default-metrics-tls" not found
Sep 17 17:35:19.506107 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:35:19.506082    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-djvzt_1d07e68d-9943-420f-a6a0-5f477792fabd/konnectivity-agent/0.log" limit=1024
Sep 17 17:35:19.590283 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:35:19.590065    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-djvzt_1d07e68d-9943-420f-a6a0-5f477792fabd/konnectivity-agent/0.log" limit=1024
Sep 17 17:35:19.933894 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:35:19.933865    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/b8b3d730-8877-45f1-8500-451ee8ac53a3-metrics-tls\") pod \"dns-default-tl8ds\" (UID: \"b8b3d730-8877-45f1-8500-451ee8ac53a3\") " pod="openshift-dns/dns-default-tl8ds"
Sep 17 17:35:19.934050 ip-10-0-143-74 kubenswrapper[2575]: E0917 17:35:19.933977    2575 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Sep 17 17:35:19.934050 ip-10-0-143-74 kubenswrapper[2575]: E0917 17:35:19.934032    2575 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b8b3d730-8877-45f1-8500-451ee8ac53a3-metrics-tls podName:b8b3d730-8877-45f1-8500-451ee8ac53a3 nodeName:}" failed. No retries permitted until 2026-09-17 17:35:23.934019203 +0000 UTC m=+43.564349156 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/b8b3d730-8877-45f1-8500-451ee8ac53a3-metrics-tls") pod "dns-default-tl8ds" (UID: "b8b3d730-8877-45f1-8500-451ee8ac53a3") : secret "dns-default-metrics-tls" not found
Sep 17 17:35:20.289248 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:35:20.289159    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-4p2w5" event={"ID":"cacfc0cf-465f-43c5-bfe6-dd4934eb4a6f","Type":"ContainerStarted","Data":"402a4c368ab6c6c0da660267bf41906aac15494049126d0ee38302b5b9326a31"}
Sep 17 17:35:20.306420 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:35:20.306373    2575 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress-canary/ingress-canary-4p2w5" podStartSLOduration=1.744038429 podStartE2EDuration="4.306358242s" totalImagesPullingTime="2.562319813s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-17 17:35:16 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-17 17:35:20.305617422 +0000 UTC m=+39.935947399" watchObservedRunningTime="2026-09-17 17:35:20.306358242 +0000 UTC m=+39.936688239"
Sep 17 17:35:20.396847 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:35:20.396812    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-djvzt_1d07e68d-9943-420f-a6a0-5f477792fabd/konnectivity-agent/0.log" limit=1024
Sep 17 17:35:20.580733 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:35:20.580644    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-djvzt_1d07e68d-9943-420f-a6a0-5f477792fabd/konnectivity-agent/0.log" limit=1024
Sep 17 17:35:23.959765 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:35:23.959727    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/b8b3d730-8877-45f1-8500-451ee8ac53a3-metrics-tls\") pod \"dns-default-tl8ds\" (UID: \"b8b3d730-8877-45f1-8500-451ee8ac53a3\") " pod="openshift-dns/dns-default-tl8ds"
Sep 17 17:35:23.960178 ip-10-0-143-74 kubenswrapper[2575]: E0917 17:35:23.959852    2575 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Sep 17 17:35:23.960178 ip-10-0-143-74 kubenswrapper[2575]: E0917 17:35:23.959902    2575 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b8b3d730-8877-45f1-8500-451ee8ac53a3-metrics-tls podName:b8b3d730-8877-45f1-8500-451ee8ac53a3 nodeName:}" failed. No retries permitted until 2026-09-17 17:35:31.959889853 +0000 UTC m=+51.590219805 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/b8b3d730-8877-45f1-8500-451ee8ac53a3-metrics-tls") pod "dns-default-tl8ds" (UID: "b8b3d730-8877-45f1-8500-451ee8ac53a3") : secret "dns-default-metrics-tls" not found
Sep 17 17:35:25.498209 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:35:25.498173    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-djvzt_1d07e68d-9943-420f-a6a0-5f477792fabd/konnectivity-agent/0.log" limit=1024
Sep 17 17:35:26.371065 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:35:26.371029    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-djvzt_1d07e68d-9943-420f-a6a0-5f477792fabd/konnectivity-agent/0.log" limit=1024
Sep 17 17:35:26.452492 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:35:26.452437    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-djvzt_1d07e68d-9943-420f-a6a0-5f477792fabd/konnectivity-agent/0.log" limit=1024
Sep 17 17:35:29.499737 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:35:29.499695    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-djvzt_1d07e68d-9943-420f-a6a0-5f477792fabd/konnectivity-agent/0.log" limit=1024
Sep 17 17:35:29.568119 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:35:29.568088    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-djvzt_1d07e68d-9943-420f-a6a0-5f477792fabd/konnectivity-agent/0.log" limit=1024
Sep 17 17:35:32.019251 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:35:32.019214    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/b8b3d730-8877-45f1-8500-451ee8ac53a3-metrics-tls\") pod \"dns-default-tl8ds\" (UID: \"b8b3d730-8877-45f1-8500-451ee8ac53a3\") " pod="openshift-dns/dns-default-tl8ds"
Sep 17 17:35:32.019746 ip-10-0-143-74 kubenswrapper[2575]: E0917 17:35:32.019333    2575 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Sep 17 17:35:32.019746 ip-10-0-143-74 kubenswrapper[2575]: E0917 17:35:32.019384    2575 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b8b3d730-8877-45f1-8500-451ee8ac53a3-metrics-tls podName:b8b3d730-8877-45f1-8500-451ee8ac53a3 nodeName:}" failed. No retries permitted until 2026-09-17 17:35:48.019370765 +0000 UTC m=+67.649700718 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/b8b3d730-8877-45f1-8500-451ee8ac53a3-metrics-tls") pod "dns-default-tl8ds" (UID: "b8b3d730-8877-45f1-8500-451ee8ac53a3") : secret "dns-default-metrics-tls" not found
Sep 17 17:35:35.408244 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:35:35.408205    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-djvzt_1d07e68d-9943-420f-a6a0-5f477792fabd/konnectivity-agent/0.log" limit=1024
Sep 17 17:35:35.508606 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:35:35.508577    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-djvzt_1d07e68d-9943-420f-a6a0-5f477792fabd/konnectivity-agent/0.log" limit=1024
Sep 17 17:35:41.262902 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:35:41.262874    2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-lkwb5"
Sep 17 17:35:41.437752 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:35:41.437718    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-djvzt_1d07e68d-9943-420f-a6a0-5f477792fabd/konnectivity-agent/0.log" limit=1024
Sep 17 17:35:41.555404 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:35:41.555313    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-djvzt_1d07e68d-9943-420f-a6a0-5f477792fabd/konnectivity-agent/0.log" limit=1024
Sep 17 17:35:46.523386 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:35:46.523351    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/7892b408-d2f1-4e39-9bf6-e68a452196b1-original-pull-secret\") pod \"global-pull-secret-syncer-9s5wl\" (UID: \"7892b408-d2f1-4e39-9bf6-e68a452196b1\") " pod="kube-system/global-pull-secret-syncer-9s5wl"
Sep 17 17:35:46.526883 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:35:46.526865    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\""
Sep 17 17:35:46.537860 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:35:46.537832    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/7892b408-d2f1-4e39-9bf6-e68a452196b1-original-pull-secret\") pod \"global-pull-secret-syncer-9s5wl\" (UID: \"7892b408-d2f1-4e39-9bf6-e68a452196b1\") " pod="kube-system/global-pull-secret-syncer-9s5wl"
Sep 17 17:35:46.624051 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:35:46.624018    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/27866e17-c231-49a6-921f-a83a01b8e173-metrics-certs\") pod \"network-metrics-daemon-wdp4h\" (UID: \"27866e17-c231-49a6-921f-a83a01b8e173\") " pod="openshift-multus/network-metrics-daemon-wdp4h"
Sep 17 17:35:46.627040 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:35:46.627023    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\""
Sep 17 17:35:46.634991 ip-10-0-143-74 kubenswrapper[2575]: E0917 17:35:46.634975    2575 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: secret "metrics-daemon-secret" not found
Sep 17 17:35:46.635052 ip-10-0-143-74 kubenswrapper[2575]: E0917 17:35:46.635042    2575 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/27866e17-c231-49a6-921f-a83a01b8e173-metrics-certs podName:27866e17-c231-49a6-921f-a83a01b8e173 nodeName:}" failed. No retries permitted until 2026-09-17 17:36:50.635022024 +0000 UTC m=+130.265351985 (durationBeforeRetry 1m4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/27866e17-c231-49a6-921f-a83a01b8e173-metrics-certs") pod "network-metrics-daemon-wdp4h" (UID: "27866e17-c231-49a6-921f-a83a01b8e173") : secret "metrics-daemon-secret" not found
Sep 17 17:35:46.677581 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:35:46.677555    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-9s5wl"
Sep 17 17:35:46.724576 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:35:46.724545    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7rzlr\" (UniqueName: \"kubernetes.io/projected/57bf735f-0367-4799-a1a8-889870d65704-kube-api-access-7rzlr\") pod \"network-check-target-pxt82\" (UID: \"57bf735f-0367-4799-a1a8-889870d65704\") " pod="openshift-network-diagnostics/network-check-target-pxt82"
Sep 17 17:35:46.728773 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:35:46.728703    2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\""
Sep 17 17:35:46.738300 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:35:46.738277    2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\""
Sep 17 17:35:46.748174 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:35:46.748152    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-7rzlr\" (UniqueName: \"kubernetes.io/projected/57bf735f-0367-4799-a1a8-889870d65704-kube-api-access-7rzlr\") pod \"network-check-target-pxt82\" (UID: \"57bf735f-0367-4799-a1a8-889870d65704\") " pod="openshift-network-diagnostics/network-check-target-pxt82"
Sep 17 17:35:46.953101 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:35:46.953070    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-9s5wl"]
Sep 17 17:35:46.956502 ip-10-0-143-74 kubenswrapper[2575]: W0917 17:35:46.956477    2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7892b408_d2f1_4e39_9bf6_e68a452196b1.slice/crio-07cc5416237faa00f30f273d0e816eb2ac96852ef43c8b19cf5adaf9c8a0a756 WatchSource:0}: Error finding container 07cc5416237faa00f30f273d0e816eb2ac96852ef43c8b19cf5adaf9c8a0a756: Status 404 returned error can't find the container with id 07cc5416237faa00f30f273d0e816eb2ac96852ef43c8b19cf5adaf9c8a0a756
Sep 17 17:35:46.984608 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:35:46.984586    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-vwwqh\""
Sep 17 17:35:46.991873 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:35:46.991848    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-pxt82"
Sep 17 17:35:47.263667 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:35:47.263587    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-pxt82"]
Sep 17 17:35:47.267047 ip-10-0-143-74 kubenswrapper[2575]: W0917 17:35:47.267020    2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod57bf735f_0367_4799_a1a8_889870d65704.slice/crio-981eda7881da6cce50efdf575a4b1612b001978148d5e4f0c73e69f7b8a3737d WatchSource:0}: Error finding container 981eda7881da6cce50efdf575a4b1612b001978148d5e4f0c73e69f7b8a3737d: Status 404 returned error can't find the container with id 981eda7881da6cce50efdf575a4b1612b001978148d5e4f0c73e69f7b8a3737d
Sep 17 17:35:47.352265 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:35:47.352234    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-pxt82" event={"ID":"57bf735f-0367-4799-a1a8-889870d65704","Type":"ContainerStarted","Data":"981eda7881da6cce50efdf575a4b1612b001978148d5e4f0c73e69f7b8a3737d"}
Sep 17 17:35:47.353382 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:35:47.353358    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-9s5wl" event={"ID":"7892b408-d2f1-4e39-9bf6-e68a452196b1","Type":"ContainerStarted","Data":"07cc5416237faa00f30f273d0e816eb2ac96852ef43c8b19cf5adaf9c8a0a756"}
Sep 17 17:35:48.034146 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:35:48.034110    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/b8b3d730-8877-45f1-8500-451ee8ac53a3-metrics-tls\") pod \"dns-default-tl8ds\" (UID: \"b8b3d730-8877-45f1-8500-451ee8ac53a3\") " pod="openshift-dns/dns-default-tl8ds"
Sep 17 17:35:48.034680 ip-10-0-143-74 kubenswrapper[2575]: E0917 17:35:48.034268    2575 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Sep 17 17:35:48.034680 ip-10-0-143-74 kubenswrapper[2575]: E0917 17:35:48.034346    2575 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b8b3d730-8877-45f1-8500-451ee8ac53a3-metrics-tls podName:b8b3d730-8877-45f1-8500-451ee8ac53a3 nodeName:}" failed. No retries permitted until 2026-09-17 17:36:20.03432337 +0000 UTC m=+99.664653336 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/b8b3d730-8877-45f1-8500-451ee8ac53a3-metrics-tls") pod "dns-default-tl8ds" (UID: "b8b3d730-8877-45f1-8500-451ee8ac53a3") : secret "dns-default-metrics-tls" not found
Sep 17 17:35:52.367499 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:35:52.367459    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-pxt82" event={"ID":"57bf735f-0367-4799-a1a8-889870d65704","Type":"ContainerStarted","Data":"951018b93c1e6b80078b22d00e940ecef3f103ecfabadbd85446ff0412911c21"}
Sep 17 17:35:52.367894 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:35:52.367561    2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-network-diagnostics/network-check-target-pxt82"
Sep 17 17:35:52.368935 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:35:52.368911    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-9s5wl" event={"ID":"7892b408-d2f1-4e39-9bf6-e68a452196b1","Type":"ContainerStarted","Data":"8cc13d4cf62618fe00ee1a6cf27e79d14426af5cde1912c3acd28c86d2f7b443"}
Sep 17 17:35:52.390517 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:35:52.390467    2575 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-diagnostics/network-check-target-pxt82" podStartSLOduration=67.116112175 podStartE2EDuration="1m11.390436289s" totalImagesPullingTime="4.274324114s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-17 17:34:41 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-17 17:35:52.389006699 +0000 UTC m=+72.019336675" watchObservedRunningTime="2026-09-17 17:35:52.390436289 +0000 UTC m=+72.020766261"
Sep 17 17:35:52.417437 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:35:52.417384    2575 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/global-pull-secret-syncer-9s5wl" podStartSLOduration=66.82504134 podStartE2EDuration="1m11.417367459s" totalImagesPullingTime="4.592326119s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-17 17:34:41 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-17 17:35:52.417188586 +0000 UTC m=+72.047518558" watchObservedRunningTime="2026-09-17 17:35:52.417367459 +0000 UTC m=+72.047697433"
Sep 17 17:35:52.450452 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:35:52.450421    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-djvzt_1d07e68d-9943-420f-a6a0-5f477792fabd/konnectivity-agent/0.log" limit=1024
Sep 17 17:35:52.546400 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:35:52.546369    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-djvzt_1d07e68d-9943-420f-a6a0-5f477792fabd/konnectivity-agent/0.log" limit=1024
Sep 17 17:35:52.641809 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:35:52.641720    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-djvzt_1d07e68d-9943-420f-a6a0-5f477792fabd/konnectivity-agent/0.log" limit=1024
Sep 17 17:36:01.500825 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:36:01.500793    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-djvzt_1d07e68d-9943-420f-a6a0-5f477792fabd/konnectivity-agent/0.log" limit=1024
Sep 17 17:36:01.603744 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:36:01.603703    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-djvzt_1d07e68d-9943-420f-a6a0-5f477792fabd/konnectivity-agent/0.log" limit=1024
Sep 17 17:36:20.053828 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:36:20.053787    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/b8b3d730-8877-45f1-8500-451ee8ac53a3-metrics-tls\") pod \"dns-default-tl8ds\" (UID: \"b8b3d730-8877-45f1-8500-451ee8ac53a3\") " pod="openshift-dns/dns-default-tl8ds"
Sep 17 17:36:20.054280 ip-10-0-143-74 kubenswrapper[2575]: E0917 17:36:20.053914    2575 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Sep 17 17:36:20.054280 ip-10-0-143-74 kubenswrapper[2575]: E0917 17:36:20.053983    2575 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b8b3d730-8877-45f1-8500-451ee8ac53a3-metrics-tls podName:b8b3d730-8877-45f1-8500-451ee8ac53a3 nodeName:}" failed. No retries permitted until 2026-09-17 17:37:24.053969583 +0000 UTC m=+163.684299536 (durationBeforeRetry 1m4s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/b8b3d730-8877-45f1-8500-451ee8ac53a3-metrics-tls") pod "dns-default-tl8ds" (UID: "b8b3d730-8877-45f1-8500-451ee8ac53a3") : secret "dns-default-metrics-tls" not found
Sep 17 17:36:23.373812 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:36:23.373783    2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-network-diagnostics/network-check-target-pxt82"
Sep 17 17:36:23.859046 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:36:23.858967    2575 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-insights/insights-operator-6f9c644dd9-dllkw"]
Sep 17 17:36:23.892067 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:36:23.892028    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-operator-6f9c644dd9-dllkw"]
Sep 17 17:36:23.892212 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:36:23.892127    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-operator-6f9c644dd9-dllkw"
Sep 17 17:36:23.894879 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:36:23.894854    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"openshift-insights-serving-cert\""
Sep 17 17:36:23.894999 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:36:23.894861    2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"service-ca-bundle\""
Sep 17 17:36:23.895800 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:36:23.895786    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"operator-dockercfg-bcscd\""
Sep 17 17:36:23.895800 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:36:23.895794    2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-root-ca.crt\""
Sep 17 17:36:23.895900 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:36:23.895835    2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"openshift-service-ca.crt\""
Sep 17 17:36:23.900140 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:36:23.900123    2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"trusted-ca-bundle\""
Sep 17 17:36:23.983983 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:36:23.983951    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/b253f0e2-6a5b-4fc4-b686-db5ae716f81b-tmp\") pod \"insights-operator-6f9c644dd9-dllkw\" (UID: \"b253f0e2-6a5b-4fc4-b686-db5ae716f81b\") " pod="openshift-insights/insights-operator-6f9c644dd9-dllkw"
Sep 17 17:36:23.984159 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:36:23.983989    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/b253f0e2-6a5b-4fc4-b686-db5ae716f81b-serving-cert\") pod \"insights-operator-6f9c644dd9-dllkw\" (UID: \"b253f0e2-6a5b-4fc4-b686-db5ae716f81b\") " pod="openshift-insights/insights-operator-6f9c644dd9-dllkw"
Sep 17 17:36:23.984159 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:36:23.984027    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"snapshots\" (UniqueName: \"kubernetes.io/empty-dir/b253f0e2-6a5b-4fc4-b686-db5ae716f81b-snapshots\") pod \"insights-operator-6f9c644dd9-dllkw\" (UID: \"b253f0e2-6a5b-4fc4-b686-db5ae716f81b\") " pod="openshift-insights/insights-operator-6f9c644dd9-dllkw"
Sep 17 17:36:23.984159 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:36:23.984095    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/b253f0e2-6a5b-4fc4-b686-db5ae716f81b-service-ca-bundle\") pod \"insights-operator-6f9c644dd9-dllkw\" (UID: \"b253f0e2-6a5b-4fc4-b686-db5ae716f81b\") " pod="openshift-insights/insights-operator-6f9c644dd9-dllkw"
Sep 17 17:36:23.984159 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:36:23.984136    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/b253f0e2-6a5b-4fc4-b686-db5ae716f81b-trusted-ca-bundle\") pod \"insights-operator-6f9c644dd9-dllkw\" (UID: \"b253f0e2-6a5b-4fc4-b686-db5ae716f81b\") " pod="openshift-insights/insights-operator-6f9c644dd9-dllkw"
Sep 17 17:36:23.984293 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:36:23.984187    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-b87d8\" (UniqueName: \"kubernetes.io/projected/b253f0e2-6a5b-4fc4-b686-db5ae716f81b-kube-api-access-b87d8\") pod \"insights-operator-6f9c644dd9-dllkw\" (UID: \"b253f0e2-6a5b-4fc4-b686-db5ae716f81b\") " pod="openshift-insights/insights-operator-6f9c644dd9-dllkw"
Sep 17 17:36:24.063769 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:36:24.063737    2575 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/router-default-76f78f8976-kwv2h"]
Sep 17 17:36:24.081283 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:36:24.081254    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/router-default-76f78f8976-kwv2h"
Sep 17 17:36:24.084118 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:36:24.084093    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"router-dockercfg-xlq4g\""
Sep 17 17:36:24.084415 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:36:24.084388    2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-service-ca.crt\""
Sep 17 17:36:24.084602 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:36:24.084414    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/b253f0e2-6a5b-4fc4-b686-db5ae716f81b-service-ca-bundle\") pod \"insights-operator-6f9c644dd9-dllkw\" (UID: \"b253f0e2-6a5b-4fc4-b686-db5ae716f81b\") " pod="openshift-insights/insights-operator-6f9c644dd9-dllkw"
Sep 17 17:36:24.084602 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:36:24.084467    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"default-ingress-cert\""
Sep 17 17:36:24.084602 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:36:24.084399    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"router-stats-default\""
Sep 17 17:36:24.084602 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:36:24.084478    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/b253f0e2-6a5b-4fc4-b686-db5ae716f81b-trusted-ca-bundle\") pod \"insights-operator-6f9c644dd9-dllkw\" (UID: \"b253f0e2-6a5b-4fc4-b686-db5ae716f81b\") " pod="openshift-insights/insights-operator-6f9c644dd9-dllkw"
Sep 17 17:36:24.084602 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:36:24.084539    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-b87d8\" (UniqueName: \"kubernetes.io/projected/b253f0e2-6a5b-4fc4-b686-db5ae716f81b-kube-api-access-b87d8\") pod \"insights-operator-6f9c644dd9-dllkw\" (UID: \"b253f0e2-6a5b-4fc4-b686-db5ae716f81b\") " pod="openshift-insights/insights-operator-6f9c644dd9-dllkw"
Sep 17 17:36:24.084602 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:36:24.084467    2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"service-ca-bundle\""
Sep 17 17:36:24.084602 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:36:24.084571    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/b253f0e2-6a5b-4fc4-b686-db5ae716f81b-tmp\") pod \"insights-operator-6f9c644dd9-dllkw\" (UID: \"b253f0e2-6a5b-4fc4-b686-db5ae716f81b\") " pod="openshift-insights/insights-operator-6f9c644dd9-dllkw"
Sep 17 17:36:24.084976 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:36:24.084674    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"router-metrics-certs-default\""
Sep 17 17:36:24.084976 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:36:24.084757    2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"kube-root-ca.crt\""
Sep 17 17:36:24.084976 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:36:24.084883    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/b253f0e2-6a5b-4fc4-b686-db5ae716f81b-serving-cert\") pod \"insights-operator-6f9c644dd9-dllkw\" (UID: \"b253f0e2-6a5b-4fc4-b686-db5ae716f81b\") " pod="openshift-insights/insights-operator-6f9c644dd9-dllkw"
Sep 17 17:36:24.084976 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:36:24.084914    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/b253f0e2-6a5b-4fc4-b686-db5ae716f81b-tmp\") pod \"insights-operator-6f9c644dd9-dllkw\" (UID: \"b253f0e2-6a5b-4fc4-b686-db5ae716f81b\") " pod="openshift-insights/insights-operator-6f9c644dd9-dllkw"
Sep 17 17:36:24.084976 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:36:24.084934    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"snapshots\" (UniqueName: \"kubernetes.io/empty-dir/b253f0e2-6a5b-4fc4-b686-db5ae716f81b-snapshots\") pod \"insights-operator-6f9c644dd9-dllkw\" (UID: \"b253f0e2-6a5b-4fc4-b686-db5ae716f81b\") " pod="openshift-insights/insights-operator-6f9c644dd9-dllkw"
Sep 17 17:36:24.085197 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:36:24.085117    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/b253f0e2-6a5b-4fc4-b686-db5ae716f81b-service-ca-bundle\") pod \"insights-operator-6f9c644dd9-dllkw\" (UID: \"b253f0e2-6a5b-4fc4-b686-db5ae716f81b\") " pod="openshift-insights/insights-operator-6f9c644dd9-dllkw"
Sep 17 17:36:24.085527 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:36:24.085507    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"snapshots\" (UniqueName: \"kubernetes.io/empty-dir/b253f0e2-6a5b-4fc4-b686-db5ae716f81b-snapshots\") pod \"insights-operator-6f9c644dd9-dllkw\" (UID: \"b253f0e2-6a5b-4fc4-b686-db5ae716f81b\") " pod="openshift-insights/insights-operator-6f9c644dd9-dllkw"
Sep 17 17:36:24.085599 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:36:24.085551    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/b253f0e2-6a5b-4fc4-b686-db5ae716f81b-trusted-ca-bundle\") pod \"insights-operator-6f9c644dd9-dllkw\" (UID: \"b253f0e2-6a5b-4fc4-b686-db5ae716f81b\") " pod="openshift-insights/insights-operator-6f9c644dd9-dllkw"
Sep 17 17:36:24.087531 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:36:24.087470    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/b253f0e2-6a5b-4fc4-b686-db5ae716f81b-serving-cert\") pod \"insights-operator-6f9c644dd9-dllkw\" (UID: \"b253f0e2-6a5b-4fc4-b686-db5ae716f81b\") " pod="openshift-insights/insights-operator-6f9c644dd9-dllkw"
Sep 17 17:36:24.088208 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:36:24.088180    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/router-default-76f78f8976-kwv2h"]
Sep 17 17:36:24.097585 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:36:24.097557    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-b87d8\" (UniqueName: \"kubernetes.io/projected/b253f0e2-6a5b-4fc4-b686-db5ae716f81b-kube-api-access-b87d8\") pod \"insights-operator-6f9c644dd9-dllkw\" (UID: \"b253f0e2-6a5b-4fc4-b686-db5ae716f81b\") " pod="openshift-insights/insights-operator-6f9c644dd9-dllkw"
Sep 17 17:36:24.185916 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:36:24.185880    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"default-certificate\" (UniqueName: \"kubernetes.io/secret/3a6e39e9-229a-440e-a565-16ef48e36247-default-certificate\") pod \"router-default-76f78f8976-kwv2h\" (UID: \"3a6e39e9-229a-440e-a565-16ef48e36247\") " pod="openshift-ingress/router-default-76f78f8976-kwv2h"
Sep 17 17:36:24.186092 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:36:24.185946    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/3a6e39e9-229a-440e-a565-16ef48e36247-metrics-certs\") pod \"router-default-76f78f8976-kwv2h\" (UID: \"3a6e39e9-229a-440e-a565-16ef48e36247\") " pod="openshift-ingress/router-default-76f78f8976-kwv2h"
Sep 17 17:36:24.186092 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:36:24.185986    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/3a6e39e9-229a-440e-a565-16ef48e36247-service-ca-bundle\") pod \"router-default-76f78f8976-kwv2h\" (UID: \"3a6e39e9-229a-440e-a565-16ef48e36247\") " pod="openshift-ingress/router-default-76f78f8976-kwv2h"
Sep 17 17:36:24.186092 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:36:24.186019    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qnvvb\" (UniqueName: \"kubernetes.io/projected/3a6e39e9-229a-440e-a565-16ef48e36247-kube-api-access-qnvvb\") pod \"router-default-76f78f8976-kwv2h\" (UID: \"3a6e39e9-229a-440e-a565-16ef48e36247\") " pod="openshift-ingress/router-default-76f78f8976-kwv2h"
Sep 17 17:36:24.186202 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:36:24.186088    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"stats-auth\" (UniqueName: \"kubernetes.io/secret/3a6e39e9-229a-440e-a565-16ef48e36247-stats-auth\") pod \"router-default-76f78f8976-kwv2h\" (UID: \"3a6e39e9-229a-440e-a565-16ef48e36247\") " pod="openshift-ingress/router-default-76f78f8976-kwv2h"
Sep 17 17:36:24.200975 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:36:24.200935    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-operator-6f9c644dd9-dllkw"
Sep 17 17:36:24.287057 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:36:24.287015    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"stats-auth\" (UniqueName: \"kubernetes.io/secret/3a6e39e9-229a-440e-a565-16ef48e36247-stats-auth\") pod \"router-default-76f78f8976-kwv2h\" (UID: \"3a6e39e9-229a-440e-a565-16ef48e36247\") " pod="openshift-ingress/router-default-76f78f8976-kwv2h"
Sep 17 17:36:24.287215 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:36:24.287078    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"default-certificate\" (UniqueName: \"kubernetes.io/secret/3a6e39e9-229a-440e-a565-16ef48e36247-default-certificate\") pod \"router-default-76f78f8976-kwv2h\" (UID: \"3a6e39e9-229a-440e-a565-16ef48e36247\") " pod="openshift-ingress/router-default-76f78f8976-kwv2h"
Sep 17 17:36:24.287215 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:36:24.287138    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/3a6e39e9-229a-440e-a565-16ef48e36247-metrics-certs\") pod \"router-default-76f78f8976-kwv2h\" (UID: \"3a6e39e9-229a-440e-a565-16ef48e36247\") " pod="openshift-ingress/router-default-76f78f8976-kwv2h"
Sep 17 17:36:24.287215 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:36:24.287166    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/3a6e39e9-229a-440e-a565-16ef48e36247-service-ca-bundle\") pod \"router-default-76f78f8976-kwv2h\" (UID: \"3a6e39e9-229a-440e-a565-16ef48e36247\") " pod="openshift-ingress/router-default-76f78f8976-kwv2h"
Sep 17 17:36:24.287403 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:36:24.287221    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qnvvb\" (UniqueName: \"kubernetes.io/projected/3a6e39e9-229a-440e-a565-16ef48e36247-kube-api-access-qnvvb\") pod \"router-default-76f78f8976-kwv2h\" (UID: \"3a6e39e9-229a-440e-a565-16ef48e36247\") " pod="openshift-ingress/router-default-76f78f8976-kwv2h"
Sep 17 17:36:24.288233 ip-10-0-143-74 kubenswrapper[2575]: E0917 17:36:24.288179    2575 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found
Sep 17 17:36:24.288354 ip-10-0-143-74 kubenswrapper[2575]: E0917 17:36:24.288257    2575 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/3a6e39e9-229a-440e-a565-16ef48e36247-metrics-certs podName:3a6e39e9-229a-440e-a565-16ef48e36247 nodeName:}" failed. No retries permitted until 2026-09-17 17:36:24.788233719 +0000 UTC m=+104.418563685 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/3a6e39e9-229a-440e-a565-16ef48e36247-metrics-certs") pod "router-default-76f78f8976-kwv2h" (UID: "3a6e39e9-229a-440e-a565-16ef48e36247") : secret "router-metrics-certs-default" not found
Sep 17 17:36:24.288354 ip-10-0-143-74 kubenswrapper[2575]: E0917 17:36:24.288301    2575 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/3a6e39e9-229a-440e-a565-16ef48e36247-service-ca-bundle podName:3a6e39e9-229a-440e-a565-16ef48e36247 nodeName:}" failed. No retries permitted until 2026-09-17 17:36:24.78827707 +0000 UTC m=+104.418607063 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/3a6e39e9-229a-440e-a565-16ef48e36247-service-ca-bundle") pod "router-default-76f78f8976-kwv2h" (UID: "3a6e39e9-229a-440e-a565-16ef48e36247") : configmap references non-existent config key: service-ca.crt
Sep 17 17:36:24.290274 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:36:24.290251    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"stats-auth\" (UniqueName: \"kubernetes.io/secret/3a6e39e9-229a-440e-a565-16ef48e36247-stats-auth\") pod \"router-default-76f78f8976-kwv2h\" (UID: \"3a6e39e9-229a-440e-a565-16ef48e36247\") " pod="openshift-ingress/router-default-76f78f8976-kwv2h"
Sep 17 17:36:24.290529 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:36:24.290510    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"default-certificate\" (UniqueName: \"kubernetes.io/secret/3a6e39e9-229a-440e-a565-16ef48e36247-default-certificate\") pod \"router-default-76f78f8976-kwv2h\" (UID: \"3a6e39e9-229a-440e-a565-16ef48e36247\") " pod="openshift-ingress/router-default-76f78f8976-kwv2h"
Sep 17 17:36:24.303904 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:36:24.303863    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-qnvvb\" (UniqueName: \"kubernetes.io/projected/3a6e39e9-229a-440e-a565-16ef48e36247-kube-api-access-qnvvb\") pod \"router-default-76f78f8976-kwv2h\" (UID: \"3a6e39e9-229a-440e-a565-16ef48e36247\") " pod="openshift-ingress/router-default-76f78f8976-kwv2h"
Sep 17 17:36:24.313762 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:36:24.313729    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-djvzt_1d07e68d-9943-420f-a6a0-5f477792fabd/konnectivity-agent/0.log" limit=1024
Sep 17 17:36:24.492804 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:36:24.492667    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-operator-6f9c644dd9-dllkw"]
Sep 17 17:36:24.496712 ip-10-0-143-74 kubenswrapper[2575]: W0917 17:36:24.496684    2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb253f0e2_6a5b_4fc4_b686_db5ae716f81b.slice/crio-3db2d6e83f9dabfbf3522b2ab76eaa1b84e2ba197bc76ddc18ecdf8d0ac67d8f WatchSource:0}: Error finding container 3db2d6e83f9dabfbf3522b2ab76eaa1b84e2ba197bc76ddc18ecdf8d0ac67d8f: Status 404 returned error can't find the container with id 3db2d6e83f9dabfbf3522b2ab76eaa1b84e2ba197bc76ddc18ecdf8d0ac67d8f
Sep 17 17:36:24.791123 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:36:24.791041    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/3a6e39e9-229a-440e-a565-16ef48e36247-metrics-certs\") pod \"router-default-76f78f8976-kwv2h\" (UID: \"3a6e39e9-229a-440e-a565-16ef48e36247\") " pod="openshift-ingress/router-default-76f78f8976-kwv2h"
Sep 17 17:36:24.791123 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:36:24.791079    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/3a6e39e9-229a-440e-a565-16ef48e36247-service-ca-bundle\") pod \"router-default-76f78f8976-kwv2h\" (UID: \"3a6e39e9-229a-440e-a565-16ef48e36247\") " pod="openshift-ingress/router-default-76f78f8976-kwv2h"
Sep 17 17:36:24.791292 ip-10-0-143-74 kubenswrapper[2575]: E0917 17:36:24.791202    2575 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found
Sep 17 17:36:24.791292 ip-10-0-143-74 kubenswrapper[2575]: E0917 17:36:24.791217    2575 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/3a6e39e9-229a-440e-a565-16ef48e36247-service-ca-bundle podName:3a6e39e9-229a-440e-a565-16ef48e36247 nodeName:}" failed. No retries permitted until 2026-09-17 17:36:25.791204778 +0000 UTC m=+105.421534745 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/3a6e39e9-229a-440e-a565-16ef48e36247-service-ca-bundle") pod "router-default-76f78f8976-kwv2h" (UID: "3a6e39e9-229a-440e-a565-16ef48e36247") : configmap references non-existent config key: service-ca.crt
Sep 17 17:36:24.791292 ip-10-0-143-74 kubenswrapper[2575]: E0917 17:36:24.791258    2575 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/3a6e39e9-229a-440e-a565-16ef48e36247-metrics-certs podName:3a6e39e9-229a-440e-a565-16ef48e36247 nodeName:}" failed. No retries permitted until 2026-09-17 17:36:25.791244389 +0000 UTC m=+105.421574342 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/3a6e39e9-229a-440e-a565-16ef48e36247-metrics-certs") pod "router-default-76f78f8976-kwv2h" (UID: "3a6e39e9-229a-440e-a565-16ef48e36247") : secret "router-metrics-certs-default" not found
Sep 17 17:36:25.449396 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:36:25.449361    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-operator-6f9c644dd9-dllkw" event={"ID":"b253f0e2-6a5b-4fc4-b686-db5ae716f81b","Type":"ContainerStarted","Data":"3db2d6e83f9dabfbf3522b2ab76eaa1b84e2ba197bc76ddc18ecdf8d0ac67d8f"}
Sep 17 17:36:25.652685 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:36:25.652657    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-djvzt_1d07e68d-9943-420f-a6a0-5f477792fabd/konnectivity-agent/0.log" limit=1024
Sep 17 17:36:25.799415 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:36:25.799337    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/3a6e39e9-229a-440e-a565-16ef48e36247-metrics-certs\") pod \"router-default-76f78f8976-kwv2h\" (UID: \"3a6e39e9-229a-440e-a565-16ef48e36247\") " pod="openshift-ingress/router-default-76f78f8976-kwv2h"
Sep 17 17:36:25.799415 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:36:25.799376    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/3a6e39e9-229a-440e-a565-16ef48e36247-service-ca-bundle\") pod \"router-default-76f78f8976-kwv2h\" (UID: \"3a6e39e9-229a-440e-a565-16ef48e36247\") " pod="openshift-ingress/router-default-76f78f8976-kwv2h"
Sep 17 17:36:25.799666 ip-10-0-143-74 kubenswrapper[2575]: E0917 17:36:25.799524    2575 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found
Sep 17 17:36:25.799666 ip-10-0-143-74 kubenswrapper[2575]: E0917 17:36:25.799532    2575 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/3a6e39e9-229a-440e-a565-16ef48e36247-service-ca-bundle podName:3a6e39e9-229a-440e-a565-16ef48e36247 nodeName:}" failed. No retries permitted until 2026-09-17 17:36:27.79951754 +0000 UTC m=+107.429847494 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/3a6e39e9-229a-440e-a565-16ef48e36247-service-ca-bundle") pod "router-default-76f78f8976-kwv2h" (UID: "3a6e39e9-229a-440e-a565-16ef48e36247") : configmap references non-existent config key: service-ca.crt
Sep 17 17:36:25.799666 ip-10-0-143-74 kubenswrapper[2575]: E0917 17:36:25.799600    2575 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/3a6e39e9-229a-440e-a565-16ef48e36247-metrics-certs podName:3a6e39e9-229a-440e-a565-16ef48e36247 nodeName:}" failed. No retries permitted until 2026-09-17 17:36:27.799582152 +0000 UTC m=+107.429912121 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/3a6e39e9-229a-440e-a565-16ef48e36247-metrics-certs") pod "router-default-76f78f8976-kwv2h" (UID: "3a6e39e9-229a-440e-a565-16ef48e36247") : secret "router-metrics-certs-default" not found
Sep 17 17:36:26.509799 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:36:26.507377    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-djvzt_1d07e68d-9943-420f-a6a0-5f477792fabd/konnectivity-agent/0.log" limit=1024
Sep 17 17:36:26.607097 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:36:26.607061    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-djvzt_1d07e68d-9943-420f-a6a0-5f477792fabd/konnectivity-agent/0.log" limit=1024
Sep 17 17:36:27.455164 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:36:27.455131    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-operator-6f9c644dd9-dllkw" event={"ID":"b253f0e2-6a5b-4fc4-b686-db5ae716f81b","Type":"ContainerStarted","Data":"ca7d12df0b0cd8b2fc7243791ba52efc31467b63015346df2f8c326f8a94e9f2"}
Sep 17 17:36:27.472476 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:36:27.472409    2575 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-insights/insights-operator-6f9c644dd9-dllkw" podStartSLOduration=2.503629526 podStartE2EDuration="4.472393892s" totalImagesPullingTime="1.968764366s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-17 17:36:23 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-17 17:36:27.4722114 +0000 UTC m=+107.102541374" watchObservedRunningTime="2026-09-17 17:36:27.472393892 +0000 UTC m=+107.102723867"
Sep 17 17:36:27.814217 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:36:27.814144    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/3a6e39e9-229a-440e-a565-16ef48e36247-metrics-certs\") pod \"router-default-76f78f8976-kwv2h\" (UID: \"3a6e39e9-229a-440e-a565-16ef48e36247\") " pod="openshift-ingress/router-default-76f78f8976-kwv2h"
Sep 17 17:36:27.814217 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:36:27.814177    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/3a6e39e9-229a-440e-a565-16ef48e36247-service-ca-bundle\") pod \"router-default-76f78f8976-kwv2h\" (UID: \"3a6e39e9-229a-440e-a565-16ef48e36247\") " pod="openshift-ingress/router-default-76f78f8976-kwv2h"
Sep 17 17:36:27.814401 ip-10-0-143-74 kubenswrapper[2575]: E0917 17:36:27.814294    2575 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found
Sep 17 17:36:27.814401 ip-10-0-143-74 kubenswrapper[2575]: E0917 17:36:27.814300    2575 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/3a6e39e9-229a-440e-a565-16ef48e36247-service-ca-bundle podName:3a6e39e9-229a-440e-a565-16ef48e36247 nodeName:}" failed. No retries permitted until 2026-09-17 17:36:31.814287379 +0000 UTC m=+111.444617332 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/3a6e39e9-229a-440e-a565-16ef48e36247-service-ca-bundle") pod "router-default-76f78f8976-kwv2h" (UID: "3a6e39e9-229a-440e-a565-16ef48e36247") : configmap references non-existent config key: service-ca.crt
Sep 17 17:36:27.814401 ip-10-0-143-74 kubenswrapper[2575]: E0917 17:36:27.814364    2575 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/3a6e39e9-229a-440e-a565-16ef48e36247-metrics-certs podName:3a6e39e9-229a-440e-a565-16ef48e36247 nodeName:}" failed. No retries permitted until 2026-09-17 17:36:31.814349963 +0000 UTC m=+111.444679916 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/3a6e39e9-229a-440e-a565-16ef48e36247-metrics-certs") pod "router-default-76f78f8976-kwv2h" (UID: "3a6e39e9-229a-440e-a565-16ef48e36247") : secret "router-metrics-certs-default" not found
Sep 17 17:36:27.844201 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:36:27.844165    2575 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-insights/insights-runtime-extractor-gspvq"]
Sep 17 17:36:27.846063 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:36:27.846047    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-gspvq"
Sep 17 17:36:27.848404 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:36:27.848382    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-tls\""
Sep 17 17:36:27.848662 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:36:27.848407    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-sa-dockercfg-4ktmh\""
Sep 17 17:36:27.848769 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:36:27.848740    2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-rbac-proxy\""
Sep 17 17:36:27.860528 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:36:27.860502    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-gspvq"]
Sep 17 17:36:27.861452 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:36:27.861420    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-djvzt_1d07e68d-9943-420f-a6a0-5f477792fabd/konnectivity-agent/0.log" limit=1024
Sep 17 17:36:27.915335 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:36:27.915304    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-np4ms\" (UniqueName: \"kubernetes.io/projected/1e261d7f-9daa-43ed-84f0-8d4d2aea69d7-kube-api-access-np4ms\") pod \"insights-runtime-extractor-gspvq\" (UID: \"1e261d7f-9daa-43ed-84f0-8d4d2aea69d7\") " pod="openshift-insights/insights-runtime-extractor-gspvq"
Sep 17 17:36:27.915531 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:36:27.915343    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/1e261d7f-9daa-43ed-84f0-8d4d2aea69d7-crio-socket\") pod \"insights-runtime-extractor-gspvq\" (UID: \"1e261d7f-9daa-43ed-84f0-8d4d2aea69d7\") " pod="openshift-insights/insights-runtime-extractor-gspvq"
Sep 17 17:36:27.915531 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:36:27.915366    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/1e261d7f-9daa-43ed-84f0-8d4d2aea69d7-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-gspvq\" (UID: \"1e261d7f-9daa-43ed-84f0-8d4d2aea69d7\") " pod="openshift-insights/insights-runtime-extractor-gspvq"
Sep 17 17:36:27.915531 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:36:27.915469    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/1e261d7f-9daa-43ed-84f0-8d4d2aea69d7-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-gspvq\" (UID: \"1e261d7f-9daa-43ed-84f0-8d4d2aea69d7\") " pod="openshift-insights/insights-runtime-extractor-gspvq"
Sep 17 17:36:27.915678 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:36:27.915571    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/1e261d7f-9daa-43ed-84f0-8d4d2aea69d7-data-volume\") pod \"insights-runtime-extractor-gspvq\" (UID: \"1e261d7f-9daa-43ed-84f0-8d4d2aea69d7\") " pod="openshift-insights/insights-runtime-extractor-gspvq"
Sep 17 17:36:27.984155 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:36:27.984114    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-djvzt_1d07e68d-9943-420f-a6a0-5f477792fabd/konnectivity-agent/0.log" limit=1024
Sep 17 17:36:28.016580 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:36:28.016548    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/1e261d7f-9daa-43ed-84f0-8d4d2aea69d7-data-volume\") pod \"insights-runtime-extractor-gspvq\" (UID: \"1e261d7f-9daa-43ed-84f0-8d4d2aea69d7\") " pod="openshift-insights/insights-runtime-extractor-gspvq"
Sep 17 17:36:28.016742 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:36:28.016594    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-np4ms\" (UniqueName: \"kubernetes.io/projected/1e261d7f-9daa-43ed-84f0-8d4d2aea69d7-kube-api-access-np4ms\") pod \"insights-runtime-extractor-gspvq\" (UID: \"1e261d7f-9daa-43ed-84f0-8d4d2aea69d7\") " pod="openshift-insights/insights-runtime-extractor-gspvq"
Sep 17 17:36:28.016742 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:36:28.016719    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/1e261d7f-9daa-43ed-84f0-8d4d2aea69d7-crio-socket\") pod \"insights-runtime-extractor-gspvq\" (UID: \"1e261d7f-9daa-43ed-84f0-8d4d2aea69d7\") " pod="openshift-insights/insights-runtime-extractor-gspvq"
Sep 17 17:36:28.016833 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:36:28.016757    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/1e261d7f-9daa-43ed-84f0-8d4d2aea69d7-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-gspvq\" (UID: \"1e261d7f-9daa-43ed-84f0-8d4d2aea69d7\") " pod="openshift-insights/insights-runtime-extractor-gspvq"
Sep 17 17:36:28.016867 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:36:28.016833    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/1e261d7f-9daa-43ed-84f0-8d4d2aea69d7-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-gspvq\" (UID: \"1e261d7f-9daa-43ed-84f0-8d4d2aea69d7\") " pod="openshift-insights/insights-runtime-extractor-gspvq"
Sep 17 17:36:28.016867 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:36:28.016839    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/1e261d7f-9daa-43ed-84f0-8d4d2aea69d7-crio-socket\") pod \"insights-runtime-extractor-gspvq\" (UID: \"1e261d7f-9daa-43ed-84f0-8d4d2aea69d7\") " pod="openshift-insights/insights-runtime-extractor-gspvq"
Sep 17 17:36:28.016924 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:36:28.016890    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/1e261d7f-9daa-43ed-84f0-8d4d2aea69d7-data-volume\") pod \"insights-runtime-extractor-gspvq\" (UID: \"1e261d7f-9daa-43ed-84f0-8d4d2aea69d7\") " pod="openshift-insights/insights-runtime-extractor-gspvq"
Sep 17 17:36:28.016924 ip-10-0-143-74 kubenswrapper[2575]: E0917 17:36:28.016916    2575 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Sep 17 17:36:28.016987 ip-10-0-143-74 kubenswrapper[2575]: E0917 17:36:28.016962    2575 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/1e261d7f-9daa-43ed-84f0-8d4d2aea69d7-kube-rbac-proxy-cm podName:1e261d7f-9daa-43ed-84f0-8d4d2aea69d7 nodeName:}" failed. No retries permitted until 2026-09-17 17:36:28.516949781 +0000 UTC m=+108.147279735 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/1e261d7f-9daa-43ed-84f0-8d4d2aea69d7-kube-rbac-proxy-cm") pod "insights-runtime-extractor-gspvq" (UID: "1e261d7f-9daa-43ed-84f0-8d4d2aea69d7") : configmap "kube-rbac-proxy" not found
Sep 17 17:36:28.017070 ip-10-0-143-74 kubenswrapper[2575]: E0917 17:36:28.017053    2575 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Sep 17 17:36:28.017112 ip-10-0-143-74 kubenswrapper[2575]: E0917 17:36:28.017103    2575 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/1e261d7f-9daa-43ed-84f0-8d4d2aea69d7-insights-runtime-extractor-tls podName:1e261d7f-9daa-43ed-84f0-8d4d2aea69d7 nodeName:}" failed. No retries permitted until 2026-09-17 17:36:28.517090967 +0000 UTC m=+108.147420924 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/1e261d7f-9daa-43ed-84f0-8d4d2aea69d7-insights-runtime-extractor-tls") pod "insights-runtime-extractor-gspvq" (UID: "1e261d7f-9daa-43ed-84f0-8d4d2aea69d7") : secret "insights-runtime-extractor-tls" not found
Sep 17 17:36:28.025355 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:36:28.025324    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-np4ms\" (UniqueName: \"kubernetes.io/projected/1e261d7f-9daa-43ed-84f0-8d4d2aea69d7-kube-api-access-np4ms\") pod \"insights-runtime-extractor-gspvq\" (UID: \"1e261d7f-9daa-43ed-84f0-8d4d2aea69d7\") " pod="openshift-insights/insights-runtime-extractor-gspvq"
Sep 17 17:36:28.520729 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:36:28.520688    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/1e261d7f-9daa-43ed-84f0-8d4d2aea69d7-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-gspvq\" (UID: \"1e261d7f-9daa-43ed-84f0-8d4d2aea69d7\") " pod="openshift-insights/insights-runtime-extractor-gspvq"
Sep 17 17:36:28.521105 ip-10-0-143-74 kubenswrapper[2575]: E0917 17:36:28.520852    2575 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Sep 17 17:36:28.521105 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:36:28.520929    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/1e261d7f-9daa-43ed-84f0-8d4d2aea69d7-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-gspvq\" (UID: \"1e261d7f-9daa-43ed-84f0-8d4d2aea69d7\") " pod="openshift-insights/insights-runtime-extractor-gspvq"
Sep 17 17:36:28.521105 ip-10-0-143-74 kubenswrapper[2575]: E0917 17:36:28.520980    2575 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Sep 17 17:36:28.521105 ip-10-0-143-74 kubenswrapper[2575]: E0917 17:36:28.521010    2575 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/1e261d7f-9daa-43ed-84f0-8d4d2aea69d7-insights-runtime-extractor-tls podName:1e261d7f-9daa-43ed-84f0-8d4d2aea69d7 nodeName:}" failed. No retries permitted until 2026-09-17 17:36:29.520987184 +0000 UTC m=+109.151317138 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/1e261d7f-9daa-43ed-84f0-8d4d2aea69d7-insights-runtime-extractor-tls") pod "insights-runtime-extractor-gspvq" (UID: "1e261d7f-9daa-43ed-84f0-8d4d2aea69d7") : secret "insights-runtime-extractor-tls" not found
Sep 17 17:36:28.521261 ip-10-0-143-74 kubenswrapper[2575]: E0917 17:36:28.521102    2575 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/1e261d7f-9daa-43ed-84f0-8d4d2aea69d7-kube-rbac-proxy-cm podName:1e261d7f-9daa-43ed-84f0-8d4d2aea69d7 nodeName:}" failed. No retries permitted until 2026-09-17 17:36:29.521087474 +0000 UTC m=+109.151417435 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/1e261d7f-9daa-43ed-84f0-8d4d2aea69d7-kube-rbac-proxy-cm") pod "insights-runtime-extractor-gspvq" (UID: "1e261d7f-9daa-43ed-84f0-8d4d2aea69d7") : configmap "kube-rbac-proxy" not found
Sep 17 17:36:29.529570 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:36:29.529516    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/1e261d7f-9daa-43ed-84f0-8d4d2aea69d7-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-gspvq\" (UID: \"1e261d7f-9daa-43ed-84f0-8d4d2aea69d7\") " pod="openshift-insights/insights-runtime-extractor-gspvq"
Sep 17 17:36:29.529570 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:36:29.529585    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/1e261d7f-9daa-43ed-84f0-8d4d2aea69d7-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-gspvq\" (UID: \"1e261d7f-9daa-43ed-84f0-8d4d2aea69d7\") " pod="openshift-insights/insights-runtime-extractor-gspvq"
Sep 17 17:36:29.530136 ip-10-0-143-74 kubenswrapper[2575]: E0917 17:36:29.529662    2575 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Sep 17 17:36:29.530136 ip-10-0-143-74 kubenswrapper[2575]: E0917 17:36:29.529691    2575 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Sep 17 17:36:29.530136 ip-10-0-143-74 kubenswrapper[2575]: E0917 17:36:29.529724    2575 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/1e261d7f-9daa-43ed-84f0-8d4d2aea69d7-kube-rbac-proxy-cm podName:1e261d7f-9daa-43ed-84f0-8d4d2aea69d7 nodeName:}" failed. No retries permitted until 2026-09-17 17:36:31.529707377 +0000 UTC m=+111.160037356 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/1e261d7f-9daa-43ed-84f0-8d4d2aea69d7-kube-rbac-proxy-cm") pod "insights-runtime-extractor-gspvq" (UID: "1e261d7f-9daa-43ed-84f0-8d4d2aea69d7") : configmap "kube-rbac-proxy" not found
Sep 17 17:36:29.530136 ip-10-0-143-74 kubenswrapper[2575]: E0917 17:36:29.529756    2575 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/1e261d7f-9daa-43ed-84f0-8d4d2aea69d7-insights-runtime-extractor-tls podName:1e261d7f-9daa-43ed-84f0-8d4d2aea69d7 nodeName:}" failed. No retries permitted until 2026-09-17 17:36:31.529736377 +0000 UTC m=+111.160066338 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/1e261d7f-9daa-43ed-84f0-8d4d2aea69d7-insights-runtime-extractor-tls") pod "insights-runtime-extractor-gspvq" (UID: "1e261d7f-9daa-43ed-84f0-8d4d2aea69d7") : secret "insights-runtime-extractor-tls" not found
Sep 17 17:36:29.598661 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:36:29.598633    2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-5549b_93fd2156-b185-4995-8e49-d6d9c0eda255/dns-node-resolver/0.log"
Sep 17 17:36:30.599144 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:36:30.599117    2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-2fp6z_173c8902-414c-4212-8d93-ce15853e8ca8/node-ca/0.log"
Sep 17 17:36:31.160725 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:36:31.160683    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-djvzt_1d07e68d-9943-420f-a6a0-5f477792fabd/konnectivity-agent/0.log" limit=1024
Sep 17 17:36:31.160895 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:36:31.160785    2575 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-kube-storage-version-migrator/migrator-76794cbd6c-mm4dm"]
Sep 17 17:36:31.162706 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:36:31.162691    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-kube-storage-version-migrator/migrator-76794cbd6c-mm4dm"
Sep 17 17:36:31.165657 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:36:31.165637    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-kube-storage-version-migrator\"/\"kube-storage-version-migrator-sa-dockercfg-pdq8f\""
Sep 17 17:36:31.166880 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:36:31.166861    2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator\"/\"kube-root-ca.crt\""
Sep 17 17:36:31.166977 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:36:31.166886    2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator\"/\"openshift-service-ca.crt\""
Sep 17 17:36:31.175597 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:36:31.175575    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-kube-storage-version-migrator/migrator-76794cbd6c-mm4dm"]
Sep 17 17:36:31.243495 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:36:31.243457    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-spzr6\" (UniqueName: \"kubernetes.io/projected/66bf1996-7788-4880-a36d-4efa474cdb85-kube-api-access-spzr6\") pod \"migrator-76794cbd6c-mm4dm\" (UID: \"66bf1996-7788-4880-a36d-4efa474cdb85\") " pod="openshift-kube-storage-version-migrator/migrator-76794cbd6c-mm4dm"
Sep 17 17:36:31.254412 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:36:31.254379    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-djvzt_1d07e68d-9943-420f-a6a0-5f477792fabd/konnectivity-agent/0.log" limit=1024
Sep 17 17:36:31.322901 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:36:31.322864    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-djvzt_1d07e68d-9943-420f-a6a0-5f477792fabd/konnectivity-agent/0.log" limit=1024
Sep 17 17:36:31.344902 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:36:31.344871    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-spzr6\" (UniqueName: \"kubernetes.io/projected/66bf1996-7788-4880-a36d-4efa474cdb85-kube-api-access-spzr6\") pod \"migrator-76794cbd6c-mm4dm\" (UID: \"66bf1996-7788-4880-a36d-4efa474cdb85\") " pod="openshift-kube-storage-version-migrator/migrator-76794cbd6c-mm4dm"
Sep 17 17:36:31.354435 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:36:31.354406    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-spzr6\" (UniqueName: \"kubernetes.io/projected/66bf1996-7788-4880-a36d-4efa474cdb85-kube-api-access-spzr6\") pod \"migrator-76794cbd6c-mm4dm\" (UID: \"66bf1996-7788-4880-a36d-4efa474cdb85\") " pod="openshift-kube-storage-version-migrator/migrator-76794cbd6c-mm4dm"
Sep 17 17:36:31.400628 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:36:31.400578    2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-4p2w5_cacfc0cf-465f-43c5-bfe6-dd4934eb4a6f/serve-healthcheck-canary/0.log"
Sep 17 17:36:31.471145 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:36:31.471063    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-kube-storage-version-migrator/migrator-76794cbd6c-mm4dm"
Sep 17 17:36:31.545944 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:36:31.545909    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/1e261d7f-9daa-43ed-84f0-8d4d2aea69d7-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-gspvq\" (UID: \"1e261d7f-9daa-43ed-84f0-8d4d2aea69d7\") " pod="openshift-insights/insights-runtime-extractor-gspvq"
Sep 17 17:36:31.546104 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:36:31.545966    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/1e261d7f-9daa-43ed-84f0-8d4d2aea69d7-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-gspvq\" (UID: \"1e261d7f-9daa-43ed-84f0-8d4d2aea69d7\") " pod="openshift-insights/insights-runtime-extractor-gspvq"
Sep 17 17:36:31.546104 ip-10-0-143-74 kubenswrapper[2575]: E0917 17:36:31.546039    2575 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Sep 17 17:36:31.546104 ip-10-0-143-74 kubenswrapper[2575]: E0917 17:36:31.546074    2575 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Sep 17 17:36:31.546104 ip-10-0-143-74 kubenswrapper[2575]: E0917 17:36:31.546099    2575 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/1e261d7f-9daa-43ed-84f0-8d4d2aea69d7-kube-rbac-proxy-cm podName:1e261d7f-9daa-43ed-84f0-8d4d2aea69d7 nodeName:}" failed. No retries permitted until 2026-09-17 17:36:35.546085328 +0000 UTC m=+115.176415281 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/1e261d7f-9daa-43ed-84f0-8d4d2aea69d7-kube-rbac-proxy-cm") pod "insights-runtime-extractor-gspvq" (UID: "1e261d7f-9daa-43ed-84f0-8d4d2aea69d7") : configmap "kube-rbac-proxy" not found
Sep 17 17:36:31.546239 ip-10-0-143-74 kubenswrapper[2575]: E0917 17:36:31.546129    2575 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/1e261d7f-9daa-43ed-84f0-8d4d2aea69d7-insights-runtime-extractor-tls podName:1e261d7f-9daa-43ed-84f0-8d4d2aea69d7 nodeName:}" failed. No retries permitted until 2026-09-17 17:36:35.54611524 +0000 UTC m=+115.176445229 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/1e261d7f-9daa-43ed-84f0-8d4d2aea69d7-insights-runtime-extractor-tls") pod "insights-runtime-extractor-gspvq" (UID: "1e261d7f-9daa-43ed-84f0-8d4d2aea69d7") : secret "insights-runtime-extractor-tls" not found
Sep 17 17:36:31.740714 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:36:31.740627    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-kube-storage-version-migrator/migrator-76794cbd6c-mm4dm"]
Sep 17 17:36:31.744750 ip-10-0-143-74 kubenswrapper[2575]: W0917 17:36:31.744723    2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod66bf1996_7788_4880_a36d_4efa474cdb85.slice/crio-09e4d54d4d6aeb4d4fb535a4cda1156569005642bf97948765453f1fb85bdce6 WatchSource:0}: Error finding container 09e4d54d4d6aeb4d4fb535a4cda1156569005642bf97948765453f1fb85bdce6: Status 404 returned error can't find the container with id 09e4d54d4d6aeb4d4fb535a4cda1156569005642bf97948765453f1fb85bdce6
Sep 17 17:36:31.847404 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:36:31.847362    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/3a6e39e9-229a-440e-a565-16ef48e36247-metrics-certs\") pod \"router-default-76f78f8976-kwv2h\" (UID: \"3a6e39e9-229a-440e-a565-16ef48e36247\") " pod="openshift-ingress/router-default-76f78f8976-kwv2h"
Sep 17 17:36:31.847404 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:36:31.847402    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/3a6e39e9-229a-440e-a565-16ef48e36247-service-ca-bundle\") pod \"router-default-76f78f8976-kwv2h\" (UID: \"3a6e39e9-229a-440e-a565-16ef48e36247\") " pod="openshift-ingress/router-default-76f78f8976-kwv2h"
Sep 17 17:36:31.847636 ip-10-0-143-74 kubenswrapper[2575]: E0917 17:36:31.847527    2575 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found
Sep 17 17:36:31.847636 ip-10-0-143-74 kubenswrapper[2575]: E0917 17:36:31.847546    2575 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/3a6e39e9-229a-440e-a565-16ef48e36247-service-ca-bundle podName:3a6e39e9-229a-440e-a565-16ef48e36247 nodeName:}" failed. No retries permitted until 2026-09-17 17:36:39.847533225 +0000 UTC m=+119.477863179 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/3a6e39e9-229a-440e-a565-16ef48e36247-service-ca-bundle") pod "router-default-76f78f8976-kwv2h" (UID: "3a6e39e9-229a-440e-a565-16ef48e36247") : configmap references non-existent config key: service-ca.crt
Sep 17 17:36:31.847636 ip-10-0-143-74 kubenswrapper[2575]: E0917 17:36:31.847583    2575 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/3a6e39e9-229a-440e-a565-16ef48e36247-metrics-certs podName:3a6e39e9-229a-440e-a565-16ef48e36247 nodeName:}" failed. No retries permitted until 2026-09-17 17:36:39.8475701 +0000 UTC m=+119.477900058 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/3a6e39e9-229a-440e-a565-16ef48e36247-metrics-certs") pod "router-default-76f78f8976-kwv2h" (UID: "3a6e39e9-229a-440e-a565-16ef48e36247") : secret "router-metrics-certs-default" not found
Sep 17 17:36:32.469040 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:36:32.469010    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator/migrator-76794cbd6c-mm4dm" event={"ID":"66bf1996-7788-4880-a36d-4efa474cdb85","Type":"ContainerStarted","Data":"09e4d54d4d6aeb4d4fb535a4cda1156569005642bf97948765453f1fb85bdce6"}
Sep 17 17:36:32.764546 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:36:32.764506    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-djvzt_1d07e68d-9943-420f-a6a0-5f477792fabd/konnectivity-agent/0.log" limit=1024
Sep 17 17:36:32.865964 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:36:32.865921    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-djvzt_1d07e68d-9943-420f-a6a0-5f477792fabd/konnectivity-agent/0.log" limit=1024
Sep 17 17:36:33.213145 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:36:33.213104    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-djvzt_1d07e68d-9943-420f-a6a0-5f477792fabd/konnectivity-agent/0.log" limit=1024
Sep 17 17:36:33.309940 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:36:33.309911    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-djvzt_1d07e68d-9943-420f-a6a0-5f477792fabd/konnectivity-agent/0.log" limit=1024
Sep 17 17:36:33.472648 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:36:33.472558    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-djvzt_1d07e68d-9943-420f-a6a0-5f477792fabd/konnectivity-agent/0.log" limit=1024
Sep 17 17:36:33.474275 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:36:33.474251    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator/migrator-76794cbd6c-mm4dm" event={"ID":"66bf1996-7788-4880-a36d-4efa474cdb85","Type":"ContainerStarted","Data":"8d07dc5c15551950da0c7f03119822be72f90d0c6eb53b19e8dcc294555c4723"}
Sep 17 17:36:33.474392 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:36:33.474296    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator/migrator-76794cbd6c-mm4dm" event={"ID":"66bf1996-7788-4880-a36d-4efa474cdb85","Type":"ContainerStarted","Data":"b7bcc4c95afe3969704f95e11f85fdb71927f2e790a428a352599314f1e02217"}
Sep 17 17:36:33.492927 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:36:33.492877    2575 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-kube-storage-version-migrator/migrator-76794cbd6c-mm4dm" podStartSLOduration=1.5582201850000001 podStartE2EDuration="2.492862697s" totalImagesPullingTime="934.642512ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-17 17:36:31 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-17 17:36:33.492272315 +0000 UTC m=+113.122602287" watchObservedRunningTime="2026-09-17 17:36:33.492862697 +0000 UTC m=+113.123192672"
Sep 17 17:36:33.577495 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:36:33.577465    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-djvzt_1d07e68d-9943-420f-a6a0-5f477792fabd/konnectivity-agent/0.log" limit=1024
Sep 17 17:36:33.674784 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:36:33.674735    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-djvzt_1d07e68d-9943-420f-a6a0-5f477792fabd/konnectivity-agent/0.log" limit=1024
Sep 17 17:36:35.074193 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:36:35.074154    2575 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-service-ca/service-ca-55f6fc7b86-5dqkh"]
Sep 17 17:36:35.076147 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:36:35.076126    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-service-ca/service-ca-55f6fc7b86-5dqkh"
Sep 17 17:36:35.079155 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:36:35.079133    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-service-ca\"/\"service-ca-dockercfg-6h95f\""
Sep 17 17:36:35.079257 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:36:35.079136    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-service-ca\"/\"signing-key\""
Sep 17 17:36:35.079562 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:36:35.079541    2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca\"/\"kube-root-ca.crt\""
Sep 17 17:36:35.079649 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:36:35.079581    2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca\"/\"openshift-service-ca.crt\""
Sep 17 17:36:35.079649 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:36:35.079632    2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca\"/\"signing-cabundle\""
Sep 17 17:36:35.086389 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:36:35.086371    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-service-ca/service-ca-55f6fc7b86-5dqkh"]
Sep 17 17:36:35.107320 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:36:35.107285    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-djvzt_1d07e68d-9943-420f-a6a0-5f477792fabd/konnectivity-agent/0.log" limit=1024
Sep 17 17:36:35.169732 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:36:35.169694    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-djvzt_1d07e68d-9943-420f-a6a0-5f477792fabd/konnectivity-agent/0.log" limit=1024
Sep 17 17:36:35.173711 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:36:35.173688    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"signing-key\" (UniqueName: \"kubernetes.io/secret/c458b447-edad-4cdb-993d-5514a5278b8e-signing-key\") pod \"service-ca-55f6fc7b86-5dqkh\" (UID: \"c458b447-edad-4cdb-993d-5514a5278b8e\") " pod="openshift-service-ca/service-ca-55f6fc7b86-5dqkh"
Sep 17 17:36:35.173799 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:36:35.173736    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"signing-cabundle\" (UniqueName: \"kubernetes.io/configmap/c458b447-edad-4cdb-993d-5514a5278b8e-signing-cabundle\") pod \"service-ca-55f6fc7b86-5dqkh\" (UID: \"c458b447-edad-4cdb-993d-5514a5278b8e\") " pod="openshift-service-ca/service-ca-55f6fc7b86-5dqkh"
Sep 17 17:36:35.173799 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:36:35.173757    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mfkf8\" (UniqueName: \"kubernetes.io/projected/c458b447-edad-4cdb-993d-5514a5278b8e-kube-api-access-mfkf8\") pod \"service-ca-55f6fc7b86-5dqkh\" (UID: \"c458b447-edad-4cdb-993d-5514a5278b8e\") " pod="openshift-service-ca/service-ca-55f6fc7b86-5dqkh"
Sep 17 17:36:35.274752 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:36:35.274698    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"signing-key\" (UniqueName: \"kubernetes.io/secret/c458b447-edad-4cdb-993d-5514a5278b8e-signing-key\") pod \"service-ca-55f6fc7b86-5dqkh\" (UID: \"c458b447-edad-4cdb-993d-5514a5278b8e\") " pod="openshift-service-ca/service-ca-55f6fc7b86-5dqkh"
Sep 17 17:36:35.274947 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:36:35.274779    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"signing-cabundle\" (UniqueName: \"kubernetes.io/configmap/c458b447-edad-4cdb-993d-5514a5278b8e-signing-cabundle\") pod \"service-ca-55f6fc7b86-5dqkh\" (UID: \"c458b447-edad-4cdb-993d-5514a5278b8e\") " pod="openshift-service-ca/service-ca-55f6fc7b86-5dqkh"
Sep 17 17:36:35.274947 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:36:35.274806    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mfkf8\" (UniqueName: \"kubernetes.io/projected/c458b447-edad-4cdb-993d-5514a5278b8e-kube-api-access-mfkf8\") pod \"service-ca-55f6fc7b86-5dqkh\" (UID: \"c458b447-edad-4cdb-993d-5514a5278b8e\") " pod="openshift-service-ca/service-ca-55f6fc7b86-5dqkh"
Sep 17 17:36:35.275433 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:36:35.275411    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"signing-cabundle\" (UniqueName: \"kubernetes.io/configmap/c458b447-edad-4cdb-993d-5514a5278b8e-signing-cabundle\") pod \"service-ca-55f6fc7b86-5dqkh\" (UID: \"c458b447-edad-4cdb-993d-5514a5278b8e\") " pod="openshift-service-ca/service-ca-55f6fc7b86-5dqkh"
Sep 17 17:36:35.277102 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:36:35.277084    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"signing-key\" (UniqueName: \"kubernetes.io/secret/c458b447-edad-4cdb-993d-5514a5278b8e-signing-key\") pod \"service-ca-55f6fc7b86-5dqkh\" (UID: \"c458b447-edad-4cdb-993d-5514a5278b8e\") " pod="openshift-service-ca/service-ca-55f6fc7b86-5dqkh"
Sep 17 17:36:35.283382 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:36:35.283361    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-mfkf8\" (UniqueName: \"kubernetes.io/projected/c458b447-edad-4cdb-993d-5514a5278b8e-kube-api-access-mfkf8\") pod \"service-ca-55f6fc7b86-5dqkh\" (UID: \"c458b447-edad-4cdb-993d-5514a5278b8e\") " pod="openshift-service-ca/service-ca-55f6fc7b86-5dqkh"
Sep 17 17:36:35.384680 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:36:35.384644    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-service-ca/service-ca-55f6fc7b86-5dqkh"
Sep 17 17:36:35.577528 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:36:35.577494    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/1e261d7f-9daa-43ed-84f0-8d4d2aea69d7-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-gspvq\" (UID: \"1e261d7f-9daa-43ed-84f0-8d4d2aea69d7\") " pod="openshift-insights/insights-runtime-extractor-gspvq"
Sep 17 17:36:35.577730 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:36:35.577561    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/1e261d7f-9daa-43ed-84f0-8d4d2aea69d7-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-gspvq\" (UID: \"1e261d7f-9daa-43ed-84f0-8d4d2aea69d7\") " pod="openshift-insights/insights-runtime-extractor-gspvq"
Sep 17 17:36:35.577730 ip-10-0-143-74 kubenswrapper[2575]: E0917 17:36:35.577642    2575 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Sep 17 17:36:35.577730 ip-10-0-143-74 kubenswrapper[2575]: E0917 17:36:35.577656    2575 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Sep 17 17:36:35.577730 ip-10-0-143-74 kubenswrapper[2575]: E0917 17:36:35.577709    2575 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/1e261d7f-9daa-43ed-84f0-8d4d2aea69d7-insights-runtime-extractor-tls podName:1e261d7f-9daa-43ed-84f0-8d4d2aea69d7 nodeName:}" failed. No retries permitted until 2026-09-17 17:36:43.577694146 +0000 UTC m=+123.208024099 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/1e261d7f-9daa-43ed-84f0-8d4d2aea69d7-insights-runtime-extractor-tls") pod "insights-runtime-extractor-gspvq" (UID: "1e261d7f-9daa-43ed-84f0-8d4d2aea69d7") : secret "insights-runtime-extractor-tls" not found
Sep 17 17:36:35.577730 ip-10-0-143-74 kubenswrapper[2575]: E0917 17:36:35.577726    2575 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/1e261d7f-9daa-43ed-84f0-8d4d2aea69d7-kube-rbac-proxy-cm podName:1e261d7f-9daa-43ed-84f0-8d4d2aea69d7 nodeName:}" failed. No retries permitted until 2026-09-17 17:36:43.577718203 +0000 UTC m=+123.208048155 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/1e261d7f-9daa-43ed-84f0-8d4d2aea69d7-kube-rbac-proxy-cm") pod "insights-runtime-extractor-gspvq" (UID: "1e261d7f-9daa-43ed-84f0-8d4d2aea69d7") : configmap "kube-rbac-proxy" not found
Sep 17 17:36:35.660640 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:36:35.660564    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-service-ca/service-ca-55f6fc7b86-5dqkh"]
Sep 17 17:36:35.664015 ip-10-0-143-74 kubenswrapper[2575]: W0917 17:36:35.663988    2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc458b447_edad_4cdb_993d_5514a5278b8e.slice/crio-97bef791658830343592e59a2ffcf9e5fb7a8475fec6ef20eba9aa2ba7d10e72 WatchSource:0}: Error finding container 97bef791658830343592e59a2ffcf9e5fb7a8475fec6ef20eba9aa2ba7d10e72: Status 404 returned error can't find the container with id 97bef791658830343592e59a2ffcf9e5fb7a8475fec6ef20eba9aa2ba7d10e72
Sep 17 17:36:36.484258 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:36:36.484221    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca/service-ca-55f6fc7b86-5dqkh" event={"ID":"c458b447-edad-4cdb-993d-5514a5278b8e","Type":"ContainerStarted","Data":"97bef791658830343592e59a2ffcf9e5fb7a8475fec6ef20eba9aa2ba7d10e72"}
Sep 17 17:36:36.683790 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:36:36.683746    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-djvzt_1d07e68d-9943-420f-a6a0-5f477792fabd/konnectivity-agent/0.log" limit=1024
Sep 17 17:36:37.488969 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:36:37.488882    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca/service-ca-55f6fc7b86-5dqkh" event={"ID":"c458b447-edad-4cdb-993d-5514a5278b8e","Type":"ContainerStarted","Data":"3f162536e4f5446dc386ffb05867c8c6ef14ca10283b3ff028f514f2b4e6a155"}
Sep 17 17:36:37.507827 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:36:37.507638    2575 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-service-ca/service-ca-55f6fc7b86-5dqkh" podStartSLOduration=0.981651406 podStartE2EDuration="2.507616888s" totalImagesPullingTime="1.525965482s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-17 17:36:35 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-17 17:36:37.507617383 +0000 UTC m=+117.137947358" watchObservedRunningTime="2026-09-17 17:36:37.507616888 +0000 UTC m=+117.137946864"
Sep 17 17:36:38.479928 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:36:38.479897    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-djvzt_1d07e68d-9943-420f-a6a0-5f477792fabd/konnectivity-agent/0.log" limit=1024
Sep 17 17:36:38.694978 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:36:38.694900    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-djvzt_1d07e68d-9943-420f-a6a0-5f477792fabd/konnectivity-agent/0.log" limit=1024
Sep 17 17:36:38.816052 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:36:38.815940    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-djvzt_1d07e68d-9943-420f-a6a0-5f477792fabd/konnectivity-agent/0.log" limit=1024
Sep 17 17:36:39.374887 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:36:39.374856    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-djvzt_1d07e68d-9943-420f-a6a0-5f477792fabd/konnectivity-agent/0.log" limit=1024
Sep 17 17:36:39.756377 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:36:39.756281    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-djvzt_1d07e68d-9943-420f-a6a0-5f477792fabd/konnectivity-agent/0.log" limit=1024
Sep 17 17:36:39.913298 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:36:39.913260    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/3a6e39e9-229a-440e-a565-16ef48e36247-metrics-certs\") pod \"router-default-76f78f8976-kwv2h\" (UID: \"3a6e39e9-229a-440e-a565-16ef48e36247\") " pod="openshift-ingress/router-default-76f78f8976-kwv2h"
Sep 17 17:36:39.913298 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:36:39.913301    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/3a6e39e9-229a-440e-a565-16ef48e36247-service-ca-bundle\") pod \"router-default-76f78f8976-kwv2h\" (UID: \"3a6e39e9-229a-440e-a565-16ef48e36247\") " pod="openshift-ingress/router-default-76f78f8976-kwv2h"
Sep 17 17:36:39.913554 ip-10-0-143-74 kubenswrapper[2575]: E0917 17:36:39.913406    2575 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found
Sep 17 17:36:39.913554 ip-10-0-143-74 kubenswrapper[2575]: E0917 17:36:39.913497    2575 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/3a6e39e9-229a-440e-a565-16ef48e36247-metrics-certs podName:3a6e39e9-229a-440e-a565-16ef48e36247 nodeName:}" failed. No retries permitted until 2026-09-17 17:36:55.913481131 +0000 UTC m=+135.543811084 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/3a6e39e9-229a-440e-a565-16ef48e36247-metrics-certs") pod "router-default-76f78f8976-kwv2h" (UID: "3a6e39e9-229a-440e-a565-16ef48e36247") : secret "router-metrics-certs-default" not found
Sep 17 17:36:39.913554 ip-10-0-143-74 kubenswrapper[2575]: E0917 17:36:39.913532    2575 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/3a6e39e9-229a-440e-a565-16ef48e36247-service-ca-bundle podName:3a6e39e9-229a-440e-a565-16ef48e36247 nodeName:}" failed. No retries permitted until 2026-09-17 17:36:55.913516806 +0000 UTC m=+135.543846764 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/3a6e39e9-229a-440e-a565-16ef48e36247-service-ca-bundle") pod "router-default-76f78f8976-kwv2h" (UID: "3a6e39e9-229a-440e-a565-16ef48e36247") : configmap references non-existent config key: service-ca.crt
Sep 17 17:36:40.368361 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:36:40.368322    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-djvzt_1d07e68d-9943-420f-a6a0-5f477792fabd/konnectivity-agent/0.log" limit=1024
Sep 17 17:36:40.965852 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:36:40.965820    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-djvzt_1d07e68d-9943-420f-a6a0-5f477792fabd/konnectivity-agent/0.log" limit=1024
Sep 17 17:36:41.069107 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:36:41.069074    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-djvzt_1d07e68d-9943-420f-a6a0-5f477792fabd/konnectivity-agent/0.log" limit=1024
Sep 17 17:36:41.171157 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:36:41.171115    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-djvzt_1d07e68d-9943-420f-a6a0-5f477792fabd/konnectivity-agent/0.log" limit=1024
Sep 17 17:36:41.364694 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:36:41.364609    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-djvzt_1d07e68d-9943-420f-a6a0-5f477792fabd/konnectivity-agent/0.log" limit=1024
Sep 17 17:36:41.553833 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:36:41.553801    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-djvzt_1d07e68d-9943-420f-a6a0-5f477792fabd/konnectivity-agent/0.log" limit=1024
Sep 17 17:36:42.066321 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:36:42.066286    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-djvzt_1d07e68d-9943-420f-a6a0-5f477792fabd/konnectivity-agent/0.log" limit=1024
Sep 17 17:36:42.164043 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:36:42.163996    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-djvzt_1d07e68d-9943-420f-a6a0-5f477792fabd/konnectivity-agent/0.log" limit=1024
Sep 17 17:36:42.359163 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:36:42.359075    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-djvzt_1d07e68d-9943-420f-a6a0-5f477792fabd/konnectivity-agent/0.log" limit=1024
Sep 17 17:36:42.745299 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:36:42.745263    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-djvzt_1d07e68d-9943-420f-a6a0-5f477792fabd/konnectivity-agent/0.log" limit=1024
Sep 17 17:36:42.963069 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:36:42.963029    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-djvzt_1d07e68d-9943-420f-a6a0-5f477792fabd/konnectivity-agent/0.log" limit=1024
Sep 17 17:36:43.389023 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:36:43.388986    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-djvzt_1d07e68d-9943-420f-a6a0-5f477792fabd/konnectivity-agent/0.log" limit=1024
Sep 17 17:36:43.605077 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:36:43.605041    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-djvzt_1d07e68d-9943-420f-a6a0-5f477792fabd/konnectivity-agent/0.log" limit=1024
Sep 17 17:36:43.644645 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:36:43.644577    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/1e261d7f-9daa-43ed-84f0-8d4d2aea69d7-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-gspvq\" (UID: \"1e261d7f-9daa-43ed-84f0-8d4d2aea69d7\") " pod="openshift-insights/insights-runtime-extractor-gspvq"
Sep 17 17:36:43.644645 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:36:43.644631    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/1e261d7f-9daa-43ed-84f0-8d4d2aea69d7-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-gspvq\" (UID: \"1e261d7f-9daa-43ed-84f0-8d4d2aea69d7\") " pod="openshift-insights/insights-runtime-extractor-gspvq"
Sep 17 17:36:43.644775 ip-10-0-143-74 kubenswrapper[2575]: E0917 17:36:43.644725    2575 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Sep 17 17:36:43.644815 ip-10-0-143-74 kubenswrapper[2575]: E0917 17:36:43.644795    2575 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/1e261d7f-9daa-43ed-84f0-8d4d2aea69d7-insights-runtime-extractor-tls podName:1e261d7f-9daa-43ed-84f0-8d4d2aea69d7 nodeName:}" failed. No retries permitted until 2026-09-17 17:36:59.6447794 +0000 UTC m=+139.275109354 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/1e261d7f-9daa-43ed-84f0-8d4d2aea69d7-insights-runtime-extractor-tls") pod "insights-runtime-extractor-gspvq" (UID: "1e261d7f-9daa-43ed-84f0-8d4d2aea69d7") : secret "insights-runtime-extractor-tls" not found
Sep 17 17:36:43.645157 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:36:43.645139    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/1e261d7f-9daa-43ed-84f0-8d4d2aea69d7-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-gspvq\" (UID: \"1e261d7f-9daa-43ed-84f0-8d4d2aea69d7\") " pod="openshift-insights/insights-runtime-extractor-gspvq"
Sep 17 17:36:43.953224 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:36:43.953140    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-djvzt_1d07e68d-9943-420f-a6a0-5f477792fabd/konnectivity-agent/0.log" limit=1024
Sep 17 17:36:44.271376 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:36:44.271287    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-djvzt_1d07e68d-9943-420f-a6a0-5f477792fabd/konnectivity-agent/0.log" limit=1024
Sep 17 17:36:44.618999 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:36:44.618971    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-djvzt_1d07e68d-9943-420f-a6a0-5f477792fabd/konnectivity-agent/0.log" limit=1024
Sep 17 17:36:44.758470 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:36:44.758423    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-djvzt_1d07e68d-9943-420f-a6a0-5f477792fabd/konnectivity-agent/0.log" limit=1024
Sep 17 17:36:44.960205 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:36:44.960111    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-djvzt_1d07e68d-9943-420f-a6a0-5f477792fabd/konnectivity-agent/0.log" limit=1024
Sep 17 17:36:45.150762 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:36:45.150732    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-djvzt_1d07e68d-9943-420f-a6a0-5f477792fabd/konnectivity-agent/0.log" limit=1024
Sep 17 17:36:46.153472 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:36:46.153423    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-djvzt_1d07e68d-9943-420f-a6a0-5f477792fabd/konnectivity-agent/0.log" limit=1024
Sep 17 17:36:46.549385 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:36:46.549298    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-djvzt_1d07e68d-9943-420f-a6a0-5f477792fabd/konnectivity-agent/0.log" limit=1024
Sep 17 17:36:46.747140 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:36:46.747095    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-djvzt_1d07e68d-9943-420f-a6a0-5f477792fabd/konnectivity-agent/0.log" limit=1024
Sep 17 17:36:46.959302 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:36:46.959264    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-djvzt_1d07e68d-9943-420f-a6a0-5f477792fabd/konnectivity-agent/0.log" limit=1024
Sep 17 17:36:47.358155 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:36:47.358072    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-djvzt_1d07e68d-9943-420f-a6a0-5f477792fabd/konnectivity-agent/0.log" limit=1024
Sep 17 17:36:47.974761 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:36:47.974719    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-djvzt_1d07e68d-9943-420f-a6a0-5f477792fabd/konnectivity-agent/0.log" limit=1024
Sep 17 17:36:48.147566 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:36:48.147531    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-djvzt_1d07e68d-9943-420f-a6a0-5f477792fabd/konnectivity-agent/0.log" limit=1024
Sep 17 17:36:48.362269 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:36:48.362177    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-djvzt_1d07e68d-9943-420f-a6a0-5f477792fabd/konnectivity-agent/0.log" limit=1024
Sep 17 17:36:48.762130 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:36:48.762096    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-djvzt_1d07e68d-9943-420f-a6a0-5f477792fabd/konnectivity-agent/0.log" limit=1024
Sep 17 17:36:48.995721 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:36:48.995680    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-djvzt_1d07e68d-9943-420f-a6a0-5f477792fabd/konnectivity-agent/0.log" limit=1024
Sep 17 17:36:49.170695 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:36:49.170662    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-djvzt_1d07e68d-9943-420f-a6a0-5f477792fabd/konnectivity-agent/0.log" limit=1024
Sep 17 17:36:49.524655 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:36:49.524552    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-djvzt_1d07e68d-9943-420f-a6a0-5f477792fabd/konnectivity-agent/0.log" limit=1024
Sep 17 17:36:49.622797 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:36:49.622760    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-djvzt_1d07e68d-9943-420f-a6a0-5f477792fabd/konnectivity-agent/0.log" limit=1024
Sep 17 17:36:50.700784 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:36:50.700756    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/27866e17-c231-49a6-921f-a83a01b8e173-metrics-certs\") pod \"network-metrics-daemon-wdp4h\" (UID: \"27866e17-c231-49a6-921f-a83a01b8e173\") " pod="openshift-multus/network-metrics-daemon-wdp4h"
Sep 17 17:36:50.703038 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:36:50.703021    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/27866e17-c231-49a6-921f-a83a01b8e173-metrics-certs\") pod \"network-metrics-daemon-wdp4h\" (UID: \"27866e17-c231-49a6-921f-a83a01b8e173\") " pod="openshift-multus/network-metrics-daemon-wdp4h"
Sep 17 17:36:50.874949 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:36:50.874918    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-j84n5\""
Sep 17 17:36:50.883092 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:36:50.883073    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-wdp4h"
Sep 17 17:36:51.156080 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:36:51.156053    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-wdp4h"]
Sep 17 17:36:51.159466 ip-10-0-143-74 kubenswrapper[2575]: W0917 17:36:51.159424    2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod27866e17_c231_49a6_921f_a83a01b8e173.slice/crio-47afc163dca6e609f3745387c821afa9a57bfb015be2e93a93ea90d3b9909a0e WatchSource:0}: Error finding container 47afc163dca6e609f3745387c821afa9a57bfb015be2e93a93ea90d3b9909a0e: Status 404 returned error can't find the container with id 47afc163dca6e609f3745387c821afa9a57bfb015be2e93a93ea90d3b9909a0e
Sep 17 17:36:51.532046 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:36:51.531964    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-wdp4h" event={"ID":"27866e17-c231-49a6-921f-a83a01b8e173","Type":"ContainerStarted","Data":"47afc163dca6e609f3745387c821afa9a57bfb015be2e93a93ea90d3b9909a0e"}
Sep 17 17:36:52.538139 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:36:52.538061    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-wdp4h" event={"ID":"27866e17-c231-49a6-921f-a83a01b8e173","Type":"ContainerStarted","Data":"b51b27c1ccfe3e4aa6702db80352ea2c91333da12392b3ae482f372889653f72"}
Sep 17 17:36:52.538139 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:36:52.538111    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-wdp4h" event={"ID":"27866e17-c231-49a6-921f-a83a01b8e173","Type":"ContainerStarted","Data":"40cf9d9e6e5895e0a45ba2254ad602a6707a3a38d3f037e4f74a48230c2a29ba"}
Sep 17 17:36:52.557763 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:36:52.557631    2575 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/network-metrics-daemon-wdp4h" podStartSLOduration=130.492047276 podStartE2EDuration="2m11.557610561s" totalImagesPullingTime="1.065563285s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-17 17:34:41 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-17 17:36:52.556500706 +0000 UTC m=+132.186830682" watchObservedRunningTime="2026-09-17 17:36:52.557610561 +0000 UTC m=+132.187940537"
Sep 17 17:36:55.939399 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:36:55.939357    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/3a6e39e9-229a-440e-a565-16ef48e36247-metrics-certs\") pod \"router-default-76f78f8976-kwv2h\" (UID: \"3a6e39e9-229a-440e-a565-16ef48e36247\") " pod="openshift-ingress/router-default-76f78f8976-kwv2h"
Sep 17 17:36:55.939399 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:36:55.939399    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/3a6e39e9-229a-440e-a565-16ef48e36247-service-ca-bundle\") pod \"router-default-76f78f8976-kwv2h\" (UID: \"3a6e39e9-229a-440e-a565-16ef48e36247\") " pod="openshift-ingress/router-default-76f78f8976-kwv2h"
Sep 17 17:36:55.940125 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:36:55.940101    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/3a6e39e9-229a-440e-a565-16ef48e36247-service-ca-bundle\") pod \"router-default-76f78f8976-kwv2h\" (UID: \"3a6e39e9-229a-440e-a565-16ef48e36247\") " pod="openshift-ingress/router-default-76f78f8976-kwv2h"
Sep 17 17:36:55.941661 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:36:55.941644    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/3a6e39e9-229a-440e-a565-16ef48e36247-metrics-certs\") pod \"router-default-76f78f8976-kwv2h\" (UID: \"3a6e39e9-229a-440e-a565-16ef48e36247\") " pod="openshift-ingress/router-default-76f78f8976-kwv2h"
Sep 17 17:36:56.197777 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:36:56.197702    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"router-dockercfg-xlq4g\""
Sep 17 17:36:56.205519 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:36:56.205486    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/router-default-76f78f8976-kwv2h"
Sep 17 17:36:56.496662 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:36:56.496571    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/router-default-76f78f8976-kwv2h"]
Sep 17 17:36:56.500925 ip-10-0-143-74 kubenswrapper[2575]: W0917 17:36:56.500894    2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3a6e39e9_229a_440e_a565_16ef48e36247.slice/crio-f8ba4781e997eed0396b284fc8435d7ce65a1808ec2f9002d54519b8a110415b WatchSource:0}: Error finding container f8ba4781e997eed0396b284fc8435d7ce65a1808ec2f9002d54519b8a110415b: Status 404 returned error can't find the container with id f8ba4781e997eed0396b284fc8435d7ce65a1808ec2f9002d54519b8a110415b
Sep 17 17:36:56.551332 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:36:56.551307    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/router-default-76f78f8976-kwv2h" event={"ID":"3a6e39e9-229a-440e-a565-16ef48e36247","Type":"ContainerStarted","Data":"f8ba4781e997eed0396b284fc8435d7ce65a1808ec2f9002d54519b8a110415b"}
Sep 17 17:36:57.555655 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:36:57.555608    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/router-default-76f78f8976-kwv2h" event={"ID":"3a6e39e9-229a-440e-a565-16ef48e36247","Type":"ContainerStarted","Data":"810ce49089e333bb94156c0dfb77ee00d626e8a081f4a48903d2b0dbb5a75bc9"}
Sep 17 17:36:57.580407 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:36:57.580357    2575 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/router-default-76f78f8976-kwv2h" podStartSLOduration=33.580339989 podStartE2EDuration="33.580339989s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-17 17:36:24 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-17 17:36:57.579474765 +0000 UTC m=+137.209804738" watchObservedRunningTime="2026-09-17 17:36:57.580339989 +0000 UTC m=+137.210669962"
Sep 17 17:36:58.206128 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:36:58.206092    2575 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/router-default-76f78f8976-kwv2h"
Sep 17 17:36:58.208588 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:36:58.208560    2575 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/router-default-76f78f8976-kwv2h"
Sep 17 17:36:58.558906 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:36:58.558828    2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/router-default-76f78f8976-kwv2h"
Sep 17 17:36:58.560020 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:36:58.560000    2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/router-default-76f78f8976-kwv2h"
Sep 17 17:36:58.592357 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:36:58.592330    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-djvzt_1d07e68d-9943-420f-a6a0-5f477792fabd/konnectivity-agent/0.log" limit=1024
Sep 17 17:36:58.700776 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:36:58.700741    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-djvzt_1d07e68d-9943-420f-a6a0-5f477792fabd/konnectivity-agent/0.log" limit=1024
Sep 17 17:36:58.790214 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:36:58.790182    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-djvzt_1d07e68d-9943-420f-a6a0-5f477792fabd/konnectivity-agent/0.log" limit=1024
Sep 17 17:36:59.102768 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:36:59.102734    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-djvzt_1d07e68d-9943-420f-a6a0-5f477792fabd/konnectivity-agent/0.log" limit=1024
Sep 17 17:36:59.664614 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:36:59.664555    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/1e261d7f-9daa-43ed-84f0-8d4d2aea69d7-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-gspvq\" (UID: \"1e261d7f-9daa-43ed-84f0-8d4d2aea69d7\") " pod="openshift-insights/insights-runtime-extractor-gspvq"
Sep 17 17:36:59.666863 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:36:59.666844    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/1e261d7f-9daa-43ed-84f0-8d4d2aea69d7-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-gspvq\" (UID: \"1e261d7f-9daa-43ed-84f0-8d4d2aea69d7\") " pod="openshift-insights/insights-runtime-extractor-gspvq"
Sep 17 17:36:59.957376 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:36:59.957284    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-sa-dockercfg-4ktmh\""
Sep 17 17:36:59.964752 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:36:59.964729    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-gspvq"
Sep 17 17:37:00.236477 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:37:00.236380    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-gspvq"]
Sep 17 17:37:00.239972 ip-10-0-143-74 kubenswrapper[2575]: W0917 17:37:00.239947    2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1e261d7f_9daa_43ed_84f0_8d4d2aea69d7.slice/crio-396d9b26655dd100f91a6f756058273ee60a2b3e0fe0f055165e4c393e6b398f WatchSource:0}: Error finding container 396d9b26655dd100f91a6f756058273ee60a2b3e0fe0f055165e4c393e6b398f: Status 404 returned error can't find the container with id 396d9b26655dd100f91a6f756058273ee60a2b3e0fe0f055165e4c393e6b398f
Sep 17 17:37:00.566500 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:37:00.566404    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-gspvq" event={"ID":"1e261d7f-9daa-43ed-84f0-8d4d2aea69d7","Type":"ContainerStarted","Data":"739f76b286e4926672e1442b0d9cd1a848a455c31092981c6d69aa49b9c9e7a4"}
Sep 17 17:37:00.566500 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:37:00.566480    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-gspvq" event={"ID":"1e261d7f-9daa-43ed-84f0-8d4d2aea69d7","Type":"ContainerStarted","Data":"396d9b26655dd100f91a6f756058273ee60a2b3e0fe0f055165e4c393e6b398f"}
Sep 17 17:37:01.570973 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:37:01.570938    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-gspvq" event={"ID":"1e261d7f-9daa-43ed-84f0-8d4d2aea69d7","Type":"ContainerStarted","Data":"8004efef73517c1004fe50ffe807f9cd3a1eed9672d509f91e3709db36a7c444"}
Sep 17 17:37:02.104084 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:37:02.104048    2575 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-6fdbbbfcd8-8ptz6"]
Sep 17 17:37:02.107493 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:37:02.107470    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-6fdbbbfcd8-8ptz6"
Sep 17 17:37:02.111299 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:37:02.111270    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"managed-serviceaccount-hub-kubeconfig\""
Sep 17 17:37:02.111299 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:37:02.111290    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"managed-serviceaccount-dockercfg-xshst\""
Sep 17 17:37:02.111515 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:37:02.111313    2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"open-cluster-management-agent-addon\"/\"openshift-service-ca.crt\""
Sep 17 17:37:02.111515 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:37:02.111323    2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"open-cluster-management-agent-addon\"/\"kube-root-ca.crt\""
Sep 17 17:37:02.111699 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:37:02.111684    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"open-cluster-management-image-pull-credentials\""
Sep 17 17:37:02.122623 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:37:02.122597    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-6fdbbbfcd8-8ptz6"]
Sep 17 17:37:02.171507 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:37:02.171470    2575 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-image-registry/image-registry-567fd9f7c4-z54rn"]
Sep 17 17:37:02.174745 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:37:02.174726    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-567fd9f7c4-z54rn"
Sep 17 17:37:02.177422 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:37:02.177400    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"installation-pull-secrets\""
Sep 17 17:37:02.177601 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:37:02.177537    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-private-configuration\""
Sep 17 17:37:02.177667 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:37:02.177596    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-tls\""
Sep 17 17:37:02.177729 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:37:02.177675    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"registry-dockercfg-55z8g\""
Sep 17 17:37:02.181927 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:37:02.181726    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5xcbp\" (UniqueName: \"kubernetes.io/projected/c7115ab6-933f-4235-9696-57e7eb55d11b-kube-api-access-5xcbp\") pod \"managed-serviceaccount-addon-agent-6fdbbbfcd8-8ptz6\" (UID: \"c7115ab6-933f-4235-9696-57e7eb55d11b\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-6fdbbbfcd8-8ptz6"
Sep 17 17:37:02.181927 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:37:02.181777    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/c7115ab6-933f-4235-9696-57e7eb55d11b-hub-kubeconfig\") pod \"managed-serviceaccount-addon-agent-6fdbbbfcd8-8ptz6\" (UID: \"c7115ab6-933f-4235-9696-57e7eb55d11b\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-6fdbbbfcd8-8ptz6"
Sep 17 17:37:02.184050 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:37:02.183974    2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"trusted-ca\""
Sep 17 17:37:02.191756 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:37:02.191734    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-567fd9f7c4-z54rn"]
Sep 17 17:37:02.216746 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:37:02.216707    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-djvzt_1d07e68d-9943-420f-a6a0-5f477792fabd/konnectivity-agent/0.log" limit=1024
Sep 17 17:37:02.282270 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:37:02.282239    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/9476f49c-219d-4d93-8ac2-b98df60c6b22-ca-trust-extracted\") pod \"image-registry-567fd9f7c4-z54rn\" (UID: \"9476f49c-219d-4d93-8ac2-b98df60c6b22\") " pod="openshift-image-registry/image-registry-567fd9f7c4-z54rn"
Sep 17 17:37:02.282474 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:37:02.282276    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/c7115ab6-933f-4235-9696-57e7eb55d11b-hub-kubeconfig\") pod \"managed-serviceaccount-addon-agent-6fdbbbfcd8-8ptz6\" (UID: \"c7115ab6-933f-4235-9696-57e7eb55d11b\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-6fdbbbfcd8-8ptz6"
Sep 17 17:37:02.282474 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:37:02.282305    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/9476f49c-219d-4d93-8ac2-b98df60c6b22-registry-tls\") pod \"image-registry-567fd9f7c4-z54rn\" (UID: \"9476f49c-219d-4d93-8ac2-b98df60c6b22\") " pod="openshift-image-registry/image-registry-567fd9f7c4-z54rn"
Sep 17 17:37:02.282474 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:37:02.282358    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rdsbf\" (UniqueName: \"kubernetes.io/projected/9476f49c-219d-4d93-8ac2-b98df60c6b22-kube-api-access-rdsbf\") pod \"image-registry-567fd9f7c4-z54rn\" (UID: \"9476f49c-219d-4d93-8ac2-b98df60c6b22\") " pod="openshift-image-registry/image-registry-567fd9f7c4-z54rn"
Sep 17 17:37:02.282474 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:37:02.282399    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/9476f49c-219d-4d93-8ac2-b98df60c6b22-bound-sa-token\") pod \"image-registry-567fd9f7c4-z54rn\" (UID: \"9476f49c-219d-4d93-8ac2-b98df60c6b22\") " pod="openshift-image-registry/image-registry-567fd9f7c4-z54rn"
Sep 17 17:37:02.282474 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:37:02.282434    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/9476f49c-219d-4d93-8ac2-b98df60c6b22-image-registry-private-configuration\") pod \"image-registry-567fd9f7c4-z54rn\" (UID: \"9476f49c-219d-4d93-8ac2-b98df60c6b22\") " pod="openshift-image-registry/image-registry-567fd9f7c4-z54rn"
Sep 17 17:37:02.282705 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:37:02.282487    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/9476f49c-219d-4d93-8ac2-b98df60c6b22-trusted-ca\") pod \"image-registry-567fd9f7c4-z54rn\" (UID: \"9476f49c-219d-4d93-8ac2-b98df60c6b22\") " pod="openshift-image-registry/image-registry-567fd9f7c4-z54rn"
Sep 17 17:37:02.282705 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:37:02.282546    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/9476f49c-219d-4d93-8ac2-b98df60c6b22-registry-certificates\") pod \"image-registry-567fd9f7c4-z54rn\" (UID: \"9476f49c-219d-4d93-8ac2-b98df60c6b22\") " pod="openshift-image-registry/image-registry-567fd9f7c4-z54rn"
Sep 17 17:37:02.282705 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:37:02.282588    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/9476f49c-219d-4d93-8ac2-b98df60c6b22-installation-pull-secrets\") pod \"image-registry-567fd9f7c4-z54rn\" (UID: \"9476f49c-219d-4d93-8ac2-b98df60c6b22\") " pod="openshift-image-registry/image-registry-567fd9f7c4-z54rn"
Sep 17 17:37:02.282829 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:37:02.282698    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5xcbp\" (UniqueName: \"kubernetes.io/projected/c7115ab6-933f-4235-9696-57e7eb55d11b-kube-api-access-5xcbp\") pod \"managed-serviceaccount-addon-agent-6fdbbbfcd8-8ptz6\" (UID: \"c7115ab6-933f-4235-9696-57e7eb55d11b\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-6fdbbbfcd8-8ptz6"
Sep 17 17:37:02.285172 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:37:02.285150    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/c7115ab6-933f-4235-9696-57e7eb55d11b-hub-kubeconfig\") pod \"managed-serviceaccount-addon-agent-6fdbbbfcd8-8ptz6\" (UID: \"c7115ab6-933f-4235-9696-57e7eb55d11b\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-6fdbbbfcd8-8ptz6"
Sep 17 17:37:02.294205 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:37:02.294176    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-5xcbp\" (UniqueName: \"kubernetes.io/projected/c7115ab6-933f-4235-9696-57e7eb55d11b-kube-api-access-5xcbp\") pod \"managed-serviceaccount-addon-agent-6fdbbbfcd8-8ptz6\" (UID: \"c7115ab6-933f-4235-9696-57e7eb55d11b\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-6fdbbbfcd8-8ptz6"
Sep 17 17:37:02.307714 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:37:02.307608    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-djvzt_1d07e68d-9943-420f-a6a0-5f477792fabd/konnectivity-agent/0.log" limit=1024
Sep 17 17:37:02.383408 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:37:02.383368    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/9476f49c-219d-4d93-8ac2-b98df60c6b22-ca-trust-extracted\") pod \"image-registry-567fd9f7c4-z54rn\" (UID: \"9476f49c-219d-4d93-8ac2-b98df60c6b22\") " pod="openshift-image-registry/image-registry-567fd9f7c4-z54rn"
Sep 17 17:37:02.383408 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:37:02.383402    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/9476f49c-219d-4d93-8ac2-b98df60c6b22-registry-tls\") pod \"image-registry-567fd9f7c4-z54rn\" (UID: \"9476f49c-219d-4d93-8ac2-b98df60c6b22\") " pod="openshift-image-registry/image-registry-567fd9f7c4-z54rn"
Sep 17 17:37:02.383670 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:37:02.383422    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rdsbf\" (UniqueName: \"kubernetes.io/projected/9476f49c-219d-4d93-8ac2-b98df60c6b22-kube-api-access-rdsbf\") pod \"image-registry-567fd9f7c4-z54rn\" (UID: \"9476f49c-219d-4d93-8ac2-b98df60c6b22\") " pod="openshift-image-registry/image-registry-567fd9f7c4-z54rn"
Sep 17 17:37:02.383670 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:37:02.383466    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/9476f49c-219d-4d93-8ac2-b98df60c6b22-bound-sa-token\") pod \"image-registry-567fd9f7c4-z54rn\" (UID: \"9476f49c-219d-4d93-8ac2-b98df60c6b22\") " pod="openshift-image-registry/image-registry-567fd9f7c4-z54rn"
Sep 17 17:37:02.383670 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:37:02.383498    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/9476f49c-219d-4d93-8ac2-b98df60c6b22-image-registry-private-configuration\") pod \"image-registry-567fd9f7c4-z54rn\" (UID: \"9476f49c-219d-4d93-8ac2-b98df60c6b22\") " pod="openshift-image-registry/image-registry-567fd9f7c4-z54rn"
Sep 17 17:37:02.383670 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:37:02.383522    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/9476f49c-219d-4d93-8ac2-b98df60c6b22-trusted-ca\") pod \"image-registry-567fd9f7c4-z54rn\" (UID: \"9476f49c-219d-4d93-8ac2-b98df60c6b22\") " pod="openshift-image-registry/image-registry-567fd9f7c4-z54rn"
Sep 17 17:37:02.383670 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:37:02.383553    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/9476f49c-219d-4d93-8ac2-b98df60c6b22-registry-certificates\") pod \"image-registry-567fd9f7c4-z54rn\" (UID: \"9476f49c-219d-4d93-8ac2-b98df60c6b22\") " pod="openshift-image-registry/image-registry-567fd9f7c4-z54rn"
Sep 17 17:37:02.383670 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:37:02.383590    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/9476f49c-219d-4d93-8ac2-b98df60c6b22-installation-pull-secrets\") pod \"image-registry-567fd9f7c4-z54rn\" (UID: \"9476f49c-219d-4d93-8ac2-b98df60c6b22\") " pod="openshift-image-registry/image-registry-567fd9f7c4-z54rn"
Sep 17 17:37:02.383911 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:37:02.383830    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/9476f49c-219d-4d93-8ac2-b98df60c6b22-ca-trust-extracted\") pod \"image-registry-567fd9f7c4-z54rn\" (UID: \"9476f49c-219d-4d93-8ac2-b98df60c6b22\") " pod="openshift-image-registry/image-registry-567fd9f7c4-z54rn"
Sep 17 17:37:02.384501 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:37:02.384472    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/9476f49c-219d-4d93-8ac2-b98df60c6b22-trusted-ca\") pod \"image-registry-567fd9f7c4-z54rn\" (UID: \"9476f49c-219d-4d93-8ac2-b98df60c6b22\") " pod="openshift-image-registry/image-registry-567fd9f7c4-z54rn"
Sep 17 17:37:02.384603 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:37:02.384574    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/9476f49c-219d-4d93-8ac2-b98df60c6b22-registry-certificates\") pod \"image-registry-567fd9f7c4-z54rn\" (UID: \"9476f49c-219d-4d93-8ac2-b98df60c6b22\") " pod="openshift-image-registry/image-registry-567fd9f7c4-z54rn"
Sep 17 17:37:02.386175 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:37:02.386151    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/9476f49c-219d-4d93-8ac2-b98df60c6b22-installation-pull-secrets\") pod \"image-registry-567fd9f7c4-z54rn\" (UID: \"9476f49c-219d-4d93-8ac2-b98df60c6b22\") " pod="openshift-image-registry/image-registry-567fd9f7c4-z54rn"
Sep 17 17:37:02.386267 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:37:02.386248    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/9476f49c-219d-4d93-8ac2-b98df60c6b22-registry-tls\") pod \"image-registry-567fd9f7c4-z54rn\" (UID: \"9476f49c-219d-4d93-8ac2-b98df60c6b22\") " pod="openshift-image-registry/image-registry-567fd9f7c4-z54rn"
Sep 17 17:37:02.386334 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:37:02.386310    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/9476f49c-219d-4d93-8ac2-b98df60c6b22-image-registry-private-configuration\") pod \"image-registry-567fd9f7c4-z54rn\" (UID: \"9476f49c-219d-4d93-8ac2-b98df60c6b22\") " pod="openshift-image-registry/image-registry-567fd9f7c4-z54rn"
Sep 17 17:37:02.392744 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:37:02.392721    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-rdsbf\" (UniqueName: \"kubernetes.io/projected/9476f49c-219d-4d93-8ac2-b98df60c6b22-kube-api-access-rdsbf\") pod \"image-registry-567fd9f7c4-z54rn\" (UID: \"9476f49c-219d-4d93-8ac2-b98df60c6b22\") " pod="openshift-image-registry/image-registry-567fd9f7c4-z54rn"
Sep 17 17:37:02.393851 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:37:02.393833    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/9476f49c-219d-4d93-8ac2-b98df60c6b22-bound-sa-token\") pod \"image-registry-567fd9f7c4-z54rn\" (UID: \"9476f49c-219d-4d93-8ac2-b98df60c6b22\") " pod="openshift-image-registry/image-registry-567fd9f7c4-z54rn"
Sep 17 17:37:02.400653 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:37:02.400605    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-djvzt_1d07e68d-9943-420f-a6a0-5f477792fabd/konnectivity-agent/0.log" limit=1024
Sep 17 17:37:02.428177 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:37:02.428156    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-6fdbbbfcd8-8ptz6"
Sep 17 17:37:02.484822 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:37:02.484796    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-567fd9f7c4-z54rn"
Sep 17 17:37:02.576317 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:37:02.576237    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-gspvq" event={"ID":"1e261d7f-9daa-43ed-84f0-8d4d2aea69d7","Type":"ContainerStarted","Data":"6839ba2ca3047075cbe017434ceaeba7aee08bebd88e189975da616872286862"}
Sep 17 17:37:02.608917 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:37:02.608861    2575 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-insights/insights-runtime-extractor-gspvq" podStartSLOduration=33.55766069 podStartE2EDuration="35.608844883s" totalImagesPullingTime="2.051184193s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-17 17:36:27 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-17 17:37:02.607685805 +0000 UTC m=+142.238015779" watchObservedRunningTime="2026-09-17 17:37:02.608844883 +0000 UTC m=+142.239174858"
Sep 17 17:37:02.705936 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:37:02.704025    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-6fdbbbfcd8-8ptz6"]
Sep 17 17:37:02.709192 ip-10-0-143-74 kubenswrapper[2575]: W0917 17:37:02.709170    2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podc7115ab6_933f_4235_9696_57e7eb55d11b.slice/crio-afd2ed6a7ce8e18513e383a674edb56fddd8bac76b570d93765d38be34c58f51 WatchSource:0}: Error finding container afd2ed6a7ce8e18513e383a674edb56fddd8bac76b570d93765d38be34c58f51: Status 404 returned error can't find the container with id afd2ed6a7ce8e18513e383a674edb56fddd8bac76b570d93765d38be34c58f51
Sep 17 17:37:02.758031 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:37:02.757997    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-567fd9f7c4-z54rn"]
Sep 17 17:37:02.760544 ip-10-0-143-74 kubenswrapper[2575]: W0917 17:37:02.760515    2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9476f49c_219d_4d93_8ac2_b98df60c6b22.slice/crio-15d44bbb344e4f835ee8d7ebac72d6deeabadbb8ea76497b9b36f53c43976d1c WatchSource:0}: Error finding container 15d44bbb344e4f835ee8d7ebac72d6deeabadbb8ea76497b9b36f53c43976d1c: Status 404 returned error can't find the container with id 15d44bbb344e4f835ee8d7ebac72d6deeabadbb8ea76497b9b36f53c43976d1c
Sep 17 17:37:03.582188 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:37:03.582114    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-567fd9f7c4-z54rn" event={"ID":"9476f49c-219d-4d93-8ac2-b98df60c6b22","Type":"ContainerStarted","Data":"0a71dba1484c75866e0f8f8097c166276f59fdd5829b7a7b6c537f02c5e42334"}
Sep 17 17:37:03.582188 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:37:03.582178    2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-image-registry/image-registry-567fd9f7c4-z54rn"
Sep 17 17:37:03.582823 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:37:03.582198    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-567fd9f7c4-z54rn" event={"ID":"9476f49c-219d-4d93-8ac2-b98df60c6b22","Type":"ContainerStarted","Data":"15d44bbb344e4f835ee8d7ebac72d6deeabadbb8ea76497b9b36f53c43976d1c"}
Sep 17 17:37:03.583798 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:37:03.583626    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-6fdbbbfcd8-8ptz6" event={"ID":"c7115ab6-933f-4235-9696-57e7eb55d11b","Type":"ContainerStarted","Data":"afd2ed6a7ce8e18513e383a674edb56fddd8bac76b570d93765d38be34c58f51"}
Sep 17 17:37:03.618605 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:37:03.618549    2575 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/image-registry-567fd9f7c4-z54rn" podStartSLOduration=1.618525014 podStartE2EDuration="1.618525014s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-17 17:37:02 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-17 17:37:03.617638279 +0000 UTC m=+143.247968257" watchObservedRunningTime="2026-09-17 17:37:03.618525014 +0000 UTC m=+143.248854990"
Sep 17 17:37:03.717663 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:37:03.717627    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-djvzt_1d07e68d-9943-420f-a6a0-5f477792fabd/konnectivity-agent/0.log" limit=1024
Sep 17 17:37:03.934088 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:37:03.934034    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-djvzt_1d07e68d-9943-420f-a6a0-5f477792fabd/konnectivity-agent/0.log" limit=1024
Sep 17 17:37:04.018097 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:37:04.018067    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-djvzt_1d07e68d-9943-420f-a6a0-5f477792fabd/konnectivity-agent/0.log" limit=1024
Sep 17 17:37:04.541574 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:37:04.541522    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-djvzt_1d07e68d-9943-420f-a6a0-5f477792fabd/konnectivity-agent/0.log" limit=1024
Sep 17 17:37:04.669978 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:37:04.669942    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-djvzt_1d07e68d-9943-420f-a6a0-5f477792fabd/konnectivity-agent/0.log" limit=1024
Sep 17 17:37:04.839711 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:37:04.839627    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-djvzt_1d07e68d-9943-420f-a6a0-5f477792fabd/konnectivity-agent/0.log" limit=1024
Sep 17 17:37:04.944732 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:37:04.944694    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-djvzt_1d07e68d-9943-420f-a6a0-5f477792fabd/konnectivity-agent/0.log" limit=1024
Sep 17 17:37:05.292698 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:37:05.292663    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-djvzt_1d07e68d-9943-420f-a6a0-5f477792fabd/konnectivity-agent/0.log" limit=1024
Sep 17 17:37:05.374223 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:37:05.374190    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-djvzt_1d07e68d-9943-420f-a6a0-5f477792fabd/konnectivity-agent/0.log" limit=1024
Sep 17 17:37:05.593891 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:37:05.593859    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-6fdbbbfcd8-8ptz6" event={"ID":"c7115ab6-933f-4235-9696-57e7eb55d11b","Type":"ContainerStarted","Data":"fe2a54cc18590e914b6d5027cb53283ae387fa574f4d5aa277d018ffba08744c"}
Sep 17 17:37:05.614717 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:37:05.614670    2575 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-6fdbbbfcd8-8ptz6" podStartSLOduration=1.2544049990000001 podStartE2EDuration="3.614652871s" totalImagesPullingTime="2.360247872s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-17 17:37:02 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-17 17:37:05.613725501 +0000 UTC m=+145.244055487" watchObservedRunningTime="2026-09-17 17:37:05.614652871 +0000 UTC m=+145.244982845"
Sep 17 17:37:08.650575 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:37:08.650537    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-djvzt_1d07e68d-9943-420f-a6a0-5f477792fabd/konnectivity-agent/0.log" limit=1024
Sep 17 17:37:10.059704 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:37:10.059672    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-djvzt_1d07e68d-9943-420f-a6a0-5f477792fabd/konnectivity-agent/0.log" limit=1024
Sep 17 17:37:10.293208 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:37:10.293160    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-djvzt_1d07e68d-9943-420f-a6a0-5f477792fabd/konnectivity-agent/0.log" limit=1024
Sep 17 17:37:10.456509 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:37:10.456473    2575 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/openshift-state-metrics-86c5bc79d6-hqncr"]
Sep 17 17:37:10.460932 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:37:10.460907    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/openshift-state-metrics-86c5bc79d6-hqncr"
Sep 17 17:37:10.463702 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:37:10.463679    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"openshift-state-metrics-tls\""
Sep 17 17:37:10.464780 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:37:10.464754    2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"openshift-service-ca.crt\""
Sep 17 17:37:10.464917 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:37:10.464753    2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kube-root-ca.crt\""
Sep 17 17:37:10.465030 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:37:10.464768    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"openshift-state-metrics-dockercfg-8phpw\""
Sep 17 17:37:10.465030 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:37:10.464815    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"openshift-state-metrics-kube-rbac-proxy-config\""
Sep 17 17:37:10.465030 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:37:10.464837    2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"metrics-client-ca\""
Sep 17 17:37:10.474632 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:37:10.474611    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/openshift-state-metrics-86c5bc79d6-hqncr"]
Sep 17 17:37:10.496310 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:37:10.496262    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-djvzt_1d07e68d-9943-420f-a6a0-5f477792fabd/konnectivity-agent/0.log" limit=1024
Sep 17 17:37:10.524916 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:37:10.524891    2575 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/node-exporter-ksdnc"]
Sep 17 17:37:10.528109 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:37:10.528091    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-ksdnc"
Sep 17 17:37:10.530705 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:37:10.530681    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-dockercfg-kjs7l\""
Sep 17 17:37:10.530705 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:37:10.530699    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-tls\""
Sep 17 17:37:10.530873 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:37:10.530738    2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"node-exporter-accelerators-collector-config\""
Sep 17 17:37:10.531067 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:37:10.531050    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-kube-rbac-proxy-config\""
Sep 17 17:37:10.588446 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:37:10.588419    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-djvzt_1d07e68d-9943-420f-a6a0-5f477792fabd/konnectivity-agent/0.log" limit=1024
Sep 17 17:37:10.641321 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:37:10.641293    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/748f74c6-2a2e-4d13-898e-662858625d39-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-ksdnc\" (UID: \"748f74c6-2a2e-4d13-898e-662858625d39\") " pod="openshift-monitoring/node-exporter-ksdnc"
Sep 17 17:37:10.641321 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:37:10.641323    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/748f74c6-2a2e-4d13-898e-662858625d39-node-exporter-wtmp\") pod \"node-exporter-ksdnc\" (UID: \"748f74c6-2a2e-4d13-898e-662858625d39\") " pod="openshift-monitoring/node-exporter-ksdnc"
Sep 17 17:37:10.641575 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:37:10.641340    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/748f74c6-2a2e-4d13-898e-662858625d39-node-exporter-accelerators-collector-config\") pod \"node-exporter-ksdnc\" (UID: \"748f74c6-2a2e-4d13-898e-662858625d39\") " pod="openshift-monitoring/node-exporter-ksdnc"
Sep 17 17:37:10.641575 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:37:10.641363    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/748f74c6-2a2e-4d13-898e-662858625d39-metrics-client-ca\") pod \"node-exporter-ksdnc\" (UID: \"748f74c6-2a2e-4d13-898e-662858625d39\") " pod="openshift-monitoring/node-exporter-ksdnc"
Sep 17 17:37:10.641575 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:37:10.641453    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/8dfa0270-d733-4438-b1e6-dde2f283d99b-metrics-client-ca\") pod \"openshift-state-metrics-86c5bc79d6-hqncr\" (UID: \"8dfa0270-d733-4438-b1e6-dde2f283d99b\") " pod="openshift-monitoring/openshift-state-metrics-86c5bc79d6-hqncr"
Sep 17 17:37:10.641575 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:37:10.641484    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/748f74c6-2a2e-4d13-898e-662858625d39-node-exporter-textfile\") pod \"node-exporter-ksdnc\" (UID: \"748f74c6-2a2e-4d13-898e-662858625d39\") " pod="openshift-monitoring/node-exporter-ksdnc"
Sep 17 17:37:10.641575 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:37:10.641511    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/748f74c6-2a2e-4d13-898e-662858625d39-root\") pod \"node-exporter-ksdnc\" (UID: \"748f74c6-2a2e-4d13-898e-662858625d39\") " pod="openshift-monitoring/node-exporter-ksdnc"
Sep 17 17:37:10.641575 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:37:10.641536    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hn5hx\" (UniqueName: \"kubernetes.io/projected/748f74c6-2a2e-4d13-898e-662858625d39-kube-api-access-hn5hx\") pod \"node-exporter-ksdnc\" (UID: \"748f74c6-2a2e-4d13-898e-662858625d39\") " pod="openshift-monitoring/node-exporter-ksdnc"
Sep 17 17:37:10.641883 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:37:10.641584    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/8dfa0270-d733-4438-b1e6-dde2f283d99b-openshift-state-metrics-tls\") pod \"openshift-state-metrics-86c5bc79d6-hqncr\" (UID: \"8dfa0270-d733-4438-b1e6-dde2f283d99b\") " pod="openshift-monitoring/openshift-state-metrics-86c5bc79d6-hqncr"
Sep 17 17:37:10.641883 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:37:10.641601    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/748f74c6-2a2e-4d13-898e-662858625d39-node-exporter-tls\") pod \"node-exporter-ksdnc\" (UID: \"748f74c6-2a2e-4d13-898e-662858625d39\") " pod="openshift-monitoring/node-exporter-ksdnc"
Sep 17 17:37:10.641883 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:37:10.641617    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hqrqz\" (UniqueName: \"kubernetes.io/projected/8dfa0270-d733-4438-b1e6-dde2f283d99b-kube-api-access-hqrqz\") pod \"openshift-state-metrics-86c5bc79d6-hqncr\" (UID: \"8dfa0270-d733-4438-b1e6-dde2f283d99b\") " pod="openshift-monitoring/openshift-state-metrics-86c5bc79d6-hqncr"
Sep 17 17:37:10.641883 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:37:10.641650    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/748f74c6-2a2e-4d13-898e-662858625d39-sys\") pod \"node-exporter-ksdnc\" (UID: \"748f74c6-2a2e-4d13-898e-662858625d39\") " pod="openshift-monitoring/node-exporter-ksdnc"
Sep 17 17:37:10.641883 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:37:10.641696    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/8dfa0270-d733-4438-b1e6-dde2f283d99b-openshift-state-metrics-kube-rbac-proxy-config\") pod \"openshift-state-metrics-86c5bc79d6-hqncr\" (UID: \"8dfa0270-d733-4438-b1e6-dde2f283d99b\") " pod="openshift-monitoring/openshift-state-metrics-86c5bc79d6-hqncr"
Sep 17 17:37:10.687910 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:37:10.687879    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-djvzt_1d07e68d-9943-420f-a6a0-5f477792fabd/konnectivity-agent/0.log" limit=1024
Sep 17 17:37:10.742070 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:37:10.741984    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/748f74c6-2a2e-4d13-898e-662858625d39-node-exporter-textfile\") pod \"node-exporter-ksdnc\" (UID: \"748f74c6-2a2e-4d13-898e-662858625d39\") " pod="openshift-monitoring/node-exporter-ksdnc"
Sep 17 17:37:10.742070 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:37:10.742020    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/748f74c6-2a2e-4d13-898e-662858625d39-root\") pod \"node-exporter-ksdnc\" (UID: \"748f74c6-2a2e-4d13-898e-662858625d39\") " pod="openshift-monitoring/node-exporter-ksdnc"
Sep 17 17:37:10.742070 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:37:10.742038    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hn5hx\" (UniqueName: \"kubernetes.io/projected/748f74c6-2a2e-4d13-898e-662858625d39-kube-api-access-hn5hx\") pod \"node-exporter-ksdnc\" (UID: \"748f74c6-2a2e-4d13-898e-662858625d39\") " pod="openshift-monitoring/node-exporter-ksdnc"
Sep 17 17:37:10.742344 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:37:10.742114    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"root\" (UniqueName: \"kubernetes.io/host-path/748f74c6-2a2e-4d13-898e-662858625d39-root\") pod \"node-exporter-ksdnc\" (UID: \"748f74c6-2a2e-4d13-898e-662858625d39\") " pod="openshift-monitoring/node-exporter-ksdnc"
Sep 17 17:37:10.742344 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:37:10.742183    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/8dfa0270-d733-4438-b1e6-dde2f283d99b-openshift-state-metrics-tls\") pod \"openshift-state-metrics-86c5bc79d6-hqncr\" (UID: \"8dfa0270-d733-4438-b1e6-dde2f283d99b\") " pod="openshift-monitoring/openshift-state-metrics-86c5bc79d6-hqncr"
Sep 17 17:37:10.742344 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:37:10.742225    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/748f74c6-2a2e-4d13-898e-662858625d39-node-exporter-tls\") pod \"node-exporter-ksdnc\" (UID: \"748f74c6-2a2e-4d13-898e-662858625d39\") " pod="openshift-monitoring/node-exporter-ksdnc"
Sep 17 17:37:10.742344 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:37:10.742254    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hqrqz\" (UniqueName: \"kubernetes.io/projected/8dfa0270-d733-4438-b1e6-dde2f283d99b-kube-api-access-hqrqz\") pod \"openshift-state-metrics-86c5bc79d6-hqncr\" (UID: \"8dfa0270-d733-4438-b1e6-dde2f283d99b\") " pod="openshift-monitoring/openshift-state-metrics-86c5bc79d6-hqncr"
Sep 17 17:37:10.742344 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:37:10.742277    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/748f74c6-2a2e-4d13-898e-662858625d39-sys\") pod \"node-exporter-ksdnc\" (UID: \"748f74c6-2a2e-4d13-898e-662858625d39\") " pod="openshift-monitoring/node-exporter-ksdnc"
Sep 17 17:37:10.742344 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:37:10.742331    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/8dfa0270-d733-4438-b1e6-dde2f283d99b-openshift-state-metrics-kube-rbac-proxy-config\") pod \"openshift-state-metrics-86c5bc79d6-hqncr\" (UID: \"8dfa0270-d733-4438-b1e6-dde2f283d99b\") " pod="openshift-monitoring/openshift-state-metrics-86c5bc79d6-hqncr"
Sep 17 17:37:10.742640 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:37:10.742363    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/748f74c6-2a2e-4d13-898e-662858625d39-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-ksdnc\" (UID: \"748f74c6-2a2e-4d13-898e-662858625d39\") " pod="openshift-monitoring/node-exporter-ksdnc"
Sep 17 17:37:10.742640 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:37:10.742374    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/748f74c6-2a2e-4d13-898e-662858625d39-node-exporter-textfile\") pod \"node-exporter-ksdnc\" (UID: \"748f74c6-2a2e-4d13-898e-662858625d39\") " pod="openshift-monitoring/node-exporter-ksdnc"
Sep 17 17:37:10.742640 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:37:10.742418    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/748f74c6-2a2e-4d13-898e-662858625d39-sys\") pod \"node-exporter-ksdnc\" (UID: \"748f74c6-2a2e-4d13-898e-662858625d39\") " pod="openshift-monitoring/node-exporter-ksdnc"
Sep 17 17:37:10.742640 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:37:10.742485    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/748f74c6-2a2e-4d13-898e-662858625d39-node-exporter-wtmp\") pod \"node-exporter-ksdnc\" (UID: \"748f74c6-2a2e-4d13-898e-662858625d39\") " pod="openshift-monitoring/node-exporter-ksdnc"
Sep 17 17:37:10.742640 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:37:10.742519    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/748f74c6-2a2e-4d13-898e-662858625d39-node-exporter-accelerators-collector-config\") pod \"node-exporter-ksdnc\" (UID: \"748f74c6-2a2e-4d13-898e-662858625d39\") " pod="openshift-monitoring/node-exporter-ksdnc"
Sep 17 17:37:10.742640 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:37:10.742556    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/748f74c6-2a2e-4d13-898e-662858625d39-metrics-client-ca\") pod \"node-exporter-ksdnc\" (UID: \"748f74c6-2a2e-4d13-898e-662858625d39\") " pod="openshift-monitoring/node-exporter-ksdnc"
Sep 17 17:37:10.742640 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:37:10.742604    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/8dfa0270-d733-4438-b1e6-dde2f283d99b-metrics-client-ca\") pod \"openshift-state-metrics-86c5bc79d6-hqncr\" (UID: \"8dfa0270-d733-4438-b1e6-dde2f283d99b\") " pod="openshift-monitoring/openshift-state-metrics-86c5bc79d6-hqncr"
Sep 17 17:37:10.742985 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:37:10.742643    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/748f74c6-2a2e-4d13-898e-662858625d39-node-exporter-wtmp\") pod \"node-exporter-ksdnc\" (UID: \"748f74c6-2a2e-4d13-898e-662858625d39\") " pod="openshift-monitoring/node-exporter-ksdnc"
Sep 17 17:37:10.743180 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:37:10.743155    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/748f74c6-2a2e-4d13-898e-662858625d39-node-exporter-accelerators-collector-config\") pod \"node-exporter-ksdnc\" (UID: \"748f74c6-2a2e-4d13-898e-662858625d39\") " pod="openshift-monitoring/node-exporter-ksdnc"
Sep 17 17:37:10.743325 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:37:10.743204    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/748f74c6-2a2e-4d13-898e-662858625d39-metrics-client-ca\") pod \"node-exporter-ksdnc\" (UID: \"748f74c6-2a2e-4d13-898e-662858625d39\") " pod="openshift-monitoring/node-exporter-ksdnc"
Sep 17 17:37:10.743627 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:37:10.743604    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/8dfa0270-d733-4438-b1e6-dde2f283d99b-metrics-client-ca\") pod \"openshift-state-metrics-86c5bc79d6-hqncr\" (UID: \"8dfa0270-d733-4438-b1e6-dde2f283d99b\") " pod="openshift-monitoring/openshift-state-metrics-86c5bc79d6-hqncr"
Sep 17 17:37:10.744971 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:37:10.744948    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/748f74c6-2a2e-4d13-898e-662858625d39-node-exporter-tls\") pod \"node-exporter-ksdnc\" (UID: \"748f74c6-2a2e-4d13-898e-662858625d39\") " pod="openshift-monitoring/node-exporter-ksdnc"
Sep 17 17:37:10.745064 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:37:10.744954    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/8dfa0270-d733-4438-b1e6-dde2f283d99b-openshift-state-metrics-tls\") pod \"openshift-state-metrics-86c5bc79d6-hqncr\" (UID: \"8dfa0270-d733-4438-b1e6-dde2f283d99b\") " pod="openshift-monitoring/openshift-state-metrics-86c5bc79d6-hqncr"
Sep 17 17:37:10.745164 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:37:10.745144    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/748f74c6-2a2e-4d13-898e-662858625d39-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-ksdnc\" (UID: \"748f74c6-2a2e-4d13-898e-662858625d39\") " pod="openshift-monitoring/node-exporter-ksdnc"
Sep 17 17:37:10.745222 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:37:10.745201    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/8dfa0270-d733-4438-b1e6-dde2f283d99b-openshift-state-metrics-kube-rbac-proxy-config\") pod \"openshift-state-metrics-86c5bc79d6-hqncr\" (UID: \"8dfa0270-d733-4438-b1e6-dde2f283d99b\") " pod="openshift-monitoring/openshift-state-metrics-86c5bc79d6-hqncr"
Sep 17 17:37:10.751926 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:37:10.751906    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-hqrqz\" (UniqueName: \"kubernetes.io/projected/8dfa0270-d733-4438-b1e6-dde2f283d99b-kube-api-access-hqrqz\") pod \"openshift-state-metrics-86c5bc79d6-hqncr\" (UID: \"8dfa0270-d733-4438-b1e6-dde2f283d99b\") " pod="openshift-monitoring/openshift-state-metrics-86c5bc79d6-hqncr"
Sep 17 17:37:10.751926 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:37:10.751916    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-hn5hx\" (UniqueName: \"kubernetes.io/projected/748f74c6-2a2e-4d13-898e-662858625d39-kube-api-access-hn5hx\") pod \"node-exporter-ksdnc\" (UID: \"748f74c6-2a2e-4d13-898e-662858625d39\") " pod="openshift-monitoring/node-exporter-ksdnc"
Sep 17 17:37:10.771526 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:37:10.771505    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/openshift-state-metrics-86c5bc79d6-hqncr"
Sep 17 17:37:10.787806 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:37:10.787775    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-djvzt_1d07e68d-9943-420f-a6a0-5f477792fabd/konnectivity-agent/0.log" limit=1024
Sep 17 17:37:10.837839 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:37:10.837803    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-ksdnc"
Sep 17 17:37:10.850026 ip-10-0-143-74 kubenswrapper[2575]: W0917 17:37:10.849998    2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod748f74c6_2a2e_4d13_898e_662858625d39.slice/crio-698f19dc482c300655c2fef2ac06e0259b0b872cb25f170ea0fa862bcb3ce4ef WatchSource:0}: Error finding container 698f19dc482c300655c2fef2ac06e0259b0b872cb25f170ea0fa862bcb3ce4ef: Status 404 returned error can't find the container with id 698f19dc482c300655c2fef2ac06e0259b0b872cb25f170ea0fa862bcb3ce4ef
Sep 17 17:37:10.923422 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:37:10.923380    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-djvzt_1d07e68d-9943-420f-a6a0-5f477792fabd/konnectivity-agent/0.log" limit=1024
Sep 17 17:37:11.053167 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:37:11.053137    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/openshift-state-metrics-86c5bc79d6-hqncr"]
Sep 17 17:37:11.055621 ip-10-0-143-74 kubenswrapper[2575]: W0917 17:37:11.055595    2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8dfa0270_d733_4438_b1e6_dde2f283d99b.slice/crio-b402e0e94b5a585cf50e3b3d06fff52a51472725103abbb1b5d7789be769b929 WatchSource:0}: Error finding container b402e0e94b5a585cf50e3b3d06fff52a51472725103abbb1b5d7789be769b929: Status 404 returned error can't find the container with id b402e0e94b5a585cf50e3b3d06fff52a51472725103abbb1b5d7789be769b929
Sep 17 17:37:11.146802 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:37:11.146772    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-djvzt_1d07e68d-9943-420f-a6a0-5f477792fabd/konnectivity-agent/0.log" limit=1024
Sep 17 17:37:11.301791 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:37:11.301745    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-djvzt_1d07e68d-9943-420f-a6a0-5f477792fabd/konnectivity-agent/0.log" limit=1024
Sep 17 17:37:11.402707 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:37:11.402669    2575 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/alertmanager-main-0"]
Sep 17 17:37:11.406725 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:37:11.406702    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0"
Sep 17 17:37:11.408956 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:37:11.408907    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-djvzt_1d07e68d-9943-420f-a6a0-5f477792fabd/konnectivity-agent/0.log" limit=1024
Sep 17 17:37:11.409651 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:37:11.409629    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-tls\""
Sep 17 17:37:11.409780 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:37:11.409689    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy\""
Sep 17 17:37:11.409939 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:37:11.409920    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-generated\""
Sep 17 17:37:11.410047 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:37:11.409920    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy-web\""
Sep 17 17:37:11.410132 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:37:11.409945    2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"alertmanager-trusted-ca-bundle\""
Sep 17 17:37:11.410194 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:37:11.409972    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-web-config\""
Sep 17 17:37:11.410249 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:37:11.410239    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-cluster-tls-config\""
Sep 17 17:37:11.410345 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:37:11.410326    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-tls-assets-0\""
Sep 17 17:37:11.410510 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:37:11.410494    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy-metric\""
Sep 17 17:37:11.410588 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:37:11.410503    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-dockercfg-pdb7v\""
Sep 17 17:37:11.424716 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:37:11.424695    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/alertmanager-main-0"]
Sep 17 17:37:11.550094 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:37:11.550060    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/b9853529-152c-4642-ab14-885220711fb3-config-volume\") pod \"alertmanager-main-0\" (UID: \"b9853529-152c-4642-ab14-885220711fb3\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 17 17:37:11.550094 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:37:11.550096    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/b9853529-152c-4642-ab14-885220711fb3-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"b9853529-152c-4642-ab14-885220711fb3\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 17 17:37:11.550315 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:37:11.550121    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mnzn6\" (UniqueName: \"kubernetes.io/projected/b9853529-152c-4642-ab14-885220711fb3-kube-api-access-mnzn6\") pod \"alertmanager-main-0\" (UID: \"b9853529-152c-4642-ab14-885220711fb3\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 17 17:37:11.550315 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:37:11.550185    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/b9853529-152c-4642-ab14-885220711fb3-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"b9853529-152c-4642-ab14-885220711fb3\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 17 17:37:11.550315 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:37:11.550252    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/b9853529-152c-4642-ab14-885220711fb3-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"b9853529-152c-4642-ab14-885220711fb3\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 17 17:37:11.550315 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:37:11.550300    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/b9853529-152c-4642-ab14-885220711fb3-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"b9853529-152c-4642-ab14-885220711fb3\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 17 17:37:11.550555 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:37:11.550335    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/b9853529-152c-4642-ab14-885220711fb3-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"b9853529-152c-4642-ab14-885220711fb3\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 17 17:37:11.550555 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:37:11.550356    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/b9853529-152c-4642-ab14-885220711fb3-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"b9853529-152c-4642-ab14-885220711fb3\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 17 17:37:11.550555 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:37:11.550380    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/b9853529-152c-4642-ab14-885220711fb3-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"b9853529-152c-4642-ab14-885220711fb3\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 17 17:37:11.550555 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:37:11.550407    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/b9853529-152c-4642-ab14-885220711fb3-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"b9853529-152c-4642-ab14-885220711fb3\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 17 17:37:11.550555 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:37:11.550434    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/b9853529-152c-4642-ab14-885220711fb3-web-config\") pod \"alertmanager-main-0\" (UID: \"b9853529-152c-4642-ab14-885220711fb3\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 17 17:37:11.550555 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:37:11.550509    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/b9853529-152c-4642-ab14-885220711fb3-config-out\") pod \"alertmanager-main-0\" (UID: \"b9853529-152c-4642-ab14-885220711fb3\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 17 17:37:11.550839 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:37:11.550569    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/b9853529-152c-4642-ab14-885220711fb3-tls-assets\") pod \"alertmanager-main-0\" (UID: \"b9853529-152c-4642-ab14-885220711fb3\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 17 17:37:11.616610 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:37:11.616574    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-ksdnc" event={"ID":"748f74c6-2a2e-4d13-898e-662858625d39","Type":"ContainerStarted","Data":"698f19dc482c300655c2fef2ac06e0259b0b872cb25f170ea0fa862bcb3ce4ef"}
Sep 17 17:37:11.618753 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:37:11.618729    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/openshift-state-metrics-86c5bc79d6-hqncr" event={"ID":"8dfa0270-d733-4438-b1e6-dde2f283d99b","Type":"ContainerStarted","Data":"60b242d552ffde3b1199e996856096156d2e253842e047f3b5e05a55bb4a1308"}
Sep 17 17:37:11.618867 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:37:11.618784    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/openshift-state-metrics-86c5bc79d6-hqncr" event={"ID":"8dfa0270-d733-4438-b1e6-dde2f283d99b","Type":"ContainerStarted","Data":"07bd8bd27f8999ec5aa572a05250be0691c556ceb2c8ed1cfd4528d11489cad4"}
Sep 17 17:37:11.618867 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:37:11.618811    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/openshift-state-metrics-86c5bc79d6-hqncr" event={"ID":"8dfa0270-d733-4438-b1e6-dde2f283d99b","Type":"ContainerStarted","Data":"b402e0e94b5a585cf50e3b3d06fff52a51472725103abbb1b5d7789be769b929"}
Sep 17 17:37:11.651208 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:37:11.651174    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/b9853529-152c-4642-ab14-885220711fb3-config-volume\") pod \"alertmanager-main-0\" (UID: \"b9853529-152c-4642-ab14-885220711fb3\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 17 17:37:11.651363 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:37:11.651217    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/b9853529-152c-4642-ab14-885220711fb3-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"b9853529-152c-4642-ab14-885220711fb3\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 17 17:37:11.651363 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:37:11.651246    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mnzn6\" (UniqueName: \"kubernetes.io/projected/b9853529-152c-4642-ab14-885220711fb3-kube-api-access-mnzn6\") pod \"alertmanager-main-0\" (UID: \"b9853529-152c-4642-ab14-885220711fb3\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 17 17:37:11.651363 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:37:11.651282    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/b9853529-152c-4642-ab14-885220711fb3-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"b9853529-152c-4642-ab14-885220711fb3\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 17 17:37:11.651363 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:37:11.651315    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/b9853529-152c-4642-ab14-885220711fb3-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"b9853529-152c-4642-ab14-885220711fb3\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 17 17:37:11.651363 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:37:11.651358    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/b9853529-152c-4642-ab14-885220711fb3-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"b9853529-152c-4642-ab14-885220711fb3\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 17 17:37:11.651646 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:37:11.651404    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/b9853529-152c-4642-ab14-885220711fb3-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"b9853529-152c-4642-ab14-885220711fb3\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 17 17:37:11.651646 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:37:11.651431    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/b9853529-152c-4642-ab14-885220711fb3-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"b9853529-152c-4642-ab14-885220711fb3\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 17 17:37:11.651646 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:37:11.651475    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/b9853529-152c-4642-ab14-885220711fb3-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"b9853529-152c-4642-ab14-885220711fb3\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 17 17:37:11.651646 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:37:11.651513    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/b9853529-152c-4642-ab14-885220711fb3-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"b9853529-152c-4642-ab14-885220711fb3\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 17 17:37:11.651646 ip-10-0-143-74 kubenswrapper[2575]: E0917 17:37:11.651529    2575 secret.go:189] Couldn't get secret openshift-monitoring/alertmanager-main-tls: secret "alertmanager-main-tls" not found
Sep 17 17:37:11.651646 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:37:11.651550    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/b9853529-152c-4642-ab14-885220711fb3-web-config\") pod \"alertmanager-main-0\" (UID: \"b9853529-152c-4642-ab14-885220711fb3\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 17 17:37:11.651646 ip-10-0-143-74 kubenswrapper[2575]: E0917 17:37:11.651591    2575 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b9853529-152c-4642-ab14-885220711fb3-secret-alertmanager-main-tls podName:b9853529-152c-4642-ab14-885220711fb3 nodeName:}" failed. No retries permitted until 2026-09-17 17:37:12.151571314 +0000 UTC m=+151.781901284 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "secret-alertmanager-main-tls" (UniqueName: "kubernetes.io/secret/b9853529-152c-4642-ab14-885220711fb3-secret-alertmanager-main-tls") pod "alertmanager-main-0" (UID: "b9853529-152c-4642-ab14-885220711fb3") : secret "alertmanager-main-tls" not found
Sep 17 17:37:11.651646 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:37:11.651609    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/b9853529-152c-4642-ab14-885220711fb3-config-out\") pod \"alertmanager-main-0\" (UID: \"b9853529-152c-4642-ab14-885220711fb3\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 17 17:37:11.651646 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:37:11.651640    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/b9853529-152c-4642-ab14-885220711fb3-tls-assets\") pod \"alertmanager-main-0\" (UID: \"b9853529-152c-4642-ab14-885220711fb3\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 17 17:37:11.652229 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:37:11.652204    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/b9853529-152c-4642-ab14-885220711fb3-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"b9853529-152c-4642-ab14-885220711fb3\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 17 17:37:11.652740 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:37:11.652676    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/b9853529-152c-4642-ab14-885220711fb3-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"b9853529-152c-4642-ab14-885220711fb3\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 17 17:37:11.652829 ip-10-0-143-74 kubenswrapper[2575]: E0917 17:37:11.652815    2575 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/b9853529-152c-4642-ab14-885220711fb3-alertmanager-trusted-ca-bundle podName:b9853529-152c-4642-ab14-885220711fb3 nodeName:}" failed. No retries permitted until 2026-09-17 17:37:12.152797461 +0000 UTC m=+151.783127428 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "alertmanager-trusted-ca-bundle" (UniqueName: "kubernetes.io/configmap/b9853529-152c-4642-ab14-885220711fb3-alertmanager-trusted-ca-bundle") pod "alertmanager-main-0" (UID: "b9853529-152c-4642-ab14-885220711fb3") : configmap references non-existent config key: ca-bundle.crt
Sep 17 17:37:11.654427 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:37:11.654403    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/b9853529-152c-4642-ab14-885220711fb3-web-config\") pod \"alertmanager-main-0\" (UID: \"b9853529-152c-4642-ab14-885220711fb3\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 17 17:37:11.654546 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:37:11.654512    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/b9853529-152c-4642-ab14-885220711fb3-config-volume\") pod \"alertmanager-main-0\" (UID: \"b9853529-152c-4642-ab14-885220711fb3\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 17 17:37:11.654679 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:37:11.654657    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/b9853529-152c-4642-ab14-885220711fb3-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"b9853529-152c-4642-ab14-885220711fb3\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 17 17:37:11.655576 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:37:11.655428    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/b9853529-152c-4642-ab14-885220711fb3-config-out\") pod \"alertmanager-main-0\" (UID: \"b9853529-152c-4642-ab14-885220711fb3\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 17 17:37:11.655839 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:37:11.655821    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/b9853529-152c-4642-ab14-885220711fb3-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"b9853529-152c-4642-ab14-885220711fb3\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 17 17:37:11.655944 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:37:11.655924    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/b9853529-152c-4642-ab14-885220711fb3-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"b9853529-152c-4642-ab14-885220711fb3\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 17 17:37:11.656197 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:37:11.656175    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/b9853529-152c-4642-ab14-885220711fb3-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"b9853529-152c-4642-ab14-885220711fb3\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 17 17:37:11.656386 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:37:11.656369    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/b9853529-152c-4642-ab14-885220711fb3-tls-assets\") pod \"alertmanager-main-0\" (UID: \"b9853529-152c-4642-ab14-885220711fb3\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 17 17:37:11.670389 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:37:11.670368    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-mnzn6\" (UniqueName: \"kubernetes.io/projected/b9853529-152c-4642-ab14-885220711fb3-kube-api-access-mnzn6\") pod \"alertmanager-main-0\" (UID: \"b9853529-152c-4642-ab14-885220711fb3\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 17 17:37:11.833228 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:37:11.833190    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-djvzt_1d07e68d-9943-420f-a6a0-5f477792fabd/konnectivity-agent/0.log" limit=1024
Sep 17 17:37:12.157124 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:37:12.157086    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/b9853529-152c-4642-ab14-885220711fb3-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"b9853529-152c-4642-ab14-885220711fb3\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 17 17:37:12.157587 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:37:12.157215    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/b9853529-152c-4642-ab14-885220711fb3-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"b9853529-152c-4642-ab14-885220711fb3\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 17 17:37:12.157951 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:37:12.157925    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/b9853529-152c-4642-ab14-885220711fb3-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"b9853529-152c-4642-ab14-885220711fb3\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 17 17:37:12.160026 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:37:12.159997    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/b9853529-152c-4642-ab14-885220711fb3-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"b9853529-152c-4642-ab14-885220711fb3\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 17 17:37:12.317502 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:37:12.317471    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0"
Sep 17 17:37:12.419411 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:37:12.419337    2575 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/thanos-querier-569d74d969-7c5rn"]
Sep 17 17:37:12.423081 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:37:12.423067    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/thanos-querier-569d74d969-7c5rn"
Sep 17 17:37:12.425738 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:37:12.425719    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-grpc-tls-533tb2i70t3n\""
Sep 17 17:37:12.425949 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:37:12.425936    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-tls\""
Sep 17 17:37:12.426435 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:37:12.426422    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-kube-rbac-proxy-metrics\""
Sep 17 17:37:12.426737 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:37:12.426720    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-kube-rbac-proxy-rules\""
Sep 17 17:37:12.426793 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:37:12.426776    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-kube-rbac-proxy-web\""
Sep 17 17:37:12.426843 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:37:12.426782    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-kube-rbac-proxy\""
Sep 17 17:37:12.427100 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:37:12.427085    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-dockercfg-gr7xf\""
Sep 17 17:37:12.438488 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:37:12.438465    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/thanos-querier-569d74d969-7c5rn"]
Sep 17 17:37:12.561150 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:37:12.561120    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-rules\" (UniqueName: \"kubernetes.io/secret/fccda717-3dbe-4bb3-8bd0-d4c8bcb2855f-secret-thanos-querier-kube-rbac-proxy-rules\") pod \"thanos-querier-569d74d969-7c5rn\" (UID: \"fccda717-3dbe-4bb3-8bd0-d4c8bcb2855f\") " pod="openshift-monitoring/thanos-querier-569d74d969-7c5rn"
Sep 17 17:37:12.561150 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:37:12.561157    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-tls\" (UniqueName: \"kubernetes.io/secret/fccda717-3dbe-4bb3-8bd0-d4c8bcb2855f-secret-thanos-querier-tls\") pod \"thanos-querier-569d74d969-7c5rn\" (UID: \"fccda717-3dbe-4bb3-8bd0-d4c8bcb2855f\") " pod="openshift-monitoring/thanos-querier-569d74d969-7c5rn"
Sep 17 17:37:12.561355 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:37:12.561199    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/fccda717-3dbe-4bb3-8bd0-d4c8bcb2855f-secret-thanos-querier-kube-rbac-proxy\") pod \"thanos-querier-569d74d969-7c5rn\" (UID: \"fccda717-3dbe-4bb3-8bd0-d4c8bcb2855f\") " pod="openshift-monitoring/thanos-querier-569d74d969-7c5rn"
Sep 17 17:37:12.561355 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:37:12.561222    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-metrics\" (UniqueName: \"kubernetes.io/secret/fccda717-3dbe-4bb3-8bd0-d4c8bcb2855f-secret-thanos-querier-kube-rbac-proxy-metrics\") pod \"thanos-querier-569d74d969-7c5rn\" (UID: \"fccda717-3dbe-4bb3-8bd0-d4c8bcb2855f\") " pod="openshift-monitoring/thanos-querier-569d74d969-7c5rn"
Sep 17 17:37:12.561355 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:37:12.561252    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/fccda717-3dbe-4bb3-8bd0-d4c8bcb2855f-secret-grpc-tls\") pod \"thanos-querier-569d74d969-7c5rn\" (UID: \"fccda717-3dbe-4bb3-8bd0-d4c8bcb2855f\") " pod="openshift-monitoring/thanos-querier-569d74d969-7c5rn"
Sep 17 17:37:12.561355 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:37:12.561296    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/fccda717-3dbe-4bb3-8bd0-d4c8bcb2855f-secret-thanos-querier-kube-rbac-proxy-web\") pod \"thanos-querier-569d74d969-7c5rn\" (UID: \"fccda717-3dbe-4bb3-8bd0-d4c8bcb2855f\") " pod="openshift-monitoring/thanos-querier-569d74d969-7c5rn"
Sep 17 17:37:12.561355 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:37:12.561329    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/fccda717-3dbe-4bb3-8bd0-d4c8bcb2855f-metrics-client-ca\") pod \"thanos-querier-569d74d969-7c5rn\" (UID: \"fccda717-3dbe-4bb3-8bd0-d4c8bcb2855f\") " pod="openshift-monitoring/thanos-querier-569d74d969-7c5rn"
Sep 17 17:37:12.561547 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:37:12.561387    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dv2vf\" (UniqueName: \"kubernetes.io/projected/fccda717-3dbe-4bb3-8bd0-d4c8bcb2855f-kube-api-access-dv2vf\") pod \"thanos-querier-569d74d969-7c5rn\" (UID: \"fccda717-3dbe-4bb3-8bd0-d4c8bcb2855f\") " pod="openshift-monitoring/thanos-querier-569d74d969-7c5rn"
Sep 17 17:37:12.628156 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:37:12.628123    2575 generic.go:356] "Generic (PLEG): container finished" podID="748f74c6-2a2e-4d13-898e-662858625d39" containerID="48f06ae26534fd771c9f006623b1766b2660f9252dcceb02f33e0db02e4faa57" exitCode=0
Sep 17 17:37:12.628354 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:37:12.628203    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-ksdnc" event={"ID":"748f74c6-2a2e-4d13-898e-662858625d39","Type":"ContainerDied","Data":"48f06ae26534fd771c9f006623b1766b2660f9252dcceb02f33e0db02e4faa57"}
Sep 17 17:37:12.630523 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:37:12.630496    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/openshift-state-metrics-86c5bc79d6-hqncr" event={"ID":"8dfa0270-d733-4438-b1e6-dde2f283d99b","Type":"ContainerStarted","Data":"99d591fef672144ef353663e2566522f99d8534e3650bf3f71b1fabb9e249717"}
Sep 17 17:37:12.648883 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:37:12.648863    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/alertmanager-main-0"]
Sep 17 17:37:12.661374 ip-10-0-143-74 kubenswrapper[2575]: W0917 17:37:12.661350    2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb9853529_152c_4642_ab14_885220711fb3.slice/crio-37eb78b233cffe72dd7f40f471cc2d9f56d24c842ee78aa80e535a32d2c08672 WatchSource:0}: Error finding container 37eb78b233cffe72dd7f40f471cc2d9f56d24c842ee78aa80e535a32d2c08672: Status 404 returned error can't find the container with id 37eb78b233cffe72dd7f40f471cc2d9f56d24c842ee78aa80e535a32d2c08672
Sep 17 17:37:12.662509 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:37:12.662487    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/fccda717-3dbe-4bb3-8bd0-d4c8bcb2855f-secret-grpc-tls\") pod \"thanos-querier-569d74d969-7c5rn\" (UID: \"fccda717-3dbe-4bb3-8bd0-d4c8bcb2855f\") " pod="openshift-monitoring/thanos-querier-569d74d969-7c5rn"
Sep 17 17:37:12.662603 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:37:12.662542    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/fccda717-3dbe-4bb3-8bd0-d4c8bcb2855f-secret-thanos-querier-kube-rbac-proxy-web\") pod \"thanos-querier-569d74d969-7c5rn\" (UID: \"fccda717-3dbe-4bb3-8bd0-d4c8bcb2855f\") " pod="openshift-monitoring/thanos-querier-569d74d969-7c5rn"
Sep 17 17:37:12.662603 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:37:12.662588    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/fccda717-3dbe-4bb3-8bd0-d4c8bcb2855f-metrics-client-ca\") pod \"thanos-querier-569d74d969-7c5rn\" (UID: \"fccda717-3dbe-4bb3-8bd0-d4c8bcb2855f\") " pod="openshift-monitoring/thanos-querier-569d74d969-7c5rn"
Sep 17 17:37:12.662690 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:37:12.662611    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dv2vf\" (UniqueName: \"kubernetes.io/projected/fccda717-3dbe-4bb3-8bd0-d4c8bcb2855f-kube-api-access-dv2vf\") pod \"thanos-querier-569d74d969-7c5rn\" (UID: \"fccda717-3dbe-4bb3-8bd0-d4c8bcb2855f\") " pod="openshift-monitoring/thanos-querier-569d74d969-7c5rn"
Sep 17 17:37:12.662690 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:37:12.662650    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-rules\" (UniqueName: \"kubernetes.io/secret/fccda717-3dbe-4bb3-8bd0-d4c8bcb2855f-secret-thanos-querier-kube-rbac-proxy-rules\") pod \"thanos-querier-569d74d969-7c5rn\" (UID: \"fccda717-3dbe-4bb3-8bd0-d4c8bcb2855f\") " pod="openshift-monitoring/thanos-querier-569d74d969-7c5rn"
Sep 17 17:37:12.662758 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:37:12.662685    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-tls\" (UniqueName: \"kubernetes.io/secret/fccda717-3dbe-4bb3-8bd0-d4c8bcb2855f-secret-thanos-querier-tls\") pod \"thanos-querier-569d74d969-7c5rn\" (UID: \"fccda717-3dbe-4bb3-8bd0-d4c8bcb2855f\") " pod="openshift-monitoring/thanos-querier-569d74d969-7c5rn"
Sep 17 17:37:12.662758 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:37:12.662732    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/fccda717-3dbe-4bb3-8bd0-d4c8bcb2855f-secret-thanos-querier-kube-rbac-proxy\") pod \"thanos-querier-569d74d969-7c5rn\" (UID: \"fccda717-3dbe-4bb3-8bd0-d4c8bcb2855f\") " pod="openshift-monitoring/thanos-querier-569d74d969-7c5rn"
Sep 17 17:37:12.662828 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:37:12.662759    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-metrics\" (UniqueName: \"kubernetes.io/secret/fccda717-3dbe-4bb3-8bd0-d4c8bcb2855f-secret-thanos-querier-kube-rbac-proxy-metrics\") pod \"thanos-querier-569d74d969-7c5rn\" (UID: \"fccda717-3dbe-4bb3-8bd0-d4c8bcb2855f\") " pod="openshift-monitoring/thanos-querier-569d74d969-7c5rn"
Sep 17 17:37:12.663353 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:37:12.663327    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/fccda717-3dbe-4bb3-8bd0-d4c8bcb2855f-metrics-client-ca\") pod \"thanos-querier-569d74d969-7c5rn\" (UID: \"fccda717-3dbe-4bb3-8bd0-d4c8bcb2855f\") " pod="openshift-monitoring/thanos-querier-569d74d969-7c5rn"
Sep 17 17:37:12.665595 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:37:12.665546    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/fccda717-3dbe-4bb3-8bd0-d4c8bcb2855f-secret-thanos-querier-kube-rbac-proxy-web\") pod \"thanos-querier-569d74d969-7c5rn\" (UID: \"fccda717-3dbe-4bb3-8bd0-d4c8bcb2855f\") " pod="openshift-monitoring/thanos-querier-569d74d969-7c5rn"
Sep 17 17:37:12.665771 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:37:12.665726    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/fccda717-3dbe-4bb3-8bd0-d4c8bcb2855f-secret-grpc-tls\") pod \"thanos-querier-569d74d969-7c5rn\" (UID: \"fccda717-3dbe-4bb3-8bd0-d4c8bcb2855f\") " pod="openshift-monitoring/thanos-querier-569d74d969-7c5rn"
Sep 17 17:37:12.665860 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:37:12.665774    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-kube-rbac-proxy-metrics\" (UniqueName: \"kubernetes.io/secret/fccda717-3dbe-4bb3-8bd0-d4c8bcb2855f-secret-thanos-querier-kube-rbac-proxy-metrics\") pod \"thanos-querier-569d74d969-7c5rn\" (UID: \"fccda717-3dbe-4bb3-8bd0-d4c8bcb2855f\") " pod="openshift-monitoring/thanos-querier-569d74d969-7c5rn"
Sep 17 17:37:12.666346 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:37:12.666325    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-kube-rbac-proxy-rules\" (UniqueName: \"kubernetes.io/secret/fccda717-3dbe-4bb3-8bd0-d4c8bcb2855f-secret-thanos-querier-kube-rbac-proxy-rules\") pod \"thanos-querier-569d74d969-7c5rn\" (UID: \"fccda717-3dbe-4bb3-8bd0-d4c8bcb2855f\") " pod="openshift-monitoring/thanos-querier-569d74d969-7c5rn"
Sep 17 17:37:12.666413 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:37:12.666393    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-tls\" (UniqueName: \"kubernetes.io/secret/fccda717-3dbe-4bb3-8bd0-d4c8bcb2855f-secret-thanos-querier-tls\") pod \"thanos-querier-569d74d969-7c5rn\" (UID: \"fccda717-3dbe-4bb3-8bd0-d4c8bcb2855f\") " pod="openshift-monitoring/thanos-querier-569d74d969-7c5rn"
Sep 17 17:37:12.666521 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:37:12.666500    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/fccda717-3dbe-4bb3-8bd0-d4c8bcb2855f-secret-thanos-querier-kube-rbac-proxy\") pod \"thanos-querier-569d74d969-7c5rn\" (UID: \"fccda717-3dbe-4bb3-8bd0-d4c8bcb2855f\") " pod="openshift-monitoring/thanos-querier-569d74d969-7c5rn"
Sep 17 17:37:12.677925 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:37:12.677870    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-dv2vf\" (UniqueName: \"kubernetes.io/projected/fccda717-3dbe-4bb3-8bd0-d4c8bcb2855f-kube-api-access-dv2vf\") pod \"thanos-querier-569d74d969-7c5rn\" (UID: \"fccda717-3dbe-4bb3-8bd0-d4c8bcb2855f\") " pod="openshift-monitoring/thanos-querier-569d74d969-7c5rn"
Sep 17 17:37:12.729875 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:37:12.729826    2575 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/openshift-state-metrics-86c5bc79d6-hqncr" podStartSLOduration=1.6663379 podStartE2EDuration="2.729804138s" totalImagesPullingTime="1.063466238s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-17 17:37:10 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-17 17:37:12.728182452 +0000 UTC m=+152.358512428" watchObservedRunningTime="2026-09-17 17:37:12.729804138 +0000 UTC m=+152.360134114"
Sep 17 17:37:12.732671 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:37:12.732647    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/thanos-querier-569d74d969-7c5rn"
Sep 17 17:37:13.018847 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:37:13.018752    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/thanos-querier-569d74d969-7c5rn"]
Sep 17 17:37:13.021174 ip-10-0-143-74 kubenswrapper[2575]: W0917 17:37:13.021143    2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podfccda717_3dbe_4bb3_8bd0_d4c8bcb2855f.slice/crio-96ad6fa51a6e49b4ad6b5070cc00e55407576bc8a11b2684922fee1f1d216f1a WatchSource:0}: Error finding container 96ad6fa51a6e49b4ad6b5070cc00e55407576bc8a11b2684922fee1f1d216f1a: Status 404 returned error can't find the container with id 96ad6fa51a6e49b4ad6b5070cc00e55407576bc8a11b2684922fee1f1d216f1a
Sep 17 17:37:13.387972 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:37:13.387942    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-djvzt_1d07e68d-9943-420f-a6a0-5f477792fabd/konnectivity-agent/0.log" limit=1024
Sep 17 17:37:13.488997 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:37:13.488952    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-djvzt_1d07e68d-9943-420f-a6a0-5f477792fabd/konnectivity-agent/0.log" limit=1024
Sep 17 17:37:13.637708 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:37:13.637669    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-569d74d969-7c5rn" event={"ID":"fccda717-3dbe-4bb3-8bd0-d4c8bcb2855f","Type":"ContainerStarted","Data":"96ad6fa51a6e49b4ad6b5070cc00e55407576bc8a11b2684922fee1f1d216f1a"}
Sep 17 17:37:13.640406 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:37:13.640338    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-ksdnc" event={"ID":"748f74c6-2a2e-4d13-898e-662858625d39","Type":"ContainerStarted","Data":"c1f6da8440c135ea1e43c0236c66fe742a01605b7c6daaef3471dfdf07cfa256"}
Sep 17 17:37:13.640406 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:37:13.640390    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-ksdnc" event={"ID":"748f74c6-2a2e-4d13-898e-662858625d39","Type":"ContainerStarted","Data":"cc458aceaae4adfff7d0203d2fc5cc26d2fb89fccc797132098989dc96dbf4eb"}
Sep 17 17:37:13.642649 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:37:13.642625    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"b9853529-152c-4642-ab14-885220711fb3","Type":"ContainerStarted","Data":"37eb78b233cffe72dd7f40f471cc2d9f56d24c842ee78aa80e535a32d2c08672"}
Sep 17 17:37:13.668423 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:37:13.668373    2575 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/node-exporter-ksdnc" podStartSLOduration=2.707713206 podStartE2EDuration="3.668357003s" totalImagesPullingTime="890.878339ms" totalInitContainerRuntime="69.765458ms" isStatefulPod=true podCreationTimestamp="2026-09-17 17:37:10 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-17 17:37:13.666725867 +0000 UTC m=+153.297055842" watchObservedRunningTime="2026-09-17 17:37:13.668357003 +0000 UTC m=+153.298686977"
Sep 17 17:37:13.836013 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:37:13.835960    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-djvzt_1d07e68d-9943-420f-a6a0-5f477792fabd/konnectivity-agent/0.log" limit=1024
Sep 17 17:37:13.940332 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:37:13.940301    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-djvzt_1d07e68d-9943-420f-a6a0-5f477792fabd/konnectivity-agent/0.log" limit=1024
Sep 17 17:37:14.648146 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:37:14.648110    2575 generic.go:356] "Generic (PLEG): container finished" podID="b9853529-152c-4642-ab14-885220711fb3" containerID="74d8a55f51add720c843cb3cd5a3b44c90f3d492827af6ffb3622c08f229932f" exitCode=0
Sep 17 17:37:14.648630 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:37:14.648207    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"b9853529-152c-4642-ab14-885220711fb3","Type":"ContainerDied","Data":"74d8a55f51add720c843cb3cd5a3b44c90f3d492827af6ffb3622c08f229932f"}
Sep 17 17:37:15.053844 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:37:15.053772    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-djvzt_1d07e68d-9943-420f-a6a0-5f477792fabd/konnectivity-agent/0.log" limit=1024
Sep 17 17:37:15.655380 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:37:15.655343    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-569d74d969-7c5rn" event={"ID":"fccda717-3dbe-4bb3-8bd0-d4c8bcb2855f","Type":"ContainerStarted","Data":"97b136ae298995ba8605757de0da08e724af2027e403cc7ddb2cb63b07bfd5ef"}
Sep 17 17:37:15.655812 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:37:15.655482    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-569d74d969-7c5rn" event={"ID":"fccda717-3dbe-4bb3-8bd0-d4c8bcb2855f","Type":"ContainerStarted","Data":"571b45ff27c7d5fae5e738fe647cfc3dedf97da77b648e2acb1eab5b67457f3a"}
Sep 17 17:37:15.655812 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:37:15.655529    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-569d74d969-7c5rn" event={"ID":"fccda717-3dbe-4bb3-8bd0-d4c8bcb2855f","Type":"ContainerStarted","Data":"7b4875933a2e3f197c2d21426d7d1b0a6cd599ece2d6219ade7d16b8d05f0e6b"}
Sep 17 17:37:15.802344 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:37:15.802306    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-djvzt_1d07e68d-9943-420f-a6a0-5f477792fabd/konnectivity-agent/0.log" limit=1024
Sep 17 17:37:16.662511 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:37:16.662479    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"b9853529-152c-4642-ab14-885220711fb3","Type":"ContainerStarted","Data":"0918796955dd9f4a25172e460fe92d63b83ba858a54da5ba3ee6a5d05a6012cf"}
Sep 17 17:37:16.662972 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:37:16.662549    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"b9853529-152c-4642-ab14-885220711fb3","Type":"ContainerStarted","Data":"0fac4cef38830b460a8482cf465f160bf3923fdafdcd27569be7fa4f0b3e0080"}
Sep 17 17:37:16.662972 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:37:16.662585    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"b9853529-152c-4642-ab14-885220711fb3","Type":"ContainerStarted","Data":"e82c34fe62828aebd281698514a38ef042f4f65b74d83cf09dd85aca1554a485"}
Sep 17 17:37:16.662972 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:37:16.662613    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"b9853529-152c-4642-ab14-885220711fb3","Type":"ContainerStarted","Data":"9c5698b6a330acd5b26f7113dcbb5215177a2cea12660a6846cb1c9aaa09808d"}
Sep 17 17:37:16.662972 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:37:16.662643    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"b9853529-152c-4642-ab14-885220711fb3","Type":"ContainerStarted","Data":"743b4c1518861787e247f1f7878550e864aa794bbbcd9eb1203c7bbe2c8f7981"}
Sep 17 17:37:16.662972 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:37:16.662669    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"b9853529-152c-4642-ab14-885220711fb3","Type":"ContainerStarted","Data":"548e1952d675442d7f2ff0e0e972e20d63d2253801c201c686551c999cf614ed"}
Sep 17 17:37:16.665564 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:37:16.665541    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-569d74d969-7c5rn" event={"ID":"fccda717-3dbe-4bb3-8bd0-d4c8bcb2855f","Type":"ContainerStarted","Data":"bf1d823b6291a23aedadf6a6c02b5362b9627a4b294a3bbc9f5f073e6b1cb10d"}
Sep 17 17:37:16.665669 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:37:16.665593    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-569d74d969-7c5rn" event={"ID":"fccda717-3dbe-4bb3-8bd0-d4c8bcb2855f","Type":"ContainerStarted","Data":"91bc5184dc7028510e08bbdc6b6193bed30b011f85392e3affeb0a5b4890a8d7"}
Sep 17 17:37:16.665669 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:37:16.665619    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-569d74d969-7c5rn" event={"ID":"fccda717-3dbe-4bb3-8bd0-d4c8bcb2855f","Type":"ContainerStarted","Data":"b1929534a3a212e126ed69f97e39353ff9266bae08b38874ec3e7c00b6d7b072"}
Sep 17 17:37:16.665669 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:37:16.665646    2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/thanos-querier-569d74d969-7c5rn"
Sep 17 17:37:16.699934 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:37:16.699888    2575 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/alertmanager-main-0" podStartSLOduration=3.184728319 podStartE2EDuration="5.699872029s" totalImagesPullingTime="2.499896509s" totalInitContainerRuntime="15.247201ms" isStatefulPod=false podCreationTimestamp="2026-09-17 17:37:11 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-17 17:37:16.698222294 +0000 UTC m=+156.328552269" watchObservedRunningTime="2026-09-17 17:37:16.699872029 +0000 UTC m=+156.330202004"
Sep 17 17:37:16.728083 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:37:16.728022    2575 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/thanos-querier-569d74d969-7c5rn" podStartSLOduration=1.6503880579999999 podStartE2EDuration="4.727998084s" totalImagesPullingTime="3.077610026s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-17 17:37:12 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-17 17:37:16.727348082 +0000 UTC m=+156.357678060" watchObservedRunningTime="2026-09-17 17:37:16.727998084 +0000 UTC m=+156.358328060"
Sep 17 17:37:16.827773 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:37:16.827734    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-djvzt_1d07e68d-9943-420f-a6a0-5f477792fabd/konnectivity-agent/0.log" limit=1024
Sep 17 17:37:19.208253 ip-10-0-143-74 kubenswrapper[2575]: E0917 17:37:19.208211    2575 pod_workers.go:1324] "Error syncing pod, skipping" err="unmounted volumes=[metrics-tls], unattached volumes=[], failed to process volumes=[]: context deadline exceeded" pod="openshift-dns/dns-default-tl8ds" podUID="b8b3d730-8877-45f1-8500-451ee8ac53a3"
Sep 17 17:37:19.676548 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:37:19.676515    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-tl8ds"
Sep 17 17:37:21.246997 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:37:21.246968    2575 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/downloads-6594f4c75-lblbf"]
Sep 17 17:37:21.251123 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:37:21.251106    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/downloads-6594f4c75-lblbf"
Sep 17 17:37:21.253765 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:37:21.253741    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"default-dockercfg-9mdcx\""
Sep 17 17:37:21.253765 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:37:21.253754    2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"kube-root-ca.crt\""
Sep 17 17:37:21.253765 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:37:21.253757    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-djvzt_1d07e68d-9943-420f-a6a0-5f477792fabd/konnectivity-agent/0.log" limit=1024
Sep 17 17:37:21.254002 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:37:21.253863    2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"openshift-service-ca.crt\""
Sep 17 17:37:21.265147 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:37:21.265117    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/downloads-6594f4c75-lblbf"]
Sep 17 17:37:21.341326 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:37:21.341283    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5hv66\" (UniqueName: \"kubernetes.io/projected/13f90145-45b6-49b9-a30e-bb2064b55386-kube-api-access-5hv66\") pod \"downloads-6594f4c75-lblbf\" (UID: \"13f90145-45b6-49b9-a30e-bb2064b55386\") " pod="openshift-console/downloads-6594f4c75-lblbf"
Sep 17 17:37:21.346965 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:37:21.346926    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-djvzt_1d07e68d-9943-420f-a6a0-5f477792fabd/konnectivity-agent/0.log" limit=1024
Sep 17 17:37:21.416315 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:37:21.416280    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-djvzt_1d07e68d-9943-420f-a6a0-5f477792fabd/konnectivity-agent/0.log" limit=1024
Sep 17 17:37:21.442556 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:37:21.442519    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5hv66\" (UniqueName: \"kubernetes.io/projected/13f90145-45b6-49b9-a30e-bb2064b55386-kube-api-access-5hv66\") pod \"downloads-6594f4c75-lblbf\" (UID: \"13f90145-45b6-49b9-a30e-bb2064b55386\") " pod="openshift-console/downloads-6594f4c75-lblbf"
Sep 17 17:37:21.451738 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:37:21.451709    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-5hv66\" (UniqueName: \"kubernetes.io/projected/13f90145-45b6-49b9-a30e-bb2064b55386-kube-api-access-5hv66\") pod \"downloads-6594f4c75-lblbf\" (UID: \"13f90145-45b6-49b9-a30e-bb2064b55386\") " pod="openshift-console/downloads-6594f4c75-lblbf"
Sep 17 17:37:21.533520 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:37:21.533416    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-djvzt_1d07e68d-9943-420f-a6a0-5f477792fabd/konnectivity-agent/0.log" limit=1024
Sep 17 17:37:21.560538 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:37:21.560513    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/downloads-6594f4c75-lblbf"
Sep 17 17:37:21.712502 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:37:21.712467    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-djvzt_1d07e68d-9943-420f-a6a0-5f477792fabd/konnectivity-agent/0.log" limit=1024
Sep 17 17:37:21.793039 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:37:21.792947    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-djvzt_1d07e68d-9943-420f-a6a0-5f477792fabd/konnectivity-agent/0.log" limit=1024
Sep 17 17:37:21.832537 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:37:21.832499    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/downloads-6594f4c75-lblbf"]
Sep 17 17:37:21.835990 ip-10-0-143-74 kubenswrapper[2575]: W0917 17:37:21.835956    2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod13f90145_45b6_49b9_a30e_bb2064b55386.slice/crio-c08443ecdefe6d5c03a209e7670ec06222a3d5c07194afce12d9e78030e45bf1 WatchSource:0}: Error finding container c08443ecdefe6d5c03a209e7670ec06222a3d5c07194afce12d9e78030e45bf1: Status 404 returned error can't find the container with id c08443ecdefe6d5c03a209e7670ec06222a3d5c07194afce12d9e78030e45bf1
Sep 17 17:37:21.986371 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:37:21.986334    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-djvzt_1d07e68d-9943-420f-a6a0-5f477792fabd/konnectivity-agent/0.log" limit=1024
Sep 17 17:37:22.112746 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:37:22.112666    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-djvzt_1d07e68d-9943-420f-a6a0-5f477792fabd/konnectivity-agent/0.log" limit=1024
Sep 17 17:37:22.213422 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:37:22.213388    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-djvzt_1d07e68d-9943-420f-a6a0-5f477792fabd/konnectivity-agent/0.log" limit=1024
Sep 17 17:37:22.491298 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:37:22.491236    2575 patch_prober.go:28] interesting pod/image-registry-567fd9f7c4-z54rn container/registry namespace/openshift-image-registry: Liveness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body={"errors":[{"code":"UNAVAILABLE","message":"service unavailable","detail":"health check failed: please see /debug/health"}]}
Sep 17 17:37:22.491776 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:37:22.491319    2575 prober.go:120] "Probe failed" probeType="Liveness" pod="openshift-image-registry/image-registry-567fd9f7c4-z54rn" podUID="9476f49c-219d-4d93-8ac2-b98df60c6b22" containerName="registry" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Sep 17 17:37:22.677331 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:37:22.677091    2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/thanos-querier-569d74d969-7c5rn"
Sep 17 17:37:22.689109 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:37:22.689077    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/downloads-6594f4c75-lblbf" event={"ID":"13f90145-45b6-49b9-a30e-bb2064b55386","Type":"ContainerStarted","Data":"c08443ecdefe6d5c03a209e7670ec06222a3d5c07194afce12d9e78030e45bf1"}
Sep 17 17:37:22.725972 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:37:22.725933    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-djvzt_1d07e68d-9943-420f-a6a0-5f477792fabd/konnectivity-agent/0.log" limit=1024
Sep 17 17:37:22.955755 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:37:22.955657    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-djvzt_1d07e68d-9943-420f-a6a0-5f477792fabd/konnectivity-agent/0.log" limit=1024
Sep 17 17:37:23.053706 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:37:23.053617    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-djvzt_1d07e68d-9943-420f-a6a0-5f477792fabd/konnectivity-agent/0.log" limit=1024
Sep 17 17:37:23.151832 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:37:23.151793    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-djvzt_1d07e68d-9943-420f-a6a0-5f477792fabd/konnectivity-agent/0.log" limit=1024
Sep 17 17:37:24.067475 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:37:24.067425    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/b8b3d730-8877-45f1-8500-451ee8ac53a3-metrics-tls\") pod \"dns-default-tl8ds\" (UID: \"b8b3d730-8877-45f1-8500-451ee8ac53a3\") " pod="openshift-dns/dns-default-tl8ds"
Sep 17 17:37:24.070087 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:37:24.070060    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/b8b3d730-8877-45f1-8500-451ee8ac53a3-metrics-tls\") pod \"dns-default-tl8ds\" (UID: \"b8b3d730-8877-45f1-8500-451ee8ac53a3\") " pod="openshift-dns/dns-default-tl8ds"
Sep 17 17:37:24.180822 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:37:24.180795    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-dng7x\""
Sep 17 17:37:24.188454 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:37:24.188417    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-tl8ds"
Sep 17 17:37:24.473123 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:37:24.473087    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-tl8ds"]
Sep 17 17:37:24.477530 ip-10-0-143-74 kubenswrapper[2575]: W0917 17:37:24.477483    2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb8b3d730_8877_45f1_8500_451ee8ac53a3.slice/crio-9fbcc08f79e6c368a4d7d089bc60c0542025183c602bdaf3426c42f54996cc5e WatchSource:0}: Error finding container 9fbcc08f79e6c368a4d7d089bc60c0542025183c602bdaf3426c42f54996cc5e: Status 404 returned error can't find the container with id 9fbcc08f79e6c368a4d7d089bc60c0542025183c602bdaf3426c42f54996cc5e
Sep 17 17:37:24.594919 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:37:24.594876    2575 patch_prober.go:28] interesting pod/image-registry-567fd9f7c4-z54rn container/registry namespace/openshift-image-registry: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body={"errors":[{"code":"UNAVAILABLE","message":"service unavailable","detail":"health check failed: please see /debug/health"}]}
Sep 17 17:37:24.595079 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:37:24.594929    2575 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-image-registry/image-registry-567fd9f7c4-z54rn" podUID="9476f49c-219d-4d93-8ac2-b98df60c6b22" containerName="registry" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Sep 17 17:37:24.698958 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:37:24.698919    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-tl8ds" event={"ID":"b8b3d730-8877-45f1-8500-451ee8ac53a3","Type":"ContainerStarted","Data":"9fbcc08f79e6c368a4d7d089bc60c0542025183c602bdaf3426c42f54996cc5e"}
Sep 17 17:37:24.950785 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:37:24.950748    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-djvzt_1d07e68d-9943-420f-a6a0-5f477792fabd/konnectivity-agent/0.log" limit=1024
Sep 17 17:37:25.032780 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:37:25.032735    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-djvzt_1d07e68d-9943-420f-a6a0-5f477792fabd/konnectivity-agent/0.log" limit=1024
Sep 17 17:37:26.709720 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:37:26.709684    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-tl8ds" event={"ID":"b8b3d730-8877-45f1-8500-451ee8ac53a3","Type":"ContainerStarted","Data":"3c2ef14e9ac022ef2b9bb0f45aeff0f35e8e8a27a43537ab050f5459fe9d464b"}
Sep 17 17:37:26.710091 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:37:26.709737    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-tl8ds" event={"ID":"b8b3d730-8877-45f1-8500-451ee8ac53a3","Type":"ContainerStarted","Data":"aa21cf0c809fb837faf6c990c7f84dc90d131447d1c7a3b39951eb230e58b3a6"}
Sep 17 17:37:26.710091 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:37:26.709806    2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-dns/dns-default-tl8ds"
Sep 17 17:37:26.730261 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:37:26.730218    2575 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/dns-default-tl8ds" podStartSLOduration=128.843146036 podStartE2EDuration="2m10.730199948s" totalImagesPullingTime="1.887053912s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-17 17:35:16 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-17 17:37:26.728573601 +0000 UTC m=+166.358903578" watchObservedRunningTime="2026-09-17 17:37:26.730199948 +0000 UTC m=+166.360529923"
Sep 17 17:37:28.161193 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:37:28.161155    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-djvzt_1d07e68d-9943-420f-a6a0-5f477792fabd/konnectivity-agent/0.log" limit=1024
Sep 17 17:37:28.263620 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:37:28.263580    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-djvzt_1d07e68d-9943-420f-a6a0-5f477792fabd/konnectivity-agent/0.log" limit=1024
Sep 17 17:37:31.161059 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:37:31.161026    2575 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-7d486c979b-bhv9r"]
Sep 17 17:37:31.165976 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:37:31.165950    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-7d486c979b-bhv9r"
Sep 17 17:37:31.168939 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:37:31.168912    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-oauth-config\""
Sep 17 17:37:31.169054 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:37:31.168920    2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"oauth-serving-cert\""
Sep 17 17:37:31.169914 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:37:31.169886    2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"service-ca\""
Sep 17 17:37:31.170015 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:37:31.169922    2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"console-config\""
Sep 17 17:37:31.170015 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:37:31.169933    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-dockercfg-zn79t\""
Sep 17 17:37:31.170015 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:37:31.169938    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-serving-cert\""
Sep 17 17:37:31.177392 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:37:31.177357    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-7d486c979b-bhv9r"]
Sep 17 17:37:31.228319 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:37:31.228287    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/a1b85463-8d53-400c-9e79-4a2991b5af22-console-serving-cert\") pod \"console-7d486c979b-bhv9r\" (UID: \"a1b85463-8d53-400c-9e79-4a2991b5af22\") " pod="openshift-console/console-7d486c979b-bhv9r"
Sep 17 17:37:31.228319 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:37:31.228324    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/a1b85463-8d53-400c-9e79-4a2991b5af22-service-ca\") pod \"console-7d486c979b-bhv9r\" (UID: \"a1b85463-8d53-400c-9e79-4a2991b5af22\") " pod="openshift-console/console-7d486c979b-bhv9r"
Sep 17 17:37:31.228563 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:37:31.228430    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/a1b85463-8d53-400c-9e79-4a2991b5af22-console-config\") pod \"console-7d486c979b-bhv9r\" (UID: \"a1b85463-8d53-400c-9e79-4a2991b5af22\") " pod="openshift-console/console-7d486c979b-bhv9r"
Sep 17 17:37:31.228563 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:37:31.228468    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/a1b85463-8d53-400c-9e79-4a2991b5af22-console-oauth-config\") pod \"console-7d486c979b-bhv9r\" (UID: \"a1b85463-8d53-400c-9e79-4a2991b5af22\") " pod="openshift-console/console-7d486c979b-bhv9r"
Sep 17 17:37:31.228563 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:37:31.228490    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/a1b85463-8d53-400c-9e79-4a2991b5af22-oauth-serving-cert\") pod \"console-7d486c979b-bhv9r\" (UID: \"a1b85463-8d53-400c-9e79-4a2991b5af22\") " pod="openshift-console/console-7d486c979b-bhv9r"
Sep 17 17:37:31.228563 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:37:31.228520    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ns5hz\" (UniqueName: \"kubernetes.io/projected/a1b85463-8d53-400c-9e79-4a2991b5af22-kube-api-access-ns5hz\") pod \"console-7d486c979b-bhv9r\" (UID: \"a1b85463-8d53-400c-9e79-4a2991b5af22\") " pod="openshift-console/console-7d486c979b-bhv9r"
Sep 17 17:37:31.329140 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:37:31.329102    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/a1b85463-8d53-400c-9e79-4a2991b5af22-console-config\") pod \"console-7d486c979b-bhv9r\" (UID: \"a1b85463-8d53-400c-9e79-4a2991b5af22\") " pod="openshift-console/console-7d486c979b-bhv9r"
Sep 17 17:37:31.329326 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:37:31.329150    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/a1b85463-8d53-400c-9e79-4a2991b5af22-console-oauth-config\") pod \"console-7d486c979b-bhv9r\" (UID: \"a1b85463-8d53-400c-9e79-4a2991b5af22\") " pod="openshift-console/console-7d486c979b-bhv9r"
Sep 17 17:37:31.329326 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:37:31.329177    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/a1b85463-8d53-400c-9e79-4a2991b5af22-oauth-serving-cert\") pod \"console-7d486c979b-bhv9r\" (UID: \"a1b85463-8d53-400c-9e79-4a2991b5af22\") " pod="openshift-console/console-7d486c979b-bhv9r"
Sep 17 17:37:31.329326 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:37:31.329206    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ns5hz\" (UniqueName: \"kubernetes.io/projected/a1b85463-8d53-400c-9e79-4a2991b5af22-kube-api-access-ns5hz\") pod \"console-7d486c979b-bhv9r\" (UID: \"a1b85463-8d53-400c-9e79-4a2991b5af22\") " pod="openshift-console/console-7d486c979b-bhv9r"
Sep 17 17:37:31.329326 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:37:31.329303    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/a1b85463-8d53-400c-9e79-4a2991b5af22-console-serving-cert\") pod \"console-7d486c979b-bhv9r\" (UID: \"a1b85463-8d53-400c-9e79-4a2991b5af22\") " pod="openshift-console/console-7d486c979b-bhv9r"
Sep 17 17:37:31.329326 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:37:31.329324    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/a1b85463-8d53-400c-9e79-4a2991b5af22-service-ca\") pod \"console-7d486c979b-bhv9r\" (UID: \"a1b85463-8d53-400c-9e79-4a2991b5af22\") " pod="openshift-console/console-7d486c979b-bhv9r"
Sep 17 17:37:31.330196 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:37:31.330159    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/a1b85463-8d53-400c-9e79-4a2991b5af22-console-config\") pod \"console-7d486c979b-bhv9r\" (UID: \"a1b85463-8d53-400c-9e79-4a2991b5af22\") " pod="openshift-console/console-7d486c979b-bhv9r"
Sep 17 17:37:31.330316 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:37:31.330248    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/a1b85463-8d53-400c-9e79-4a2991b5af22-oauth-serving-cert\") pod \"console-7d486c979b-bhv9r\" (UID: \"a1b85463-8d53-400c-9e79-4a2991b5af22\") " pod="openshift-console/console-7d486c979b-bhv9r"
Sep 17 17:37:31.330416 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:37:31.330392    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/a1b85463-8d53-400c-9e79-4a2991b5af22-service-ca\") pod \"console-7d486c979b-bhv9r\" (UID: \"a1b85463-8d53-400c-9e79-4a2991b5af22\") " pod="openshift-console/console-7d486c979b-bhv9r"
Sep 17 17:37:31.332129 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:37:31.332111    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/a1b85463-8d53-400c-9e79-4a2991b5af22-console-oauth-config\") pod \"console-7d486c979b-bhv9r\" (UID: \"a1b85463-8d53-400c-9e79-4a2991b5af22\") " pod="openshift-console/console-7d486c979b-bhv9r"
Sep 17 17:37:31.332291 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:37:31.332271    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/a1b85463-8d53-400c-9e79-4a2991b5af22-console-serving-cert\") pod \"console-7d486c979b-bhv9r\" (UID: \"a1b85463-8d53-400c-9e79-4a2991b5af22\") " pod="openshift-console/console-7d486c979b-bhv9r"
Sep 17 17:37:31.339513 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:37:31.339492    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-ns5hz\" (UniqueName: \"kubernetes.io/projected/a1b85463-8d53-400c-9e79-4a2991b5af22-kube-api-access-ns5hz\") pod \"console-7d486c979b-bhv9r\" (UID: \"a1b85463-8d53-400c-9e79-4a2991b5af22\") " pod="openshift-console/console-7d486c979b-bhv9r"
Sep 17 17:37:31.477386 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:37:31.477302    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-7d486c979b-bhv9r"
Sep 17 17:37:32.489814 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:37:32.489774    2575 patch_prober.go:28] interesting pod/image-registry-567fd9f7c4-z54rn container/registry namespace/openshift-image-registry: Liveness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body={"errors":[{"code":"UNAVAILABLE","message":"service unavailable","detail":"health check failed: please see /debug/health"}]}
Sep 17 17:37:32.490314 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:37:32.489833    2575 prober.go:120] "Probe failed" probeType="Liveness" pod="openshift-image-registry/image-registry-567fd9f7c4-z54rn" podUID="9476f49c-219d-4d93-8ac2-b98df60c6b22" containerName="registry" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Sep 17 17:37:34.558286 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:37:34.558196    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-djvzt_1d07e68d-9943-420f-a6a0-5f477792fabd/konnectivity-agent/0.log" limit=1024
Sep 17 17:37:34.594079 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:37:34.594036    2575 patch_prober.go:28] interesting pod/image-registry-567fd9f7c4-z54rn container/registry namespace/openshift-image-registry: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body={"errors":[{"code":"UNAVAILABLE","message":"service unavailable","detail":"health check failed: please see /debug/health"}]}
Sep 17 17:37:34.594242 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:37:34.594106    2575 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-image-registry/image-registry-567fd9f7c4-z54rn" podUID="9476f49c-219d-4d93-8ac2-b98df60c6b22" containerName="registry" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Sep 17 17:37:34.655198 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:37:34.655159    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-djvzt_1d07e68d-9943-420f-a6a0-5f477792fabd/konnectivity-agent/0.log" limit=1024
Sep 17 17:37:36.717718 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:37:36.717687    2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-dns/dns-default-tl8ds"
Sep 17 17:37:38.210292 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:37:38.210223    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-7d486c979b-bhv9r"]
Sep 17 17:37:38.213272 ip-10-0-143-74 kubenswrapper[2575]: W0917 17:37:38.213234    2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda1b85463_8d53_400c_9e79_4a2991b5af22.slice/crio-abd60eb19a0250b4f8ff7910c4d0e5b35179ace35f8c0327f2e82ffa93a3e60a WatchSource:0}: Error finding container abd60eb19a0250b4f8ff7910c4d0e5b35179ace35f8c0327f2e82ffa93a3e60a: Status 404 returned error can't find the container with id abd60eb19a0250b4f8ff7910c4d0e5b35179ace35f8c0327f2e82ffa93a3e60a
Sep 17 17:37:38.769114 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:37:38.769008    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-7d486c979b-bhv9r" event={"ID":"a1b85463-8d53-400c-9e79-4a2991b5af22","Type":"ContainerStarted","Data":"abd60eb19a0250b4f8ff7910c4d0e5b35179ace35f8c0327f2e82ffa93a3e60a"}
Sep 17 17:37:38.774203 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:37:38.773298    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/downloads-6594f4c75-lblbf" event={"ID":"13f90145-45b6-49b9-a30e-bb2064b55386","Type":"ContainerStarted","Data":"f553c734d526aa3ccc809e688cfe0b981a06e59f492a4fef302e0b996f9a7236"}
Sep 17 17:37:38.774797 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:37:38.774743    2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/downloads-6594f4c75-lblbf"
Sep 17 17:37:38.785883 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:37:38.785825    2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/downloads-6594f4c75-lblbf"
Sep 17 17:37:38.796840 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:37:38.795030    2575 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/downloads-6594f4c75-lblbf" podStartSLOduration=1.614339179 podStartE2EDuration="17.795016216s" totalImagesPullingTime="16.180677037s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-17 17:37:21 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-17 17:37:38.794034471 +0000 UTC m=+178.424364447" watchObservedRunningTime="2026-09-17 17:37:38.795016216 +0000 UTC m=+178.425346190"
Sep 17 17:37:41.014132 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:37:41.014039    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-djvzt_1d07e68d-9943-420f-a6a0-5f477792fabd/konnectivity-agent/0.log" limit=1024
Sep 17 17:37:41.115251 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:37:41.115214    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-djvzt_1d07e68d-9943-420f-a6a0-5f477792fabd/konnectivity-agent/0.log" limit=1024
Sep 17 17:37:42.490933 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:37:42.490887    2575 patch_prober.go:28] interesting pod/image-registry-567fd9f7c4-z54rn container/registry namespace/openshift-image-registry: Liveness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body={"errors":[{"code":"UNAVAILABLE","message":"service unavailable","detail":"health check failed: please see /debug/health"}]}
Sep 17 17:37:42.491664 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:37:42.490954    2575 prober.go:120] "Probe failed" probeType="Liveness" pod="openshift-image-registry/image-registry-567fd9f7c4-z54rn" podUID="9476f49c-219d-4d93-8ac2-b98df60c6b22" containerName="registry" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Sep 17 17:37:42.491664 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:37:42.490995    2575 kubelet.go:2654] "SyncLoop (probe)" probe="liveness" status="unhealthy" pod="openshift-image-registry/image-registry-567fd9f7c4-z54rn"
Sep 17 17:37:42.491664 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:37:42.491584    2575 kuberuntime_manager.go:1192] "Message for Container of pod" containerName="registry" containerStatusID={"Type":"cri-o","ID":"0a71dba1484c75866e0f8f8097c166276f59fdd5829b7a7b6c537f02c5e42334"} pod="openshift-image-registry/image-registry-567fd9f7c4-z54rn" containerMessage="Container registry failed liveness probe, will be restarted"
Sep 17 17:37:42.496497 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:37:42.496462    2575 patch_prober.go:28] interesting pod/image-registry-567fd9f7c4-z54rn container/registry namespace/openshift-image-registry: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body={"errors":[{"code":"UNAVAILABLE","message":"service unavailable","detail":"health check failed: please see /debug/health"}]}
Sep 17 17:37:42.496651 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:37:42.496520    2575 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-image-registry/image-registry-567fd9f7c4-z54rn" podUID="9476f49c-219d-4d93-8ac2-b98df60c6b22" containerName="registry" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Sep 17 17:37:43.777589 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:37:43.777545    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-djvzt_1d07e68d-9943-420f-a6a0-5f477792fabd/konnectivity-agent/0.log" limit=1024
Sep 17 17:37:43.878470 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:37:43.878422    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-djvzt_1d07e68d-9943-420f-a6a0-5f477792fabd/konnectivity-agent/0.log" limit=1024
Sep 17 17:37:43.980039 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:37:43.979995    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-djvzt_1d07e68d-9943-420f-a6a0-5f477792fabd/konnectivity-agent/0.log" limit=1024
Sep 17 17:37:44.819269 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:37:44.819234    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-7d486c979b-bhv9r" event={"ID":"a1b85463-8d53-400c-9e79-4a2991b5af22","Type":"ContainerStarted","Data":"ba9651b4b371ea40b76382c65b71ecbda6f697e5b32931fe0ff2b5ef6693ee82"}
Sep 17 17:37:44.842572 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:37:44.842515    2575 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-7d486c979b-bhv9r" podStartSLOduration=8.056920972 podStartE2EDuration="13.842494048s" totalImagesPullingTime="5.785573076s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-17 17:37:31 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-17 17:37:44.841156718 +0000 UTC m=+184.471486709" watchObservedRunningTime="2026-09-17 17:37:44.842494048 +0000 UTC m=+184.472824023"
Sep 17 17:37:45.358068 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:37:45.358028    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-djvzt_1d07e68d-9943-420f-a6a0-5f477792fabd/konnectivity-agent/0.log" limit=1024
Sep 17 17:37:45.478721 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:37:45.478681    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-djvzt_1d07e68d-9943-420f-a6a0-5f477792fabd/konnectivity-agent/0.log" limit=1024
Sep 17 17:37:45.840318 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:37:45.840274    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-djvzt_1d07e68d-9943-420f-a6a0-5f477792fabd/konnectivity-agent/0.log" limit=1024
Sep 17 17:37:45.948625 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:37:45.948586    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-djvzt_1d07e68d-9943-420f-a6a0-5f477792fabd/konnectivity-agent/0.log" limit=1024
Sep 17 17:37:46.172352 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:37:46.172311    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-djvzt_1d07e68d-9943-420f-a6a0-5f477792fabd/konnectivity-agent/0.log" limit=1024
Sep 17 17:37:46.271583 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:37:46.271544    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-djvzt_1d07e68d-9943-420f-a6a0-5f477792fabd/konnectivity-agent/0.log" limit=1024
Sep 17 17:37:48.950634 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:37:48.950595    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-djvzt_1d07e68d-9943-420f-a6a0-5f477792fabd/konnectivity-agent/0.log" limit=1024
Sep 17 17:37:49.054186 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:37:49.054151    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-djvzt_1d07e68d-9943-420f-a6a0-5f477792fabd/konnectivity-agent/0.log" limit=1024
Sep 17 17:37:49.510240 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:37:49.510202    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-djvzt_1d07e68d-9943-420f-a6a0-5f477792fabd/konnectivity-agent/0.log" limit=1024
Sep 17 17:37:49.610572 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:37:49.610530    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-djvzt_1d07e68d-9943-420f-a6a0-5f477792fabd/konnectivity-agent/0.log" limit=1024
Sep 17 17:37:50.758640 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:37:50.758610    2575 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-7d486c979b-bhv9r"]
Sep 17 17:37:50.911639 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:37:50.911602    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-djvzt_1d07e68d-9943-420f-a6a0-5f477792fabd/konnectivity-agent/0.log" limit=1024
Sep 17 17:37:51.011752 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:37:51.011661    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-djvzt_1d07e68d-9943-420f-a6a0-5f477792fabd/konnectivity-agent/0.log" limit=1024
Sep 17 17:37:51.162729 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:37:51.162689    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-djvzt_1d07e68d-9943-420f-a6a0-5f477792fabd/konnectivity-agent/0.log" limit=1024
Sep 17 17:37:51.261046 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:37:51.261009    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-djvzt_1d07e68d-9943-420f-a6a0-5f477792fabd/konnectivity-agent/0.log" limit=1024
Sep 17 17:37:51.478378 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:37:51.478342    2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-7d486c979b-bhv9r"
Sep 17 17:37:52.010044 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:37:52.010008    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-djvzt_1d07e68d-9943-420f-a6a0-5f477792fabd/konnectivity-agent/0.log" limit=1024
Sep 17 17:37:52.128353 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:37:52.128308    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-djvzt_1d07e68d-9943-420f-a6a0-5f477792fabd/konnectivity-agent/0.log" limit=1024
Sep 17 17:37:52.496599 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:37:52.496568    2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-image-registry/image-registry-567fd9f7c4-z54rn"
Sep 17 17:37:52.711695 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:37:52.711659    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-djvzt_1d07e68d-9943-420f-a6a0-5f477792fabd/konnectivity-agent/0.log" limit=1024
Sep 17 17:37:52.862205 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:37:52.862099    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-djvzt_1d07e68d-9943-420f-a6a0-5f477792fabd/konnectivity-agent/0.log" limit=1024
Sep 17 17:37:54.402138 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:37:54.402097    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-djvzt_1d07e68d-9943-420f-a6a0-5f477792fabd/konnectivity-agent/0.log" limit=1024
Sep 17 17:37:54.494353 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:37:54.494318    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-djvzt_1d07e68d-9943-420f-a6a0-5f477792fabd/konnectivity-agent/0.log" limit=1024
Sep 17 17:37:54.898417 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:37:54.898359    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-djvzt_1d07e68d-9943-420f-a6a0-5f477792fabd/konnectivity-agent/0.log" limit=1024
Sep 17 17:37:54.988142 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:37:54.988113    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-djvzt_1d07e68d-9943-420f-a6a0-5f477792fabd/konnectivity-agent/0.log" limit=1024
Sep 17 17:37:58.136669 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:37:58.136626    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-djvzt_1d07e68d-9943-420f-a6a0-5f477792fabd/konnectivity-agent/0.log" limit=1024
Sep 17 17:37:58.202871 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:37:58.202837    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-djvzt_1d07e68d-9943-420f-a6a0-5f477792fabd/konnectivity-agent/0.log" limit=1024
Sep 17 17:38:02.892079 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:38:02.892038    2575 generic.go:356] "Generic (PLEG): container finished" podID="b253f0e2-6a5b-4fc4-b686-db5ae716f81b" containerID="ca7d12df0b0cd8b2fc7243791ba52efc31467b63015346df2f8c326f8a94e9f2" exitCode=0
Sep 17 17:38:02.892483 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:38:02.892101    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-operator-6f9c644dd9-dllkw" event={"ID":"b253f0e2-6a5b-4fc4-b686-db5ae716f81b","Type":"ContainerDied","Data":"ca7d12df0b0cd8b2fc7243791ba52efc31467b63015346df2f8c326f8a94e9f2"}
Sep 17 17:38:02.892533 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:38:02.892502    2575 scope.go:117] "RemoveContainer" containerID="ca7d12df0b0cd8b2fc7243791ba52efc31467b63015346df2f8c326f8a94e9f2"
Sep 17 17:38:03.391782 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:38:03.391738    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-djvzt_1d07e68d-9943-420f-a6a0-5f477792fabd/konnectivity-agent/0.log" limit=1024
Sep 17 17:38:03.491915 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:38:03.491883    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-djvzt_1d07e68d-9943-420f-a6a0-5f477792fabd/konnectivity-agent/0.log" limit=1024
Sep 17 17:38:03.897900 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:38:03.897864    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-operator-6f9c644dd9-dllkw" event={"ID":"b253f0e2-6a5b-4fc4-b686-db5ae716f81b","Type":"ContainerStarted","Data":"1ad49ac255cfc97fecb77a93c6e5b9d1b5d37cd1383647d959522df01e268f84"}
Sep 17 17:38:04.528891 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:38:04.528858    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-djvzt_1d07e68d-9943-420f-a6a0-5f477792fabd/konnectivity-agent/0.log" limit=1024
Sep 17 17:38:04.653064 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:38:04.653032    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-djvzt_1d07e68d-9943-420f-a6a0-5f477792fabd/konnectivity-agent/0.log" limit=1024
Sep 17 17:38:07.516153 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:38:07.516096    2575 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-image-registry/image-registry-567fd9f7c4-z54rn" podUID="9476f49c-219d-4d93-8ac2-b98df60c6b22" containerName="registry" containerID="cri-o://0a71dba1484c75866e0f8f8097c166276f59fdd5829b7a7b6c537f02c5e42334" gracePeriod=30
Sep 17 17:38:08.919555 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:38:08.919525    2575 generic.go:356] "Generic (PLEG): container finished" podID="9476f49c-219d-4d93-8ac2-b98df60c6b22" containerID="0a71dba1484c75866e0f8f8097c166276f59fdd5829b7a7b6c537f02c5e42334" exitCode=0
Sep 17 17:38:08.919892 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:38:08.919597    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-567fd9f7c4-z54rn" event={"ID":"9476f49c-219d-4d93-8ac2-b98df60c6b22","Type":"ContainerDied","Data":"0a71dba1484c75866e0f8f8097c166276f59fdd5829b7a7b6c537f02c5e42334"}
Sep 17 17:38:09.924743 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:38:09.924709    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-567fd9f7c4-z54rn" event={"ID":"9476f49c-219d-4d93-8ac2-b98df60c6b22","Type":"ContainerStarted","Data":"c6e488b6fc5444415a6aeb977f1fb1bfa08034b2311cf05a6efc002e32723fea"}
Sep 17 17:38:09.925142 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:38:09.924812    2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-image-registry/image-registry-567fd9f7c4-z54rn"
Sep 17 17:38:10.247737 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:38:10.247651    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-djvzt_1d07e68d-9943-420f-a6a0-5f477792fabd/konnectivity-agent/0.log" limit=1024
Sep 17 17:38:10.348754 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:38:10.348716    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-djvzt_1d07e68d-9943-420f-a6a0-5f477792fabd/konnectivity-agent/0.log" limit=1024
Sep 17 17:38:15.777890 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:38:15.777839    2575 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-console/console-7d486c979b-bhv9r" podUID="a1b85463-8d53-400c-9e79-4a2991b5af22" containerName="console" containerID="cri-o://ba9651b4b371ea40b76382c65b71ecbda6f697e5b32931fe0ff2b5ef6693ee82" gracePeriod=15
Sep 17 17:38:15.953499 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:38:15.953459    2575 generic.go:356] "Generic (PLEG): container finished" podID="a1b85463-8d53-400c-9e79-4a2991b5af22" containerID="ba9651b4b371ea40b76382c65b71ecbda6f697e5b32931fe0ff2b5ef6693ee82" exitCode=0
Sep 17 17:38:15.953654 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:38:15.953477    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-7d486c979b-bhv9r" event={"ID":"a1b85463-8d53-400c-9e79-4a2991b5af22","Type":"ContainerDied","Data":"ba9651b4b371ea40b76382c65b71ecbda6f697e5b32931fe0ff2b5ef6693ee82"}
Sep 17 17:38:16.092648 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:38:16.092618    2575 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-7d486c979b-bhv9r"
Sep 17 17:38:16.127973 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:38:16.127655    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/a1b85463-8d53-400c-9e79-4a2991b5af22-console-config\") pod \"a1b85463-8d53-400c-9e79-4a2991b5af22\" (UID: \"a1b85463-8d53-400c-9e79-4a2991b5af22\") "
Sep 17 17:38:16.127973 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:38:16.127766    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/a1b85463-8d53-400c-9e79-4a2991b5af22-oauth-serving-cert\") pod \"a1b85463-8d53-400c-9e79-4a2991b5af22\" (UID: \"a1b85463-8d53-400c-9e79-4a2991b5af22\") "
Sep 17 17:38:16.127973 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:38:16.127802    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/a1b85463-8d53-400c-9e79-4a2991b5af22-console-serving-cert\") pod \"a1b85463-8d53-400c-9e79-4a2991b5af22\" (UID: \"a1b85463-8d53-400c-9e79-4a2991b5af22\") "
Sep 17 17:38:16.127973 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:38:16.127844    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/a1b85463-8d53-400c-9e79-4a2991b5af22-console-oauth-config\") pod \"a1b85463-8d53-400c-9e79-4a2991b5af22\" (UID: \"a1b85463-8d53-400c-9e79-4a2991b5af22\") "
Sep 17 17:38:16.127973 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:38:16.127883    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/a1b85463-8d53-400c-9e79-4a2991b5af22-service-ca\") pod \"a1b85463-8d53-400c-9e79-4a2991b5af22\" (UID: \"a1b85463-8d53-400c-9e79-4a2991b5af22\") "
Sep 17 17:38:16.127973 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:38:16.127913    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-ns5hz\" (UniqueName: \"kubernetes.io/projected/a1b85463-8d53-400c-9e79-4a2991b5af22-kube-api-access-ns5hz\") pod \"a1b85463-8d53-400c-9e79-4a2991b5af22\" (UID: \"a1b85463-8d53-400c-9e79-4a2991b5af22\") "
Sep 17 17:38:16.128457 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:38:16.128412    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/a1b85463-8d53-400c-9e79-4a2991b5af22-oauth-serving-cert" (OuterVolumeSpecName: "oauth-serving-cert") pod "a1b85463-8d53-400c-9e79-4a2991b5af22" (UID: "a1b85463-8d53-400c-9e79-4a2991b5af22"). InnerVolumeSpecName "oauth-serving-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 17 17:38:16.128627 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:38:16.128523    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/a1b85463-8d53-400c-9e79-4a2991b5af22-console-config" (OuterVolumeSpecName: "console-config") pod "a1b85463-8d53-400c-9e79-4a2991b5af22" (UID: "a1b85463-8d53-400c-9e79-4a2991b5af22"). InnerVolumeSpecName "console-config". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 17 17:38:16.128742 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:38:16.128612    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/a1b85463-8d53-400c-9e79-4a2991b5af22-service-ca" (OuterVolumeSpecName: "service-ca") pod "a1b85463-8d53-400c-9e79-4a2991b5af22" (UID: "a1b85463-8d53-400c-9e79-4a2991b5af22"). InnerVolumeSpecName "service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 17 17:38:16.130723 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:38:16.130672    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a1b85463-8d53-400c-9e79-4a2991b5af22-kube-api-access-ns5hz" (OuterVolumeSpecName: "kube-api-access-ns5hz") pod "a1b85463-8d53-400c-9e79-4a2991b5af22" (UID: "a1b85463-8d53-400c-9e79-4a2991b5af22"). InnerVolumeSpecName "kube-api-access-ns5hz". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 17 17:38:16.130933 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:38:16.130909    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/a1b85463-8d53-400c-9e79-4a2991b5af22-console-oauth-config" (OuterVolumeSpecName: "console-oauth-config") pod "a1b85463-8d53-400c-9e79-4a2991b5af22" (UID: "a1b85463-8d53-400c-9e79-4a2991b5af22"). InnerVolumeSpecName "console-oauth-config". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 17 17:38:16.131027 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:38:16.130934    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/a1b85463-8d53-400c-9e79-4a2991b5af22-console-serving-cert" (OuterVolumeSpecName: "console-serving-cert") pod "a1b85463-8d53-400c-9e79-4a2991b5af22" (UID: "a1b85463-8d53-400c-9e79-4a2991b5af22"). InnerVolumeSpecName "console-serving-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 17 17:38:16.228970 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:38:16.228937    2575 reconciler_common.go:299] "Volume detached for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/a1b85463-8d53-400c-9e79-4a2991b5af22-service-ca\") on node \"ip-10-0-143-74.ec2.internal\" DevicePath \"\""
Sep 17 17:38:16.229231 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:38:16.229211    2575 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-ns5hz\" (UniqueName: \"kubernetes.io/projected/a1b85463-8d53-400c-9e79-4a2991b5af22-kube-api-access-ns5hz\") on node \"ip-10-0-143-74.ec2.internal\" DevicePath \"\""
Sep 17 17:38:16.229359 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:38:16.229344    2575 reconciler_common.go:299] "Volume detached for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/a1b85463-8d53-400c-9e79-4a2991b5af22-console-config\") on node \"ip-10-0-143-74.ec2.internal\" DevicePath \"\""
Sep 17 17:38:16.229487 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:38:16.229473    2575 reconciler_common.go:299] "Volume detached for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/a1b85463-8d53-400c-9e79-4a2991b5af22-oauth-serving-cert\") on node \"ip-10-0-143-74.ec2.internal\" DevicePath \"\""
Sep 17 17:38:16.229605 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:38:16.229592    2575 reconciler_common.go:299] "Volume detached for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/a1b85463-8d53-400c-9e79-4a2991b5af22-console-serving-cert\") on node \"ip-10-0-143-74.ec2.internal\" DevicePath \"\""
Sep 17 17:38:16.229711 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:38:16.229699    2575 reconciler_common.go:299] "Volume detached for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/a1b85463-8d53-400c-9e79-4a2991b5af22-console-oauth-config\") on node \"ip-10-0-143-74.ec2.internal\" DevicePath \"\""
Sep 17 17:38:16.959826 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:38:16.959795    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-7d486c979b-bhv9r" event={"ID":"a1b85463-8d53-400c-9e79-4a2991b5af22","Type":"ContainerDied","Data":"abd60eb19a0250b4f8ff7910c4d0e5b35179ace35f8c0327f2e82ffa93a3e60a"}
Sep 17 17:38:16.960218 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:38:16.959880    2575 scope.go:117] "RemoveContainer" containerID="ba9651b4b371ea40b76382c65b71ecbda6f697e5b32931fe0ff2b5ef6693ee82"
Sep 17 17:38:16.960218 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:38:16.959803    2575 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-7d486c979b-bhv9r"
Sep 17 17:38:17.000807 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:38:17.000771    2575 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-7d486c979b-bhv9r"]
Sep 17 17:38:17.004293 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:38:17.004267    2575 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-console/console-7d486c979b-bhv9r"]
Sep 17 17:38:18.966944 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:38:18.966863    2575 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="a1b85463-8d53-400c-9e79-4a2991b5af22" path="/var/lib/kubelet/pods/a1b85463-8d53-400c-9e79-4a2991b5af22/volumes"
Sep 17 17:38:19.502901 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:38:19.502864    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-djvzt_1d07e68d-9943-420f-a6a0-5f477792fabd/konnectivity-agent/0.log" limit=1024
Sep 17 17:38:19.579370 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:38:19.579335    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-djvzt_1d07e68d-9943-420f-a6a0-5f477792fabd/konnectivity-agent/0.log" limit=1024
Sep 17 17:38:26.605915 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:38:26.605879    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-djvzt_1d07e68d-9943-420f-a6a0-5f477792fabd/konnectivity-agent/0.log" limit=1024
Sep 17 17:38:26.707349 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:38:26.707314    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-djvzt_1d07e68d-9943-420f-a6a0-5f477792fabd/konnectivity-agent/0.log" limit=1024
Sep 17 17:38:30.887452 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:38:30.887416    2575 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-monitoring/alertmanager-main-0"]
Sep 17 17:38:30.887836 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:38:30.887710    2575 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="b9853529-152c-4642-ab14-885220711fb3" containerName="alertmanager" containerID="cri-o://548e1952d675442d7f2ff0e0e972e20d63d2253801c201c686551c999cf614ed" gracePeriod=120
Sep 17 17:38:30.887836 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:38:30.887798    2575 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="b9853529-152c-4642-ab14-885220711fb3" containerName="kube-rbac-proxy-web" containerID="cri-o://9c5698b6a330acd5b26f7113dcbb5215177a2cea12660a6846cb1c9aaa09808d" gracePeriod=120
Sep 17 17:38:30.887836 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:38:30.887788    2575 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="b9853529-152c-4642-ab14-885220711fb3" containerName="kube-rbac-proxy-metric" containerID="cri-o://0fac4cef38830b460a8482cf465f160bf3923fdafdcd27569be7fa4f0b3e0080" gracePeriod=120
Sep 17 17:38:30.888001 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:38:30.887818    2575 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="b9853529-152c-4642-ab14-885220711fb3" containerName="config-reloader" containerID="cri-o://743b4c1518861787e247f1f7878550e864aa794bbbcd9eb1203c7bbe2c8f7981" gracePeriod=120
Sep 17 17:38:30.888001 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:38:30.887847    2575 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="b9853529-152c-4642-ab14-885220711fb3" containerName="kube-rbac-proxy" containerID="cri-o://e82c34fe62828aebd281698514a38ef042f4f65b74d83cf09dd85aca1554a485" gracePeriod=120
Sep 17 17:38:30.888001 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:38:30.887880    2575 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="b9853529-152c-4642-ab14-885220711fb3" containerName="prom-label-proxy" containerID="cri-o://0918796955dd9f4a25172e460fe92d63b83ba858a54da5ba3ee6a5d05a6012cf" gracePeriod=120
Sep 17 17:38:30.932951 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:38:30.932922    2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-image-registry/image-registry-567fd9f7c4-z54rn"
Sep 17 17:38:31.020632 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:38:31.020606    2575 generic.go:356] "Generic (PLEG): container finished" podID="b9853529-152c-4642-ab14-885220711fb3" containerID="0918796955dd9f4a25172e460fe92d63b83ba858a54da5ba3ee6a5d05a6012cf" exitCode=0
Sep 17 17:38:31.020632 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:38:31.020627    2575 generic.go:356] "Generic (PLEG): container finished" podID="b9853529-152c-4642-ab14-885220711fb3" containerID="0fac4cef38830b460a8482cf465f160bf3923fdafdcd27569be7fa4f0b3e0080" exitCode=0
Sep 17 17:38:31.020632 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:38:31.020634    2575 generic.go:356] "Generic (PLEG): container finished" podID="b9853529-152c-4642-ab14-885220711fb3" containerID="e82c34fe62828aebd281698514a38ef042f4f65b74d83cf09dd85aca1554a485" exitCode=0
Sep 17 17:38:31.020819 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:38:31.020641    2575 generic.go:356] "Generic (PLEG): container finished" podID="b9853529-152c-4642-ab14-885220711fb3" containerID="743b4c1518861787e247f1f7878550e864aa794bbbcd9eb1203c7bbe2c8f7981" exitCode=0
Sep 17 17:38:31.020819 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:38:31.020646    2575 generic.go:356] "Generic (PLEG): container finished" podID="b9853529-152c-4642-ab14-885220711fb3" containerID="548e1952d675442d7f2ff0e0e972e20d63d2253801c201c686551c999cf614ed" exitCode=0
Sep 17 17:38:31.020819 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:38:31.020673    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"b9853529-152c-4642-ab14-885220711fb3","Type":"ContainerDied","Data":"0918796955dd9f4a25172e460fe92d63b83ba858a54da5ba3ee6a5d05a6012cf"}
Sep 17 17:38:31.020819 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:38:31.020736    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"b9853529-152c-4642-ab14-885220711fb3","Type":"ContainerDied","Data":"0fac4cef38830b460a8482cf465f160bf3923fdafdcd27569be7fa4f0b3e0080"}
Sep 17 17:38:31.020819 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:38:31.020764    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"b9853529-152c-4642-ab14-885220711fb3","Type":"ContainerDied","Data":"e82c34fe62828aebd281698514a38ef042f4f65b74d83cf09dd85aca1554a485"}
Sep 17 17:38:31.020819 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:38:31.020796    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"b9853529-152c-4642-ab14-885220711fb3","Type":"ContainerDied","Data":"743b4c1518861787e247f1f7878550e864aa794bbbcd9eb1203c7bbe2c8f7981"}
Sep 17 17:38:31.021012 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:38:31.020842    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"b9853529-152c-4642-ab14-885220711fb3","Type":"ContainerDied","Data":"548e1952d675442d7f2ff0e0e972e20d63d2253801c201c686551c999cf614ed"}
Sep 17 17:38:31.099316 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:38:31.099278    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-djvzt_1d07e68d-9943-420f-a6a0-5f477792fabd/konnectivity-agent/0.log" limit=1024
Sep 17 17:38:31.170920 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:38:31.170841    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-djvzt_1d07e68d-9943-420f-a6a0-5f477792fabd/konnectivity-agent/0.log" limit=1024
Sep 17 17:38:31.323272 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:38:31.323235    2575 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-6b674b467b-7nf59"]
Sep 17 17:38:31.323645 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:38:31.323627    2575 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a1b85463-8d53-400c-9e79-4a2991b5af22" containerName="console"
Sep 17 17:38:31.323743 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:38:31.323647    2575 state_mem.go:107] "Deleted CPUSet assignment" podUID="a1b85463-8d53-400c-9e79-4a2991b5af22" containerName="console"
Sep 17 17:38:31.323743 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:38:31.323726    2575 memory_manager.go:368] "RemoveStaleState removing state" podUID="a1b85463-8d53-400c-9e79-4a2991b5af22" containerName="console"
Sep 17 17:38:31.325587 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:38:31.325567    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-6b674b467b-7nf59"
Sep 17 17:38:31.328213 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:38:31.328191    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-serving-cert\""
Sep 17 17:38:31.328306 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:38:31.328235    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-dockercfg-zn79t\""
Sep 17 17:38:31.328306 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:38:31.328295    2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"console-config\""
Sep 17 17:38:31.328411 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:38:31.328334    2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"oauth-serving-cert\""
Sep 17 17:38:31.328504 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:38:31.328490    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-oauth-config\""
Sep 17 17:38:31.328979 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:38:31.328965    2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"service-ca\""
Sep 17 17:38:31.336140 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:38:31.334835    2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"trusted-ca-bundle\""
Sep 17 17:38:31.338750 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:38:31.338730    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-6b674b467b-7nf59"]
Sep 17 17:38:31.391634 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:38:31.391587    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-djvzt_1d07e68d-9943-420f-a6a0-5f477792fabd/konnectivity-agent/0.log" limit=1024
Sep 17 17:38:31.441567 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:38:31.441509    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/879b8249-fc8d-41d5-9708-8c396841dd12-console-serving-cert\") pod \"console-6b674b467b-7nf59\" (UID: \"879b8249-fc8d-41d5-9708-8c396841dd12\") " pod="openshift-console/console-6b674b467b-7nf59"
Sep 17 17:38:31.441567 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:38:31.441540    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/879b8249-fc8d-41d5-9708-8c396841dd12-oauth-serving-cert\") pod \"console-6b674b467b-7nf59\" (UID: \"879b8249-fc8d-41d5-9708-8c396841dd12\") " pod="openshift-console/console-6b674b467b-7nf59"
Sep 17 17:38:31.441691 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:38:31.441567    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/879b8249-fc8d-41d5-9708-8c396841dd12-service-ca\") pod \"console-6b674b467b-7nf59\" (UID: \"879b8249-fc8d-41d5-9708-8c396841dd12\") " pod="openshift-console/console-6b674b467b-7nf59"
Sep 17 17:38:31.441691 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:38:31.441625    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/879b8249-fc8d-41d5-9708-8c396841dd12-console-config\") pod \"console-6b674b467b-7nf59\" (UID: \"879b8249-fc8d-41d5-9708-8c396841dd12\") " pod="openshift-console/console-6b674b467b-7nf59"
Sep 17 17:38:31.441691 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:38:31.441651    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/879b8249-fc8d-41d5-9708-8c396841dd12-console-oauth-config\") pod \"console-6b674b467b-7nf59\" (UID: \"879b8249-fc8d-41d5-9708-8c396841dd12\") " pod="openshift-console/console-6b674b467b-7nf59"
Sep 17 17:38:31.441691 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:38:31.441674    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-x4np7\" (UniqueName: \"kubernetes.io/projected/879b8249-fc8d-41d5-9708-8c396841dd12-kube-api-access-x4np7\") pod \"console-6b674b467b-7nf59\" (UID: \"879b8249-fc8d-41d5-9708-8c396841dd12\") " pod="openshift-console/console-6b674b467b-7nf59"
Sep 17 17:38:31.441813 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:38:31.441713    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/879b8249-fc8d-41d5-9708-8c396841dd12-trusted-ca-bundle\") pod \"console-6b674b467b-7nf59\" (UID: \"879b8249-fc8d-41d5-9708-8c396841dd12\") " pod="openshift-console/console-6b674b467b-7nf59"
Sep 17 17:38:31.493687 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:38:31.493658    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-djvzt_1d07e68d-9943-420f-a6a0-5f477792fabd/konnectivity-agent/0.log" limit=1024
Sep 17 17:38:31.542880 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:38:31.542848    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/879b8249-fc8d-41d5-9708-8c396841dd12-console-serving-cert\") pod \"console-6b674b467b-7nf59\" (UID: \"879b8249-fc8d-41d5-9708-8c396841dd12\") " pod="openshift-console/console-6b674b467b-7nf59"
Sep 17 17:38:31.542880 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:38:31.542876    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/879b8249-fc8d-41d5-9708-8c396841dd12-oauth-serving-cert\") pod \"console-6b674b467b-7nf59\" (UID: \"879b8249-fc8d-41d5-9708-8c396841dd12\") " pod="openshift-console/console-6b674b467b-7nf59"
Sep 17 17:38:31.543056 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:38:31.542898    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/879b8249-fc8d-41d5-9708-8c396841dd12-service-ca\") pod \"console-6b674b467b-7nf59\" (UID: \"879b8249-fc8d-41d5-9708-8c396841dd12\") " pod="openshift-console/console-6b674b467b-7nf59"
Sep 17 17:38:31.543056 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:38:31.542925    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/879b8249-fc8d-41d5-9708-8c396841dd12-console-config\") pod \"console-6b674b467b-7nf59\" (UID: \"879b8249-fc8d-41d5-9708-8c396841dd12\") " pod="openshift-console/console-6b674b467b-7nf59"
Sep 17 17:38:31.543056 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:38:31.542988    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/879b8249-fc8d-41d5-9708-8c396841dd12-console-oauth-config\") pod \"console-6b674b467b-7nf59\" (UID: \"879b8249-fc8d-41d5-9708-8c396841dd12\") " pod="openshift-console/console-6b674b467b-7nf59"
Sep 17 17:38:31.543056 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:38:31.543044    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-x4np7\" (UniqueName: \"kubernetes.io/projected/879b8249-fc8d-41d5-9708-8c396841dd12-kube-api-access-x4np7\") pod \"console-6b674b467b-7nf59\" (UID: \"879b8249-fc8d-41d5-9708-8c396841dd12\") " pod="openshift-console/console-6b674b467b-7nf59"
Sep 17 17:38:31.543183 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:38:31.543109    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/879b8249-fc8d-41d5-9708-8c396841dd12-trusted-ca-bundle\") pod \"console-6b674b467b-7nf59\" (UID: \"879b8249-fc8d-41d5-9708-8c396841dd12\") " pod="openshift-console/console-6b674b467b-7nf59"
Sep 17 17:38:31.543789 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:38:31.543764    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/879b8249-fc8d-41d5-9708-8c396841dd12-oauth-serving-cert\") pod \"console-6b674b467b-7nf59\" (UID: \"879b8249-fc8d-41d5-9708-8c396841dd12\") " pod="openshift-console/console-6b674b467b-7nf59"
Sep 17 17:38:31.543889 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:38:31.543767    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/879b8249-fc8d-41d5-9708-8c396841dd12-service-ca\") pod \"console-6b674b467b-7nf59\" (UID: \"879b8249-fc8d-41d5-9708-8c396841dd12\") " pod="openshift-console/console-6b674b467b-7nf59"
Sep 17 17:38:31.543889 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:38:31.543859    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/879b8249-fc8d-41d5-9708-8c396841dd12-console-config\") pod \"console-6b674b467b-7nf59\" (UID: \"879b8249-fc8d-41d5-9708-8c396841dd12\") " pod="openshift-console/console-6b674b467b-7nf59"
Sep 17 17:38:31.543956 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:38:31.543924    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/879b8249-fc8d-41d5-9708-8c396841dd12-trusted-ca-bundle\") pod \"console-6b674b467b-7nf59\" (UID: \"879b8249-fc8d-41d5-9708-8c396841dd12\") " pod="openshift-console/console-6b674b467b-7nf59"
Sep 17 17:38:31.545317 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:38:31.545299    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/879b8249-fc8d-41d5-9708-8c396841dd12-console-serving-cert\") pod \"console-6b674b467b-7nf59\" (UID: \"879b8249-fc8d-41d5-9708-8c396841dd12\") " pod="openshift-console/console-6b674b467b-7nf59"
Sep 17 17:38:31.545406 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:38:31.545340    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/879b8249-fc8d-41d5-9708-8c396841dd12-console-oauth-config\") pod \"console-6b674b467b-7nf59\" (UID: \"879b8249-fc8d-41d5-9708-8c396841dd12\") " pod="openshift-console/console-6b674b467b-7nf59"
Sep 17 17:38:31.553559 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:38:31.553533    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-x4np7\" (UniqueName: \"kubernetes.io/projected/879b8249-fc8d-41d5-9708-8c396841dd12-kube-api-access-x4np7\") pod \"console-6b674b467b-7nf59\" (UID: \"879b8249-fc8d-41d5-9708-8c396841dd12\") " pod="openshift-console/console-6b674b467b-7nf59"
Sep 17 17:38:31.639068 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:38:31.639040    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-6b674b467b-7nf59"
Sep 17 17:38:31.910141 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:38:31.909984    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-6b674b467b-7nf59"]
Sep 17 17:38:31.912317 ip-10-0-143-74 kubenswrapper[2575]: W0917 17:38:31.912293    2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod879b8249_fc8d_41d5_9708_8c396841dd12.slice/crio-a364018b71975cabd0d4dbde33dd2a8768d0fe389d64f040e89bc6bd8bd41f16 WatchSource:0}: Error finding container a364018b71975cabd0d4dbde33dd2a8768d0fe389d64f040e89bc6bd8bd41f16: Status 404 returned error can't find the container with id a364018b71975cabd0d4dbde33dd2a8768d0fe389d64f040e89bc6bd8bd41f16
Sep 17 17:38:32.030197 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:38:32.030162    2575 generic.go:356] "Generic (PLEG): container finished" podID="b9853529-152c-4642-ab14-885220711fb3" containerID="9c5698b6a330acd5b26f7113dcbb5215177a2cea12660a6846cb1c9aaa09808d" exitCode=0
Sep 17 17:38:32.030368 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:38:32.030228    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"b9853529-152c-4642-ab14-885220711fb3","Type":"ContainerDied","Data":"9c5698b6a330acd5b26f7113dcbb5215177a2cea12660a6846cb1c9aaa09808d"}
Sep 17 17:38:32.032811 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:38:32.032028    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-6b674b467b-7nf59" event={"ID":"879b8249-fc8d-41d5-9708-8c396841dd12","Type":"ContainerStarted","Data":"bf3e822e6949b272c505b790371bb1405217bb6b1a7dea798e4d96a518147010"}
Sep 17 17:38:32.032811 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:38:32.032076    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-6b674b467b-7nf59" event={"ID":"879b8249-fc8d-41d5-9708-8c396841dd12","Type":"ContainerStarted","Data":"a364018b71975cabd0d4dbde33dd2a8768d0fe389d64f040e89bc6bd8bd41f16"}
Sep 17 17:38:32.053012 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:38:32.052688    2575 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-6b674b467b-7nf59" podStartSLOduration=1.052668306 podStartE2EDuration="1.052668306s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-17 17:38:31 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-17 17:38:32.052213139 +0000 UTC m=+231.682543112" watchObservedRunningTime="2026-09-17 17:38:32.052668306 +0000 UTC m=+231.682998280"
Sep 17 17:38:32.125360 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:38:32.125338    2575 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0"
Sep 17 17:38:32.251000 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:38:32.250927    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/b9853529-152c-4642-ab14-885220711fb3-secret-alertmanager-kube-rbac-proxy\") pod \"b9853529-152c-4642-ab14-885220711fb3\" (UID: \"b9853529-152c-4642-ab14-885220711fb3\") "
Sep 17 17:38:32.251000 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:38:32.250978    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/b9853529-152c-4642-ab14-885220711fb3-config-out\") pod \"b9853529-152c-4642-ab14-885220711fb3\" (UID: \"b9853529-152c-4642-ab14-885220711fb3\") "
Sep 17 17:38:32.251208 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:38:32.251023    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/b9853529-152c-4642-ab14-885220711fb3-secret-alertmanager-kube-rbac-proxy-web\") pod \"b9853529-152c-4642-ab14-885220711fb3\" (UID: \"b9853529-152c-4642-ab14-885220711fb3\") "
Sep 17 17:38:32.251208 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:38:32.251050    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/b9853529-152c-4642-ab14-885220711fb3-metrics-client-ca\") pod \"b9853529-152c-4642-ab14-885220711fb3\" (UID: \"b9853529-152c-4642-ab14-885220711fb3\") "
Sep 17 17:38:32.251208 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:38:32.251080    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/b9853529-152c-4642-ab14-885220711fb3-alertmanager-main-db\") pod \"b9853529-152c-4642-ab14-885220711fb3\" (UID: \"b9853529-152c-4642-ab14-885220711fb3\") "
Sep 17 17:38:32.251368 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:38:32.251209    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/b9853529-152c-4642-ab14-885220711fb3-config-volume\") pod \"b9853529-152c-4642-ab14-885220711fb3\" (UID: \"b9853529-152c-4642-ab14-885220711fb3\") "
Sep 17 17:38:32.251368 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:38:32.251260    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/b9853529-152c-4642-ab14-885220711fb3-secret-alertmanager-main-tls\") pod \"b9853529-152c-4642-ab14-885220711fb3\" (UID: \"b9853529-152c-4642-ab14-885220711fb3\") "
Sep 17 17:38:32.251368 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:38:32.251289    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/b9853529-152c-4642-ab14-885220711fb3-alertmanager-trusted-ca-bundle\") pod \"b9853529-152c-4642-ab14-885220711fb3\" (UID: \"b9853529-152c-4642-ab14-885220711fb3\") "
Sep 17 17:38:32.251368 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:38:32.251322    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/b9853529-152c-4642-ab14-885220711fb3-web-config\") pod \"b9853529-152c-4642-ab14-885220711fb3\" (UID: \"b9853529-152c-4642-ab14-885220711fb3\") "
Sep 17 17:38:32.251368 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:38:32.251346    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/b9853529-152c-4642-ab14-885220711fb3-tls-assets\") pod \"b9853529-152c-4642-ab14-885220711fb3\" (UID: \"b9853529-152c-4642-ab14-885220711fb3\") "
Sep 17 17:38:32.251626 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:38:32.251383    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/b9853529-152c-4642-ab14-885220711fb3-cluster-tls-config\") pod \"b9853529-152c-4642-ab14-885220711fb3\" (UID: \"b9853529-152c-4642-ab14-885220711fb3\") "
Sep 17 17:38:32.251626 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:38:32.251407    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-mnzn6\" (UniqueName: \"kubernetes.io/projected/b9853529-152c-4642-ab14-885220711fb3-kube-api-access-mnzn6\") pod \"b9853529-152c-4642-ab14-885220711fb3\" (UID: \"b9853529-152c-4642-ab14-885220711fb3\") "
Sep 17 17:38:32.251626 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:38:32.251470    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/b9853529-152c-4642-ab14-885220711fb3-secret-alertmanager-kube-rbac-proxy-metric\") pod \"b9853529-152c-4642-ab14-885220711fb3\" (UID: \"b9853529-152c-4642-ab14-885220711fb3\") "
Sep 17 17:38:32.254315 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:38:32.251403    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/b9853529-152c-4642-ab14-885220711fb3-metrics-client-ca" (OuterVolumeSpecName: "metrics-client-ca") pod "b9853529-152c-4642-ab14-885220711fb3" (UID: "b9853529-152c-4642-ab14-885220711fb3"). InnerVolumeSpecName "metrics-client-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 17 17:38:32.254315 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:38:32.251415    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b9853529-152c-4642-ab14-885220711fb3-alertmanager-main-db" (OuterVolumeSpecName: "alertmanager-main-db") pod "b9853529-152c-4642-ab14-885220711fb3" (UID: "b9853529-152c-4642-ab14-885220711fb3"). InnerVolumeSpecName "alertmanager-main-db". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 17 17:38:32.254534 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:38:32.253974    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/b9853529-152c-4642-ab14-885220711fb3-config-volume" (OuterVolumeSpecName: "config-volume") pod "b9853529-152c-4642-ab14-885220711fb3" (UID: "b9853529-152c-4642-ab14-885220711fb3"). InnerVolumeSpecName "config-volume". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 17 17:38:32.254534 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:38:32.254327    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/b9853529-152c-4642-ab14-885220711fb3-secret-alertmanager-kube-rbac-proxy" (OuterVolumeSpecName: "secret-alertmanager-kube-rbac-proxy") pod "b9853529-152c-4642-ab14-885220711fb3" (UID: "b9853529-152c-4642-ab14-885220711fb3"). InnerVolumeSpecName "secret-alertmanager-kube-rbac-proxy". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 17 17:38:32.254534 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:38:32.254281    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/b9853529-152c-4642-ab14-885220711fb3-secret-alertmanager-kube-rbac-proxy-metric" (OuterVolumeSpecName: "secret-alertmanager-kube-rbac-proxy-metric") pod "b9853529-152c-4642-ab14-885220711fb3" (UID: "b9853529-152c-4642-ab14-885220711fb3"). InnerVolumeSpecName "secret-alertmanager-kube-rbac-proxy-metric". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 17 17:38:32.254534 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:38:32.254353    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/b9853529-152c-4642-ab14-885220711fb3-secret-alertmanager-kube-rbac-proxy-web" (OuterVolumeSpecName: "secret-alertmanager-kube-rbac-proxy-web") pod "b9853529-152c-4642-ab14-885220711fb3" (UID: "b9853529-152c-4642-ab14-885220711fb3"). InnerVolumeSpecName "secret-alertmanager-kube-rbac-proxy-web". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 17 17:38:32.254534 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:38:32.254351    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/b9853529-152c-4642-ab14-885220711fb3-kube-api-access-mnzn6" (OuterVolumeSpecName: "kube-api-access-mnzn6") pod "b9853529-152c-4642-ab14-885220711fb3" (UID: "b9853529-152c-4642-ab14-885220711fb3"). InnerVolumeSpecName "kube-api-access-mnzn6". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 17 17:38:32.254779 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:38:32.254702    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b9853529-152c-4642-ab14-885220711fb3-config-out" (OuterVolumeSpecName: "config-out") pod "b9853529-152c-4642-ab14-885220711fb3" (UID: "b9853529-152c-4642-ab14-885220711fb3"). InnerVolumeSpecName "config-out". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 17 17:38:32.254779 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:38:32.254736    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/b9853529-152c-4642-ab14-885220711fb3-alertmanager-trusted-ca-bundle" (OuterVolumeSpecName: "alertmanager-trusted-ca-bundle") pod "b9853529-152c-4642-ab14-885220711fb3" (UID: "b9853529-152c-4642-ab14-885220711fb3"). InnerVolumeSpecName "alertmanager-trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 17 17:38:32.255709 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:38:32.255686    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/b9853529-152c-4642-ab14-885220711fb3-secret-alertmanager-main-tls" (OuterVolumeSpecName: "secret-alertmanager-main-tls") pod "b9853529-152c-4642-ab14-885220711fb3" (UID: "b9853529-152c-4642-ab14-885220711fb3"). InnerVolumeSpecName "secret-alertmanager-main-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 17 17:38:32.256508 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:38:32.256488    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/b9853529-152c-4642-ab14-885220711fb3-tls-assets" (OuterVolumeSpecName: "tls-assets") pod "b9853529-152c-4642-ab14-885220711fb3" (UID: "b9853529-152c-4642-ab14-885220711fb3"). InnerVolumeSpecName "tls-assets". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 17 17:38:32.258359 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:38:32.258337    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/b9853529-152c-4642-ab14-885220711fb3-cluster-tls-config" (OuterVolumeSpecName: "cluster-tls-config") pod "b9853529-152c-4642-ab14-885220711fb3" (UID: "b9853529-152c-4642-ab14-885220711fb3"). InnerVolumeSpecName "cluster-tls-config". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 17 17:38:32.263600 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:38:32.263583    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/b9853529-152c-4642-ab14-885220711fb3-web-config" (OuterVolumeSpecName: "web-config") pod "b9853529-152c-4642-ab14-885220711fb3" (UID: "b9853529-152c-4642-ab14-885220711fb3"). InnerVolumeSpecName "web-config". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 17 17:38:32.353040 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:38:32.353007    2575 reconciler_common.go:299] "Volume detached for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/b9853529-152c-4642-ab14-885220711fb3-config-out\") on node \"ip-10-0-143-74.ec2.internal\" DevicePath \"\""
Sep 17 17:38:32.353040 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:38:32.353033    2575 reconciler_common.go:299] "Volume detached for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/b9853529-152c-4642-ab14-885220711fb3-secret-alertmanager-kube-rbac-proxy-web\") on node \"ip-10-0-143-74.ec2.internal\" DevicePath \"\""
Sep 17 17:38:32.353040 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:38:32.353043    2575 reconciler_common.go:299] "Volume detached for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/b9853529-152c-4642-ab14-885220711fb3-metrics-client-ca\") on node \"ip-10-0-143-74.ec2.internal\" DevicePath \"\""
Sep 17 17:38:32.353232 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:38:32.353051    2575 reconciler_common.go:299] "Volume detached for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/b9853529-152c-4642-ab14-885220711fb3-alertmanager-main-db\") on node \"ip-10-0-143-74.ec2.internal\" DevicePath \"\""
Sep 17 17:38:32.353232 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:38:32.353060    2575 reconciler_common.go:299] "Volume detached for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/b9853529-152c-4642-ab14-885220711fb3-config-volume\") on node \"ip-10-0-143-74.ec2.internal\" DevicePath \"\""
Sep 17 17:38:32.353232 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:38:32.353068    2575 reconciler_common.go:299] "Volume detached for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/b9853529-152c-4642-ab14-885220711fb3-secret-alertmanager-main-tls\") on node \"ip-10-0-143-74.ec2.internal\" DevicePath \"\""
Sep 17 17:38:32.353232 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:38:32.353076    2575 reconciler_common.go:299] "Volume detached for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/b9853529-152c-4642-ab14-885220711fb3-alertmanager-trusted-ca-bundle\") on node \"ip-10-0-143-74.ec2.internal\" DevicePath \"\""
Sep 17 17:38:32.353232 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:38:32.353084    2575 reconciler_common.go:299] "Volume detached for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/b9853529-152c-4642-ab14-885220711fb3-web-config\") on node \"ip-10-0-143-74.ec2.internal\" DevicePath \"\""
Sep 17 17:38:32.353232 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:38:32.353094    2575 reconciler_common.go:299] "Volume detached for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/b9853529-152c-4642-ab14-885220711fb3-tls-assets\") on node \"ip-10-0-143-74.ec2.internal\" DevicePath \"\""
Sep 17 17:38:32.353232 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:38:32.353101    2575 reconciler_common.go:299] "Volume detached for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/b9853529-152c-4642-ab14-885220711fb3-cluster-tls-config\") on node \"ip-10-0-143-74.ec2.internal\" DevicePath \"\""
Sep 17 17:38:32.353232 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:38:32.353109    2575 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-mnzn6\" (UniqueName: \"kubernetes.io/projected/b9853529-152c-4642-ab14-885220711fb3-kube-api-access-mnzn6\") on node \"ip-10-0-143-74.ec2.internal\" DevicePath \"\""
Sep 17 17:38:32.353232 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:38:32.353117    2575 reconciler_common.go:299] "Volume detached for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/b9853529-152c-4642-ab14-885220711fb3-secret-alertmanager-kube-rbac-proxy-metric\") on node \"ip-10-0-143-74.ec2.internal\" DevicePath \"\""
Sep 17 17:38:32.353232 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:38:32.353125    2575 reconciler_common.go:299] "Volume detached for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/b9853529-152c-4642-ab14-885220711fb3-secret-alertmanager-kube-rbac-proxy\") on node \"ip-10-0-143-74.ec2.internal\" DevicePath \"\""
Sep 17 17:38:33.039876 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:38:33.039839    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"b9853529-152c-4642-ab14-885220711fb3","Type":"ContainerDied","Data":"37eb78b233cffe72dd7f40f471cc2d9f56d24c842ee78aa80e535a32d2c08672"}
Sep 17 17:38:33.040280 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:38:33.039901    2575 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0"
Sep 17 17:38:33.040280 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:38:33.039913    2575 scope.go:117] "RemoveContainer" containerID="0918796955dd9f4a25172e460fe92d63b83ba858a54da5ba3ee6a5d05a6012cf"
Sep 17 17:38:33.048522 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:38:33.048425    2575 scope.go:117] "RemoveContainer" containerID="0fac4cef38830b460a8482cf465f160bf3923fdafdcd27569be7fa4f0b3e0080"
Sep 17 17:38:33.056104 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:38:33.056087    2575 scope.go:117] "RemoveContainer" containerID="e82c34fe62828aebd281698514a38ef042f4f65b74d83cf09dd85aca1554a485"
Sep 17 17:38:33.062491 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:38:33.062469    2575 scope.go:117] "RemoveContainer" containerID="9c5698b6a330acd5b26f7113dcbb5215177a2cea12660a6846cb1c9aaa09808d"
Sep 17 17:38:33.065091 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:38:33.065071    2575 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-monitoring/alertmanager-main-0"]
Sep 17 17:38:33.070321 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:38:33.070303    2575 scope.go:117] "RemoveContainer" containerID="743b4c1518861787e247f1f7878550e864aa794bbbcd9eb1203c7bbe2c8f7981"
Sep 17 17:38:33.070675 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:38:33.070660    2575 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-monitoring/alertmanager-main-0"]
Sep 17 17:38:33.076995 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:38:33.076979    2575 scope.go:117] "RemoveContainer" containerID="548e1952d675442d7f2ff0e0e972e20d63d2253801c201c686551c999cf614ed"
Sep 17 17:38:33.083453 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:38:33.083426    2575 scope.go:117] "RemoveContainer" containerID="74d8a55f51add720c843cb3cd5a3b44c90f3d492827af6ffb3622c08f229932f"
Sep 17 17:38:33.113927 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:38:33.111682    2575 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/alertmanager-main-0"]
Sep 17 17:38:33.113927 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:38:33.112251    2575 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b9853529-152c-4642-ab14-885220711fb3" containerName="kube-rbac-proxy"
Sep 17 17:38:33.113927 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:38:33.112273    2575 state_mem.go:107] "Deleted CPUSet assignment" podUID="b9853529-152c-4642-ab14-885220711fb3" containerName="kube-rbac-proxy"
Sep 17 17:38:33.113927 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:38:33.112303    2575 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b9853529-152c-4642-ab14-885220711fb3" containerName="kube-rbac-proxy-metric"
Sep 17 17:38:33.113927 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:38:33.112312    2575 state_mem.go:107] "Deleted CPUSet assignment" podUID="b9853529-152c-4642-ab14-885220711fb3" containerName="kube-rbac-proxy-metric"
Sep 17 17:38:33.113927 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:38:33.112342    2575 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b9853529-152c-4642-ab14-885220711fb3" containerName="kube-rbac-proxy-web"
Sep 17 17:38:33.113927 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:38:33.112351    2575 state_mem.go:107] "Deleted CPUSet assignment" podUID="b9853529-152c-4642-ab14-885220711fb3" containerName="kube-rbac-proxy-web"
Sep 17 17:38:33.113927 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:38:33.112361    2575 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b9853529-152c-4642-ab14-885220711fb3" containerName="init-config-reloader"
Sep 17 17:38:33.113927 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:38:33.112370    2575 state_mem.go:107] "Deleted CPUSet assignment" podUID="b9853529-152c-4642-ab14-885220711fb3" containerName="init-config-reloader"
Sep 17 17:38:33.113927 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:38:33.112391    2575 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b9853529-152c-4642-ab14-885220711fb3" containerName="alertmanager"
Sep 17 17:38:33.113927 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:38:33.112399    2575 state_mem.go:107] "Deleted CPUSet assignment" podUID="b9853529-152c-4642-ab14-885220711fb3" containerName="alertmanager"
Sep 17 17:38:33.113927 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:38:33.112407    2575 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b9853529-152c-4642-ab14-885220711fb3" containerName="config-reloader"
Sep 17 17:38:33.113927 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:38:33.112415    2575 state_mem.go:107] "Deleted CPUSet assignment" podUID="b9853529-152c-4642-ab14-885220711fb3" containerName="config-reloader"
Sep 17 17:38:33.113927 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:38:33.112423    2575 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b9853529-152c-4642-ab14-885220711fb3" containerName="prom-label-proxy"
Sep 17 17:38:33.113927 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:38:33.112431    2575 state_mem.go:107] "Deleted CPUSet assignment" podUID="b9853529-152c-4642-ab14-885220711fb3" containerName="prom-label-proxy"
Sep 17 17:38:33.113927 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:38:33.112562    2575 memory_manager.go:368] "RemoveStaleState removing state" podUID="b9853529-152c-4642-ab14-885220711fb3" containerName="kube-rbac-proxy-web"
Sep 17 17:38:33.113927 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:38:33.112581    2575 memory_manager.go:368] "RemoveStaleState removing state" podUID="b9853529-152c-4642-ab14-885220711fb3" containerName="prom-label-proxy"
Sep 17 17:38:33.113927 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:38:33.112596    2575 memory_manager.go:368] "RemoveStaleState removing state" podUID="b9853529-152c-4642-ab14-885220711fb3" containerName="config-reloader"
Sep 17 17:38:33.113927 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:38:33.112604    2575 memory_manager.go:368] "RemoveStaleState removing state" podUID="b9853529-152c-4642-ab14-885220711fb3" containerName="kube-rbac-proxy"
Sep 17 17:38:33.113927 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:38:33.112612    2575 memory_manager.go:368] "RemoveStaleState removing state" podUID="b9853529-152c-4642-ab14-885220711fb3" containerName="kube-rbac-proxy-metric"
Sep 17 17:38:33.113927 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:38:33.112623    2575 memory_manager.go:368] "RemoveStaleState removing state" podUID="b9853529-152c-4642-ab14-885220711fb3" containerName="alertmanager"
Sep 17 17:38:33.117405 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:38:33.117384    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0"
Sep 17 17:38:33.120296 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:38:33.120269    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-tls\""
Sep 17 17:38:33.120296 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:38:33.120269    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy\""
Sep 17 17:38:33.120296 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:38:33.120309    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-cluster-tls-config\""
Sep 17 17:38:33.120600 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:38:33.120375    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-generated\""
Sep 17 17:38:33.120600 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:38:33.120269    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-dockercfg-pdb7v\""
Sep 17 17:38:33.120600 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:38:33.120387    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy-web\""
Sep 17 17:38:33.120862 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:38:33.120704    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-web-config\""
Sep 17 17:38:33.120862 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:38:33.120800    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy-metric\""
Sep 17 17:38:33.120862 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:38:33.120703    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-tls-assets-0\""
Sep 17 17:38:33.126289 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:38:33.126259    2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"alertmanager-trusted-ca-bundle\""
Sep 17 17:38:33.130775 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:38:33.130752    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/alertmanager-main-0"]
Sep 17 17:38:33.158686 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:38:33.158617    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/26063eb2-0a4f-460f-a6ba-6d19ab96af70-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"26063eb2-0a4f-460f-a6ba-6d19ab96af70\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 17 17:38:33.158686 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:38:33.158668    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/26063eb2-0a4f-460f-a6ba-6d19ab96af70-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"26063eb2-0a4f-460f-a6ba-6d19ab96af70\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 17 17:38:33.158842 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:38:33.158693    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/26063eb2-0a4f-460f-a6ba-6d19ab96af70-web-config\") pod \"alertmanager-main-0\" (UID: \"26063eb2-0a4f-460f-a6ba-6d19ab96af70\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 17 17:38:33.158842 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:38:33.158791    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/26063eb2-0a4f-460f-a6ba-6d19ab96af70-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"26063eb2-0a4f-460f-a6ba-6d19ab96af70\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 17 17:38:33.158842 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:38:33.158837    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/26063eb2-0a4f-460f-a6ba-6d19ab96af70-config-out\") pod \"alertmanager-main-0\" (UID: \"26063eb2-0a4f-460f-a6ba-6d19ab96af70\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 17 17:38:33.159007 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:38:33.158873    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8srtf\" (UniqueName: \"kubernetes.io/projected/26063eb2-0a4f-460f-a6ba-6d19ab96af70-kube-api-access-8srtf\") pod \"alertmanager-main-0\" (UID: \"26063eb2-0a4f-460f-a6ba-6d19ab96af70\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 17 17:38:33.159007 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:38:33.158929    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/26063eb2-0a4f-460f-a6ba-6d19ab96af70-tls-assets\") pod \"alertmanager-main-0\" (UID: \"26063eb2-0a4f-460f-a6ba-6d19ab96af70\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 17 17:38:33.159007 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:38:33.158995    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/26063eb2-0a4f-460f-a6ba-6d19ab96af70-config-volume\") pod \"alertmanager-main-0\" (UID: \"26063eb2-0a4f-460f-a6ba-6d19ab96af70\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 17 17:38:33.159147 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:38:33.159075    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/26063eb2-0a4f-460f-a6ba-6d19ab96af70-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"26063eb2-0a4f-460f-a6ba-6d19ab96af70\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 17 17:38:33.159147 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:38:33.159115    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/26063eb2-0a4f-460f-a6ba-6d19ab96af70-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"26063eb2-0a4f-460f-a6ba-6d19ab96af70\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 17 17:38:33.159251 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:38:33.159204    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/26063eb2-0a4f-460f-a6ba-6d19ab96af70-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"26063eb2-0a4f-460f-a6ba-6d19ab96af70\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 17 17:38:33.159304 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:38:33.159254    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/26063eb2-0a4f-460f-a6ba-6d19ab96af70-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"26063eb2-0a4f-460f-a6ba-6d19ab96af70\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 17 17:38:33.159304 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:38:33.159280    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/26063eb2-0a4f-460f-a6ba-6d19ab96af70-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"26063eb2-0a4f-460f-a6ba-6d19ab96af70\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 17 17:38:33.260511 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:38:33.260475    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/26063eb2-0a4f-460f-a6ba-6d19ab96af70-web-config\") pod \"alertmanager-main-0\" (UID: \"26063eb2-0a4f-460f-a6ba-6d19ab96af70\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 17 17:38:33.260511 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:38:33.260516    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/26063eb2-0a4f-460f-a6ba-6d19ab96af70-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"26063eb2-0a4f-460f-a6ba-6d19ab96af70\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 17 17:38:33.260757 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:38:33.260538    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/26063eb2-0a4f-460f-a6ba-6d19ab96af70-config-out\") pod \"alertmanager-main-0\" (UID: \"26063eb2-0a4f-460f-a6ba-6d19ab96af70\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 17 17:38:33.260757 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:38:33.260561    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8srtf\" (UniqueName: \"kubernetes.io/projected/26063eb2-0a4f-460f-a6ba-6d19ab96af70-kube-api-access-8srtf\") pod \"alertmanager-main-0\" (UID: \"26063eb2-0a4f-460f-a6ba-6d19ab96af70\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 17 17:38:33.260757 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:38:33.260729    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/26063eb2-0a4f-460f-a6ba-6d19ab96af70-tls-assets\") pod \"alertmanager-main-0\" (UID: \"26063eb2-0a4f-460f-a6ba-6d19ab96af70\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 17 17:38:33.260757 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:38:33.260752    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/26063eb2-0a4f-460f-a6ba-6d19ab96af70-config-volume\") pod \"alertmanager-main-0\" (UID: \"26063eb2-0a4f-460f-a6ba-6d19ab96af70\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 17 17:38:33.260976 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:38:33.260778    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/26063eb2-0a4f-460f-a6ba-6d19ab96af70-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"26063eb2-0a4f-460f-a6ba-6d19ab96af70\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 17 17:38:33.260976 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:38:33.260816    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/26063eb2-0a4f-460f-a6ba-6d19ab96af70-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"26063eb2-0a4f-460f-a6ba-6d19ab96af70\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 17 17:38:33.260976 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:38:33.260870    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/26063eb2-0a4f-460f-a6ba-6d19ab96af70-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"26063eb2-0a4f-460f-a6ba-6d19ab96af70\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 17 17:38:33.260976 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:38:33.260905    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/26063eb2-0a4f-460f-a6ba-6d19ab96af70-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"26063eb2-0a4f-460f-a6ba-6d19ab96af70\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 17 17:38:33.260976 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:38:33.260930    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/26063eb2-0a4f-460f-a6ba-6d19ab96af70-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"26063eb2-0a4f-460f-a6ba-6d19ab96af70\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 17 17:38:33.260976 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:38:33.260966    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/26063eb2-0a4f-460f-a6ba-6d19ab96af70-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"26063eb2-0a4f-460f-a6ba-6d19ab96af70\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 17 17:38:33.261271 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:38:33.260993    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/26063eb2-0a4f-460f-a6ba-6d19ab96af70-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"26063eb2-0a4f-460f-a6ba-6d19ab96af70\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 17 17:38:33.261328 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:38:33.261273    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/26063eb2-0a4f-460f-a6ba-6d19ab96af70-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"26063eb2-0a4f-460f-a6ba-6d19ab96af70\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 17 17:38:33.261590 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:38:33.261565    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/26063eb2-0a4f-460f-a6ba-6d19ab96af70-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"26063eb2-0a4f-460f-a6ba-6d19ab96af70\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 17 17:38:33.261672 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:38:33.261611    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/26063eb2-0a4f-460f-a6ba-6d19ab96af70-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"26063eb2-0a4f-460f-a6ba-6d19ab96af70\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 17 17:38:33.263552 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:38:33.263529    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/26063eb2-0a4f-460f-a6ba-6d19ab96af70-config-out\") pod \"alertmanager-main-0\" (UID: \"26063eb2-0a4f-460f-a6ba-6d19ab96af70\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 17 17:38:33.263732 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:38:33.263708    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/26063eb2-0a4f-460f-a6ba-6d19ab96af70-web-config\") pod \"alertmanager-main-0\" (UID: \"26063eb2-0a4f-460f-a6ba-6d19ab96af70\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 17 17:38:33.263976 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:38:33.263958    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/26063eb2-0a4f-460f-a6ba-6d19ab96af70-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"26063eb2-0a4f-460f-a6ba-6d19ab96af70\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 17 17:38:33.264051 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:38:33.263967    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/26063eb2-0a4f-460f-a6ba-6d19ab96af70-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"26063eb2-0a4f-460f-a6ba-6d19ab96af70\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 17 17:38:33.264140 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:38:33.264118    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/26063eb2-0a4f-460f-a6ba-6d19ab96af70-tls-assets\") pod \"alertmanager-main-0\" (UID: \"26063eb2-0a4f-460f-a6ba-6d19ab96af70\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 17 17:38:33.264347 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:38:33.264329    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/26063eb2-0a4f-460f-a6ba-6d19ab96af70-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"26063eb2-0a4f-460f-a6ba-6d19ab96af70\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 17 17:38:33.264498 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:38:33.264476    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/26063eb2-0a4f-460f-a6ba-6d19ab96af70-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"26063eb2-0a4f-460f-a6ba-6d19ab96af70\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 17 17:38:33.265114 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:38:33.265093    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/26063eb2-0a4f-460f-a6ba-6d19ab96af70-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"26063eb2-0a4f-460f-a6ba-6d19ab96af70\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 17 17:38:33.265515 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:38:33.265499    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/26063eb2-0a4f-460f-a6ba-6d19ab96af70-config-volume\") pod \"alertmanager-main-0\" (UID: \"26063eb2-0a4f-460f-a6ba-6d19ab96af70\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 17 17:38:33.270068 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:38:33.270047    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-8srtf\" (UniqueName: \"kubernetes.io/projected/26063eb2-0a4f-460f-a6ba-6d19ab96af70-kube-api-access-8srtf\") pod \"alertmanager-main-0\" (UID: \"26063eb2-0a4f-460f-a6ba-6d19ab96af70\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 17 17:38:33.427495 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:38:33.427401    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0"
Sep 17 17:38:33.705551 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:38:33.705527    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/alertmanager-main-0"]
Sep 17 17:38:33.707821 ip-10-0-143-74 kubenswrapper[2575]: W0917 17:38:33.707792    2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod26063eb2_0a4f_460f_a6ba_6d19ab96af70.slice/crio-e620b827361feaa729b209c4701c08b3db93e3747feaa7c5ae17835b1ce7d948 WatchSource:0}: Error finding container e620b827361feaa729b209c4701c08b3db93e3747feaa7c5ae17835b1ce7d948: Status 404 returned error can't find the container with id e620b827361feaa729b209c4701c08b3db93e3747feaa7c5ae17835b1ce7d948
Sep 17 17:38:34.045530 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:38:34.045424    2575 generic.go:356] "Generic (PLEG): container finished" podID="26063eb2-0a4f-460f-a6ba-6d19ab96af70" containerID="b76593626765fd63c4cfa4c6713c8f4ef75765a1320aad62b431e2a4b128bc91" exitCode=0
Sep 17 17:38:34.045530 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:38:34.045487    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"26063eb2-0a4f-460f-a6ba-6d19ab96af70","Type":"ContainerDied","Data":"b76593626765fd63c4cfa4c6713c8f4ef75765a1320aad62b431e2a4b128bc91"}
Sep 17 17:38:34.045890 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:38:34.045547    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"26063eb2-0a4f-460f-a6ba-6d19ab96af70","Type":"ContainerStarted","Data":"e620b827361feaa729b209c4701c08b3db93e3747feaa7c5ae17835b1ce7d948"}
Sep 17 17:38:34.528511 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:38:34.528476    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-djvzt_1d07e68d-9943-420f-a6a0-5f477792fabd/konnectivity-agent/0.log" limit=1024
Sep 17 17:38:34.637731 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:38:34.637697    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-djvzt_1d07e68d-9943-420f-a6a0-5f477792fabd/konnectivity-agent/0.log" limit=1024
Sep 17 17:38:34.966636 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:38:34.966565    2575 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="b9853529-152c-4642-ab14-885220711fb3" path="/var/lib/kubelet/pods/b9853529-152c-4642-ab14-885220711fb3/volumes"
Sep 17 17:38:35.053045 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:38:35.053009    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"26063eb2-0a4f-460f-a6ba-6d19ab96af70","Type":"ContainerStarted","Data":"627bf6b66210c1e5c7a53d85804cbc67f11ea92df858419090e94363f06da7b8"}
Sep 17 17:38:35.053408 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:38:35.053070    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"26063eb2-0a4f-460f-a6ba-6d19ab96af70","Type":"ContainerStarted","Data":"5d3484211ced81120b09f8cce1f49b5742fba6075f4f7db6ae8c45ea854144ba"}
Sep 17 17:38:35.053408 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:38:35.053102    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"26063eb2-0a4f-460f-a6ba-6d19ab96af70","Type":"ContainerStarted","Data":"1907080a9b56492fa794cb4d3b794994614b7834b72d5ca01732a4ce2108d600"}
Sep 17 17:38:35.053408 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:38:35.053128    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"26063eb2-0a4f-460f-a6ba-6d19ab96af70","Type":"ContainerStarted","Data":"b89456f24ac42964b92a1be9d1f64200e23b8a4b522c3859d2d831e6fa210d46"}
Sep 17 17:38:35.053408 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:38:35.053150    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"26063eb2-0a4f-460f-a6ba-6d19ab96af70","Type":"ContainerStarted","Data":"ba19ead859b2356e967b985f8e730440b624b6d5f1ba6f31ceda8d1739285a2d"}
Sep 17 17:38:35.053408 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:38:35.053176    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"26063eb2-0a4f-460f-a6ba-6d19ab96af70","Type":"ContainerStarted","Data":"a909741bcaa5e81669fc9fecbaeaf5bc21324014f0eb71f0559834a1705b80b5"}
Sep 17 17:38:35.087750 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:38:35.087700    2575 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/alertmanager-main-0" podStartSLOduration=2.067148953 podStartE2EDuration="2.087685674s" totalImagesPullingTime="0s" totalInitContainerRuntime="20.536721ms" isStatefulPod=false podCreationTimestamp="2026-09-17 17:38:33 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-17 17:38:35.085380084 +0000 UTC m=+234.715710058" watchObservedRunningTime="2026-09-17 17:38:35.087685674 +0000 UTC m=+234.718015648"
Sep 17 17:38:37.804999 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:38:37.804960    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-djvzt_1d07e68d-9943-420f-a6a0-5f477792fabd/konnectivity-agent/0.log" limit=1024
Sep 17 17:38:37.904175 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:38:37.904138    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-djvzt_1d07e68d-9943-420f-a6a0-5f477792fabd/konnectivity-agent/0.log" limit=1024
Sep 17 17:38:38.885740 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:38:38.885702    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-djvzt_1d07e68d-9943-420f-a6a0-5f477792fabd/konnectivity-agent/0.log" limit=1024
Sep 17 17:38:41.639327 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:38:41.639277    2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-6b674b467b-7nf59"
Sep 17 17:38:41.639327 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:38:41.639347    2575 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-6b674b467b-7nf59"
Sep 17 17:38:41.644171 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:38:41.644144    2575 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-6b674b467b-7nf59"
Sep 17 17:38:42.084828 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:38:42.084741    2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-6b674b467b-7nf59"
Sep 17 17:38:42.267645 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:38:42.267610    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-djvzt_1d07e68d-9943-420f-a6a0-5f477792fabd/konnectivity-agent/0.log" limit=1024
Sep 17 17:38:42.366254 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:38:42.366170    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-djvzt_1d07e68d-9943-420f-a6a0-5f477792fabd/konnectivity-agent/0.log" limit=1024
Sep 17 17:38:42.426278 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:38:42.426248    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-djvzt_1d07e68d-9943-420f-a6a0-5f477792fabd/konnectivity-agent/0.log" limit=1024
Sep 17 17:38:47.836907 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:38:47.836862    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-djvzt_1d07e68d-9943-420f-a6a0-5f477792fabd/konnectivity-agent/0.log" limit=1024
Sep 17 17:38:47.935706 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:38:47.935667    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-djvzt_1d07e68d-9943-420f-a6a0-5f477792fabd/konnectivity-agent/0.log" limit=1024
Sep 17 17:39:33.507156 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:39:33.507116    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-djvzt_1d07e68d-9943-420f-a6a0-5f477792fabd/konnectivity-agent/0.log" limit=1024
Sep 17 17:39:33.653382 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:39:33.653334    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-djvzt_1d07e68d-9943-420f-a6a0-5f477792fabd/konnectivity-agent/0.log" limit=1024
Sep 17 17:39:40.747462 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:39:40.747419    2575 kubelet.go:1648] "Image garbage collection succeeded"
Sep 17 17:41:03.493612 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:41:03.493570    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-djvzt_1d07e68d-9943-420f-a6a0-5f477792fabd/konnectivity-agent/0.log" limit=1024
Sep 17 17:41:03.602999 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:41:03.602956    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-djvzt_1d07e68d-9943-420f-a6a0-5f477792fabd/konnectivity-agent/0.log" limit=1024
Sep 17 17:41:43.365477 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:41:43.365414    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-djvzt_1d07e68d-9943-420f-a6a0-5f477792fabd/konnectivity-agent/0.log" limit=1024
Sep 17 17:41:49.217825 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:41:49.217794    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-djvzt_1d07e68d-9943-420f-a6a0-5f477792fabd/konnectivity-agent/0.log" limit=1024
Sep 17 17:41:49.285155 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:41:49.285121    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-djvzt_1d07e68d-9943-420f-a6a0-5f477792fabd/konnectivity-agent/0.log" limit=1024
Sep 17 17:42:34.989233 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:42:34.989144    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-djvzt_1d07e68d-9943-420f-a6a0-5f477792fabd/konnectivity-agent/0.log" limit=1024
Sep 17 17:43:02.716041 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:43:02.715995    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-djvzt_1d07e68d-9943-420f-a6a0-5f477792fabd/konnectivity-agent/0.log" limit=1024
Sep 17 17:43:22.188883 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:43:22.188844    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-djvzt_1d07e68d-9943-420f-a6a0-5f477792fabd/konnectivity-agent/0.log" limit=1024
Sep 17 17:43:22.331378 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:43:22.331339    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-djvzt_1d07e68d-9943-420f-a6a0-5f477792fabd/konnectivity-agent/0.log" limit=1024
Sep 17 17:43:22.435939 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:43:22.435906    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-djvzt_1d07e68d-9943-420f-a6a0-5f477792fabd/konnectivity-agent/0.log" limit=1024
Sep 17 17:43:22.862021 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:43:22.861988    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-djvzt_1d07e68d-9943-420f-a6a0-5f477792fabd/konnectivity-agent/0.log" limit=1024
Sep 17 17:43:23.011081 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:43:23.011047    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-djvzt_1d07e68d-9943-420f-a6a0-5f477792fabd/konnectivity-agent/0.log" limit=1024
Sep 17 17:43:32.952684 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:43:32.952645    2575 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-6b674b467b-7nf59"]
Sep 17 17:43:33.137589 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:43:33.137537    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-djvzt_1d07e68d-9943-420f-a6a0-5f477792fabd/konnectivity-agent/0.log" limit=1024
Sep 17 17:43:33.294107 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:43:33.293981    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-djvzt_1d07e68d-9943-420f-a6a0-5f477792fabd/konnectivity-agent/0.log" limit=1024
Sep 17 17:43:57.976964 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:43:57.976894    2575 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-console/console-6b674b467b-7nf59" podUID="879b8249-fc8d-41d5-9708-8c396841dd12" containerName="console" containerID="cri-o://bf3e822e6949b272c505b790371bb1405217bb6b1a7dea798e4d96a518147010" gracePeriod=15
Sep 17 17:43:58.220851 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:43:58.220823    2575 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-6b674b467b-7nf59"
Sep 17 17:43:58.293473 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:43:58.293362    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-x4np7\" (UniqueName: \"kubernetes.io/projected/879b8249-fc8d-41d5-9708-8c396841dd12-kube-api-access-x4np7\") pod \"879b8249-fc8d-41d5-9708-8c396841dd12\" (UID: \"879b8249-fc8d-41d5-9708-8c396841dd12\") "
Sep 17 17:43:58.293473 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:43:58.293404    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/879b8249-fc8d-41d5-9708-8c396841dd12-trusted-ca-bundle\") pod \"879b8249-fc8d-41d5-9708-8c396841dd12\" (UID: \"879b8249-fc8d-41d5-9708-8c396841dd12\") "
Sep 17 17:43:58.293473 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:43:58.293427    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/879b8249-fc8d-41d5-9708-8c396841dd12-service-ca\") pod \"879b8249-fc8d-41d5-9708-8c396841dd12\" (UID: \"879b8249-fc8d-41d5-9708-8c396841dd12\") "
Sep 17 17:43:58.293764 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:43:58.293479    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/879b8249-fc8d-41d5-9708-8c396841dd12-console-config\") pod \"879b8249-fc8d-41d5-9708-8c396841dd12\" (UID: \"879b8249-fc8d-41d5-9708-8c396841dd12\") "
Sep 17 17:43:58.293764 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:43:58.293521    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/879b8249-fc8d-41d5-9708-8c396841dd12-oauth-serving-cert\") pod \"879b8249-fc8d-41d5-9708-8c396841dd12\" (UID: \"879b8249-fc8d-41d5-9708-8c396841dd12\") "
Sep 17 17:43:58.293764 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:43:58.293544    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/879b8249-fc8d-41d5-9708-8c396841dd12-console-oauth-config\") pod \"879b8249-fc8d-41d5-9708-8c396841dd12\" (UID: \"879b8249-fc8d-41d5-9708-8c396841dd12\") "
Sep 17 17:43:58.293764 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:43:58.293582    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/879b8249-fc8d-41d5-9708-8c396841dd12-console-serving-cert\") pod \"879b8249-fc8d-41d5-9708-8c396841dd12\" (UID: \"879b8249-fc8d-41d5-9708-8c396841dd12\") "
Sep 17 17:43:58.293964 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:43:58.293913    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/879b8249-fc8d-41d5-9708-8c396841dd12-service-ca" (OuterVolumeSpecName: "service-ca") pod "879b8249-fc8d-41d5-9708-8c396841dd12" (UID: "879b8249-fc8d-41d5-9708-8c396841dd12"). InnerVolumeSpecName "service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 17 17:43:58.294016 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:43:58.293966    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/879b8249-fc8d-41d5-9708-8c396841dd12-oauth-serving-cert" (OuterVolumeSpecName: "oauth-serving-cert") pod "879b8249-fc8d-41d5-9708-8c396841dd12" (UID: "879b8249-fc8d-41d5-9708-8c396841dd12"). InnerVolumeSpecName "oauth-serving-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 17 17:43:58.294016 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:43:58.293992    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/879b8249-fc8d-41d5-9708-8c396841dd12-console-config" (OuterVolumeSpecName: "console-config") pod "879b8249-fc8d-41d5-9708-8c396841dd12" (UID: "879b8249-fc8d-41d5-9708-8c396841dd12"). InnerVolumeSpecName "console-config". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 17 17:43:58.294081 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:43:58.294016    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/879b8249-fc8d-41d5-9708-8c396841dd12-trusted-ca-bundle" (OuterVolumeSpecName: "trusted-ca-bundle") pod "879b8249-fc8d-41d5-9708-8c396841dd12" (UID: "879b8249-fc8d-41d5-9708-8c396841dd12"). InnerVolumeSpecName "trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 17 17:43:58.295839 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:43:58.295821    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/879b8249-fc8d-41d5-9708-8c396841dd12-console-serving-cert" (OuterVolumeSpecName: "console-serving-cert") pod "879b8249-fc8d-41d5-9708-8c396841dd12" (UID: "879b8249-fc8d-41d5-9708-8c396841dd12"). InnerVolumeSpecName "console-serving-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 17 17:43:58.296152 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:43:58.296122    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/879b8249-fc8d-41d5-9708-8c396841dd12-kube-api-access-x4np7" (OuterVolumeSpecName: "kube-api-access-x4np7") pod "879b8249-fc8d-41d5-9708-8c396841dd12" (UID: "879b8249-fc8d-41d5-9708-8c396841dd12"). InnerVolumeSpecName "kube-api-access-x4np7". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 17 17:43:58.296152 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:43:58.296122    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/879b8249-fc8d-41d5-9708-8c396841dd12-console-oauth-config" (OuterVolumeSpecName: "console-oauth-config") pod "879b8249-fc8d-41d5-9708-8c396841dd12" (UID: "879b8249-fc8d-41d5-9708-8c396841dd12"). InnerVolumeSpecName "console-oauth-config". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 17 17:43:58.327779 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:43:58.327747    2575 generic.go:356] "Generic (PLEG): container finished" podID="879b8249-fc8d-41d5-9708-8c396841dd12" containerID="bf3e822e6949b272c505b790371bb1405217bb6b1a7dea798e4d96a518147010" exitCode=0
Sep 17 17:43:58.327942 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:43:58.327795    2575 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-6b674b467b-7nf59"
Sep 17 17:43:58.327942 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:43:58.327825    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-6b674b467b-7nf59" event={"ID":"879b8249-fc8d-41d5-9708-8c396841dd12","Type":"ContainerDied","Data":"bf3e822e6949b272c505b790371bb1405217bb6b1a7dea798e4d96a518147010"}
Sep 17 17:43:58.327942 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:43:58.327877    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-6b674b467b-7nf59" event={"ID":"879b8249-fc8d-41d5-9708-8c396841dd12","Type":"ContainerDied","Data":"a364018b71975cabd0d4dbde33dd2a8768d0fe389d64f040e89bc6bd8bd41f16"}
Sep 17 17:43:58.327942 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:43:58.327897    2575 scope.go:117] "RemoveContainer" containerID="bf3e822e6949b272c505b790371bb1405217bb6b1a7dea798e4d96a518147010"
Sep 17 17:43:58.337029 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:43:58.337009    2575 scope.go:117] "RemoveContainer" containerID="bf3e822e6949b272c505b790371bb1405217bb6b1a7dea798e4d96a518147010"
Sep 17 17:43:58.337343 ip-10-0-143-74 kubenswrapper[2575]: E0917 17:43:58.337320    2575 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"bf3e822e6949b272c505b790371bb1405217bb6b1a7dea798e4d96a518147010\": container with ID starting with bf3e822e6949b272c505b790371bb1405217bb6b1a7dea798e4d96a518147010 not found: ID does not exist" containerID="bf3e822e6949b272c505b790371bb1405217bb6b1a7dea798e4d96a518147010"
Sep 17 17:43:58.337416 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:43:58.337362    2575 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"bf3e822e6949b272c505b790371bb1405217bb6b1a7dea798e4d96a518147010"} err="failed to get container status \"bf3e822e6949b272c505b790371bb1405217bb6b1a7dea798e4d96a518147010\": rpc error: code = NotFound desc = could not find container \"bf3e822e6949b272c505b790371bb1405217bb6b1a7dea798e4d96a518147010\": container with ID starting with bf3e822e6949b272c505b790371bb1405217bb6b1a7dea798e4d96a518147010 not found: ID does not exist"
Sep 17 17:43:58.350857 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:43:58.350829    2575 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-6b674b467b-7nf59"]
Sep 17 17:43:58.355388 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:43:58.355363    2575 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-console/console-6b674b467b-7nf59"]
Sep 17 17:43:58.394936 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:43:58.394906    2575 reconciler_common.go:299] "Volume detached for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/879b8249-fc8d-41d5-9708-8c396841dd12-oauth-serving-cert\") on node \"ip-10-0-143-74.ec2.internal\" DevicePath \"\""
Sep 17 17:43:58.394936 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:43:58.394931    2575 reconciler_common.go:299] "Volume detached for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/879b8249-fc8d-41d5-9708-8c396841dd12-console-oauth-config\") on node \"ip-10-0-143-74.ec2.internal\" DevicePath \"\""
Sep 17 17:43:58.394936 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:43:58.394943    2575 reconciler_common.go:299] "Volume detached for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/879b8249-fc8d-41d5-9708-8c396841dd12-console-serving-cert\") on node \"ip-10-0-143-74.ec2.internal\" DevicePath \"\""
Sep 17 17:43:58.395146 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:43:58.394952    2575 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-x4np7\" (UniqueName: \"kubernetes.io/projected/879b8249-fc8d-41d5-9708-8c396841dd12-kube-api-access-x4np7\") on node \"ip-10-0-143-74.ec2.internal\" DevicePath \"\""
Sep 17 17:43:58.395146 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:43:58.394961    2575 reconciler_common.go:299] "Volume detached for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/879b8249-fc8d-41d5-9708-8c396841dd12-trusted-ca-bundle\") on node \"ip-10-0-143-74.ec2.internal\" DevicePath \"\""
Sep 17 17:43:58.395146 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:43:58.394970    2575 reconciler_common.go:299] "Volume detached for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/879b8249-fc8d-41d5-9708-8c396841dd12-service-ca\") on node \"ip-10-0-143-74.ec2.internal\" DevicePath \"\""
Sep 17 17:43:58.395146 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:43:58.394977    2575 reconciler_common.go:299] "Volume detached for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/879b8249-fc8d-41d5-9708-8c396841dd12-console-config\") on node \"ip-10-0-143-74.ec2.internal\" DevicePath \"\""
Sep 17 17:43:58.966967 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:43:58.966932    2575 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="879b8249-fc8d-41d5-9708-8c396841dd12" path="/var/lib/kubelet/pods/879b8249-fc8d-41d5-9708-8c396841dd12/volumes"
Sep 17 17:44:42.837177 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:44:42.837120    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-djvzt_1d07e68d-9943-420f-a6a0-5f477792fabd/konnectivity-agent/0.log" limit=1024
Sep 17 17:44:42.935958 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:44:42.935933    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-djvzt_1d07e68d-9943-420f-a6a0-5f477792fabd/konnectivity-agent/0.log" limit=1024
Sep 17 17:46:09.517706 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:46:09.517655    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-djvzt_1d07e68d-9943-420f-a6a0-5f477792fabd/konnectivity-agent/0.log" limit=1024
Sep 17 17:48:56.200394 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:48:56.200350    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-djvzt_1d07e68d-9943-420f-a6a0-5f477792fabd/konnectivity-agent/0.log" limit=1024
Sep 17 17:48:57.752019 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:48:57.751975    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-djvzt_1d07e68d-9943-420f-a6a0-5f477792fabd/konnectivity-agent/0.log" limit=1024
Sep 17 17:48:58.014869 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:48:58.014798    2575 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-marketplace/1acd376c2f780381bc434f5f1985f786a96c8309f7a005b52ed03f0c1awzjvm"]
Sep 17 17:48:58.015122 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:48:58.015111    2575 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="879b8249-fc8d-41d5-9708-8c396841dd12" containerName="console"
Sep 17 17:48:58.015165 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:48:58.015125    2575 state_mem.go:107] "Deleted CPUSet assignment" podUID="879b8249-fc8d-41d5-9708-8c396841dd12" containerName="console"
Sep 17 17:48:58.015203 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:48:58.015175    2575 memory_manager.go:368] "RemoveStaleState removing state" podUID="879b8249-fc8d-41d5-9708-8c396841dd12" containerName="console"
Sep 17 17:48:58.017980 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:48:58.017949    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/1acd376c2f780381bc434f5f1985f786a96c8309f7a005b52ed03f0c1awzjvm"
Sep 17 17:48:58.020392 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:48:58.020374    2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"kube-root-ca.crt\""
Sep 17 17:48:58.020522 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:48:58.020380    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-marketplace\"/\"default-dockercfg-p47rs\""
Sep 17 17:48:58.021166 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:48:58.021151    2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"openshift-service-ca.crt\""
Sep 17 17:48:58.033522 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:48:58.033494    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/1acd376c2f780381bc434f5f1985f786a96c8309f7a005b52ed03f0c1awzjvm"]
Sep 17 17:48:58.158979 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:48:58.158941    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/23b63ac5-af44-4d3f-ba86-7ea73d7aac2d-util\") pod \"1acd376c2f780381bc434f5f1985f786a96c8309f7a005b52ed03f0c1awzjvm\" (UID: \"23b63ac5-af44-4d3f-ba86-7ea73d7aac2d\") " pod="openshift-marketplace/1acd376c2f780381bc434f5f1985f786a96c8309f7a005b52ed03f0c1awzjvm"
Sep 17 17:48:58.158979 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:48:58.158982    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/23b63ac5-af44-4d3f-ba86-7ea73d7aac2d-bundle\") pod \"1acd376c2f780381bc434f5f1985f786a96c8309f7a005b52ed03f0c1awzjvm\" (UID: \"23b63ac5-af44-4d3f-ba86-7ea73d7aac2d\") " pod="openshift-marketplace/1acd376c2f780381bc434f5f1985f786a96c8309f7a005b52ed03f0c1awzjvm"
Sep 17 17:48:58.159180 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:48:58.159055    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5sz5q\" (UniqueName: \"kubernetes.io/projected/23b63ac5-af44-4d3f-ba86-7ea73d7aac2d-kube-api-access-5sz5q\") pod \"1acd376c2f780381bc434f5f1985f786a96c8309f7a005b52ed03f0c1awzjvm\" (UID: \"23b63ac5-af44-4d3f-ba86-7ea73d7aac2d\") " pod="openshift-marketplace/1acd376c2f780381bc434f5f1985f786a96c8309f7a005b52ed03f0c1awzjvm"
Sep 17 17:48:58.259689 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:48:58.259659    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5sz5q\" (UniqueName: \"kubernetes.io/projected/23b63ac5-af44-4d3f-ba86-7ea73d7aac2d-kube-api-access-5sz5q\") pod \"1acd376c2f780381bc434f5f1985f786a96c8309f7a005b52ed03f0c1awzjvm\" (UID: \"23b63ac5-af44-4d3f-ba86-7ea73d7aac2d\") " pod="openshift-marketplace/1acd376c2f780381bc434f5f1985f786a96c8309f7a005b52ed03f0c1awzjvm"
Sep 17 17:48:58.259847 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:48:58.259714    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/23b63ac5-af44-4d3f-ba86-7ea73d7aac2d-util\") pod \"1acd376c2f780381bc434f5f1985f786a96c8309f7a005b52ed03f0c1awzjvm\" (UID: \"23b63ac5-af44-4d3f-ba86-7ea73d7aac2d\") " pod="openshift-marketplace/1acd376c2f780381bc434f5f1985f786a96c8309f7a005b52ed03f0c1awzjvm"
Sep 17 17:48:58.259847 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:48:58.259732    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/23b63ac5-af44-4d3f-ba86-7ea73d7aac2d-bundle\") pod \"1acd376c2f780381bc434f5f1985f786a96c8309f7a005b52ed03f0c1awzjvm\" (UID: \"23b63ac5-af44-4d3f-ba86-7ea73d7aac2d\") " pod="openshift-marketplace/1acd376c2f780381bc434f5f1985f786a96c8309f7a005b52ed03f0c1awzjvm"
Sep 17 17:48:58.260068 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:48:58.260049    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/23b63ac5-af44-4d3f-ba86-7ea73d7aac2d-util\") pod \"1acd376c2f780381bc434f5f1985f786a96c8309f7a005b52ed03f0c1awzjvm\" (UID: \"23b63ac5-af44-4d3f-ba86-7ea73d7aac2d\") " pod="openshift-marketplace/1acd376c2f780381bc434f5f1985f786a96c8309f7a005b52ed03f0c1awzjvm"
Sep 17 17:48:58.260111 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:48:58.260057    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/23b63ac5-af44-4d3f-ba86-7ea73d7aac2d-bundle\") pod \"1acd376c2f780381bc434f5f1985f786a96c8309f7a005b52ed03f0c1awzjvm\" (UID: \"23b63ac5-af44-4d3f-ba86-7ea73d7aac2d\") " pod="openshift-marketplace/1acd376c2f780381bc434f5f1985f786a96c8309f7a005b52ed03f0c1awzjvm"
Sep 17 17:48:58.267840 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:48:58.267784    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-5sz5q\" (UniqueName: \"kubernetes.io/projected/23b63ac5-af44-4d3f-ba86-7ea73d7aac2d-kube-api-access-5sz5q\") pod \"1acd376c2f780381bc434f5f1985f786a96c8309f7a005b52ed03f0c1awzjvm\" (UID: \"23b63ac5-af44-4d3f-ba86-7ea73d7aac2d\") " pod="openshift-marketplace/1acd376c2f780381bc434f5f1985f786a96c8309f7a005b52ed03f0c1awzjvm"
Sep 17 17:48:58.327029 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:48:58.326977    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/1acd376c2f780381bc434f5f1985f786a96c8309f7a005b52ed03f0c1awzjvm"
Sep 17 17:48:58.599554 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:48:58.599460    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/1acd376c2f780381bc434f5f1985f786a96c8309f7a005b52ed03f0c1awzjvm"]
Sep 17 17:48:58.603367 ip-10-0-143-74 kubenswrapper[2575]: W0917 17:48:58.603335    2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod23b63ac5_af44_4d3f_ba86_7ea73d7aac2d.slice/crio-79e2101e908f9b19d0070b914e6129b1e2bc0ff6232f23e709c97747fdf8b77a WatchSource:0}: Error finding container 79e2101e908f9b19d0070b914e6129b1e2bc0ff6232f23e709c97747fdf8b77a: Status 404 returned error can't find the container with id 79e2101e908f9b19d0070b914e6129b1e2bc0ff6232f23e709c97747fdf8b77a
Sep 17 17:48:58.605372 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:48:58.605354    2575 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Sep 17 17:48:59.491484 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:48:59.491431    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/1acd376c2f780381bc434f5f1985f786a96c8309f7a005b52ed03f0c1awzjvm" event={"ID":"23b63ac5-af44-4d3f-ba86-7ea73d7aac2d","Type":"ContainerStarted","Data":"79e2101e908f9b19d0070b914e6129b1e2bc0ff6232f23e709c97747fdf8b77a"}
Sep 17 17:49:06.524211 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:49:06.524174    2575 generic.go:356] "Generic (PLEG): container finished" podID="23b63ac5-af44-4d3f-ba86-7ea73d7aac2d" containerID="57762be380af879d4c66e7bdf8e7c2a5ff27be7cfda4bc7831e2eb6d66fe3382" exitCode=0
Sep 17 17:49:06.524696 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:49:06.524237    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/1acd376c2f780381bc434f5f1985f786a96c8309f7a005b52ed03f0c1awzjvm" event={"ID":"23b63ac5-af44-4d3f-ba86-7ea73d7aac2d","Type":"ContainerDied","Data":"57762be380af879d4c66e7bdf8e7c2a5ff27be7cfda4bc7831e2eb6d66fe3382"}
Sep 17 17:49:09.540683 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:49:09.540650    2575 generic.go:356] "Generic (PLEG): container finished" podID="23b63ac5-af44-4d3f-ba86-7ea73d7aac2d" containerID="d50d37c827012ea4dd633a4e4a62393e8817c1b2bd555bfcc0b08b0ee2352a14" exitCode=0
Sep 17 17:49:09.541086 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:49:09.540718    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/1acd376c2f780381bc434f5f1985f786a96c8309f7a005b52ed03f0c1awzjvm" event={"ID":"23b63ac5-af44-4d3f-ba86-7ea73d7aac2d","Type":"ContainerDied","Data":"d50d37c827012ea4dd633a4e4a62393e8817c1b2bd555bfcc0b08b0ee2352a14"}
Sep 17 17:49:16.574450 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:49:16.574416    2575 generic.go:356] "Generic (PLEG): container finished" podID="23b63ac5-af44-4d3f-ba86-7ea73d7aac2d" containerID="c91dde0ff32ec1d08a1a7d6c533870882ec2a56e69c0390f171a0b7569785eed" exitCode=0
Sep 17 17:49:16.574824 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:49:16.574475    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/1acd376c2f780381bc434f5f1985f786a96c8309f7a005b52ed03f0c1awzjvm" event={"ID":"23b63ac5-af44-4d3f-ba86-7ea73d7aac2d","Type":"ContainerDied","Data":"c91dde0ff32ec1d08a1a7d6c533870882ec2a56e69c0390f171a0b7569785eed"}
Sep 17 17:49:17.717919 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:49:17.717894    2575 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/1acd376c2f780381bc434f5f1985f786a96c8309f7a005b52ed03f0c1awzjvm"
Sep 17 17:49:17.731199 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:49:17.731173    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/23b63ac5-af44-4d3f-ba86-7ea73d7aac2d-bundle\") pod \"23b63ac5-af44-4d3f-ba86-7ea73d7aac2d\" (UID: \"23b63ac5-af44-4d3f-ba86-7ea73d7aac2d\") "
Sep 17 17:49:17.731326 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:49:17.731226    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/23b63ac5-af44-4d3f-ba86-7ea73d7aac2d-util\") pod \"23b63ac5-af44-4d3f-ba86-7ea73d7aac2d\" (UID: \"23b63ac5-af44-4d3f-ba86-7ea73d7aac2d\") "
Sep 17 17:49:17.731326 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:49:17.731275    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-5sz5q\" (UniqueName: \"kubernetes.io/projected/23b63ac5-af44-4d3f-ba86-7ea73d7aac2d-kube-api-access-5sz5q\") pod \"23b63ac5-af44-4d3f-ba86-7ea73d7aac2d\" (UID: \"23b63ac5-af44-4d3f-ba86-7ea73d7aac2d\") "
Sep 17 17:49:17.731850 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:49:17.731820    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/23b63ac5-af44-4d3f-ba86-7ea73d7aac2d-bundle" (OuterVolumeSpecName: "bundle") pod "23b63ac5-af44-4d3f-ba86-7ea73d7aac2d" (UID: "23b63ac5-af44-4d3f-ba86-7ea73d7aac2d"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 17 17:49:17.733453 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:49:17.733418    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/23b63ac5-af44-4d3f-ba86-7ea73d7aac2d-kube-api-access-5sz5q" (OuterVolumeSpecName: "kube-api-access-5sz5q") pod "23b63ac5-af44-4d3f-ba86-7ea73d7aac2d" (UID: "23b63ac5-af44-4d3f-ba86-7ea73d7aac2d"). InnerVolumeSpecName "kube-api-access-5sz5q". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 17 17:49:17.736239 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:49:17.736217    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/23b63ac5-af44-4d3f-ba86-7ea73d7aac2d-util" (OuterVolumeSpecName: "util") pod "23b63ac5-af44-4d3f-ba86-7ea73d7aac2d" (UID: "23b63ac5-af44-4d3f-ba86-7ea73d7aac2d"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 17 17:49:17.832629 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:49:17.832536    2575 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-5sz5q\" (UniqueName: \"kubernetes.io/projected/23b63ac5-af44-4d3f-ba86-7ea73d7aac2d-kube-api-access-5sz5q\") on node \"ip-10-0-143-74.ec2.internal\" DevicePath \"\""
Sep 17 17:49:17.832629 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:49:17.832569    2575 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/23b63ac5-af44-4d3f-ba86-7ea73d7aac2d-bundle\") on node \"ip-10-0-143-74.ec2.internal\" DevicePath \"\""
Sep 17 17:49:17.832629 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:49:17.832580    2575 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/23b63ac5-af44-4d3f-ba86-7ea73d7aac2d-util\") on node \"ip-10-0-143-74.ec2.internal\" DevicePath \"\""
Sep 17 17:49:18.583747 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:49:18.583709    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/1acd376c2f780381bc434f5f1985f786a96c8309f7a005b52ed03f0c1awzjvm" event={"ID":"23b63ac5-af44-4d3f-ba86-7ea73d7aac2d","Type":"ContainerDied","Data":"79e2101e908f9b19d0070b914e6129b1e2bc0ff6232f23e709c97747fdf8b77a"}
Sep 17 17:49:18.583915 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:49:18.583759    2575 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/1acd376c2f780381bc434f5f1985f786a96c8309f7a005b52ed03f0c1awzjvm"
Sep 17 17:49:18.583915 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:49:18.583769    2575 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="79e2101e908f9b19d0070b914e6129b1e2bc0ff6232f23e709c97747fdf8b77a"
Sep 17 17:49:20.551095 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:49:20.551055    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-djvzt_1d07e68d-9943-420f-a6a0-5f477792fabd/konnectivity-agent/0.log" limit=1024
Sep 17 17:49:20.642357 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:49:20.642324    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-djvzt_1d07e68d-9943-420f-a6a0-5f477792fabd/konnectivity-agent/0.log" limit=1024
Sep 17 17:49:26.431148 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:49:26.431107    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-djvzt_1d07e68d-9943-420f-a6a0-5f477792fabd/konnectivity-agent/0.log" limit=1024
Sep 17 17:49:26.672940 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:49:26.672906    2575 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-keda/keda-operator-579c8958f7-tcj9g"]
Sep 17 17:49:26.673233 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:49:26.673220    2575 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="23b63ac5-af44-4d3f-ba86-7ea73d7aac2d" containerName="util"
Sep 17 17:49:26.673279 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:49:26.673236    2575 state_mem.go:107] "Deleted CPUSet assignment" podUID="23b63ac5-af44-4d3f-ba86-7ea73d7aac2d" containerName="util"
Sep 17 17:49:26.673279 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:49:26.673249    2575 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="23b63ac5-af44-4d3f-ba86-7ea73d7aac2d" containerName="extract"
Sep 17 17:49:26.673279 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:49:26.673254    2575 state_mem.go:107] "Deleted CPUSet assignment" podUID="23b63ac5-af44-4d3f-ba86-7ea73d7aac2d" containerName="extract"
Sep 17 17:49:26.673279 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:49:26.673262    2575 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="23b63ac5-af44-4d3f-ba86-7ea73d7aac2d" containerName="pull"
Sep 17 17:49:26.673279 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:49:26.673267    2575 state_mem.go:107] "Deleted CPUSet assignment" podUID="23b63ac5-af44-4d3f-ba86-7ea73d7aac2d" containerName="pull"
Sep 17 17:49:26.673429 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:49:26.673335    2575 memory_manager.go:368] "RemoveStaleState removing state" podUID="23b63ac5-af44-4d3f-ba86-7ea73d7aac2d" containerName="extract"
Sep 17 17:49:26.676970 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:49:26.676953    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-keda/keda-operator-579c8958f7-tcj9g"
Sep 17 17:49:26.679511 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:49:26.679486    2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-keda\"/\"keda-ocp-cabundle\""
Sep 17 17:49:26.679671 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:49:26.679652    2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-keda\"/\"kube-root-ca.crt\""
Sep 17 17:49:26.680493 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:49:26.680473    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-keda\"/\"keda-operator-certs\""
Sep 17 17:49:26.680616 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:49:26.680518    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-keda\"/\"kedaorg-certs\""
Sep 17 17:49:26.680616 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:49:26.680528    2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-keda\"/\"openshift-service-ca.crt\""
Sep 17 17:49:26.680616 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:49:26.680535    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-keda\"/\"keda-operator-dockercfg-rk7tc\""
Sep 17 17:49:26.687538 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:49:26.687517    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-keda/keda-operator-579c8958f7-tcj9g"]
Sep 17 17:49:26.700672 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:49:26.700640    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cabundle0\" (UniqueName: \"kubernetes.io/configmap/a731fa3e-457b-47ee-b553-4cd9fed175db-cabundle0\") pod \"keda-operator-579c8958f7-tcj9g\" (UID: \"a731fa3e-457b-47ee-b553-4cd9fed175db\") " pod="openshift-keda/keda-operator-579c8958f7-tcj9g"
Sep 17 17:49:26.700864 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:49:26.700689    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-l4vrx\" (UniqueName: \"kubernetes.io/projected/a731fa3e-457b-47ee-b553-4cd9fed175db-kube-api-access-l4vrx\") pod \"keda-operator-579c8958f7-tcj9g\" (UID: \"a731fa3e-457b-47ee-b553-4cd9fed175db\") " pod="openshift-keda/keda-operator-579c8958f7-tcj9g"
Sep 17 17:49:26.700864 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:49:26.700763    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"certificates\" (UniqueName: \"kubernetes.io/projected/a731fa3e-457b-47ee-b553-4cd9fed175db-certificates\") pod \"keda-operator-579c8958f7-tcj9g\" (UID: \"a731fa3e-457b-47ee-b553-4cd9fed175db\") " pod="openshift-keda/keda-operator-579c8958f7-tcj9g"
Sep 17 17:49:26.703547 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:49:26.703517    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-djvzt_1d07e68d-9943-420f-a6a0-5f477792fabd/konnectivity-agent/0.log" limit=1024
Sep 17 17:49:26.801546 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:49:26.801507    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cabundle0\" (UniqueName: \"kubernetes.io/configmap/a731fa3e-457b-47ee-b553-4cd9fed175db-cabundle0\") pod \"keda-operator-579c8958f7-tcj9g\" (UID: \"a731fa3e-457b-47ee-b553-4cd9fed175db\") " pod="openshift-keda/keda-operator-579c8958f7-tcj9g"
Sep 17 17:49:26.801743 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:49:26.801555    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-l4vrx\" (UniqueName: \"kubernetes.io/projected/a731fa3e-457b-47ee-b553-4cd9fed175db-kube-api-access-l4vrx\") pod \"keda-operator-579c8958f7-tcj9g\" (UID: \"a731fa3e-457b-47ee-b553-4cd9fed175db\") " pod="openshift-keda/keda-operator-579c8958f7-tcj9g"
Sep 17 17:49:26.801743 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:49:26.801590    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"certificates\" (UniqueName: \"kubernetes.io/projected/a731fa3e-457b-47ee-b553-4cd9fed175db-certificates\") pod \"keda-operator-579c8958f7-tcj9g\" (UID: \"a731fa3e-457b-47ee-b553-4cd9fed175db\") " pod="openshift-keda/keda-operator-579c8958f7-tcj9g"
Sep 17 17:49:26.801860 ip-10-0-143-74 kubenswrapper[2575]: E0917 17:49:26.801752    2575 projected.go:266] Couldn't get secret openshift-keda/keda-operator-certs: secret "keda-operator-certs" not found
Sep 17 17:49:26.801860 ip-10-0-143-74 kubenswrapper[2575]: E0917 17:49:26.801771    2575 secret.go:281] references non-existent secret key: ca.crt
Sep 17 17:49:26.801860 ip-10-0-143-74 kubenswrapper[2575]: E0917 17:49:26.801782    2575 projected.go:279] Couldn't get secret payload openshift-keda/kedaorg-certs: references non-existent secret key: ca.crt
Sep 17 17:49:26.801860 ip-10-0-143-74 kubenswrapper[2575]: E0917 17:49:26.801798    2575 projected.go:196] Error preparing data for projected volume certificates for pod openshift-keda/keda-operator-579c8958f7-tcj9g: [secret "keda-operator-certs" not found, references non-existent secret key: ca.crt]
Sep 17 17:49:26.802052 ip-10-0-143-74 kubenswrapper[2575]: E0917 17:49:26.801862    2575 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/a731fa3e-457b-47ee-b553-4cd9fed175db-certificates podName:a731fa3e-457b-47ee-b553-4cd9fed175db nodeName:}" failed. No retries permitted until 2026-09-17 17:49:27.301842301 +0000 UTC m=+886.932172271 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "certificates" (UniqueName: "kubernetes.io/projected/a731fa3e-457b-47ee-b553-4cd9fed175db-certificates") pod "keda-operator-579c8958f7-tcj9g" (UID: "a731fa3e-457b-47ee-b553-4cd9fed175db") : [secret "keda-operator-certs" not found, references non-existent secret key: ca.crt]
Sep 17 17:49:26.802132 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:49:26.802109    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cabundle0\" (UniqueName: \"kubernetes.io/configmap/a731fa3e-457b-47ee-b553-4cd9fed175db-cabundle0\") pod \"keda-operator-579c8958f7-tcj9g\" (UID: \"a731fa3e-457b-47ee-b553-4cd9fed175db\") " pod="openshift-keda/keda-operator-579c8958f7-tcj9g"
Sep 17 17:49:26.813773 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:49:26.813730    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-l4vrx\" (UniqueName: \"kubernetes.io/projected/a731fa3e-457b-47ee-b553-4cd9fed175db-kube-api-access-l4vrx\") pod \"keda-operator-579c8958f7-tcj9g\" (UID: \"a731fa3e-457b-47ee-b553-4cd9fed175db\") " pod="openshift-keda/keda-operator-579c8958f7-tcj9g"
Sep 17 17:49:26.866372 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:49:26.866341    2575 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-keda/keda-metrics-apiserver-77857b7b98-hdsqb"]
Sep 17 17:49:26.868468 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:49:26.868430    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-keda/keda-metrics-apiserver-77857b7b98-hdsqb"
Sep 17 17:49:26.870945 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:49:26.870928    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-keda\"/\"keda-metrics-apiserver-certs\""
Sep 17 17:49:26.879245 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:49:26.879223    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-keda/keda-metrics-apiserver-77857b7b98-hdsqb"]
Sep 17 17:49:26.882414 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:49:26.882389    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-djvzt_1d07e68d-9943-420f-a6a0-5f477792fabd/konnectivity-agent/0.log" limit=1024
Sep 17 17:49:26.902528 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:49:26.902497    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"temp-vol\" (UniqueName: \"kubernetes.io/empty-dir/b8da88cf-4c60-4289-b8d2-a218944a78ce-temp-vol\") pod \"keda-metrics-apiserver-77857b7b98-hdsqb\" (UID: \"b8da88cf-4c60-4289-b8d2-a218944a78ce\") " pod="openshift-keda/keda-metrics-apiserver-77857b7b98-hdsqb"
Sep 17 17:49:26.902675 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:49:26.902552    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-x4cs2\" (UniqueName: \"kubernetes.io/projected/b8da88cf-4c60-4289-b8d2-a218944a78ce-kube-api-access-x4cs2\") pod \"keda-metrics-apiserver-77857b7b98-hdsqb\" (UID: \"b8da88cf-4c60-4289-b8d2-a218944a78ce\") " pod="openshift-keda/keda-metrics-apiserver-77857b7b98-hdsqb"
Sep 17 17:49:26.902734 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:49:26.902665    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"certificates\" (UniqueName: \"kubernetes.io/projected/b8da88cf-4c60-4289-b8d2-a218944a78ce-certificates\") pod \"keda-metrics-apiserver-77857b7b98-hdsqb\" (UID: \"b8da88cf-4c60-4289-b8d2-a218944a78ce\") " pod="openshift-keda/keda-metrics-apiserver-77857b7b98-hdsqb"
Sep 17 17:49:27.003330 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:49:27.003238    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"temp-vol\" (UniqueName: \"kubernetes.io/empty-dir/b8da88cf-4c60-4289-b8d2-a218944a78ce-temp-vol\") pod \"keda-metrics-apiserver-77857b7b98-hdsqb\" (UID: \"b8da88cf-4c60-4289-b8d2-a218944a78ce\") " pod="openshift-keda/keda-metrics-apiserver-77857b7b98-hdsqb"
Sep 17 17:49:27.003330 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:49:27.003277    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-x4cs2\" (UniqueName: \"kubernetes.io/projected/b8da88cf-4c60-4289-b8d2-a218944a78ce-kube-api-access-x4cs2\") pod \"keda-metrics-apiserver-77857b7b98-hdsqb\" (UID: \"b8da88cf-4c60-4289-b8d2-a218944a78ce\") " pod="openshift-keda/keda-metrics-apiserver-77857b7b98-hdsqb"
Sep 17 17:49:27.003330 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:49:27.003318    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"certificates\" (UniqueName: \"kubernetes.io/projected/b8da88cf-4c60-4289-b8d2-a218944a78ce-certificates\") pod \"keda-metrics-apiserver-77857b7b98-hdsqb\" (UID: \"b8da88cf-4c60-4289-b8d2-a218944a78ce\") " pod="openshift-keda/keda-metrics-apiserver-77857b7b98-hdsqb"
Sep 17 17:49:27.003629 ip-10-0-143-74 kubenswrapper[2575]: E0917 17:49:27.003415    2575 secret.go:281] references non-existent secret key: tls.crt
Sep 17 17:49:27.003629 ip-10-0-143-74 kubenswrapper[2575]: E0917 17:49:27.003432    2575 projected.go:279] Couldn't get secret payload openshift-keda/kedaorg-certs: references non-existent secret key: tls.crt
Sep 17 17:49:27.003629 ip-10-0-143-74 kubenswrapper[2575]: E0917 17:49:27.003472    2575 projected.go:266] Couldn't get secret openshift-keda/keda-metrics-apiserver-certs: secret "keda-metrics-apiserver-certs" not found
Sep 17 17:49:27.003629 ip-10-0-143-74 kubenswrapper[2575]: E0917 17:49:27.003493    2575 projected.go:196] Error preparing data for projected volume certificates for pod openshift-keda/keda-metrics-apiserver-77857b7b98-hdsqb: [references non-existent secret key: tls.crt, secret "keda-metrics-apiserver-certs" not found]
Sep 17 17:49:27.003629 ip-10-0-143-74 kubenswrapper[2575]: E0917 17:49:27.003558    2575 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/b8da88cf-4c60-4289-b8d2-a218944a78ce-certificates podName:b8da88cf-4c60-4289-b8d2-a218944a78ce nodeName:}" failed. No retries permitted until 2026-09-17 17:49:27.503540668 +0000 UTC m=+887.133870620 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "certificates" (UniqueName: "kubernetes.io/projected/b8da88cf-4c60-4289-b8d2-a218944a78ce-certificates") pod "keda-metrics-apiserver-77857b7b98-hdsqb" (UID: "b8da88cf-4c60-4289-b8d2-a218944a78ce") : [references non-existent secret key: tls.crt, secret "keda-metrics-apiserver-certs" not found]
Sep 17 17:49:27.003890 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:49:27.003749    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"temp-vol\" (UniqueName: \"kubernetes.io/empty-dir/b8da88cf-4c60-4289-b8d2-a218944a78ce-temp-vol\") pod \"keda-metrics-apiserver-77857b7b98-hdsqb\" (UID: \"b8da88cf-4c60-4289-b8d2-a218944a78ce\") " pod="openshift-keda/keda-metrics-apiserver-77857b7b98-hdsqb"
Sep 17 17:49:27.013810 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:49:27.013783    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-x4cs2\" (UniqueName: \"kubernetes.io/projected/b8da88cf-4c60-4289-b8d2-a218944a78ce-kube-api-access-x4cs2\") pod \"keda-metrics-apiserver-77857b7b98-hdsqb\" (UID: \"b8da88cf-4c60-4289-b8d2-a218944a78ce\") " pod="openshift-keda/keda-metrics-apiserver-77857b7b98-hdsqb"
Sep 17 17:49:27.041765 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:49:27.041730    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-djvzt_1d07e68d-9943-420f-a6a0-5f477792fabd/konnectivity-agent/0.log" limit=1024
Sep 17 17:49:27.199318 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:49:27.199281    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-djvzt_1d07e68d-9943-420f-a6a0-5f477792fabd/konnectivity-agent/0.log" limit=1024
Sep 17 17:49:27.306963 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:49:27.306871    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"certificates\" (UniqueName: \"kubernetes.io/projected/a731fa3e-457b-47ee-b553-4cd9fed175db-certificates\") pod \"keda-operator-579c8958f7-tcj9g\" (UID: \"a731fa3e-457b-47ee-b553-4cd9fed175db\") " pod="openshift-keda/keda-operator-579c8958f7-tcj9g"
Sep 17 17:49:27.307127 ip-10-0-143-74 kubenswrapper[2575]: E0917 17:49:27.307023    2575 secret.go:281] references non-existent secret key: ca.crt
Sep 17 17:49:27.307127 ip-10-0-143-74 kubenswrapper[2575]: E0917 17:49:27.307042    2575 projected.go:279] Couldn't get secret payload openshift-keda/kedaorg-certs: references non-existent secret key: ca.crt
Sep 17 17:49:27.307127 ip-10-0-143-74 kubenswrapper[2575]: E0917 17:49:27.307055    2575 projected.go:196] Error preparing data for projected volume certificates for pod openshift-keda/keda-operator-579c8958f7-tcj9g: references non-existent secret key: ca.crt
Sep 17 17:49:27.307127 ip-10-0-143-74 kubenswrapper[2575]: E0917 17:49:27.307118    2575 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/a731fa3e-457b-47ee-b553-4cd9fed175db-certificates podName:a731fa3e-457b-47ee-b553-4cd9fed175db nodeName:}" failed. No retries permitted until 2026-09-17 17:49:28.307100555 +0000 UTC m=+887.937430524 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "certificates" (UniqueName: "kubernetes.io/projected/a731fa3e-457b-47ee-b553-4cd9fed175db-certificates") pod "keda-operator-579c8958f7-tcj9g" (UID: "a731fa3e-457b-47ee-b553-4cd9fed175db") : references non-existent secret key: ca.crt
Sep 17 17:49:27.509550 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:49:27.509508    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"certificates\" (UniqueName: \"kubernetes.io/projected/b8da88cf-4c60-4289-b8d2-a218944a78ce-certificates\") pod \"keda-metrics-apiserver-77857b7b98-hdsqb\" (UID: \"b8da88cf-4c60-4289-b8d2-a218944a78ce\") " pod="openshift-keda/keda-metrics-apiserver-77857b7b98-hdsqb"
Sep 17 17:49:27.509951 ip-10-0-143-74 kubenswrapper[2575]: E0917 17:49:27.509672    2575 secret.go:281] references non-existent secret key: tls.crt
Sep 17 17:49:27.509951 ip-10-0-143-74 kubenswrapper[2575]: E0917 17:49:27.509693    2575 projected.go:279] Couldn't get secret payload openshift-keda/kedaorg-certs: references non-existent secret key: tls.crt
Sep 17 17:49:27.509951 ip-10-0-143-74 kubenswrapper[2575]: E0917 17:49:27.509712    2575 projected.go:196] Error preparing data for projected volume certificates for pod openshift-keda/keda-metrics-apiserver-77857b7b98-hdsqb: references non-existent secret key: tls.crt
Sep 17 17:49:27.509951 ip-10-0-143-74 kubenswrapper[2575]: E0917 17:49:27.509776    2575 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/b8da88cf-4c60-4289-b8d2-a218944a78ce-certificates podName:b8da88cf-4c60-4289-b8d2-a218944a78ce nodeName:}" failed. No retries permitted until 2026-09-17 17:49:28.509760802 +0000 UTC m=+888.140090755 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "certificates" (UniqueName: "kubernetes.io/projected/b8da88cf-4c60-4289-b8d2-a218944a78ce-certificates") pod "keda-metrics-apiserver-77857b7b98-hdsqb" (UID: "b8da88cf-4c60-4289-b8d2-a218944a78ce") : references non-existent secret key: tls.crt
Sep 17 17:49:27.661778 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:49:27.661743    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-djvzt_1d07e68d-9943-420f-a6a0-5f477792fabd/konnectivity-agent/0.log" limit=1024
Sep 17 17:49:27.792379 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:49:27.792345    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-djvzt_1d07e68d-9943-420f-a6a0-5f477792fabd/konnectivity-agent/0.log" limit=1024
Sep 17 17:49:28.035318 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:49:28.035229    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-djvzt_1d07e68d-9943-420f-a6a0-5f477792fabd/konnectivity-agent/0.log" limit=1024
Sep 17 17:49:28.246762 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:49:28.246734    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-djvzt_1d07e68d-9943-420f-a6a0-5f477792fabd/konnectivity-agent/0.log" limit=1024
Sep 17 17:49:28.314715 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:49:28.314635    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"certificates\" (UniqueName: \"kubernetes.io/projected/a731fa3e-457b-47ee-b553-4cd9fed175db-certificates\") pod \"keda-operator-579c8958f7-tcj9g\" (UID: \"a731fa3e-457b-47ee-b553-4cd9fed175db\") " pod="openshift-keda/keda-operator-579c8958f7-tcj9g"
Sep 17 17:49:28.316976 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:49:28.316945    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"certificates\" (UniqueName: \"kubernetes.io/projected/a731fa3e-457b-47ee-b553-4cd9fed175db-certificates\") pod \"keda-operator-579c8958f7-tcj9g\" (UID: \"a731fa3e-457b-47ee-b553-4cd9fed175db\") " pod="openshift-keda/keda-operator-579c8958f7-tcj9g"
Sep 17 17:49:28.445475 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:49:28.445423    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-djvzt_1d07e68d-9943-420f-a6a0-5f477792fabd/konnectivity-agent/0.log" limit=1024
Sep 17 17:49:28.487880 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:49:28.487845    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-keda/keda-operator-579c8958f7-tcj9g"
Sep 17 17:49:28.517195 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:49:28.517160    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"certificates\" (UniqueName: \"kubernetes.io/projected/b8da88cf-4c60-4289-b8d2-a218944a78ce-certificates\") pod \"keda-metrics-apiserver-77857b7b98-hdsqb\" (UID: \"b8da88cf-4c60-4289-b8d2-a218944a78ce\") " pod="openshift-keda/keda-metrics-apiserver-77857b7b98-hdsqb"
Sep 17 17:49:28.519731 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:49:28.519698    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"certificates\" (UniqueName: \"kubernetes.io/projected/b8da88cf-4c60-4289-b8d2-a218944a78ce-certificates\") pod \"keda-metrics-apiserver-77857b7b98-hdsqb\" (UID: \"b8da88cf-4c60-4289-b8d2-a218944a78ce\") " pod="openshift-keda/keda-metrics-apiserver-77857b7b98-hdsqb"
Sep 17 17:49:28.561049 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:49:28.561011    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-djvzt_1d07e68d-9943-420f-a6a0-5f477792fabd/konnectivity-agent/0.log" limit=1024
Sep 17 17:49:28.679711 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:49:28.679676    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-keda/keda-metrics-apiserver-77857b7b98-hdsqb"
Sep 17 17:49:28.758110 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:49:28.758082    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-keda/keda-operator-579c8958f7-tcj9g"]
Sep 17 17:49:28.761376 ip-10-0-143-74 kubenswrapper[2575]: W0917 17:49:28.761352    2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda731fa3e_457b_47ee_b553_4cd9fed175db.slice/crio-a5cbabc0cdb8be14191e4ec8d222d5d1a781ac5eb3aaabec77659d86b79c13c6 WatchSource:0}: Error finding container a5cbabc0cdb8be14191e4ec8d222d5d1a781ac5eb3aaabec77659d86b79c13c6: Status 404 returned error can't find the container with id a5cbabc0cdb8be14191e4ec8d222d5d1a781ac5eb3aaabec77659d86b79c13c6
Sep 17 17:49:28.951263 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:49:28.951240    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-keda/keda-metrics-apiserver-77857b7b98-hdsqb"]
Sep 17 17:49:28.953589 ip-10-0-143-74 kubenswrapper[2575]: W0917 17:49:28.953564    2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb8da88cf_4c60_4289_b8d2_a218944a78ce.slice/crio-ac36d85eb4236f1b0c698a354367dd36f73ee40a9e660bb115f2df9450b33a7f WatchSource:0}: Error finding container ac36d85eb4236f1b0c698a354367dd36f73ee40a9e660bb115f2df9450b33a7f: Status 404 returned error can't find the container with id ac36d85eb4236f1b0c698a354367dd36f73ee40a9e660bb115f2df9450b33a7f
Sep 17 17:49:29.629973 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:49:29.629897    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-keda/keda-metrics-apiserver-77857b7b98-hdsqb" event={"ID":"b8da88cf-4c60-4289-b8d2-a218944a78ce","Type":"ContainerStarted","Data":"ac36d85eb4236f1b0c698a354367dd36f73ee40a9e660bb115f2df9450b33a7f"}
Sep 17 17:49:29.631436 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:49:29.631404    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-keda/keda-operator-579c8958f7-tcj9g" event={"ID":"a731fa3e-457b-47ee-b553-4cd9fed175db","Type":"ContainerStarted","Data":"a5cbabc0cdb8be14191e4ec8d222d5d1a781ac5eb3aaabec77659d86b79c13c6"}
Sep 17 17:49:33.654720 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:49:33.654633    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-keda/keda-metrics-apiserver-77857b7b98-hdsqb" event={"ID":"b8da88cf-4c60-4289-b8d2-a218944a78ce","Type":"ContainerStarted","Data":"90589a5c032c2201c4f380a26e422c4e3b04b7a2f44bf6b8dde43df468ae8d64"}
Sep 17 17:49:33.655178 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:49:33.654749    2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-keda/keda-metrics-apiserver-77857b7b98-hdsqb"
Sep 17 17:49:33.656568 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:49:33.656542    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-keda/keda-operator-579c8958f7-tcj9g" event={"ID":"a731fa3e-457b-47ee-b553-4cd9fed175db","Type":"ContainerStarted","Data":"fb2ce6777fe33ca5423696cb1d4434c93898a195e4e131739816fedc4575e51f"}
Sep 17 17:49:33.656688 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:49:33.656641    2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-keda/keda-operator-579c8958f7-tcj9g"
Sep 17 17:49:33.674981 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:49:33.674935    2575 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-keda/keda-metrics-apiserver-77857b7b98-hdsqb" podStartSLOduration=4.042814249 podStartE2EDuration="7.674920625s" totalImagesPullingTime="3.632106376s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-17 17:49:26 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-17 17:49:33.673345685 +0000 UTC m=+893.303675672" watchObservedRunningTime="2026-09-17 17:49:33.674920625 +0000 UTC m=+893.305250598"
Sep 17 17:49:33.695666 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:49:33.695620    2575 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-keda/keda-operator-579c8958f7-tcj9g" podStartSLOduration=3.868600873 podStartE2EDuration="7.695599751s" totalImagesPullingTime="3.826998878s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-17 17:49:26 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-17 17:49:33.694323094 +0000 UTC m=+893.324653069" watchObservedRunningTime="2026-09-17 17:49:33.695599751 +0000 UTC m=+893.325929725"
Sep 17 17:49:44.665596 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:49:44.665567    2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-keda/keda-metrics-apiserver-77857b7b98-hdsqb"
Sep 17 17:49:54.663819 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:49:54.663784    2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-keda/keda-operator-579c8958f7-tcj9g"
Sep 17 17:50:00.882787 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:50:00.882752    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-djvzt_1d07e68d-9943-420f-a6a0-5f477792fabd/konnectivity-agent/0.log" limit=1024
Sep 17 17:50:06.060525 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:50:06.060484    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-djvzt_1d07e68d-9943-420f-a6a0-5f477792fabd/konnectivity-agent/0.log" limit=1024
Sep 17 17:50:12.193755 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:50:12.193721    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-djvzt_1d07e68d-9943-420f-a6a0-5f477792fabd/konnectivity-agent/0.log" limit=1024
Sep 17 17:50:12.304705 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:50:12.304667    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-djvzt_1d07e68d-9943-420f-a6a0-5f477792fabd/konnectivity-agent/0.log" limit=1024
Sep 17 17:50:17.171494 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:50:17.171463    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-djvzt_1d07e68d-9943-420f-a6a0-5f477792fabd/konnectivity-agent/0.log" limit=1024
Sep 17 17:50:17.320800 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:50:17.320768    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-djvzt_1d07e68d-9943-420f-a6a0-5f477792fabd/konnectivity-agent/0.log" limit=1024
Sep 17 17:50:17.620960 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:50:17.620930    2575 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e675406vc7k"]
Sep 17 17:50:17.623203 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:50:17.623187    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e675406vc7k"
Sep 17 17:50:17.626142 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:50:17.626120    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-marketplace\"/\"default-dockercfg-p47rs\""
Sep 17 17:50:17.626142 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:50:17.626119    2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"openshift-service-ca.crt\""
Sep 17 17:50:17.627009 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:50:17.626984    2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"kube-root-ca.crt\""
Sep 17 17:50:17.631805 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:50:17.631785    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/57d5a33b-43ee-4ea9-aaa8-aa5fb3861ed7-util\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e675406vc7k\" (UID: \"57d5a33b-43ee-4ea9-aaa8-aa5fb3861ed7\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e675406vc7k"
Sep 17 17:50:17.631873 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:50:17.631827    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-z9dsj\" (UniqueName: \"kubernetes.io/projected/57d5a33b-43ee-4ea9-aaa8-aa5fb3861ed7-kube-api-access-z9dsj\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e675406vc7k\" (UID: \"57d5a33b-43ee-4ea9-aaa8-aa5fb3861ed7\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e675406vc7k"
Sep 17 17:50:17.631873 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:50:17.631856    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/57d5a33b-43ee-4ea9-aaa8-aa5fb3861ed7-bundle\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e675406vc7k\" (UID: \"57d5a33b-43ee-4ea9-aaa8-aa5fb3861ed7\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e675406vc7k"
Sep 17 17:50:17.642977 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:50:17.642958    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e675406vc7k"]
Sep 17 17:50:17.732540 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:50:17.732506    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/57d5a33b-43ee-4ea9-aaa8-aa5fb3861ed7-util\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e675406vc7k\" (UID: \"57d5a33b-43ee-4ea9-aaa8-aa5fb3861ed7\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e675406vc7k"
Sep 17 17:50:17.732713 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:50:17.732560    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-z9dsj\" (UniqueName: \"kubernetes.io/projected/57d5a33b-43ee-4ea9-aaa8-aa5fb3861ed7-kube-api-access-z9dsj\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e675406vc7k\" (UID: \"57d5a33b-43ee-4ea9-aaa8-aa5fb3861ed7\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e675406vc7k"
Sep 17 17:50:17.732713 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:50:17.732676    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/57d5a33b-43ee-4ea9-aaa8-aa5fb3861ed7-bundle\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e675406vc7k\" (UID: \"57d5a33b-43ee-4ea9-aaa8-aa5fb3861ed7\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e675406vc7k"
Sep 17 17:50:17.732939 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:50:17.732923    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/57d5a33b-43ee-4ea9-aaa8-aa5fb3861ed7-util\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e675406vc7k\" (UID: \"57d5a33b-43ee-4ea9-aaa8-aa5fb3861ed7\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e675406vc7k"
Sep 17 17:50:17.732995 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:50:17.732978    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/57d5a33b-43ee-4ea9-aaa8-aa5fb3861ed7-bundle\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e675406vc7k\" (UID: \"57d5a33b-43ee-4ea9-aaa8-aa5fb3861ed7\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e675406vc7k"
Sep 17 17:50:17.742662 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:50:17.742633    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-z9dsj\" (UniqueName: \"kubernetes.io/projected/57d5a33b-43ee-4ea9-aaa8-aa5fb3861ed7-kube-api-access-z9dsj\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e675406vc7k\" (UID: \"57d5a33b-43ee-4ea9-aaa8-aa5fb3861ed7\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e675406vc7k"
Sep 17 17:50:17.932697 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:50:17.932611    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e675406vc7k"
Sep 17 17:50:18.215831 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:50:18.215760    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e675406vc7k"]
Sep 17 17:50:18.219363 ip-10-0-143-74 kubenswrapper[2575]: W0917 17:50:18.219335    2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod57d5a33b_43ee_4ea9_aaa8_aa5fb3861ed7.slice/crio-125e33a0904bebd1a8f64e31cde86e1f27d35165b835aaa8e484c0de877f56be WatchSource:0}: Error finding container 125e33a0904bebd1a8f64e31cde86e1f27d35165b835aaa8e484c0de877f56be: Status 404 returned error can't find the container with id 125e33a0904bebd1a8f64e31cde86e1f27d35165b835aaa8e484c0de877f56be
Sep 17 17:50:18.849373 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:50:18.849341    2575 generic.go:356] "Generic (PLEG): container finished" podID="57d5a33b-43ee-4ea9-aaa8-aa5fb3861ed7" containerID="c3815e350bca9a54bd43d337d52109d81a850f98aefd033ed03cb2bb801bcf1e" exitCode=0
Sep 17 17:50:18.849558 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:50:18.849432    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e675406vc7k" event={"ID":"57d5a33b-43ee-4ea9-aaa8-aa5fb3861ed7","Type":"ContainerDied","Data":"c3815e350bca9a54bd43d337d52109d81a850f98aefd033ed03cb2bb801bcf1e"}
Sep 17 17:50:18.849558 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:50:18.849493    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e675406vc7k" event={"ID":"57d5a33b-43ee-4ea9-aaa8-aa5fb3861ed7","Type":"ContainerStarted","Data":"125e33a0904bebd1a8f64e31cde86e1f27d35165b835aaa8e484c0de877f56be"}
Sep 17 17:50:20.860087 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:50:20.860056    2575 generic.go:356] "Generic (PLEG): container finished" podID="57d5a33b-43ee-4ea9-aaa8-aa5fb3861ed7" containerID="372969a63403eb4399d1adebb7efb07e602e0fdb3fc2b38f06dc746a7c8d7bca" exitCode=0
Sep 17 17:50:20.860391 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:50:20.860123    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e675406vc7k" event={"ID":"57d5a33b-43ee-4ea9-aaa8-aa5fb3861ed7","Type":"ContainerDied","Data":"372969a63403eb4399d1adebb7efb07e602e0fdb3fc2b38f06dc746a7c8d7bca"}
Sep 17 17:50:21.866797 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:50:21.866768    2575 generic.go:356] "Generic (PLEG): container finished" podID="57d5a33b-43ee-4ea9-aaa8-aa5fb3861ed7" containerID="e0cd5f92e7846c26987299b4dd9bba94aa5a92376dfdf6f34f49ffeaf20118a7" exitCode=0
Sep 17 17:50:21.867195 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:50:21.866817    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e675406vc7k" event={"ID":"57d5a33b-43ee-4ea9-aaa8-aa5fb3861ed7","Type":"ContainerDied","Data":"e0cd5f92e7846c26987299b4dd9bba94aa5a92376dfdf6f34f49ffeaf20118a7"}
Sep 17 17:50:22.995375 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:50:22.995353    2575 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e675406vc7k"
Sep 17 17:50:23.074981 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:50:23.074955    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/57d5a33b-43ee-4ea9-aaa8-aa5fb3861ed7-util\") pod \"57d5a33b-43ee-4ea9-aaa8-aa5fb3861ed7\" (UID: \"57d5a33b-43ee-4ea9-aaa8-aa5fb3861ed7\") "
Sep 17 17:50:23.075093 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:50:23.075006    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-z9dsj\" (UniqueName: \"kubernetes.io/projected/57d5a33b-43ee-4ea9-aaa8-aa5fb3861ed7-kube-api-access-z9dsj\") pod \"57d5a33b-43ee-4ea9-aaa8-aa5fb3861ed7\" (UID: \"57d5a33b-43ee-4ea9-aaa8-aa5fb3861ed7\") "
Sep 17 17:50:23.075093 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:50:23.075029    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/57d5a33b-43ee-4ea9-aaa8-aa5fb3861ed7-bundle\") pod \"57d5a33b-43ee-4ea9-aaa8-aa5fb3861ed7\" (UID: \"57d5a33b-43ee-4ea9-aaa8-aa5fb3861ed7\") "
Sep 17 17:50:23.075931 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:50:23.075898    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/57d5a33b-43ee-4ea9-aaa8-aa5fb3861ed7-bundle" (OuterVolumeSpecName: "bundle") pod "57d5a33b-43ee-4ea9-aaa8-aa5fb3861ed7" (UID: "57d5a33b-43ee-4ea9-aaa8-aa5fb3861ed7"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 17 17:50:23.077116 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:50:23.077095    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/57d5a33b-43ee-4ea9-aaa8-aa5fb3861ed7-kube-api-access-z9dsj" (OuterVolumeSpecName: "kube-api-access-z9dsj") pod "57d5a33b-43ee-4ea9-aaa8-aa5fb3861ed7" (UID: "57d5a33b-43ee-4ea9-aaa8-aa5fb3861ed7"). InnerVolumeSpecName "kube-api-access-z9dsj". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 17 17:50:23.080491 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:50:23.080474    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/57d5a33b-43ee-4ea9-aaa8-aa5fb3861ed7-util" (OuterVolumeSpecName: "util") pod "57d5a33b-43ee-4ea9-aaa8-aa5fb3861ed7" (UID: "57d5a33b-43ee-4ea9-aaa8-aa5fb3861ed7"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 17 17:50:23.175776 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:50:23.175720    2575 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/57d5a33b-43ee-4ea9-aaa8-aa5fb3861ed7-bundle\") on node \"ip-10-0-143-74.ec2.internal\" DevicePath \"\""
Sep 17 17:50:23.175776 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:50:23.175741    2575 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/57d5a33b-43ee-4ea9-aaa8-aa5fb3861ed7-util\") on node \"ip-10-0-143-74.ec2.internal\" DevicePath \"\""
Sep 17 17:50:23.175776 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:50:23.175750    2575 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-z9dsj\" (UniqueName: \"kubernetes.io/projected/57d5a33b-43ee-4ea9-aaa8-aa5fb3861ed7-kube-api-access-z9dsj\") on node \"ip-10-0-143-74.ec2.internal\" DevicePath \"\""
Sep 17 17:50:23.877433 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:50:23.877403    2575 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e675406vc7k"
Sep 17 17:50:23.877433 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:50:23.877414    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e675406vc7k" event={"ID":"57d5a33b-43ee-4ea9-aaa8-aa5fb3861ed7","Type":"ContainerDied","Data":"125e33a0904bebd1a8f64e31cde86e1f27d35165b835aaa8e484c0de877f56be"}
Sep 17 17:50:23.877638 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:50:23.877504    2575 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="125e33a0904bebd1a8f64e31cde86e1f27d35165b835aaa8e484c0de877f56be"
Sep 17 17:50:29.412864 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:50:29.412829    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-djvzt_1d07e68d-9943-420f-a6a0-5f477792fabd/konnectivity-agent/0.log" limit=1024
Sep 17 17:50:29.789758 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:50:29.789669    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-djvzt_1d07e68d-9943-420f-a6a0-5f477792fabd/konnectivity-agent/0.log" limit=1024
Sep 17 17:50:31.866923 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:50:31.866892    2575 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["cert-manager-operator/cert-manager-operator-controller-manager-684f79c9b9-l8vcw"]
Sep 17 17:50:31.867377 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:50:31.867336    2575 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="57d5a33b-43ee-4ea9-aaa8-aa5fb3861ed7" containerName="extract"
Sep 17 17:50:31.867377 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:50:31.867353    2575 state_mem.go:107] "Deleted CPUSet assignment" podUID="57d5a33b-43ee-4ea9-aaa8-aa5fb3861ed7" containerName="extract"
Sep 17 17:50:31.867377 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:50:31.867373    2575 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="57d5a33b-43ee-4ea9-aaa8-aa5fb3861ed7" containerName="util"
Sep 17 17:50:31.867578 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:50:31.867381    2575 state_mem.go:107] "Deleted CPUSet assignment" podUID="57d5a33b-43ee-4ea9-aaa8-aa5fb3861ed7" containerName="util"
Sep 17 17:50:31.867578 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:50:31.867407    2575 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="57d5a33b-43ee-4ea9-aaa8-aa5fb3861ed7" containerName="pull"
Sep 17 17:50:31.867578 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:50:31.867416    2575 state_mem.go:107] "Deleted CPUSet assignment" podUID="57d5a33b-43ee-4ea9-aaa8-aa5fb3861ed7" containerName="pull"
Sep 17 17:50:31.867578 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:50:31.867523    2575 memory_manager.go:368] "RemoveStaleState removing state" podUID="57d5a33b-43ee-4ea9-aaa8-aa5fb3861ed7" containerName="extract"
Sep 17 17:50:31.870730 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:50:31.870709    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager-operator/cert-manager-operator-controller-manager-684f79c9b9-l8vcw"
Sep 17 17:50:31.873221 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:50:31.873202    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"cert-manager-operator\"/\"cert-manager-operator-controller-manager-dockercfg-jqw8z\""
Sep 17 17:50:31.873381 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:50:31.873223    2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager-operator\"/\"kube-root-ca.crt\""
Sep 17 17:50:31.873381 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:50:31.873207    2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager-operator\"/\"openshift-service-ca.crt\""
Sep 17 17:50:31.890322 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:50:31.890292    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager-operator/cert-manager-operator-controller-manager-684f79c9b9-l8vcw"]
Sep 17 17:50:31.946618 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:50:31.946587    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/a86e0cc5-7bc6-40a2-93d1-a3404b87243c-tmp\") pod \"cert-manager-operator-controller-manager-684f79c9b9-l8vcw\" (UID: \"a86e0cc5-7bc6-40a2-93d1-a3404b87243c\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-684f79c9b9-l8vcw"
Sep 17 17:50:31.946747 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:50:31.946633    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hzx5m\" (UniqueName: \"kubernetes.io/projected/a86e0cc5-7bc6-40a2-93d1-a3404b87243c-kube-api-access-hzx5m\") pod \"cert-manager-operator-controller-manager-684f79c9b9-l8vcw\" (UID: \"a86e0cc5-7bc6-40a2-93d1-a3404b87243c\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-684f79c9b9-l8vcw"
Sep 17 17:50:32.047931 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:50:32.047893    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/a86e0cc5-7bc6-40a2-93d1-a3404b87243c-tmp\") pod \"cert-manager-operator-controller-manager-684f79c9b9-l8vcw\" (UID: \"a86e0cc5-7bc6-40a2-93d1-a3404b87243c\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-684f79c9b9-l8vcw"
Sep 17 17:50:32.048024 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:50:32.047945    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hzx5m\" (UniqueName: \"kubernetes.io/projected/a86e0cc5-7bc6-40a2-93d1-a3404b87243c-kube-api-access-hzx5m\") pod \"cert-manager-operator-controller-manager-684f79c9b9-l8vcw\" (UID: \"a86e0cc5-7bc6-40a2-93d1-a3404b87243c\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-684f79c9b9-l8vcw"
Sep 17 17:50:32.048269 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:50:32.048251    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/a86e0cc5-7bc6-40a2-93d1-a3404b87243c-tmp\") pod \"cert-manager-operator-controller-manager-684f79c9b9-l8vcw\" (UID: \"a86e0cc5-7bc6-40a2-93d1-a3404b87243c\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-684f79c9b9-l8vcw"
Sep 17 17:50:32.056903 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:50:32.056875    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-hzx5m\" (UniqueName: \"kubernetes.io/projected/a86e0cc5-7bc6-40a2-93d1-a3404b87243c-kube-api-access-hzx5m\") pod \"cert-manager-operator-controller-manager-684f79c9b9-l8vcw\" (UID: \"a86e0cc5-7bc6-40a2-93d1-a3404b87243c\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-684f79c9b9-l8vcw"
Sep 17 17:50:32.179975 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:50:32.179912    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager-operator/cert-manager-operator-controller-manager-684f79c9b9-l8vcw"
Sep 17 17:50:32.463134 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:50:32.463112    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager-operator/cert-manager-operator-controller-manager-684f79c9b9-l8vcw"]
Sep 17 17:50:32.465193 ip-10-0-143-74 kubenswrapper[2575]: W0917 17:50:32.465165    2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda86e0cc5_7bc6_40a2_93d1_a3404b87243c.slice/crio-f4af4ef928c027f158b8d20cf0fe30d5364af58db6ce9f263706a3c26269cb7c WatchSource:0}: Error finding container f4af4ef928c027f158b8d20cf0fe30d5364af58db6ce9f263706a3c26269cb7c: Status 404 returned error can't find the container with id f4af4ef928c027f158b8d20cf0fe30d5364af58db6ce9f263706a3c26269cb7c
Sep 17 17:50:32.915415 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:50:32.915378    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager-operator/cert-manager-operator-controller-manager-684f79c9b9-l8vcw" event={"ID":"a86e0cc5-7bc6-40a2-93d1-a3404b87243c","Type":"ContainerStarted","Data":"f4af4ef928c027f158b8d20cf0fe30d5364af58db6ce9f263706a3c26269cb7c"}
Sep 17 17:50:33.409044 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:50:33.409013    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-djvzt_1d07e68d-9943-420f-a6a0-5f477792fabd/konnectivity-agent/0.log" limit=1024
Sep 17 17:50:33.544796 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:50:33.544753    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-djvzt_1d07e68d-9943-420f-a6a0-5f477792fabd/konnectivity-agent/0.log" limit=1024
Sep 17 17:50:35.930684 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:50:35.930647    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager-operator/cert-manager-operator-controller-manager-684f79c9b9-l8vcw" event={"ID":"a86e0cc5-7bc6-40a2-93d1-a3404b87243c","Type":"ContainerStarted","Data":"a7e3aef6c139431f1d5e06ccf0eaff0b954b22f90c2d6e6bf58bb20bea225ff7"}
Sep 17 17:50:35.960866 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:50:35.960813    2575 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="cert-manager-operator/cert-manager-operator-controller-manager-684f79c9b9-l8vcw" podStartSLOduration=1.9525679139999998 podStartE2EDuration="4.960795484s" totalImagesPullingTime="3.00822757s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-17 17:50:31 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-17 17:50:35.959022755 +0000 UTC m=+955.589352729" watchObservedRunningTime="2026-09-17 17:50:35.960795484 +0000 UTC m=+955.591125459"
Sep 17 17:50:36.317120 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:50:36.317015    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-djvzt_1d07e68d-9943-420f-a6a0-5f477792fabd/konnectivity-agent/0.log" limit=1024
Sep 17 17:50:36.565350 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:50:36.565320    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-djvzt_1d07e68d-9943-420f-a6a0-5f477792fabd/konnectivity-agent/0.log" limit=1024
Sep 17 17:50:38.655348 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:50:38.655298    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-djvzt_1d07e68d-9943-420f-a6a0-5f477792fabd/konnectivity-agent/0.log" limit=1024
Sep 17 17:50:38.815225 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:50:38.815191    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-djvzt_1d07e68d-9943-420f-a6a0-5f477792fabd/konnectivity-agent/0.log" limit=1024
Sep 17 17:50:38.979571 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:50:38.979497    2575 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f2w9fv"]
Sep 17 17:50:38.982776 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:50:38.982760    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f2w9fv"
Sep 17 17:50:38.985552 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:50:38.985530    2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"openshift-service-ca.crt\""
Sep 17 17:50:38.985651 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:50:38.985530    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-marketplace\"/\"default-dockercfg-p47rs\""
Sep 17 17:50:38.986219 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:50:38.986204    2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"kube-root-ca.crt\""
Sep 17 17:50:38.993362 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:50:38.993340    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f2w9fv"]
Sep 17 17:50:39.107968 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:50:39.107929    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/1f35cbc3-81fa-479f-8076-371dc8ecdc37-util\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f2w9fv\" (UID: \"1f35cbc3-81fa-479f-8076-371dc8ecdc37\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f2w9fv"
Sep 17 17:50:39.108136 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:50:39.108002    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/1f35cbc3-81fa-479f-8076-371dc8ecdc37-bundle\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f2w9fv\" (UID: \"1f35cbc3-81fa-479f-8076-371dc8ecdc37\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f2w9fv"
Sep 17 17:50:39.108136 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:50:39.108035    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-r6gzf\" (UniqueName: \"kubernetes.io/projected/1f35cbc3-81fa-479f-8076-371dc8ecdc37-kube-api-access-r6gzf\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f2w9fv\" (UID: \"1f35cbc3-81fa-479f-8076-371dc8ecdc37\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f2w9fv"
Sep 17 17:50:39.208683 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:50:39.208647    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/1f35cbc3-81fa-479f-8076-371dc8ecdc37-bundle\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f2w9fv\" (UID: \"1f35cbc3-81fa-479f-8076-371dc8ecdc37\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f2w9fv"
Sep 17 17:50:39.208884 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:50:39.208705    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-r6gzf\" (UniqueName: \"kubernetes.io/projected/1f35cbc3-81fa-479f-8076-371dc8ecdc37-kube-api-access-r6gzf\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f2w9fv\" (UID: \"1f35cbc3-81fa-479f-8076-371dc8ecdc37\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f2w9fv"
Sep 17 17:50:39.208884 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:50:39.208761    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/1f35cbc3-81fa-479f-8076-371dc8ecdc37-util\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f2w9fv\" (UID: \"1f35cbc3-81fa-479f-8076-371dc8ecdc37\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f2w9fv"
Sep 17 17:50:39.209137 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:50:39.209112    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/1f35cbc3-81fa-479f-8076-371dc8ecdc37-bundle\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f2w9fv\" (UID: \"1f35cbc3-81fa-479f-8076-371dc8ecdc37\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f2w9fv"
Sep 17 17:50:39.209210 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:50:39.209124    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/1f35cbc3-81fa-479f-8076-371dc8ecdc37-util\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f2w9fv\" (UID: \"1f35cbc3-81fa-479f-8076-371dc8ecdc37\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f2w9fv"
Sep 17 17:50:39.210363 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:50:39.210329    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-djvzt_1d07e68d-9943-420f-a6a0-5f477792fabd/konnectivity-agent/0.log" limit=1024
Sep 17 17:50:39.223372 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:50:39.223355    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-r6gzf\" (UniqueName: \"kubernetes.io/projected/1f35cbc3-81fa-479f-8076-371dc8ecdc37-kube-api-access-r6gzf\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f2w9fv\" (UID: \"1f35cbc3-81fa-479f-8076-371dc8ecdc37\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f2w9fv"
Sep 17 17:50:39.291957 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:50:39.291909    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f2w9fv"
Sep 17 17:50:39.567564 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:50:39.567473    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f2w9fv"]
Sep 17 17:50:39.570153 ip-10-0-143-74 kubenswrapper[2575]: W0917 17:50:39.570124    2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1f35cbc3_81fa_479f_8076_371dc8ecdc37.slice/crio-3b161532ea79b5d8e1b260bece285cb6d044c5bc53cfceb2e1d2f761d523ea55 WatchSource:0}: Error finding container 3b161532ea79b5d8e1b260bece285cb6d044c5bc53cfceb2e1d2f761d523ea55: Status 404 returned error can't find the container with id 3b161532ea79b5d8e1b260bece285cb6d044c5bc53cfceb2e1d2f761d523ea55
Sep 17 17:50:39.950835 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:50:39.950802    2575 generic.go:356] "Generic (PLEG): container finished" podID="1f35cbc3-81fa-479f-8076-371dc8ecdc37" containerID="700f739a9849da918e3cb61b5b22cd8b0a2c811461118371a9d123de9b8eb981" exitCode=0
Sep 17 17:50:39.951307 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:50:39.950853    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f2w9fv" event={"ID":"1f35cbc3-81fa-479f-8076-371dc8ecdc37","Type":"ContainerDied","Data":"700f739a9849da918e3cb61b5b22cd8b0a2c811461118371a9d123de9b8eb981"}
Sep 17 17:50:39.951307 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:50:39.950887    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f2w9fv" event={"ID":"1f35cbc3-81fa-479f-8076-371dc8ecdc37","Type":"ContainerStarted","Data":"3b161532ea79b5d8e1b260bece285cb6d044c5bc53cfceb2e1d2f761d523ea55"}
Sep 17 17:50:42.966266 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:50:42.966233    2575 generic.go:356] "Generic (PLEG): container finished" podID="1f35cbc3-81fa-479f-8076-371dc8ecdc37" containerID="49c1e1459ea66d07f1a8eb8e0d4e59418353d205c832fede8dd863c0a33d795c" exitCode=0
Sep 17 17:50:42.967362 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:50:42.967331    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f2w9fv" event={"ID":"1f35cbc3-81fa-479f-8076-371dc8ecdc37","Type":"ContainerDied","Data":"49c1e1459ea66d07f1a8eb8e0d4e59418353d205c832fede8dd863c0a33d795c"}
Sep 17 17:50:43.972828 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:50:43.972794    2575 generic.go:356] "Generic (PLEG): container finished" podID="1f35cbc3-81fa-479f-8076-371dc8ecdc37" containerID="7865bd94756decbb49f628e8f2dab7a34cadfd3d6959704deffe4016b6601d0f" exitCode=0
Sep 17 17:50:43.972828 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:50:43.972829    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f2w9fv" event={"ID":"1f35cbc3-81fa-479f-8076-371dc8ecdc37","Type":"ContainerDied","Data":"7865bd94756decbb49f628e8f2dab7a34cadfd3d6959704deffe4016b6601d0f"}
Sep 17 17:50:45.102699 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:50:45.102677    2575 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f2w9fv"
Sep 17 17:50:45.262826 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:50:45.262740    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/1f35cbc3-81fa-479f-8076-371dc8ecdc37-util\") pod \"1f35cbc3-81fa-479f-8076-371dc8ecdc37\" (UID: \"1f35cbc3-81fa-479f-8076-371dc8ecdc37\") "
Sep 17 17:50:45.262826 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:50:45.262779    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/1f35cbc3-81fa-479f-8076-371dc8ecdc37-bundle\") pod \"1f35cbc3-81fa-479f-8076-371dc8ecdc37\" (UID: \"1f35cbc3-81fa-479f-8076-371dc8ecdc37\") "
Sep 17 17:50:45.262826 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:50:45.262826    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-r6gzf\" (UniqueName: \"kubernetes.io/projected/1f35cbc3-81fa-479f-8076-371dc8ecdc37-kube-api-access-r6gzf\") pod \"1f35cbc3-81fa-479f-8076-371dc8ecdc37\" (UID: \"1f35cbc3-81fa-479f-8076-371dc8ecdc37\") "
Sep 17 17:50:45.263239 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:50:45.263206    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/1f35cbc3-81fa-479f-8076-371dc8ecdc37-bundle" (OuterVolumeSpecName: "bundle") pod "1f35cbc3-81fa-479f-8076-371dc8ecdc37" (UID: "1f35cbc3-81fa-479f-8076-371dc8ecdc37"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 17 17:50:45.264930 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:50:45.264905    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/1f35cbc3-81fa-479f-8076-371dc8ecdc37-kube-api-access-r6gzf" (OuterVolumeSpecName: "kube-api-access-r6gzf") pod "1f35cbc3-81fa-479f-8076-371dc8ecdc37" (UID: "1f35cbc3-81fa-479f-8076-371dc8ecdc37"). InnerVolumeSpecName "kube-api-access-r6gzf". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 17 17:50:45.268919 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:50:45.268883    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/1f35cbc3-81fa-479f-8076-371dc8ecdc37-util" (OuterVolumeSpecName: "util") pod "1f35cbc3-81fa-479f-8076-371dc8ecdc37" (UID: "1f35cbc3-81fa-479f-8076-371dc8ecdc37"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 17 17:50:45.363628 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:50:45.363578    2575 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/1f35cbc3-81fa-479f-8076-371dc8ecdc37-util\") on node \"ip-10-0-143-74.ec2.internal\" DevicePath \"\""
Sep 17 17:50:45.363628 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:50:45.363623    2575 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/1f35cbc3-81fa-479f-8076-371dc8ecdc37-bundle\") on node \"ip-10-0-143-74.ec2.internal\" DevicePath \"\""
Sep 17 17:50:45.363628 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:50:45.363633    2575 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-r6gzf\" (UniqueName: \"kubernetes.io/projected/1f35cbc3-81fa-479f-8076-371dc8ecdc37-kube-api-access-r6gzf\") on node \"ip-10-0-143-74.ec2.internal\" DevicePath \"\""
Sep 17 17:50:45.984366 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:50:45.984333    2575 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f2w9fv"
Sep 17 17:50:45.984553 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:50:45.984367    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f2w9fv" event={"ID":"1f35cbc3-81fa-479f-8076-371dc8ecdc37","Type":"ContainerDied","Data":"3b161532ea79b5d8e1b260bece285cb6d044c5bc53cfceb2e1d2f761d523ea55"}
Sep 17 17:50:45.984553 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:50:45.984416    2575 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="3b161532ea79b5d8e1b260bece285cb6d044c5bc53cfceb2e1d2f761d523ea55"
Sep 17 17:50:46.973698 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:50:46.973656    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-djvzt_1d07e68d-9943-420f-a6a0-5f477792fabd/konnectivity-agent/0.log" limit=1024
Sep 17 17:50:47.100034 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:50:47.099991    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-djvzt_1d07e68d-9943-420f-a6a0-5f477792fabd/konnectivity-agent/0.log" limit=1024
Sep 17 17:50:49.732460 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:50:49.732420    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-djvzt_1d07e68d-9943-420f-a6a0-5f477792fabd/konnectivity-agent/0.log" limit=1024
Sep 17 17:50:49.915148 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:50:49.915112    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-djvzt_1d07e68d-9943-420f-a6a0-5f477792fabd/konnectivity-agent/0.log" limit=1024
Sep 17 17:50:56.925661 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:50:56.925625    2575 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["cert-manager/cert-manager-55b4d954d5-j9pqj"]
Sep 17 17:50:56.926159 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:50:56.925932    2575 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="1f35cbc3-81fa-479f-8076-371dc8ecdc37" containerName="extract"
Sep 17 17:50:56.926159 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:50:56.925944    2575 state_mem.go:107] "Deleted CPUSet assignment" podUID="1f35cbc3-81fa-479f-8076-371dc8ecdc37" containerName="extract"
Sep 17 17:50:56.926159 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:50:56.925970    2575 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="1f35cbc3-81fa-479f-8076-371dc8ecdc37" containerName="pull"
Sep 17 17:50:56.926159 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:50:56.925975    2575 state_mem.go:107] "Deleted CPUSet assignment" podUID="1f35cbc3-81fa-479f-8076-371dc8ecdc37" containerName="pull"
Sep 17 17:50:56.926159 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:50:56.925990    2575 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="1f35cbc3-81fa-479f-8076-371dc8ecdc37" containerName="util"
Sep 17 17:50:56.926159 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:50:56.925995    2575 state_mem.go:107] "Deleted CPUSet assignment" podUID="1f35cbc3-81fa-479f-8076-371dc8ecdc37" containerName="util"
Sep 17 17:50:56.926159 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:50:56.926056    2575 memory_manager.go:368] "RemoveStaleState removing state" podUID="1f35cbc3-81fa-479f-8076-371dc8ecdc37" containerName="extract"
Sep 17 17:50:56.930594 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:50:56.930575    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-55b4d954d5-j9pqj"
Sep 17 17:50:56.933789 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:50:56.933670    2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager\"/\"kube-root-ca.crt\""
Sep 17 17:50:56.933789 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:50:56.933709    2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager\"/\"openshift-service-ca.crt\""
Sep 17 17:50:56.933789 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:50:56.933719    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"cert-manager\"/\"cert-manager-dockercfg-8p5l7\""
Sep 17 17:50:56.938914 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:50:56.938888    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-55b4d954d5-j9pqj"]
Sep 17 17:50:57.059399 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:50:57.059370    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-kvn5d\" (UniqueName: \"kubernetes.io/projected/457123e6-7da3-4894-a018-52ea386a51b5-kube-api-access-kvn5d\") pod \"cert-manager-55b4d954d5-j9pqj\" (UID: \"457123e6-7da3-4894-a018-52ea386a51b5\") " pod="cert-manager/cert-manager-55b4d954d5-j9pqj"
Sep 17 17:50:57.059598 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:50:57.059575    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/457123e6-7da3-4894-a018-52ea386a51b5-bound-sa-token\") pod \"cert-manager-55b4d954d5-j9pqj\" (UID: \"457123e6-7da3-4894-a018-52ea386a51b5\") " pod="cert-manager/cert-manager-55b4d954d5-j9pqj"
Sep 17 17:50:57.161651 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:50:57.161619    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/457123e6-7da3-4894-a018-52ea386a51b5-bound-sa-token\") pod \"cert-manager-55b4d954d5-j9pqj\" (UID: \"457123e6-7da3-4894-a018-52ea386a51b5\") " pod="cert-manager/cert-manager-55b4d954d5-j9pqj"
Sep 17 17:50:57.161804 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:50:57.161676    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kvn5d\" (UniqueName: \"kubernetes.io/projected/457123e6-7da3-4894-a018-52ea386a51b5-kube-api-access-kvn5d\") pod \"cert-manager-55b4d954d5-j9pqj\" (UID: \"457123e6-7da3-4894-a018-52ea386a51b5\") " pod="cert-manager/cert-manager-55b4d954d5-j9pqj"
Sep 17 17:50:57.170525 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:50:57.170496    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/457123e6-7da3-4894-a018-52ea386a51b5-bound-sa-token\") pod \"cert-manager-55b4d954d5-j9pqj\" (UID: \"457123e6-7da3-4894-a018-52ea386a51b5\") " pod="cert-manager/cert-manager-55b4d954d5-j9pqj"
Sep 17 17:50:57.170625 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:50:57.170501    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-kvn5d\" (UniqueName: \"kubernetes.io/projected/457123e6-7da3-4894-a018-52ea386a51b5-kube-api-access-kvn5d\") pod \"cert-manager-55b4d954d5-j9pqj\" (UID: \"457123e6-7da3-4894-a018-52ea386a51b5\") " pod="cert-manager/cert-manager-55b4d954d5-j9pqj"
Sep 17 17:50:57.240194 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:50:57.240124    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-55b4d954d5-j9pqj"
Sep 17 17:50:57.509648 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:50:57.509618    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-55b4d954d5-j9pqj"]
Sep 17 17:50:57.512097 ip-10-0-143-74 kubenswrapper[2575]: W0917 17:50:57.512070    2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod457123e6_7da3_4894_a018_52ea386a51b5.slice/crio-6dee42edea14e1f43189d448a48c504b0d0edd4ac43217be9ad891380307629a WatchSource:0}: Error finding container 6dee42edea14e1f43189d448a48c504b0d0edd4ac43217be9ad891380307629a: Status 404 returned error can't find the container with id 6dee42edea14e1f43189d448a48c504b0d0edd4ac43217be9ad891380307629a
Sep 17 17:50:58.038256 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:50:58.038222    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-55b4d954d5-j9pqj" event={"ID":"457123e6-7da3-4894-a018-52ea386a51b5","Type":"ContainerStarted","Data":"6dee42edea14e1f43189d448a48c504b0d0edd4ac43217be9ad891380307629a"}
Sep 17 17:51:01.056059 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:51:01.056013    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-55b4d954d5-j9pqj" event={"ID":"457123e6-7da3-4894-a018-52ea386a51b5","Type":"ContainerStarted","Data":"86d75e29fb14314df9c8b5b9ec66cb34a9cd62ec3ee944fe68443bcd9c898b47"}
Sep 17 17:51:01.076685 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:51:01.076638    2575 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="cert-manager/cert-manager-55b4d954d5-j9pqj" podStartSLOduration=2.590816003 podStartE2EDuration="5.076624299s" totalImagesPullingTime="2.485808296s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-17 17:50:56 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-17 17:51:01.074251269 +0000 UTC m=+980.704581243" watchObservedRunningTime="2026-09-17 17:51:01.076624299 +0000 UTC m=+980.706954273"
Sep 17 17:51:02.064729 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:51:02.064677    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-djvzt_1d07e68d-9943-420f-a6a0-5f477792fabd/konnectivity-agent/0.log" limit=1024
Sep 17 17:51:06.298403 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:51:06.298315    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-djvzt_1d07e68d-9943-420f-a6a0-5f477792fabd/konnectivity-agent/0.log" limit=1024
Sep 17 17:51:06.394514 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:51:06.394470    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-djvzt_1d07e68d-9943-420f-a6a0-5f477792fabd/konnectivity-agent/0.log" limit=1024
Sep 17 17:51:07.453676 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:51:07.453639    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-djvzt_1d07e68d-9943-420f-a6a0-5f477792fabd/konnectivity-agent/0.log" limit=1024
Sep 17 17:51:07.531595 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:51:07.531559    2575 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7b7995d5cd-mbwr4"]
Sep 17 17:51:07.534541 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:51:07.534524    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-mbwr4"
Sep 17 17:51:07.539195 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:51:07.539175    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"cacerts\""
Sep 17 17:51:07.539309 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:51:07.539200    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istiod-tls\""
Sep 17 17:51:07.539422 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:51:07.539407    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istio-kubeconfig\""
Sep 17 17:51:07.539514 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:51:07.539411    2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-gw-ca-root-cert\""
Sep 17 17:51:07.539514 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:51:07.539454    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istiod-openshift-gateway-dockercfg-wt6qv\""
Sep 17 17:51:07.544140 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:51:07.544121    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/449a6010-6e6b-4fa8-aeb6-570e53fc9081-local-certs\") pod \"istiod-openshift-gateway-7b7995d5cd-mbwr4\" (UID: \"449a6010-6e6b-4fa8-aeb6-570e53fc9081\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-mbwr4"
Sep 17 17:51:07.544244 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:51:07.544148    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/449a6010-6e6b-4fa8-aeb6-570e53fc9081-istio-token\") pod \"istiod-openshift-gateway-7b7995d5cd-mbwr4\" (UID: \"449a6010-6e6b-4fa8-aeb6-570e53fc9081\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-mbwr4"
Sep 17 17:51:07.544244 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:51:07.544163    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/449a6010-6e6b-4fa8-aeb6-570e53fc9081-istio-kubeconfig\") pod \"istiod-openshift-gateway-7b7995d5cd-mbwr4\" (UID: \"449a6010-6e6b-4fa8-aeb6-570e53fc9081\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-mbwr4"
Sep 17 17:51:07.544244 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:51:07.544178    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xppgz\" (UniqueName: \"kubernetes.io/projected/449a6010-6e6b-4fa8-aeb6-570e53fc9081-kube-api-access-xppgz\") pod \"istiod-openshift-gateway-7b7995d5cd-mbwr4\" (UID: \"449a6010-6e6b-4fa8-aeb6-570e53fc9081\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-mbwr4"
Sep 17 17:51:07.544244 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:51:07.544196    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/449a6010-6e6b-4fa8-aeb6-570e53fc9081-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-7b7995d5cd-mbwr4\" (UID: \"449a6010-6e6b-4fa8-aeb6-570e53fc9081\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-mbwr4"
Sep 17 17:51:07.544244 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:51:07.544239    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/449a6010-6e6b-4fa8-aeb6-570e53fc9081-cacerts\") pod \"istiod-openshift-gateway-7b7995d5cd-mbwr4\" (UID: \"449a6010-6e6b-4fa8-aeb6-570e53fc9081\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-mbwr4"
Sep 17 17:51:07.544428 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:51:07.544263    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/449a6010-6e6b-4fa8-aeb6-570e53fc9081-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-7b7995d5cd-mbwr4\" (UID: \"449a6010-6e6b-4fa8-aeb6-570e53fc9081\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-mbwr4"
Sep 17 17:51:07.560141 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:51:07.560117    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7b7995d5cd-mbwr4"]
Sep 17 17:51:07.604165 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:51:07.604134    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-djvzt_1d07e68d-9943-420f-a6a0-5f477792fabd/konnectivity-agent/0.log" limit=1024
Sep 17 17:51:07.645371 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:51:07.645342    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/449a6010-6e6b-4fa8-aeb6-570e53fc9081-local-certs\") pod \"istiod-openshift-gateway-7b7995d5cd-mbwr4\" (UID: \"449a6010-6e6b-4fa8-aeb6-570e53fc9081\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-mbwr4"
Sep 17 17:51:07.645538 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:51:07.645375    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/449a6010-6e6b-4fa8-aeb6-570e53fc9081-istio-token\") pod \"istiod-openshift-gateway-7b7995d5cd-mbwr4\" (UID: \"449a6010-6e6b-4fa8-aeb6-570e53fc9081\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-mbwr4"
Sep 17 17:51:07.645538 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:51:07.645395    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/449a6010-6e6b-4fa8-aeb6-570e53fc9081-istio-kubeconfig\") pod \"istiod-openshift-gateway-7b7995d5cd-mbwr4\" (UID: \"449a6010-6e6b-4fa8-aeb6-570e53fc9081\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-mbwr4"
Sep 17 17:51:07.645538 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:51:07.645410    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xppgz\" (UniqueName: \"kubernetes.io/projected/449a6010-6e6b-4fa8-aeb6-570e53fc9081-kube-api-access-xppgz\") pod \"istiod-openshift-gateway-7b7995d5cd-mbwr4\" (UID: \"449a6010-6e6b-4fa8-aeb6-570e53fc9081\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-mbwr4"
Sep 17 17:51:07.645538 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:51:07.645427    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/449a6010-6e6b-4fa8-aeb6-570e53fc9081-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-7b7995d5cd-mbwr4\" (UID: \"449a6010-6e6b-4fa8-aeb6-570e53fc9081\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-mbwr4"
Sep 17 17:51:07.645538 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:51:07.645474    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/449a6010-6e6b-4fa8-aeb6-570e53fc9081-cacerts\") pod \"istiod-openshift-gateway-7b7995d5cd-mbwr4\" (UID: \"449a6010-6e6b-4fa8-aeb6-570e53fc9081\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-mbwr4"
Sep 17 17:51:07.645538 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:51:07.645510    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/449a6010-6e6b-4fa8-aeb6-570e53fc9081-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-7b7995d5cd-mbwr4\" (UID: \"449a6010-6e6b-4fa8-aeb6-570e53fc9081\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-mbwr4"
Sep 17 17:51:07.646147 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:51:07.646119    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/449a6010-6e6b-4fa8-aeb6-570e53fc9081-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-7b7995d5cd-mbwr4\" (UID: \"449a6010-6e6b-4fa8-aeb6-570e53fc9081\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-mbwr4"
Sep 17 17:51:07.647902 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:51:07.647873    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/449a6010-6e6b-4fa8-aeb6-570e53fc9081-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-7b7995d5cd-mbwr4\" (UID: \"449a6010-6e6b-4fa8-aeb6-570e53fc9081\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-mbwr4"
Sep 17 17:51:07.647983 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:51:07.647943    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/449a6010-6e6b-4fa8-aeb6-570e53fc9081-istio-kubeconfig\") pod \"istiod-openshift-gateway-7b7995d5cd-mbwr4\" (UID: \"449a6010-6e6b-4fa8-aeb6-570e53fc9081\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-mbwr4"
Sep 17 17:51:07.648067 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:51:07.648048    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/449a6010-6e6b-4fa8-aeb6-570e53fc9081-local-certs\") pod \"istiod-openshift-gateway-7b7995d5cd-mbwr4\" (UID: \"449a6010-6e6b-4fa8-aeb6-570e53fc9081\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-mbwr4"
Sep 17 17:51:07.648102 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:51:07.648070    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/449a6010-6e6b-4fa8-aeb6-570e53fc9081-cacerts\") pod \"istiod-openshift-gateway-7b7995d5cd-mbwr4\" (UID: \"449a6010-6e6b-4fa8-aeb6-570e53fc9081\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-mbwr4"
Sep 17 17:51:07.658144 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:51:07.658121    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-xppgz\" (UniqueName: \"kubernetes.io/projected/449a6010-6e6b-4fa8-aeb6-570e53fc9081-kube-api-access-xppgz\") pod \"istiod-openshift-gateway-7b7995d5cd-mbwr4\" (UID: \"449a6010-6e6b-4fa8-aeb6-570e53fc9081\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-mbwr4"
Sep 17 17:51:07.663341 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:51:07.663320    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/449a6010-6e6b-4fa8-aeb6-570e53fc9081-istio-token\") pod \"istiod-openshift-gateway-7b7995d5cd-mbwr4\" (UID: \"449a6010-6e6b-4fa8-aeb6-570e53fc9081\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-mbwr4"
Sep 17 17:51:07.844215 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:51:07.844134    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-mbwr4"
Sep 17 17:51:08.120204 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:51:08.120180    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7b7995d5cd-mbwr4"]
Sep 17 17:51:08.122060 ip-10-0-143-74 kubenswrapper[2575]: W0917 17:51:08.122033    2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod449a6010_6e6b_4fa8_aeb6_570e53fc9081.slice/crio-b9d08b703bc7d0031db7b9640cb918ffff6e6fadb40284cef6452c3a1dd172f7 WatchSource:0}: Error finding container b9d08b703bc7d0031db7b9640cb918ffff6e6fadb40284cef6452c3a1dd172f7: Status 404 returned error can't find the container with id b9d08b703bc7d0031db7b9640cb918ffff6e6fadb40284cef6452c3a1dd172f7
Sep 17 17:51:08.869582 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:51:08.869536    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-djvzt_1d07e68d-9943-420f-a6a0-5f477792fabd/konnectivity-agent/0.log" limit=1024
Sep 17 17:51:09.096916 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:51:09.096880    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-mbwr4" event={"ID":"449a6010-6e6b-4fa8-aeb6-570e53fc9081","Type":"ContainerStarted","Data":"b9d08b703bc7d0031db7b9640cb918ffff6e6fadb40284cef6452c3a1dd172f7"}
Sep 17 17:51:10.579917 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:51:10.579182    2575 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236216Ki","pods":"250"}
Sep 17 17:51:10.579917 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:51:10.579245    2575 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236216Ki","pods":"250"}
Sep 17 17:51:11.109035 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:51:11.109003    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-mbwr4" event={"ID":"449a6010-6e6b-4fa8-aeb6-570e53fc9081","Type":"ContainerStarted","Data":"7a0874efd22cda87acd4f3ca1a804886936ccb00790b2d63061231d68325312e"}
Sep 17 17:51:11.109235 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:51:11.109060    2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-mbwr4"
Sep 17 17:51:11.110872 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:51:11.110847    2575 patch_prober.go:28] interesting pod/istiod-openshift-gateway-7b7995d5cd-mbwr4 container/discovery namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body=
Sep 17 17:51:11.110982 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:51:11.110898    2575 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-mbwr4" podUID="449a6010-6e6b-4fa8-aeb6-570e53fc9081" containerName="discovery" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Sep 17 17:51:11.150314 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:51:11.150262    2575 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-mbwr4" podStartSLOduration=1.6952188019999999 podStartE2EDuration="4.150247959s" totalImagesPullingTime="2.455029157s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-17 17:51:07 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-17 17:51:11.148532189 +0000 UTC m=+990.778862164" watchObservedRunningTime="2026-09-17 17:51:11.150247959 +0000 UTC m=+990.780577932"
Sep 17 17:51:11.638951 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:51:11.638920    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-djvzt_1d07e68d-9943-420f-a6a0-5f477792fabd/konnectivity-agent/0.log" limit=1024
Sep 17 17:51:11.746248 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:51:11.746211    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-djvzt_1d07e68d-9943-420f-a6a0-5f477792fabd/konnectivity-agent/0.log" limit=1024
Sep 17 17:51:12.115274 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:51:12.115199    2575 patch_prober.go:28] interesting pod/istiod-openshift-gateway-7b7995d5cd-mbwr4 container/discovery namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body=
Sep 17 17:51:12.115274 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:51:12.115260    2575 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-mbwr4" podUID="449a6010-6e6b-4fa8-aeb6-570e53fc9081" containerName="discovery" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Sep 17 17:51:15.115961 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:51:15.115929    2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-mbwr4"
Sep 17 17:51:15.660417 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:51:15.660344    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-djvzt_1d07e68d-9943-420f-a6a0-5f477792fabd/konnectivity-agent/0.log" limit=1024
Sep 17 17:51:15.773741 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:51:15.773706    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-djvzt_1d07e68d-9943-420f-a6a0-5f477792fabd/konnectivity-agent/0.log" limit=1024
Sep 17 17:51:19.292924 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:51:19.292887    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-djvzt_1d07e68d-9943-420f-a6a0-5f477792fabd/konnectivity-agent/0.log" limit=1024
Sep 17 17:51:24.095620 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:51:24.095580    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-djvzt_1d07e68d-9943-420f-a6a0-5f477792fabd/konnectivity-agent/0.log" limit=1024
Sep 17 17:51:24.286964 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:51:24.286922    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-djvzt_1d07e68d-9943-420f-a6a0-5f477792fabd/konnectivity-agent/0.log" limit=1024
Sep 17 17:51:29.995744 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:51:29.995712    2575 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-marketplace/e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4c54qm9"]
Sep 17 17:51:29.999412 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:51:29.999397    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4c54qm9"
Sep 17 17:51:30.002089 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:51:30.002065    2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"kube-root-ca.crt\""
Sep 17 17:51:30.002278 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:51:30.002262    2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"openshift-service-ca.crt\""
Sep 17 17:51:30.003160 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:51:30.003139    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-marketplace\"/\"default-dockercfg-p47rs\""
Sep 17 17:51:30.009260 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:51:30.009239    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4c54qm9"]
Sep 17 17:51:30.068889 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:51:30.068855    2575 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-marketplace/ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6pc498"]
Sep 17 17:51:30.072360 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:51:30.072339    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6pc498"
Sep 17 17:51:30.089597 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:51:30.089569    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6pc498"]
Sep 17 17:51:30.135525 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:51:30.135493    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/9a95c99c-1cdf-428d-9a4e-87c0998c4b80-util\") pod \"ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6pc498\" (UID: \"9a95c99c-1cdf-428d-9a4e-87c0998c4b80\") " pod="openshift-marketplace/ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6pc498"
Sep 17 17:51:30.135525 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:51:30.135528    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hr2jg\" (UniqueName: \"kubernetes.io/projected/9a95c99c-1cdf-428d-9a4e-87c0998c4b80-kube-api-access-hr2jg\") pod \"ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6pc498\" (UID: \"9a95c99c-1cdf-428d-9a4e-87c0998c4b80\") " pod="openshift-marketplace/ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6pc498"
Sep 17 17:51:30.135714 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:51:30.135555    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/07cf78e0-a8fe-43c6-8307-b559dee0ae66-bundle\") pod \"e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4c54qm9\" (UID: \"07cf78e0-a8fe-43c6-8307-b559dee0ae66\") " pod="openshift-marketplace/e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4c54qm9"
Sep 17 17:51:30.135714 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:51:30.135609    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/9a95c99c-1cdf-428d-9a4e-87c0998c4b80-bundle\") pod \"ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6pc498\" (UID: \"9a95c99c-1cdf-428d-9a4e-87c0998c4b80\") " pod="openshift-marketplace/ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6pc498"
Sep 17 17:51:30.135714 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:51:30.135628    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/07cf78e0-a8fe-43c6-8307-b559dee0ae66-util\") pod \"e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4c54qm9\" (UID: \"07cf78e0-a8fe-43c6-8307-b559dee0ae66\") " pod="openshift-marketplace/e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4c54qm9"
Sep 17 17:51:30.135714 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:51:30.135695    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-c2wv6\" (UniqueName: \"kubernetes.io/projected/07cf78e0-a8fe-43c6-8307-b559dee0ae66-kube-api-access-c2wv6\") pod \"e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4c54qm9\" (UID: \"07cf78e0-a8fe-43c6-8307-b559dee0ae66\") " pod="openshift-marketplace/e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4c54qm9"
Sep 17 17:51:30.215063 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:51:30.215029    2575 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-marketplace/6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847asdtmz"]
Sep 17 17:51:30.218483 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:51:30.218466    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847asdtmz"
Sep 17 17:51:30.230075 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:51:30.230052    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847asdtmz"]
Sep 17 17:51:30.236867 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:51:30.236841    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/9a95c99c-1cdf-428d-9a4e-87c0998c4b80-bundle\") pod \"ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6pc498\" (UID: \"9a95c99c-1cdf-428d-9a4e-87c0998c4b80\") " pod="openshift-marketplace/ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6pc498"
Sep 17 17:51:30.236982 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:51:30.236880    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/07cf78e0-a8fe-43c6-8307-b559dee0ae66-util\") pod \"e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4c54qm9\" (UID: \"07cf78e0-a8fe-43c6-8307-b559dee0ae66\") " pod="openshift-marketplace/e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4c54qm9"
Sep 17 17:51:30.236982 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:51:30.236966    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-c2wv6\" (UniqueName: \"kubernetes.io/projected/07cf78e0-a8fe-43c6-8307-b559dee0ae66-kube-api-access-c2wv6\") pod \"e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4c54qm9\" (UID: \"07cf78e0-a8fe-43c6-8307-b559dee0ae66\") " pod="openshift-marketplace/e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4c54qm9"
Sep 17 17:51:30.237056 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:51:30.237010    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/9a95c99c-1cdf-428d-9a4e-87c0998c4b80-util\") pod \"ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6pc498\" (UID: \"9a95c99c-1cdf-428d-9a4e-87c0998c4b80\") " pod="openshift-marketplace/ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6pc498"
Sep 17 17:51:30.237056 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:51:30.237035    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hr2jg\" (UniqueName: \"kubernetes.io/projected/9a95c99c-1cdf-428d-9a4e-87c0998c4b80-kube-api-access-hr2jg\") pod \"ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6pc498\" (UID: \"9a95c99c-1cdf-428d-9a4e-87c0998c4b80\") " pod="openshift-marketplace/ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6pc498"
Sep 17 17:51:30.237150 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:51:30.237063    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/07cf78e0-a8fe-43c6-8307-b559dee0ae66-bundle\") pod \"e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4c54qm9\" (UID: \"07cf78e0-a8fe-43c6-8307-b559dee0ae66\") " pod="openshift-marketplace/e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4c54qm9"
Sep 17 17:51:30.237263 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:51:30.237244    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/9a95c99c-1cdf-428d-9a4e-87c0998c4b80-bundle\") pod \"ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6pc498\" (UID: \"9a95c99c-1cdf-428d-9a4e-87c0998c4b80\") " pod="openshift-marketplace/ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6pc498"
Sep 17 17:51:30.237317 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:51:30.237253    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/07cf78e0-a8fe-43c6-8307-b559dee0ae66-util\") pod \"e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4c54qm9\" (UID: \"07cf78e0-a8fe-43c6-8307-b559dee0ae66\") " pod="openshift-marketplace/e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4c54qm9"
Sep 17 17:51:30.237359 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:51:30.237322    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/9a95c99c-1cdf-428d-9a4e-87c0998c4b80-util\") pod \"ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6pc498\" (UID: \"9a95c99c-1cdf-428d-9a4e-87c0998c4b80\") " pod="openshift-marketplace/ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6pc498"
Sep 17 17:51:30.237459 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:51:30.237418    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/07cf78e0-a8fe-43c6-8307-b559dee0ae66-bundle\") pod \"e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4c54qm9\" (UID: \"07cf78e0-a8fe-43c6-8307-b559dee0ae66\") " pod="openshift-marketplace/e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4c54qm9"
Sep 17 17:51:30.246565 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:51:30.246513    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-hr2jg\" (UniqueName: \"kubernetes.io/projected/9a95c99c-1cdf-428d-9a4e-87c0998c4b80-kube-api-access-hr2jg\") pod \"ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6pc498\" (UID: \"9a95c99c-1cdf-428d-9a4e-87c0998c4b80\") " pod="openshift-marketplace/ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6pc498"
Sep 17 17:51:30.246636 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:51:30.246610    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-c2wv6\" (UniqueName: \"kubernetes.io/projected/07cf78e0-a8fe-43c6-8307-b559dee0ae66-kube-api-access-c2wv6\") pod \"e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4c54qm9\" (UID: \"07cf78e0-a8fe-43c6-8307-b559dee0ae66\") " pod="openshift-marketplace/e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4c54qm9"
Sep 17 17:51:30.304468 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:51:30.304426    2575 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-marketplace/da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7hhmqk"]
Sep 17 17:51:30.308213 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:51:30.308195    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4c54qm9"
Sep 17 17:51:30.308282 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:51:30.308239    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7hhmqk"
Sep 17 17:51:30.319090 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:51:30.319063    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7hhmqk"]
Sep 17 17:51:30.338008 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:51:30.337965    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/968beef5-2813-42c1-b94d-402cc2b23425-util\") pod \"6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847asdtmz\" (UID: \"968beef5-2813-42c1-b94d-402cc2b23425\") " pod="openshift-marketplace/6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847asdtmz"
Sep 17 17:51:30.338155 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:51:30.338100    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mwv4p\" (UniqueName: \"kubernetes.io/projected/968beef5-2813-42c1-b94d-402cc2b23425-kube-api-access-mwv4p\") pod \"6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847asdtmz\" (UID: \"968beef5-2813-42c1-b94d-402cc2b23425\") " pod="openshift-marketplace/6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847asdtmz"
Sep 17 17:51:30.338155 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:51:30.338145    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/968beef5-2813-42c1-b94d-402cc2b23425-bundle\") pod \"6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847asdtmz\" (UID: \"968beef5-2813-42c1-b94d-402cc2b23425\") " pod="openshift-marketplace/6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847asdtmz"
Sep 17 17:51:30.381145 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:51:30.381116    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6pc498"
Sep 17 17:51:30.439412 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:51:30.438973    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/6f9584a2-c363-4a56-80b5-3ae4aa8ab6e7-bundle\") pod \"da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7hhmqk\" (UID: \"6f9584a2-c363-4a56-80b5-3ae4aa8ab6e7\") " pod="openshift-marketplace/da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7hhmqk"
Sep 17 17:51:30.439412 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:51:30.439011    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/968beef5-2813-42c1-b94d-402cc2b23425-util\") pod \"6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847asdtmz\" (UID: \"968beef5-2813-42c1-b94d-402cc2b23425\") " pod="openshift-marketplace/6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847asdtmz"
Sep 17 17:51:30.439412 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:51:30.439034    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/6f9584a2-c363-4a56-80b5-3ae4aa8ab6e7-util\") pod \"da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7hhmqk\" (UID: \"6f9584a2-c363-4a56-80b5-3ae4aa8ab6e7\") " pod="openshift-marketplace/da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7hhmqk"
Sep 17 17:51:30.439412 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:51:30.439064    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mxv8l\" (UniqueName: \"kubernetes.io/projected/6f9584a2-c363-4a56-80b5-3ae4aa8ab6e7-kube-api-access-mxv8l\") pod \"da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7hhmqk\" (UID: \"6f9584a2-c363-4a56-80b5-3ae4aa8ab6e7\") " pod="openshift-marketplace/da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7hhmqk"
Sep 17 17:51:30.439412 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:51:30.439088    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mwv4p\" (UniqueName: \"kubernetes.io/projected/968beef5-2813-42c1-b94d-402cc2b23425-kube-api-access-mwv4p\") pod \"6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847asdtmz\" (UID: \"968beef5-2813-42c1-b94d-402cc2b23425\") " pod="openshift-marketplace/6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847asdtmz"
Sep 17 17:51:30.439412 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:51:30.439114    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/968beef5-2813-42c1-b94d-402cc2b23425-bundle\") pod \"6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847asdtmz\" (UID: \"968beef5-2813-42c1-b94d-402cc2b23425\") " pod="openshift-marketplace/6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847asdtmz"
Sep 17 17:51:30.439412 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:51:30.439396    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/968beef5-2813-42c1-b94d-402cc2b23425-bundle\") pod \"6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847asdtmz\" (UID: \"968beef5-2813-42c1-b94d-402cc2b23425\") " pod="openshift-marketplace/6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847asdtmz"
Sep 17 17:51:30.440426 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:51:30.440382    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/968beef5-2813-42c1-b94d-402cc2b23425-util\") pod \"6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847asdtmz\" (UID: \"968beef5-2813-42c1-b94d-402cc2b23425\") " pod="openshift-marketplace/6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847asdtmz"
Sep 17 17:51:30.448456 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:51:30.448427    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-mwv4p\" (UniqueName: \"kubernetes.io/projected/968beef5-2813-42c1-b94d-402cc2b23425-kube-api-access-mwv4p\") pod \"6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847asdtmz\" (UID: \"968beef5-2813-42c1-b94d-402cc2b23425\") " pod="openshift-marketplace/6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847asdtmz"
Sep 17 17:51:30.527653 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:51:30.527572    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847asdtmz"
Sep 17 17:51:30.539891 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:51:30.539854    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/6f9584a2-c363-4a56-80b5-3ae4aa8ab6e7-util\") pod \"da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7hhmqk\" (UID: \"6f9584a2-c363-4a56-80b5-3ae4aa8ab6e7\") " pod="openshift-marketplace/da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7hhmqk"
Sep 17 17:51:30.540004 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:51:30.539949    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mxv8l\" (UniqueName: \"kubernetes.io/projected/6f9584a2-c363-4a56-80b5-3ae4aa8ab6e7-kube-api-access-mxv8l\") pod \"da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7hhmqk\" (UID: \"6f9584a2-c363-4a56-80b5-3ae4aa8ab6e7\") " pod="openshift-marketplace/da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7hhmqk"
Sep 17 17:51:30.540077 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:51:30.540059    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/6f9584a2-c363-4a56-80b5-3ae4aa8ab6e7-bundle\") pod \"da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7hhmqk\" (UID: \"6f9584a2-c363-4a56-80b5-3ae4aa8ab6e7\") " pod="openshift-marketplace/da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7hhmqk"
Sep 17 17:51:30.540257 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:51:30.540235    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/6f9584a2-c363-4a56-80b5-3ae4aa8ab6e7-util\") pod \"da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7hhmqk\" (UID: \"6f9584a2-c363-4a56-80b5-3ae4aa8ab6e7\") " pod="openshift-marketplace/da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7hhmqk"
Sep 17 17:51:30.540394 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:51:30.540375    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/6f9584a2-c363-4a56-80b5-3ae4aa8ab6e7-bundle\") pod \"da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7hhmqk\" (UID: \"6f9584a2-c363-4a56-80b5-3ae4aa8ab6e7\") " pod="openshift-marketplace/da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7hhmqk"
Sep 17 17:51:30.549456 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:51:30.549418    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-mxv8l\" (UniqueName: \"kubernetes.io/projected/6f9584a2-c363-4a56-80b5-3ae4aa8ab6e7-kube-api-access-mxv8l\") pod \"da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7hhmqk\" (UID: \"6f9584a2-c363-4a56-80b5-3ae4aa8ab6e7\") " pod="openshift-marketplace/da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7hhmqk"
Sep 17 17:51:30.589975 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:51:30.589950    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4c54qm9"]
Sep 17 17:51:30.591876 ip-10-0-143-74 kubenswrapper[2575]: W0917 17:51:30.591848    2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod07cf78e0_a8fe_43c6_8307_b559dee0ae66.slice/crio-64ff08a5d7327eedaa71b8026fbd4f960d157b73aeeb7a25e08f673f677473ca WatchSource:0}: Error finding container 64ff08a5d7327eedaa71b8026fbd4f960d157b73aeeb7a25e08f673f677473ca: Status 404 returned error can't find the container with id 64ff08a5d7327eedaa71b8026fbd4f960d157b73aeeb7a25e08f673f677473ca
Sep 17 17:51:30.631186 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:51:30.631160    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7hhmqk"
Sep 17 17:51:30.659797 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:51:30.659769    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6pc498"]
Sep 17 17:51:30.661544 ip-10-0-143-74 kubenswrapper[2575]: W0917 17:51:30.661508    2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9a95c99c_1cdf_428d_9a4e_87c0998c4b80.slice/crio-a9ecc96406d41e23a50fcd92806238174d539209cd6e7a83a43224f37d697c1e WatchSource:0}: Error finding container a9ecc96406d41e23a50fcd92806238174d539209cd6e7a83a43224f37d697c1e: Status 404 returned error can't find the container with id a9ecc96406d41e23a50fcd92806238174d539209cd6e7a83a43224f37d697c1e
Sep 17 17:51:30.799168 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:51:30.799142    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847asdtmz"]
Sep 17 17:51:30.801105 ip-10-0-143-74 kubenswrapper[2575]: W0917 17:51:30.801079    2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod968beef5_2813_42c1_b94d_402cc2b23425.slice/crio-20b462d3cc3412dc0ee26b6f3ace859d6bfdf8861e06ccb7da97478a10c86c6e WatchSource:0}: Error finding container 20b462d3cc3412dc0ee26b6f3ace859d6bfdf8861e06ccb7da97478a10c86c6e: Status 404 returned error can't find the container with id 20b462d3cc3412dc0ee26b6f3ace859d6bfdf8861e06ccb7da97478a10c86c6e
Sep 17 17:51:30.913701 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:51:30.913622    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7hhmqk"]
Sep 17 17:51:30.915841 ip-10-0-143-74 kubenswrapper[2575]: W0917 17:51:30.915809    2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6f9584a2_c363_4a56_80b5_3ae4aa8ab6e7.slice/crio-e4fd32336690bf77afadb1c513b356b183ff7c2136cb81c0adda377567c9e3c5 WatchSource:0}: Error finding container e4fd32336690bf77afadb1c513b356b183ff7c2136cb81c0adda377567c9e3c5: Status 404 returned error can't find the container with id e4fd32336690bf77afadb1c513b356b183ff7c2136cb81c0adda377567c9e3c5
Sep 17 17:51:31.204472 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:51:31.204421    2575 generic.go:356] "Generic (PLEG): container finished" podID="9a95c99c-1cdf-428d-9a4e-87c0998c4b80" containerID="afc3ad8afb7a973982d9b8d5f0913c03e8b98c7ddc9b1d51f9093600b8bc6141" exitCode=0
Sep 17 17:51:31.204957 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:51:31.204504    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6pc498" event={"ID":"9a95c99c-1cdf-428d-9a4e-87c0998c4b80","Type":"ContainerDied","Data":"afc3ad8afb7a973982d9b8d5f0913c03e8b98c7ddc9b1d51f9093600b8bc6141"}
Sep 17 17:51:31.204957 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:51:31.204553    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6pc498" event={"ID":"9a95c99c-1cdf-428d-9a4e-87c0998c4b80","Type":"ContainerStarted","Data":"a9ecc96406d41e23a50fcd92806238174d539209cd6e7a83a43224f37d697c1e"}
Sep 17 17:51:31.206032 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:51:31.206012    2575 generic.go:356] "Generic (PLEG): container finished" podID="968beef5-2813-42c1-b94d-402cc2b23425" containerID="941dec4294380716a19ea6d4241db5b6c53d11cc1a1f71539b16d32158383b68" exitCode=0
Sep 17 17:51:31.206106 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:51:31.206062    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847asdtmz" event={"ID":"968beef5-2813-42c1-b94d-402cc2b23425","Type":"ContainerDied","Data":"941dec4294380716a19ea6d4241db5b6c53d11cc1a1f71539b16d32158383b68"}
Sep 17 17:51:31.206157 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:51:31.206106    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847asdtmz" event={"ID":"968beef5-2813-42c1-b94d-402cc2b23425","Type":"ContainerStarted","Data":"20b462d3cc3412dc0ee26b6f3ace859d6bfdf8861e06ccb7da97478a10c86c6e"}
Sep 17 17:51:31.207600 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:51:31.207583    2575 generic.go:356] "Generic (PLEG): container finished" podID="07cf78e0-a8fe-43c6-8307-b559dee0ae66" containerID="f8407f6e5e9fd96a33b6291d16ebb75c5343fc412224a582fa5661a3fa1a8367" exitCode=0
Sep 17 17:51:31.207674 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:51:31.207653    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4c54qm9" event={"ID":"07cf78e0-a8fe-43c6-8307-b559dee0ae66","Type":"ContainerDied","Data":"f8407f6e5e9fd96a33b6291d16ebb75c5343fc412224a582fa5661a3fa1a8367"}
Sep 17 17:51:31.207806 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:51:31.207697    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4c54qm9" event={"ID":"07cf78e0-a8fe-43c6-8307-b559dee0ae66","Type":"ContainerStarted","Data":"64ff08a5d7327eedaa71b8026fbd4f960d157b73aeeb7a25e08f673f677473ca"}
Sep 17 17:51:31.209355 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:51:31.209338    2575 generic.go:356] "Generic (PLEG): container finished" podID="6f9584a2-c363-4a56-80b5-3ae4aa8ab6e7" containerID="7939f4288ec945571ba96512aa4bc8f979aca7b8f6b82c9fd050b722c9794a00" exitCode=0
Sep 17 17:51:31.209427 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:51:31.209403    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7hhmqk" event={"ID":"6f9584a2-c363-4a56-80b5-3ae4aa8ab6e7","Type":"ContainerDied","Data":"7939f4288ec945571ba96512aa4bc8f979aca7b8f6b82c9fd050b722c9794a00"}
Sep 17 17:51:31.209516 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:51:31.209434    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7hhmqk" event={"ID":"6f9584a2-c363-4a56-80b5-3ae4aa8ab6e7","Type":"ContainerStarted","Data":"e4fd32336690bf77afadb1c513b356b183ff7c2136cb81c0adda377567c9e3c5"}
Sep 17 17:51:32.221201 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:51:32.221176    2575 generic.go:356] "Generic (PLEG): container finished" podID="07cf78e0-a8fe-43c6-8307-b559dee0ae66" containerID="fdce3e85879da5eb3a2a60fed341712459252a54795d2a1ed7d43af34bc433aa" exitCode=0
Sep 17 17:51:32.221573 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:51:32.221258    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4c54qm9" event={"ID":"07cf78e0-a8fe-43c6-8307-b559dee0ae66","Type":"ContainerDied","Data":"fdce3e85879da5eb3a2a60fed341712459252a54795d2a1ed7d43af34bc433aa"}
Sep 17 17:51:33.233095 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:51:33.233056    2575 generic.go:356] "Generic (PLEG): container finished" podID="968beef5-2813-42c1-b94d-402cc2b23425" containerID="12ec49d46f84cbfd5d45f64a27369e7b446bb6ff5fe08f05f283c09cb3114d71" exitCode=0
Sep 17 17:51:33.233540 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:51:33.233136    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847asdtmz" event={"ID":"968beef5-2813-42c1-b94d-402cc2b23425","Type":"ContainerDied","Data":"12ec49d46f84cbfd5d45f64a27369e7b446bb6ff5fe08f05f283c09cb3114d71"}
Sep 17 17:51:33.235237 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:51:33.235215    2575 generic.go:356] "Generic (PLEG): container finished" podID="07cf78e0-a8fe-43c6-8307-b559dee0ae66" containerID="d9b60cf1a36144f01884d37e172d388457c2e69d6b02a7c443ac16bd22f27930" exitCode=0
Sep 17 17:51:33.235318 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:51:33.235275    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4c54qm9" event={"ID":"07cf78e0-a8fe-43c6-8307-b559dee0ae66","Type":"ContainerDied","Data":"d9b60cf1a36144f01884d37e172d388457c2e69d6b02a7c443ac16bd22f27930"}
Sep 17 17:51:33.237073 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:51:33.237051    2575 generic.go:356] "Generic (PLEG): container finished" podID="6f9584a2-c363-4a56-80b5-3ae4aa8ab6e7" containerID="9a8824ebef21c9efb37c06b862b12efce975b4cba3cc34f10c25af16f9750ddd" exitCode=0
Sep 17 17:51:33.237147 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:51:33.237088    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7hhmqk" event={"ID":"6f9584a2-c363-4a56-80b5-3ae4aa8ab6e7","Type":"ContainerDied","Data":"9a8824ebef21c9efb37c06b862b12efce975b4cba3cc34f10c25af16f9750ddd"}
Sep 17 17:51:34.243546 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:51:34.243511    2575 generic.go:356] "Generic (PLEG): container finished" podID="968beef5-2813-42c1-b94d-402cc2b23425" containerID="8cb2da35aaf5e12cb9b807419793576848425c8b98db01abcfc9e0488bcef720" exitCode=0
Sep 17 17:51:34.243892 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:51:34.243593    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847asdtmz" event={"ID":"968beef5-2813-42c1-b94d-402cc2b23425","Type":"ContainerDied","Data":"8cb2da35aaf5e12cb9b807419793576848425c8b98db01abcfc9e0488bcef720"}
Sep 17 17:51:34.245463 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:51:34.245425    2575 generic.go:356] "Generic (PLEG): container finished" podID="6f9584a2-c363-4a56-80b5-3ae4aa8ab6e7" containerID="2bc6fd8186700131ebe0c77a5c396ddb0ae4b13427f91c0d960857c6ad7f2a7b" exitCode=0
Sep 17 17:51:34.245572 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:51:34.245476    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7hhmqk" event={"ID":"6f9584a2-c363-4a56-80b5-3ae4aa8ab6e7","Type":"ContainerDied","Data":"2bc6fd8186700131ebe0c77a5c396ddb0ae4b13427f91c0d960857c6ad7f2a7b"}
Sep 17 17:51:34.370245 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:51:34.370218    2575 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4c54qm9"
Sep 17 17:51:34.472849 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:51:34.472814    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-c2wv6\" (UniqueName: \"kubernetes.io/projected/07cf78e0-a8fe-43c6-8307-b559dee0ae66-kube-api-access-c2wv6\") pod \"07cf78e0-a8fe-43c6-8307-b559dee0ae66\" (UID: \"07cf78e0-a8fe-43c6-8307-b559dee0ae66\") "
Sep 17 17:51:34.472988 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:51:34.472862    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/07cf78e0-a8fe-43c6-8307-b559dee0ae66-util\") pod \"07cf78e0-a8fe-43c6-8307-b559dee0ae66\" (UID: \"07cf78e0-a8fe-43c6-8307-b559dee0ae66\") "
Sep 17 17:51:34.472988 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:51:34.472895    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/07cf78e0-a8fe-43c6-8307-b559dee0ae66-bundle\") pod \"07cf78e0-a8fe-43c6-8307-b559dee0ae66\" (UID: \"07cf78e0-a8fe-43c6-8307-b559dee0ae66\") "
Sep 17 17:51:34.473461 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:51:34.473414    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/07cf78e0-a8fe-43c6-8307-b559dee0ae66-bundle" (OuterVolumeSpecName: "bundle") pod "07cf78e0-a8fe-43c6-8307-b559dee0ae66" (UID: "07cf78e0-a8fe-43c6-8307-b559dee0ae66"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 17 17:51:34.474855 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:51:34.474836    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/07cf78e0-a8fe-43c6-8307-b559dee0ae66-kube-api-access-c2wv6" (OuterVolumeSpecName: "kube-api-access-c2wv6") pod "07cf78e0-a8fe-43c6-8307-b559dee0ae66" (UID: "07cf78e0-a8fe-43c6-8307-b559dee0ae66"). InnerVolumeSpecName "kube-api-access-c2wv6". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 17 17:51:34.477830 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:51:34.477810    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/07cf78e0-a8fe-43c6-8307-b559dee0ae66-util" (OuterVolumeSpecName: "util") pod "07cf78e0-a8fe-43c6-8307-b559dee0ae66" (UID: "07cf78e0-a8fe-43c6-8307-b559dee0ae66"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 17 17:51:34.574487 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:51:34.574388    2575 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-c2wv6\" (UniqueName: \"kubernetes.io/projected/07cf78e0-a8fe-43c6-8307-b559dee0ae66-kube-api-access-c2wv6\") on node \"ip-10-0-143-74.ec2.internal\" DevicePath \"\""
Sep 17 17:51:34.574487 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:51:34.574421    2575 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/07cf78e0-a8fe-43c6-8307-b559dee0ae66-util\") on node \"ip-10-0-143-74.ec2.internal\" DevicePath \"\""
Sep 17 17:51:34.574487 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:51:34.574434    2575 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/07cf78e0-a8fe-43c6-8307-b559dee0ae66-bundle\") on node \"ip-10-0-143-74.ec2.internal\" DevicePath \"\""
Sep 17 17:51:35.251837 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:51:35.251765    2575 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4c54qm9"
Sep 17 17:51:35.251837 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:51:35.251793    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4c54qm9" event={"ID":"07cf78e0-a8fe-43c6-8307-b559dee0ae66","Type":"ContainerDied","Data":"64ff08a5d7327eedaa71b8026fbd4f960d157b73aeeb7a25e08f673f677473ca"}
Sep 17 17:51:35.251837 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:51:35.251839    2575 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="64ff08a5d7327eedaa71b8026fbd4f960d157b73aeeb7a25e08f673f677473ca"
Sep 17 17:51:35.398389 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:51:35.398371    2575 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847asdtmz"
Sep 17 17:51:35.400573 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:51:35.400552    2575 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7hhmqk"
Sep 17 17:51:35.482313 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:51:35.482281    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/968beef5-2813-42c1-b94d-402cc2b23425-util\") pod \"968beef5-2813-42c1-b94d-402cc2b23425\" (UID: \"968beef5-2813-42c1-b94d-402cc2b23425\") "
Sep 17 17:51:35.482499 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:51:35.482379    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/968beef5-2813-42c1-b94d-402cc2b23425-bundle\") pod \"968beef5-2813-42c1-b94d-402cc2b23425\" (UID: \"968beef5-2813-42c1-b94d-402cc2b23425\") "
Sep 17 17:51:35.482499 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:51:35.482406    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/6f9584a2-c363-4a56-80b5-3ae4aa8ab6e7-bundle\") pod \"6f9584a2-c363-4a56-80b5-3ae4aa8ab6e7\" (UID: \"6f9584a2-c363-4a56-80b5-3ae4aa8ab6e7\") "
Sep 17 17:51:35.482499 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:51:35.482465    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-mwv4p\" (UniqueName: \"kubernetes.io/projected/968beef5-2813-42c1-b94d-402cc2b23425-kube-api-access-mwv4p\") pod \"968beef5-2813-42c1-b94d-402cc2b23425\" (UID: \"968beef5-2813-42c1-b94d-402cc2b23425\") "
Sep 17 17:51:35.482664 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:51:35.482509    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-mxv8l\" (UniqueName: \"kubernetes.io/projected/6f9584a2-c363-4a56-80b5-3ae4aa8ab6e7-kube-api-access-mxv8l\") pod \"6f9584a2-c363-4a56-80b5-3ae4aa8ab6e7\" (UID: \"6f9584a2-c363-4a56-80b5-3ae4aa8ab6e7\") "
Sep 17 17:51:35.482664 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:51:35.482537    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/6f9584a2-c363-4a56-80b5-3ae4aa8ab6e7-util\") pod \"6f9584a2-c363-4a56-80b5-3ae4aa8ab6e7\" (UID: \"6f9584a2-c363-4a56-80b5-3ae4aa8ab6e7\") "
Sep 17 17:51:35.482998 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:51:35.482970    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/6f9584a2-c363-4a56-80b5-3ae4aa8ab6e7-bundle" (OuterVolumeSpecName: "bundle") pod "6f9584a2-c363-4a56-80b5-3ae4aa8ab6e7" (UID: "6f9584a2-c363-4a56-80b5-3ae4aa8ab6e7"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 17 17:51:35.483180 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:51:35.483156    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/968beef5-2813-42c1-b94d-402cc2b23425-bundle" (OuterVolumeSpecName: "bundle") pod "968beef5-2813-42c1-b94d-402cc2b23425" (UID: "968beef5-2813-42c1-b94d-402cc2b23425"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 17 17:51:35.485085 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:51:35.485055    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/6f9584a2-c363-4a56-80b5-3ae4aa8ab6e7-kube-api-access-mxv8l" (OuterVolumeSpecName: "kube-api-access-mxv8l") pod "6f9584a2-c363-4a56-80b5-3ae4aa8ab6e7" (UID: "6f9584a2-c363-4a56-80b5-3ae4aa8ab6e7"). InnerVolumeSpecName "kube-api-access-mxv8l". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 17 17:51:35.485085 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:51:35.485070    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/968beef5-2813-42c1-b94d-402cc2b23425-kube-api-access-mwv4p" (OuterVolumeSpecName: "kube-api-access-mwv4p") pod "968beef5-2813-42c1-b94d-402cc2b23425" (UID: "968beef5-2813-42c1-b94d-402cc2b23425"). InnerVolumeSpecName "kube-api-access-mwv4p". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 17 17:51:35.490388 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:51:35.490353    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/968beef5-2813-42c1-b94d-402cc2b23425-util" (OuterVolumeSpecName: "util") pod "968beef5-2813-42c1-b94d-402cc2b23425" (UID: "968beef5-2813-42c1-b94d-402cc2b23425"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 17 17:51:35.490497 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:51:35.490477    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/6f9584a2-c363-4a56-80b5-3ae4aa8ab6e7-util" (OuterVolumeSpecName: "util") pod "6f9584a2-c363-4a56-80b5-3ae4aa8ab6e7" (UID: "6f9584a2-c363-4a56-80b5-3ae4aa8ab6e7"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 17 17:51:35.583949 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:51:35.583862    2575 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/968beef5-2813-42c1-b94d-402cc2b23425-util\") on node \"ip-10-0-143-74.ec2.internal\" DevicePath \"\""
Sep 17 17:51:35.583949 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:51:35.583903    2575 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/968beef5-2813-42c1-b94d-402cc2b23425-bundle\") on node \"ip-10-0-143-74.ec2.internal\" DevicePath \"\""
Sep 17 17:51:35.583949 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:51:35.583912    2575 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/6f9584a2-c363-4a56-80b5-3ae4aa8ab6e7-bundle\") on node \"ip-10-0-143-74.ec2.internal\" DevicePath \"\""
Sep 17 17:51:35.583949 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:51:35.583921    2575 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-mwv4p\" (UniqueName: \"kubernetes.io/projected/968beef5-2813-42c1-b94d-402cc2b23425-kube-api-access-mwv4p\") on node \"ip-10-0-143-74.ec2.internal\" DevicePath \"\""
Sep 17 17:51:35.583949 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:51:35.583930    2575 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-mxv8l\" (UniqueName: \"kubernetes.io/projected/6f9584a2-c363-4a56-80b5-3ae4aa8ab6e7-kube-api-access-mxv8l\") on node \"ip-10-0-143-74.ec2.internal\" DevicePath \"\""
Sep 17 17:51:35.583949 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:51:35.583939    2575 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/6f9584a2-c363-4a56-80b5-3ae4aa8ab6e7-util\") on node \"ip-10-0-143-74.ec2.internal\" DevicePath \"\""
Sep 17 17:51:36.258662 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:51:36.258632    2575 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847asdtmz"
Sep 17 17:51:36.259110 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:51:36.258636    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847asdtmz" event={"ID":"968beef5-2813-42c1-b94d-402cc2b23425","Type":"ContainerDied","Data":"20b462d3cc3412dc0ee26b6f3ace859d6bfdf8861e06ccb7da97478a10c86c6e"}
Sep 17 17:51:36.259110 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:51:36.258756    2575 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="20b462d3cc3412dc0ee26b6f3ace859d6bfdf8861e06ccb7da97478a10c86c6e"
Sep 17 17:51:36.260568 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:51:36.260551    2575 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7hhmqk"
Sep 17 17:51:36.260681 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:51:36.260569    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7hhmqk" event={"ID":"6f9584a2-c363-4a56-80b5-3ae4aa8ab6e7","Type":"ContainerDied","Data":"e4fd32336690bf77afadb1c513b356b183ff7c2136cb81c0adda377567c9e3c5"}
Sep 17 17:51:36.260681 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:51:36.260602    2575 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="e4fd32336690bf77afadb1c513b356b183ff7c2136cb81c0adda377567c9e3c5"
Sep 17 17:51:38.271463 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:51:38.271418    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6pc498" event={"ID":"9a95c99c-1cdf-428d-9a4e-87c0998c4b80","Type":"ContainerStarted","Data":"d5ceee5a1a3808d782abcf2292e47d73b5951865cee8b6325e658f1f7de315d5"}
Sep 17 17:51:39.277992 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:51:39.277956    2575 generic.go:356] "Generic (PLEG): container finished" podID="9a95c99c-1cdf-428d-9a4e-87c0998c4b80" containerID="d5ceee5a1a3808d782abcf2292e47d73b5951865cee8b6325e658f1f7de315d5" exitCode=0
Sep 17 17:51:39.278370 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:51:39.278013    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6pc498" event={"ID":"9a95c99c-1cdf-428d-9a4e-87c0998c4b80","Type":"ContainerDied","Data":"d5ceee5a1a3808d782abcf2292e47d73b5951865cee8b6325e658f1f7de315d5"}
Sep 17 17:51:40.285480 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:51:40.285425    2575 generic.go:356] "Generic (PLEG): container finished" podID="9a95c99c-1cdf-428d-9a4e-87c0998c4b80" containerID="3f93bf65d73271e96e4b535b5c9c80fc475c90c456255a9b041b4d43c2334d32" exitCode=0
Sep 17 17:51:40.285480 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:51:40.285473    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6pc498" event={"ID":"9a95c99c-1cdf-428d-9a4e-87c0998c4b80","Type":"ContainerDied","Data":"3f93bf65d73271e96e4b535b5c9c80fc475c90c456255a9b041b4d43c2334d32"}
Sep 17 17:51:41.416638 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:51:41.416611    2575 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6pc498"
Sep 17 17:51:41.535639 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:51:41.535606    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/9a95c99c-1cdf-428d-9a4e-87c0998c4b80-util\") pod \"9a95c99c-1cdf-428d-9a4e-87c0998c4b80\" (UID: \"9a95c99c-1cdf-428d-9a4e-87c0998c4b80\") "
Sep 17 17:51:41.535822 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:51:41.535659    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/9a95c99c-1cdf-428d-9a4e-87c0998c4b80-bundle\") pod \"9a95c99c-1cdf-428d-9a4e-87c0998c4b80\" (UID: \"9a95c99c-1cdf-428d-9a4e-87c0998c4b80\") "
Sep 17 17:51:41.535822 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:51:41.535707    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-hr2jg\" (UniqueName: \"kubernetes.io/projected/9a95c99c-1cdf-428d-9a4e-87c0998c4b80-kube-api-access-hr2jg\") pod \"9a95c99c-1cdf-428d-9a4e-87c0998c4b80\" (UID: \"9a95c99c-1cdf-428d-9a4e-87c0998c4b80\") "
Sep 17 17:51:41.536126 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:51:41.536098    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/9a95c99c-1cdf-428d-9a4e-87c0998c4b80-bundle" (OuterVolumeSpecName: "bundle") pod "9a95c99c-1cdf-428d-9a4e-87c0998c4b80" (UID: "9a95c99c-1cdf-428d-9a4e-87c0998c4b80"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 17 17:51:41.537809 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:51:41.537773    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/9a95c99c-1cdf-428d-9a4e-87c0998c4b80-kube-api-access-hr2jg" (OuterVolumeSpecName: "kube-api-access-hr2jg") pod "9a95c99c-1cdf-428d-9a4e-87c0998c4b80" (UID: "9a95c99c-1cdf-428d-9a4e-87c0998c4b80"). InnerVolumeSpecName "kube-api-access-hr2jg". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 17 17:51:41.540078 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:51:41.540043    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/9a95c99c-1cdf-428d-9a4e-87c0998c4b80-util" (OuterVolumeSpecName: "util") pod "9a95c99c-1cdf-428d-9a4e-87c0998c4b80" (UID: "9a95c99c-1cdf-428d-9a4e-87c0998c4b80"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 17 17:51:41.636202 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:51:41.636166    2575 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-hr2jg\" (UniqueName: \"kubernetes.io/projected/9a95c99c-1cdf-428d-9a4e-87c0998c4b80-kube-api-access-hr2jg\") on node \"ip-10-0-143-74.ec2.internal\" DevicePath \"\""
Sep 17 17:51:41.636202 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:51:41.636196    2575 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/9a95c99c-1cdf-428d-9a4e-87c0998c4b80-util\") on node \"ip-10-0-143-74.ec2.internal\" DevicePath \"\""
Sep 17 17:51:41.636202 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:51:41.636208    2575 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/9a95c99c-1cdf-428d-9a4e-87c0998c4b80-bundle\") on node \"ip-10-0-143-74.ec2.internal\" DevicePath \"\""
Sep 17 17:51:42.297048 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:51:42.297011    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6pc498" event={"ID":"9a95c99c-1cdf-428d-9a4e-87c0998c4b80","Type":"ContainerDied","Data":"a9ecc96406d41e23a50fcd92806238174d539209cd6e7a83a43224f37d697c1e"}
Sep 17 17:51:42.297048 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:51:42.297045    2575 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6pc498"
Sep 17 17:51:42.297250 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:51:42.297066    2575 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="a9ecc96406d41e23a50fcd92806238174d539209cd6e7a83a43224f37d697c1e"
Sep 17 17:51:47.791005 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:51:47.790966    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-djvzt_1d07e68d-9943-420f-a6a0-5f477792fabd/konnectivity-agent/0.log" limit=1024
Sep 17 17:51:50.057266 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:51:50.057227    2575 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-5ffb4b4ccb-gn5dk"]
Sep 17 17:51:50.057637 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:51:50.057584    2575 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="9a95c99c-1cdf-428d-9a4e-87c0998c4b80" containerName="pull"
Sep 17 17:51:50.057637 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:51:50.057596    2575 state_mem.go:107] "Deleted CPUSet assignment" podUID="9a95c99c-1cdf-428d-9a4e-87c0998c4b80" containerName="pull"
Sep 17 17:51:50.057637 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:51:50.057606    2575 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="9a95c99c-1cdf-428d-9a4e-87c0998c4b80" containerName="util"
Sep 17 17:51:50.057637 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:51:50.057611    2575 state_mem.go:107] "Deleted CPUSet assignment" podUID="9a95c99c-1cdf-428d-9a4e-87c0998c4b80" containerName="util"
Sep 17 17:51:50.057637 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:51:50.057620    2575 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="6f9584a2-c363-4a56-80b5-3ae4aa8ab6e7" containerName="util"
Sep 17 17:51:50.057637 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:51:50.057626    2575 state_mem.go:107] "Deleted CPUSet assignment" podUID="6f9584a2-c363-4a56-80b5-3ae4aa8ab6e7" containerName="util"
Sep 17 17:51:50.057637 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:51:50.057633    2575 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="968beef5-2813-42c1-b94d-402cc2b23425" containerName="pull"
Sep 17 17:51:50.057637 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:51:50.057638    2575 state_mem.go:107] "Deleted CPUSet assignment" podUID="968beef5-2813-42c1-b94d-402cc2b23425" containerName="pull"
Sep 17 17:51:50.057867 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:51:50.057646    2575 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="07cf78e0-a8fe-43c6-8307-b559dee0ae66" containerName="extract"
Sep 17 17:51:50.057867 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:51:50.057652    2575 state_mem.go:107] "Deleted CPUSet assignment" podUID="07cf78e0-a8fe-43c6-8307-b559dee0ae66" containerName="extract"
Sep 17 17:51:50.057867 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:51:50.057661    2575 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="968beef5-2813-42c1-b94d-402cc2b23425" containerName="util"
Sep 17 17:51:50.057867 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:51:50.057666    2575 state_mem.go:107] "Deleted CPUSet assignment" podUID="968beef5-2813-42c1-b94d-402cc2b23425" containerName="util"
Sep 17 17:51:50.057867 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:51:50.057673    2575 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="968beef5-2813-42c1-b94d-402cc2b23425" containerName="extract"
Sep 17 17:51:50.057867 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:51:50.057677    2575 state_mem.go:107] "Deleted CPUSet assignment" podUID="968beef5-2813-42c1-b94d-402cc2b23425" containerName="extract"
Sep 17 17:51:50.057867 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:51:50.057685    2575 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="07cf78e0-a8fe-43c6-8307-b559dee0ae66" containerName="pull"
Sep 17 17:51:50.057867 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:51:50.057690    2575 state_mem.go:107] "Deleted CPUSet assignment" podUID="07cf78e0-a8fe-43c6-8307-b559dee0ae66" containerName="pull"
Sep 17 17:51:50.057867 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:51:50.057699    2575 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="6f9584a2-c363-4a56-80b5-3ae4aa8ab6e7" containerName="extract"
Sep 17 17:51:50.057867 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:51:50.057704    2575 state_mem.go:107] "Deleted CPUSet assignment" podUID="6f9584a2-c363-4a56-80b5-3ae4aa8ab6e7" containerName="extract"
Sep 17 17:51:50.057867 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:51:50.057710    2575 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="6f9584a2-c363-4a56-80b5-3ae4aa8ab6e7" containerName="pull"
Sep 17 17:51:50.057867 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:51:50.057715    2575 state_mem.go:107] "Deleted CPUSet assignment" podUID="6f9584a2-c363-4a56-80b5-3ae4aa8ab6e7" containerName="pull"
Sep 17 17:51:50.057867 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:51:50.057720    2575 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="9a95c99c-1cdf-428d-9a4e-87c0998c4b80" containerName="extract"
Sep 17 17:51:50.057867 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:51:50.057725    2575 state_mem.go:107] "Deleted CPUSet assignment" podUID="9a95c99c-1cdf-428d-9a4e-87c0998c4b80" containerName="extract"
Sep 17 17:51:50.057867 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:51:50.057732    2575 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="07cf78e0-a8fe-43c6-8307-b559dee0ae66" containerName="util"
Sep 17 17:51:50.057867 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:51:50.057737    2575 state_mem.go:107] "Deleted CPUSet assignment" podUID="07cf78e0-a8fe-43c6-8307-b559dee0ae66" containerName="util"
Sep 17 17:51:50.057867 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:51:50.057784    2575 memory_manager.go:368] "RemoveStaleState removing state" podUID="07cf78e0-a8fe-43c6-8307-b559dee0ae66" containerName="extract"
Sep 17 17:51:50.057867 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:51:50.057795    2575 memory_manager.go:368] "RemoveStaleState removing state" podUID="968beef5-2813-42c1-b94d-402cc2b23425" containerName="extract"
Sep 17 17:51:50.057867 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:51:50.057802    2575 memory_manager.go:368] "RemoveStaleState removing state" podUID="9a95c99c-1cdf-428d-9a4e-87c0998c4b80" containerName="extract"
Sep 17 17:51:50.057867 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:51:50.057808    2575 memory_manager.go:368] "RemoveStaleState removing state" podUID="6f9584a2-c363-4a56-80b5-3ae4aa8ab6e7" containerName="extract"
Sep 17 17:51:50.066776 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:51:50.066754    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-5ffb4b4ccb-gn5dk"
Sep 17 17:51:50.070297 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:51:50.070275    2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"kube-root-ca.crt\""
Sep 17 17:51:50.071177 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:51:50.071161    2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"openshift-service-ca.crt\""
Sep 17 17:51:50.071243 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:51:50.071164    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"limitador-operator-controller-manager-dockercfg-gcdgb\""
Sep 17 17:51:50.076009 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:51:50.075988    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-5ffb4b4ccb-gn5dk"]
Sep 17 17:51:50.104638 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:51:50.104611    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-fvszt\" (UniqueName: \"kubernetes.io/projected/049f5402-5027-46b4-92c6-36c31dbb26b0-kube-api-access-fvszt\") pod \"limitador-operator-controller-manager-5ffb4b4ccb-gn5dk\" (UID: \"049f5402-5027-46b4-92c6-36c31dbb26b0\") " pod="kuadrant-system/limitador-operator-controller-manager-5ffb4b4ccb-gn5dk"
Sep 17 17:51:50.205087 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:51:50.205049    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fvszt\" (UniqueName: \"kubernetes.io/projected/049f5402-5027-46b4-92c6-36c31dbb26b0-kube-api-access-fvszt\") pod \"limitador-operator-controller-manager-5ffb4b4ccb-gn5dk\" (UID: \"049f5402-5027-46b4-92c6-36c31dbb26b0\") " pod="kuadrant-system/limitador-operator-controller-manager-5ffb4b4ccb-gn5dk"
Sep 17 17:51:50.214729 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:51:50.214694    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-fvszt\" (UniqueName: \"kubernetes.io/projected/049f5402-5027-46b4-92c6-36c31dbb26b0-kube-api-access-fvszt\") pod \"limitador-operator-controller-manager-5ffb4b4ccb-gn5dk\" (UID: \"049f5402-5027-46b4-92c6-36c31dbb26b0\") " pod="kuadrant-system/limitador-operator-controller-manager-5ffb4b4ccb-gn5dk"
Sep 17 17:51:50.378073 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:51:50.378048    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-5ffb4b4ccb-gn5dk"
Sep 17 17:51:50.425775 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:51:50.425729    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-djvzt_1d07e68d-9943-420f-a6a0-5f477792fabd/konnectivity-agent/0.log" limit=1024
Sep 17 17:51:50.658256 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:51:50.658227    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-5ffb4b4ccb-gn5dk"]
Sep 17 17:51:50.732727 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:51:50.732690    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-djvzt_1d07e68d-9943-420f-a6a0-5f477792fabd/konnectivity-agent/0.log" limit=1024
Sep 17 17:51:51.338942 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:51:51.338909    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-operator-controller-manager-5ffb4b4ccb-gn5dk" event={"ID":"049f5402-5027-46b4-92c6-36c31dbb26b0","Type":"ContainerStarted","Data":"d4078017720b8c68296b6db1fe324f733e5ee3810b29b4599ec4cd5e51b2e7d9"}
Sep 17 17:51:52.007525 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:51:52.007491    2575 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/dns-operator-controller-manager-65dc8f76f5-8s7cq"]
Sep 17 17:51:52.011276 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:51:52.011254    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/dns-operator-controller-manager-65dc8f76f5-8s7cq"
Sep 17 17:51:52.019496 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:51:52.019462    2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"dns-operator-controller-env\""
Sep 17 17:51:52.019628 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:51:52.019465    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"dns-operator-controller-manager-dockercfg-k9p64\""
Sep 17 17:51:52.055260 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:51:52.055224    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/dns-operator-controller-manager-65dc8f76f5-8s7cq"]
Sep 17 17:51:52.116933 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:51:52.116890    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xc9tn\" (UniqueName: \"kubernetes.io/projected/9c5e897b-2d08-490f-b194-ef54c1d56ba3-kube-api-access-xc9tn\") pod \"dns-operator-controller-manager-65dc8f76f5-8s7cq\" (UID: \"9c5e897b-2d08-490f-b194-ef54c1d56ba3\") " pod="kuadrant-system/dns-operator-controller-manager-65dc8f76f5-8s7cq"
Sep 17 17:51:52.218131 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:51:52.218091    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xc9tn\" (UniqueName: \"kubernetes.io/projected/9c5e897b-2d08-490f-b194-ef54c1d56ba3-kube-api-access-xc9tn\") pod \"dns-operator-controller-manager-65dc8f76f5-8s7cq\" (UID: \"9c5e897b-2d08-490f-b194-ef54c1d56ba3\") " pod="kuadrant-system/dns-operator-controller-manager-65dc8f76f5-8s7cq"
Sep 17 17:51:52.228550 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:51:52.228526    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-xc9tn\" (UniqueName: \"kubernetes.io/projected/9c5e897b-2d08-490f-b194-ef54c1d56ba3-kube-api-access-xc9tn\") pod \"dns-operator-controller-manager-65dc8f76f5-8s7cq\" (UID: \"9c5e897b-2d08-490f-b194-ef54c1d56ba3\") " pod="kuadrant-system/dns-operator-controller-manager-65dc8f76f5-8s7cq"
Sep 17 17:51:52.322705 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:51:52.322670    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/dns-operator-controller-manager-65dc8f76f5-8s7cq"
Sep 17 17:51:52.347626 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:51:52.347586    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-operator-controller-manager-5ffb4b4ccb-gn5dk" event={"ID":"049f5402-5027-46b4-92c6-36c31dbb26b0","Type":"ContainerStarted","Data":"da934ea3288a76cdfe96ce8ed017b09218e79212ad0409d4861f24a5b1f38adc"}
Sep 17 17:51:52.348095 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:51:52.347660    2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/limitador-operator-controller-manager-5ffb4b4ccb-gn5dk"
Sep 17 17:51:52.399271 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:51:52.399219    2575 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/limitador-operator-controller-manager-5ffb4b4ccb-gn5dk" podStartSLOduration=0.790872762 podStartE2EDuration="2.39920183s" totalImagesPullingTime="1.608329068s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-17 17:51:50 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-17 17:51:52.396876332 +0000 UTC m=+1032.027206307" watchObservedRunningTime="2026-09-17 17:51:52.39920183 +0000 UTC m=+1032.029531803"
Sep 17 17:51:52.604416 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:51:52.604366    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/dns-operator-controller-manager-65dc8f76f5-8s7cq"]
Sep 17 17:51:52.607064 ip-10-0-143-74 kubenswrapper[2575]: W0917 17:51:52.607037    2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9c5e897b_2d08_490f_b194_ef54c1d56ba3.slice/crio-cda5b48ca881cc7029aac7e1668b7cc6f48e773eae8e895e4db1cda08e28b414 WatchSource:0}: Error finding container cda5b48ca881cc7029aac7e1668b7cc6f48e773eae8e895e4db1cda08e28b414: Status 404 returned error can't find the container with id cda5b48ca881cc7029aac7e1668b7cc6f48e773eae8e895e4db1cda08e28b414
Sep 17 17:51:53.013751 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:51:53.013719    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-djvzt_1d07e68d-9943-420f-a6a0-5f477792fabd/konnectivity-agent/0.log" limit=1024
Sep 17 17:51:53.319524 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:51:53.319424    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-djvzt_1d07e68d-9943-420f-a6a0-5f477792fabd/konnectivity-agent/0.log" limit=1024
Sep 17 17:51:53.354032 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:51:53.354002    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/dns-operator-controller-manager-65dc8f76f5-8s7cq" event={"ID":"9c5e897b-2d08-490f-b194-ef54c1d56ba3","Type":"ContainerStarted","Data":"cda5b48ca881cc7029aac7e1668b7cc6f48e773eae8e895e4db1cda08e28b414"}
Sep 17 17:51:53.569411 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:51:53.569362    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-djvzt_1d07e68d-9943-420f-a6a0-5f477792fabd/konnectivity-agent/0.log" limit=1024
Sep 17 17:51:53.997420 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:51:53.997387    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-djvzt_1d07e68d-9943-420f-a6a0-5f477792fabd/konnectivity-agent/0.log" limit=1024
Sep 17 17:51:54.481960 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:51:54.481917    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-djvzt_1d07e68d-9943-420f-a6a0-5f477792fabd/konnectivity-agent/0.log" limit=1024
Sep 17 17:51:55.244900 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:51:55.244834    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-djvzt_1d07e68d-9943-420f-a6a0-5f477792fabd/konnectivity-agent/0.log" limit=1024
Sep 17 17:51:55.502756 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:51:55.502641    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-djvzt_1d07e68d-9943-420f-a6a0-5f477792fabd/konnectivity-agent/0.log" limit=1024
Sep 17 17:51:56.371833 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:51:56.371800    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/dns-operator-controller-manager-65dc8f76f5-8s7cq" event={"ID":"9c5e897b-2d08-490f-b194-ef54c1d56ba3","Type":"ContainerStarted","Data":"6248d413df63b791256baf053b20a2ef10106cd4298877451f216f5da3357091"}
Sep 17 17:51:56.372077 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:51:56.371872    2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/dns-operator-controller-manager-65dc8f76f5-8s7cq"
Sep 17 17:51:56.392321 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:51:56.392267    2575 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/dns-operator-controller-manager-65dc8f76f5-8s7cq" podStartSLOduration=2.413804126 podStartE2EDuration="5.39225296s" totalImagesPullingTime="2.978448834s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-17 17:51:51 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-17 17:51:56.390522039 +0000 UTC m=+1036.020852015" watchObservedRunningTime="2026-09-17 17:51:56.39225296 +0000 UTC m=+1036.022582933"
Sep 17 17:51:57.405028 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:51:57.404956    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-djvzt_1d07e68d-9943-420f-a6a0-5f477792fabd/konnectivity-agent/0.log" limit=1024
Sep 17 17:51:57.771824 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:51:57.771710    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-djvzt_1d07e68d-9943-420f-a6a0-5f477792fabd/konnectivity-agent/0.log" limit=1024
Sep 17 17:51:58.100673 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:51:58.100581    2575 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-6d4cb79c-25rvf"]
Sep 17 17:51:58.104034 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:51:58.104013    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-6d4cb79c-25rvf"
Sep 17 17:51:58.110919 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:51:58.110901    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"kuadrant-operator-controller-manager-dockercfg-8pjwg\""
Sep 17 17:51:58.122508 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:51:58.122478    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-6d4cb79c-25rvf"]
Sep 17 17:51:58.165132 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:51:58.165092    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/3cb0e8b5-8b54-49a6-aa86-0dcf1b07f4a2-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-6d4cb79c-25rvf\" (UID: \"3cb0e8b5-8b54-49a6-aa86-0dcf1b07f4a2\") " pod="kuadrant-system/kuadrant-operator-controller-manager-6d4cb79c-25rvf"
Sep 17 17:51:58.165308 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:51:58.165149    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-tz78p\" (UniqueName: \"kubernetes.io/projected/3cb0e8b5-8b54-49a6-aa86-0dcf1b07f4a2-kube-api-access-tz78p\") pod \"kuadrant-operator-controller-manager-6d4cb79c-25rvf\" (UID: \"3cb0e8b5-8b54-49a6-aa86-0dcf1b07f4a2\") " pod="kuadrant-system/kuadrant-operator-controller-manager-6d4cb79c-25rvf"
Sep 17 17:51:58.266161 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:51:58.266130    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/3cb0e8b5-8b54-49a6-aa86-0dcf1b07f4a2-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-6d4cb79c-25rvf\" (UID: \"3cb0e8b5-8b54-49a6-aa86-0dcf1b07f4a2\") " pod="kuadrant-system/kuadrant-operator-controller-manager-6d4cb79c-25rvf"
Sep 17 17:51:58.266340 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:51:58.266187    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tz78p\" (UniqueName: \"kubernetes.io/projected/3cb0e8b5-8b54-49a6-aa86-0dcf1b07f4a2-kube-api-access-tz78p\") pod \"kuadrant-operator-controller-manager-6d4cb79c-25rvf\" (UID: \"3cb0e8b5-8b54-49a6-aa86-0dcf1b07f4a2\") " pod="kuadrant-system/kuadrant-operator-controller-manager-6d4cb79c-25rvf"
Sep 17 17:51:58.266496 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:51:58.266476    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/3cb0e8b5-8b54-49a6-aa86-0dcf1b07f4a2-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-6d4cb79c-25rvf\" (UID: \"3cb0e8b5-8b54-49a6-aa86-0dcf1b07f4a2\") " pod="kuadrant-system/kuadrant-operator-controller-manager-6d4cb79c-25rvf"
Sep 17 17:51:58.278244 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:51:58.278223    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-tz78p\" (UniqueName: \"kubernetes.io/projected/3cb0e8b5-8b54-49a6-aa86-0dcf1b07f4a2-kube-api-access-tz78p\") pod \"kuadrant-operator-controller-manager-6d4cb79c-25rvf\" (UID: \"3cb0e8b5-8b54-49a6-aa86-0dcf1b07f4a2\") " pod="kuadrant-system/kuadrant-operator-controller-manager-6d4cb79c-25rvf"
Sep 17 17:51:58.413142 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:51:58.413105    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-6d4cb79c-25rvf"
Sep 17 17:51:58.693224 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:51:58.693199    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-6d4cb79c-25rvf"]
Sep 17 17:51:58.694714 ip-10-0-143-74 kubenswrapper[2575]: W0917 17:51:58.694689    2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3cb0e8b5_8b54_49a6_aa86_0dcf1b07f4a2.slice/crio-dd175c15705ea51037bc91cd8f39ac18b861722d9197da02fa11ee70a4969992 WatchSource:0}: Error finding container dd175c15705ea51037bc91cd8f39ac18b861722d9197da02fa11ee70a4969992: Status 404 returned error can't find the container with id dd175c15705ea51037bc91cd8f39ac18b861722d9197da02fa11ee70a4969992
Sep 17 17:51:59.103500 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:51:59.103364    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-djvzt_1d07e68d-9943-420f-a6a0-5f477792fabd/konnectivity-agent/0.log" limit=1024
Sep 17 17:51:59.387033 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:51:59.386995    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-6d4cb79c-25rvf" event={"ID":"3cb0e8b5-8b54-49a6-aa86-0dcf1b07f4a2","Type":"ContainerStarted","Data":"dd175c15705ea51037bc91cd8f39ac18b861722d9197da02fa11ee70a4969992"}
Sep 17 17:52:01.524781 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:52:01.524311    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-djvzt_1d07e68d-9943-420f-a6a0-5f477792fabd/konnectivity-agent/0.log" limit=1024
Sep 17 17:52:01.953479 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:52:01.953416    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-djvzt_1d07e68d-9943-420f-a6a0-5f477792fabd/konnectivity-agent/0.log" limit=1024
Sep 17 17:52:03.072401 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:52:03.072363    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-djvzt_1d07e68d-9943-420f-a6a0-5f477792fabd/konnectivity-agent/0.log" limit=1024
Sep 17 17:52:03.356193 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:52:03.356121    2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/limitador-operator-controller-manager-5ffb4b4ccb-gn5dk"
Sep 17 17:52:03.408716 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:52:03.408679    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-6d4cb79c-25rvf" event={"ID":"3cb0e8b5-8b54-49a6-aa86-0dcf1b07f4a2","Type":"ContainerStarted","Data":"d5934b2a28ed60acddd99d87dade79fc17c416ba76683da7bead7327d91be401"}
Sep 17 17:52:03.408890 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:52:03.408826    2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-controller-manager-6d4cb79c-25rvf"
Sep 17 17:52:03.433679 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:52:03.433631    2575 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-controller-manager-6d4cb79c-25rvf" podStartSLOduration=1.690952007 podStartE2EDuration="5.433616967s" totalImagesPullingTime="3.74266496s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-17 17:51:58 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-17 17:52:03.433004993 +0000 UTC m=+1043.063334969" watchObservedRunningTime="2026-09-17 17:52:03.433616967 +0000 UTC m=+1043.063946940"
Sep 17 17:52:03.555248 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:52:03.555216    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-djvzt_1d07e68d-9943-420f-a6a0-5f477792fabd/konnectivity-agent/0.log" limit=1024
Sep 17 17:52:05.193892 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:52:05.193852    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-djvzt_1d07e68d-9943-420f-a6a0-5f477792fabd/konnectivity-agent/0.log" limit=1024
Sep 17 17:52:05.443165 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:52:05.443130    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-djvzt_1d07e68d-9943-420f-a6a0-5f477792fabd/konnectivity-agent/0.log" limit=1024
Sep 17 17:52:07.378738 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:52:07.378700    2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/dns-operator-controller-manager-65dc8f76f5-8s7cq"
Sep 17 17:52:10.216938 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:52:10.216901    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-djvzt_1d07e68d-9943-420f-a6a0-5f477792fabd/konnectivity-agent/0.log" limit=1024
Sep 17 17:52:10.471995 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:52:10.471897    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-djvzt_1d07e68d-9943-420f-a6a0-5f477792fabd/konnectivity-agent/0.log" limit=1024
Sep 17 17:52:11.541577 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:52:11.541540    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-djvzt_1d07e68d-9943-420f-a6a0-5f477792fabd/konnectivity-agent/0.log" limit=1024
Sep 17 17:52:11.714738 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:52:11.714706    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-djvzt_1d07e68d-9943-420f-a6a0-5f477792fabd/konnectivity-agent/0.log" limit=1024
Sep 17 17:52:14.416857 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:52:14.416824    2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/kuadrant-operator-controller-manager-6d4cb79c-25rvf"
Sep 17 17:52:19.089510 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:52:19.089468    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-djvzt_1d07e68d-9943-420f-a6a0-5f477792fabd/konnectivity-agent/0.log" limit=1024
Sep 17 17:52:19.213093 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:52:19.213046    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-djvzt_1d07e68d-9943-420f-a6a0-5f477792fabd/konnectivity-agent/0.log" limit=1024
Sep 17 17:52:19.509661 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:52:19.509617    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-djvzt_1d07e68d-9943-420f-a6a0-5f477792fabd/konnectivity-agent/0.log" limit=1024
Sep 17 17:52:21.291529 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:52:21.291497    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-djvzt_1d07e68d-9943-420f-a6a0-5f477792fabd/konnectivity-agent/0.log" limit=1024
Sep 17 17:52:21.434627 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:52:21.434593    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-djvzt_1d07e68d-9943-420f-a6a0-5f477792fabd/konnectivity-agent/0.log" limit=1024
Sep 17 17:52:21.779487 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:52:21.779438    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-djvzt_1d07e68d-9943-420f-a6a0-5f477792fabd/konnectivity-agent/0.log" limit=1024
Sep 17 17:52:22.123520 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:52:22.123488    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-djvzt_1d07e68d-9943-420f-a6a0-5f477792fabd/konnectivity-agent/0.log" limit=1024
Sep 17 17:52:24.454332 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:52:24.454295    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-djvzt_1d07e68d-9943-420f-a6a0-5f477792fabd/konnectivity-agent/0.log" limit=1024
Sep 17 17:52:24.576134 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:52:24.576087    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-djvzt_1d07e68d-9943-420f-a6a0-5f477792fabd/konnectivity-agent/0.log" limit=1024
Sep 17 17:52:29.580341 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:52:29.580313    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-djvzt_1d07e68d-9943-420f-a6a0-5f477792fabd/konnectivity-agent/0.log" limit=1024
Sep 17 17:52:29.743324 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:52:29.743288    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-djvzt_1d07e68d-9943-420f-a6a0-5f477792fabd/konnectivity-agent/0.log" limit=1024
Sep 17 17:52:38.408552 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:52:38.408466    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-djvzt_1d07e68d-9943-420f-a6a0-5f477792fabd/konnectivity-agent/0.log" limit=1024
Sep 17 17:52:38.531392 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:52:38.531354    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-djvzt_1d07e68d-9943-420f-a6a0-5f477792fabd/konnectivity-agent/0.log" limit=1024
Sep 17 17:52:38.648325 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:52:38.648286    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-djvzt_1d07e68d-9943-420f-a6a0-5f477792fabd/konnectivity-agent/0.log" limit=1024
Sep 17 17:52:47.038607 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:52:47.038573    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-djvzt_1d07e68d-9943-420f-a6a0-5f477792fabd/konnectivity-agent/0.log" limit=1024
Sep 17 17:52:47.062729 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:52:47.062695    2575 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/limitador-limitador-8588749977-pxqz2"]
Sep 17 17:52:47.065698 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:52:47.065682    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-8588749977-pxqz2"
Sep 17 17:52:47.070615 ip-10-0-143-74 kubenswrapper[2575]: E0917 17:52:47.070588    2575 reflector.go:205] "Failed to watch" err="failed to list *v1.Secret: secrets \"default-dockercfg-s4fpv\" is forbidden: User \"system:node:ip-10-0-143-74.ec2.internal\" cannot list resource \"secrets\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-143-74.ec2.internal' and this object" logger="UnhandledError" reflector="object-\"kuadrant-system\"/\"default-dockercfg-s4fpv\"" type="*v1.Secret"
Sep 17 17:52:47.070836 ip-10-0-143-74 kubenswrapper[2575]: E0917 17:52:47.070816    2575 status_manager.go:1018] "Failed to get status for pod" err="pods \"limitador-limitador-8588749977-pxqz2\" is forbidden: User \"system:node:ip-10-0-143-74.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-143-74.ec2.internal' and this object" podUID="51020090-cb3b-4e16-86f3-6158afb1f9f4" pod="kuadrant-system/limitador-limitador-8588749977-pxqz2"
Sep 17 17:52:47.070900 ip-10-0-143-74 kubenswrapper[2575]: E0917 17:52:47.070851    2575 reflector.go:205] "Failed to watch" err="failed to list *v1.ConfigMap: configmaps \"limitador-limits-config-limitador\" is forbidden: User \"system:node:ip-10-0-143-74.ec2.internal\" cannot list resource \"configmaps\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-143-74.ec2.internal' and this object" logger="UnhandledError" reflector="object-\"kuadrant-system\"/\"limitador-limits-config-limitador\"" type="*v1.ConfigMap"
Sep 17 17:52:47.105185 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:52:47.105144    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-8588749977-pxqz2"]
Sep 17 17:52:47.161322 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:52:47.161285    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-8588749977-pxqz2"]
Sep 17 17:52:47.178426 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:52:47.178393    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/51020090-cb3b-4e16-86f3-6158afb1f9f4-config-file\") pod \"limitador-limitador-8588749977-pxqz2\" (UID: \"51020090-cb3b-4e16-86f3-6158afb1f9f4\") " pod="kuadrant-system/limitador-limitador-8588749977-pxqz2"
Sep 17 17:52:47.178426 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:52:47.178426    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4f5m4\" (UniqueName: \"kubernetes.io/projected/51020090-cb3b-4e16-86f3-6158afb1f9f4-kube-api-access-4f5m4\") pod \"limitador-limitador-8588749977-pxqz2\" (UID: \"51020090-cb3b-4e16-86f3-6158afb1f9f4\") " pod="kuadrant-system/limitador-limitador-8588749977-pxqz2"
Sep 17 17:52:47.185587 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:52:47.185556    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-djvzt_1d07e68d-9943-420f-a6a0-5f477792fabd/konnectivity-agent/0.log" limit=1024
Sep 17 17:52:47.279775 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:52:47.279743    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/51020090-cb3b-4e16-86f3-6158afb1f9f4-config-file\") pod \"limitador-limitador-8588749977-pxqz2\" (UID: \"51020090-cb3b-4e16-86f3-6158afb1f9f4\") " pod="kuadrant-system/limitador-limitador-8588749977-pxqz2"
Sep 17 17:52:47.279775 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:52:47.279778    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4f5m4\" (UniqueName: \"kubernetes.io/projected/51020090-cb3b-4e16-86f3-6158afb1f9f4-kube-api-access-4f5m4\") pod \"limitador-limitador-8588749977-pxqz2\" (UID: \"51020090-cb3b-4e16-86f3-6158afb1f9f4\") " pod="kuadrant-system/limitador-limitador-8588749977-pxqz2"
Sep 17 17:52:47.296205 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:52:47.296134    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-4f5m4\" (UniqueName: \"kubernetes.io/projected/51020090-cb3b-4e16-86f3-6158afb1f9f4-kube-api-access-4f5m4\") pod \"limitador-limitador-8588749977-pxqz2\" (UID: \"51020090-cb3b-4e16-86f3-6158afb1f9f4\") " pod="kuadrant-system/limitador-limitador-8588749977-pxqz2"
Sep 17 17:52:47.329396 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:52:47.329358    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-djvzt_1d07e68d-9943-420f-a6a0-5f477792fabd/konnectivity-agent/0.log" limit=1024
Sep 17 17:52:47.931558 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:52:47.931526    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-djvzt_1d07e68d-9943-420f-a6a0-5f477792fabd/konnectivity-agent/0.log" limit=1024
Sep 17 17:52:47.932167 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:52:47.932148    2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"limitador-limits-config-limitador\""
Sep 17 17:52:47.941335 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:52:47.941309    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/51020090-cb3b-4e16-86f3-6158afb1f9f4-config-file\") pod \"limitador-limitador-8588749977-pxqz2\" (UID: \"51020090-cb3b-4e16-86f3-6158afb1f9f4\") " pod="kuadrant-system/limitador-limitador-8588749977-pxqz2"
Sep 17 17:52:48.033338 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:52:48.033302    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-djvzt_1d07e68d-9943-420f-a6a0-5f477792fabd/konnectivity-agent/0.log" limit=1024
Sep 17 17:52:48.568342 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:52:48.568310    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"default-dockercfg-s4fpv\""
Sep 17 17:52:48.576906 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:52:48.576885    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-8588749977-pxqz2"
Sep 17 17:52:48.850686 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:52:48.850659    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-8588749977-pxqz2"]
Sep 17 17:52:48.852599 ip-10-0-143-74 kubenswrapper[2575]: W0917 17:52:48.852572    2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod51020090_cb3b_4e16_86f3_6158afb1f9f4.slice/crio-0faed4152edceceed6c254ea40d234b98c95bc838298999cf99d256cecb06ee9 WatchSource:0}: Error finding container 0faed4152edceceed6c254ea40d234b98c95bc838298999cf99d256cecb06ee9: Status 404 returned error can't find the container with id 0faed4152edceceed6c254ea40d234b98c95bc838298999cf99d256cecb06ee9
Sep 17 17:52:49.654653 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:52:49.654616    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-8588749977-pxqz2" event={"ID":"51020090-cb3b-4e16-86f3-6158afb1f9f4","Type":"ContainerStarted","Data":"0faed4152edceceed6c254ea40d234b98c95bc838298999cf99d256cecb06ee9"}
Sep 17 17:52:50.661788 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:52:50.661756    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-8588749977-pxqz2" event={"ID":"51020090-cb3b-4e16-86f3-6158afb1f9f4","Type":"ContainerStarted","Data":"6c496432ac0dea2eb7bfe7025986d31e7736051a8083c165e875b03400715f09"}
Sep 17 17:52:50.662174 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:52:50.661885    2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/limitador-limitador-8588749977-pxqz2"
Sep 17 17:52:50.686294 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:52:50.683111    2575 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/limitador-limitador-8588749977-pxqz2" podStartSLOduration=2.61055846 podStartE2EDuration="3.683090054s" totalImagesPullingTime="1.072531594s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-17 17:52:47 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-17 17:52:50.680501703 +0000 UTC m=+1090.310831672" watchObservedRunningTime="2026-09-17 17:52:50.683090054 +0000 UTC m=+1090.313420029"
Sep 17 17:53:00.925257 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:53:00.925218    2575 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/limitador-limitador-8588749977-pxqz2"]
Sep 17 17:53:00.925774 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:53:00.925422    2575 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/limitador-limitador-8588749977-pxqz2" podUID="51020090-cb3b-4e16-86f3-6158afb1f9f4" containerName="limitador" containerID="cri-o://6c496432ac0dea2eb7bfe7025986d31e7736051a8083c165e875b03400715f09" gracePeriod=30
Sep 17 17:53:00.926156 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:53:00.926121    2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/limitador-limitador-8588749977-pxqz2"
Sep 17 17:53:01.719916 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:53:01.719884    2575 generic.go:356] "Generic (PLEG): container finished" podID="51020090-cb3b-4e16-86f3-6158afb1f9f4" containerID="6c496432ac0dea2eb7bfe7025986d31e7736051a8083c165e875b03400715f09" exitCode=0
Sep 17 17:53:01.720094 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:53:01.719955    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-8588749977-pxqz2" event={"ID":"51020090-cb3b-4e16-86f3-6158afb1f9f4","Type":"ContainerDied","Data":"6c496432ac0dea2eb7bfe7025986d31e7736051a8083c165e875b03400715f09"}
Sep 17 17:53:01.868750 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:53:01.868683    2575 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-8588749977-pxqz2"
Sep 17 17:53:01.904362 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:53:01.904324    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-4f5m4\" (UniqueName: \"kubernetes.io/projected/51020090-cb3b-4e16-86f3-6158afb1f9f4-kube-api-access-4f5m4\") pod \"51020090-cb3b-4e16-86f3-6158afb1f9f4\" (UID: \"51020090-cb3b-4e16-86f3-6158afb1f9f4\") "
Sep 17 17:53:01.904528 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:53:01.904389    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/51020090-cb3b-4e16-86f3-6158afb1f9f4-config-file\") pod \"51020090-cb3b-4e16-86f3-6158afb1f9f4\" (UID: \"51020090-cb3b-4e16-86f3-6158afb1f9f4\") "
Sep 17 17:53:01.904748 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:53:01.904721    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/51020090-cb3b-4e16-86f3-6158afb1f9f4-config-file" (OuterVolumeSpecName: "config-file") pod "51020090-cb3b-4e16-86f3-6158afb1f9f4" (UID: "51020090-cb3b-4e16-86f3-6158afb1f9f4"). InnerVolumeSpecName "config-file". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 17 17:53:01.906508 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:53:01.906483    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/51020090-cb3b-4e16-86f3-6158afb1f9f4-kube-api-access-4f5m4" (OuterVolumeSpecName: "kube-api-access-4f5m4") pod "51020090-cb3b-4e16-86f3-6158afb1f9f4" (UID: "51020090-cb3b-4e16-86f3-6158afb1f9f4"). InnerVolumeSpecName "kube-api-access-4f5m4". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 17 17:53:02.005189 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:53:02.005103    2575 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-4f5m4\" (UniqueName: \"kubernetes.io/projected/51020090-cb3b-4e16-86f3-6158afb1f9f4-kube-api-access-4f5m4\") on node \"ip-10-0-143-74.ec2.internal\" DevicePath \"\""
Sep 17 17:53:02.005189 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:53:02.005135    2575 reconciler_common.go:299] "Volume detached for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/51020090-cb3b-4e16-86f3-6158afb1f9f4-config-file\") on node \"ip-10-0-143-74.ec2.internal\" DevicePath \"\""
Sep 17 17:53:02.726636 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:53:02.726607    2575 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-8588749977-pxqz2"
Sep 17 17:53:02.726833 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:53:02.726642    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-8588749977-pxqz2" event={"ID":"51020090-cb3b-4e16-86f3-6158afb1f9f4","Type":"ContainerDied","Data":"0faed4152edceceed6c254ea40d234b98c95bc838298999cf99d256cecb06ee9"}
Sep 17 17:53:02.726833 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:53:02.726709    2575 scope.go:117] "RemoveContainer" containerID="6c496432ac0dea2eb7bfe7025986d31e7736051a8083c165e875b03400715f09"
Sep 17 17:53:02.751519 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:53:02.751489    2575 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/limitador-limitador-8588749977-pxqz2"]
Sep 17 17:53:02.757783 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:53:02.757759    2575 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/limitador-limitador-8588749977-pxqz2"]
Sep 17 17:53:02.967994 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:53:02.967961    2575 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="51020090-cb3b-4e16-86f3-6158afb1f9f4" path="/var/lib/kubelet/pods/51020090-cb3b-4e16-86f3-6158afb1f9f4/volumes"
Sep 17 17:53:07.052390 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:53:07.052355    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-djvzt_1d07e68d-9943-420f-a6a0-5f477792fabd/konnectivity-agent/0.log" limit=1024
Sep 17 17:53:07.201031 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:53:07.200990    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-djvzt_1d07e68d-9943-420f-a6a0-5f477792fabd/konnectivity-agent/0.log" limit=1024
Sep 17 17:53:07.311946 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:53:07.311863    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-djvzt_1d07e68d-9943-420f-a6a0-5f477792fabd/konnectivity-agent/0.log" limit=1024
Sep 17 17:53:16.736369 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:53:16.736332    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-djvzt_1d07e68d-9943-420f-a6a0-5f477792fabd/konnectivity-agent/0.log" limit=1024
Sep 17 17:53:16.835551 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:53:16.835512    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-djvzt_1d07e68d-9943-420f-a6a0-5f477792fabd/konnectivity-agent/0.log" limit=1024
Sep 17 17:53:32.440966 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:53:32.440925    2575 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/istiod-openshift-gateway-5677ccb49b-ft57x"]
Sep 17 17:53:32.441554 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:53:32.441495    2575 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="51020090-cb3b-4e16-86f3-6158afb1f9f4" containerName="limitador"
Sep 17 17:53:32.441554 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:53:32.441516    2575 state_mem.go:107] "Deleted CPUSet assignment" podUID="51020090-cb3b-4e16-86f3-6158afb1f9f4" containerName="limitador"
Sep 17 17:53:32.441680 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:53:32.441617    2575 memory_manager.go:368] "RemoveStaleState removing state" podUID="51020090-cb3b-4e16-86f3-6158afb1f9f4" containerName="limitador"
Sep 17 17:53:32.444699 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:53:32.444679    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-ft57x"
Sep 17 17:53:32.457412 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:53:32.457387    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-5677ccb49b-ft57x"]
Sep 17 17:53:32.579820 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:53:32.579783    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/7ca824c6-1031-48be-af6a-0d0b5de1aa51-local-certs\") pod \"istiod-openshift-gateway-5677ccb49b-ft57x\" (UID: \"7ca824c6-1031-48be-af6a-0d0b5de1aa51\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-ft57x"
Sep 17 17:53:32.579820 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:53:32.579821    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/7ca824c6-1031-48be-af6a-0d0b5de1aa51-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-5677ccb49b-ft57x\" (UID: \"7ca824c6-1031-48be-af6a-0d0b5de1aa51\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-ft57x"
Sep 17 17:53:32.580033 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:53:32.579883    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/7ca824c6-1031-48be-af6a-0d0b5de1aa51-cacerts\") pod \"istiod-openshift-gateway-5677ccb49b-ft57x\" (UID: \"7ca824c6-1031-48be-af6a-0d0b5de1aa51\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-ft57x"
Sep 17 17:53:32.580033 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:53:32.579939    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/7ca824c6-1031-48be-af6a-0d0b5de1aa51-istio-kubeconfig\") pod \"istiod-openshift-gateway-5677ccb49b-ft57x\" (UID: \"7ca824c6-1031-48be-af6a-0d0b5de1aa51\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-ft57x"
Sep 17 17:53:32.580033 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:53:32.579984    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ngm9p\" (UniqueName: \"kubernetes.io/projected/7ca824c6-1031-48be-af6a-0d0b5de1aa51-kube-api-access-ngm9p\") pod \"istiod-openshift-gateway-5677ccb49b-ft57x\" (UID: \"7ca824c6-1031-48be-af6a-0d0b5de1aa51\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-ft57x"
Sep 17 17:53:32.580163 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:53:32.580039    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/7ca824c6-1031-48be-af6a-0d0b5de1aa51-istio-token\") pod \"istiod-openshift-gateway-5677ccb49b-ft57x\" (UID: \"7ca824c6-1031-48be-af6a-0d0b5de1aa51\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-ft57x"
Sep 17 17:53:32.580163 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:53:32.580073    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/7ca824c6-1031-48be-af6a-0d0b5de1aa51-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-5677ccb49b-ft57x\" (UID: \"7ca824c6-1031-48be-af6a-0d0b5de1aa51\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-ft57x"
Sep 17 17:53:32.680640 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:53:32.680591    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ngm9p\" (UniqueName: \"kubernetes.io/projected/7ca824c6-1031-48be-af6a-0d0b5de1aa51-kube-api-access-ngm9p\") pod \"istiod-openshift-gateway-5677ccb49b-ft57x\" (UID: \"7ca824c6-1031-48be-af6a-0d0b5de1aa51\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-ft57x"
Sep 17 17:53:32.680640 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:53:32.680648    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/7ca824c6-1031-48be-af6a-0d0b5de1aa51-istio-token\") pod \"istiod-openshift-gateway-5677ccb49b-ft57x\" (UID: \"7ca824c6-1031-48be-af6a-0d0b5de1aa51\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-ft57x"
Sep 17 17:53:32.680908 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:53:32.680680    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/7ca824c6-1031-48be-af6a-0d0b5de1aa51-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-5677ccb49b-ft57x\" (UID: \"7ca824c6-1031-48be-af6a-0d0b5de1aa51\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-ft57x"
Sep 17 17:53:32.680908 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:53:32.680712    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/7ca824c6-1031-48be-af6a-0d0b5de1aa51-local-certs\") pod \"istiod-openshift-gateway-5677ccb49b-ft57x\" (UID: \"7ca824c6-1031-48be-af6a-0d0b5de1aa51\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-ft57x"
Sep 17 17:53:32.680908 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:53:32.680738    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/7ca824c6-1031-48be-af6a-0d0b5de1aa51-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-5677ccb49b-ft57x\" (UID: \"7ca824c6-1031-48be-af6a-0d0b5de1aa51\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-ft57x"
Sep 17 17:53:32.680908 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:53:32.680789    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/7ca824c6-1031-48be-af6a-0d0b5de1aa51-cacerts\") pod \"istiod-openshift-gateway-5677ccb49b-ft57x\" (UID: \"7ca824c6-1031-48be-af6a-0d0b5de1aa51\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-ft57x"
Sep 17 17:53:32.680908 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:53:32.680854    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/7ca824c6-1031-48be-af6a-0d0b5de1aa51-istio-kubeconfig\") pod \"istiod-openshift-gateway-5677ccb49b-ft57x\" (UID: \"7ca824c6-1031-48be-af6a-0d0b5de1aa51\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-ft57x"
Sep 17 17:53:32.681532 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:53:32.681505    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/7ca824c6-1031-48be-af6a-0d0b5de1aa51-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-5677ccb49b-ft57x\" (UID: \"7ca824c6-1031-48be-af6a-0d0b5de1aa51\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-ft57x"
Sep 17 17:53:32.683116 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:53:32.683090    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/7ca824c6-1031-48be-af6a-0d0b5de1aa51-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-5677ccb49b-ft57x\" (UID: \"7ca824c6-1031-48be-af6a-0d0b5de1aa51\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-ft57x"
Sep 17 17:53:32.683216 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:53:32.683094    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/7ca824c6-1031-48be-af6a-0d0b5de1aa51-local-certs\") pod \"istiod-openshift-gateway-5677ccb49b-ft57x\" (UID: \"7ca824c6-1031-48be-af6a-0d0b5de1aa51\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-ft57x"
Sep 17 17:53:32.683469 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:53:32.683433    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/7ca824c6-1031-48be-af6a-0d0b5de1aa51-istio-kubeconfig\") pod \"istiod-openshift-gateway-5677ccb49b-ft57x\" (UID: \"7ca824c6-1031-48be-af6a-0d0b5de1aa51\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-ft57x"
Sep 17 17:53:32.683601 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:53:32.683579    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/7ca824c6-1031-48be-af6a-0d0b5de1aa51-cacerts\") pod \"istiod-openshift-gateway-5677ccb49b-ft57x\" (UID: \"7ca824c6-1031-48be-af6a-0d0b5de1aa51\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-ft57x"
Sep 17 17:53:32.690335 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:53:32.690317    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-ngm9p\" (UniqueName: \"kubernetes.io/projected/7ca824c6-1031-48be-af6a-0d0b5de1aa51-kube-api-access-ngm9p\") pod \"istiod-openshift-gateway-5677ccb49b-ft57x\" (UID: \"7ca824c6-1031-48be-af6a-0d0b5de1aa51\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-ft57x"
Sep 17 17:53:32.690707 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:53:32.690687    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/7ca824c6-1031-48be-af6a-0d0b5de1aa51-istio-token\") pod \"istiod-openshift-gateway-5677ccb49b-ft57x\" (UID: \"7ca824c6-1031-48be-af6a-0d0b5de1aa51\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-ft57x"
Sep 17 17:53:32.753611 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:53:32.753530    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-ft57x"
Sep 17 17:53:33.049560 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:53:33.049470    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-5677ccb49b-ft57x"]
Sep 17 17:53:33.052845 ip-10-0-143-74 kubenswrapper[2575]: W0917 17:53:33.052810    2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7ca824c6_1031_48be_af6a_0d0b5de1aa51.slice/crio-f91cc6d084682c3b75320cfe253c0ef7c5639d9f1b9b56d58776de4c37129a22 WatchSource:0}: Error finding container f91cc6d084682c3b75320cfe253c0ef7c5639d9f1b9b56d58776de4c37129a22: Status 404 returned error can't find the container with id f91cc6d084682c3b75320cfe253c0ef7c5639d9f1b9b56d58776de4c37129a22
Sep 17 17:53:33.054890 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:53:33.054850    2575 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236216Ki","pods":"250"}
Sep 17 17:53:33.054975 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:53:33.054934    2575 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236216Ki","pods":"250"}
Sep 17 17:53:33.885727 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:53:33.885688    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-ft57x" event={"ID":"7ca824c6-1031-48be-af6a-0d0b5de1aa51","Type":"ContainerStarted","Data":"2470e1a6bf38aadd6b4bea64c294f100559e5ffa4b187e40da14c81d228aafce"}
Sep 17 17:53:33.886104 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:53:33.885749    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-ft57x" event={"ID":"7ca824c6-1031-48be-af6a-0d0b5de1aa51","Type":"ContainerStarted","Data":"f91cc6d084682c3b75320cfe253c0ef7c5639d9f1b9b56d58776de4c37129a22"}
Sep 17 17:53:33.886104 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:53:33.885862    2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-ft57x"
Sep 17 17:53:33.957818 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:53:33.957766    2575 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-ft57x" podStartSLOduration=1.957746251 podStartE2EDuration="1.957746251s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-17 17:53:32 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-17 17:53:33.955352693 +0000 UTC m=+1133.585682661" watchObservedRunningTime="2026-09-17 17:53:33.957746251 +0000 UTC m=+1133.588076225"
Sep 17 17:53:34.893380 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:53:34.893349    2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-ft57x"
Sep 17 17:53:34.968523 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:53:34.968485    2575 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7b7995d5cd-mbwr4"]
Sep 17 17:53:34.968700 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:53:34.968678    2575 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-mbwr4" podUID="449a6010-6e6b-4fa8-aeb6-570e53fc9081" containerName="discovery" containerID="cri-o://7a0874efd22cda87acd4f3ca1a804886936ccb00790b2d63061231d68325312e" gracePeriod=30
Sep 17 17:53:35.220112 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:53:35.220085    2575 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-mbwr4"
Sep 17 17:53:35.404152 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:53:35.404115    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/449a6010-6e6b-4fa8-aeb6-570e53fc9081-local-certs\") pod \"449a6010-6e6b-4fa8-aeb6-570e53fc9081\" (UID: \"449a6010-6e6b-4fa8-aeb6-570e53fc9081\") "
Sep 17 17:53:35.404323 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:53:35.404163    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/449a6010-6e6b-4fa8-aeb6-570e53fc9081-istio-csr-ca-configmap\") pod \"449a6010-6e6b-4fa8-aeb6-570e53fc9081\" (UID: \"449a6010-6e6b-4fa8-aeb6-570e53fc9081\") "
Sep 17 17:53:35.404323 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:53:35.404222    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/449a6010-6e6b-4fa8-aeb6-570e53fc9081-istio-token\") pod \"449a6010-6e6b-4fa8-aeb6-570e53fc9081\" (UID: \"449a6010-6e6b-4fa8-aeb6-570e53fc9081\") "
Sep 17 17:53:35.404323 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:53:35.404245    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-xppgz\" (UniqueName: \"kubernetes.io/projected/449a6010-6e6b-4fa8-aeb6-570e53fc9081-kube-api-access-xppgz\") pod \"449a6010-6e6b-4fa8-aeb6-570e53fc9081\" (UID: \"449a6010-6e6b-4fa8-aeb6-570e53fc9081\") "
Sep 17 17:53:35.404323 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:53:35.404300    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/449a6010-6e6b-4fa8-aeb6-570e53fc9081-istio-csr-dns-cert\") pod \"449a6010-6e6b-4fa8-aeb6-570e53fc9081\" (UID: \"449a6010-6e6b-4fa8-aeb6-570e53fc9081\") "
Sep 17 17:53:35.404534 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:53:35.404372    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/449a6010-6e6b-4fa8-aeb6-570e53fc9081-cacerts\") pod \"449a6010-6e6b-4fa8-aeb6-570e53fc9081\" (UID: \"449a6010-6e6b-4fa8-aeb6-570e53fc9081\") "
Sep 17 17:53:35.404534 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:53:35.404397    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/449a6010-6e6b-4fa8-aeb6-570e53fc9081-istio-kubeconfig\") pod \"449a6010-6e6b-4fa8-aeb6-570e53fc9081\" (UID: \"449a6010-6e6b-4fa8-aeb6-570e53fc9081\") "
Sep 17 17:53:35.404899 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:53:35.404834    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/449a6010-6e6b-4fa8-aeb6-570e53fc9081-istio-csr-ca-configmap" (OuterVolumeSpecName: "istio-csr-ca-configmap") pod "449a6010-6e6b-4fa8-aeb6-570e53fc9081" (UID: "449a6010-6e6b-4fa8-aeb6-570e53fc9081"). InnerVolumeSpecName "istio-csr-ca-configmap". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 17 17:53:35.406678 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:53:35.406650    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/449a6010-6e6b-4fa8-aeb6-570e53fc9081-istio-token" (OuterVolumeSpecName: "istio-token") pod "449a6010-6e6b-4fa8-aeb6-570e53fc9081" (UID: "449a6010-6e6b-4fa8-aeb6-570e53fc9081"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 17 17:53:35.406782 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:53:35.406690    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/449a6010-6e6b-4fa8-aeb6-570e53fc9081-cacerts" (OuterVolumeSpecName: "cacerts") pod "449a6010-6e6b-4fa8-aeb6-570e53fc9081" (UID: "449a6010-6e6b-4fa8-aeb6-570e53fc9081"). InnerVolumeSpecName "cacerts". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 17 17:53:35.406892 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:53:35.406875    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/449a6010-6e6b-4fa8-aeb6-570e53fc9081-local-certs" (OuterVolumeSpecName: "local-certs") pod "449a6010-6e6b-4fa8-aeb6-570e53fc9081" (UID: "449a6010-6e6b-4fa8-aeb6-570e53fc9081"). InnerVolumeSpecName "local-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 17 17:53:35.406957 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:53:35.406926    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/449a6010-6e6b-4fa8-aeb6-570e53fc9081-kube-api-access-xppgz" (OuterVolumeSpecName: "kube-api-access-xppgz") pod "449a6010-6e6b-4fa8-aeb6-570e53fc9081" (UID: "449a6010-6e6b-4fa8-aeb6-570e53fc9081"). InnerVolumeSpecName "kube-api-access-xppgz". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 17 17:53:35.407147 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:53:35.407127    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/449a6010-6e6b-4fa8-aeb6-570e53fc9081-istio-kubeconfig" (OuterVolumeSpecName: "istio-kubeconfig") pod "449a6010-6e6b-4fa8-aeb6-570e53fc9081" (UID: "449a6010-6e6b-4fa8-aeb6-570e53fc9081"). InnerVolumeSpecName "istio-kubeconfig". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 17 17:53:35.407236 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:53:35.407155    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/449a6010-6e6b-4fa8-aeb6-570e53fc9081-istio-csr-dns-cert" (OuterVolumeSpecName: "istio-csr-dns-cert") pod "449a6010-6e6b-4fa8-aeb6-570e53fc9081" (UID: "449a6010-6e6b-4fa8-aeb6-570e53fc9081"). InnerVolumeSpecName "istio-csr-dns-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 17 17:53:35.506009 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:53:35.505927    2575 reconciler_common.go:299] "Volume detached for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/449a6010-6e6b-4fa8-aeb6-570e53fc9081-local-certs\") on node \"ip-10-0-143-74.ec2.internal\" DevicePath \"\""
Sep 17 17:53:35.506009 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:53:35.505957    2575 reconciler_common.go:299] "Volume detached for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/449a6010-6e6b-4fa8-aeb6-570e53fc9081-istio-csr-ca-configmap\") on node \"ip-10-0-143-74.ec2.internal\" DevicePath \"\""
Sep 17 17:53:35.506009 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:53:35.505967    2575 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/449a6010-6e6b-4fa8-aeb6-570e53fc9081-istio-token\") on node \"ip-10-0-143-74.ec2.internal\" DevicePath \"\""
Sep 17 17:53:35.506009 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:53:35.505977    2575 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-xppgz\" (UniqueName: \"kubernetes.io/projected/449a6010-6e6b-4fa8-aeb6-570e53fc9081-kube-api-access-xppgz\") on node \"ip-10-0-143-74.ec2.internal\" DevicePath \"\""
Sep 17 17:53:35.506009 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:53:35.505986    2575 reconciler_common.go:299] "Volume detached for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/449a6010-6e6b-4fa8-aeb6-570e53fc9081-istio-csr-dns-cert\") on node \"ip-10-0-143-74.ec2.internal\" DevicePath \"\""
Sep 17 17:53:35.506009 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:53:35.505994    2575 reconciler_common.go:299] "Volume detached for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/449a6010-6e6b-4fa8-aeb6-570e53fc9081-cacerts\") on node \"ip-10-0-143-74.ec2.internal\" DevicePath \"\""
Sep 17 17:53:35.506009 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:53:35.506002    2575 reconciler_common.go:299] "Volume detached for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/449a6010-6e6b-4fa8-aeb6-570e53fc9081-istio-kubeconfig\") on node \"ip-10-0-143-74.ec2.internal\" DevicePath \"\""
Sep 17 17:53:35.897763 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:53:35.897727    2575 generic.go:356] "Generic (PLEG): container finished" podID="449a6010-6e6b-4fa8-aeb6-570e53fc9081" containerID="7a0874efd22cda87acd4f3ca1a804886936ccb00790b2d63061231d68325312e" exitCode=0
Sep 17 17:53:35.898213 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:53:35.897777    2575 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-mbwr4"
Sep 17 17:53:35.898213 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:53:35.897809    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-mbwr4" event={"ID":"449a6010-6e6b-4fa8-aeb6-570e53fc9081","Type":"ContainerDied","Data":"7a0874efd22cda87acd4f3ca1a804886936ccb00790b2d63061231d68325312e"}
Sep 17 17:53:35.898213 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:53:35.897855    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-mbwr4" event={"ID":"449a6010-6e6b-4fa8-aeb6-570e53fc9081","Type":"ContainerDied","Data":"b9d08b703bc7d0031db7b9640cb918ffff6e6fadb40284cef6452c3a1dd172f7"}
Sep 17 17:53:35.898213 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:53:35.897875    2575 scope.go:117] "RemoveContainer" containerID="7a0874efd22cda87acd4f3ca1a804886936ccb00790b2d63061231d68325312e"
Sep 17 17:53:35.907413 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:53:35.907393    2575 scope.go:117] "RemoveContainer" containerID="7a0874efd22cda87acd4f3ca1a804886936ccb00790b2d63061231d68325312e"
Sep 17 17:53:35.907701 ip-10-0-143-74 kubenswrapper[2575]: E0917 17:53:35.907685    2575 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"7a0874efd22cda87acd4f3ca1a804886936ccb00790b2d63061231d68325312e\": container with ID starting with 7a0874efd22cda87acd4f3ca1a804886936ccb00790b2d63061231d68325312e not found: ID does not exist" containerID="7a0874efd22cda87acd4f3ca1a804886936ccb00790b2d63061231d68325312e"
Sep 17 17:53:35.907748 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:53:35.907709    2575 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"7a0874efd22cda87acd4f3ca1a804886936ccb00790b2d63061231d68325312e"} err="failed to get container status \"7a0874efd22cda87acd4f3ca1a804886936ccb00790b2d63061231d68325312e\": rpc error: code = NotFound desc = could not find container \"7a0874efd22cda87acd4f3ca1a804886936ccb00790b2d63061231d68325312e\": container with ID starting with 7a0874efd22cda87acd4f3ca1a804886936ccb00790b2d63061231d68325312e not found: ID does not exist"
Sep 17 17:53:35.925338 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:53:35.925305    2575 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7b7995d5cd-mbwr4"]
Sep 17 17:53:35.931717 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:53:35.931684    2575 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7b7995d5cd-mbwr4"]
Sep 17 17:53:36.967753 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:53:36.967710    2575 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="449a6010-6e6b-4fa8-aeb6-570e53fc9081" path="/var/lib/kubelet/pods/449a6010-6e6b-4fa8-aeb6-570e53fc9081/volumes"
Sep 17 17:53:43.935336 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:53:43.935293    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-djvzt_1d07e68d-9943-420f-a6a0-5f477792fabd/konnectivity-agent/0.log" limit=1024
Sep 17 17:53:46.724950 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:53:46.724911    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-djvzt_1d07e68d-9943-420f-a6a0-5f477792fabd/konnectivity-agent/0.log" limit=1024
Sep 17 17:53:46.803338 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:53:46.803296    2575 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kserve/kserve-controller-manager-9b487cb79-m49hq"]
Sep 17 17:53:46.803853 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:53:46.803833    2575 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="449a6010-6e6b-4fa8-aeb6-570e53fc9081" containerName="discovery"
Sep 17 17:53:46.803955 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:53:46.803855    2575 state_mem.go:107] "Deleted CPUSet assignment" podUID="449a6010-6e6b-4fa8-aeb6-570e53fc9081" containerName="discovery"
Sep 17 17:53:46.803955 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:53:46.803941    2575 memory_manager.go:368] "RemoveStaleState removing state" podUID="449a6010-6e6b-4fa8-aeb6-570e53fc9081" containerName="discovery"
Sep 17 17:53:46.807798 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:53:46.807770    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-djvzt_1d07e68d-9943-420f-a6a0-5f477792fabd/konnectivity-agent/0.log" limit=1024
Sep 17 17:53:46.808084 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:53:46.808067    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve/kserve-controller-manager-9b487cb79-m49hq"
Sep 17 17:53:46.810432 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:53:46.810407    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kserve\"/\"kserve-webhook-server-cert\""
Sep 17 17:53:46.810900 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:53:46.810879    2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kserve\"/\"openshift-service-ca.crt\""
Sep 17 17:53:46.811490 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:53:46.811472    2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kserve\"/\"kube-root-ca.crt\""
Sep 17 17:53:46.811581 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:53:46.811511    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kserve\"/\"kserve-controller-manager-dockercfg-vl57m\""
Sep 17 17:53:46.818791 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:53:46.818772    2575 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kserve/llmisvc-controller-manager-6b54c95c95-m69ds"]
Sep 17 17:53:46.821814 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:53:46.821788    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve/llmisvc-controller-manager-6b54c95c95-m69ds"
Sep 17 17:53:46.824524 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:53:46.824501    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kserve\"/\"llmisvc-controller-manager-dockercfg-5bqr9\""
Sep 17 17:53:46.824650 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:53:46.824526    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kserve\"/\"llmisvc-webhook-server-cert\""
Sep 17 17:53:46.825776 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:53:46.825750    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve/kserve-controller-manager-9b487cb79-m49hq"]
Sep 17 17:53:46.841535 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:53:46.841503    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve/llmisvc-controller-manager-6b54c95c95-m69ds"]
Sep 17 17:53:46.860108 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:53:46.860078    2575 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kserve/seaweedfs-64568bcd49-6mnw4"]
Sep 17 17:53:46.863285 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:53:46.863262    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve/seaweedfs-64568bcd49-6mnw4"
Sep 17 17:53:46.865978 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:53:46.865959    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kserve\"/\"mlpipeline-s3-artifact\""
Sep 17 17:53:46.866210 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:53:46.866193    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kserve\"/\"default-dockercfg-65dq2\""
Sep 17 17:53:46.886964 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:53:46.886934    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve/seaweedfs-64568bcd49-6mnw4"]
Sep 17 17:53:46.891228 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:53:46.891201    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xpb9l\" (UniqueName: \"kubernetes.io/projected/473587de-f311-4519-9e1c-10154aa00c2b-kube-api-access-xpb9l\") pod \"llmisvc-controller-manager-6b54c95c95-m69ds\" (UID: \"473587de-f311-4519-9e1c-10154aa00c2b\") " pod="kserve/llmisvc-controller-manager-6b54c95c95-m69ds"
Sep 17 17:53:46.891380 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:53:46.891232    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/77968197-ca21-4f69-b362-4a8677611969-cert\") pod \"kserve-controller-manager-9b487cb79-m49hq\" (UID: \"77968197-ca21-4f69-b362-4a8677611969\") " pod="kserve/kserve-controller-manager-9b487cb79-m49hq"
Sep 17 17:53:46.891380 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:53:46.891267    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/473587de-f311-4519-9e1c-10154aa00c2b-cert\") pod \"llmisvc-controller-manager-6b54c95c95-m69ds\" (UID: \"473587de-f311-4519-9e1c-10154aa00c2b\") " pod="kserve/llmisvc-controller-manager-6b54c95c95-m69ds"
Sep 17 17:53:46.891524 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:53:46.891371    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gmbpl\" (UniqueName: \"kubernetes.io/projected/77968197-ca21-4f69-b362-4a8677611969-kube-api-access-gmbpl\") pod \"kserve-controller-manager-9b487cb79-m49hq\" (UID: \"77968197-ca21-4f69-b362-4a8677611969\") " pod="kserve/kserve-controller-manager-9b487cb79-m49hq"
Sep 17 17:53:46.891524 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:53:46.891424    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"data\" (UniqueName: \"kubernetes.io/empty-dir/29930da5-fff7-42a4-bf2c-30f5d52ed31e-data\") pod \"seaweedfs-64568bcd49-6mnw4\" (UID: \"29930da5-fff7-42a4-bf2c-30f5d52ed31e\") " pod="kserve/seaweedfs-64568bcd49-6mnw4"
Sep 17 17:53:46.891524 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:53:46.891490    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wdk64\" (UniqueName: \"kubernetes.io/projected/29930da5-fff7-42a4-bf2c-30f5d52ed31e-kube-api-access-wdk64\") pod \"seaweedfs-64568bcd49-6mnw4\" (UID: \"29930da5-fff7-42a4-bf2c-30f5d52ed31e\") " pod="kserve/seaweedfs-64568bcd49-6mnw4"
Sep 17 17:53:46.933157 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:53:46.933118    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-djvzt_1d07e68d-9943-420f-a6a0-5f477792fabd/konnectivity-agent/0.log" limit=1024
Sep 17 17:53:46.992040 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:53:46.991935    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xpb9l\" (UniqueName: \"kubernetes.io/projected/473587de-f311-4519-9e1c-10154aa00c2b-kube-api-access-xpb9l\") pod \"llmisvc-controller-manager-6b54c95c95-m69ds\" (UID: \"473587de-f311-4519-9e1c-10154aa00c2b\") " pod="kserve/llmisvc-controller-manager-6b54c95c95-m69ds"
Sep 17 17:53:46.992040 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:53:46.991981    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/77968197-ca21-4f69-b362-4a8677611969-cert\") pod \"kserve-controller-manager-9b487cb79-m49hq\" (UID: \"77968197-ca21-4f69-b362-4a8677611969\") " pod="kserve/kserve-controller-manager-9b487cb79-m49hq"
Sep 17 17:53:46.992040 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:53:46.992023    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/473587de-f311-4519-9e1c-10154aa00c2b-cert\") pod \"llmisvc-controller-manager-6b54c95c95-m69ds\" (UID: \"473587de-f311-4519-9e1c-10154aa00c2b\") " pod="kserve/llmisvc-controller-manager-6b54c95c95-m69ds"
Sep 17 17:53:46.992317 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:53:46.992060    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gmbpl\" (UniqueName: \"kubernetes.io/projected/77968197-ca21-4f69-b362-4a8677611969-kube-api-access-gmbpl\") pod \"kserve-controller-manager-9b487cb79-m49hq\" (UID: \"77968197-ca21-4f69-b362-4a8677611969\") " pod="kserve/kserve-controller-manager-9b487cb79-m49hq"
Sep 17 17:53:46.992317 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:53:46.992125    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"data\" (UniqueName: \"kubernetes.io/empty-dir/29930da5-fff7-42a4-bf2c-30f5d52ed31e-data\") pod \"seaweedfs-64568bcd49-6mnw4\" (UID: \"29930da5-fff7-42a4-bf2c-30f5d52ed31e\") " pod="kserve/seaweedfs-64568bcd49-6mnw4"
Sep 17 17:53:46.992317 ip-10-0-143-74 kubenswrapper[2575]: E0917 17:53:46.992176    2575 secret.go:189] Couldn't get secret kserve/llmisvc-webhook-server-cert: secret "llmisvc-webhook-server-cert" not found
Sep 17 17:53:46.992317 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:53:46.992196    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wdk64\" (UniqueName: \"kubernetes.io/projected/29930da5-fff7-42a4-bf2c-30f5d52ed31e-kube-api-access-wdk64\") pod \"seaweedfs-64568bcd49-6mnw4\" (UID: \"29930da5-fff7-42a4-bf2c-30f5d52ed31e\") " pod="kserve/seaweedfs-64568bcd49-6mnw4"
Sep 17 17:53:46.992317 ip-10-0-143-74 kubenswrapper[2575]: E0917 17:53:46.992240    2575 secret.go:189] Couldn't get secret kserve/kserve-webhook-server-cert: secret "kserve-webhook-server-cert" not found
Sep 17 17:53:46.992317 ip-10-0-143-74 kubenswrapper[2575]: E0917 17:53:46.992254    2575 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/473587de-f311-4519-9e1c-10154aa00c2b-cert podName:473587de-f311-4519-9e1c-10154aa00c2b nodeName:}" failed. No retries permitted until 2026-09-17 17:53:47.492233295 +0000 UTC m=+1147.122563254 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "cert" (UniqueName: "kubernetes.io/secret/473587de-f311-4519-9e1c-10154aa00c2b-cert") pod "llmisvc-controller-manager-6b54c95c95-m69ds" (UID: "473587de-f311-4519-9e1c-10154aa00c2b") : secret "llmisvc-webhook-server-cert" not found
Sep 17 17:53:46.992317 ip-10-0-143-74 kubenswrapper[2575]: E0917 17:53:46.992309    2575 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/77968197-ca21-4f69-b362-4a8677611969-cert podName:77968197-ca21-4f69-b362-4a8677611969 nodeName:}" failed. No retries permitted until 2026-09-17 17:53:47.49229302 +0000 UTC m=+1147.122622977 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "cert" (UniqueName: "kubernetes.io/secret/77968197-ca21-4f69-b362-4a8677611969-cert") pod "kserve-controller-manager-9b487cb79-m49hq" (UID: "77968197-ca21-4f69-b362-4a8677611969") : secret "kserve-webhook-server-cert" not found
Sep 17 17:53:46.992717 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:53:46.992569    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"data\" (UniqueName: \"kubernetes.io/empty-dir/29930da5-fff7-42a4-bf2c-30f5d52ed31e-data\") pod \"seaweedfs-64568bcd49-6mnw4\" (UID: \"29930da5-fff7-42a4-bf2c-30f5d52ed31e\") " pod="kserve/seaweedfs-64568bcd49-6mnw4"
Sep 17 17:53:47.004261 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:53:47.004231    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-xpb9l\" (UniqueName: \"kubernetes.io/projected/473587de-f311-4519-9e1c-10154aa00c2b-kube-api-access-xpb9l\") pod \"llmisvc-controller-manager-6b54c95c95-m69ds\" (UID: \"473587de-f311-4519-9e1c-10154aa00c2b\") " pod="kserve/llmisvc-controller-manager-6b54c95c95-m69ds"
Sep 17 17:53:47.005051 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:53:47.005028    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-gmbpl\" (UniqueName: \"kubernetes.io/projected/77968197-ca21-4f69-b362-4a8677611969-kube-api-access-gmbpl\") pod \"kserve-controller-manager-9b487cb79-m49hq\" (UID: \"77968197-ca21-4f69-b362-4a8677611969\") " pod="kserve/kserve-controller-manager-9b487cb79-m49hq"
Sep 17 17:53:47.005322 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:53:47.005301    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-wdk64\" (UniqueName: \"kubernetes.io/projected/29930da5-fff7-42a4-bf2c-30f5d52ed31e-kube-api-access-wdk64\") pod \"seaweedfs-64568bcd49-6mnw4\" (UID: \"29930da5-fff7-42a4-bf2c-30f5d52ed31e\") " pod="kserve/seaweedfs-64568bcd49-6mnw4"
Sep 17 17:53:47.174218 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:53:47.174179    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve/seaweedfs-64568bcd49-6mnw4"
Sep 17 17:53:47.260265 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:53:47.260182    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-djvzt_1d07e68d-9943-420f-a6a0-5f477792fabd/konnectivity-agent/0.log" limit=1024
Sep 17 17:53:47.381045 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:53:47.381011    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-djvzt_1d07e68d-9943-420f-a6a0-5f477792fabd/konnectivity-agent/0.log" limit=1024
Sep 17 17:53:47.466103 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:53:47.466079    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve/seaweedfs-64568bcd49-6mnw4"]
Sep 17 17:53:47.468332 ip-10-0-143-74 kubenswrapper[2575]: W0917 17:53:47.468303    2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod29930da5_fff7_42a4_bf2c_30f5d52ed31e.slice/crio-17df5306115679f8732e6837e323017b84ebf18a1aceb16ac7a8e09b4fdfc145 WatchSource:0}: Error finding container 17df5306115679f8732e6837e323017b84ebf18a1aceb16ac7a8e09b4fdfc145: Status 404 returned error can't find the container with id 17df5306115679f8732e6837e323017b84ebf18a1aceb16ac7a8e09b4fdfc145
Sep 17 17:53:47.469527 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:53:47.469436    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-djvzt_1d07e68d-9943-420f-a6a0-5f477792fabd/konnectivity-agent/0.log" limit=1024
Sep 17 17:53:47.495429 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:53:47.495399    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/77968197-ca21-4f69-b362-4a8677611969-cert\") pod \"kserve-controller-manager-9b487cb79-m49hq\" (UID: \"77968197-ca21-4f69-b362-4a8677611969\") " pod="kserve/kserve-controller-manager-9b487cb79-m49hq"
Sep 17 17:53:47.495599 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:53:47.495490    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/473587de-f311-4519-9e1c-10154aa00c2b-cert\") pod \"llmisvc-controller-manager-6b54c95c95-m69ds\" (UID: \"473587de-f311-4519-9e1c-10154aa00c2b\") " pod="kserve/llmisvc-controller-manager-6b54c95c95-m69ds"
Sep 17 17:53:47.497909 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:53:47.497877    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/77968197-ca21-4f69-b362-4a8677611969-cert\") pod \"kserve-controller-manager-9b487cb79-m49hq\" (UID: \"77968197-ca21-4f69-b362-4a8677611969\") " pod="kserve/kserve-controller-manager-9b487cb79-m49hq"
Sep 17 17:53:47.497909 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:53:47.497892    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/473587de-f311-4519-9e1c-10154aa00c2b-cert\") pod \"llmisvc-controller-manager-6b54c95c95-m69ds\" (UID: \"473587de-f311-4519-9e1c-10154aa00c2b\") " pod="kserve/llmisvc-controller-manager-6b54c95c95-m69ds"
Sep 17 17:53:47.572182 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:53:47.572100    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-djvzt_1d07e68d-9943-420f-a6a0-5f477792fabd/konnectivity-agent/0.log" limit=1024
Sep 17 17:53:47.719736 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:53:47.719690    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve/kserve-controller-manager-9b487cb79-m49hq"
Sep 17 17:53:47.734542 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:53:47.734511    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve/llmisvc-controller-manager-6b54c95c95-m69ds"
Sep 17 17:53:47.964530 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:53:47.964480    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve/seaweedfs-64568bcd49-6mnw4" event={"ID":"29930da5-fff7-42a4-bf2c-30f5d52ed31e","Type":"ContainerStarted","Data":"17df5306115679f8732e6837e323017b84ebf18a1aceb16ac7a8e09b4fdfc145"}
Sep 17 17:53:48.007892 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:53:48.007860    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve/kserve-controller-manager-9b487cb79-m49hq"]
Sep 17 17:53:48.010384 ip-10-0-143-74 kubenswrapper[2575]: W0917 17:53:48.010343    2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod77968197_ca21_4f69_b362_4a8677611969.slice/crio-b330a6701fe188e16b753edb072e43c9a028c487dbc3e400186a63fb24c385fc WatchSource:0}: Error finding container b330a6701fe188e16b753edb072e43c9a028c487dbc3e400186a63fb24c385fc: Status 404 returned error can't find the container with id b330a6701fe188e16b753edb072e43c9a028c487dbc3e400186a63fb24c385fc
Sep 17 17:53:48.032918 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:53:48.032834    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve/llmisvc-controller-manager-6b54c95c95-m69ds"]
Sep 17 17:53:48.034905 ip-10-0-143-74 kubenswrapper[2575]: W0917 17:53:48.034876    2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-pod473587de_f311_4519_9e1c_10154aa00c2b.slice/crio-e41b86393d54aa8843ca6e2f91085384787179cb8ac813d50e68836374c68876 WatchSource:0}: Error finding container e41b86393d54aa8843ca6e2f91085384787179cb8ac813d50e68836374c68876: Status 404 returned error can't find the container with id e41b86393d54aa8843ca6e2f91085384787179cb8ac813d50e68836374c68876
Sep 17 17:53:48.976197 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:53:48.976156    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve/llmisvc-controller-manager-6b54c95c95-m69ds" event={"ID":"473587de-f311-4519-9e1c-10154aa00c2b","Type":"ContainerStarted","Data":"e41b86393d54aa8843ca6e2f91085384787179cb8ac813d50e68836374c68876"}
Sep 17 17:53:48.977933 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:53:48.977898    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve/kserve-controller-manager-9b487cb79-m49hq" event={"ID":"77968197-ca21-4f69-b362-4a8677611969","Type":"ContainerStarted","Data":"b330a6701fe188e16b753edb072e43c9a028c487dbc3e400186a63fb24c385fc"}
Sep 17 17:53:53.006344 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:53:53.006307    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve/llmisvc-controller-manager-6b54c95c95-m69ds" event={"ID":"473587de-f311-4519-9e1c-10154aa00c2b","Type":"ContainerStarted","Data":"f7474c3c5dcf774e113870e373b988d5dad63cd4b89429db6a80790a1e07d3d4"}
Sep 17 17:53:53.006837 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:53:53.006378    2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve/llmisvc-controller-manager-6b54c95c95-m69ds"
Sep 17 17:53:53.007886 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:53:53.007861    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve/seaweedfs-64568bcd49-6mnw4" event={"ID":"29930da5-fff7-42a4-bf2c-30f5d52ed31e","Type":"ContainerStarted","Data":"aa4519194d1674ec2d583c04c114d068591f252b6588e81ecde9aa23f35e9515"}
Sep 17 17:53:53.008018 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:53:53.007910    2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve/seaweedfs-64568bcd49-6mnw4"
Sep 17 17:53:53.009371 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:53:53.009350    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve/kserve-controller-manager-9b487cb79-m49hq" event={"ID":"77968197-ca21-4f69-b362-4a8677611969","Type":"ContainerStarted","Data":"a0316cea1cc7edfae2929141c78b8be444ff86320cba2920fcd1d4817c213950"}
Sep 17 17:53:53.009494 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:53:53.009479    2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve/kserve-controller-manager-9b487cb79-m49hq"
Sep 17 17:53:53.029491 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:53:53.029422    2575 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kserve/llmisvc-controller-manager-6b54c95c95-m69ds" podStartSLOduration=2.722090553 podStartE2EDuration="7.029403092s" totalImagesPullingTime="4.307312539s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-17 17:53:46 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-17 17:53:53.028697908 +0000 UTC m=+1152.659027896" watchObservedRunningTime="2026-09-17 17:53:53.029403092 +0000 UTC m=+1152.659733066"
Sep 17 17:53:53.054910 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:53:53.054853    2575 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kserve/kserve-controller-manager-9b487cb79-m49hq" podStartSLOduration=2.723046687 podStartE2EDuration="7.05483845s" totalImagesPullingTime="4.331791763s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-17 17:53:46 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-17 17:53:53.053115352 +0000 UTC m=+1152.683445331" watchObservedRunningTime="2026-09-17 17:53:53.05483845 +0000 UTC m=+1152.685168425"
Sep 17 17:53:53.077147 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:53:53.077096    2575 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kserve/seaweedfs-64568bcd49-6mnw4" podStartSLOduration=2.14998927 podStartE2EDuration="7.077082025s" totalImagesPullingTime="4.927092755s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-17 17:53:46 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-17 17:53:53.076087914 +0000 UTC m=+1152.706417890" watchObservedRunningTime="2026-09-17 17:53:53.077082025 +0000 UTC m=+1152.707412000"
Sep 17 17:53:59.017852 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:53:59.017819    2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kserve/seaweedfs-64568bcd49-6mnw4"
Sep 17 17:54:24.016946 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:54:24.016916    2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kserve/llmisvc-controller-manager-6b54c95c95-m69ds"
Sep 17 17:54:24.020329 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:54:24.020295    2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kserve/kserve-controller-manager-9b487cb79-m49hq"
Sep 17 17:54:32.177013 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:54:32.176974    2575 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kserve/kserve-controller-manager-9b487cb79-m49hq"]
Sep 17 17:54:32.177419 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:54:32.177161    2575 kuberuntime_container.go:873] "Killing container with a grace period" pod="kserve/kserve-controller-manager-9b487cb79-m49hq" podUID="77968197-ca21-4f69-b362-4a8677611969" containerName="manager" containerID="cri-o://a0316cea1cc7edfae2929141c78b8be444ff86320cba2920fcd1d4817c213950" gracePeriod=10
Sep 17 17:54:32.208898 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:54:32.208868    2575 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kserve/kserve-controller-manager-9b487cb79-x9vqw"]
Sep 17 17:54:32.212321 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:54:32.212300    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve/kserve-controller-manager-9b487cb79-x9vqw"
Sep 17 17:54:32.232404 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:54:32.232368    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve/kserve-controller-manager-9b487cb79-x9vqw"]
Sep 17 17:54:32.383004 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:54:32.382970    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ql7rw\" (UniqueName: \"kubernetes.io/projected/1b0ee150-8018-4525-9a50-8f2c70195c99-kube-api-access-ql7rw\") pod \"kserve-controller-manager-9b487cb79-x9vqw\" (UID: \"1b0ee150-8018-4525-9a50-8f2c70195c99\") " pod="kserve/kserve-controller-manager-9b487cb79-x9vqw"
Sep 17 17:54:32.383168 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:54:32.383020    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/1b0ee150-8018-4525-9a50-8f2c70195c99-cert\") pod \"kserve-controller-manager-9b487cb79-x9vqw\" (UID: \"1b0ee150-8018-4525-9a50-8f2c70195c99\") " pod="kserve/kserve-controller-manager-9b487cb79-x9vqw"
Sep 17 17:54:32.428338 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:54:32.428277    2575 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve/kserve-controller-manager-9b487cb79-m49hq"
Sep 17 17:54:32.484270 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:54:32.484237    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/1b0ee150-8018-4525-9a50-8f2c70195c99-cert\") pod \"kserve-controller-manager-9b487cb79-x9vqw\" (UID: \"1b0ee150-8018-4525-9a50-8f2c70195c99\") " pod="kserve/kserve-controller-manager-9b487cb79-x9vqw"
Sep 17 17:54:32.484432 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:54:32.484326    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ql7rw\" (UniqueName: \"kubernetes.io/projected/1b0ee150-8018-4525-9a50-8f2c70195c99-kube-api-access-ql7rw\") pod \"kserve-controller-manager-9b487cb79-x9vqw\" (UID: \"1b0ee150-8018-4525-9a50-8f2c70195c99\") " pod="kserve/kserve-controller-manager-9b487cb79-x9vqw"
Sep 17 17:54:32.486756 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:54:32.486726    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/1b0ee150-8018-4525-9a50-8f2c70195c99-cert\") pod \"kserve-controller-manager-9b487cb79-x9vqw\" (UID: \"1b0ee150-8018-4525-9a50-8f2c70195c99\") " pod="kserve/kserve-controller-manager-9b487cb79-x9vqw"
Sep 17 17:54:32.493347 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:54:32.493319    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-ql7rw\" (UniqueName: \"kubernetes.io/projected/1b0ee150-8018-4525-9a50-8f2c70195c99-kube-api-access-ql7rw\") pod \"kserve-controller-manager-9b487cb79-x9vqw\" (UID: \"1b0ee150-8018-4525-9a50-8f2c70195c99\") " pod="kserve/kserve-controller-manager-9b487cb79-x9vqw"
Sep 17 17:54:32.558247 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:54:32.558211    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve/kserve-controller-manager-9b487cb79-x9vqw"
Sep 17 17:54:32.584648 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:54:32.584615    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/77968197-ca21-4f69-b362-4a8677611969-cert\") pod \"77968197-ca21-4f69-b362-4a8677611969\" (UID: \"77968197-ca21-4f69-b362-4a8677611969\") "
Sep 17 17:54:32.584812 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:54:32.584668    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-gmbpl\" (UniqueName: \"kubernetes.io/projected/77968197-ca21-4f69-b362-4a8677611969-kube-api-access-gmbpl\") pod \"77968197-ca21-4f69-b362-4a8677611969\" (UID: \"77968197-ca21-4f69-b362-4a8677611969\") "
Sep 17 17:54:32.586887 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:54:32.586840    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/77968197-ca21-4f69-b362-4a8677611969-cert" (OuterVolumeSpecName: "cert") pod "77968197-ca21-4f69-b362-4a8677611969" (UID: "77968197-ca21-4f69-b362-4a8677611969"). InnerVolumeSpecName "cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 17 17:54:32.587042 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:54:32.587008    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/77968197-ca21-4f69-b362-4a8677611969-kube-api-access-gmbpl" (OuterVolumeSpecName: "kube-api-access-gmbpl") pod "77968197-ca21-4f69-b362-4a8677611969" (UID: "77968197-ca21-4f69-b362-4a8677611969"). InnerVolumeSpecName "kube-api-access-gmbpl". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 17 17:54:32.686152 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:54:32.686066    2575 reconciler_common.go:299] "Volume detached for volume \"cert\" (UniqueName: \"kubernetes.io/secret/77968197-ca21-4f69-b362-4a8677611969-cert\") on node \"ip-10-0-143-74.ec2.internal\" DevicePath \"\""
Sep 17 17:54:32.686152 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:54:32.686097    2575 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-gmbpl\" (UniqueName: \"kubernetes.io/projected/77968197-ca21-4f69-b362-4a8677611969-kube-api-access-gmbpl\") on node \"ip-10-0-143-74.ec2.internal\" DevicePath \"\""
Sep 17 17:54:32.839424 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:54:32.839378    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve/kserve-controller-manager-9b487cb79-x9vqw"]
Sep 17 17:54:32.844644 ip-10-0-143-74 kubenswrapper[2575]: W0917 17:54:32.844616    2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1b0ee150_8018_4525_9a50_8f2c70195c99.slice/crio-e2b1e908ef998559fc49f92c59256510d593a18568b271a512c8cba1b10f2b08 WatchSource:0}: Error finding container e2b1e908ef998559fc49f92c59256510d593a18568b271a512c8cba1b10f2b08: Status 404 returned error can't find the container with id e2b1e908ef998559fc49f92c59256510d593a18568b271a512c8cba1b10f2b08
Sep 17 17:54:32.845935 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:54:32.845915    2575 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Sep 17 17:54:33.221718 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:54:33.221679    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve/kserve-controller-manager-9b487cb79-x9vqw" event={"ID":"1b0ee150-8018-4525-9a50-8f2c70195c99","Type":"ContainerStarted","Data":"e2b1e908ef998559fc49f92c59256510d593a18568b271a512c8cba1b10f2b08"}
Sep 17 17:54:33.223004 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:54:33.222980    2575 generic.go:356] "Generic (PLEG): container finished" podID="77968197-ca21-4f69-b362-4a8677611969" containerID="a0316cea1cc7edfae2929141c78b8be444ff86320cba2920fcd1d4817c213950" exitCode=0
Sep 17 17:54:33.223129 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:54:33.223015    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve/kserve-controller-manager-9b487cb79-m49hq" event={"ID":"77968197-ca21-4f69-b362-4a8677611969","Type":"ContainerDied","Data":"a0316cea1cc7edfae2929141c78b8be444ff86320cba2920fcd1d4817c213950"}
Sep 17 17:54:33.223129 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:54:33.223044    2575 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve/kserve-controller-manager-9b487cb79-m49hq"
Sep 17 17:54:33.223129 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:54:33.223053    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve/kserve-controller-manager-9b487cb79-m49hq" event={"ID":"77968197-ca21-4f69-b362-4a8677611969","Type":"ContainerDied","Data":"b330a6701fe188e16b753edb072e43c9a028c487dbc3e400186a63fb24c385fc"}
Sep 17 17:54:33.223129 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:54:33.223084    2575 scope.go:117] "RemoveContainer" containerID="a0316cea1cc7edfae2929141c78b8be444ff86320cba2920fcd1d4817c213950"
Sep 17 17:54:33.232133 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:54:33.232110    2575 scope.go:117] "RemoveContainer" containerID="a0316cea1cc7edfae2929141c78b8be444ff86320cba2920fcd1d4817c213950"
Sep 17 17:54:33.232417 ip-10-0-143-74 kubenswrapper[2575]: E0917 17:54:33.232395    2575 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"a0316cea1cc7edfae2929141c78b8be444ff86320cba2920fcd1d4817c213950\": container with ID starting with a0316cea1cc7edfae2929141c78b8be444ff86320cba2920fcd1d4817c213950 not found: ID does not exist" containerID="a0316cea1cc7edfae2929141c78b8be444ff86320cba2920fcd1d4817c213950"
Sep 17 17:54:33.232518 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:54:33.232427    2575 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"a0316cea1cc7edfae2929141c78b8be444ff86320cba2920fcd1d4817c213950"} err="failed to get container status \"a0316cea1cc7edfae2929141c78b8be444ff86320cba2920fcd1d4817c213950\": rpc error: code = NotFound desc = could not find container \"a0316cea1cc7edfae2929141c78b8be444ff86320cba2920fcd1d4817c213950\": container with ID starting with a0316cea1cc7edfae2929141c78b8be444ff86320cba2920fcd1d4817c213950 not found: ID does not exist"
Sep 17 17:54:33.239978 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:54:33.239952    2575 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kserve/kserve-controller-manager-9b487cb79-m49hq"]
Sep 17 17:54:33.244101 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:54:33.244077    2575 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kserve/kserve-controller-manager-9b487cb79-m49hq"]
Sep 17 17:54:34.230845 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:54:34.230811    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve/kserve-controller-manager-9b487cb79-x9vqw" event={"ID":"1b0ee150-8018-4525-9a50-8f2c70195c99","Type":"ContainerStarted","Data":"3ee6ef73a5a4c2ecb9d26671950b7e449919209a270042e697b839dc30d5e2f3"}
Sep 17 17:54:34.231227 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:54:34.230925    2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve/kserve-controller-manager-9b487cb79-x9vqw"
Sep 17 17:54:34.256092 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:54:34.256041    2575 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kserve/kserve-controller-manager-9b487cb79-x9vqw" podStartSLOduration=1.805983651 podStartE2EDuration="2.256018595s" totalImagesPullingTime="450.034944ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-17 17:54:32 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-17 17:54:34.255138363 +0000 UTC m=+1193.885468337" watchObservedRunningTime="2026-09-17 17:54:34.256018595 +0000 UTC m=+1193.886348571"
Sep 17 17:54:34.967913 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:54:34.967879    2575 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="77968197-ca21-4f69-b362-4a8677611969" path="/var/lib/kubelet/pods/77968197-ca21-4f69-b362-4a8677611969/volumes"
Sep 17 17:55:05.242085 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:55:05.242049    2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kserve/kserve-controller-manager-9b487cb79-x9vqw"
Sep 17 17:55:06.556205 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:55:06.556169    2575 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kserve/odh-model-controller-7b966d7984-5cnqv"]
Sep 17 17:55:06.556598 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:55:06.556583    2575 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="77968197-ca21-4f69-b362-4a8677611969" containerName="manager"
Sep 17 17:55:06.556647 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:55:06.556600    2575 state_mem.go:107] "Deleted CPUSet assignment" podUID="77968197-ca21-4f69-b362-4a8677611969" containerName="manager"
Sep 17 17:55:06.556732 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:55:06.556713    2575 memory_manager.go:368] "RemoveStaleState removing state" podUID="77968197-ca21-4f69-b362-4a8677611969" containerName="manager"
Sep 17 17:55:06.560145 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:55:06.560122    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve/odh-model-controller-7b966d7984-5cnqv"
Sep 17 17:55:06.563592 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:55:06.563574    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kserve\"/\"odh-model-controller-webhook-cert\""
Sep 17 17:55:06.563713 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:55:06.563692    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kserve\"/\"odh-model-controller-metrics-tls\""
Sep 17 17:55:06.563773 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:55:06.563688    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kserve\"/\"odh-model-controller-dockercfg-6hb9g\""
Sep 17 17:55:06.572166 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:55:06.572141    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve/odh-model-controller-7b966d7984-5cnqv"]
Sep 17 17:55:06.576750 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:55:06.576724    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-djvzt_1d07e68d-9943-420f-a6a0-5f477792fabd/konnectivity-agent/0.log" limit=1024
Sep 17 17:55:06.680938 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:55:06.680902    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/8dd98cfc-69b0-4814-beca-bee51a1fdaf8-cert\") pod \"odh-model-controller-7b966d7984-5cnqv\" (UID: \"8dd98cfc-69b0-4814-beca-bee51a1fdaf8\") " pod="kserve/odh-model-controller-7b966d7984-5cnqv"
Sep 17 17:55:06.681099 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:55:06.680949    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-kgrdw\" (UniqueName: \"kubernetes.io/projected/8dd98cfc-69b0-4814-beca-bee51a1fdaf8-kube-api-access-kgrdw\") pod \"odh-model-controller-7b966d7984-5cnqv\" (UID: \"8dd98cfc-69b0-4814-beca-bee51a1fdaf8\") " pod="kserve/odh-model-controller-7b966d7984-5cnqv"
Sep 17 17:55:06.681099 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:55:06.681021    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/8dd98cfc-69b0-4814-beca-bee51a1fdaf8-metrics-certs\") pod \"odh-model-controller-7b966d7984-5cnqv\" (UID: \"8dd98cfc-69b0-4814-beca-bee51a1fdaf8\") " pod="kserve/odh-model-controller-7b966d7984-5cnqv"
Sep 17 17:55:06.781856 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:55:06.781820    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/8dd98cfc-69b0-4814-beca-bee51a1fdaf8-cert\") pod \"odh-model-controller-7b966d7984-5cnqv\" (UID: \"8dd98cfc-69b0-4814-beca-bee51a1fdaf8\") " pod="kserve/odh-model-controller-7b966d7984-5cnqv"
Sep 17 17:55:06.782028 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:55:06.781867    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kgrdw\" (UniqueName: \"kubernetes.io/projected/8dd98cfc-69b0-4814-beca-bee51a1fdaf8-kube-api-access-kgrdw\") pod \"odh-model-controller-7b966d7984-5cnqv\" (UID: \"8dd98cfc-69b0-4814-beca-bee51a1fdaf8\") " pod="kserve/odh-model-controller-7b966d7984-5cnqv"
Sep 17 17:55:06.782028 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:55:06.781906    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/8dd98cfc-69b0-4814-beca-bee51a1fdaf8-metrics-certs\") pod \"odh-model-controller-7b966d7984-5cnqv\" (UID: \"8dd98cfc-69b0-4814-beca-bee51a1fdaf8\") " pod="kserve/odh-model-controller-7b966d7984-5cnqv"
Sep 17 17:55:06.782028 ip-10-0-143-74 kubenswrapper[2575]: E0917 17:55:06.781974    2575 secret.go:189] Couldn't get secret kserve/odh-model-controller-webhook-cert: secret "odh-model-controller-webhook-cert" not found
Sep 17 17:55:06.782185 ip-10-0-143-74 kubenswrapper[2575]: E0917 17:55:06.782051    2575 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/8dd98cfc-69b0-4814-beca-bee51a1fdaf8-cert podName:8dd98cfc-69b0-4814-beca-bee51a1fdaf8 nodeName:}" failed. No retries permitted until 2026-09-17 17:55:07.282032054 +0000 UTC m=+1226.912362007 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "cert" (UniqueName: "kubernetes.io/secret/8dd98cfc-69b0-4814-beca-bee51a1fdaf8-cert") pod "odh-model-controller-7b966d7984-5cnqv" (UID: "8dd98cfc-69b0-4814-beca-bee51a1fdaf8") : secret "odh-model-controller-webhook-cert" not found
Sep 17 17:55:06.784399 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:55:06.784376    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/8dd98cfc-69b0-4814-beca-bee51a1fdaf8-metrics-certs\") pod \"odh-model-controller-7b966d7984-5cnqv\" (UID: \"8dd98cfc-69b0-4814-beca-bee51a1fdaf8\") " pod="kserve/odh-model-controller-7b966d7984-5cnqv"
Sep 17 17:55:06.795553 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:55:06.795533    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-kgrdw\" (UniqueName: \"kubernetes.io/projected/8dd98cfc-69b0-4814-beca-bee51a1fdaf8-kube-api-access-kgrdw\") pod \"odh-model-controller-7b966d7984-5cnqv\" (UID: \"8dd98cfc-69b0-4814-beca-bee51a1fdaf8\") " pod="kserve/odh-model-controller-7b966d7984-5cnqv"
Sep 17 17:55:07.286838 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:55:07.286805    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/8dd98cfc-69b0-4814-beca-bee51a1fdaf8-cert\") pod \"odh-model-controller-7b966d7984-5cnqv\" (UID: \"8dd98cfc-69b0-4814-beca-bee51a1fdaf8\") " pod="kserve/odh-model-controller-7b966d7984-5cnqv"
Sep 17 17:55:07.289133 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:55:07.289104    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/8dd98cfc-69b0-4814-beca-bee51a1fdaf8-cert\") pod \"odh-model-controller-7b966d7984-5cnqv\" (UID: \"8dd98cfc-69b0-4814-beca-bee51a1fdaf8\") " pod="kserve/odh-model-controller-7b966d7984-5cnqv"
Sep 17 17:55:07.471279 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:55:07.471243    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve/odh-model-controller-7b966d7984-5cnqv"
Sep 17 17:55:07.747046 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:55:07.747018    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve/odh-model-controller-7b966d7984-5cnqv"]
Sep 17 17:55:07.748275 ip-10-0-143-74 kubenswrapper[2575]: W0917 17:55:07.748247    2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8dd98cfc_69b0_4814_beca_bee51a1fdaf8.slice/crio-c76b4927afe66e45cf66e48033c0f8213858fe1742d15c68b1c4ca0df5776d6c WatchSource:0}: Error finding container c76b4927afe66e45cf66e48033c0f8213858fe1742d15c68b1c4ca0df5776d6c: Status 404 returned error can't find the container with id c76b4927afe66e45cf66e48033c0f8213858fe1742d15c68b1c4ca0df5776d6c
Sep 17 17:55:07.906841 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:55:07.906809    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-djvzt_1d07e68d-9943-420f-a6a0-5f477792fabd/konnectivity-agent/0.log" limit=1024
Sep 17 17:55:08.004873 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:55:08.004779    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-djvzt_1d07e68d-9943-420f-a6a0-5f477792fabd/konnectivity-agent/0.log" limit=1024
Sep 17 17:55:08.426676 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:55:08.426635    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve/odh-model-controller-7b966d7984-5cnqv" event={"ID":"8dd98cfc-69b0-4814-beca-bee51a1fdaf8","Type":"ContainerStarted","Data":"c76b4927afe66e45cf66e48033c0f8213858fe1742d15c68b1c4ca0df5776d6c"}
Sep 17 17:55:10.441191 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:55:10.441144    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve/odh-model-controller-7b966d7984-5cnqv" event={"ID":"8dd98cfc-69b0-4814-beca-bee51a1fdaf8","Type":"ContainerStarted","Data":"0b5a5303b2b559022065acb06af206cc3cd5e7057fcf8373e443ad60a725b454"}
Sep 17 17:55:10.441629 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:55:10.441355    2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve/odh-model-controller-7b966d7984-5cnqv"
Sep 17 17:55:10.462569 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:55:10.462515    2575 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kserve/odh-model-controller-7b966d7984-5cnqv" podStartSLOduration=2.167875346 podStartE2EDuration="4.462499963s" totalImagesPullingTime="2.294624617s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-17 17:55:06 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-17 17:55:10.459742573 +0000 UTC m=+1230.090072552" watchObservedRunningTime="2026-09-17 17:55:10.462499963 +0000 UTC m=+1230.092829935"
Sep 17 17:55:21.449636 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:55:21.449597    2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kserve/odh-model-controller-7b966d7984-5cnqv"
Sep 17 17:56:41.316179 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:56:41.316144    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-djvzt_1d07e68d-9943-420f-a6a0-5f477792fabd/konnectivity-agent/0.log" limit=1024
Sep 17 17:56:41.463647 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:56:41.463614    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-djvzt_1d07e68d-9943-420f-a6a0-5f477792fabd/konnectivity-agent/0.log" limit=1024
Sep 17 17:56:51.640897 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:56:51.640861    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-djvzt_1d07e68d-9943-420f-a6a0-5f477792fabd/konnectivity-agent/0.log" limit=1024
Sep 17 17:56:53.391008 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:56:53.390976    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-djvzt_1d07e68d-9943-420f-a6a0-5f477792fabd/konnectivity-agent/0.log" limit=1024
Sep 17 17:56:53.592122 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:56:53.592086    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-djvzt_1d07e68d-9943-420f-a6a0-5f477792fabd/konnectivity-agent/0.log" limit=1024
Sep 17 17:57:06.572693 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:57:06.572586    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-djvzt_1d07e68d-9943-420f-a6a0-5f477792fabd/konnectivity-agent/0.log" limit=1024
Sep 17 17:57:06.874650 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:57:06.874614    2575 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["e2e-test-llm-inference-service-2541cd65/scheduler-ha-replicas-test-kserve-router-scheduler-557bbc5p9tkw"]
Sep 17 17:57:06.878679 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:57:06.878654    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-test-llm-inference-service-2541cd65/scheduler-ha-replicas-test-kserve-router-scheduler-557bbc5p9tkw"
Sep 17 17:57:06.883417 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:57:06.883394    2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-test-llm-inference-service-2541cd65\"/\"kube-root-ca.crt\""
Sep 17 17:57:06.884144 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:57:06.884090    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-test-llm-inference-service-2541cd65\"/\"scheduler-ha-replicas-test-kserve-self-signed-certs\""
Sep 17 17:57:06.884144 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:57:06.884114    2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-test-llm-inference-service-2541cd65\"/\"openshift-service-ca.crt\""
Sep 17 17:57:06.884144 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:57:06.884142    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-test-llm-inference-service-2541cd65\"/\"default-dockercfg-h64nn\""
Sep 17 17:57:06.884418 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:57:06.884397    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-test-llm-inference-service-2541cd65\"/\"scheduler-ha-replicas-test-epp-sa-dockercfg-lxczf\""
Sep 17 17:57:06.892623 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:57:06.892600    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["e2e-test-llm-inference-service-2541cd65/scheduler-ha-replicas-test-kserve-router-scheduler-557bbc5p9tkw"]
Sep 17 17:57:06.919102 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:57:06.919066    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-djvzt_1d07e68d-9943-420f-a6a0-5f477792fabd/konnectivity-agent/0.log" limit=1024
Sep 17 17:57:06.971369 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:57:06.971328    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/68a11ba5-3c58-45c8-9190-c0030a9799a1-tls-certs\") pod \"scheduler-ha-replicas-test-kserve-router-scheduler-557bbc5p9tkw\" (UID: \"68a11ba5-3c58-45c8-9190-c0030a9799a1\") " pod="e2e-test-llm-inference-service-2541cd65/scheduler-ha-replicas-test-kserve-router-scheduler-557bbc5p9tkw"
Sep 17 17:57:06.971581 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:57:06.971378    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-q7gf5\" (UniqueName: \"kubernetes.io/projected/68a11ba5-3c58-45c8-9190-c0030a9799a1-kube-api-access-q7gf5\") pod \"scheduler-ha-replicas-test-kserve-router-scheduler-557bbc5p9tkw\" (UID: \"68a11ba5-3c58-45c8-9190-c0030a9799a1\") " pod="e2e-test-llm-inference-service-2541cd65/scheduler-ha-replicas-test-kserve-router-scheduler-557bbc5p9tkw"
Sep 17 17:57:07.072671 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:57:07.072638    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/68a11ba5-3c58-45c8-9190-c0030a9799a1-tls-certs\") pod \"scheduler-ha-replicas-test-kserve-router-scheduler-557bbc5p9tkw\" (UID: \"68a11ba5-3c58-45c8-9190-c0030a9799a1\") " pod="e2e-test-llm-inference-service-2541cd65/scheduler-ha-replicas-test-kserve-router-scheduler-557bbc5p9tkw"
Sep 17 17:57:07.072671 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:57:07.072674    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-q7gf5\" (UniqueName: \"kubernetes.io/projected/68a11ba5-3c58-45c8-9190-c0030a9799a1-kube-api-access-q7gf5\") pod \"scheduler-ha-replicas-test-kserve-router-scheduler-557bbc5p9tkw\" (UID: \"68a11ba5-3c58-45c8-9190-c0030a9799a1\") " pod="e2e-test-llm-inference-service-2541cd65/scheduler-ha-replicas-test-kserve-router-scheduler-557bbc5p9tkw"
Sep 17 17:57:07.076034 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:57:07.075995    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/68a11ba5-3c58-45c8-9190-c0030a9799a1-tls-certs\") pod \"scheduler-ha-replicas-test-kserve-router-scheduler-557bbc5p9tkw\" (UID: \"68a11ba5-3c58-45c8-9190-c0030a9799a1\") " pod="e2e-test-llm-inference-service-2541cd65/scheduler-ha-replicas-test-kserve-router-scheduler-557bbc5p9tkw"
Sep 17 17:57:07.082707 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:57:07.082681    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-q7gf5\" (UniqueName: \"kubernetes.io/projected/68a11ba5-3c58-45c8-9190-c0030a9799a1-kube-api-access-q7gf5\") pod \"scheduler-ha-replicas-test-kserve-router-scheduler-557bbc5p9tkw\" (UID: \"68a11ba5-3c58-45c8-9190-c0030a9799a1\") " pod="e2e-test-llm-inference-service-2541cd65/scheduler-ha-replicas-test-kserve-router-scheduler-557bbc5p9tkw"
Sep 17 17:57:07.191052 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:57:07.190956    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-test-llm-inference-service-2541cd65/scheduler-ha-replicas-test-kserve-router-scheduler-557bbc5p9tkw"
Sep 17 17:57:07.472241 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:57:07.472151    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["e2e-test-llm-inference-service-2541cd65/scheduler-ha-replicas-test-kserve-router-scheduler-557bbc5p9tkw"]
Sep 17 17:57:07.474957 ip-10-0-143-74 kubenswrapper[2575]: W0917 17:57:07.474927    2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod68a11ba5_3c58_45c8_9190_c0030a9799a1.slice/crio-d1f66684ed6ea318a9a1d18a392b0004cfe4d1f93ccf5f944371c5ec15f3ea9c WatchSource:0}: Error finding container d1f66684ed6ea318a9a1d18a392b0004cfe4d1f93ccf5f944371c5ec15f3ea9c: Status 404 returned error can't find the container with id d1f66684ed6ea318a9a1d18a392b0004cfe4d1f93ccf5f944371c5ec15f3ea9c
Sep 17 17:57:08.084986 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:57:08.084895    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-test-llm-inference-service-2541cd65/scheduler-ha-replicas-test-kserve-router-scheduler-557bbc5p9tkw" event={"ID":"68a11ba5-3c58-45c8-9190-c0030a9799a1","Type":"ContainerStarted","Data":"d1f66684ed6ea318a9a1d18a392b0004cfe4d1f93ccf5f944371c5ec15f3ea9c"}
Sep 17 17:57:10.098456 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:57:10.098406    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-test-llm-inference-service-2541cd65/scheduler-ha-replicas-test-kserve-router-scheduler-557bbc5p9tkw" event={"ID":"68a11ba5-3c58-45c8-9190-c0030a9799a1","Type":"ContainerStarted","Data":"05e19c488b5f40797e234def570e19498919981cd960c6318f1dbab2227cf6c6"}
Sep 17 17:57:10.098868 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:57:10.098494    2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-test-llm-inference-service-2541cd65/scheduler-ha-replicas-test-kserve-router-scheduler-557bbc5p9tkw"
Sep 17 17:57:10.120255 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:57:10.120199    2575 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="e2e-test-llm-inference-service-2541cd65/scheduler-ha-replicas-test-kserve-router-scheduler-557bbc5p9tkw" podStartSLOduration=1.7352381110000001 podStartE2EDuration="4.12018324s" totalImagesPullingTime="2.384945129s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-17 17:57:06 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-17 17:57:10.118902761 +0000 UTC m=+1349.749232736" watchObservedRunningTime="2026-09-17 17:57:10.12018324 +0000 UTC m=+1349.750513214"
Sep 17 17:57:16.199682 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:57:16.199635    2575 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["e2e-test-flow-control-smoke-4f8df83c/fc-smoke-test-kserve-7f4687897-646df"]
Sep 17 17:57:16.203710 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:57:16.203689    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-test-flow-control-smoke-4f8df83c/fc-smoke-test-kserve-7f4687897-646df"
Sep 17 17:57:16.207321 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:57:16.207299    2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-test-flow-control-smoke-4f8df83c\"/\"kube-root-ca.crt\""
Sep 17 17:57:16.207431 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:57:16.207369    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-test-flow-control-smoke-4f8df83c\"/\"fc-smoke-test-kserve-self-signed-certs\""
Sep 17 17:57:16.207431 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:57:16.207406    2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-test-flow-control-smoke-4f8df83c\"/\"openshift-service-ca.crt\""
Sep 17 17:57:16.207558 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:57:16.207435    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-test-flow-control-smoke-4f8df83c\"/\"default-dockercfg-5zv78\""
Sep 17 17:57:16.215132 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:57:16.215099    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["e2e-test-flow-control-smoke-4f8df83c/fc-smoke-test-kserve-7f4687897-646df"]
Sep 17 17:57:16.245182 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:57:16.245151    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-djvzt_1d07e68d-9943-420f-a6a0-5f477792fabd/konnectivity-agent/0.log" limit=1024
Sep 17 17:57:16.366402 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:57:16.366367    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/b76e9eae-0573-47b2-bea0-e85ee4360e97-home\") pod \"fc-smoke-test-kserve-7f4687897-646df\" (UID: \"b76e9eae-0573-47b2-bea0-e85ee4360e97\") " pod="e2e-test-flow-control-smoke-4f8df83c/fc-smoke-test-kserve-7f4687897-646df"
Sep 17 17:57:16.366402 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:57:16.366402    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/b76e9eae-0573-47b2-bea0-e85ee4360e97-model-cache\") pod \"fc-smoke-test-kserve-7f4687897-646df\" (UID: \"b76e9eae-0573-47b2-bea0-e85ee4360e97\") " pod="e2e-test-flow-control-smoke-4f8df83c/fc-smoke-test-kserve-7f4687897-646df"
Sep 17 17:57:16.366617 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:57:16.366422    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-krzlf\" (UniqueName: \"kubernetes.io/projected/b76e9eae-0573-47b2-bea0-e85ee4360e97-kube-api-access-krzlf\") pod \"fc-smoke-test-kserve-7f4687897-646df\" (UID: \"b76e9eae-0573-47b2-bea0-e85ee4360e97\") " pod="e2e-test-flow-control-smoke-4f8df83c/fc-smoke-test-kserve-7f4687897-646df"
Sep 17 17:57:16.366617 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:57:16.366494    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/b76e9eae-0573-47b2-bea0-e85ee4360e97-dshm\") pod \"fc-smoke-test-kserve-7f4687897-646df\" (UID: \"b76e9eae-0573-47b2-bea0-e85ee4360e97\") " pod="e2e-test-flow-control-smoke-4f8df83c/fc-smoke-test-kserve-7f4687897-646df"
Sep 17 17:57:16.366617 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:57:16.366534    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/b76e9eae-0573-47b2-bea0-e85ee4360e97-tmp-dir\") pod \"fc-smoke-test-kserve-7f4687897-646df\" (UID: \"b76e9eae-0573-47b2-bea0-e85ee4360e97\") " pod="e2e-test-flow-control-smoke-4f8df83c/fc-smoke-test-kserve-7f4687897-646df"
Sep 17 17:57:16.366617 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:57:16.366584    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/b76e9eae-0573-47b2-bea0-e85ee4360e97-tls-certs\") pod \"fc-smoke-test-kserve-7f4687897-646df\" (UID: \"b76e9eae-0573-47b2-bea0-e85ee4360e97\") " pod="e2e-test-flow-control-smoke-4f8df83c/fc-smoke-test-kserve-7f4687897-646df"
Sep 17 17:57:16.453278 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:57:16.453195    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-djvzt_1d07e68d-9943-420f-a6a0-5f477792fabd/konnectivity-agent/0.log" limit=1024
Sep 17 17:57:16.467078 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:57:16.467053    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/b76e9eae-0573-47b2-bea0-e85ee4360e97-home\") pod \"fc-smoke-test-kserve-7f4687897-646df\" (UID: \"b76e9eae-0573-47b2-bea0-e85ee4360e97\") " pod="e2e-test-flow-control-smoke-4f8df83c/fc-smoke-test-kserve-7f4687897-646df"
Sep 17 17:57:16.467152 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:57:16.467082    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/b76e9eae-0573-47b2-bea0-e85ee4360e97-model-cache\") pod \"fc-smoke-test-kserve-7f4687897-646df\" (UID: \"b76e9eae-0573-47b2-bea0-e85ee4360e97\") " pod="e2e-test-flow-control-smoke-4f8df83c/fc-smoke-test-kserve-7f4687897-646df"
Sep 17 17:57:16.467152 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:57:16.467098    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-krzlf\" (UniqueName: \"kubernetes.io/projected/b76e9eae-0573-47b2-bea0-e85ee4360e97-kube-api-access-krzlf\") pod \"fc-smoke-test-kserve-7f4687897-646df\" (UID: \"b76e9eae-0573-47b2-bea0-e85ee4360e97\") " pod="e2e-test-flow-control-smoke-4f8df83c/fc-smoke-test-kserve-7f4687897-646df"
Sep 17 17:57:16.467240 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:57:16.467215    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/b76e9eae-0573-47b2-bea0-e85ee4360e97-dshm\") pod \"fc-smoke-test-kserve-7f4687897-646df\" (UID: \"b76e9eae-0573-47b2-bea0-e85ee4360e97\") " pod="e2e-test-flow-control-smoke-4f8df83c/fc-smoke-test-kserve-7f4687897-646df"
Sep 17 17:57:16.467297 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:57:16.467271    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/b76e9eae-0573-47b2-bea0-e85ee4360e97-tmp-dir\") pod \"fc-smoke-test-kserve-7f4687897-646df\" (UID: \"b76e9eae-0573-47b2-bea0-e85ee4360e97\") " pod="e2e-test-flow-control-smoke-4f8df83c/fc-smoke-test-kserve-7f4687897-646df"
Sep 17 17:57:16.467366 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:57:16.467349    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/b76e9eae-0573-47b2-bea0-e85ee4360e97-tls-certs\") pod \"fc-smoke-test-kserve-7f4687897-646df\" (UID: \"b76e9eae-0573-47b2-bea0-e85ee4360e97\") " pod="e2e-test-flow-control-smoke-4f8df83c/fc-smoke-test-kserve-7f4687897-646df"
Sep 17 17:57:16.467428 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:57:16.467409    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/b76e9eae-0573-47b2-bea0-e85ee4360e97-model-cache\") pod \"fc-smoke-test-kserve-7f4687897-646df\" (UID: \"b76e9eae-0573-47b2-bea0-e85ee4360e97\") " pod="e2e-test-flow-control-smoke-4f8df83c/fc-smoke-test-kserve-7f4687897-646df"
Sep 17 17:57:16.467515 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:57:16.467480    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/b76e9eae-0573-47b2-bea0-e85ee4360e97-home\") pod \"fc-smoke-test-kserve-7f4687897-646df\" (UID: \"b76e9eae-0573-47b2-bea0-e85ee4360e97\") " pod="e2e-test-flow-control-smoke-4f8df83c/fc-smoke-test-kserve-7f4687897-646df"
Sep 17 17:57:16.467637 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:57:16.467618    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/b76e9eae-0573-47b2-bea0-e85ee4360e97-tmp-dir\") pod \"fc-smoke-test-kserve-7f4687897-646df\" (UID: \"b76e9eae-0573-47b2-bea0-e85ee4360e97\") " pod="e2e-test-flow-control-smoke-4f8df83c/fc-smoke-test-kserve-7f4687897-646df"
Sep 17 17:57:16.469503 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:57:16.469478    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/b76e9eae-0573-47b2-bea0-e85ee4360e97-dshm\") pod \"fc-smoke-test-kserve-7f4687897-646df\" (UID: \"b76e9eae-0573-47b2-bea0-e85ee4360e97\") " pod="e2e-test-flow-control-smoke-4f8df83c/fc-smoke-test-kserve-7f4687897-646df"
Sep 17 17:57:16.469777 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:57:16.469762    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/b76e9eae-0573-47b2-bea0-e85ee4360e97-tls-certs\") pod \"fc-smoke-test-kserve-7f4687897-646df\" (UID: \"b76e9eae-0573-47b2-bea0-e85ee4360e97\") " pod="e2e-test-flow-control-smoke-4f8df83c/fc-smoke-test-kserve-7f4687897-646df"
Sep 17 17:57:16.476137 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:57:16.476116    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-krzlf\" (UniqueName: \"kubernetes.io/projected/b76e9eae-0573-47b2-bea0-e85ee4360e97-kube-api-access-krzlf\") pod \"fc-smoke-test-kserve-7f4687897-646df\" (UID: \"b76e9eae-0573-47b2-bea0-e85ee4360e97\") " pod="e2e-test-flow-control-smoke-4f8df83c/fc-smoke-test-kserve-7f4687897-646df"
Sep 17 17:57:16.515737 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:57:16.515707    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-test-flow-control-smoke-4f8df83c/fc-smoke-test-kserve-7f4687897-646df"
Sep 17 17:57:16.797816 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:57:16.797790    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["e2e-test-flow-control-smoke-4f8df83c/fc-smoke-test-kserve-7f4687897-646df"]
Sep 17 17:57:16.800171 ip-10-0-143-74 kubenswrapper[2575]: W0917 17:57:16.800144    2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb76e9eae_0573_47b2_bea0_e85ee4360e97.slice/crio-b05c2e3bd8d134b7a6e5d05f4b6d1aa7588a1370c596d9826bac7c04f1292162 WatchSource:0}: Error finding container b05c2e3bd8d134b7a6e5d05f4b6d1aa7588a1370c596d9826bac7c04f1292162: Status 404 returned error can't find the container with id b05c2e3bd8d134b7a6e5d05f4b6d1aa7588a1370c596d9826bac7c04f1292162
Sep 17 17:57:17.139730 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:57:17.139690    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-test-flow-control-smoke-4f8df83c/fc-smoke-test-kserve-7f4687897-646df" event={"ID":"b76e9eae-0573-47b2-bea0-e85ee4360e97","Type":"ContainerStarted","Data":"b05c2e3bd8d134b7a6e5d05f4b6d1aa7588a1370c596d9826bac7c04f1292162"}
Sep 17 17:57:19.153386 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:57:19.153351    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-test-flow-control-smoke-4f8df83c/fc-smoke-test-kserve-7f4687897-646df" event={"ID":"b76e9eae-0573-47b2-bea0-e85ee4360e97","Type":"ContainerStarted","Data":"e461f2131b926e4ab5875bfc82d404bf077bc4a7c084b8516e65e686e7b0d692"}
Sep 17 17:57:19.176668 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:57:19.176616    2575 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="e2e-test-flow-control-smoke-4f8df83c/fc-smoke-test-kserve-7f4687897-646df" podStartSLOduration=1.661632787 podStartE2EDuration="3.176600988s" totalImagesPullingTime="1.514968201s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-17 17:57:16 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-17 17:57:19.174365733 +0000 UTC m=+1358.804695712" watchObservedRunningTime="2026-09-17 17:57:19.176600988 +0000 UTC m=+1358.806930994"
Sep 17 17:57:19.516322 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:57:19.516235    2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-test-flow-control-smoke-4f8df83c/fc-smoke-test-kserve-7f4687897-646df"
Sep 17 17:57:26.516476 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:57:26.516420    2575 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="e2e-test-flow-control-smoke-4f8df83c/fc-smoke-test-kserve-7f4687897-646df"
Sep 17 17:57:26.528717 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:57:26.528693    2575 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="e2e-test-flow-control-smoke-4f8df83c/fc-smoke-test-kserve-7f4687897-646df"
Sep 17 17:57:27.206818 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:57:27.206787    2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-test-flow-control-smoke-4f8df83c/fc-smoke-test-kserve-7f4687897-646df"
Sep 17 17:57:28.215599 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:57:28.215566    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-djvzt_1d07e68d-9943-420f-a6a0-5f477792fabd/konnectivity-agent/0.log" limit=1024
Sep 17 17:57:33.983390 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:57:33.983352    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-djvzt_1d07e68d-9943-420f-a6a0-5f477792fabd/konnectivity-agent/0.log" limit=1024
Sep 17 17:57:34.424195 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:57:34.424165    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-djvzt_1d07e68d-9943-420f-a6a0-5f477792fabd/konnectivity-agent/0.log" limit=1024
Sep 17 17:57:41.107651 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:57:41.107608    2575 prober.go:120] "Probe failed" probeType="Readiness" pod="e2e-test-llm-inference-service-2541cd65/scheduler-ha-replicas-test-kserve-router-scheduler-557bbc5p9tkw" podUID="68a11ba5-3c58-45c8-9190-c0030a9799a1" containerName="main" probeResult="failure" output="service unhealthy (responded with \"NOT_SERVING\")"
Sep 17 17:57:46.553222 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:57:46.553185    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-djvzt_1d07e68d-9943-420f-a6a0-5f477792fabd/konnectivity-agent/0.log" limit=1024
Sep 17 17:57:51.106978 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:57:51.106942    2575 prober.go:120] "Probe failed" probeType="Readiness" pod="e2e-test-llm-inference-service-2541cd65/scheduler-ha-replicas-test-kserve-router-scheduler-557bbc5p9tkw" podUID="68a11ba5-3c58-45c8-9190-c0030a9799a1" containerName="main" probeResult="failure" output="service unhealthy (responded with \"NOT_SERVING\")"
Sep 17 17:57:51.216152 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:57:51.216115    2575 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["e2e-test-llm-inference-service-2541cd65/scheduler-ha-replicas-test-kserve-router-scheduler-557bbc5p9tkw"]
Sep 17 17:57:51.222562 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:57:51.222515    2575 prober.go:120] "Probe failed" probeType="Readiness" pod="e2e-test-llm-inference-service-2541cd65/scheduler-ha-replicas-test-kserve-router-scheduler-557bbc5p9tkw" podUID="68a11ba5-3c58-45c8-9190-c0030a9799a1" containerName="main" probeResult="failure" output="service unhealthy (responded with \"NOT_SERVING\")"
Sep 17 17:57:51.235817 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:57:51.235781    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-djvzt_1d07e68d-9943-420f-a6a0-5f477792fabd/konnectivity-agent/0.log" limit=1024
Sep 17 17:57:51.321106 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:57:51.321073    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-djvzt_1d07e68d-9943-420f-a6a0-5f477792fabd/konnectivity-agent/0.log" limit=1024
Sep 17 17:57:59.034424 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:57:59.034388    2575 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["e2e-test-flow-control-smoke-4f8df83c/fc-smoke-test-kserve-7f4687897-646df"]
Sep 17 17:57:59.045551 ip-10-0-143-74 kubenswrapper[2575]: E0917 17:57:59.045406    2575 log.go:32] "ExecSync cmd from runtime service failed" err=<
Sep 17 17:57:59.045551 ip-10-0-143-74 kubenswrapper[2575]:         rpc error: code = Unknown desc = command error: executable file `/bin/sleep` not found: No such file or directory
Sep 17 17:57:59.045551 ip-10-0-143-74 kubenswrapper[2575]:         , stdout: , stderr: , exit code -1
Sep 17 17:57:59.045551 ip-10-0-143-74 kubenswrapper[2575]:  > containerID="e461f2131b926e4ab5875bfc82d404bf077bc4a7c084b8516e65e686e7b0d692" cmd=["/bin/sleep","15"]
Sep 17 17:57:59.045551 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:57:59.045518    2575 handlers.go:82] "Exec lifecycle hook for Container in Pod failed" execCommand=["/bin/sleep","15"] containerName="main" pod="e2e-test-flow-control-smoke-4f8df83c/fc-smoke-test-kserve-7f4687897-646df" message="" err=<
Sep 17 17:57:59.045551 ip-10-0-143-74 kubenswrapper[2575]:         rpc error: code = Unknown desc = command error: executable file `/bin/sleep` not found: No such file or directory
Sep 17 17:57:59.045551 ip-10-0-143-74 kubenswrapper[2575]:         , stdout: , stderr: , exit code -1
Sep 17 17:57:59.045551 ip-10-0-143-74 kubenswrapper[2575]:  >
Sep 17 17:57:59.045551 ip-10-0-143-74 kubenswrapper[2575]: E0917 17:57:59.045537    2575 kuberuntime_container.go:755] "PreStop hook failed" err=<
Sep 17 17:57:59.045551 ip-10-0-143-74 kubenswrapper[2575]:         rpc error: code = Unknown desc = command error: executable file `/bin/sleep` not found: No such file or directory
Sep 17 17:57:59.045551 ip-10-0-143-74 kubenswrapper[2575]:         , stdout: , stderr: , exit code -1
Sep 17 17:57:59.045551 ip-10-0-143-74 kubenswrapper[2575]:  > pod="e2e-test-flow-control-smoke-4f8df83c/fc-smoke-test-kserve-7f4687897-646df" podUID="b76e9eae-0573-47b2-bea0-e85ee4360e97" containerName="main" containerID="cri-o://e461f2131b926e4ab5875bfc82d404bf077bc4a7c084b8516e65e686e7b0d692"
Sep 17 17:57:59.046168 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:57:59.045583    2575 kuberuntime_container.go:873] "Killing container with a grace period" pod="e2e-test-flow-control-smoke-4f8df83c/fc-smoke-test-kserve-7f4687897-646df" podUID="b76e9eae-0573-47b2-bea0-e85ee4360e97" containerName="main" containerID="cri-o://e461f2131b926e4ab5875bfc82d404bf077bc4a7c084b8516e65e686e7b0d692" gracePeriod=60
Sep 17 17:57:59.072349 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:57:59.072303    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-djvzt_1d07e68d-9943-420f-a6a0-5f477792fabd/konnectivity-agent/0.log" limit=1024
Sep 17 17:57:59.196634 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:57:59.196587    2575 prober.go:120] "Probe failed" probeType="Readiness" pod="e2e-test-flow-control-smoke-4f8df83c/fc-smoke-test-kserve-7f4687897-646df" podUID="b76e9eae-0573-47b2-bea0-e85ee4360e97" containerName="main" probeResult="failure" output="Get \"https://10.133.0.45:8000/health\": dial tcp 10.133.0.45:8000: connect: connection refused"
Sep 17 17:57:59.275541 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:57:59.275502    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-djvzt_1d07e68d-9943-420f-a6a0-5f477792fabd/konnectivity-agent/0.log" limit=1024
Sep 17 17:57:59.370806 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:57:59.370770    2575 generic.go:356] "Generic (PLEG): container finished" podID="b76e9eae-0573-47b2-bea0-e85ee4360e97" containerID="e461f2131b926e4ab5875bfc82d404bf077bc4a7c084b8516e65e686e7b0d692" exitCode=0
Sep 17 17:57:59.370966 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:57:59.370840    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-test-flow-control-smoke-4f8df83c/fc-smoke-test-kserve-7f4687897-646df" event={"ID":"b76e9eae-0573-47b2-bea0-e85ee4360e97","Type":"ContainerDied","Data":"e461f2131b926e4ab5875bfc82d404bf077bc4a7c084b8516e65e686e7b0d692"}
Sep 17 17:57:59.800904 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:57:59.800879    2575 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-test-flow-control-smoke-4f8df83c/fc-smoke-test-kserve-7f4687897-646df"
Sep 17 17:57:59.953679 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:57:59.953582    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/b76e9eae-0573-47b2-bea0-e85ee4360e97-tmp-dir\") pod \"b76e9eae-0573-47b2-bea0-e85ee4360e97\" (UID: \"b76e9eae-0573-47b2-bea0-e85ee4360e97\") "
Sep 17 17:57:59.953679 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:57:59.953653    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/b76e9eae-0573-47b2-bea0-e85ee4360e97-model-cache\") pod \"b76e9eae-0573-47b2-bea0-e85ee4360e97\" (UID: \"b76e9eae-0573-47b2-bea0-e85ee4360e97\") "
Sep 17 17:57:59.953914 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:57:59.953689    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-krzlf\" (UniqueName: \"kubernetes.io/projected/b76e9eae-0573-47b2-bea0-e85ee4360e97-kube-api-access-krzlf\") pod \"b76e9eae-0573-47b2-bea0-e85ee4360e97\" (UID: \"b76e9eae-0573-47b2-bea0-e85ee4360e97\") "
Sep 17 17:57:59.953914 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:57:59.953750    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/b76e9eae-0573-47b2-bea0-e85ee4360e97-home\") pod \"b76e9eae-0573-47b2-bea0-e85ee4360e97\" (UID: \"b76e9eae-0573-47b2-bea0-e85ee4360e97\") "
Sep 17 17:57:59.953914 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:57:59.953782    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/b76e9eae-0573-47b2-bea0-e85ee4360e97-dshm\") pod \"b76e9eae-0573-47b2-bea0-e85ee4360e97\" (UID: \"b76e9eae-0573-47b2-bea0-e85ee4360e97\") "
Sep 17 17:57:59.953914 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:57:59.953809    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/b76e9eae-0573-47b2-bea0-e85ee4360e97-tls-certs\") pod \"b76e9eae-0573-47b2-bea0-e85ee4360e97\" (UID: \"b76e9eae-0573-47b2-bea0-e85ee4360e97\") "
Sep 17 17:57:59.953914 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:57:59.953845    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b76e9eae-0573-47b2-bea0-e85ee4360e97-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "b76e9eae-0573-47b2-bea0-e85ee4360e97" (UID: "b76e9eae-0573-47b2-bea0-e85ee4360e97"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 17 17:57:59.954129 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:57:59.953961    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b76e9eae-0573-47b2-bea0-e85ee4360e97-home" (OuterVolumeSpecName: "home") pod "b76e9eae-0573-47b2-bea0-e85ee4360e97" (UID: "b76e9eae-0573-47b2-bea0-e85ee4360e97"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 17 17:57:59.954129 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:57:59.953961    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b76e9eae-0573-47b2-bea0-e85ee4360e97-model-cache" (OuterVolumeSpecName: "model-cache") pod "b76e9eae-0573-47b2-bea0-e85ee4360e97" (UID: "b76e9eae-0573-47b2-bea0-e85ee4360e97"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 17 17:57:59.954129 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:57:59.954110    2575 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/b76e9eae-0573-47b2-bea0-e85ee4360e97-tmp-dir\") on node \"ip-10-0-143-74.ec2.internal\" DevicePath \"\""
Sep 17 17:57:59.954129 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:57:59.954122    2575 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/b76e9eae-0573-47b2-bea0-e85ee4360e97-model-cache\") on node \"ip-10-0-143-74.ec2.internal\" DevicePath \"\""
Sep 17 17:57:59.954263 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:57:59.954134    2575 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/b76e9eae-0573-47b2-bea0-e85ee4360e97-home\") on node \"ip-10-0-143-74.ec2.internal\" DevicePath \"\""
Sep 17 17:57:59.956099 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:57:59.956070    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/b76e9eae-0573-47b2-bea0-e85ee4360e97-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "b76e9eae-0573-47b2-bea0-e85ee4360e97" (UID: "b76e9eae-0573-47b2-bea0-e85ee4360e97"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 17 17:57:59.956195 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:57:59.956082    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/b76e9eae-0573-47b2-bea0-e85ee4360e97-kube-api-access-krzlf" (OuterVolumeSpecName: "kube-api-access-krzlf") pod "b76e9eae-0573-47b2-bea0-e85ee4360e97" (UID: "b76e9eae-0573-47b2-bea0-e85ee4360e97"). InnerVolumeSpecName "kube-api-access-krzlf". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 17 17:57:59.956195 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:57:59.956135    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b76e9eae-0573-47b2-bea0-e85ee4360e97-dshm" (OuterVolumeSpecName: "dshm") pod "b76e9eae-0573-47b2-bea0-e85ee4360e97" (UID: "b76e9eae-0573-47b2-bea0-e85ee4360e97"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 17 17:58:00.055020 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:58:00.054986    2575 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-krzlf\" (UniqueName: \"kubernetes.io/projected/b76e9eae-0573-47b2-bea0-e85ee4360e97-kube-api-access-krzlf\") on node \"ip-10-0-143-74.ec2.internal\" DevicePath \"\""
Sep 17 17:58:00.055020 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:58:00.055015    2575 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/b76e9eae-0573-47b2-bea0-e85ee4360e97-dshm\") on node \"ip-10-0-143-74.ec2.internal\" DevicePath \"\""
Sep 17 17:58:00.055584 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:58:00.055031    2575 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/b76e9eae-0573-47b2-bea0-e85ee4360e97-tls-certs\") on node \"ip-10-0-143-74.ec2.internal\" DevicePath \"\""
Sep 17 17:58:00.378275 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:58:00.378238    2575 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-test-flow-control-smoke-4f8df83c/fc-smoke-test-kserve-7f4687897-646df"
Sep 17 17:58:00.378461 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:58:00.378273    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-test-flow-control-smoke-4f8df83c/fc-smoke-test-kserve-7f4687897-646df" event={"ID":"b76e9eae-0573-47b2-bea0-e85ee4360e97","Type":"ContainerDied","Data":"b05c2e3bd8d134b7a6e5d05f4b6d1aa7588a1370c596d9826bac7c04f1292162"}
Sep 17 17:58:00.378461 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:58:00.378335    2575 scope.go:117] "RemoveContainer" containerID="e461f2131b926e4ab5875bfc82d404bf077bc4a7c084b8516e65e686e7b0d692"
Sep 17 17:58:00.400410 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:58:00.400387    2575 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["e2e-test-flow-control-smoke-4f8df83c/fc-smoke-test-kserve-7f4687897-646df"]
Sep 17 17:58:00.405659 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:58:00.405631    2575 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["e2e-test-flow-control-smoke-4f8df83c/fc-smoke-test-kserve-7f4687897-646df"]
Sep 17 17:58:00.968124 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:58:00.968085    2575 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="b76e9eae-0573-47b2-bea0-e85ee4360e97" path="/var/lib/kubelet/pods/b76e9eae-0573-47b2-bea0-e85ee4360e97/volumes"
Sep 17 17:58:01.218912 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:58:01.218827    2575 prober.go:120] "Probe failed" probeType="Readiness" pod="e2e-test-llm-inference-service-2541cd65/scheduler-ha-replicas-test-kserve-router-scheduler-557bbc5p9tkw" podUID="68a11ba5-3c58-45c8-9190-c0030a9799a1" containerName="main" probeResult="failure" output="service unhealthy (responded with \"NOT_SERVING\")"
Sep 17 17:58:06.242102 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:58:06.242036    2575 kuberuntime_container.go:873] "Killing container with a grace period" pod="e2e-test-llm-inference-service-2541cd65/scheduler-ha-replicas-test-kserve-router-scheduler-557bbc5p9tkw" podUID="68a11ba5-3c58-45c8-9190-c0030a9799a1" containerName="main" containerID="cri-o://05e19c488b5f40797e234def570e19498919981cd960c6318f1dbab2227cf6c6" gracePeriod=45
Sep 17 17:58:11.218496 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:58:11.218459    2575 prober.go:120] "Probe failed" probeType="Readiness" pod="e2e-test-llm-inference-service-2541cd65/scheduler-ha-replicas-test-kserve-router-scheduler-557bbc5p9tkw" podUID="68a11ba5-3c58-45c8-9190-c0030a9799a1" containerName="main" probeResult="failure" output="service unhealthy (responded with \"NOT_SERVING\")"
Sep 17 17:58:21.218495 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:58:21.218431    2575 prober.go:120] "Probe failed" probeType="Readiness" pod="e2e-test-llm-inference-service-2541cd65/scheduler-ha-replicas-test-kserve-router-scheduler-557bbc5p9tkw" podUID="68a11ba5-3c58-45c8-9190-c0030a9799a1" containerName="main" probeResult="failure" output="service unhealthy (responded with \"NOT_SERVING\")"
Sep 17 17:58:25.269335 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:58:25.269298    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-djvzt_1d07e68d-9943-420f-a6a0-5f477792fabd/konnectivity-agent/0.log" limit=1024
Sep 17 17:58:31.219161 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:58:31.219124    2575 prober.go:120] "Probe failed" probeType="Readiness" pod="e2e-test-llm-inference-service-2541cd65/scheduler-ha-replicas-test-kserve-router-scheduler-557bbc5p9tkw" podUID="68a11ba5-3c58-45c8-9190-c0030a9799a1" containerName="main" probeResult="failure" output="service unhealthy (responded with \"NOT_SERVING\")"
Sep 17 17:58:36.488987 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:58:36.488961    2575 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-test-llm-inference-service-2541cd65/scheduler-ha-replicas-test-kserve-router-scheduler-557bbc5p9tkw"
Sep 17 17:58:36.564895 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:58:36.564813    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-q7gf5\" (UniqueName: \"kubernetes.io/projected/68a11ba5-3c58-45c8-9190-c0030a9799a1-kube-api-access-q7gf5\") pod \"68a11ba5-3c58-45c8-9190-c0030a9799a1\" (UID: \"68a11ba5-3c58-45c8-9190-c0030a9799a1\") "
Sep 17 17:58:36.564895 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:58:36.564866    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/68a11ba5-3c58-45c8-9190-c0030a9799a1-tls-certs\") pod \"68a11ba5-3c58-45c8-9190-c0030a9799a1\" (UID: \"68a11ba5-3c58-45c8-9190-c0030a9799a1\") "
Sep 17 17:58:36.567007 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:58:36.566974    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/68a11ba5-3c58-45c8-9190-c0030a9799a1-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "68a11ba5-3c58-45c8-9190-c0030a9799a1" (UID: "68a11ba5-3c58-45c8-9190-c0030a9799a1"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 17 17:58:36.567007 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:58:36.566989    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/68a11ba5-3c58-45c8-9190-c0030a9799a1-kube-api-access-q7gf5" (OuterVolumeSpecName: "kube-api-access-q7gf5") pod "68a11ba5-3c58-45c8-9190-c0030a9799a1" (UID: "68a11ba5-3c58-45c8-9190-c0030a9799a1"). InnerVolumeSpecName "kube-api-access-q7gf5". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 17 17:58:36.581561 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:58:36.581538    2575 generic.go:356] "Generic (PLEG): container finished" podID="68a11ba5-3c58-45c8-9190-c0030a9799a1" containerID="05e19c488b5f40797e234def570e19498919981cd960c6318f1dbab2227cf6c6" exitCode=0
Sep 17 17:58:36.581677 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:58:36.581581    2575 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-test-llm-inference-service-2541cd65/scheduler-ha-replicas-test-kserve-router-scheduler-557bbc5p9tkw"
Sep 17 17:58:36.581677 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:58:36.581622    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-test-llm-inference-service-2541cd65/scheduler-ha-replicas-test-kserve-router-scheduler-557bbc5p9tkw" event={"ID":"68a11ba5-3c58-45c8-9190-c0030a9799a1","Type":"ContainerDied","Data":"05e19c488b5f40797e234def570e19498919981cd960c6318f1dbab2227cf6c6"}
Sep 17 17:58:36.581790 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:58:36.581680    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-test-llm-inference-service-2541cd65/scheduler-ha-replicas-test-kserve-router-scheduler-557bbc5p9tkw" event={"ID":"68a11ba5-3c58-45c8-9190-c0030a9799a1","Type":"ContainerDied","Data":"d1f66684ed6ea318a9a1d18a392b0004cfe4d1f93ccf5f944371c5ec15f3ea9c"}
Sep 17 17:58:36.581790 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:58:36.581713    2575 scope.go:117] "RemoveContainer" containerID="05e19c488b5f40797e234def570e19498919981cd960c6318f1dbab2227cf6c6"
Sep 17 17:58:36.590945 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:58:36.590925    2575 scope.go:117] "RemoveContainer" containerID="05e19c488b5f40797e234def570e19498919981cd960c6318f1dbab2227cf6c6"
Sep 17 17:58:36.591249 ip-10-0-143-74 kubenswrapper[2575]: E0917 17:58:36.591225    2575 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"05e19c488b5f40797e234def570e19498919981cd960c6318f1dbab2227cf6c6\": container with ID starting with 05e19c488b5f40797e234def570e19498919981cd960c6318f1dbab2227cf6c6 not found: ID does not exist" containerID="05e19c488b5f40797e234def570e19498919981cd960c6318f1dbab2227cf6c6"
Sep 17 17:58:36.591313 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:58:36.591257    2575 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"05e19c488b5f40797e234def570e19498919981cd960c6318f1dbab2227cf6c6"} err="failed to get container status \"05e19c488b5f40797e234def570e19498919981cd960c6318f1dbab2227cf6c6\": rpc error: code = NotFound desc = could not find container \"05e19c488b5f40797e234def570e19498919981cd960c6318f1dbab2227cf6c6\": container with ID starting with 05e19c488b5f40797e234def570e19498919981cd960c6318f1dbab2227cf6c6 not found: ID does not exist"
Sep 17 17:58:36.621007 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:58:36.620980    2575 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["e2e-test-llm-inference-service-2541cd65/scheduler-ha-replicas-test-kserve-router-scheduler-557bbc5p9tkw"]
Sep 17 17:58:36.626970 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:58:36.626947    2575 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["e2e-test-llm-inference-service-2541cd65/scheduler-ha-replicas-test-kserve-router-scheduler-557bbc5p9tkw"]
Sep 17 17:58:36.666251 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:58:36.666226    2575 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/68a11ba5-3c58-45c8-9190-c0030a9799a1-tls-certs\") on node \"ip-10-0-143-74.ec2.internal\" DevicePath \"\""
Sep 17 17:58:36.666251 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:58:36.666246    2575 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-q7gf5\" (UniqueName: \"kubernetes.io/projected/68a11ba5-3c58-45c8-9190-c0030a9799a1-kube-api-access-q7gf5\") on node \"ip-10-0-143-74.ec2.internal\" DevicePath \"\""
Sep 17 17:58:36.967067 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:58:36.967035    2575 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="68a11ba5-3c58-45c8-9190-c0030a9799a1" path="/var/lib/kubelet/pods/68a11ba5-3c58-45c8-9190-c0030a9799a1/volumes"
Sep 17 17:58:39.589863 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:58:39.589829    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-djvzt_1d07e68d-9943-420f-a6a0-5f477792fabd/konnectivity-agent/0.log" limit=1024
Sep 17 17:58:39.686959 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:58:39.686926    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-djvzt_1d07e68d-9943-420f-a6a0-5f477792fabd/konnectivity-agent/0.log" limit=1024
Sep 17 17:58:46.189730 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:58:46.189693    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-djvzt_1d07e68d-9943-420f-a6a0-5f477792fabd/konnectivity-agent/0.log" limit=1024
Sep 17 17:58:49.077706 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:58:49.077667    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-djvzt_1d07e68d-9943-420f-a6a0-5f477792fabd/konnectivity-agent/0.log" limit=1024
Sep 17 17:58:49.188273 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:58:49.188237    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-djvzt_1d07e68d-9943-420f-a6a0-5f477792fabd/konnectivity-agent/0.log" limit=1024
Sep 17 17:58:52.132735 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:58:52.132705    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-djvzt_1d07e68d-9943-420f-a6a0-5f477792fabd/konnectivity-agent/0.log" limit=1024
Sep 17 17:58:52.246109 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:58:52.246075    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-djvzt_1d07e68d-9943-420f-a6a0-5f477792fabd/konnectivity-agent/0.log" limit=1024
Sep 17 17:58:55.227822 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:58:55.227786    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-djvzt_1d07e68d-9943-420f-a6a0-5f477792fabd/konnectivity-agent/0.log" limit=1024
Sep 17 17:59:01.165420 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:59:01.165389    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-djvzt_1d07e68d-9943-420f-a6a0-5f477792fabd/konnectivity-agent/0.log" limit=1024
Sep 17 17:59:01.239017 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:59:01.238989    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-djvzt_1d07e68d-9943-420f-a6a0-5f477792fabd/konnectivity-agent/0.log" limit=1024
Sep 17 17:59:02.299598 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:59:02.299568    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-djvzt_1d07e68d-9943-420f-a6a0-5f477792fabd/konnectivity-agent/0.log" limit=1024
Sep 17 17:59:02.499259 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:59:02.499218    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-djvzt_1d07e68d-9943-420f-a6a0-5f477792fabd/konnectivity-agent/0.log" limit=1024
Sep 17 17:59:08.821159 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:59:08.821122    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-djvzt_1d07e68d-9943-420f-a6a0-5f477792fabd/konnectivity-agent/0.log" limit=1024
Sep 17 17:59:22.005000 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:59:22.004965    2575 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["e2e-test-flow-control-smoke-1e98f46d/fc-concurrency-test-kserve-b5d44467c-bq7hv"]
Sep 17 17:59:22.005430 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:59:22.005414    2575 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b76e9eae-0573-47b2-bea0-e85ee4360e97" containerName="main"
Sep 17 17:59:22.005507 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:59:22.005432    2575 state_mem.go:107] "Deleted CPUSet assignment" podUID="b76e9eae-0573-47b2-bea0-e85ee4360e97" containerName="main"
Sep 17 17:59:22.005507 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:59:22.005478    2575 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="68a11ba5-3c58-45c8-9190-c0030a9799a1" containerName="main"
Sep 17 17:59:22.005507 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:59:22.005486    2575 state_mem.go:107] "Deleted CPUSet assignment" podUID="68a11ba5-3c58-45c8-9190-c0030a9799a1" containerName="main"
Sep 17 17:59:22.005622 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:59:22.005593    2575 memory_manager.go:368] "RemoveStaleState removing state" podUID="68a11ba5-3c58-45c8-9190-c0030a9799a1" containerName="main"
Sep 17 17:59:22.005622 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:59:22.005606    2575 memory_manager.go:368] "RemoveStaleState removing state" podUID="b76e9eae-0573-47b2-bea0-e85ee4360e97" containerName="main"
Sep 17 17:59:22.008715 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:59:22.008696    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-test-flow-control-smoke-1e98f46d/fc-concurrency-test-kserve-b5d44467c-bq7hv"
Sep 17 17:59:22.011404 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:59:22.011382    2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-test-flow-control-smoke-1e98f46d\"/\"kube-root-ca.crt\""
Sep 17 17:59:22.012357 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:59:22.012338    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-test-flow-control-smoke-1e98f46d\"/\"default-dockercfg-kg6s5\""
Sep 17 17:59:22.012473 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:59:22.012382    2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-test-flow-control-smoke-1e98f46d\"/\"openshift-service-ca.crt\""
Sep 17 17:59:22.012473 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:59:22.012389    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-test-flow-control-smoke-1e98f46d\"/\"fc-concurrency-test-kserve-self-signed-certs\""
Sep 17 17:59:22.019666 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:59:22.019643    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["e2e-test-flow-control-smoke-1e98f46d/fc-concurrency-test-kserve-b5d44467c-bq7hv"]
Sep 17 17:59:22.031867 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:59:22.031838    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-djvzt_1d07e68d-9943-420f-a6a0-5f477792fabd/konnectivity-agent/0.log" limit=1024
Sep 17 17:59:22.048858 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:59:22.048831    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/753c9293-bc29-4523-aa9e-25dd0db6893a-model-cache\") pod \"fc-concurrency-test-kserve-b5d44467c-bq7hv\" (UID: \"753c9293-bc29-4523-aa9e-25dd0db6893a\") " pod="e2e-test-flow-control-smoke-1e98f46d/fc-concurrency-test-kserve-b5d44467c-bq7hv"
Sep 17 17:59:22.048981 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:59:22.048895    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/753c9293-bc29-4523-aa9e-25dd0db6893a-tls-certs\") pod \"fc-concurrency-test-kserve-b5d44467c-bq7hv\" (UID: \"753c9293-bc29-4523-aa9e-25dd0db6893a\") " pod="e2e-test-flow-control-smoke-1e98f46d/fc-concurrency-test-kserve-b5d44467c-bq7hv"
Sep 17 17:59:22.049086 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:59:22.049063    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/753c9293-bc29-4523-aa9e-25dd0db6893a-home\") pod \"fc-concurrency-test-kserve-b5d44467c-bq7hv\" (UID: \"753c9293-bc29-4523-aa9e-25dd0db6893a\") " pod="e2e-test-flow-control-smoke-1e98f46d/fc-concurrency-test-kserve-b5d44467c-bq7hv"
Sep 17 17:59:22.049179 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:59:22.049151    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xpfk7\" (UniqueName: \"kubernetes.io/projected/753c9293-bc29-4523-aa9e-25dd0db6893a-kube-api-access-xpfk7\") pod \"fc-concurrency-test-kserve-b5d44467c-bq7hv\" (UID: \"753c9293-bc29-4523-aa9e-25dd0db6893a\") " pod="e2e-test-flow-control-smoke-1e98f46d/fc-concurrency-test-kserve-b5d44467c-bq7hv"
Sep 17 17:59:22.049308 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:59:22.049258    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/753c9293-bc29-4523-aa9e-25dd0db6893a-tmp-dir\") pod \"fc-concurrency-test-kserve-b5d44467c-bq7hv\" (UID: \"753c9293-bc29-4523-aa9e-25dd0db6893a\") " pod="e2e-test-flow-control-smoke-1e98f46d/fc-concurrency-test-kserve-b5d44467c-bq7hv"
Sep 17 17:59:22.049373 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:59:22.049313    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/753c9293-bc29-4523-aa9e-25dd0db6893a-dshm\") pod \"fc-concurrency-test-kserve-b5d44467c-bq7hv\" (UID: \"753c9293-bc29-4523-aa9e-25dd0db6893a\") " pod="e2e-test-flow-control-smoke-1e98f46d/fc-concurrency-test-kserve-b5d44467c-bq7hv"
Sep 17 17:59:22.149851 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:59:22.149816    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/753c9293-bc29-4523-aa9e-25dd0db6893a-tmp-dir\") pod \"fc-concurrency-test-kserve-b5d44467c-bq7hv\" (UID: \"753c9293-bc29-4523-aa9e-25dd0db6893a\") " pod="e2e-test-flow-control-smoke-1e98f46d/fc-concurrency-test-kserve-b5d44467c-bq7hv"
Sep 17 17:59:22.150067 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:59:22.149866    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/753c9293-bc29-4523-aa9e-25dd0db6893a-dshm\") pod \"fc-concurrency-test-kserve-b5d44467c-bq7hv\" (UID: \"753c9293-bc29-4523-aa9e-25dd0db6893a\") " pod="e2e-test-flow-control-smoke-1e98f46d/fc-concurrency-test-kserve-b5d44467c-bq7hv"
Sep 17 17:59:22.150067 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:59:22.149908    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/753c9293-bc29-4523-aa9e-25dd0db6893a-model-cache\") pod \"fc-concurrency-test-kserve-b5d44467c-bq7hv\" (UID: \"753c9293-bc29-4523-aa9e-25dd0db6893a\") " pod="e2e-test-flow-control-smoke-1e98f46d/fc-concurrency-test-kserve-b5d44467c-bq7hv"
Sep 17 17:59:22.150067 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:59:22.150049    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/753c9293-bc29-4523-aa9e-25dd0db6893a-tls-certs\") pod \"fc-concurrency-test-kserve-b5d44467c-bq7hv\" (UID: \"753c9293-bc29-4523-aa9e-25dd0db6893a\") " pod="e2e-test-flow-control-smoke-1e98f46d/fc-concurrency-test-kserve-b5d44467c-bq7hv"
Sep 17 17:59:22.150258 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:59:22.150145    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/753c9293-bc29-4523-aa9e-25dd0db6893a-home\") pod \"fc-concurrency-test-kserve-b5d44467c-bq7hv\" (UID: \"753c9293-bc29-4523-aa9e-25dd0db6893a\") " pod="e2e-test-flow-control-smoke-1e98f46d/fc-concurrency-test-kserve-b5d44467c-bq7hv"
Sep 17 17:59:22.150258 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:59:22.150177    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xpfk7\" (UniqueName: \"kubernetes.io/projected/753c9293-bc29-4523-aa9e-25dd0db6893a-kube-api-access-xpfk7\") pod \"fc-concurrency-test-kserve-b5d44467c-bq7hv\" (UID: \"753c9293-bc29-4523-aa9e-25dd0db6893a\") " pod="e2e-test-flow-control-smoke-1e98f46d/fc-concurrency-test-kserve-b5d44467c-bq7hv"
Sep 17 17:59:22.150258 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:59:22.150185    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/753c9293-bc29-4523-aa9e-25dd0db6893a-tmp-dir\") pod \"fc-concurrency-test-kserve-b5d44467c-bq7hv\" (UID: \"753c9293-bc29-4523-aa9e-25dd0db6893a\") " pod="e2e-test-flow-control-smoke-1e98f46d/fc-concurrency-test-kserve-b5d44467c-bq7hv"
Sep 17 17:59:22.150258 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:59:22.150238    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/753c9293-bc29-4523-aa9e-25dd0db6893a-model-cache\") pod \"fc-concurrency-test-kserve-b5d44467c-bq7hv\" (UID: \"753c9293-bc29-4523-aa9e-25dd0db6893a\") " pod="e2e-test-flow-control-smoke-1e98f46d/fc-concurrency-test-kserve-b5d44467c-bq7hv"
Sep 17 17:59:22.150560 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:59:22.150537    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/753c9293-bc29-4523-aa9e-25dd0db6893a-home\") pod \"fc-concurrency-test-kserve-b5d44467c-bq7hv\" (UID: \"753c9293-bc29-4523-aa9e-25dd0db6893a\") " pod="e2e-test-flow-control-smoke-1e98f46d/fc-concurrency-test-kserve-b5d44467c-bq7hv"
Sep 17 17:59:22.152161 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:59:22.152139    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/753c9293-bc29-4523-aa9e-25dd0db6893a-dshm\") pod \"fc-concurrency-test-kserve-b5d44467c-bq7hv\" (UID: \"753c9293-bc29-4523-aa9e-25dd0db6893a\") " pod="e2e-test-flow-control-smoke-1e98f46d/fc-concurrency-test-kserve-b5d44467c-bq7hv"
Sep 17 17:59:22.152430 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:59:22.152410    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/753c9293-bc29-4523-aa9e-25dd0db6893a-tls-certs\") pod \"fc-concurrency-test-kserve-b5d44467c-bq7hv\" (UID: \"753c9293-bc29-4523-aa9e-25dd0db6893a\") " pod="e2e-test-flow-control-smoke-1e98f46d/fc-concurrency-test-kserve-b5d44467c-bq7hv"
Sep 17 17:59:22.159900 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:59:22.159878    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-xpfk7\" (UniqueName: \"kubernetes.io/projected/753c9293-bc29-4523-aa9e-25dd0db6893a-kube-api-access-xpfk7\") pod \"fc-concurrency-test-kserve-b5d44467c-bq7hv\" (UID: \"753c9293-bc29-4523-aa9e-25dd0db6893a\") " pod="e2e-test-flow-control-smoke-1e98f46d/fc-concurrency-test-kserve-b5d44467c-bq7hv"
Sep 17 17:59:22.244319 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:59:22.244284    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-djvzt_1d07e68d-9943-420f-a6a0-5f477792fabd/konnectivity-agent/0.log" limit=1024
Sep 17 17:59:22.320591 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:59:22.320513    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-test-flow-control-smoke-1e98f46d/fc-concurrency-test-kserve-b5d44467c-bq7hv"
Sep 17 17:59:22.599422 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:59:22.599392    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["e2e-test-flow-control-smoke-1e98f46d/fc-concurrency-test-kserve-b5d44467c-bq7hv"]
Sep 17 17:59:22.601214 ip-10-0-143-74 kubenswrapper[2575]: W0917 17:59:22.601182    2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod753c9293_bc29_4523_aa9e_25dd0db6893a.slice/crio-b4105f5f071e8567d4f3b195b7f7c656613813c55a9fd083a5ec227e9c2303fc WatchSource:0}: Error finding container b4105f5f071e8567d4f3b195b7f7c656613813c55a9fd083a5ec227e9c2303fc: Status 404 returned error can't find the container with id b4105f5f071e8567d4f3b195b7f7c656613813c55a9fd083a5ec227e9c2303fc
Sep 17 17:59:22.840525 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:59:22.840489    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-test-flow-control-smoke-1e98f46d/fc-concurrency-test-kserve-b5d44467c-bq7hv" event={"ID":"753c9293-bc29-4523-aa9e-25dd0db6893a","Type":"ContainerStarted","Data":"f7b32b3758051f37c65e7f23ca53692de6e1413cafae2e25600dc5632bd1fac0"}
Sep 17 17:59:22.840674 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:59:22.840540    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-test-flow-control-smoke-1e98f46d/fc-concurrency-test-kserve-b5d44467c-bq7hv" event={"ID":"753c9293-bc29-4523-aa9e-25dd0db6893a","Type":"ContainerStarted","Data":"b4105f5f071e8567d4f3b195b7f7c656613813c55a9fd083a5ec227e9c2303fc"}
Sep 17 17:59:22.894671 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:59:22.894619    2575 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="e2e-test-flow-control-smoke-1e98f46d/fc-concurrency-test-kserve-b5d44467c-bq7hv" podStartSLOduration=1.89460198 podStartE2EDuration="1.89460198s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-17 17:59:21 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-17 17:59:22.891466518 +0000 UTC m=+1482.521796488" watchObservedRunningTime="2026-09-17 17:59:22.89460198 +0000 UTC m=+1482.524931955"
Sep 17 17:59:23.321009 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:59:23.320930    2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-test-flow-control-smoke-1e98f46d/fc-concurrency-test-kserve-b5d44467c-bq7hv"
Sep 17 17:59:32.321169 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:59:32.321129    2575 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="e2e-test-flow-control-smoke-1e98f46d/fc-concurrency-test-kserve-b5d44467c-bq7hv"
Sep 17 17:59:32.333763 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:59:32.333734    2575 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="e2e-test-flow-control-smoke-1e98f46d/fc-concurrency-test-kserve-b5d44467c-bq7hv"
Sep 17 17:59:32.908473 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:59:32.908429    2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-test-flow-control-smoke-1e98f46d/fc-concurrency-test-kserve-b5d44467c-bq7hv"
Sep 17 17:59:34.319645 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:59:34.319590    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-djvzt_1d07e68d-9943-420f-a6a0-5f477792fabd/konnectivity-agent/0.log" limit=1024
Sep 17 17:59:35.298919 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:59:35.298808    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-djvzt_1d07e68d-9943-420f-a6a0-5f477792fabd/konnectivity-agent/0.log" limit=1024
Sep 17 17:59:39.420947 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:59:39.420911    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-djvzt_1d07e68d-9943-420f-a6a0-5f477792fabd/konnectivity-agent/0.log" limit=1024
Sep 17 17:59:39.486329 ip-10-0-143-74 kubenswrapper[2575]: I0917 17:59:39.486290    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-djvzt_1d07e68d-9943-420f-a6a0-5f477792fabd/konnectivity-agent/0.log" limit=1024
Sep 17 18:00:00.649497 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:00:00.649458    2575 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["e2e-test-flow-control-smoke-1e98f46d/fc-concurrency-test-kserve-b5d44467c-bq7hv"]
Sep 17 18:00:00.659117 ip-10-0-143-74 kubenswrapper[2575]: E0917 18:00:00.659030    2575 log.go:32] "ExecSync cmd from runtime service failed" err=<
Sep 17 18:00:00.659117 ip-10-0-143-74 kubenswrapper[2575]:         rpc error: code = Unknown desc = command error: executable file `/bin/sleep` not found: No such file or directory
Sep 17 18:00:00.659117 ip-10-0-143-74 kubenswrapper[2575]:         , stdout: , stderr: , exit code -1
Sep 17 18:00:00.659117 ip-10-0-143-74 kubenswrapper[2575]:  > containerID="f7b32b3758051f37c65e7f23ca53692de6e1413cafae2e25600dc5632bd1fac0" cmd=["/bin/sleep","15"]
Sep 17 18:00:00.659570 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:00:00.659542    2575 handlers.go:82] "Exec lifecycle hook for Container in Pod failed" execCommand=["/bin/sleep","15"] containerName="main" pod="e2e-test-flow-control-smoke-1e98f46d/fc-concurrency-test-kserve-b5d44467c-bq7hv" message="" err=<
Sep 17 18:00:00.659570 ip-10-0-143-74 kubenswrapper[2575]:         rpc error: code = Unknown desc = command error: executable file `/bin/sleep` not found: No such file or directory
Sep 17 18:00:00.659570 ip-10-0-143-74 kubenswrapper[2575]:         , stdout: , stderr: , exit code -1
Sep 17 18:00:00.659570 ip-10-0-143-74 kubenswrapper[2575]:  >
Sep 17 18:00:00.659781 ip-10-0-143-74 kubenswrapper[2575]: E0917 18:00:00.659587    2575 kuberuntime_container.go:755] "PreStop hook failed" err=<
Sep 17 18:00:00.659781 ip-10-0-143-74 kubenswrapper[2575]:         rpc error: code = Unknown desc = command error: executable file `/bin/sleep` not found: No such file or directory
Sep 17 18:00:00.659781 ip-10-0-143-74 kubenswrapper[2575]:         , stdout: , stderr: , exit code -1
Sep 17 18:00:00.659781 ip-10-0-143-74 kubenswrapper[2575]:  > pod="e2e-test-flow-control-smoke-1e98f46d/fc-concurrency-test-kserve-b5d44467c-bq7hv" podUID="753c9293-bc29-4523-aa9e-25dd0db6893a" containerName="main" containerID="cri-o://f7b32b3758051f37c65e7f23ca53692de6e1413cafae2e25600dc5632bd1fac0"
Sep 17 18:00:00.659781 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:00:00.659684    2575 kuberuntime_container.go:873] "Killing container with a grace period" pod="e2e-test-flow-control-smoke-1e98f46d/fc-concurrency-test-kserve-b5d44467c-bq7hv" podUID="753c9293-bc29-4523-aa9e-25dd0db6893a" containerName="main" containerID="cri-o://f7b32b3758051f37c65e7f23ca53692de6e1413cafae2e25600dc5632bd1fac0" gracePeriod=60
Sep 17 18:00:00.687216 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:00:00.687190    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-djvzt_1d07e68d-9943-420f-a6a0-5f477792fabd/konnectivity-agent/0.log" limit=1024
Sep 17 18:00:00.754161 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:00:00.754127    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-djvzt_1d07e68d-9943-420f-a6a0-5f477792fabd/konnectivity-agent/0.log" limit=1024
Sep 17 18:00:00.898235 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:00:00.898193    2575 prober.go:120] "Probe failed" probeType="Readiness" pod="e2e-test-flow-control-smoke-1e98f46d/fc-concurrency-test-kserve-b5d44467c-bq7hv" podUID="753c9293-bc29-4523-aa9e-25dd0db6893a" containerName="main" probeResult="failure" output="Get \"https://10.133.0.46:8000/health\": dial tcp 10.133.0.46:8000: connect: connection refused"
Sep 17 18:00:01.051021 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:00:01.050932    2575 generic.go:356] "Generic (PLEG): container finished" podID="753c9293-bc29-4523-aa9e-25dd0db6893a" containerID="f7b32b3758051f37c65e7f23ca53692de6e1413cafae2e25600dc5632bd1fac0" exitCode=0
Sep 17 18:00:01.051208 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:00:01.051013    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-test-flow-control-smoke-1e98f46d/fc-concurrency-test-kserve-b5d44467c-bq7hv" event={"ID":"753c9293-bc29-4523-aa9e-25dd0db6893a","Type":"ContainerDied","Data":"f7b32b3758051f37c65e7f23ca53692de6e1413cafae2e25600dc5632bd1fac0"}
Sep 17 18:00:01.414895 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:00:01.414872    2575 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-test-flow-control-smoke-1e98f46d/fc-concurrency-test-kserve-b5d44467c-bq7hv"
Sep 17 18:00:01.515401 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:00:01.515364    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/753c9293-bc29-4523-aa9e-25dd0db6893a-model-cache\") pod \"753c9293-bc29-4523-aa9e-25dd0db6893a\" (UID: \"753c9293-bc29-4523-aa9e-25dd0db6893a\") "
Sep 17 18:00:01.515581 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:00:01.515409    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/753c9293-bc29-4523-aa9e-25dd0db6893a-dshm\") pod \"753c9293-bc29-4523-aa9e-25dd0db6893a\" (UID: \"753c9293-bc29-4523-aa9e-25dd0db6893a\") "
Sep 17 18:00:01.515581 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:00:01.515478    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/753c9293-bc29-4523-aa9e-25dd0db6893a-home\") pod \"753c9293-bc29-4523-aa9e-25dd0db6893a\" (UID: \"753c9293-bc29-4523-aa9e-25dd0db6893a\") "
Sep 17 18:00:01.515581 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:00:01.515515    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/753c9293-bc29-4523-aa9e-25dd0db6893a-tmp-dir\") pod \"753c9293-bc29-4523-aa9e-25dd0db6893a\" (UID: \"753c9293-bc29-4523-aa9e-25dd0db6893a\") "
Sep 17 18:00:01.515581 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:00:01.515558    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/753c9293-bc29-4523-aa9e-25dd0db6893a-tls-certs\") pod \"753c9293-bc29-4523-aa9e-25dd0db6893a\" (UID: \"753c9293-bc29-4523-aa9e-25dd0db6893a\") "
Sep 17 18:00:01.515581 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:00:01.515579    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-xpfk7\" (UniqueName: \"kubernetes.io/projected/753c9293-bc29-4523-aa9e-25dd0db6893a-kube-api-access-xpfk7\") pod \"753c9293-bc29-4523-aa9e-25dd0db6893a\" (UID: \"753c9293-bc29-4523-aa9e-25dd0db6893a\") "
Sep 17 18:00:01.515866 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:00:01.515747    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/753c9293-bc29-4523-aa9e-25dd0db6893a-model-cache" (OuterVolumeSpecName: "model-cache") pod "753c9293-bc29-4523-aa9e-25dd0db6893a" (UID: "753c9293-bc29-4523-aa9e-25dd0db6893a"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 17 18:00:01.515866 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:00:01.515799    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/753c9293-bc29-4523-aa9e-25dd0db6893a-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "753c9293-bc29-4523-aa9e-25dd0db6893a" (UID: "753c9293-bc29-4523-aa9e-25dd0db6893a"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 17 18:00:01.515866 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:00:01.515735    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/753c9293-bc29-4523-aa9e-25dd0db6893a-home" (OuterVolumeSpecName: "home") pod "753c9293-bc29-4523-aa9e-25dd0db6893a" (UID: "753c9293-bc29-4523-aa9e-25dd0db6893a"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 17 18:00:01.516032 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:00:01.515895    2575 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/753c9293-bc29-4523-aa9e-25dd0db6893a-model-cache\") on node \"ip-10-0-143-74.ec2.internal\" DevicePath \"\""
Sep 17 18:00:01.516032 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:00:01.515912    2575 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/753c9293-bc29-4523-aa9e-25dd0db6893a-home\") on node \"ip-10-0-143-74.ec2.internal\" DevicePath \"\""
Sep 17 18:00:01.516032 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:00:01.515924    2575 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/753c9293-bc29-4523-aa9e-25dd0db6893a-tmp-dir\") on node \"ip-10-0-143-74.ec2.internal\" DevicePath \"\""
Sep 17 18:00:01.517828 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:00:01.517795    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/753c9293-bc29-4523-aa9e-25dd0db6893a-dshm" (OuterVolumeSpecName: "dshm") pod "753c9293-bc29-4523-aa9e-25dd0db6893a" (UID: "753c9293-bc29-4523-aa9e-25dd0db6893a"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 17 18:00:01.517954 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:00:01.517884    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/753c9293-bc29-4523-aa9e-25dd0db6893a-kube-api-access-xpfk7" (OuterVolumeSpecName: "kube-api-access-xpfk7") pod "753c9293-bc29-4523-aa9e-25dd0db6893a" (UID: "753c9293-bc29-4523-aa9e-25dd0db6893a"). InnerVolumeSpecName "kube-api-access-xpfk7". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 17 18:00:01.517954 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:00:01.517890    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/753c9293-bc29-4523-aa9e-25dd0db6893a-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "753c9293-bc29-4523-aa9e-25dd0db6893a" (UID: "753c9293-bc29-4523-aa9e-25dd0db6893a"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 17 18:00:01.616346 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:00:01.616265    2575 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/753c9293-bc29-4523-aa9e-25dd0db6893a-dshm\") on node \"ip-10-0-143-74.ec2.internal\" DevicePath \"\""
Sep 17 18:00:01.616346 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:00:01.616291    2575 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/753c9293-bc29-4523-aa9e-25dd0db6893a-tls-certs\") on node \"ip-10-0-143-74.ec2.internal\" DevicePath \"\""
Sep 17 18:00:01.616346 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:00:01.616302    2575 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-xpfk7\" (UniqueName: \"kubernetes.io/projected/753c9293-bc29-4523-aa9e-25dd0db6893a-kube-api-access-xpfk7\") on node \"ip-10-0-143-74.ec2.internal\" DevicePath \"\""
Sep 17 18:00:02.058061 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:00:02.058035    2575 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-test-flow-control-smoke-1e98f46d/fc-concurrency-test-kserve-b5d44467c-bq7hv"
Sep 17 18:00:02.058061 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:00:02.058044    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-test-flow-control-smoke-1e98f46d/fc-concurrency-test-kserve-b5d44467c-bq7hv" event={"ID":"753c9293-bc29-4523-aa9e-25dd0db6893a","Type":"ContainerDied","Data":"b4105f5f071e8567d4f3b195b7f7c656613813c55a9fd083a5ec227e9c2303fc"}
Sep 17 18:00:02.058579 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:00:02.058101    2575 scope.go:117] "RemoveContainer" containerID="f7b32b3758051f37c65e7f23ca53692de6e1413cafae2e25600dc5632bd1fac0"
Sep 17 18:00:02.080739 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:00:02.080715    2575 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["e2e-test-flow-control-smoke-1e98f46d/fc-concurrency-test-kserve-b5d44467c-bq7hv"]
Sep 17 18:00:02.085356 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:00:02.085336    2575 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["e2e-test-flow-control-smoke-1e98f46d/fc-concurrency-test-kserve-b5d44467c-bq7hv"]
Sep 17 18:00:02.967250 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:00:02.967215    2575 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="753c9293-bc29-4523-aa9e-25dd0db6893a" path="/var/lib/kubelet/pods/753c9293-bc29-4523-aa9e-25dd0db6893a/volumes"
Sep 17 18:00:23.338351 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:00:23.338311    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-djvzt_1d07e68d-9943-420f-a6a0-5f477792fabd/konnectivity-agent/0.log" limit=1024
Sep 17 18:00:26.186137 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:00:26.186098    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-djvzt_1d07e68d-9943-420f-a6a0-5f477792fabd/konnectivity-agent/0.log" limit=1024
Sep 17 18:00:28.530949 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:00:28.530916    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-djvzt_1d07e68d-9943-420f-a6a0-5f477792fabd/konnectivity-agent/0.log" limit=1024
Sep 17 18:00:28.628913 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:00:28.628877    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-djvzt_1d07e68d-9943-420f-a6a0-5f477792fabd/konnectivity-agent/0.log" limit=1024
Sep 17 18:00:34.916129 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:00:34.916090    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-djvzt_1d07e68d-9943-420f-a6a0-5f477792fabd/konnectivity-agent/0.log" limit=1024
Sep 17 18:00:40.189575 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:00:40.189539    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-djvzt_1d07e68d-9943-420f-a6a0-5f477792fabd/konnectivity-agent/0.log" limit=1024
Sep 17 18:00:40.395985 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:00:40.395954    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-djvzt_1d07e68d-9943-420f-a6a0-5f477792fabd/konnectivity-agent/0.log" limit=1024
Sep 17 18:00:41.996691 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:00:41.996652    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-djvzt_1d07e68d-9943-420f-a6a0-5f477792fabd/konnectivity-agent/0.log" limit=1024
Sep 17 18:00:42.097539 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:00:42.097510    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-djvzt_1d07e68d-9943-420f-a6a0-5f477792fabd/konnectivity-agent/0.log" limit=1024
Sep 17 18:00:50.705022 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:00:50.704989    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-djvzt_1d07e68d-9943-420f-a6a0-5f477792fabd/konnectivity-agent/0.log" limit=1024
Sep 17 18:00:50.809478 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:00:50.809422    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-djvzt_1d07e68d-9943-420f-a6a0-5f477792fabd/konnectivity-agent/0.log" limit=1024
Sep 17 18:00:50.918151 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:00:50.918116    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-djvzt_1d07e68d-9943-420f-a6a0-5f477792fabd/konnectivity-agent/0.log" limit=1024
Sep 17 18:00:51.181530 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:00:51.181500    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-djvzt_1d07e68d-9943-420f-a6a0-5f477792fabd/konnectivity-agent/0.log" limit=1024
Sep 17 18:00:51.344072 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:00:51.344034    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-djvzt_1d07e68d-9943-420f-a6a0-5f477792fabd/konnectivity-agent/0.log" limit=1024
Sep 17 18:00:54.616279 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:00:54.616248    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-djvzt_1d07e68d-9943-420f-a6a0-5f477792fabd/konnectivity-agent/0.log" limit=1024
Sep 17 18:01:03.378642 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:01:03.378608    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-djvzt_1d07e68d-9943-420f-a6a0-5f477792fabd/konnectivity-agent/0.log" limit=1024
Sep 17 18:01:03.479533 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:01:03.479501    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-djvzt_1d07e68d-9943-420f-a6a0-5f477792fabd/konnectivity-agent/0.log" limit=1024
Sep 17 18:01:05.749552 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:01:05.749521    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-djvzt_1d07e68d-9943-420f-a6a0-5f477792fabd/konnectivity-agent/0.log" limit=1024
Sep 17 18:01:13.637048 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:01:13.637018    2575 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["e2e-test-gateway-section-name-propagation-49b5edd9/gw-section-name-router-with-gat-2f0a622e-kserve-8dc8c8494-bzbfg"]
Sep 17 18:01:13.637510 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:01:13.637395    2575 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="753c9293-bc29-4523-aa9e-25dd0db6893a" containerName="main"
Sep 17 18:01:13.637510 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:01:13.637407    2575 state_mem.go:107] "Deleted CPUSet assignment" podUID="753c9293-bc29-4523-aa9e-25dd0db6893a" containerName="main"
Sep 17 18:01:13.637510 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:01:13.637499    2575 memory_manager.go:368] "RemoveStaleState removing state" podUID="753c9293-bc29-4523-aa9e-25dd0db6893a" containerName="main"
Sep 17 18:01:13.640286 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:01:13.640262    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-test-gateway-section-name-propagation-49b5edd9/gw-section-name-router-with-gat-2f0a622e-kserve-8dc8c8494-bzbfg"
Sep 17 18:01:13.647705 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:01:13.647680    2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-test-gateway-section-name-propagation-49b5edd9\"/\"kube-root-ca.crt\""
Sep 17 18:01:13.648372 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:01:13.648350    2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-test-gateway-section-name-propagation-49b5edd9\"/\"openshift-service-ca.crt\""
Sep 17 18:01:13.648860 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:01:13.648841    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-test-gateway-section-name-propagation-49b5edd9\"/\"default-dockercfg-pv4jh\""
Sep 17 18:01:13.649314 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:01:13.649296    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-test-gateway-section-name-propagation-49b5edd9\"/\"gw-sec0c69dceeb48768325d1a53a749e65786-kserve-self-signed-certs\""
Sep 17 18:01:13.659732 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:01:13.659707    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["e2e-test-gateway-section-name-propagation-49b5edd9/gw-section-name-router-with-gat-2f0a622e-kserve-8dc8c8494-bzbfg"]
Sep 17 18:01:13.689673 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:01:13.689637    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-djvzt_1d07e68d-9943-420f-a6a0-5f477792fabd/konnectivity-agent/0.log" limit=1024
Sep 17 18:01:13.731789 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:01:13.731752    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2v8r2\" (UniqueName: \"kubernetes.io/projected/59567967-5a1d-472f-ba9a-d6156cda174f-kube-api-access-2v8r2\") pod \"gw-section-name-router-with-gat-2f0a622e-kserve-8dc8c8494-bzbfg\" (UID: \"59567967-5a1d-472f-ba9a-d6156cda174f\") " pod="e2e-test-gateway-section-name-propagation-49b5edd9/gw-section-name-router-with-gat-2f0a622e-kserve-8dc8c8494-bzbfg"
Sep 17 18:01:13.731949 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:01:13.731825    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/59567967-5a1d-472f-ba9a-d6156cda174f-tls-certs\") pod \"gw-section-name-router-with-gat-2f0a622e-kserve-8dc8c8494-bzbfg\" (UID: \"59567967-5a1d-472f-ba9a-d6156cda174f\") " pod="e2e-test-gateway-section-name-propagation-49b5edd9/gw-section-name-router-with-gat-2f0a622e-kserve-8dc8c8494-bzbfg"
Sep 17 18:01:13.731949 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:01:13.731869    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/59567967-5a1d-472f-ba9a-d6156cda174f-home\") pod \"gw-section-name-router-with-gat-2f0a622e-kserve-8dc8c8494-bzbfg\" (UID: \"59567967-5a1d-472f-ba9a-d6156cda174f\") " pod="e2e-test-gateway-section-name-propagation-49b5edd9/gw-section-name-router-with-gat-2f0a622e-kserve-8dc8c8494-bzbfg"
Sep 17 18:01:13.731949 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:01:13.731899    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/59567967-5a1d-472f-ba9a-d6156cda174f-model-cache\") pod \"gw-section-name-router-with-gat-2f0a622e-kserve-8dc8c8494-bzbfg\" (UID: \"59567967-5a1d-472f-ba9a-d6156cda174f\") " pod="e2e-test-gateway-section-name-propagation-49b5edd9/gw-section-name-router-with-gat-2f0a622e-kserve-8dc8c8494-bzbfg"
Sep 17 18:01:13.731949 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:01:13.731943    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/59567967-5a1d-472f-ba9a-d6156cda174f-tmp-dir\") pod \"gw-section-name-router-with-gat-2f0a622e-kserve-8dc8c8494-bzbfg\" (UID: \"59567967-5a1d-472f-ba9a-d6156cda174f\") " pod="e2e-test-gateway-section-name-propagation-49b5edd9/gw-section-name-router-with-gat-2f0a622e-kserve-8dc8c8494-bzbfg"
Sep 17 18:01:13.732129 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:01:13.731987    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/59567967-5a1d-472f-ba9a-d6156cda174f-dshm\") pod \"gw-section-name-router-with-gat-2f0a622e-kserve-8dc8c8494-bzbfg\" (UID: \"59567967-5a1d-472f-ba9a-d6156cda174f\") " pod="e2e-test-gateway-section-name-propagation-49b5edd9/gw-section-name-router-with-gat-2f0a622e-kserve-8dc8c8494-bzbfg"
Sep 17 18:01:13.832828 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:01:13.832790    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/59567967-5a1d-472f-ba9a-d6156cda174f-home\") pod \"gw-section-name-router-with-gat-2f0a622e-kserve-8dc8c8494-bzbfg\" (UID: \"59567967-5a1d-472f-ba9a-d6156cda174f\") " pod="e2e-test-gateway-section-name-propagation-49b5edd9/gw-section-name-router-with-gat-2f0a622e-kserve-8dc8c8494-bzbfg"
Sep 17 18:01:13.832828 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:01:13.832832    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/59567967-5a1d-472f-ba9a-d6156cda174f-model-cache\") pod \"gw-section-name-router-with-gat-2f0a622e-kserve-8dc8c8494-bzbfg\" (UID: \"59567967-5a1d-472f-ba9a-d6156cda174f\") " pod="e2e-test-gateway-section-name-propagation-49b5edd9/gw-section-name-router-with-gat-2f0a622e-kserve-8dc8c8494-bzbfg"
Sep 17 18:01:13.833060 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:01:13.832856    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/59567967-5a1d-472f-ba9a-d6156cda174f-tmp-dir\") pod \"gw-section-name-router-with-gat-2f0a622e-kserve-8dc8c8494-bzbfg\" (UID: \"59567967-5a1d-472f-ba9a-d6156cda174f\") " pod="e2e-test-gateway-section-name-propagation-49b5edd9/gw-section-name-router-with-gat-2f0a622e-kserve-8dc8c8494-bzbfg"
Sep 17 18:01:13.833060 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:01:13.832977    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/59567967-5a1d-472f-ba9a-d6156cda174f-dshm\") pod \"gw-section-name-router-with-gat-2f0a622e-kserve-8dc8c8494-bzbfg\" (UID: \"59567967-5a1d-472f-ba9a-d6156cda174f\") " pod="e2e-test-gateway-section-name-propagation-49b5edd9/gw-section-name-router-with-gat-2f0a622e-kserve-8dc8c8494-bzbfg"
Sep 17 18:01:13.833060 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:01:13.833032    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2v8r2\" (UniqueName: \"kubernetes.io/projected/59567967-5a1d-472f-ba9a-d6156cda174f-kube-api-access-2v8r2\") pod \"gw-section-name-router-with-gat-2f0a622e-kserve-8dc8c8494-bzbfg\" (UID: \"59567967-5a1d-472f-ba9a-d6156cda174f\") " pod="e2e-test-gateway-section-name-propagation-49b5edd9/gw-section-name-router-with-gat-2f0a622e-kserve-8dc8c8494-bzbfg"
Sep 17 18:01:13.833197 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:01:13.833113    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/59567967-5a1d-472f-ba9a-d6156cda174f-tls-certs\") pod \"gw-section-name-router-with-gat-2f0a622e-kserve-8dc8c8494-bzbfg\" (UID: \"59567967-5a1d-472f-ba9a-d6156cda174f\") " pod="e2e-test-gateway-section-name-propagation-49b5edd9/gw-section-name-router-with-gat-2f0a622e-kserve-8dc8c8494-bzbfg"
Sep 17 18:01:13.833273 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:01:13.833253    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/59567967-5a1d-472f-ba9a-d6156cda174f-home\") pod \"gw-section-name-router-with-gat-2f0a622e-kserve-8dc8c8494-bzbfg\" (UID: \"59567967-5a1d-472f-ba9a-d6156cda174f\") " pod="e2e-test-gateway-section-name-propagation-49b5edd9/gw-section-name-router-with-gat-2f0a622e-kserve-8dc8c8494-bzbfg"
Sep 17 18:01:13.833328 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:01:13.833236    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/59567967-5a1d-472f-ba9a-d6156cda174f-tmp-dir\") pod \"gw-section-name-router-with-gat-2f0a622e-kserve-8dc8c8494-bzbfg\" (UID: \"59567967-5a1d-472f-ba9a-d6156cda174f\") " pod="e2e-test-gateway-section-name-propagation-49b5edd9/gw-section-name-router-with-gat-2f0a622e-kserve-8dc8c8494-bzbfg"
Sep 17 18:01:13.833328 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:01:13.833306    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/59567967-5a1d-472f-ba9a-d6156cda174f-model-cache\") pod \"gw-section-name-router-with-gat-2f0a622e-kserve-8dc8c8494-bzbfg\" (UID: \"59567967-5a1d-472f-ba9a-d6156cda174f\") " pod="e2e-test-gateway-section-name-propagation-49b5edd9/gw-section-name-router-with-gat-2f0a622e-kserve-8dc8c8494-bzbfg"
Sep 17 18:01:13.835157 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:01:13.835137    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/59567967-5a1d-472f-ba9a-d6156cda174f-dshm\") pod \"gw-section-name-router-with-gat-2f0a622e-kserve-8dc8c8494-bzbfg\" (UID: \"59567967-5a1d-472f-ba9a-d6156cda174f\") " pod="e2e-test-gateway-section-name-propagation-49b5edd9/gw-section-name-router-with-gat-2f0a622e-kserve-8dc8c8494-bzbfg"
Sep 17 18:01:13.835610 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:01:13.835590    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/59567967-5a1d-472f-ba9a-d6156cda174f-tls-certs\") pod \"gw-section-name-router-with-gat-2f0a622e-kserve-8dc8c8494-bzbfg\" (UID: \"59567967-5a1d-472f-ba9a-d6156cda174f\") " pod="e2e-test-gateway-section-name-propagation-49b5edd9/gw-section-name-router-with-gat-2f0a622e-kserve-8dc8c8494-bzbfg"
Sep 17 18:01:13.842392 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:01:13.842363    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-2v8r2\" (UniqueName: \"kubernetes.io/projected/59567967-5a1d-472f-ba9a-d6156cda174f-kube-api-access-2v8r2\") pod \"gw-section-name-router-with-gat-2f0a622e-kserve-8dc8c8494-bzbfg\" (UID: \"59567967-5a1d-472f-ba9a-d6156cda174f\") " pod="e2e-test-gateway-section-name-propagation-49b5edd9/gw-section-name-router-with-gat-2f0a622e-kserve-8dc8c8494-bzbfg"
Sep 17 18:01:13.951230 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:01:13.951138    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-test-gateway-section-name-propagation-49b5edd9/gw-section-name-router-with-gat-2f0a622e-kserve-8dc8c8494-bzbfg"
Sep 17 18:01:14.229722 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:01:14.229693    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["e2e-test-gateway-section-name-propagation-49b5edd9/gw-section-name-router-with-gat-2f0a622e-kserve-8dc8c8494-bzbfg"]
Sep 17 18:01:14.231946 ip-10-0-143-74 kubenswrapper[2575]: W0917 18:01:14.231914    2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod59567967_5a1d_472f_ba9a_d6156cda174f.slice/crio-22e71d81be6dece59b64d9cc5963b3c551a2877ee1cf52fb16409050e5b1bca9 WatchSource:0}: Error finding container 22e71d81be6dece59b64d9cc5963b3c551a2877ee1cf52fb16409050e5b1bca9: Status 404 returned error can't find the container with id 22e71d81be6dece59b64d9cc5963b3c551a2877ee1cf52fb16409050e5b1bca9
Sep 17 18:01:14.445636 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:01:14.445600    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-test-gateway-section-name-propagation-49b5edd9/gw-section-name-router-with-gat-2f0a622e-kserve-8dc8c8494-bzbfg" event={"ID":"59567967-5a1d-472f-ba9a-d6156cda174f","Type":"ContainerStarted","Data":"0f5cefee55738c6d327afe81f8f7124181f84e1ae068f36d0e96aa90a2c80753"}
Sep 17 18:01:14.445790 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:01:14.445655    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-test-gateway-section-name-propagation-49b5edd9/gw-section-name-router-with-gat-2f0a622e-kserve-8dc8c8494-bzbfg" event={"ID":"59567967-5a1d-472f-ba9a-d6156cda174f","Type":"ContainerStarted","Data":"22e71d81be6dece59b64d9cc5963b3c551a2877ee1cf52fb16409050e5b1bca9"}
Sep 17 18:01:14.467029 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:01:14.466978    2575 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="e2e-test-gateway-section-name-propagation-49b5edd9/gw-section-name-router-with-gat-2f0a622e-kserve-8dc8c8494-bzbfg" podStartSLOduration=1.466962861 podStartE2EDuration="1.466962861s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-17 18:01:13 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-17 18:01:14.466560946 +0000 UTC m=+1594.096890921" watchObservedRunningTime="2026-09-17 18:01:14.466962861 +0000 UTC m=+1594.097292835"
Sep 17 18:01:14.952008 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:01:14.951969    2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-test-gateway-section-name-propagation-49b5edd9/gw-section-name-router-with-gat-2f0a622e-kserve-8dc8c8494-bzbfg"
Sep 17 18:01:16.034230 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:01:16.034194    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-djvzt_1d07e68d-9943-420f-a6a0-5f477792fabd/konnectivity-agent/0.log" limit=1024
Sep 17 18:01:16.134541 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:01:16.134506    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-djvzt_1d07e68d-9943-420f-a6a0-5f477792fabd/konnectivity-agent/0.log" limit=1024
Sep 17 18:01:16.239202 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:01:16.239171    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-djvzt_1d07e68d-9943-420f-a6a0-5f477792fabd/konnectivity-agent/0.log" limit=1024
Sep 17 18:01:16.352938 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:01:16.352898    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-djvzt_1d07e68d-9943-420f-a6a0-5f477792fabd/konnectivity-agent/0.log" limit=1024
Sep 17 18:01:16.452366 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:01:16.452331    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-djvzt_1d07e68d-9943-420f-a6a0-5f477792fabd/konnectivity-agent/0.log" limit=1024
Sep 17 18:01:19.528161 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:01:19.528129    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-djvzt_1d07e68d-9943-420f-a6a0-5f477792fabd/konnectivity-agent/0.log" limit=1024
Sep 17 18:01:20.104535 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:01:20.104497    2575 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["e2e-test-gateway-section-name-propagation-49b5edd9/gw-section-name-router-with-gat-2f0a622e-kserve-8dc8c8494-bzbfg"]
Sep 17 18:01:20.114253 ip-10-0-143-74 kubenswrapper[2575]: E0917 18:01:20.114212    2575 log.go:32] "ExecSync cmd from runtime service failed" err=<
Sep 17 18:01:20.114253 ip-10-0-143-74 kubenswrapper[2575]:         rpc error: code = Unknown desc = command error: executable file `/bin/sleep` not found: No such file or directory
Sep 17 18:01:20.114253 ip-10-0-143-74 kubenswrapper[2575]:         , stdout: , stderr: , exit code -1
Sep 17 18:01:20.114253 ip-10-0-143-74 kubenswrapper[2575]:  > containerID="0f5cefee55738c6d327afe81f8f7124181f84e1ae068f36d0e96aa90a2c80753" cmd=["/bin/sleep","15"]
Sep 17 18:01:20.114566 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:01:20.114269    2575 handlers.go:82] "Exec lifecycle hook for Container in Pod failed" execCommand=["/bin/sleep","15"] containerName="main" pod="e2e-test-gateway-section-name-propagation-49b5edd9/gw-section-name-router-with-gat-2f0a622e-kserve-8dc8c8494-bzbfg" message="" err=<
Sep 17 18:01:20.114566 ip-10-0-143-74 kubenswrapper[2575]:         rpc error: code = Unknown desc = command error: executable file `/bin/sleep` not found: No such file or directory
Sep 17 18:01:20.114566 ip-10-0-143-74 kubenswrapper[2575]:         , stdout: , stderr: , exit code -1
Sep 17 18:01:20.114566 ip-10-0-143-74 kubenswrapper[2575]:  >
Sep 17 18:01:20.114566 ip-10-0-143-74 kubenswrapper[2575]: E0917 18:01:20.114282    2575 kuberuntime_container.go:755] "PreStop hook failed" err=<
Sep 17 18:01:20.114566 ip-10-0-143-74 kubenswrapper[2575]:         rpc error: code = Unknown desc = command error: executable file `/bin/sleep` not found: No such file or directory
Sep 17 18:01:20.114566 ip-10-0-143-74 kubenswrapper[2575]:         , stdout: , stderr: , exit code -1
Sep 17 18:01:20.114566 ip-10-0-143-74 kubenswrapper[2575]:  > pod="e2e-test-gateway-section-name-propagation-49b5edd9/gw-section-name-router-with-gat-2f0a622e-kserve-8dc8c8494-bzbfg" podUID="59567967-5a1d-472f-ba9a-d6156cda174f" containerName="main" containerID="cri-o://0f5cefee55738c6d327afe81f8f7124181f84e1ae068f36d0e96aa90a2c80753"
Sep 17 18:01:20.114566 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:01:20.114333    2575 kuberuntime_container.go:873] "Killing container with a grace period" pod="e2e-test-gateway-section-name-propagation-49b5edd9/gw-section-name-router-with-gat-2f0a622e-kserve-8dc8c8494-bzbfg" podUID="59567967-5a1d-472f-ba9a-d6156cda174f" containerName="main" containerID="cri-o://0f5cefee55738c6d327afe81f8f7124181f84e1ae068f36d0e96aa90a2c80753" gracePeriod=60
Sep 17 18:01:20.142118 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:01:20.142088    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-djvzt_1d07e68d-9943-420f-a6a0-5f477792fabd/konnectivity-agent/0.log" limit=1024
Sep 17 18:01:20.373918 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:01:20.373892    2575 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-test-gateway-section-name-propagation-49b5edd9/gw-section-name-router-with-gat-2f0a622e-kserve-8dc8c8494-bzbfg"
Sep 17 18:01:20.478744 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:01:20.478706    2575 generic.go:356] "Generic (PLEG): container finished" podID="59567967-5a1d-472f-ba9a-d6156cda174f" containerID="0f5cefee55738c6d327afe81f8f7124181f84e1ae068f36d0e96aa90a2c80753" exitCode=0
Sep 17 18:01:20.478914 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:01:20.478757    2575 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-test-gateway-section-name-propagation-49b5edd9/gw-section-name-router-with-gat-2f0a622e-kserve-8dc8c8494-bzbfg"
Sep 17 18:01:20.478914 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:01:20.478787    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-test-gateway-section-name-propagation-49b5edd9/gw-section-name-router-with-gat-2f0a622e-kserve-8dc8c8494-bzbfg" event={"ID":"59567967-5a1d-472f-ba9a-d6156cda174f","Type":"ContainerDied","Data":"0f5cefee55738c6d327afe81f8f7124181f84e1ae068f36d0e96aa90a2c80753"}
Sep 17 18:01:20.478914 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:01:20.478840    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-test-gateway-section-name-propagation-49b5edd9/gw-section-name-router-with-gat-2f0a622e-kserve-8dc8c8494-bzbfg" event={"ID":"59567967-5a1d-472f-ba9a-d6156cda174f","Type":"ContainerDied","Data":"22e71d81be6dece59b64d9cc5963b3c551a2877ee1cf52fb16409050e5b1bca9"}
Sep 17 18:01:20.478914 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:01:20.478869    2575 scope.go:117] "RemoveContainer" containerID="0f5cefee55738c6d327afe81f8f7124181f84e1ae068f36d0e96aa90a2c80753"
Sep 17 18:01:20.488130 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:01:20.488104    2575 scope.go:117] "RemoveContainer" containerID="0f5cefee55738c6d327afe81f8f7124181f84e1ae068f36d0e96aa90a2c80753"
Sep 17 18:01:20.488391 ip-10-0-143-74 kubenswrapper[2575]: E0917 18:01:20.488374    2575 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"0f5cefee55738c6d327afe81f8f7124181f84e1ae068f36d0e96aa90a2c80753\": container with ID starting with 0f5cefee55738c6d327afe81f8f7124181f84e1ae068f36d0e96aa90a2c80753 not found: ID does not exist" containerID="0f5cefee55738c6d327afe81f8f7124181f84e1ae068f36d0e96aa90a2c80753"
Sep 17 18:01:20.488436 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:01:20.488399    2575 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"0f5cefee55738c6d327afe81f8f7124181f84e1ae068f36d0e96aa90a2c80753"} err="failed to get container status \"0f5cefee55738c6d327afe81f8f7124181f84e1ae068f36d0e96aa90a2c80753\": rpc error: code = NotFound desc = could not find container \"0f5cefee55738c6d327afe81f8f7124181f84e1ae068f36d0e96aa90a2c80753\": container with ID starting with 0f5cefee55738c6d327afe81f8f7124181f84e1ae068f36d0e96aa90a2c80753 not found: ID does not exist"
Sep 17 18:01:20.501142 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:01:20.501115    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/59567967-5a1d-472f-ba9a-d6156cda174f-tmp-dir\") pod \"59567967-5a1d-472f-ba9a-d6156cda174f\" (UID: \"59567967-5a1d-472f-ba9a-d6156cda174f\") "
Sep 17 18:01:20.501255 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:01:20.501213    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-2v8r2\" (UniqueName: \"kubernetes.io/projected/59567967-5a1d-472f-ba9a-d6156cda174f-kube-api-access-2v8r2\") pod \"59567967-5a1d-472f-ba9a-d6156cda174f\" (UID: \"59567967-5a1d-472f-ba9a-d6156cda174f\") "
Sep 17 18:01:20.501255 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:01:20.501229    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/59567967-5a1d-472f-ba9a-d6156cda174f-tls-certs\") pod \"59567967-5a1d-472f-ba9a-d6156cda174f\" (UID: \"59567967-5a1d-472f-ba9a-d6156cda174f\") "
Sep 17 18:01:20.501255 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:01:20.501250    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/59567967-5a1d-472f-ba9a-d6156cda174f-model-cache\") pod \"59567967-5a1d-472f-ba9a-d6156cda174f\" (UID: \"59567967-5a1d-472f-ba9a-d6156cda174f\") "
Sep 17 18:01:20.501409 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:01:20.501292    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/59567967-5a1d-472f-ba9a-d6156cda174f-dshm\") pod \"59567967-5a1d-472f-ba9a-d6156cda174f\" (UID: \"59567967-5a1d-472f-ba9a-d6156cda174f\") "
Sep 17 18:01:20.501409 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:01:20.501342    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/59567967-5a1d-472f-ba9a-d6156cda174f-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "59567967-5a1d-472f-ba9a-d6156cda174f" (UID: "59567967-5a1d-472f-ba9a-d6156cda174f"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 17 18:01:20.501409 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:01:20.501401    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/59567967-5a1d-472f-ba9a-d6156cda174f-home\") pod \"59567967-5a1d-472f-ba9a-d6156cda174f\" (UID: \"59567967-5a1d-472f-ba9a-d6156cda174f\") "
Sep 17 18:01:20.501602 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:01:20.501572    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/59567967-5a1d-472f-ba9a-d6156cda174f-model-cache" (OuterVolumeSpecName: "model-cache") pod "59567967-5a1d-472f-ba9a-d6156cda174f" (UID: "59567967-5a1d-472f-ba9a-d6156cda174f"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 17 18:01:20.501670 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:01:20.501641    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/59567967-5a1d-472f-ba9a-d6156cda174f-home" (OuterVolumeSpecName: "home") pod "59567967-5a1d-472f-ba9a-d6156cda174f" (UID: "59567967-5a1d-472f-ba9a-d6156cda174f"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 17 18:01:20.501736 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:01:20.501730    2575 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/59567967-5a1d-472f-ba9a-d6156cda174f-model-cache\") on node \"ip-10-0-143-74.ec2.internal\" DevicePath \"\""
Sep 17 18:01:20.501789 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:01:20.501740    2575 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/59567967-5a1d-472f-ba9a-d6156cda174f-home\") on node \"ip-10-0-143-74.ec2.internal\" DevicePath \"\""
Sep 17 18:01:20.501789 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:01:20.501752    2575 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/59567967-5a1d-472f-ba9a-d6156cda174f-tmp-dir\") on node \"ip-10-0-143-74.ec2.internal\" DevicePath \"\""
Sep 17 18:01:20.503515 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:01:20.503481    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/59567967-5a1d-472f-ba9a-d6156cda174f-kube-api-access-2v8r2" (OuterVolumeSpecName: "kube-api-access-2v8r2") pod "59567967-5a1d-472f-ba9a-d6156cda174f" (UID: "59567967-5a1d-472f-ba9a-d6156cda174f"). InnerVolumeSpecName "kube-api-access-2v8r2". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 17 18:01:20.503515 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:01:20.503504    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/59567967-5a1d-472f-ba9a-d6156cda174f-dshm" (OuterVolumeSpecName: "dshm") pod "59567967-5a1d-472f-ba9a-d6156cda174f" (UID: "59567967-5a1d-472f-ba9a-d6156cda174f"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 17 18:01:20.503678 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:01:20.503655    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/59567967-5a1d-472f-ba9a-d6156cda174f-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "59567967-5a1d-472f-ba9a-d6156cda174f" (UID: "59567967-5a1d-472f-ba9a-d6156cda174f"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 17 18:01:20.602744 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:01:20.602712    2575 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/59567967-5a1d-472f-ba9a-d6156cda174f-dshm\") on node \"ip-10-0-143-74.ec2.internal\" DevicePath \"\""
Sep 17 18:01:20.602744 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:01:20.602740    2575 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-2v8r2\" (UniqueName: \"kubernetes.io/projected/59567967-5a1d-472f-ba9a-d6156cda174f-kube-api-access-2v8r2\") on node \"ip-10-0-143-74.ec2.internal\" DevicePath \"\""
Sep 17 18:01:20.603161 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:01:20.602753    2575 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/59567967-5a1d-472f-ba9a-d6156cda174f-tls-certs\") on node \"ip-10-0-143-74.ec2.internal\" DevicePath \"\""
Sep 17 18:01:20.801957 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:01:20.801922    2575 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["e2e-test-gateway-section-name-propagation-49b5edd9/gw-section-name-router-with-gat-2f0a622e-kserve-8dc8c8494-bzbfg"]
Sep 17 18:01:20.807960 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:01:20.807929    2575 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["e2e-test-gateway-section-name-propagation-49b5edd9/gw-section-name-router-with-gat-2f0a622e-kserve-8dc8c8494-bzbfg"]
Sep 17 18:01:20.969860 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:01:20.969823    2575 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="59567967-5a1d-472f-ba9a-d6156cda174f" path="/var/lib/kubelet/pods/59567967-5a1d-472f-ba9a-d6156cda174f/volumes"
Sep 17 18:01:21.104534 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:01:21.104430    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-djvzt_1d07e68d-9943-420f-a6a0-5f477792fabd/konnectivity-agent/0.log" limit=1024
Sep 17 18:01:21.350276 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:01:21.350240    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-djvzt_1d07e68d-9943-420f-a6a0-5f477792fabd/konnectivity-agent/0.log" limit=1024
Sep 17 18:01:26.402655 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:01:26.402622    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-djvzt_1d07e68d-9943-420f-a6a0-5f477792fabd/konnectivity-agent/0.log" limit=1024
Sep 17 18:01:27.216720 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:01:27.216684    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-djvzt_1d07e68d-9943-420f-a6a0-5f477792fabd/konnectivity-agent/0.log" limit=1024
Sep 17 18:01:32.704606 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:01:32.704568    2575 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["e2e-test-gateway-section-name-propagation-94799d44/gw-section-name-router-with-gat-f1d92d0f-kserve-5f5b776bbfjngts"]
Sep 17 18:01:32.704998 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:01:32.704986    2575 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="59567967-5a1d-472f-ba9a-d6156cda174f" containerName="main"
Sep 17 18:01:32.705049 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:01:32.705000    2575 state_mem.go:107] "Deleted CPUSet assignment" podUID="59567967-5a1d-472f-ba9a-d6156cda174f" containerName="main"
Sep 17 18:01:32.705086 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:01:32.705064    2575 memory_manager.go:368] "RemoveStaleState removing state" podUID="59567967-5a1d-472f-ba9a-d6156cda174f" containerName="main"
Sep 17 18:01:32.709520 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:01:32.709498    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-test-gateway-section-name-propagation-94799d44/gw-section-name-router-with-gat-f1d92d0f-kserve-5f5b776bbfjngts"
Sep 17 18:01:32.713216 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:01:32.713178    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-test-gateway-section-name-propagation-94799d44\"/\"default-dockercfg-kgdvx\""
Sep 17 18:01:32.713411 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:01:32.713185    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-test-gateway-section-name-propagation-94799d44\"/\"gw-sec2774c263d49959f50d9eebc552e13bf9-kserve-self-signed-certs\""
Sep 17 18:01:32.713411 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:01:32.713344    2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-test-gateway-section-name-propagation-94799d44\"/\"kube-root-ca.crt\""
Sep 17 18:01:32.713757 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:01:32.713734    2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-test-gateway-section-name-propagation-94799d44\"/\"openshift-service-ca.crt\""
Sep 17 18:01:32.714428 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:01:32.714401    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/411a782c-9423-495d-8972-6824fdae7f86-dshm\") pod \"gw-section-name-router-with-gat-f1d92d0f-kserve-5f5b776bbfjngts\" (UID: \"411a782c-9423-495d-8972-6824fdae7f86\") " pod="e2e-test-gateway-section-name-propagation-94799d44/gw-section-name-router-with-gat-f1d92d0f-kserve-5f5b776bbfjngts"
Sep 17 18:01:32.714566 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:01:32.714503    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/411a782c-9423-495d-8972-6824fdae7f86-tls-certs\") pod \"gw-section-name-router-with-gat-f1d92d0f-kserve-5f5b776bbfjngts\" (UID: \"411a782c-9423-495d-8972-6824fdae7f86\") " pod="e2e-test-gateway-section-name-propagation-94799d44/gw-section-name-router-with-gat-f1d92d0f-kserve-5f5b776bbfjngts"
Sep 17 18:01:32.714566 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:01:32.714549    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ts46c\" (UniqueName: \"kubernetes.io/projected/411a782c-9423-495d-8972-6824fdae7f86-kube-api-access-ts46c\") pod \"gw-section-name-router-with-gat-f1d92d0f-kserve-5f5b776bbfjngts\" (UID: \"411a782c-9423-495d-8972-6824fdae7f86\") " pod="e2e-test-gateway-section-name-propagation-94799d44/gw-section-name-router-with-gat-f1d92d0f-kserve-5f5b776bbfjngts"
Sep 17 18:01:32.714724 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:01:32.714641    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/411a782c-9423-495d-8972-6824fdae7f86-tmp-dir\") pod \"gw-section-name-router-with-gat-f1d92d0f-kserve-5f5b776bbfjngts\" (UID: \"411a782c-9423-495d-8972-6824fdae7f86\") " pod="e2e-test-gateway-section-name-propagation-94799d44/gw-section-name-router-with-gat-f1d92d0f-kserve-5f5b776bbfjngts"
Sep 17 18:01:32.714724 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:01:32.714694    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/411a782c-9423-495d-8972-6824fdae7f86-home\") pod \"gw-section-name-router-with-gat-f1d92d0f-kserve-5f5b776bbfjngts\" (UID: \"411a782c-9423-495d-8972-6824fdae7f86\") " pod="e2e-test-gateway-section-name-propagation-94799d44/gw-section-name-router-with-gat-f1d92d0f-kserve-5f5b776bbfjngts"
Sep 17 18:01:32.714808 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:01:32.714728    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/411a782c-9423-495d-8972-6824fdae7f86-model-cache\") pod \"gw-section-name-router-with-gat-f1d92d0f-kserve-5f5b776bbfjngts\" (UID: \"411a782c-9423-495d-8972-6824fdae7f86\") " pod="e2e-test-gateway-section-name-propagation-94799d44/gw-section-name-router-with-gat-f1d92d0f-kserve-5f5b776bbfjngts"
Sep 17 18:01:32.717497 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:01:32.717031    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["e2e-test-gateway-section-name-propagation-94799d44/gw-section-name-router-with-gat-f1d92d0f-kserve-5f5b776bbfjngts"]
Sep 17 18:01:32.733359 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:01:32.733315    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-djvzt_1d07e68d-9943-420f-a6a0-5f477792fabd/konnectivity-agent/0.log" limit=1024
Sep 17 18:01:32.815141 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:01:32.815098    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/411a782c-9423-495d-8972-6824fdae7f86-tmp-dir\") pod \"gw-section-name-router-with-gat-f1d92d0f-kserve-5f5b776bbfjngts\" (UID: \"411a782c-9423-495d-8972-6824fdae7f86\") " pod="e2e-test-gateway-section-name-propagation-94799d44/gw-section-name-router-with-gat-f1d92d0f-kserve-5f5b776bbfjngts"
Sep 17 18:01:32.815286 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:01:32.815155    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/411a782c-9423-495d-8972-6824fdae7f86-home\") pod \"gw-section-name-router-with-gat-f1d92d0f-kserve-5f5b776bbfjngts\" (UID: \"411a782c-9423-495d-8972-6824fdae7f86\") " pod="e2e-test-gateway-section-name-propagation-94799d44/gw-section-name-router-with-gat-f1d92d0f-kserve-5f5b776bbfjngts"
Sep 17 18:01:32.815286 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:01:32.815181    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/411a782c-9423-495d-8972-6824fdae7f86-model-cache\") pod \"gw-section-name-router-with-gat-f1d92d0f-kserve-5f5b776bbfjngts\" (UID: \"411a782c-9423-495d-8972-6824fdae7f86\") " pod="e2e-test-gateway-section-name-propagation-94799d44/gw-section-name-router-with-gat-f1d92d0f-kserve-5f5b776bbfjngts"
Sep 17 18:01:32.815286 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:01:32.815244    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/411a782c-9423-495d-8972-6824fdae7f86-dshm\") pod \"gw-section-name-router-with-gat-f1d92d0f-kserve-5f5b776bbfjngts\" (UID: \"411a782c-9423-495d-8972-6824fdae7f86\") " pod="e2e-test-gateway-section-name-propagation-94799d44/gw-section-name-router-with-gat-f1d92d0f-kserve-5f5b776bbfjngts"
Sep 17 18:01:32.815286 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:01:32.815273    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/411a782c-9423-495d-8972-6824fdae7f86-tls-certs\") pod \"gw-section-name-router-with-gat-f1d92d0f-kserve-5f5b776bbfjngts\" (UID: \"411a782c-9423-495d-8972-6824fdae7f86\") " pod="e2e-test-gateway-section-name-propagation-94799d44/gw-section-name-router-with-gat-f1d92d0f-kserve-5f5b776bbfjngts"
Sep 17 18:01:32.815494 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:01:32.815305    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ts46c\" (UniqueName: \"kubernetes.io/projected/411a782c-9423-495d-8972-6824fdae7f86-kube-api-access-ts46c\") pod \"gw-section-name-router-with-gat-f1d92d0f-kserve-5f5b776bbfjngts\" (UID: \"411a782c-9423-495d-8972-6824fdae7f86\") " pod="e2e-test-gateway-section-name-propagation-94799d44/gw-section-name-router-with-gat-f1d92d0f-kserve-5f5b776bbfjngts"
Sep 17 18:01:32.815605 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:01:32.815585    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/411a782c-9423-495d-8972-6824fdae7f86-tmp-dir\") pod \"gw-section-name-router-with-gat-f1d92d0f-kserve-5f5b776bbfjngts\" (UID: \"411a782c-9423-495d-8972-6824fdae7f86\") " pod="e2e-test-gateway-section-name-propagation-94799d44/gw-section-name-router-with-gat-f1d92d0f-kserve-5f5b776bbfjngts"
Sep 17 18:01:32.815687 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:01:32.815665    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/411a782c-9423-495d-8972-6824fdae7f86-model-cache\") pod \"gw-section-name-router-with-gat-f1d92d0f-kserve-5f5b776bbfjngts\" (UID: \"411a782c-9423-495d-8972-6824fdae7f86\") " pod="e2e-test-gateway-section-name-propagation-94799d44/gw-section-name-router-with-gat-f1d92d0f-kserve-5f5b776bbfjngts"
Sep 17 18:01:32.815734 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:01:32.815690    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/411a782c-9423-495d-8972-6824fdae7f86-home\") pod \"gw-section-name-router-with-gat-f1d92d0f-kserve-5f5b776bbfjngts\" (UID: \"411a782c-9423-495d-8972-6824fdae7f86\") " pod="e2e-test-gateway-section-name-propagation-94799d44/gw-section-name-router-with-gat-f1d92d0f-kserve-5f5b776bbfjngts"
Sep 17 18:01:32.817719 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:01:32.817694    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/411a782c-9423-495d-8972-6824fdae7f86-dshm\") pod \"gw-section-name-router-with-gat-f1d92d0f-kserve-5f5b776bbfjngts\" (UID: \"411a782c-9423-495d-8972-6824fdae7f86\") " pod="e2e-test-gateway-section-name-propagation-94799d44/gw-section-name-router-with-gat-f1d92d0f-kserve-5f5b776bbfjngts"
Sep 17 18:01:32.817873 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:01:32.817797    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/411a782c-9423-495d-8972-6824fdae7f86-tls-certs\") pod \"gw-section-name-router-with-gat-f1d92d0f-kserve-5f5b776bbfjngts\" (UID: \"411a782c-9423-495d-8972-6824fdae7f86\") " pod="e2e-test-gateway-section-name-propagation-94799d44/gw-section-name-router-with-gat-f1d92d0f-kserve-5f5b776bbfjngts"
Sep 17 18:01:32.823474 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:01:32.823432    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-ts46c\" (UniqueName: \"kubernetes.io/projected/411a782c-9423-495d-8972-6824fdae7f86-kube-api-access-ts46c\") pod \"gw-section-name-router-with-gat-f1d92d0f-kserve-5f5b776bbfjngts\" (UID: \"411a782c-9423-495d-8972-6824fdae7f86\") " pod="e2e-test-gateway-section-name-propagation-94799d44/gw-section-name-router-with-gat-f1d92d0f-kserve-5f5b776bbfjngts"
Sep 17 18:01:33.021158 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:01:33.021074    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-test-gateway-section-name-propagation-94799d44/gw-section-name-router-with-gat-f1d92d0f-kserve-5f5b776bbfjngts"
Sep 17 18:01:33.296720 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:01:33.296692    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["e2e-test-gateway-section-name-propagation-94799d44/gw-section-name-router-with-gat-f1d92d0f-kserve-5f5b776bbfjngts"]
Sep 17 18:01:33.298424 ip-10-0-143-74 kubenswrapper[2575]: W0917 18:01:33.298389    2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod411a782c_9423_495d_8972_6824fdae7f86.slice/crio-0fc563a19f67dc197a6405566b47fd15652d7c074613fad0f36b0cb350d1f4ba WatchSource:0}: Error finding container 0fc563a19f67dc197a6405566b47fd15652d7c074613fad0f36b0cb350d1f4ba: Status 404 returned error can't find the container with id 0fc563a19f67dc197a6405566b47fd15652d7c074613fad0f36b0cb350d1f4ba
Sep 17 18:01:33.553265 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:01:33.553167    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-test-gateway-section-name-propagation-94799d44/gw-section-name-router-with-gat-f1d92d0f-kserve-5f5b776bbfjngts" event={"ID":"411a782c-9423-495d-8972-6824fdae7f86","Type":"ContainerStarted","Data":"9619db8a3e529e505200dd89ce6dd3001cd9b9fee7898fb1d358a51051dc1823"}
Sep 17 18:01:33.553265 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:01:33.553232    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-test-gateway-section-name-propagation-94799d44/gw-section-name-router-with-gat-f1d92d0f-kserve-5f5b776bbfjngts" event={"ID":"411a782c-9423-495d-8972-6824fdae7f86","Type":"ContainerStarted","Data":"0fc563a19f67dc197a6405566b47fd15652d7c074613fad0f36b0cb350d1f4ba"}
Sep 17 18:01:33.574869 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:01:33.574816    2575 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="e2e-test-gateway-section-name-propagation-94799d44/gw-section-name-router-with-gat-f1d92d0f-kserve-5f5b776bbfjngts" podStartSLOduration=1.5747986520000001 podStartE2EDuration="1.574798652s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-17 18:01:32 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-17 18:01:33.573671062 +0000 UTC m=+1613.204001038" watchObservedRunningTime="2026-09-17 18:01:33.574798652 +0000 UTC m=+1613.205128628"
Sep 17 18:01:34.022102 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:01:34.022067    2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-test-gateway-section-name-propagation-94799d44/gw-section-name-router-with-gat-f1d92d0f-kserve-5f5b776bbfjngts"
Sep 17 18:01:34.876211 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:01:34.876178    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-djvzt_1d07e68d-9943-420f-a6a0-5f477792fabd/konnectivity-agent/0.log" limit=1024
Sep 17 18:01:35.534044 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:01:35.534011    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-djvzt_1d07e68d-9943-420f-a6a0-5f477792fabd/konnectivity-agent/0.log" limit=1024
Sep 17 18:01:35.634472 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:01:35.634408    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-djvzt_1d07e68d-9943-420f-a6a0-5f477792fabd/konnectivity-agent/0.log" limit=1024
Sep 17 18:01:36.987860 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:01:36.987814    2575 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["e2e-test-gateway-section-name-propagation-94799d44/gw-section-name-router-with-gat-f1d92d0f-kserve-5f5b776bbfjngts"]
Sep 17 18:01:36.997612 ip-10-0-143-74 kubenswrapper[2575]: E0917 18:01:36.997567    2575 log.go:32] "ExecSync cmd from runtime service failed" err=<
Sep 17 18:01:36.997612 ip-10-0-143-74 kubenswrapper[2575]:         rpc error: code = Unknown desc = command error: executable file `/bin/sleep` not found: No such file or directory
Sep 17 18:01:36.997612 ip-10-0-143-74 kubenswrapper[2575]:         , stdout: , stderr: , exit code -1
Sep 17 18:01:36.997612 ip-10-0-143-74 kubenswrapper[2575]:  > containerID="9619db8a3e529e505200dd89ce6dd3001cd9b9fee7898fb1d358a51051dc1823" cmd=["/bin/sleep","15"]
Sep 17 18:01:36.997902 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:01:36.997619    2575 handlers.go:82] "Exec lifecycle hook for Container in Pod failed" execCommand=["/bin/sleep","15"] containerName="main" pod="e2e-test-gateway-section-name-propagation-94799d44/gw-section-name-router-with-gat-f1d92d0f-kserve-5f5b776bbfjngts" message="" err=<
Sep 17 18:01:36.997902 ip-10-0-143-74 kubenswrapper[2575]:         rpc error: code = Unknown desc = command error: executable file `/bin/sleep` not found: No such file or directory
Sep 17 18:01:36.997902 ip-10-0-143-74 kubenswrapper[2575]:         , stdout: , stderr: , exit code -1
Sep 17 18:01:36.997902 ip-10-0-143-74 kubenswrapper[2575]:  >
Sep 17 18:01:36.997902 ip-10-0-143-74 kubenswrapper[2575]: E0917 18:01:36.997632    2575 kuberuntime_container.go:755] "PreStop hook failed" err=<
Sep 17 18:01:36.997902 ip-10-0-143-74 kubenswrapper[2575]:         rpc error: code = Unknown desc = command error: executable file `/bin/sleep` not found: No such file or directory
Sep 17 18:01:36.997902 ip-10-0-143-74 kubenswrapper[2575]:         , stdout: , stderr: , exit code -1
Sep 17 18:01:36.997902 ip-10-0-143-74 kubenswrapper[2575]:  > pod="e2e-test-gateway-section-name-propagation-94799d44/gw-section-name-router-with-gat-f1d92d0f-kserve-5f5b776bbfjngts" podUID="411a782c-9423-495d-8972-6824fdae7f86" containerName="main" containerID="cri-o://9619db8a3e529e505200dd89ce6dd3001cd9b9fee7898fb1d358a51051dc1823"
Sep 17 18:01:36.997902 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:01:36.997668    2575 kuberuntime_container.go:873] "Killing container with a grace period" pod="e2e-test-gateway-section-name-propagation-94799d44/gw-section-name-router-with-gat-f1d92d0f-kserve-5f5b776bbfjngts" podUID="411a782c-9423-495d-8972-6824fdae7f86" containerName="main" containerID="cri-o://9619db8a3e529e505200dd89ce6dd3001cd9b9fee7898fb1d358a51051dc1823" gracePeriod=60
Sep 17 18:01:37.000982 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:01:37.000959    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-djvzt_1d07e68d-9943-420f-a6a0-5f477792fabd/konnectivity-agent/0.log" limit=1024
Sep 17 18:01:37.244134 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:01:37.244063    2575 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-test-gateway-section-name-propagation-94799d44/gw-section-name-router-with-gat-f1d92d0f-kserve-5f5b776bbfjngts"
Sep 17 18:01:37.348467 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:01:37.348416    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/411a782c-9423-495d-8972-6824fdae7f86-tmp-dir\") pod \"411a782c-9423-495d-8972-6824fdae7f86\" (UID: \"411a782c-9423-495d-8972-6824fdae7f86\") "
Sep 17 18:01:37.348467 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:01:37.348471    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/411a782c-9423-495d-8972-6824fdae7f86-home\") pod \"411a782c-9423-495d-8972-6824fdae7f86\" (UID: \"411a782c-9423-495d-8972-6824fdae7f86\") "
Sep 17 18:01:37.348724 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:01:37.348499    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-ts46c\" (UniqueName: \"kubernetes.io/projected/411a782c-9423-495d-8972-6824fdae7f86-kube-api-access-ts46c\") pod \"411a782c-9423-495d-8972-6824fdae7f86\" (UID: \"411a782c-9423-495d-8972-6824fdae7f86\") "
Sep 17 18:01:37.348724 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:01:37.348559    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/411a782c-9423-495d-8972-6824fdae7f86-model-cache\") pod \"411a782c-9423-495d-8972-6824fdae7f86\" (UID: \"411a782c-9423-495d-8972-6824fdae7f86\") "
Sep 17 18:01:37.348724 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:01:37.348594    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/411a782c-9423-495d-8972-6824fdae7f86-dshm\") pod \"411a782c-9423-495d-8972-6824fdae7f86\" (UID: \"411a782c-9423-495d-8972-6824fdae7f86\") "
Sep 17 18:01:37.348724 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:01:37.348614    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/411a782c-9423-495d-8972-6824fdae7f86-tls-certs\") pod \"411a782c-9423-495d-8972-6824fdae7f86\" (UID: \"411a782c-9423-495d-8972-6824fdae7f86\") "
Sep 17 18:01:37.348920 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:01:37.348762    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/411a782c-9423-495d-8972-6824fdae7f86-home" (OuterVolumeSpecName: "home") pod "411a782c-9423-495d-8972-6824fdae7f86" (UID: "411a782c-9423-495d-8972-6824fdae7f86"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 17 18:01:37.348920 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:01:37.348777    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/411a782c-9423-495d-8972-6824fdae7f86-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "411a782c-9423-495d-8972-6824fdae7f86" (UID: "411a782c-9423-495d-8972-6824fdae7f86"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 17 18:01:37.348920 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:01:37.348845    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/411a782c-9423-495d-8972-6824fdae7f86-model-cache" (OuterVolumeSpecName: "model-cache") pod "411a782c-9423-495d-8972-6824fdae7f86" (UID: "411a782c-9423-495d-8972-6824fdae7f86"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 17 18:01:37.350825 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:01:37.350804    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/411a782c-9423-495d-8972-6824fdae7f86-dshm" (OuterVolumeSpecName: "dshm") pod "411a782c-9423-495d-8972-6824fdae7f86" (UID: "411a782c-9423-495d-8972-6824fdae7f86"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 17 18:01:37.350940 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:01:37.350920    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/411a782c-9423-495d-8972-6824fdae7f86-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "411a782c-9423-495d-8972-6824fdae7f86" (UID: "411a782c-9423-495d-8972-6824fdae7f86"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 17 18:01:37.350990 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:01:37.350960    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/411a782c-9423-495d-8972-6824fdae7f86-kube-api-access-ts46c" (OuterVolumeSpecName: "kube-api-access-ts46c") pod "411a782c-9423-495d-8972-6824fdae7f86" (UID: "411a782c-9423-495d-8972-6824fdae7f86"). InnerVolumeSpecName "kube-api-access-ts46c". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 17 18:01:37.449289 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:01:37.449256    2575 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/411a782c-9423-495d-8972-6824fdae7f86-dshm\") on node \"ip-10-0-143-74.ec2.internal\" DevicePath \"\""
Sep 17 18:01:37.449289 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:01:37.449285    2575 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/411a782c-9423-495d-8972-6824fdae7f86-tls-certs\") on node \"ip-10-0-143-74.ec2.internal\" DevicePath \"\""
Sep 17 18:01:37.449289 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:01:37.449295    2575 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/411a782c-9423-495d-8972-6824fdae7f86-tmp-dir\") on node \"ip-10-0-143-74.ec2.internal\" DevicePath \"\""
Sep 17 18:01:37.449289 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:01:37.449304    2575 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/411a782c-9423-495d-8972-6824fdae7f86-home\") on node \"ip-10-0-143-74.ec2.internal\" DevicePath \"\""
Sep 17 18:01:37.449611 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:01:37.449313    2575 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-ts46c\" (UniqueName: \"kubernetes.io/projected/411a782c-9423-495d-8972-6824fdae7f86-kube-api-access-ts46c\") on node \"ip-10-0-143-74.ec2.internal\" DevicePath \"\""
Sep 17 18:01:37.449611 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:01:37.449322    2575 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/411a782c-9423-495d-8972-6824fdae7f86-model-cache\") on node \"ip-10-0-143-74.ec2.internal\" DevicePath \"\""
Sep 17 18:01:37.578714 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:01:37.578618    2575 generic.go:356] "Generic (PLEG): container finished" podID="411a782c-9423-495d-8972-6824fdae7f86" containerID="9619db8a3e529e505200dd89ce6dd3001cd9b9fee7898fb1d358a51051dc1823" exitCode=0
Sep 17 18:01:37.578714 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:01:37.578671    2575 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-test-gateway-section-name-propagation-94799d44/gw-section-name-router-with-gat-f1d92d0f-kserve-5f5b776bbfjngts"
Sep 17 18:01:37.578714 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:01:37.578702    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-test-gateway-section-name-propagation-94799d44/gw-section-name-router-with-gat-f1d92d0f-kserve-5f5b776bbfjngts" event={"ID":"411a782c-9423-495d-8972-6824fdae7f86","Type":"ContainerDied","Data":"9619db8a3e529e505200dd89ce6dd3001cd9b9fee7898fb1d358a51051dc1823"}
Sep 17 18:01:37.578958 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:01:37.578754    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-test-gateway-section-name-propagation-94799d44/gw-section-name-router-with-gat-f1d92d0f-kserve-5f5b776bbfjngts" event={"ID":"411a782c-9423-495d-8972-6824fdae7f86","Type":"ContainerDied","Data":"0fc563a19f67dc197a6405566b47fd15652d7c074613fad0f36b0cb350d1f4ba"}
Sep 17 18:01:37.578958 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:01:37.578778    2575 scope.go:117] "RemoveContainer" containerID="9619db8a3e529e505200dd89ce6dd3001cd9b9fee7898fb1d358a51051dc1823"
Sep 17 18:01:37.588353 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:01:37.588333    2575 scope.go:117] "RemoveContainer" containerID="9619db8a3e529e505200dd89ce6dd3001cd9b9fee7898fb1d358a51051dc1823"
Sep 17 18:01:37.588712 ip-10-0-143-74 kubenswrapper[2575]: E0917 18:01:37.588688    2575 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"9619db8a3e529e505200dd89ce6dd3001cd9b9fee7898fb1d358a51051dc1823\": container with ID starting with 9619db8a3e529e505200dd89ce6dd3001cd9b9fee7898fb1d358a51051dc1823 not found: ID does not exist" containerID="9619db8a3e529e505200dd89ce6dd3001cd9b9fee7898fb1d358a51051dc1823"
Sep 17 18:01:37.588796 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:01:37.588719    2575 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"9619db8a3e529e505200dd89ce6dd3001cd9b9fee7898fb1d358a51051dc1823"} err="failed to get container status \"9619db8a3e529e505200dd89ce6dd3001cd9b9fee7898fb1d358a51051dc1823\": rpc error: code = NotFound desc = could not find container \"9619db8a3e529e505200dd89ce6dd3001cd9b9fee7898fb1d358a51051dc1823\": container with ID starting with 9619db8a3e529e505200dd89ce6dd3001cd9b9fee7898fb1d358a51051dc1823 not found: ID does not exist"
Sep 17 18:01:37.601369 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:01:37.601341    2575 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["e2e-test-gateway-section-name-propagation-94799d44/gw-section-name-router-with-gat-f1d92d0f-kserve-5f5b776bbfjngts"]
Sep 17 18:01:37.605409 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:01:37.605384    2575 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["e2e-test-gateway-section-name-propagation-94799d44/gw-section-name-router-with-gat-f1d92d0f-kserve-5f5b776bbfjngts"]
Sep 17 18:01:38.967493 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:01:38.967457    2575 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="411a782c-9423-495d-8972-6824fdae7f86" path="/var/lib/kubelet/pods/411a782c-9423-495d-8972-6824fdae7f86/volumes"
Sep 17 18:01:39.420247 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:01:39.420215    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-djvzt_1d07e68d-9943-420f-a6a0-5f477792fabd/konnectivity-agent/0.log" limit=1024
Sep 17 18:01:40.562769 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:01:40.562733    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-djvzt_1d07e68d-9943-420f-a6a0-5f477792fabd/konnectivity-agent/0.log" limit=1024
Sep 17 18:01:42.417272 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:01:42.417239    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-djvzt_1d07e68d-9943-420f-a6a0-5f477792fabd/konnectivity-agent/0.log" limit=1024
Sep 17 18:01:42.486147 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:01:42.486116    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-djvzt_1d07e68d-9943-420f-a6a0-5f477792fabd/konnectivity-agent/0.log" limit=1024
Sep 17 18:01:45.649298 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:01:45.649265    2575 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["e2e-test-llm-auth-enabled-requires-token-d4ed2c08/auth-enabled-test-kserve-8465f8f45c-nmxcb"]
Sep 17 18:01:45.649807 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:01:45.649658    2575 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="411a782c-9423-495d-8972-6824fdae7f86" containerName="main"
Sep 17 18:01:45.649807 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:01:45.649672    2575 state_mem.go:107] "Deleted CPUSet assignment" podUID="411a782c-9423-495d-8972-6824fdae7f86" containerName="main"
Sep 17 18:01:45.649807 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:01:45.649737    2575 memory_manager.go:368] "RemoveStaleState removing state" podUID="411a782c-9423-495d-8972-6824fdae7f86" containerName="main"
Sep 17 18:01:45.652712 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:01:45.652686    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-test-llm-auth-enabled-requires-token-d4ed2c08/auth-enabled-test-kserve-8465f8f45c-nmxcb"
Sep 17 18:01:45.657076 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:01:45.657059    2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-test-llm-auth-enabled-requires-token-d4ed2c08\"/\"kube-root-ca.crt\""
Sep 17 18:01:45.657544 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:01:45.657525    2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-test-llm-auth-enabled-requires-token-d4ed2c08\"/\"openshift-service-ca.crt\""
Sep 17 18:01:45.657970 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:01:45.657950    2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-test-llm-auth-enabled-requires-token-d4ed2c08\"/\"odh-kserve-custom-ca-bundle\""
Sep 17 18:01:45.658048 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:01:45.658019    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-test-llm-auth-enabled-requires-token-d4ed2c08\"/\"auth-enabled-test-kserve-self-signed-certs\""
Sep 17 18:01:45.658762 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:01:45.658732    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-test-llm-auth-enabled-requires-token-d4ed2c08\"/\"default-dockercfg-txq77\""
Sep 17 18:01:45.660287 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:01:45.660262    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-test-llm-auth-enabled-requires-token-d4ed2c08\"/\"seaweedfs-s3-creds\""
Sep 17 18:01:45.690973 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:01:45.690937    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["e2e-test-llm-auth-enabled-requires-token-d4ed2c08/auth-enabled-test-kserve-8465f8f45c-nmxcb"]
Sep 17 18:01:45.709099 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:01:45.709070    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/d4ceaea7-ec2b-4e61-8b9c-5b3f598e732d-tmp-dir\") pod \"auth-enabled-test-kserve-8465f8f45c-nmxcb\" (UID: \"d4ceaea7-ec2b-4e61-8b9c-5b3f598e732d\") " pod="e2e-test-llm-auth-enabled-requires-token-d4ed2c08/auth-enabled-test-kserve-8465f8f45c-nmxcb"
Sep 17 18:01:45.709099 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:01:45.709102    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wt4rw\" (UniqueName: \"kubernetes.io/projected/d4ceaea7-ec2b-4e61-8b9c-5b3f598e732d-kube-api-access-wt4rw\") pod \"auth-enabled-test-kserve-8465f8f45c-nmxcb\" (UID: \"d4ceaea7-ec2b-4e61-8b9c-5b3f598e732d\") " pod="e2e-test-llm-auth-enabled-requires-token-d4ed2c08/auth-enabled-test-kserve-8465f8f45c-nmxcb"
Sep 17 18:01:45.709296 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:01:45.709127    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/d4ceaea7-ec2b-4e61-8b9c-5b3f598e732d-kserve-provision-location\") pod \"auth-enabled-test-kserve-8465f8f45c-nmxcb\" (UID: \"d4ceaea7-ec2b-4e61-8b9c-5b3f598e732d\") " pod="e2e-test-llm-auth-enabled-requires-token-d4ed2c08/auth-enabled-test-kserve-8465f8f45c-nmxcb"
Sep 17 18:01:45.709296 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:01:45.709153    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cabundle-cert\" (UniqueName: \"kubernetes.io/configmap/d4ceaea7-ec2b-4e61-8b9c-5b3f598e732d-cabundle-cert\") pod \"auth-enabled-test-kserve-8465f8f45c-nmxcb\" (UID: \"d4ceaea7-ec2b-4e61-8b9c-5b3f598e732d\") " pod="e2e-test-llm-auth-enabled-requires-token-d4ed2c08/auth-enabled-test-kserve-8465f8f45c-nmxcb"
Sep 17 18:01:45.709296 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:01:45.709249    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/d4ceaea7-ec2b-4e61-8b9c-5b3f598e732d-dshm\") pod \"auth-enabled-test-kserve-8465f8f45c-nmxcb\" (UID: \"d4ceaea7-ec2b-4e61-8b9c-5b3f598e732d\") " pod="e2e-test-llm-auth-enabled-requires-token-d4ed2c08/auth-enabled-test-kserve-8465f8f45c-nmxcb"
Sep 17 18:01:45.709404 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:01:45.709295    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/d4ceaea7-ec2b-4e61-8b9c-5b3f598e732d-model-cache\") pod \"auth-enabled-test-kserve-8465f8f45c-nmxcb\" (UID: \"d4ceaea7-ec2b-4e61-8b9c-5b3f598e732d\") " pod="e2e-test-llm-auth-enabled-requires-token-d4ed2c08/auth-enabled-test-kserve-8465f8f45c-nmxcb"
Sep 17 18:01:45.709404 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:01:45.709322    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/d4ceaea7-ec2b-4e61-8b9c-5b3f598e732d-tls-certs\") pod \"auth-enabled-test-kserve-8465f8f45c-nmxcb\" (UID: \"d4ceaea7-ec2b-4e61-8b9c-5b3f598e732d\") " pod="e2e-test-llm-auth-enabled-requires-token-d4ed2c08/auth-enabled-test-kserve-8465f8f45c-nmxcb"
Sep 17 18:01:45.709487 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:01:45.709462    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/d4ceaea7-ec2b-4e61-8b9c-5b3f598e732d-home\") pod \"auth-enabled-test-kserve-8465f8f45c-nmxcb\" (UID: \"d4ceaea7-ec2b-4e61-8b9c-5b3f598e732d\") " pod="e2e-test-llm-auth-enabled-requires-token-d4ed2c08/auth-enabled-test-kserve-8465f8f45c-nmxcb"
Sep 17 18:01:45.790305 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:01:45.790274    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-djvzt_1d07e68d-9943-420f-a6a0-5f477792fabd/konnectivity-agent/0.log" limit=1024
Sep 17 18:01:45.810139 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:01:45.810110    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/d4ceaea7-ec2b-4e61-8b9c-5b3f598e732d-kserve-provision-location\") pod \"auth-enabled-test-kserve-8465f8f45c-nmxcb\" (UID: \"d4ceaea7-ec2b-4e61-8b9c-5b3f598e732d\") " pod="e2e-test-llm-auth-enabled-requires-token-d4ed2c08/auth-enabled-test-kserve-8465f8f45c-nmxcb"
Sep 17 18:01:45.810258 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:01:45.810147    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cabundle-cert\" (UniqueName: \"kubernetes.io/configmap/d4ceaea7-ec2b-4e61-8b9c-5b3f598e732d-cabundle-cert\") pod \"auth-enabled-test-kserve-8465f8f45c-nmxcb\" (UID: \"d4ceaea7-ec2b-4e61-8b9c-5b3f598e732d\") " pod="e2e-test-llm-auth-enabled-requires-token-d4ed2c08/auth-enabled-test-kserve-8465f8f45c-nmxcb"
Sep 17 18:01:45.810258 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:01:45.810170    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/d4ceaea7-ec2b-4e61-8b9c-5b3f598e732d-dshm\") pod \"auth-enabled-test-kserve-8465f8f45c-nmxcb\" (UID: \"d4ceaea7-ec2b-4e61-8b9c-5b3f598e732d\") " pod="e2e-test-llm-auth-enabled-requires-token-d4ed2c08/auth-enabled-test-kserve-8465f8f45c-nmxcb"
Sep 17 18:01:45.810258 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:01:45.810192    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/d4ceaea7-ec2b-4e61-8b9c-5b3f598e732d-model-cache\") pod \"auth-enabled-test-kserve-8465f8f45c-nmxcb\" (UID: \"d4ceaea7-ec2b-4e61-8b9c-5b3f598e732d\") " pod="e2e-test-llm-auth-enabled-requires-token-d4ed2c08/auth-enabled-test-kserve-8465f8f45c-nmxcb"
Sep 17 18:01:45.810388 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:01:45.810310    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/d4ceaea7-ec2b-4e61-8b9c-5b3f598e732d-tls-certs\") pod \"auth-enabled-test-kserve-8465f8f45c-nmxcb\" (UID: \"d4ceaea7-ec2b-4e61-8b9c-5b3f598e732d\") " pod="e2e-test-llm-auth-enabled-requires-token-d4ed2c08/auth-enabled-test-kserve-8465f8f45c-nmxcb"
Sep 17 18:01:45.810454 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:01:45.810422    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/d4ceaea7-ec2b-4e61-8b9c-5b3f598e732d-home\") pod \"auth-enabled-test-kserve-8465f8f45c-nmxcb\" (UID: \"d4ceaea7-ec2b-4e61-8b9c-5b3f598e732d\") " pod="e2e-test-llm-auth-enabled-requires-token-d4ed2c08/auth-enabled-test-kserve-8465f8f45c-nmxcb"
Sep 17 18:01:45.810567 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:01:45.810491    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/d4ceaea7-ec2b-4e61-8b9c-5b3f598e732d-tmp-dir\") pod \"auth-enabled-test-kserve-8465f8f45c-nmxcb\" (UID: \"d4ceaea7-ec2b-4e61-8b9c-5b3f598e732d\") " pod="e2e-test-llm-auth-enabled-requires-token-d4ed2c08/auth-enabled-test-kserve-8465f8f45c-nmxcb"
Sep 17 18:01:45.810567 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:01:45.810520    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wt4rw\" (UniqueName: \"kubernetes.io/projected/d4ceaea7-ec2b-4e61-8b9c-5b3f598e732d-kube-api-access-wt4rw\") pod \"auth-enabled-test-kserve-8465f8f45c-nmxcb\" (UID: \"d4ceaea7-ec2b-4e61-8b9c-5b3f598e732d\") " pod="e2e-test-llm-auth-enabled-requires-token-d4ed2c08/auth-enabled-test-kserve-8465f8f45c-nmxcb"
Sep 17 18:01:45.810689 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:01:45.810577    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/d4ceaea7-ec2b-4e61-8b9c-5b3f598e732d-kserve-provision-location\") pod \"auth-enabled-test-kserve-8465f8f45c-nmxcb\" (UID: \"d4ceaea7-ec2b-4e61-8b9c-5b3f598e732d\") " pod="e2e-test-llm-auth-enabled-requires-token-d4ed2c08/auth-enabled-test-kserve-8465f8f45c-nmxcb"
Sep 17 18:01:45.810775 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:01:45.810754    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/d4ceaea7-ec2b-4e61-8b9c-5b3f598e732d-model-cache\") pod \"auth-enabled-test-kserve-8465f8f45c-nmxcb\" (UID: \"d4ceaea7-ec2b-4e61-8b9c-5b3f598e732d\") " pod="e2e-test-llm-auth-enabled-requires-token-d4ed2c08/auth-enabled-test-kserve-8465f8f45c-nmxcb"
Sep 17 18:01:45.810831 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:01:45.810813    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/d4ceaea7-ec2b-4e61-8b9c-5b3f598e732d-home\") pod \"auth-enabled-test-kserve-8465f8f45c-nmxcb\" (UID: \"d4ceaea7-ec2b-4e61-8b9c-5b3f598e732d\") " pod="e2e-test-llm-auth-enabled-requires-token-d4ed2c08/auth-enabled-test-kserve-8465f8f45c-nmxcb"
Sep 17 18:01:45.810965 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:01:45.810942    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/d4ceaea7-ec2b-4e61-8b9c-5b3f598e732d-tmp-dir\") pod \"auth-enabled-test-kserve-8465f8f45c-nmxcb\" (UID: \"d4ceaea7-ec2b-4e61-8b9c-5b3f598e732d\") " pod="e2e-test-llm-auth-enabled-requires-token-d4ed2c08/auth-enabled-test-kserve-8465f8f45c-nmxcb"
Sep 17 18:01:45.810965 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:01:45.810960    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cabundle-cert\" (UniqueName: \"kubernetes.io/configmap/d4ceaea7-ec2b-4e61-8b9c-5b3f598e732d-cabundle-cert\") pod \"auth-enabled-test-kserve-8465f8f45c-nmxcb\" (UID: \"d4ceaea7-ec2b-4e61-8b9c-5b3f598e732d\") " pod="e2e-test-llm-auth-enabled-requires-token-d4ed2c08/auth-enabled-test-kserve-8465f8f45c-nmxcb"
Sep 17 18:01:45.812562 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:01:45.812536    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/d4ceaea7-ec2b-4e61-8b9c-5b3f598e732d-dshm\") pod \"auth-enabled-test-kserve-8465f8f45c-nmxcb\" (UID: \"d4ceaea7-ec2b-4e61-8b9c-5b3f598e732d\") " pod="e2e-test-llm-auth-enabled-requires-token-d4ed2c08/auth-enabled-test-kserve-8465f8f45c-nmxcb"
Sep 17 18:01:45.812828 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:01:45.812812    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/d4ceaea7-ec2b-4e61-8b9c-5b3f598e732d-tls-certs\") pod \"auth-enabled-test-kserve-8465f8f45c-nmxcb\" (UID: \"d4ceaea7-ec2b-4e61-8b9c-5b3f598e732d\") " pod="e2e-test-llm-auth-enabled-requires-token-d4ed2c08/auth-enabled-test-kserve-8465f8f45c-nmxcb"
Sep 17 18:01:45.829762 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:01:45.829737    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-wt4rw\" (UniqueName: \"kubernetes.io/projected/d4ceaea7-ec2b-4e61-8b9c-5b3f598e732d-kube-api-access-wt4rw\") pod \"auth-enabled-test-kserve-8465f8f45c-nmxcb\" (UID: \"d4ceaea7-ec2b-4e61-8b9c-5b3f598e732d\") " pod="e2e-test-llm-auth-enabled-requires-token-d4ed2c08/auth-enabled-test-kserve-8465f8f45c-nmxcb"
Sep 17 18:01:45.969753 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:01:45.969664    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-test-llm-auth-enabled-requires-token-d4ed2c08/auth-enabled-test-kserve-8465f8f45c-nmxcb"
Sep 17 18:01:46.136502 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:01:46.136467    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-djvzt_1d07e68d-9943-420f-a6a0-5f477792fabd/konnectivity-agent/0.log" limit=1024
Sep 17 18:01:46.263178 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:01:46.263151    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["e2e-test-llm-auth-enabled-requires-token-d4ed2c08/auth-enabled-test-kserve-8465f8f45c-nmxcb"]
Sep 17 18:01:46.263772 ip-10-0-143-74 kubenswrapper[2575]: W0917 18:01:46.263736    2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd4ceaea7_ec2b_4e61_8b9c_5b3f598e732d.slice/crio-b1bb7e63ff0cf1e2bdfe9e61aae8166d8455de59b8985b393bf44603ff2a180b WatchSource:0}: Error finding container b1bb7e63ff0cf1e2bdfe9e61aae8166d8455de59b8985b393bf44603ff2a180b: Status 404 returned error can't find the container with id b1bb7e63ff0cf1e2bdfe9e61aae8166d8455de59b8985b393bf44603ff2a180b
Sep 17 18:01:46.265571 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:01:46.265552    2575 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Sep 17 18:01:46.630040 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:01:46.630009    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-test-llm-auth-enabled-requires-token-d4ed2c08/auth-enabled-test-kserve-8465f8f45c-nmxcb" event={"ID":"d4ceaea7-ec2b-4e61-8b9c-5b3f598e732d","Type":"ContainerStarted","Data":"b1bb7e63ff0cf1e2bdfe9e61aae8166d8455de59b8985b393bf44603ff2a180b"}
Sep 17 18:01:47.607903 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:01:47.607868    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-djvzt_1d07e68d-9943-420f-a6a0-5f477792fabd/konnectivity-agent/0.log" limit=1024
Sep 17 18:01:48.625524 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:01:48.625486    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-djvzt_1d07e68d-9943-420f-a6a0-5f477792fabd/konnectivity-agent/0.log" limit=1024
Sep 17 18:01:53.668807 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:01:53.668763    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-test-llm-auth-enabled-requires-token-d4ed2c08/auth-enabled-test-kserve-8465f8f45c-nmxcb" event={"ID":"d4ceaea7-ec2b-4e61-8b9c-5b3f598e732d","Type":"ContainerStarted","Data":"c62df334f5c32e9f09e9f6cfb76497cf7416f37d81b4252eec64bbf58fe8aa50"}
Sep 17 18:01:54.324724 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:01:54.324693    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-djvzt_1d07e68d-9943-420f-a6a0-5f477792fabd/konnectivity-agent/0.log" limit=1024
Sep 17 18:01:55.680986 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:01:55.680897    2575 generic.go:356] "Generic (PLEG): container finished" podID="d4ceaea7-ec2b-4e61-8b9c-5b3f598e732d" containerID="c62df334f5c32e9f09e9f6cfb76497cf7416f37d81b4252eec64bbf58fe8aa50" exitCode=0
Sep 17 18:01:55.680986 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:01:55.680972    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-test-llm-auth-enabled-requires-token-d4ed2c08/auth-enabled-test-kserve-8465f8f45c-nmxcb" event={"ID":"d4ceaea7-ec2b-4e61-8b9c-5b3f598e732d","Type":"ContainerDied","Data":"c62df334f5c32e9f09e9f6cfb76497cf7416f37d81b4252eec64bbf58fe8aa50"}
Sep 17 18:02:03.792984 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:02:03.792941    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-djvzt_1d07e68d-9943-420f-a6a0-5f477792fabd/konnectivity-agent/0.log" limit=1024
Sep 17 18:02:03.894982 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:02:03.894943    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-djvzt_1d07e68d-9943-420f-a6a0-5f477792fabd/konnectivity-agent/0.log" limit=1024
Sep 17 18:02:06.341792 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:02:06.341744    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-djvzt_1d07e68d-9943-420f-a6a0-5f477792fabd/konnectivity-agent/0.log" limit=1024
Sep 17 18:02:25.888763 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:02:25.888728    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-test-llm-auth-enabled-requires-token-d4ed2c08/auth-enabled-test-kserve-8465f8f45c-nmxcb" event={"ID":"d4ceaea7-ec2b-4e61-8b9c-5b3f598e732d","Type":"ContainerStarted","Data":"8bfbd8cfff9bd0cb1f3474f2d6e3ecdc249a463e717a362ffe443c3229c0c217"}
Sep 17 18:02:25.915529 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:02:25.915476    2575 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="e2e-test-llm-auth-enabled-requires-token-d4ed2c08/auth-enabled-test-kserve-8465f8f45c-nmxcb" podStartSLOduration=2.374378333 podStartE2EDuration="40.915424248s" totalImagesPullingTime="36.318658389s" totalInitContainerRuntime="2.222387526s" isStatefulPod=false podCreationTimestamp="2026-09-17 18:01:45 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-17 18:02:25.912773995 +0000 UTC m=+1665.543103970" watchObservedRunningTime="2026-09-17 18:02:25.915424248 +0000 UTC m=+1665.545754222"
Sep 17 18:02:25.970624 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:02:25.970589    2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-test-llm-auth-enabled-requires-token-d4ed2c08/auth-enabled-test-kserve-8465f8f45c-nmxcb"
Sep 17 18:02:25.970786 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:02:25.970658    2575 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="e2e-test-llm-auth-enabled-requires-token-d4ed2c08/auth-enabled-test-kserve-8465f8f45c-nmxcb"
Sep 17 18:02:25.972094 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:02:25.972066    2575 prober.go:120] "Probe failed" probeType="Startup" pod="e2e-test-llm-auth-enabled-requires-token-d4ed2c08/auth-enabled-test-kserve-8465f8f45c-nmxcb" podUID="d4ceaea7-ec2b-4e61-8b9c-5b3f598e732d" containerName="main" probeResult="failure" output="Get \"https://10.133.0.49:8000/health\": dial tcp 10.133.0.49:8000: connect: connection refused"
Sep 17 18:02:26.648802 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:02:26.648765    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-djvzt_1d07e68d-9943-420f-a6a0-5f477792fabd/konnectivity-agent/0.log" limit=1024
Sep 17 18:02:26.716352 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:02:26.716314    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-djvzt_1d07e68d-9943-420f-a6a0-5f477792fabd/konnectivity-agent/0.log" limit=1024
Sep 17 18:02:35.970638 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:02:35.970595    2575 prober.go:120] "Probe failed" probeType="Startup" pod="e2e-test-llm-auth-enabled-requires-token-d4ed2c08/auth-enabled-test-kserve-8465f8f45c-nmxcb" podUID="d4ceaea7-ec2b-4e61-8b9c-5b3f598e732d" containerName="main" probeResult="failure" output="Get \"https://10.133.0.49:8000/health\": dial tcp 10.133.0.49:8000: connect: connection refused"
Sep 17 18:02:45.971074 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:02:45.971029    2575 prober.go:120] "Probe failed" probeType="Startup" pod="e2e-test-llm-auth-enabled-requires-token-d4ed2c08/auth-enabled-test-kserve-8465f8f45c-nmxcb" podUID="d4ceaea7-ec2b-4e61-8b9c-5b3f598e732d" containerName="main" probeResult="failure" output="Get \"https://10.133.0.49:8000/health\": dial tcp 10.133.0.49:8000: connect: connection refused"
Sep 17 18:02:55.970285 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:02:55.970243    2575 prober.go:120] "Probe failed" probeType="Startup" pod="e2e-test-llm-auth-enabled-requires-token-d4ed2c08/auth-enabled-test-kserve-8465f8f45c-nmxcb" podUID="d4ceaea7-ec2b-4e61-8b9c-5b3f598e732d" containerName="main" probeResult="failure" output="Get \"https://10.133.0.49:8000/health\": dial tcp 10.133.0.49:8000: connect: connection refused"
Sep 17 18:03:05.970531 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:03:05.970403    2575 prober.go:120] "Probe failed" probeType="Startup" pod="e2e-test-llm-auth-enabled-requires-token-d4ed2c08/auth-enabled-test-kserve-8465f8f45c-nmxcb" podUID="d4ceaea7-ec2b-4e61-8b9c-5b3f598e732d" containerName="main" probeResult="failure" output="Get \"https://10.133.0.49:8000/health\": dial tcp 10.133.0.49:8000: connect: connection refused"
Sep 17 18:03:15.971090 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:03:15.971028    2575 prober.go:120] "Probe failed" probeType="Startup" pod="e2e-test-llm-auth-enabled-requires-token-d4ed2c08/auth-enabled-test-kserve-8465f8f45c-nmxcb" podUID="d4ceaea7-ec2b-4e61-8b9c-5b3f598e732d" containerName="main" probeResult="failure" output="Get \"https://10.133.0.49:8000/health\": dial tcp 10.133.0.49:8000: connect: connection refused"
Sep 17 18:03:16.759253 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:03:16.759220    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-djvzt_1d07e68d-9943-420f-a6a0-5f477792fabd/konnectivity-agent/0.log" limit=1024
Sep 17 18:03:16.851507 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:03:16.851464    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-djvzt_1d07e68d-9943-420f-a6a0-5f477792fabd/konnectivity-agent/0.log" limit=1024
Sep 17 18:03:23.191119 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:03:23.191073    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-djvzt_1d07e68d-9943-420f-a6a0-5f477792fabd/konnectivity-agent/0.log" limit=1024
Sep 17 18:03:25.970587 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:03:25.970536    2575 prober.go:120] "Probe failed" probeType="Startup" pod="e2e-test-llm-auth-enabled-requires-token-d4ed2c08/auth-enabled-test-kserve-8465f8f45c-nmxcb" podUID="d4ceaea7-ec2b-4e61-8b9c-5b3f598e732d" containerName="main" probeResult="failure" output="Get \"https://10.133.0.49:8000/health\": dial tcp 10.133.0.49:8000: connect: connection refused"
Sep 17 18:03:28.987347 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:03:28.987309    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-djvzt_1d07e68d-9943-420f-a6a0-5f477792fabd/konnectivity-agent/0.log" limit=1024
Sep 17 18:03:29.086956 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:03:29.086916    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-djvzt_1d07e68d-9943-420f-a6a0-5f477792fabd/konnectivity-agent/0.log" limit=1024
Sep 17 18:03:31.398640 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:03:31.398603    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-djvzt_1d07e68d-9943-420f-a6a0-5f477792fabd/konnectivity-agent/0.log" limit=1024
Sep 17 18:03:31.728682 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:03:31.728594    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-djvzt_1d07e68d-9943-420f-a6a0-5f477792fabd/konnectivity-agent/0.log" limit=1024
Sep 17 18:03:35.970747 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:03:35.970702    2575 prober.go:120] "Probe failed" probeType="Startup" pod="e2e-test-llm-auth-enabled-requires-token-d4ed2c08/auth-enabled-test-kserve-8465f8f45c-nmxcb" podUID="d4ceaea7-ec2b-4e61-8b9c-5b3f598e732d" containerName="main" probeResult="failure" output="Get \"https://10.133.0.49:8000/health\": dial tcp 10.133.0.49:8000: connect: connection refused"
Sep 17 18:03:45.970049 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:03:45.970002    2575 prober.go:120] "Probe failed" probeType="Startup" pod="e2e-test-llm-auth-enabled-requires-token-d4ed2c08/auth-enabled-test-kserve-8465f8f45c-nmxcb" podUID="d4ceaea7-ec2b-4e61-8b9c-5b3f598e732d" containerName="main" probeResult="failure" output="Get \"https://10.133.0.49:8000/health\": dial tcp 10.133.0.49:8000: connect: connection refused"
Sep 17 18:03:50.171863 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:03:50.171815    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-djvzt_1d07e68d-9943-420f-a6a0-5f477792fabd/konnectivity-agent/0.log" limit=1024
Sep 17 18:03:55.970649 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:03:55.970595    2575 prober.go:120] "Probe failed" probeType="Startup" pod="e2e-test-llm-auth-enabled-requires-token-d4ed2c08/auth-enabled-test-kserve-8465f8f45c-nmxcb" podUID="d4ceaea7-ec2b-4e61-8b9c-5b3f598e732d" containerName="main" probeResult="failure" output="Get \"https://10.133.0.49:8000/health\": dial tcp 10.133.0.49:8000: connect: connection refused"
Sep 17 18:04:05.970822 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:04:05.970773    2575 prober.go:120] "Probe failed" probeType="Startup" pod="e2e-test-llm-auth-enabled-requires-token-d4ed2c08/auth-enabled-test-kserve-8465f8f45c-nmxcb" podUID="d4ceaea7-ec2b-4e61-8b9c-5b3f598e732d" containerName="main" probeResult="failure" output="Get \"https://10.133.0.49:8000/health\": dial tcp 10.133.0.49:8000: connect: connection refused"
Sep 17 18:04:07.176044 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:04:07.176001    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-djvzt_1d07e68d-9943-420f-a6a0-5f477792fabd/konnectivity-agent/0.log" limit=1024
Sep 17 18:04:07.245735 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:04:07.245700    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-djvzt_1d07e68d-9943-420f-a6a0-5f477792fabd/konnectivity-agent/0.log" limit=1024
Sep 17 18:04:15.970465 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:04:15.970400    2575 prober.go:120] "Probe failed" probeType="Startup" pod="e2e-test-llm-auth-enabled-requires-token-d4ed2c08/auth-enabled-test-kserve-8465f8f45c-nmxcb" podUID="d4ceaea7-ec2b-4e61-8b9c-5b3f598e732d" containerName="main" probeResult="failure" output="Get \"https://10.133.0.49:8000/health\": dial tcp 10.133.0.49:8000: connect: connection refused"
Sep 17 18:04:25.980674 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:04:25.980641    2575 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="e2e-test-llm-auth-enabled-requires-token-d4ed2c08/auth-enabled-test-kserve-8465f8f45c-nmxcb"
Sep 17 18:04:25.988721 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:04:25.988691    2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-test-llm-auth-enabled-requires-token-d4ed2c08/auth-enabled-test-kserve-8465f8f45c-nmxcb"
Sep 17 18:04:31.487576 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:04:31.487532    2575 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["e2e-test-llm-auth-enabled-requires-token-d4ed2c08/auth-enabled-test-kserve-8465f8f45c-nmxcb"]
Sep 17 18:04:31.490880 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:04:31.490833    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-djvzt_1d07e68d-9943-420f-a6a0-5f477792fabd/konnectivity-agent/0.log" limit=1024
Sep 17 18:04:31.565208 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:04:31.565176    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-djvzt_1d07e68d-9943-420f-a6a0-5f477792fabd/konnectivity-agent/0.log" limit=1024
Sep 17 18:04:46.510599 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:04:46.510542    2575 kuberuntime_container.go:873] "Killing container with a grace period" pod="e2e-test-llm-auth-enabled-requires-token-d4ed2c08/auth-enabled-test-kserve-8465f8f45c-nmxcb" podUID="d4ceaea7-ec2b-4e61-8b9c-5b3f598e732d" containerName="main" containerID="cri-o://8bfbd8cfff9bd0cb1f3474f2d6e3ecdc249a463e717a362ffe443c3229c0c217" gracePeriod=45
Sep 17 18:04:46.982189 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:04:46.982150    2575 prober.go:120] "Probe failed" probeType="Readiness" pod="e2e-test-llm-auth-enabled-requires-token-d4ed2c08/auth-enabled-test-kserve-8465f8f45c-nmxcb" podUID="d4ceaea7-ec2b-4e61-8b9c-5b3f598e732d" containerName="main" probeResult="failure" output="Get \"https://10.133.0.49:8000/health\": dial tcp 10.133.0.49:8000: connect: connection refused"
Sep 17 18:04:47.170903 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:04:47.170872    2575 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-test-llm-auth-enabled-requires-token-d4ed2c08/auth-enabled-test-kserve-8465f8f45c-nmxcb"
Sep 17 18:04:47.179180 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:04:47.179155    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/d4ceaea7-ec2b-4e61-8b9c-5b3f598e732d-home\") pod \"d4ceaea7-ec2b-4e61-8b9c-5b3f598e732d\" (UID: \"d4ceaea7-ec2b-4e61-8b9c-5b3f598e732d\") "
Sep 17 18:04:47.179258 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:04:47.179188    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/d4ceaea7-ec2b-4e61-8b9c-5b3f598e732d-model-cache\") pod \"d4ceaea7-ec2b-4e61-8b9c-5b3f598e732d\" (UID: \"d4ceaea7-ec2b-4e61-8b9c-5b3f598e732d\") "
Sep 17 18:04:47.179258 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:04:47.179213    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/d4ceaea7-ec2b-4e61-8b9c-5b3f598e732d-dshm\") pod \"d4ceaea7-ec2b-4e61-8b9c-5b3f598e732d\" (UID: \"d4ceaea7-ec2b-4e61-8b9c-5b3f598e732d\") "
Sep 17 18:04:47.179341 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:04:47.179263    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/d4ceaea7-ec2b-4e61-8b9c-5b3f598e732d-tmp-dir\") pod \"d4ceaea7-ec2b-4e61-8b9c-5b3f598e732d\" (UID: \"d4ceaea7-ec2b-4e61-8b9c-5b3f598e732d\") "
Sep 17 18:04:47.179341 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:04:47.179296    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/d4ceaea7-ec2b-4e61-8b9c-5b3f598e732d-tls-certs\") pod \"d4ceaea7-ec2b-4e61-8b9c-5b3f598e732d\" (UID: \"d4ceaea7-ec2b-4e61-8b9c-5b3f598e732d\") "
Sep 17 18:04:47.179425 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:04:47.179364    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-wt4rw\" (UniqueName: \"kubernetes.io/projected/d4ceaea7-ec2b-4e61-8b9c-5b3f598e732d-kube-api-access-wt4rw\") pod \"d4ceaea7-ec2b-4e61-8b9c-5b3f598e732d\" (UID: \"d4ceaea7-ec2b-4e61-8b9c-5b3f598e732d\") "
Sep 17 18:04:47.179512 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:04:47.179427    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"cabundle-cert\" (UniqueName: \"kubernetes.io/configmap/d4ceaea7-ec2b-4e61-8b9c-5b3f598e732d-cabundle-cert\") pod \"d4ceaea7-ec2b-4e61-8b9c-5b3f598e732d\" (UID: \"d4ceaea7-ec2b-4e61-8b9c-5b3f598e732d\") "
Sep 17 18:04:47.179512 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:04:47.179486    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/d4ceaea7-ec2b-4e61-8b9c-5b3f598e732d-kserve-provision-location\") pod \"d4ceaea7-ec2b-4e61-8b9c-5b3f598e732d\" (UID: \"d4ceaea7-ec2b-4e61-8b9c-5b3f598e732d\") "
Sep 17 18:04:47.179747 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:04:47.179460    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d4ceaea7-ec2b-4e61-8b9c-5b3f598e732d-model-cache" (OuterVolumeSpecName: "model-cache") pod "d4ceaea7-ec2b-4e61-8b9c-5b3f598e732d" (UID: "d4ceaea7-ec2b-4e61-8b9c-5b3f598e732d"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 17 18:04:47.179837 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:04:47.179815    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/d4ceaea7-ec2b-4e61-8b9c-5b3f598e732d-cabundle-cert" (OuterVolumeSpecName: "cabundle-cert") pod "d4ceaea7-ec2b-4e61-8b9c-5b3f598e732d" (UID: "d4ceaea7-ec2b-4e61-8b9c-5b3f598e732d"). InnerVolumeSpecName "cabundle-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 17 18:04:47.180179 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:04:47.180153    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d4ceaea7-ec2b-4e61-8b9c-5b3f598e732d-home" (OuterVolumeSpecName: "home") pod "d4ceaea7-ec2b-4e61-8b9c-5b3f598e732d" (UID: "d4ceaea7-ec2b-4e61-8b9c-5b3f598e732d"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 17 18:04:47.181631 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:04:47.181601    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/d4ceaea7-ec2b-4e61-8b9c-5b3f598e732d-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "d4ceaea7-ec2b-4e61-8b9c-5b3f598e732d" (UID: "d4ceaea7-ec2b-4e61-8b9c-5b3f598e732d"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 17 18:04:47.181746 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:04:47.181725    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d4ceaea7-ec2b-4e61-8b9c-5b3f598e732d-kube-api-access-wt4rw" (OuterVolumeSpecName: "kube-api-access-wt4rw") pod "d4ceaea7-ec2b-4e61-8b9c-5b3f598e732d" (UID: "d4ceaea7-ec2b-4e61-8b9c-5b3f598e732d"). InnerVolumeSpecName "kube-api-access-wt4rw". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 17 18:04:47.181746 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:04:47.181738    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d4ceaea7-ec2b-4e61-8b9c-5b3f598e732d-dshm" (OuterVolumeSpecName: "dshm") pod "d4ceaea7-ec2b-4e61-8b9c-5b3f598e732d" (UID: "d4ceaea7-ec2b-4e61-8b9c-5b3f598e732d"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 17 18:04:47.198831 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:04:47.198794    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d4ceaea7-ec2b-4e61-8b9c-5b3f598e732d-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "d4ceaea7-ec2b-4e61-8b9c-5b3f598e732d" (UID: "d4ceaea7-ec2b-4e61-8b9c-5b3f598e732d"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 17 18:04:47.207399 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:04:47.207365    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d4ceaea7-ec2b-4e61-8b9c-5b3f598e732d-kserve-provision-location" (OuterVolumeSpecName: "kserve-provision-location") pod "d4ceaea7-ec2b-4e61-8b9c-5b3f598e732d" (UID: "d4ceaea7-ec2b-4e61-8b9c-5b3f598e732d"). InnerVolumeSpecName "kserve-provision-location". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 17 18:04:47.280395 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:04:47.280305    2575 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/d4ceaea7-ec2b-4e61-8b9c-5b3f598e732d-tmp-dir\") on node \"ip-10-0-143-74.ec2.internal\" DevicePath \"\""
Sep 17 18:04:47.280395 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:04:47.280340    2575 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/d4ceaea7-ec2b-4e61-8b9c-5b3f598e732d-tls-certs\") on node \"ip-10-0-143-74.ec2.internal\" DevicePath \"\""
Sep 17 18:04:47.280395 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:04:47.280352    2575 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-wt4rw\" (UniqueName: \"kubernetes.io/projected/d4ceaea7-ec2b-4e61-8b9c-5b3f598e732d-kube-api-access-wt4rw\") on node \"ip-10-0-143-74.ec2.internal\" DevicePath \"\""
Sep 17 18:04:47.280395 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:04:47.280360    2575 reconciler_common.go:299] "Volume detached for volume \"cabundle-cert\" (UniqueName: \"kubernetes.io/configmap/d4ceaea7-ec2b-4e61-8b9c-5b3f598e732d-cabundle-cert\") on node \"ip-10-0-143-74.ec2.internal\" DevicePath \"\""
Sep 17 18:04:47.280395 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:04:47.280370    2575 reconciler_common.go:299] "Volume detached for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/d4ceaea7-ec2b-4e61-8b9c-5b3f598e732d-kserve-provision-location\") on node \"ip-10-0-143-74.ec2.internal\" DevicePath \"\""
Sep 17 18:04:47.280395 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:04:47.280378    2575 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/d4ceaea7-ec2b-4e61-8b9c-5b3f598e732d-home\") on node \"ip-10-0-143-74.ec2.internal\" DevicePath \"\""
Sep 17 18:04:47.280395 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:04:47.280386    2575 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/d4ceaea7-ec2b-4e61-8b9c-5b3f598e732d-model-cache\") on node \"ip-10-0-143-74.ec2.internal\" DevicePath \"\""
Sep 17 18:04:47.280395 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:04:47.280393    2575 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/d4ceaea7-ec2b-4e61-8b9c-5b3f598e732d-dshm\") on node \"ip-10-0-143-74.ec2.internal\" DevicePath \"\""
Sep 17 18:04:47.720799 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:04:47.720762    2575 generic.go:356] "Generic (PLEG): container finished" podID="d4ceaea7-ec2b-4e61-8b9c-5b3f598e732d" containerID="8bfbd8cfff9bd0cb1f3474f2d6e3ecdc249a463e717a362ffe443c3229c0c217" exitCode=0
Sep 17 18:04:47.721196 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:04:47.720816    2575 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-test-llm-auth-enabled-requires-token-d4ed2c08/auth-enabled-test-kserve-8465f8f45c-nmxcb"
Sep 17 18:04:47.721196 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:04:47.720847    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-test-llm-auth-enabled-requires-token-d4ed2c08/auth-enabled-test-kserve-8465f8f45c-nmxcb" event={"ID":"d4ceaea7-ec2b-4e61-8b9c-5b3f598e732d","Type":"ContainerDied","Data":"8bfbd8cfff9bd0cb1f3474f2d6e3ecdc249a463e717a362ffe443c3229c0c217"}
Sep 17 18:04:47.721196 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:04:47.720937    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-test-llm-auth-enabled-requires-token-d4ed2c08/auth-enabled-test-kserve-8465f8f45c-nmxcb" event={"ID":"d4ceaea7-ec2b-4e61-8b9c-5b3f598e732d","Type":"ContainerDied","Data":"b1bb7e63ff0cf1e2bdfe9e61aae8166d8455de59b8985b393bf44603ff2a180b"}
Sep 17 18:04:47.721196 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:04:47.720961    2575 scope.go:117] "RemoveContainer" containerID="8bfbd8cfff9bd0cb1f3474f2d6e3ecdc249a463e717a362ffe443c3229c0c217"
Sep 17 18:04:47.730535 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:04:47.730513    2575 scope.go:117] "RemoveContainer" containerID="c62df334f5c32e9f09e9f6cfb76497cf7416f37d81b4252eec64bbf58fe8aa50"
Sep 17 18:04:47.739391 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:04:47.739364    2575 scope.go:117] "RemoveContainer" containerID="8bfbd8cfff9bd0cb1f3474f2d6e3ecdc249a463e717a362ffe443c3229c0c217"
Sep 17 18:04:47.739738 ip-10-0-143-74 kubenswrapper[2575]: E0917 18:04:47.739719    2575 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"8bfbd8cfff9bd0cb1f3474f2d6e3ecdc249a463e717a362ffe443c3229c0c217\": container with ID starting with 8bfbd8cfff9bd0cb1f3474f2d6e3ecdc249a463e717a362ffe443c3229c0c217 not found: ID does not exist" containerID="8bfbd8cfff9bd0cb1f3474f2d6e3ecdc249a463e717a362ffe443c3229c0c217"
Sep 17 18:04:47.739814 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:04:47.739746    2575 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"8bfbd8cfff9bd0cb1f3474f2d6e3ecdc249a463e717a362ffe443c3229c0c217"} err="failed to get container status \"8bfbd8cfff9bd0cb1f3474f2d6e3ecdc249a463e717a362ffe443c3229c0c217\": rpc error: code = NotFound desc = could not find container \"8bfbd8cfff9bd0cb1f3474f2d6e3ecdc249a463e717a362ffe443c3229c0c217\": container with ID starting with 8bfbd8cfff9bd0cb1f3474f2d6e3ecdc249a463e717a362ffe443c3229c0c217 not found: ID does not exist"
Sep 17 18:04:47.739814 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:04:47.739764    2575 scope.go:117] "RemoveContainer" containerID="c62df334f5c32e9f09e9f6cfb76497cf7416f37d81b4252eec64bbf58fe8aa50"
Sep 17 18:04:47.740059 ip-10-0-143-74 kubenswrapper[2575]: E0917 18:04:47.740038    2575 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"c62df334f5c32e9f09e9f6cfb76497cf7416f37d81b4252eec64bbf58fe8aa50\": container with ID starting with c62df334f5c32e9f09e9f6cfb76497cf7416f37d81b4252eec64bbf58fe8aa50 not found: ID does not exist" containerID="c62df334f5c32e9f09e9f6cfb76497cf7416f37d81b4252eec64bbf58fe8aa50"
Sep 17 18:04:47.740106 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:04:47.740066    2575 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"c62df334f5c32e9f09e9f6cfb76497cf7416f37d81b4252eec64bbf58fe8aa50"} err="failed to get container status \"c62df334f5c32e9f09e9f6cfb76497cf7416f37d81b4252eec64bbf58fe8aa50\": rpc error: code = NotFound desc = could not find container \"c62df334f5c32e9f09e9f6cfb76497cf7416f37d81b4252eec64bbf58fe8aa50\": container with ID starting with c62df334f5c32e9f09e9f6cfb76497cf7416f37d81b4252eec64bbf58fe8aa50 not found: ID does not exist"
Sep 17 18:04:47.743967 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:04:47.743940    2575 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["e2e-test-llm-auth-enabled-requires-token-d4ed2c08/auth-enabled-test-kserve-8465f8f45c-nmxcb"]
Sep 17 18:04:47.749967 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:04:47.749940    2575 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["e2e-test-llm-auth-enabled-requires-token-d4ed2c08/auth-enabled-test-kserve-8465f8f45c-nmxcb"]
Sep 17 18:04:48.967681 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:04:48.967646    2575 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="d4ceaea7-ec2b-4e61-8b9c-5b3f598e732d" path="/var/lib/kubelet/pods/d4ceaea7-ec2b-4e61-8b9c-5b3f598e732d/volumes"
Sep 17 18:04:57.252827 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:04:57.252786    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-djvzt_1d07e68d-9943-420f-a6a0-5f477792fabd/konnectivity-agent/0.log" limit=1024
Sep 17 18:04:57.348178 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:04:57.348146    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-djvzt_1d07e68d-9943-420f-a6a0-5f477792fabd/konnectivity-agent/0.log" limit=1024
Sep 17 18:05:03.612980 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:05:03.612936    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-djvzt_1d07e68d-9943-420f-a6a0-5f477792fabd/konnectivity-agent/0.log" limit=1024
Sep 17 18:05:07.204398 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:05:07.204360    2575 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["e2e-test-llm-inference-service-cf47448b/tokenizer-clean-path-test-kserve-5b769465b9-4cdcj"]
Sep 17 18:05:07.205009 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:05:07.204990    2575 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d4ceaea7-ec2b-4e61-8b9c-5b3f598e732d" containerName="storage-initializer"
Sep 17 18:05:07.205071 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:05:07.205011    2575 state_mem.go:107] "Deleted CPUSet assignment" podUID="d4ceaea7-ec2b-4e61-8b9c-5b3f598e732d" containerName="storage-initializer"
Sep 17 18:05:07.205071 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:05:07.205020    2575 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d4ceaea7-ec2b-4e61-8b9c-5b3f598e732d" containerName="main"
Sep 17 18:05:07.205071 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:05:07.205025    2575 state_mem.go:107] "Deleted CPUSet assignment" podUID="d4ceaea7-ec2b-4e61-8b9c-5b3f598e732d" containerName="main"
Sep 17 18:05:07.205186 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:05:07.205101    2575 memory_manager.go:368] "RemoveStaleState removing state" podUID="d4ceaea7-ec2b-4e61-8b9c-5b3f598e732d" containerName="main"
Sep 17 18:05:07.207188 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:05:07.207166    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-test-llm-inference-service-cf47448b/tokenizer-clean-path-test-kserve-5b769465b9-4cdcj"
Sep 17 18:05:07.209906 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:05:07.209880    2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-test-llm-inference-service-cf47448b\"/\"kube-root-ca.crt\""
Sep 17 18:05:07.210031 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:05:07.209929    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-test-llm-inference-service-cf47448b\"/\"tokenizer-clean-path-test-kserve-self-signed-certs\""
Sep 17 18:05:07.210031 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:05:07.209956    2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-test-llm-inference-service-cf47448b\"/\"odh-kserve-custom-ca-bundle\""
Sep 17 18:05:07.210812 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:05:07.210789    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-test-llm-inference-service-cf47448b\"/\"seaweedfs-s3-creds\""
Sep 17 18:05:07.210892 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:05:07.210848    2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-test-llm-inference-service-cf47448b\"/\"openshift-service-ca.crt\""
Sep 17 18:05:07.210957 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:05:07.210937    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-test-llm-inference-service-cf47448b\"/\"default-dockercfg-9wzmx\""
Sep 17 18:05:07.227287 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:05:07.227254    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-djvzt_1d07e68d-9943-420f-a6a0-5f477792fabd/konnectivity-agent/0.log" limit=1024
Sep 17 18:05:07.228631 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:05:07.228611    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["e2e-test-llm-inference-service-cf47448b/tokenizer-clean-path-test-kserve-5b769465b9-4cdcj"]
Sep 17 18:05:07.248365 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:05:07.248339    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vkzvk\" (UniqueName: \"kubernetes.io/projected/b6f4ed78-4222-460f-bf36-ad4ff96eb150-kube-api-access-vkzvk\") pod \"tokenizer-clean-path-test-kserve-5b769465b9-4cdcj\" (UID: \"b6f4ed78-4222-460f-bf36-ad4ff96eb150\") " pod="e2e-test-llm-inference-service-cf47448b/tokenizer-clean-path-test-kserve-5b769465b9-4cdcj"
Sep 17 18:05:07.248555 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:05:07.248373    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/b6f4ed78-4222-460f-bf36-ad4ff96eb150-tmp-dir\") pod \"tokenizer-clean-path-test-kserve-5b769465b9-4cdcj\" (UID: \"b6f4ed78-4222-460f-bf36-ad4ff96eb150\") " pod="e2e-test-llm-inference-service-cf47448b/tokenizer-clean-path-test-kserve-5b769465b9-4cdcj"
Sep 17 18:05:07.248555 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:05:07.248400    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/b6f4ed78-4222-460f-bf36-ad4ff96eb150-tls-certs\") pod \"tokenizer-clean-path-test-kserve-5b769465b9-4cdcj\" (UID: \"b6f4ed78-4222-460f-bf36-ad4ff96eb150\") " pod="e2e-test-llm-inference-service-cf47448b/tokenizer-clean-path-test-kserve-5b769465b9-4cdcj"
Sep 17 18:05:07.248555 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:05:07.248477    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/b6f4ed78-4222-460f-bf36-ad4ff96eb150-dshm\") pod \"tokenizer-clean-path-test-kserve-5b769465b9-4cdcj\" (UID: \"b6f4ed78-4222-460f-bf36-ad4ff96eb150\") " pod="e2e-test-llm-inference-service-cf47448b/tokenizer-clean-path-test-kserve-5b769465b9-4cdcj"
Sep 17 18:05:07.248555 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:05:07.248500    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cabundle-cert\" (UniqueName: \"kubernetes.io/configmap/b6f4ed78-4222-460f-bf36-ad4ff96eb150-cabundle-cert\") pod \"tokenizer-clean-path-test-kserve-5b769465b9-4cdcj\" (UID: \"b6f4ed78-4222-460f-bf36-ad4ff96eb150\") " pod="e2e-test-llm-inference-service-cf47448b/tokenizer-clean-path-test-kserve-5b769465b9-4cdcj"
Sep 17 18:05:07.248555 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:05:07.248520    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/b6f4ed78-4222-460f-bf36-ad4ff96eb150-home\") pod \"tokenizer-clean-path-test-kserve-5b769465b9-4cdcj\" (UID: \"b6f4ed78-4222-460f-bf36-ad4ff96eb150\") " pod="e2e-test-llm-inference-service-cf47448b/tokenizer-clean-path-test-kserve-5b769465b9-4cdcj"
Sep 17 18:05:07.248555 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:05:07.248534    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/b6f4ed78-4222-460f-bf36-ad4ff96eb150-kserve-provision-location\") pod \"tokenizer-clean-path-test-kserve-5b769465b9-4cdcj\" (UID: \"b6f4ed78-4222-460f-bf36-ad4ff96eb150\") " pod="e2e-test-llm-inference-service-cf47448b/tokenizer-clean-path-test-kserve-5b769465b9-4cdcj"
Sep 17 18:05:07.248759 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:05:07.248634    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/b6f4ed78-4222-460f-bf36-ad4ff96eb150-model-cache\") pod \"tokenizer-clean-path-test-kserve-5b769465b9-4cdcj\" (UID: \"b6f4ed78-4222-460f-bf36-ad4ff96eb150\") " pod="e2e-test-llm-inference-service-cf47448b/tokenizer-clean-path-test-kserve-5b769465b9-4cdcj"
Sep 17 18:05:07.350020 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:05:07.349970    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/b6f4ed78-4222-460f-bf36-ad4ff96eb150-tls-certs\") pod \"tokenizer-clean-path-test-kserve-5b769465b9-4cdcj\" (UID: \"b6f4ed78-4222-460f-bf36-ad4ff96eb150\") " pod="e2e-test-llm-inference-service-cf47448b/tokenizer-clean-path-test-kserve-5b769465b9-4cdcj"
Sep 17 18:05:07.350236 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:05:07.350037    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/b6f4ed78-4222-460f-bf36-ad4ff96eb150-dshm\") pod \"tokenizer-clean-path-test-kserve-5b769465b9-4cdcj\" (UID: \"b6f4ed78-4222-460f-bf36-ad4ff96eb150\") " pod="e2e-test-llm-inference-service-cf47448b/tokenizer-clean-path-test-kserve-5b769465b9-4cdcj"
Sep 17 18:05:07.350236 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:05:07.350069    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cabundle-cert\" (UniqueName: \"kubernetes.io/configmap/b6f4ed78-4222-460f-bf36-ad4ff96eb150-cabundle-cert\") pod \"tokenizer-clean-path-test-kserve-5b769465b9-4cdcj\" (UID: \"b6f4ed78-4222-460f-bf36-ad4ff96eb150\") " pod="e2e-test-llm-inference-service-cf47448b/tokenizer-clean-path-test-kserve-5b769465b9-4cdcj"
Sep 17 18:05:07.350236 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:05:07.350099    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/b6f4ed78-4222-460f-bf36-ad4ff96eb150-home\") pod \"tokenizer-clean-path-test-kserve-5b769465b9-4cdcj\" (UID: \"b6f4ed78-4222-460f-bf36-ad4ff96eb150\") " pod="e2e-test-llm-inference-service-cf47448b/tokenizer-clean-path-test-kserve-5b769465b9-4cdcj"
Sep 17 18:05:07.350236 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:05:07.350121    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/b6f4ed78-4222-460f-bf36-ad4ff96eb150-kserve-provision-location\") pod \"tokenizer-clean-path-test-kserve-5b769465b9-4cdcj\" (UID: \"b6f4ed78-4222-460f-bf36-ad4ff96eb150\") " pod="e2e-test-llm-inference-service-cf47448b/tokenizer-clean-path-test-kserve-5b769465b9-4cdcj"
Sep 17 18:05:07.350236 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:05:07.350183    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/b6f4ed78-4222-460f-bf36-ad4ff96eb150-model-cache\") pod \"tokenizer-clean-path-test-kserve-5b769465b9-4cdcj\" (UID: \"b6f4ed78-4222-460f-bf36-ad4ff96eb150\") " pod="e2e-test-llm-inference-service-cf47448b/tokenizer-clean-path-test-kserve-5b769465b9-4cdcj"
Sep 17 18:05:07.350236 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:05:07.350209    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vkzvk\" (UniqueName: \"kubernetes.io/projected/b6f4ed78-4222-460f-bf36-ad4ff96eb150-kube-api-access-vkzvk\") pod \"tokenizer-clean-path-test-kserve-5b769465b9-4cdcj\" (UID: \"b6f4ed78-4222-460f-bf36-ad4ff96eb150\") " pod="e2e-test-llm-inference-service-cf47448b/tokenizer-clean-path-test-kserve-5b769465b9-4cdcj"
Sep 17 18:05:07.350587 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:05:07.350242    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/b6f4ed78-4222-460f-bf36-ad4ff96eb150-tmp-dir\") pod \"tokenizer-clean-path-test-kserve-5b769465b9-4cdcj\" (UID: \"b6f4ed78-4222-460f-bf36-ad4ff96eb150\") " pod="e2e-test-llm-inference-service-cf47448b/tokenizer-clean-path-test-kserve-5b769465b9-4cdcj"
Sep 17 18:05:07.350640 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:05:07.350613    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/b6f4ed78-4222-460f-bf36-ad4ff96eb150-home\") pod \"tokenizer-clean-path-test-kserve-5b769465b9-4cdcj\" (UID: \"b6f4ed78-4222-460f-bf36-ad4ff96eb150\") " pod="e2e-test-llm-inference-service-cf47448b/tokenizer-clean-path-test-kserve-5b769465b9-4cdcj"
Sep 17 18:05:07.350959 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:05:07.350928    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cabundle-cert\" (UniqueName: \"kubernetes.io/configmap/b6f4ed78-4222-460f-bf36-ad4ff96eb150-cabundle-cert\") pod \"tokenizer-clean-path-test-kserve-5b769465b9-4cdcj\" (UID: \"b6f4ed78-4222-460f-bf36-ad4ff96eb150\") " pod="e2e-test-llm-inference-service-cf47448b/tokenizer-clean-path-test-kserve-5b769465b9-4cdcj"
Sep 17 18:05:07.351059 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:05:07.350958    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/b6f4ed78-4222-460f-bf36-ad4ff96eb150-model-cache\") pod \"tokenizer-clean-path-test-kserve-5b769465b9-4cdcj\" (UID: \"b6f4ed78-4222-460f-bf36-ad4ff96eb150\") " pod="e2e-test-llm-inference-service-cf47448b/tokenizer-clean-path-test-kserve-5b769465b9-4cdcj"
Sep 17 18:05:07.351059 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:05:07.350987    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/b6f4ed78-4222-460f-bf36-ad4ff96eb150-tmp-dir\") pod \"tokenizer-clean-path-test-kserve-5b769465b9-4cdcj\" (UID: \"b6f4ed78-4222-460f-bf36-ad4ff96eb150\") " pod="e2e-test-llm-inference-service-cf47448b/tokenizer-clean-path-test-kserve-5b769465b9-4cdcj"
Sep 17 18:05:07.351059 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:05:07.351018    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/b6f4ed78-4222-460f-bf36-ad4ff96eb150-kserve-provision-location\") pod \"tokenizer-clean-path-test-kserve-5b769465b9-4cdcj\" (UID: \"b6f4ed78-4222-460f-bf36-ad4ff96eb150\") " pod="e2e-test-llm-inference-service-cf47448b/tokenizer-clean-path-test-kserve-5b769465b9-4cdcj"
Sep 17 18:05:07.352525 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:05:07.352506    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/b6f4ed78-4222-460f-bf36-ad4ff96eb150-dshm\") pod \"tokenizer-clean-path-test-kserve-5b769465b9-4cdcj\" (UID: \"b6f4ed78-4222-460f-bf36-ad4ff96eb150\") " pod="e2e-test-llm-inference-service-cf47448b/tokenizer-clean-path-test-kserve-5b769465b9-4cdcj"
Sep 17 18:05:07.352676 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:05:07.352658    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/b6f4ed78-4222-460f-bf36-ad4ff96eb150-tls-certs\") pod \"tokenizer-clean-path-test-kserve-5b769465b9-4cdcj\" (UID: \"b6f4ed78-4222-460f-bf36-ad4ff96eb150\") " pod="e2e-test-llm-inference-service-cf47448b/tokenizer-clean-path-test-kserve-5b769465b9-4cdcj"
Sep 17 18:05:07.359268 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:05:07.359246    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-vkzvk\" (UniqueName: \"kubernetes.io/projected/b6f4ed78-4222-460f-bf36-ad4ff96eb150-kube-api-access-vkzvk\") pod \"tokenizer-clean-path-test-kserve-5b769465b9-4cdcj\" (UID: \"b6f4ed78-4222-460f-bf36-ad4ff96eb150\") " pod="e2e-test-llm-inference-service-cf47448b/tokenizer-clean-path-test-kserve-5b769465b9-4cdcj"
Sep 17 18:05:07.495006 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:05:07.494919    2575 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["e2e-test-llm-inference-service-cf47448b/tokenizer-clean-path-test-tokenizer-579dc4b99b-22sl5"]
Sep 17 18:05:07.498425 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:05:07.498399    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-test-llm-inference-service-cf47448b/tokenizer-clean-path-test-tokenizer-579dc4b99b-22sl5"
Sep 17 18:05:07.512739 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:05:07.512706    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["e2e-test-llm-inference-service-cf47448b/tokenizer-clean-path-test-tokenizer-579dc4b99b-22sl5"]
Sep 17 18:05:07.518206 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:05:07.518181    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-test-llm-inference-service-cf47448b/tokenizer-clean-path-test-kserve-5b769465b9-4cdcj"
Sep 17 18:05:07.551955 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:05:07.551914    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/e26b7d5e-3d4f-48db-a066-cd260746c32a-kserve-provision-location\") pod \"tokenizer-clean-path-test-tokenizer-579dc4b99b-22sl5\" (UID: \"e26b7d5e-3d4f-48db-a066-cd260746c32a\") " pod="e2e-test-llm-inference-service-cf47448b/tokenizer-clean-path-test-tokenizer-579dc4b99b-22sl5"
Sep 17 18:05:07.552164 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:05:07.551972    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/e26b7d5e-3d4f-48db-a066-cd260746c32a-tls-certs\") pod \"tokenizer-clean-path-test-tokenizer-579dc4b99b-22sl5\" (UID: \"e26b7d5e-3d4f-48db-a066-cd260746c32a\") " pod="e2e-test-llm-inference-service-cf47448b/tokenizer-clean-path-test-tokenizer-579dc4b99b-22sl5"
Sep 17 18:05:07.552164 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:05:07.552090    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/e26b7d5e-3d4f-48db-a066-cd260746c32a-tmp-dir\") pod \"tokenizer-clean-path-test-tokenizer-579dc4b99b-22sl5\" (UID: \"e26b7d5e-3d4f-48db-a066-cd260746c32a\") " pod="e2e-test-llm-inference-service-cf47448b/tokenizer-clean-path-test-tokenizer-579dc4b99b-22sl5"
Sep 17 18:05:07.552164 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:05:07.552129    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cabundle-cert\" (UniqueName: \"kubernetes.io/configmap/e26b7d5e-3d4f-48db-a066-cd260746c32a-cabundle-cert\") pod \"tokenizer-clean-path-test-tokenizer-579dc4b99b-22sl5\" (UID: \"e26b7d5e-3d4f-48db-a066-cd260746c32a\") " pod="e2e-test-llm-inference-service-cf47448b/tokenizer-clean-path-test-tokenizer-579dc4b99b-22sl5"
Sep 17 18:05:07.653009 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:05:07.652969    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/e26b7d5e-3d4f-48db-a066-cd260746c32a-kserve-provision-location\") pod \"tokenizer-clean-path-test-tokenizer-579dc4b99b-22sl5\" (UID: \"e26b7d5e-3d4f-48db-a066-cd260746c32a\") " pod="e2e-test-llm-inference-service-cf47448b/tokenizer-clean-path-test-tokenizer-579dc4b99b-22sl5"
Sep 17 18:05:07.653170 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:05:07.653022    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/e26b7d5e-3d4f-48db-a066-cd260746c32a-tls-certs\") pod \"tokenizer-clean-path-test-tokenizer-579dc4b99b-22sl5\" (UID: \"e26b7d5e-3d4f-48db-a066-cd260746c32a\") " pod="e2e-test-llm-inference-service-cf47448b/tokenizer-clean-path-test-tokenizer-579dc4b99b-22sl5"
Sep 17 18:05:07.653170 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:05:07.653069    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/e26b7d5e-3d4f-48db-a066-cd260746c32a-tmp-dir\") pod \"tokenizer-clean-path-test-tokenizer-579dc4b99b-22sl5\" (UID: \"e26b7d5e-3d4f-48db-a066-cd260746c32a\") " pod="e2e-test-llm-inference-service-cf47448b/tokenizer-clean-path-test-tokenizer-579dc4b99b-22sl5"
Sep 17 18:05:07.653170 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:05:07.653090    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cabundle-cert\" (UniqueName: \"kubernetes.io/configmap/e26b7d5e-3d4f-48db-a066-cd260746c32a-cabundle-cert\") pod \"tokenizer-clean-path-test-tokenizer-579dc4b99b-22sl5\" (UID: \"e26b7d5e-3d4f-48db-a066-cd260746c32a\") " pod="e2e-test-llm-inference-service-cf47448b/tokenizer-clean-path-test-tokenizer-579dc4b99b-22sl5"
Sep 17 18:05:07.653498 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:05:07.653469    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/e26b7d5e-3d4f-48db-a066-cd260746c32a-kserve-provision-location\") pod \"tokenizer-clean-path-test-tokenizer-579dc4b99b-22sl5\" (UID: \"e26b7d5e-3d4f-48db-a066-cd260746c32a\") " pod="e2e-test-llm-inference-service-cf47448b/tokenizer-clean-path-test-tokenizer-579dc4b99b-22sl5"
Sep 17 18:05:07.653556 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:05:07.653484    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/e26b7d5e-3d4f-48db-a066-cd260746c32a-tmp-dir\") pod \"tokenizer-clean-path-test-tokenizer-579dc4b99b-22sl5\" (UID: \"e26b7d5e-3d4f-48db-a066-cd260746c32a\") " pod="e2e-test-llm-inference-service-cf47448b/tokenizer-clean-path-test-tokenizer-579dc4b99b-22sl5"
Sep 17 18:05:07.653725 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:05:07.653707    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cabundle-cert\" (UniqueName: \"kubernetes.io/configmap/e26b7d5e-3d4f-48db-a066-cd260746c32a-cabundle-cert\") pod \"tokenizer-clean-path-test-tokenizer-579dc4b99b-22sl5\" (UID: \"e26b7d5e-3d4f-48db-a066-cd260746c32a\") " pod="e2e-test-llm-inference-service-cf47448b/tokenizer-clean-path-test-tokenizer-579dc4b99b-22sl5"
Sep 17 18:05:07.655608 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:05:07.655589    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/e26b7d5e-3d4f-48db-a066-cd260746c32a-tls-certs\") pod \"tokenizer-clean-path-test-tokenizer-579dc4b99b-22sl5\" (UID: \"e26b7d5e-3d4f-48db-a066-cd260746c32a\") " pod="e2e-test-llm-inference-service-cf47448b/tokenizer-clean-path-test-tokenizer-579dc4b99b-22sl5"
Sep 17 18:05:07.775499 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:05:07.775388    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-djvzt_1d07e68d-9943-420f-a6a0-5f477792fabd/konnectivity-agent/0.log" limit=1024
Sep 17 18:05:07.806092 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:05:07.806063    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["e2e-test-llm-inference-service-cf47448b/tokenizer-clean-path-test-kserve-5b769465b9-4cdcj"]
Sep 17 18:05:07.807818 ip-10-0-143-74 kubenswrapper[2575]: W0917 18:05:07.807796    2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb6f4ed78_4222_460f_bf36_ad4ff96eb150.slice/crio-943fdb650e1ddef0a0b12cfcc20155c587112c6ef868006a0d40d1ad8add8be5 WatchSource:0}: Error finding container 943fdb650e1ddef0a0b12cfcc20155c587112c6ef868006a0d40d1ad8add8be5: Status 404 returned error can't find the container with id 943fdb650e1ddef0a0b12cfcc20155c587112c6ef868006a0d40d1ad8add8be5
Sep 17 18:05:07.809855 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:05:07.809834    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-test-llm-inference-service-cf47448b/tokenizer-clean-path-test-tokenizer-579dc4b99b-22sl5"
Sep 17 18:05:07.833920 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:05:07.833891    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-test-llm-inference-service-cf47448b/tokenizer-clean-path-test-kserve-5b769465b9-4cdcj" event={"ID":"b6f4ed78-4222-460f-bf36-ad4ff96eb150","Type":"ContainerStarted","Data":"943fdb650e1ddef0a0b12cfcc20155c587112c6ef868006a0d40d1ad8add8be5"}
Sep 17 18:05:07.929876 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:05:07.929838    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-djvzt_1d07e68d-9943-420f-a6a0-5f477792fabd/konnectivity-agent/0.log" limit=1024
Sep 17 18:05:08.098029 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:05:08.097999    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["e2e-test-llm-inference-service-cf47448b/tokenizer-clean-path-test-tokenizer-579dc4b99b-22sl5"]
Sep 17 18:05:08.099519 ip-10-0-143-74 kubenswrapper[2575]: W0917 18:05:08.099492    2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode26b7d5e_3d4f_48db_a066_cd260746c32a.slice/crio-30485a9b43507fca940c627004e86c6f02d75e4438778f764fdbdb1f54491ce6 WatchSource:0}: Error finding container 30485a9b43507fca940c627004e86c6f02d75e4438778f764fdbdb1f54491ce6: Status 404 returned error can't find the container with id 30485a9b43507fca940c627004e86c6f02d75e4438778f764fdbdb1f54491ce6
Sep 17 18:05:08.843180 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:05:08.843138    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-test-llm-inference-service-cf47448b/tokenizer-clean-path-test-kserve-5b769465b9-4cdcj" event={"ID":"b6f4ed78-4222-460f-bf36-ad4ff96eb150","Type":"ContainerStarted","Data":"30c9cb90bbb21ed35bf8f3563c04de2e735e750edc62c53e0ed67d85566c91c6"}
Sep 17 18:05:08.845934 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:05:08.845901    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-test-llm-inference-service-cf47448b/tokenizer-clean-path-test-tokenizer-579dc4b99b-22sl5" event={"ID":"e26b7d5e-3d4f-48db-a066-cd260746c32a","Type":"ContainerStarted","Data":"5d2da5ccaaac351fb5adf20e7a3da495999b557576c3b48a291c9df0507d7324"}
Sep 17 18:05:08.846055 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:05:08.845957    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-test-llm-inference-service-cf47448b/tokenizer-clean-path-test-tokenizer-579dc4b99b-22sl5" event={"ID":"e26b7d5e-3d4f-48db-a066-cd260746c32a","Type":"ContainerStarted","Data":"30485a9b43507fca940c627004e86c6f02d75e4438778f764fdbdb1f54491ce6"}
Sep 17 18:05:10.692168 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:05:10.692132    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-djvzt_1d07e68d-9943-420f-a6a0-5f477792fabd/konnectivity-agent/0.log" limit=1024
Sep 17 18:05:10.778173 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:05:10.778088    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-djvzt_1d07e68d-9943-420f-a6a0-5f477792fabd/konnectivity-agent/0.log" limit=1024
Sep 17 18:05:10.862921 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:05:10.862887    2575 generic.go:356] "Generic (PLEG): container finished" podID="b6f4ed78-4222-460f-bf36-ad4ff96eb150" containerID="30c9cb90bbb21ed35bf8f3563c04de2e735e750edc62c53e0ed67d85566c91c6" exitCode=0
Sep 17 18:05:10.863090 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:05:10.862962    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-test-llm-inference-service-cf47448b/tokenizer-clean-path-test-kserve-5b769465b9-4cdcj" event={"ID":"b6f4ed78-4222-460f-bf36-ad4ff96eb150","Type":"ContainerDied","Data":"30c9cb90bbb21ed35bf8f3563c04de2e735e750edc62c53e0ed67d85566c91c6"}
Sep 17 18:05:10.864635 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:05:10.864615    2575 generic.go:356] "Generic (PLEG): container finished" podID="e26b7d5e-3d4f-48db-a066-cd260746c32a" containerID="5d2da5ccaaac351fb5adf20e7a3da495999b557576c3b48a291c9df0507d7324" exitCode=0
Sep 17 18:05:10.864751 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:05:10.864685    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-test-llm-inference-service-cf47448b/tokenizer-clean-path-test-tokenizer-579dc4b99b-22sl5" event={"ID":"e26b7d5e-3d4f-48db-a066-cd260746c32a","Type":"ContainerDied","Data":"5d2da5ccaaac351fb5adf20e7a3da495999b557576c3b48a291c9df0507d7324"}
Sep 17 18:05:11.871591 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:05:11.871556    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-test-llm-inference-service-cf47448b/tokenizer-clean-path-test-kserve-5b769465b9-4cdcj" event={"ID":"b6f4ed78-4222-460f-bf36-ad4ff96eb150","Type":"ContainerStarted","Data":"cadd029afc7ee36dcdd930abdca4926a02b2a41d42856e8c326cf2fb79bd9de1"}
Sep 17 18:05:11.897359 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:05:11.897304    2575 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="e2e-test-llm-inference-service-cf47448b/tokenizer-clean-path-test-kserve-5b769465b9-4cdcj" podStartSLOduration=2.26578357 podStartE2EDuration="4.897285678s" totalImagesPullingTime="0s" totalInitContainerRuntime="2.631502108s" isStatefulPod=false podCreationTimestamp="2026-09-17 18:05:07 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-17 18:05:11.896629005 +0000 UTC m=+1831.526958982" watchObservedRunningTime="2026-09-17 18:05:11.897285678 +0000 UTC m=+1831.527615654"
Sep 17 18:05:12.518791 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:05:12.518668    2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-test-llm-inference-service-cf47448b/tokenizer-clean-path-test-kserve-5b769465b9-4cdcj"
Sep 17 18:05:17.518939 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:05:17.518899    2575 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="e2e-test-llm-inference-service-cf47448b/tokenizer-clean-path-test-kserve-5b769465b9-4cdcj"
Sep 17 18:05:17.536189 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:05:17.536119    2575 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="e2e-test-llm-inference-service-cf47448b/tokenizer-clean-path-test-kserve-5b769465b9-4cdcj"
Sep 17 18:05:17.944457 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:05:17.944407    2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-test-llm-inference-service-cf47448b/tokenizer-clean-path-test-kserve-5b769465b9-4cdcj"
Sep 17 18:05:21.592979 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:05:21.592938    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-djvzt_1d07e68d-9943-420f-a6a0-5f477792fabd/konnectivity-agent/0.log" limit=1024
Sep 17 18:05:21.689121 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:05:21.689046    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-djvzt_1d07e68d-9943-420f-a6a0-5f477792fabd/konnectivity-agent/0.log" limit=1024
Sep 17 18:05:28.273363 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:05:28.273294    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-djvzt_1d07e68d-9943-420f-a6a0-5f477792fabd/konnectivity-agent/0.log" limit=1024
Sep 17 18:05:34.290329 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:05:34.290289    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-djvzt_1d07e68d-9943-420f-a6a0-5f477792fabd/konnectivity-agent/0.log" limit=1024
Sep 17 18:05:34.390562 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:05:34.390530    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-djvzt_1d07e68d-9943-420f-a6a0-5f477792fabd/konnectivity-agent/0.log" limit=1024
Sep 17 18:05:35.669789 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:05:35.669750    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-djvzt_1d07e68d-9943-420f-a6a0-5f477792fabd/konnectivity-agent/0.log" limit=1024
Sep 17 18:05:38.232975 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:05:38.232941    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-djvzt_1d07e68d-9943-420f-a6a0-5f477792fabd/konnectivity-agent/0.log" limit=1024
Sep 17 18:05:58.189904 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:05:58.189864    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-test-llm-inference-service-cf47448b/tokenizer-clean-path-test-tokenizer-579dc4b99b-22sl5" event={"ID":"e26b7d5e-3d4f-48db-a066-cd260746c32a","Type":"ContainerStarted","Data":"091649526fc3c9db00309799fa86e4fd03e033962940b1d39d069affd9560e8b"}
Sep 17 18:05:58.190312 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:05:58.190064    2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-test-llm-inference-service-cf47448b/tokenizer-clean-path-test-tokenizer-579dc4b99b-22sl5"
Sep 17 18:05:58.211923 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:05:58.211851    2575 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="e2e-test-llm-inference-service-cf47448b/tokenizer-clean-path-test-tokenizer-579dc4b99b-22sl5" podStartSLOduration=1.947369648 podStartE2EDuration="51.211828218s" totalImagesPullingTime="46.62687546s" totalInitContainerRuntime="2.63758311s" isStatefulPod=false podCreationTimestamp="2026-09-17 18:05:07 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-17 18:05:58.20888852 +0000 UTC m=+1877.839218518" watchObservedRunningTime="2026-09-17 18:05:58.211828218 +0000 UTC m=+1877.842158196"
Sep 17 18:06:12.161773 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:06:12.161726    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-djvzt_1d07e68d-9943-420f-a6a0-5f477792fabd/konnectivity-agent/0.log" limit=1024
Sep 17 18:06:12.452531 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:06:12.452424    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-djvzt_1d07e68d-9943-420f-a6a0-5f477792fabd/konnectivity-agent/0.log" limit=1024
Sep 17 18:06:22.345780 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:06:22.345744    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-djvzt_1d07e68d-9943-420f-a6a0-5f477792fabd/konnectivity-agent/0.log" limit=1024
Sep 17 18:06:29.209285 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:06:29.209251    2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-test-llm-inference-service-cf47448b/tokenizer-clean-path-test-tokenizer-579dc4b99b-22sl5"
Sep 17 18:06:31.117870 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:06:31.117833    2575 log.go:25] "Finished parsing log file" path="/var/log/pods/e2e-test-llm-inference-service-cf47448b_tokenizer-clean-path-test-tokenizer-579dc4b99b-22sl5_e26b7d5e-3d4f-48db-a066-cd260746c32a/main/0.log"
Sep 17 18:06:31.377825 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:06:31.377747    2575 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["e2e-test-llm-inference-service-cf47448b/tokenizer-clean-path-test-tokenizer-579dc4b99b-22sl5"]
Sep 17 18:06:31.377976 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:06:31.377949    2575 kuberuntime_container.go:873] "Killing container with a grace period" pod="e2e-test-llm-inference-service-cf47448b/tokenizer-clean-path-test-tokenizer-579dc4b99b-22sl5" podUID="e26b7d5e-3d4f-48db-a066-cd260746c32a" containerName="main" containerID="cri-o://091649526fc3c9db00309799fa86e4fd03e033962940b1d39d069affd9560e8b" gracePeriod=30
Sep 17 18:06:31.389260 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:06:31.389226    2575 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["e2e-test-llm-inference-service-cf47448b/tokenizer-clean-path-test-kserve-5b769465b9-4cdcj"]
Sep 17 18:06:31.399709 ip-10-0-143-74 kubenswrapper[2575]: E0917 18:06:31.399674    2575 log.go:32] "ExecSync cmd from runtime service failed" err=<
Sep 17 18:06:31.399709 ip-10-0-143-74 kubenswrapper[2575]:         rpc error: code = Unknown desc = command error: executable file `/bin/sleep` not found: No such file or directory
Sep 17 18:06:31.399709 ip-10-0-143-74 kubenswrapper[2575]:         , stdout: , stderr: , exit code -1
Sep 17 18:06:31.399709 ip-10-0-143-74 kubenswrapper[2575]:  > containerID="cadd029afc7ee36dcdd930abdca4926a02b2a41d42856e8c326cf2fb79bd9de1" cmd=["/bin/sleep","15"]
Sep 17 18:06:31.400002 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:06:31.399727    2575 handlers.go:82] "Exec lifecycle hook for Container in Pod failed" execCommand=["/bin/sleep","15"] containerName="main" pod="e2e-test-llm-inference-service-cf47448b/tokenizer-clean-path-test-kserve-5b769465b9-4cdcj" message="" err=<
Sep 17 18:06:31.400002 ip-10-0-143-74 kubenswrapper[2575]:         rpc error: code = Unknown desc = command error: executable file `/bin/sleep` not found: No such file or directory
Sep 17 18:06:31.400002 ip-10-0-143-74 kubenswrapper[2575]:         , stdout: , stderr: , exit code -1
Sep 17 18:06:31.400002 ip-10-0-143-74 kubenswrapper[2575]:  >
Sep 17 18:06:31.400002 ip-10-0-143-74 kubenswrapper[2575]: E0917 18:06:31.399740    2575 kuberuntime_container.go:755] "PreStop hook failed" err=<
Sep 17 18:06:31.400002 ip-10-0-143-74 kubenswrapper[2575]:         rpc error: code = Unknown desc = command error: executable file `/bin/sleep` not found: No such file or directory
Sep 17 18:06:31.400002 ip-10-0-143-74 kubenswrapper[2575]:         , stdout: , stderr: , exit code -1
Sep 17 18:06:31.400002 ip-10-0-143-74 kubenswrapper[2575]:  > pod="e2e-test-llm-inference-service-cf47448b/tokenizer-clean-path-test-kserve-5b769465b9-4cdcj" podUID="b6f4ed78-4222-460f-bf36-ad4ff96eb150" containerName="main" containerID="cri-o://cadd029afc7ee36dcdd930abdca4926a02b2a41d42856e8c326cf2fb79bd9de1"
Sep 17 18:06:31.400002 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:06:31.399784    2575 kuberuntime_container.go:873] "Killing container with a grace period" pod="e2e-test-llm-inference-service-cf47448b/tokenizer-clean-path-test-kserve-5b769465b9-4cdcj" podUID="b6f4ed78-4222-460f-bf36-ad4ff96eb150" containerName="main" containerID="cri-o://cadd029afc7ee36dcdd930abdca4926a02b2a41d42856e8c326cf2fb79bd9de1" gracePeriod=60
Sep 17 18:06:31.413708 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:06:31.413680    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-djvzt_1d07e68d-9943-420f-a6a0-5f477792fabd/konnectivity-agent/0.log" limit=1024
Sep 17 18:06:31.515727 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:06:31.515695    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-djvzt_1d07e68d-9943-420f-a6a0-5f477792fabd/konnectivity-agent/0.log" limit=1024
Sep 17 18:06:31.922068 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:06:31.922031    2575 prober.go:120] "Probe failed" probeType="Readiness" pod="e2e-test-llm-inference-service-cf47448b/tokenizer-clean-path-test-kserve-5b769465b9-4cdcj" podUID="b6f4ed78-4222-460f-bf36-ad4ff96eb150" containerName="main" probeResult="failure" output="Get \"https://10.133.0.50:8000/health\": dial tcp 10.133.0.50:8000: connect: connection refused"
Sep 17 18:06:32.412412 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:06:32.412380    2575 generic.go:356] "Generic (PLEG): container finished" podID="b6f4ed78-4222-460f-bf36-ad4ff96eb150" containerID="cadd029afc7ee36dcdd930abdca4926a02b2a41d42856e8c326cf2fb79bd9de1" exitCode=0
Sep 17 18:06:32.412846 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:06:32.412471    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-test-llm-inference-service-cf47448b/tokenizer-clean-path-test-kserve-5b769465b9-4cdcj" event={"ID":"b6f4ed78-4222-460f-bf36-ad4ff96eb150","Type":"ContainerDied","Data":"cadd029afc7ee36dcdd930abdca4926a02b2a41d42856e8c326cf2fb79bd9de1"}
Sep 17 18:06:32.414489 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:06:32.414398    2575 generic.go:356] "Generic (PLEG): container finished" podID="e26b7d5e-3d4f-48db-a066-cd260746c32a" containerID="091649526fc3c9db00309799fa86e4fd03e033962940b1d39d069affd9560e8b" exitCode=0
Sep 17 18:06:32.414586 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:06:32.414485    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-test-llm-inference-service-cf47448b/tokenizer-clean-path-test-tokenizer-579dc4b99b-22sl5" event={"ID":"e26b7d5e-3d4f-48db-a066-cd260746c32a","Type":"ContainerDied","Data":"091649526fc3c9db00309799fa86e4fd03e033962940b1d39d069affd9560e8b"}
Sep 17 18:06:32.599899 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:06:32.599865    2575 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-test-llm-inference-service-cf47448b/tokenizer-clean-path-test-tokenizer-579dc4b99b-22sl5"
Sep 17 18:06:32.677157 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:06:32.677063    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/e26b7d5e-3d4f-48db-a066-cd260746c32a-tls-certs\") pod \"e26b7d5e-3d4f-48db-a066-cd260746c32a\" (UID: \"e26b7d5e-3d4f-48db-a066-cd260746c32a\") "
Sep 17 18:06:32.677157 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:06:32.677108    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/e26b7d5e-3d4f-48db-a066-cd260746c32a-kserve-provision-location\") pod \"e26b7d5e-3d4f-48db-a066-cd260746c32a\" (UID: \"e26b7d5e-3d4f-48db-a066-cd260746c32a\") "
Sep 17 18:06:32.677157 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:06:32.677134    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"cabundle-cert\" (UniqueName: \"kubernetes.io/configmap/e26b7d5e-3d4f-48db-a066-cd260746c32a-cabundle-cert\") pod \"e26b7d5e-3d4f-48db-a066-cd260746c32a\" (UID: \"e26b7d5e-3d4f-48db-a066-cd260746c32a\") "
Sep 17 18:06:32.677157 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:06:32.677156    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/e26b7d5e-3d4f-48db-a066-cd260746c32a-tmp-dir\") pod \"e26b7d5e-3d4f-48db-a066-cd260746c32a\" (UID: \"e26b7d5e-3d4f-48db-a066-cd260746c32a\") "
Sep 17 18:06:32.677550 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:06:32.677523    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/e26b7d5e-3d4f-48db-a066-cd260746c32a-cabundle-cert" (OuterVolumeSpecName: "cabundle-cert") pod "e26b7d5e-3d4f-48db-a066-cd260746c32a" (UID: "e26b7d5e-3d4f-48db-a066-cd260746c32a"). InnerVolumeSpecName "cabundle-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 17 18:06:32.677634 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:06:32.677613    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e26b7d5e-3d4f-48db-a066-cd260746c32a-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "e26b7d5e-3d4f-48db-a066-cd260746c32a" (UID: "e26b7d5e-3d4f-48db-a066-cd260746c32a"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 17 18:06:32.679993 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:06:32.679968    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/e26b7d5e-3d4f-48db-a066-cd260746c32a-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "e26b7d5e-3d4f-48db-a066-cd260746c32a" (UID: "e26b7d5e-3d4f-48db-a066-cd260746c32a"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 17 18:06:32.702803 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:06:32.702764    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e26b7d5e-3d4f-48db-a066-cd260746c32a-kserve-provision-location" (OuterVolumeSpecName: "kserve-provision-location") pod "e26b7d5e-3d4f-48db-a066-cd260746c32a" (UID: "e26b7d5e-3d4f-48db-a066-cd260746c32a"). InnerVolumeSpecName "kserve-provision-location". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 17 18:06:32.777886 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:06:32.777849    2575 reconciler_common.go:299] "Volume detached for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/e26b7d5e-3d4f-48db-a066-cd260746c32a-kserve-provision-location\") on node \"ip-10-0-143-74.ec2.internal\" DevicePath \"\""
Sep 17 18:06:32.777886 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:06:32.777882    2575 reconciler_common.go:299] "Volume detached for volume \"cabundle-cert\" (UniqueName: \"kubernetes.io/configmap/e26b7d5e-3d4f-48db-a066-cd260746c32a-cabundle-cert\") on node \"ip-10-0-143-74.ec2.internal\" DevicePath \"\""
Sep 17 18:06:32.778147 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:06:32.777916    2575 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/e26b7d5e-3d4f-48db-a066-cd260746c32a-tmp-dir\") on node \"ip-10-0-143-74.ec2.internal\" DevicePath \"\""
Sep 17 18:06:32.778147 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:06:32.777930    2575 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/e26b7d5e-3d4f-48db-a066-cd260746c32a-tls-certs\") on node \"ip-10-0-143-74.ec2.internal\" DevicePath \"\""
Sep 17 18:06:32.899757 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:06:32.899724    2575 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-test-llm-inference-service-cf47448b/tokenizer-clean-path-test-kserve-5b769465b9-4cdcj"
Sep 17 18:06:32.979791 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:06:32.979766    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/b6f4ed78-4222-460f-bf36-ad4ff96eb150-tls-certs\") pod \"b6f4ed78-4222-460f-bf36-ad4ff96eb150\" (UID: \"b6f4ed78-4222-460f-bf36-ad4ff96eb150\") "
Sep 17 18:06:32.979914 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:06:32.979815    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/b6f4ed78-4222-460f-bf36-ad4ff96eb150-tmp-dir\") pod \"b6f4ed78-4222-460f-bf36-ad4ff96eb150\" (UID: \"b6f4ed78-4222-460f-bf36-ad4ff96eb150\") "
Sep 17 18:06:32.979914 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:06:32.979831    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/b6f4ed78-4222-460f-bf36-ad4ff96eb150-home\") pod \"b6f4ed78-4222-460f-bf36-ad4ff96eb150\" (UID: \"b6f4ed78-4222-460f-bf36-ad4ff96eb150\") "
Sep 17 18:06:32.979914 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:06:32.979857    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-vkzvk\" (UniqueName: \"kubernetes.io/projected/b6f4ed78-4222-460f-bf36-ad4ff96eb150-kube-api-access-vkzvk\") pod \"b6f4ed78-4222-460f-bf36-ad4ff96eb150\" (UID: \"b6f4ed78-4222-460f-bf36-ad4ff96eb150\") "
Sep 17 18:06:32.980072 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:06:32.979979    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"cabundle-cert\" (UniqueName: \"kubernetes.io/configmap/b6f4ed78-4222-460f-bf36-ad4ff96eb150-cabundle-cert\") pod \"b6f4ed78-4222-460f-bf36-ad4ff96eb150\" (UID: \"b6f4ed78-4222-460f-bf36-ad4ff96eb150\") "
Sep 17 18:06:32.980130 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:06:32.980073    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/b6f4ed78-4222-460f-bf36-ad4ff96eb150-kserve-provision-location\") pod \"b6f4ed78-4222-460f-bf36-ad4ff96eb150\" (UID: \"b6f4ed78-4222-460f-bf36-ad4ff96eb150\") "
Sep 17 18:06:32.980130 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:06:32.980102    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/b6f4ed78-4222-460f-bf36-ad4ff96eb150-model-cache\") pod \"b6f4ed78-4222-460f-bf36-ad4ff96eb150\" (UID: \"b6f4ed78-4222-460f-bf36-ad4ff96eb150\") "
Sep 17 18:06:32.980130 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:06:32.980103    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b6f4ed78-4222-460f-bf36-ad4ff96eb150-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "b6f4ed78-4222-460f-bf36-ad4ff96eb150" (UID: "b6f4ed78-4222-460f-bf36-ad4ff96eb150"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 17 18:06:32.980276 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:06:32.980125    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b6f4ed78-4222-460f-bf36-ad4ff96eb150-home" (OuterVolumeSpecName: "home") pod "b6f4ed78-4222-460f-bf36-ad4ff96eb150" (UID: "b6f4ed78-4222-460f-bf36-ad4ff96eb150"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 17 18:06:32.980276 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:06:32.980166    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/b6f4ed78-4222-460f-bf36-ad4ff96eb150-dshm\") pod \"b6f4ed78-4222-460f-bf36-ad4ff96eb150\" (UID: \"b6f4ed78-4222-460f-bf36-ad4ff96eb150\") "
Sep 17 18:06:32.980688 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:06:32.980389    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/b6f4ed78-4222-460f-bf36-ad4ff96eb150-cabundle-cert" (OuterVolumeSpecName: "cabundle-cert") pod "b6f4ed78-4222-460f-bf36-ad4ff96eb150" (UID: "b6f4ed78-4222-460f-bf36-ad4ff96eb150"). InnerVolumeSpecName "cabundle-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 17 18:06:32.980688 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:06:32.980409    2575 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/b6f4ed78-4222-460f-bf36-ad4ff96eb150-tmp-dir\") on node \"ip-10-0-143-74.ec2.internal\" DevicePath \"\""
Sep 17 18:06:32.980688 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:06:32.980426    2575 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/b6f4ed78-4222-460f-bf36-ad4ff96eb150-home\") on node \"ip-10-0-143-74.ec2.internal\" DevicePath \"\""
Sep 17 18:06:32.980688 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:06:32.980651    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b6f4ed78-4222-460f-bf36-ad4ff96eb150-model-cache" (OuterVolumeSpecName: "model-cache") pod "b6f4ed78-4222-460f-bf36-ad4ff96eb150" (UID: "b6f4ed78-4222-460f-bf36-ad4ff96eb150"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 17 18:06:32.982423 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:06:32.982400    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/b6f4ed78-4222-460f-bf36-ad4ff96eb150-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "b6f4ed78-4222-460f-bf36-ad4ff96eb150" (UID: "b6f4ed78-4222-460f-bf36-ad4ff96eb150"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 17 18:06:32.982548 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:06:32.982428    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/b6f4ed78-4222-460f-bf36-ad4ff96eb150-kube-api-access-vkzvk" (OuterVolumeSpecName: "kube-api-access-vkzvk") pod "b6f4ed78-4222-460f-bf36-ad4ff96eb150" (UID: "b6f4ed78-4222-460f-bf36-ad4ff96eb150"). InnerVolumeSpecName "kube-api-access-vkzvk". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 17 18:06:32.982757 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:06:32.982734    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b6f4ed78-4222-460f-bf36-ad4ff96eb150-dshm" (OuterVolumeSpecName: "dshm") pod "b6f4ed78-4222-460f-bf36-ad4ff96eb150" (UID: "b6f4ed78-4222-460f-bf36-ad4ff96eb150"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 17 18:06:33.003154 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:06:33.003113    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b6f4ed78-4222-460f-bf36-ad4ff96eb150-kserve-provision-location" (OuterVolumeSpecName: "kserve-provision-location") pod "b6f4ed78-4222-460f-bf36-ad4ff96eb150" (UID: "b6f4ed78-4222-460f-bf36-ad4ff96eb150"). InnerVolumeSpecName "kserve-provision-location". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 17 18:06:33.081973 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:06:33.081931    2575 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/b6f4ed78-4222-460f-bf36-ad4ff96eb150-tls-certs\") on node \"ip-10-0-143-74.ec2.internal\" DevicePath \"\""
Sep 17 18:06:33.081973 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:06:33.081975    2575 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-vkzvk\" (UniqueName: \"kubernetes.io/projected/b6f4ed78-4222-460f-bf36-ad4ff96eb150-kube-api-access-vkzvk\") on node \"ip-10-0-143-74.ec2.internal\" DevicePath \"\""
Sep 17 18:06:33.081973 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:06:33.081989    2575 reconciler_common.go:299] "Volume detached for volume \"cabundle-cert\" (UniqueName: \"kubernetes.io/configmap/b6f4ed78-4222-460f-bf36-ad4ff96eb150-cabundle-cert\") on node \"ip-10-0-143-74.ec2.internal\" DevicePath \"\""
Sep 17 18:06:33.082285 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:06:33.082004    2575 reconciler_common.go:299] "Volume detached for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/b6f4ed78-4222-460f-bf36-ad4ff96eb150-kserve-provision-location\") on node \"ip-10-0-143-74.ec2.internal\" DevicePath \"\""
Sep 17 18:06:33.082285 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:06:33.082018    2575 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/b6f4ed78-4222-460f-bf36-ad4ff96eb150-model-cache\") on node \"ip-10-0-143-74.ec2.internal\" DevicePath \"\""
Sep 17 18:06:33.082285 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:06:33.082032    2575 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/b6f4ed78-4222-460f-bf36-ad4ff96eb150-dshm\") on node \"ip-10-0-143-74.ec2.internal\" DevicePath \"\""
Sep 17 18:06:33.421861 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:06:33.421827    2575 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-test-llm-inference-service-cf47448b/tokenizer-clean-path-test-tokenizer-579dc4b99b-22sl5"
Sep 17 18:06:33.421861 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:06:33.421858    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-test-llm-inference-service-cf47448b/tokenizer-clean-path-test-tokenizer-579dc4b99b-22sl5" event={"ID":"e26b7d5e-3d4f-48db-a066-cd260746c32a","Type":"ContainerDied","Data":"30485a9b43507fca940c627004e86c6f02d75e4438778f764fdbdb1f54491ce6"}
Sep 17 18:06:33.422362 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:06:33.421912    2575 scope.go:117] "RemoveContainer" containerID="091649526fc3c9db00309799fa86e4fd03e033962940b1d39d069affd9560e8b"
Sep 17 18:06:33.423874 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:06:33.423841    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-test-llm-inference-service-cf47448b/tokenizer-clean-path-test-kserve-5b769465b9-4cdcj" event={"ID":"b6f4ed78-4222-460f-bf36-ad4ff96eb150","Type":"ContainerDied","Data":"943fdb650e1ddef0a0b12cfcc20155c587112c6ef868006a0d40d1ad8add8be5"}
Sep 17 18:06:33.423989 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:06:33.423934    2575 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-test-llm-inference-service-cf47448b/tokenizer-clean-path-test-kserve-5b769465b9-4cdcj"
Sep 17 18:06:33.431687 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:06:33.431639    2575 scope.go:117] "RemoveContainer" containerID="5d2da5ccaaac351fb5adf20e7a3da495999b557576c3b48a291c9df0507d7324"
Sep 17 18:06:33.440502 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:06:33.440484    2575 scope.go:117] "RemoveContainer" containerID="cadd029afc7ee36dcdd930abdca4926a02b2a41d42856e8c326cf2fb79bd9de1"
Sep 17 18:06:33.442286 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:06:33.442263    2575 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["e2e-test-llm-inference-service-cf47448b/tokenizer-clean-path-test-tokenizer-579dc4b99b-22sl5"]
Sep 17 18:06:33.446049 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:06:33.446029    2575 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["e2e-test-llm-inference-service-cf47448b/tokenizer-clean-path-test-tokenizer-579dc4b99b-22sl5"]
Sep 17 18:06:33.450084 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:06:33.450059    2575 scope.go:117] "RemoveContainer" containerID="30c9cb90bbb21ed35bf8f3563c04de2e735e750edc62c53e0ed67d85566c91c6"
Sep 17 18:06:33.462164 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:06:33.462142    2575 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["e2e-test-llm-inference-service-cf47448b/tokenizer-clean-path-test-kserve-5b769465b9-4cdcj"]
Sep 17 18:06:33.466754 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:06:33.466733    2575 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["e2e-test-llm-inference-service-cf47448b/tokenizer-clean-path-test-kserve-5b769465b9-4cdcj"]
Sep 17 18:06:34.967997 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:06:34.967957    2575 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="b6f4ed78-4222-460f-bf36-ad4ff96eb150" path="/var/lib/kubelet/pods/b6f4ed78-4222-460f-bf36-ad4ff96eb150/volumes"
Sep 17 18:06:34.968542 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:06:34.968457    2575 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="e26b7d5e-3d4f-48db-a066-cd260746c32a" path="/var/lib/kubelet/pods/e26b7d5e-3d4f-48db-a066-cd260746c32a/volumes"
Sep 17 18:06:53.169081 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:06:53.169041    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-djvzt_1d07e68d-9943-420f-a6a0-5f477792fabd/konnectivity-agent/0.log" limit=1024
Sep 17 18:06:53.264466 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:06:53.264410    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-djvzt_1d07e68d-9943-420f-a6a0-5f477792fabd/konnectivity-agent/0.log" limit=1024
Sep 17 18:07:19.300907 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:07:19.300870    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-djvzt_1d07e68d-9943-420f-a6a0-5f477792fabd/konnectivity-agent/0.log" limit=1024
Sep 17 18:07:21.455982 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:07:21.455944    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-djvzt_1d07e68d-9943-420f-a6a0-5f477792fabd/konnectivity-agent/0.log" limit=1024
Sep 17 18:07:21.552574 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:07:21.552538    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-djvzt_1d07e68d-9943-420f-a6a0-5f477792fabd/konnectivity-agent/0.log" limit=1024
Sep 17 18:07:26.520095 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:07:26.520065    2575 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["e2e-test-llm-inference-service-312caf2d/tokenizer-migration-path-test-kserve-85f864cf88-nzrvs"]
Sep 17 18:07:26.520680 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:07:26.520658    2575 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e26b7d5e-3d4f-48db-a066-cd260746c32a" containerName="main"
Sep 17 18:07:26.520680 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:07:26.520681    2575 state_mem.go:107] "Deleted CPUSet assignment" podUID="e26b7d5e-3d4f-48db-a066-cd260746c32a" containerName="main"
Sep 17 18:07:26.520888 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:07:26.520699    2575 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e26b7d5e-3d4f-48db-a066-cd260746c32a" containerName="storage-initializer"
Sep 17 18:07:26.520888 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:07:26.520708    2575 state_mem.go:107] "Deleted CPUSet assignment" podUID="e26b7d5e-3d4f-48db-a066-cd260746c32a" containerName="storage-initializer"
Sep 17 18:07:26.520888 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:07:26.520730    2575 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b6f4ed78-4222-460f-bf36-ad4ff96eb150" containerName="storage-initializer"
Sep 17 18:07:26.520888 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:07:26.520738    2575 state_mem.go:107] "Deleted CPUSet assignment" podUID="b6f4ed78-4222-460f-bf36-ad4ff96eb150" containerName="storage-initializer"
Sep 17 18:07:26.520888 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:07:26.520747    2575 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b6f4ed78-4222-460f-bf36-ad4ff96eb150" containerName="main"
Sep 17 18:07:26.520888 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:07:26.520755    2575 state_mem.go:107] "Deleted CPUSet assignment" podUID="b6f4ed78-4222-460f-bf36-ad4ff96eb150" containerName="main"
Sep 17 18:07:26.520888 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:07:26.520866    2575 memory_manager.go:368] "RemoveStaleState removing state" podUID="b6f4ed78-4222-460f-bf36-ad4ff96eb150" containerName="main"
Sep 17 18:07:26.520888 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:07:26.520879    2575 memory_manager.go:368] "RemoveStaleState removing state" podUID="e26b7d5e-3d4f-48db-a066-cd260746c32a" containerName="main"
Sep 17 18:07:26.524168 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:07:26.524146    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-test-llm-inference-service-312caf2d/tokenizer-migration-path-test-kserve-85f864cf88-nzrvs"
Sep 17 18:07:26.526932 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:07:26.526906    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-test-llm-inference-service-312caf2d\"/\"seaweedfs-s3-creds\""
Sep 17 18:07:26.527089 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:07:26.527059    2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-test-llm-inference-service-312caf2d\"/\"odh-kserve-custom-ca-bundle\""
Sep 17 18:07:26.528218 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:07:26.528191    2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-test-llm-inference-service-312caf2d\"/\"openshift-service-ca.crt\""
Sep 17 18:07:26.528384 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:07:26.528199    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-test-llm-inference-service-312caf2d\"/\"default-dockercfg-fsg6t\""
Sep 17 18:07:26.528501 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:07:26.528217    2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-test-llm-inference-service-312caf2d\"/\"kube-root-ca.crt\""
Sep 17 18:07:26.528501 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:07:26.528226    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-test-llm-inference-service-312caf2d\"/\"tokenizer-migration-path-test-kserve-self-signed-certs\""
Sep 17 18:07:26.538476 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:07:26.538418    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["e2e-test-llm-inference-service-312caf2d/tokenizer-migration-path-test-kserve-85f864cf88-nzrvs"]
Sep 17 18:07:26.563826 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:07:26.563792    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-djvzt_1d07e68d-9943-420f-a6a0-5f477792fabd/konnectivity-agent/0.log" limit=1024
Sep 17 18:07:26.655683 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:07:26.655647    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cabundle-cert\" (UniqueName: \"kubernetes.io/configmap/5757cc18-3321-427d-9177-16c0987bdb2e-cabundle-cert\") pod \"tokenizer-migration-path-test-kserve-85f864cf88-nzrvs\" (UID: \"5757cc18-3321-427d-9177-16c0987bdb2e\") " pod="e2e-test-llm-inference-service-312caf2d/tokenizer-migration-path-test-kserve-85f864cf88-nzrvs"
Sep 17 18:07:26.655877 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:07:26.655688    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/5757cc18-3321-427d-9177-16c0987bdb2e-model-cache\") pod \"tokenizer-migration-path-test-kserve-85f864cf88-nzrvs\" (UID: \"5757cc18-3321-427d-9177-16c0987bdb2e\") " pod="e2e-test-llm-inference-service-312caf2d/tokenizer-migration-path-test-kserve-85f864cf88-nzrvs"
Sep 17 18:07:26.655877 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:07:26.655809    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/5757cc18-3321-427d-9177-16c0987bdb2e-dshm\") pod \"tokenizer-migration-path-test-kserve-85f864cf88-nzrvs\" (UID: \"5757cc18-3321-427d-9177-16c0987bdb2e\") " pod="e2e-test-llm-inference-service-312caf2d/tokenizer-migration-path-test-kserve-85f864cf88-nzrvs"
Sep 17 18:07:26.655877 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:07:26.655859    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/5757cc18-3321-427d-9177-16c0987bdb2e-kserve-provision-location\") pod \"tokenizer-migration-path-test-kserve-85f864cf88-nzrvs\" (UID: \"5757cc18-3321-427d-9177-16c0987bdb2e\") " pod="e2e-test-llm-inference-service-312caf2d/tokenizer-migration-path-test-kserve-85f864cf88-nzrvs"
Sep 17 18:07:26.656056 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:07:26.655896    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/5757cc18-3321-427d-9177-16c0987bdb2e-tmp-dir\") pod \"tokenizer-migration-path-test-kserve-85f864cf88-nzrvs\" (UID: \"5757cc18-3321-427d-9177-16c0987bdb2e\") " pod="e2e-test-llm-inference-service-312caf2d/tokenizer-migration-path-test-kserve-85f864cf88-nzrvs"
Sep 17 18:07:26.656056 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:07:26.655924    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-tw8tj\" (UniqueName: \"kubernetes.io/projected/5757cc18-3321-427d-9177-16c0987bdb2e-kube-api-access-tw8tj\") pod \"tokenizer-migration-path-test-kserve-85f864cf88-nzrvs\" (UID: \"5757cc18-3321-427d-9177-16c0987bdb2e\") " pod="e2e-test-llm-inference-service-312caf2d/tokenizer-migration-path-test-kserve-85f864cf88-nzrvs"
Sep 17 18:07:26.656056 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:07:26.655970    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/5757cc18-3321-427d-9177-16c0987bdb2e-home\") pod \"tokenizer-migration-path-test-kserve-85f864cf88-nzrvs\" (UID: \"5757cc18-3321-427d-9177-16c0987bdb2e\") " pod="e2e-test-llm-inference-service-312caf2d/tokenizer-migration-path-test-kserve-85f864cf88-nzrvs"
Sep 17 18:07:26.656056 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:07:26.656020    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/5757cc18-3321-427d-9177-16c0987bdb2e-tls-certs\") pod \"tokenizer-migration-path-test-kserve-85f864cf88-nzrvs\" (UID: \"5757cc18-3321-427d-9177-16c0987bdb2e\") " pod="e2e-test-llm-inference-service-312caf2d/tokenizer-migration-path-test-kserve-85f864cf88-nzrvs"
Sep 17 18:07:26.757249 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:07:26.757209    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/5757cc18-3321-427d-9177-16c0987bdb2e-dshm\") pod \"tokenizer-migration-path-test-kserve-85f864cf88-nzrvs\" (UID: \"5757cc18-3321-427d-9177-16c0987bdb2e\") " pod="e2e-test-llm-inference-service-312caf2d/tokenizer-migration-path-test-kserve-85f864cf88-nzrvs"
Sep 17 18:07:26.757466 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:07:26.757263    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/5757cc18-3321-427d-9177-16c0987bdb2e-kserve-provision-location\") pod \"tokenizer-migration-path-test-kserve-85f864cf88-nzrvs\" (UID: \"5757cc18-3321-427d-9177-16c0987bdb2e\") " pod="e2e-test-llm-inference-service-312caf2d/tokenizer-migration-path-test-kserve-85f864cf88-nzrvs"
Sep 17 18:07:26.757466 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:07:26.757297    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/5757cc18-3321-427d-9177-16c0987bdb2e-tmp-dir\") pod \"tokenizer-migration-path-test-kserve-85f864cf88-nzrvs\" (UID: \"5757cc18-3321-427d-9177-16c0987bdb2e\") " pod="e2e-test-llm-inference-service-312caf2d/tokenizer-migration-path-test-kserve-85f864cf88-nzrvs"
Sep 17 18:07:26.757466 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:07:26.757313    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tw8tj\" (UniqueName: \"kubernetes.io/projected/5757cc18-3321-427d-9177-16c0987bdb2e-kube-api-access-tw8tj\") pod \"tokenizer-migration-path-test-kserve-85f864cf88-nzrvs\" (UID: \"5757cc18-3321-427d-9177-16c0987bdb2e\") " pod="e2e-test-llm-inference-service-312caf2d/tokenizer-migration-path-test-kserve-85f864cf88-nzrvs"
Sep 17 18:07:26.757466 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:07:26.757342    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/5757cc18-3321-427d-9177-16c0987bdb2e-home\") pod \"tokenizer-migration-path-test-kserve-85f864cf88-nzrvs\" (UID: \"5757cc18-3321-427d-9177-16c0987bdb2e\") " pod="e2e-test-llm-inference-service-312caf2d/tokenizer-migration-path-test-kserve-85f864cf88-nzrvs"
Sep 17 18:07:26.757466 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:07:26.757371    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/5757cc18-3321-427d-9177-16c0987bdb2e-tls-certs\") pod \"tokenizer-migration-path-test-kserve-85f864cf88-nzrvs\" (UID: \"5757cc18-3321-427d-9177-16c0987bdb2e\") " pod="e2e-test-llm-inference-service-312caf2d/tokenizer-migration-path-test-kserve-85f864cf88-nzrvs"
Sep 17 18:07:26.757466 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:07:26.757403    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cabundle-cert\" (UniqueName: \"kubernetes.io/configmap/5757cc18-3321-427d-9177-16c0987bdb2e-cabundle-cert\") pod \"tokenizer-migration-path-test-kserve-85f864cf88-nzrvs\" (UID: \"5757cc18-3321-427d-9177-16c0987bdb2e\") " pod="e2e-test-llm-inference-service-312caf2d/tokenizer-migration-path-test-kserve-85f864cf88-nzrvs"
Sep 17 18:07:26.757790 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:07:26.757576    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/5757cc18-3321-427d-9177-16c0987bdb2e-model-cache\") pod \"tokenizer-migration-path-test-kserve-85f864cf88-nzrvs\" (UID: \"5757cc18-3321-427d-9177-16c0987bdb2e\") " pod="e2e-test-llm-inference-service-312caf2d/tokenizer-migration-path-test-kserve-85f864cf88-nzrvs"
Sep 17 18:07:26.757790 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:07:26.757772    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/5757cc18-3321-427d-9177-16c0987bdb2e-kserve-provision-location\") pod \"tokenizer-migration-path-test-kserve-85f864cf88-nzrvs\" (UID: \"5757cc18-3321-427d-9177-16c0987bdb2e\") " pod="e2e-test-llm-inference-service-312caf2d/tokenizer-migration-path-test-kserve-85f864cf88-nzrvs"
Sep 17 18:07:26.757904 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:07:26.757855    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/5757cc18-3321-427d-9177-16c0987bdb2e-home\") pod \"tokenizer-migration-path-test-kserve-85f864cf88-nzrvs\" (UID: \"5757cc18-3321-427d-9177-16c0987bdb2e\") " pod="e2e-test-llm-inference-service-312caf2d/tokenizer-migration-path-test-kserve-85f864cf88-nzrvs"
Sep 17 18:07:26.758052 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:07:26.758029    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/5757cc18-3321-427d-9177-16c0987bdb2e-model-cache\") pod \"tokenizer-migration-path-test-kserve-85f864cf88-nzrvs\" (UID: \"5757cc18-3321-427d-9177-16c0987bdb2e\") " pod="e2e-test-llm-inference-service-312caf2d/tokenizer-migration-path-test-kserve-85f864cf88-nzrvs"
Sep 17 18:07:26.758108 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:07:26.758039    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/5757cc18-3321-427d-9177-16c0987bdb2e-tmp-dir\") pod \"tokenizer-migration-path-test-kserve-85f864cf88-nzrvs\" (UID: \"5757cc18-3321-427d-9177-16c0987bdb2e\") " pod="e2e-test-llm-inference-service-312caf2d/tokenizer-migration-path-test-kserve-85f864cf88-nzrvs"
Sep 17 18:07:26.758189 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:07:26.758168    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cabundle-cert\" (UniqueName: \"kubernetes.io/configmap/5757cc18-3321-427d-9177-16c0987bdb2e-cabundle-cert\") pod \"tokenizer-migration-path-test-kserve-85f864cf88-nzrvs\" (UID: \"5757cc18-3321-427d-9177-16c0987bdb2e\") " pod="e2e-test-llm-inference-service-312caf2d/tokenizer-migration-path-test-kserve-85f864cf88-nzrvs"
Sep 17 18:07:26.759864 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:07:26.759840    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/5757cc18-3321-427d-9177-16c0987bdb2e-dshm\") pod \"tokenizer-migration-path-test-kserve-85f864cf88-nzrvs\" (UID: \"5757cc18-3321-427d-9177-16c0987bdb2e\") " pod="e2e-test-llm-inference-service-312caf2d/tokenizer-migration-path-test-kserve-85f864cf88-nzrvs"
Sep 17 18:07:26.760009 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:07:26.759990    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/5757cc18-3321-427d-9177-16c0987bdb2e-tls-certs\") pod \"tokenizer-migration-path-test-kserve-85f864cf88-nzrvs\" (UID: \"5757cc18-3321-427d-9177-16c0987bdb2e\") " pod="e2e-test-llm-inference-service-312caf2d/tokenizer-migration-path-test-kserve-85f864cf88-nzrvs"
Sep 17 18:07:26.765148 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:07:26.765129    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-tw8tj\" (UniqueName: \"kubernetes.io/projected/5757cc18-3321-427d-9177-16c0987bdb2e-kube-api-access-tw8tj\") pod \"tokenizer-migration-path-test-kserve-85f864cf88-nzrvs\" (UID: \"5757cc18-3321-427d-9177-16c0987bdb2e\") " pod="e2e-test-llm-inference-service-312caf2d/tokenizer-migration-path-test-kserve-85f864cf88-nzrvs"
Sep 17 18:07:26.831305 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:07:26.831230    2575 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["e2e-test-llm-inference-service-312caf2d/tokenizer-migration-path-test-tokenizer-7944b6fdb6-mqrn6"]
Sep 17 18:07:26.835798 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:07:26.835765    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-test-llm-inference-service-312caf2d/tokenizer-migration-path-test-kserve-85f864cf88-nzrvs"
Sep 17 18:07:26.836563 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:07:26.836543    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-test-llm-inference-service-312caf2d/tokenizer-migration-path-test-tokenizer-7944b6fdb6-mqrn6"
Sep 17 18:07:26.853157 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:07:26.853124    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["e2e-test-llm-inference-service-312caf2d/tokenizer-migration-path-test-tokenizer-7944b6fdb6-mqrn6"]
Sep 17 18:07:26.905418 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:07:26.905390    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-djvzt_1d07e68d-9943-420f-a6a0-5f477792fabd/konnectivity-agent/0.log" limit=1024
Sep 17 18:07:26.959208 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:07:26.959169    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/4cec6973-4ada-4405-84d2-c41e264d234a-tmp-dir\") pod \"tokenizer-migration-path-test-tokenizer-7944b6fdb6-mqrn6\" (UID: \"4cec6973-4ada-4405-84d2-c41e264d234a\") " pod="e2e-test-llm-inference-service-312caf2d/tokenizer-migration-path-test-tokenizer-7944b6fdb6-mqrn6"
Sep 17 18:07:26.959208 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:07:26.959218    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cabundle-cert\" (UniqueName: \"kubernetes.io/configmap/4cec6973-4ada-4405-84d2-c41e264d234a-cabundle-cert\") pod \"tokenizer-migration-path-test-tokenizer-7944b6fdb6-mqrn6\" (UID: \"4cec6973-4ada-4405-84d2-c41e264d234a\") " pod="e2e-test-llm-inference-service-312caf2d/tokenizer-migration-path-test-tokenizer-7944b6fdb6-mqrn6"
Sep 17 18:07:26.959506 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:07:26.959334    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/4cec6973-4ada-4405-84d2-c41e264d234a-kserve-provision-location\") pod \"tokenizer-migration-path-test-tokenizer-7944b6fdb6-mqrn6\" (UID: \"4cec6973-4ada-4405-84d2-c41e264d234a\") " pod="e2e-test-llm-inference-service-312caf2d/tokenizer-migration-path-test-tokenizer-7944b6fdb6-mqrn6"
Sep 17 18:07:26.959506 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:07:26.959385    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/4cec6973-4ada-4405-84d2-c41e264d234a-tls-certs\") pod \"tokenizer-migration-path-test-tokenizer-7944b6fdb6-mqrn6\" (UID: \"4cec6973-4ada-4405-84d2-c41e264d234a\") " pod="e2e-test-llm-inference-service-312caf2d/tokenizer-migration-path-test-tokenizer-7944b6fdb6-mqrn6"
Sep 17 18:07:27.021294 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:07:27.021254    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-djvzt_1d07e68d-9943-420f-a6a0-5f477792fabd/konnectivity-agent/0.log" limit=1024
Sep 17 18:07:27.060398 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:07:27.060360    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/4cec6973-4ada-4405-84d2-c41e264d234a-kserve-provision-location\") pod \"tokenizer-migration-path-test-tokenizer-7944b6fdb6-mqrn6\" (UID: \"4cec6973-4ada-4405-84d2-c41e264d234a\") " pod="e2e-test-llm-inference-service-312caf2d/tokenizer-migration-path-test-tokenizer-7944b6fdb6-mqrn6"
Sep 17 18:07:27.060398 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:07:27.060403    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/4cec6973-4ada-4405-84d2-c41e264d234a-tls-certs\") pod \"tokenizer-migration-path-test-tokenizer-7944b6fdb6-mqrn6\" (UID: \"4cec6973-4ada-4405-84d2-c41e264d234a\") " pod="e2e-test-llm-inference-service-312caf2d/tokenizer-migration-path-test-tokenizer-7944b6fdb6-mqrn6"
Sep 17 18:07:27.060651 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:07:27.060459    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/4cec6973-4ada-4405-84d2-c41e264d234a-tmp-dir\") pod \"tokenizer-migration-path-test-tokenizer-7944b6fdb6-mqrn6\" (UID: \"4cec6973-4ada-4405-84d2-c41e264d234a\") " pod="e2e-test-llm-inference-service-312caf2d/tokenizer-migration-path-test-tokenizer-7944b6fdb6-mqrn6"
Sep 17 18:07:27.060651 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:07:27.060484    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cabundle-cert\" (UniqueName: \"kubernetes.io/configmap/4cec6973-4ada-4405-84d2-c41e264d234a-cabundle-cert\") pod \"tokenizer-migration-path-test-tokenizer-7944b6fdb6-mqrn6\" (UID: \"4cec6973-4ada-4405-84d2-c41e264d234a\") " pod="e2e-test-llm-inference-service-312caf2d/tokenizer-migration-path-test-tokenizer-7944b6fdb6-mqrn6"
Sep 17 18:07:27.060774 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:07:27.060752    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/4cec6973-4ada-4405-84d2-c41e264d234a-kserve-provision-location\") pod \"tokenizer-migration-path-test-tokenizer-7944b6fdb6-mqrn6\" (UID: \"4cec6973-4ada-4405-84d2-c41e264d234a\") " pod="e2e-test-llm-inference-service-312caf2d/tokenizer-migration-path-test-tokenizer-7944b6fdb6-mqrn6"
Sep 17 18:07:27.060861 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:07:27.060837    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/4cec6973-4ada-4405-84d2-c41e264d234a-tmp-dir\") pod \"tokenizer-migration-path-test-tokenizer-7944b6fdb6-mqrn6\" (UID: \"4cec6973-4ada-4405-84d2-c41e264d234a\") " pod="e2e-test-llm-inference-service-312caf2d/tokenizer-migration-path-test-tokenizer-7944b6fdb6-mqrn6"
Sep 17 18:07:27.061114 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:07:27.061096    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cabundle-cert\" (UniqueName: \"kubernetes.io/configmap/4cec6973-4ada-4405-84d2-c41e264d234a-cabundle-cert\") pod \"tokenizer-migration-path-test-tokenizer-7944b6fdb6-mqrn6\" (UID: \"4cec6973-4ada-4405-84d2-c41e264d234a\") " pod="e2e-test-llm-inference-service-312caf2d/tokenizer-migration-path-test-tokenizer-7944b6fdb6-mqrn6"
Sep 17 18:07:27.062996 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:07:27.062971    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/4cec6973-4ada-4405-84d2-c41e264d234a-tls-certs\") pod \"tokenizer-migration-path-test-tokenizer-7944b6fdb6-mqrn6\" (UID: \"4cec6973-4ada-4405-84d2-c41e264d234a\") " pod="e2e-test-llm-inference-service-312caf2d/tokenizer-migration-path-test-tokenizer-7944b6fdb6-mqrn6"
Sep 17 18:07:27.129497 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:07:27.129463    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["e2e-test-llm-inference-service-312caf2d/tokenizer-migration-path-test-kserve-85f864cf88-nzrvs"]
Sep 17 18:07:27.131952 ip-10-0-143-74 kubenswrapper[2575]: W0917 18:07:27.131908    2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5757cc18_3321_427d_9177_16c0987bdb2e.slice/crio-2aed918a7f7cc437476a70cf51c25a3bc395ac535084dbe79dae789c736a6825 WatchSource:0}: Error finding container 2aed918a7f7cc437476a70cf51c25a3bc395ac535084dbe79dae789c736a6825: Status 404 returned error can't find the container with id 2aed918a7f7cc437476a70cf51c25a3bc395ac535084dbe79dae789c736a6825
Sep 17 18:07:27.169174 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:07:27.169145    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-test-llm-inference-service-312caf2d/tokenizer-migration-path-test-tokenizer-7944b6fdb6-mqrn6"
Sep 17 18:07:27.445735 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:07:27.445689    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["e2e-test-llm-inference-service-312caf2d/tokenizer-migration-path-test-tokenizer-7944b6fdb6-mqrn6"]
Sep 17 18:07:27.448814 ip-10-0-143-74 kubenswrapper[2575]: W0917 18:07:27.448773    2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4cec6973_4ada_4405_84d2_c41e264d234a.slice/crio-6e47a4f20ff45601956e603e3e0e2381d42ae3c8cf7eb57e6b2b0de780f1c0d6 WatchSource:0}: Error finding container 6e47a4f20ff45601956e603e3e0e2381d42ae3c8cf7eb57e6b2b0de780f1c0d6: Status 404 returned error can't find the container with id 6e47a4f20ff45601956e603e3e0e2381d42ae3c8cf7eb57e6b2b0de780f1c0d6
Sep 17 18:07:27.724327 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:07:27.724202    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-test-llm-inference-service-312caf2d/tokenizer-migration-path-test-tokenizer-7944b6fdb6-mqrn6" event={"ID":"4cec6973-4ada-4405-84d2-c41e264d234a","Type":"ContainerStarted","Data":"e5e08b0377f6d0ab71535aba26896f4233ea9b4b3f3da3d9305fae778e4c10e2"}
Sep 17 18:07:27.724327 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:07:27.724279    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-test-llm-inference-service-312caf2d/tokenizer-migration-path-test-tokenizer-7944b6fdb6-mqrn6" event={"ID":"4cec6973-4ada-4405-84d2-c41e264d234a","Type":"ContainerStarted","Data":"6e47a4f20ff45601956e603e3e0e2381d42ae3c8cf7eb57e6b2b0de780f1c0d6"}
Sep 17 18:07:27.726032 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:07:27.726005    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-test-llm-inference-service-312caf2d/tokenizer-migration-path-test-kserve-85f864cf88-nzrvs" event={"ID":"5757cc18-3321-427d-9177-16c0987bdb2e","Type":"ContainerStarted","Data":"a756fd6583e80a08bfb2672ce6cb2fb4e95f1caab37d1630f90d77d17a562cf2"}
Sep 17 18:07:27.726140 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:07:27.726050    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-test-llm-inference-service-312caf2d/tokenizer-migration-path-test-kserve-85f864cf88-nzrvs" event={"ID":"5757cc18-3321-427d-9177-16c0987bdb2e","Type":"ContainerStarted","Data":"2aed918a7f7cc437476a70cf51c25a3bc395ac535084dbe79dae789c736a6825"}
Sep 17 18:07:30.089635 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:07:30.089595    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-djvzt_1d07e68d-9943-420f-a6a0-5f477792fabd/konnectivity-agent/0.log" limit=1024
Sep 17 18:07:30.745058 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:07:30.745023    2575 generic.go:356] "Generic (PLEG): container finished" podID="5757cc18-3321-427d-9177-16c0987bdb2e" containerID="a756fd6583e80a08bfb2672ce6cb2fb4e95f1caab37d1630f90d77d17a562cf2" exitCode=0
Sep 17 18:07:30.745237 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:07:30.745094    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-test-llm-inference-service-312caf2d/tokenizer-migration-path-test-kserve-85f864cf88-nzrvs" event={"ID":"5757cc18-3321-427d-9177-16c0987bdb2e","Type":"ContainerDied","Data":"a756fd6583e80a08bfb2672ce6cb2fb4e95f1caab37d1630f90d77d17a562cf2"}
Sep 17 18:07:30.746921 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:07:30.746900    2575 generic.go:356] "Generic (PLEG): container finished" podID="4cec6973-4ada-4405-84d2-c41e264d234a" containerID="e5e08b0377f6d0ab71535aba26896f4233ea9b4b3f3da3d9305fae778e4c10e2" exitCode=0
Sep 17 18:07:30.747011 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:07:30.746932    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-test-llm-inference-service-312caf2d/tokenizer-migration-path-test-tokenizer-7944b6fdb6-mqrn6" event={"ID":"4cec6973-4ada-4405-84d2-c41e264d234a","Type":"ContainerDied","Data":"e5e08b0377f6d0ab71535aba26896f4233ea9b4b3f3da3d9305fae778e4c10e2"}
Sep 17 18:07:31.759010 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:07:31.758969    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-test-llm-inference-service-312caf2d/tokenizer-migration-path-test-tokenizer-7944b6fdb6-mqrn6" event={"ID":"4cec6973-4ada-4405-84d2-c41e264d234a","Type":"ContainerStarted","Data":"b32246b6e112b36441ecde73b21f6ce04bdd6f0bd2b2f9a8937feaed14529475"}
Sep 17 18:07:31.759489 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:07:31.759195    2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-test-llm-inference-service-312caf2d/tokenizer-migration-path-test-tokenizer-7944b6fdb6-mqrn6"
Sep 17 18:07:31.761194 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:07:31.761171    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-test-llm-inference-service-312caf2d/tokenizer-migration-path-test-kserve-85f864cf88-nzrvs" event={"ID":"5757cc18-3321-427d-9177-16c0987bdb2e","Type":"ContainerStarted","Data":"e1c1d8301b44bf42588551c2d55a26f16467f1794cc26cab3a7a1406fcc58c70"}
Sep 17 18:07:31.781208 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:07:31.781153    2575 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="e2e-test-llm-inference-service-312caf2d/tokenizer-migration-path-test-tokenizer-7944b6fdb6-mqrn6" podStartSLOduration=3.194711939 podStartE2EDuration="5.781135725s" totalImagesPullingTime="0s" totalInitContainerRuntime="2.586423786s" isStatefulPod=false podCreationTimestamp="2026-09-17 18:07:26 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-17 18:07:31.779472293 +0000 UTC m=+1971.409802269" watchObservedRunningTime="2026-09-17 18:07:31.781135725 +0000 UTC m=+1971.411465700"
Sep 17 18:07:31.802306 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:07:31.802240    2575 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="e2e-test-llm-inference-service-312caf2d/tokenizer-migration-path-test-kserve-85f864cf88-nzrvs" podStartSLOduration=3.152168074 podStartE2EDuration="5.802223492s" totalImagesPullingTime="0s" totalInitContainerRuntime="2.650055418s" isStatefulPod=false podCreationTimestamp="2026-09-17 18:07:26 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-17 18:07:31.800555915 +0000 UTC m=+1971.430885897" watchObservedRunningTime="2026-09-17 18:07:31.802223492 +0000 UTC m=+1971.432553467"
Sep 17 18:07:31.836926 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:07:31.836887    2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-test-llm-inference-service-312caf2d/tokenizer-migration-path-test-kserve-85f864cf88-nzrvs"
Sep 17 18:07:32.382719 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:07:32.382650    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-djvzt_1d07e68d-9943-420f-a6a0-5f477792fabd/konnectivity-agent/0.log" limit=1024
Sep 17 18:07:36.506028 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:07:36.505941    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-djvzt_1d07e68d-9943-420f-a6a0-5f477792fabd/konnectivity-agent/0.log" limit=1024
Sep 17 18:07:36.712144 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:07:36.712108    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-djvzt_1d07e68d-9943-420f-a6a0-5f477792fabd/konnectivity-agent/0.log" limit=1024
Sep 17 18:07:36.836186 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:07:36.836099    2575 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="e2e-test-llm-inference-service-312caf2d/tokenizer-migration-path-test-kserve-85f864cf88-nzrvs"
Sep 17 18:07:36.849616 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:07:36.849588    2575 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="e2e-test-llm-inference-service-312caf2d/tokenizer-migration-path-test-kserve-85f864cf88-nzrvs"
Sep 17 18:07:37.812315 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:07:37.812282    2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-test-llm-inference-service-312caf2d/tokenizer-migration-path-test-kserve-85f864cf88-nzrvs"
Sep 17 18:08:02.779536 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:08:02.779506    2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-test-llm-inference-service-312caf2d/tokenizer-migration-path-test-tokenizer-7944b6fdb6-mqrn6"
Sep 17 18:08:03.846392 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:08:03.846358    2575 log.go:25] "Finished parsing log file" path="/var/log/pods/e2e-test-llm-inference-service-312caf2d_tokenizer-migration-path-test-tokenizer-7944b6fdb6-mqrn6_4cec6973-4ada-4405-84d2-c41e264d234a/main/0.log"
Sep 17 18:08:04.343914 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:08:04.343867    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-djvzt_1d07e68d-9943-420f-a6a0-5f477792fabd/konnectivity-agent/0.log" limit=1024
Sep 17 18:08:04.377425 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:08:04.377383    2575 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["e2e-test-llm-inference-service-312caf2d/tokenizer-migration-path-test-tokenizer-7944b6fdb6-mqrn6"]
Sep 17 18:08:04.377748 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:08:04.377688    2575 kuberuntime_container.go:873] "Killing container with a grace period" pod="e2e-test-llm-inference-service-312caf2d/tokenizer-migration-path-test-tokenizer-7944b6fdb6-mqrn6" podUID="4cec6973-4ada-4405-84d2-c41e264d234a" containerName="main" containerID="cri-o://b32246b6e112b36441ecde73b21f6ce04bdd6f0bd2b2f9a8937feaed14529475" gracePeriod=30
Sep 17 18:08:04.380638 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:08:04.380606    2575 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["e2e-test-llm-inference-service-312caf2d/tokenizer-migration-path-test-kserve-85f864cf88-nzrvs"]
Sep 17 18:08:04.389403 ip-10-0-143-74 kubenswrapper[2575]: E0917 18:08:04.389369    2575 log.go:32] "ExecSync cmd from runtime service failed" err=<
Sep 17 18:08:04.389403 ip-10-0-143-74 kubenswrapper[2575]:         rpc error: code = Unknown desc = command error: executable file `/bin/sleep` not found: No such file or directory
Sep 17 18:08:04.389403 ip-10-0-143-74 kubenswrapper[2575]:         , stdout: , stderr: , exit code -1
Sep 17 18:08:04.389403 ip-10-0-143-74 kubenswrapper[2575]:  > containerID="e1c1d8301b44bf42588551c2d55a26f16467f1794cc26cab3a7a1406fcc58c70" cmd=["/bin/sleep","15"]
Sep 17 18:08:04.389678 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:08:04.389428    2575 handlers.go:82] "Exec lifecycle hook for Container in Pod failed" execCommand=["/bin/sleep","15"] containerName="main" pod="e2e-test-llm-inference-service-312caf2d/tokenizer-migration-path-test-kserve-85f864cf88-nzrvs" message="" err=<
Sep 17 18:08:04.389678 ip-10-0-143-74 kubenswrapper[2575]:         rpc error: code = Unknown desc = command error: executable file `/bin/sleep` not found: No such file or directory
Sep 17 18:08:04.389678 ip-10-0-143-74 kubenswrapper[2575]:         , stdout: , stderr: , exit code -1
Sep 17 18:08:04.389678 ip-10-0-143-74 kubenswrapper[2575]:  >
Sep 17 18:08:04.389678 ip-10-0-143-74 kubenswrapper[2575]: E0917 18:08:04.389474    2575 kuberuntime_container.go:755] "PreStop hook failed" err=<
Sep 17 18:08:04.389678 ip-10-0-143-74 kubenswrapper[2575]:         rpc error: code = Unknown desc = command error: executable file `/bin/sleep` not found: No such file or directory
Sep 17 18:08:04.389678 ip-10-0-143-74 kubenswrapper[2575]:         , stdout: , stderr: , exit code -1
Sep 17 18:08:04.389678 ip-10-0-143-74 kubenswrapper[2575]:  > pod="e2e-test-llm-inference-service-312caf2d/tokenizer-migration-path-test-kserve-85f864cf88-nzrvs" podUID="5757cc18-3321-427d-9177-16c0987bdb2e" containerName="main" containerID="cri-o://e1c1d8301b44bf42588551c2d55a26f16467f1794cc26cab3a7a1406fcc58c70"
Sep 17 18:08:04.389678 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:08:04.389528    2575 kuberuntime_container.go:873] "Killing container with a grace period" pod="e2e-test-llm-inference-service-312caf2d/tokenizer-migration-path-test-kserve-85f864cf88-nzrvs" podUID="5757cc18-3321-427d-9177-16c0987bdb2e" containerName="main" containerID="cri-o://e1c1d8301b44bf42588551c2d55a26f16467f1794cc26cab3a7a1406fcc58c70" gracePeriod=60
Sep 17 18:08:04.448748 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:08:04.448712    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-djvzt_1d07e68d-9943-420f-a6a0-5f477792fabd/konnectivity-agent/0.log" limit=1024
Sep 17 18:08:04.801379 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:08:04.801339    2575 prober.go:120] "Probe failed" probeType="Readiness" pod="e2e-test-llm-inference-service-312caf2d/tokenizer-migration-path-test-kserve-85f864cf88-nzrvs" podUID="5757cc18-3321-427d-9177-16c0987bdb2e" containerName="main" probeResult="failure" output="Get \"https://10.133.0.52:8000/health\": dial tcp 10.133.0.52:8000: connect: connection refused"
Sep 17 18:08:04.962176 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:08:04.962139    2575 generic.go:356] "Generic (PLEG): container finished" podID="5757cc18-3321-427d-9177-16c0987bdb2e" containerID="e1c1d8301b44bf42588551c2d55a26f16467f1794cc26cab3a7a1406fcc58c70" exitCode=0
Sep 17 18:08:04.964399 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:08:04.964374    2575 generic.go:356] "Generic (PLEG): container finished" podID="4cec6973-4ada-4405-84d2-c41e264d234a" containerID="b32246b6e112b36441ecde73b21f6ce04bdd6f0bd2b2f9a8937feaed14529475" exitCode=0
Sep 17 18:08:04.969972 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:08:04.969943    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-test-llm-inference-service-312caf2d/tokenizer-migration-path-test-kserve-85f864cf88-nzrvs" event={"ID":"5757cc18-3321-427d-9177-16c0987bdb2e","Type":"ContainerDied","Data":"e1c1d8301b44bf42588551c2d55a26f16467f1794cc26cab3a7a1406fcc58c70"}
Sep 17 18:08:04.970104 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:08:04.970014    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-test-llm-inference-service-312caf2d/tokenizer-migration-path-test-tokenizer-7944b6fdb6-mqrn6" event={"ID":"4cec6973-4ada-4405-84d2-c41e264d234a","Type":"ContainerDied","Data":"b32246b6e112b36441ecde73b21f6ce04bdd6f0bd2b2f9a8937feaed14529475"}
Sep 17 18:08:05.038573 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:08:05.038545    2575 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-test-llm-inference-service-312caf2d/tokenizer-migration-path-test-tokenizer-7944b6fdb6-mqrn6"
Sep 17 18:08:05.130451 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:08:05.130414    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"cabundle-cert\" (UniqueName: \"kubernetes.io/configmap/4cec6973-4ada-4405-84d2-c41e264d234a-cabundle-cert\") pod \"4cec6973-4ada-4405-84d2-c41e264d234a\" (UID: \"4cec6973-4ada-4405-84d2-c41e264d234a\") "
Sep 17 18:08:05.130596 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:08:05.130505    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/4cec6973-4ada-4405-84d2-c41e264d234a-tls-certs\") pod \"4cec6973-4ada-4405-84d2-c41e264d234a\" (UID: \"4cec6973-4ada-4405-84d2-c41e264d234a\") "
Sep 17 18:08:05.130596 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:08:05.130556    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/4cec6973-4ada-4405-84d2-c41e264d234a-kserve-provision-location\") pod \"4cec6973-4ada-4405-84d2-c41e264d234a\" (UID: \"4cec6973-4ada-4405-84d2-c41e264d234a\") "
Sep 17 18:08:05.130596 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:08:05.130579    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/4cec6973-4ada-4405-84d2-c41e264d234a-tmp-dir\") pod \"4cec6973-4ada-4405-84d2-c41e264d234a\" (UID: \"4cec6973-4ada-4405-84d2-c41e264d234a\") "
Sep 17 18:08:05.130869 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:08:05.130838    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/4cec6973-4ada-4405-84d2-c41e264d234a-cabundle-cert" (OuterVolumeSpecName: "cabundle-cert") pod "4cec6973-4ada-4405-84d2-c41e264d234a" (UID: "4cec6973-4ada-4405-84d2-c41e264d234a"). InnerVolumeSpecName "cabundle-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 17 18:08:05.131005 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:08:05.130980    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/4cec6973-4ada-4405-84d2-c41e264d234a-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "4cec6973-4ada-4405-84d2-c41e264d234a" (UID: "4cec6973-4ada-4405-84d2-c41e264d234a"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 17 18:08:05.132758 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:08:05.132726    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/4cec6973-4ada-4405-84d2-c41e264d234a-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "4cec6973-4ada-4405-84d2-c41e264d234a" (UID: "4cec6973-4ada-4405-84d2-c41e264d234a"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 17 18:08:05.148989 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:08:05.148966    2575 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-test-llm-inference-service-312caf2d/tokenizer-migration-path-test-kserve-85f864cf88-nzrvs"
Sep 17 18:08:05.160241 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:08:05.160200    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/4cec6973-4ada-4405-84d2-c41e264d234a-kserve-provision-location" (OuterVolumeSpecName: "kserve-provision-location") pod "4cec6973-4ada-4405-84d2-c41e264d234a" (UID: "4cec6973-4ada-4405-84d2-c41e264d234a"). InnerVolumeSpecName "kserve-provision-location". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 17 18:08:05.231950 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:08:05.231891    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/5757cc18-3321-427d-9177-16c0987bdb2e-tmp-dir\") pod \"5757cc18-3321-427d-9177-16c0987bdb2e\" (UID: \"5757cc18-3321-427d-9177-16c0987bdb2e\") "
Sep 17 18:08:05.231950 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:08:05.231958    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-tw8tj\" (UniqueName: \"kubernetes.io/projected/5757cc18-3321-427d-9177-16c0987bdb2e-kube-api-access-tw8tj\") pod \"5757cc18-3321-427d-9177-16c0987bdb2e\" (UID: \"5757cc18-3321-427d-9177-16c0987bdb2e\") "
Sep 17 18:08:05.232184 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:08:05.231988    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/5757cc18-3321-427d-9177-16c0987bdb2e-kserve-provision-location\") pod \"5757cc18-3321-427d-9177-16c0987bdb2e\" (UID: \"5757cc18-3321-427d-9177-16c0987bdb2e\") "
Sep 17 18:08:05.232184 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:08:05.232021    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"cabundle-cert\" (UniqueName: \"kubernetes.io/configmap/5757cc18-3321-427d-9177-16c0987bdb2e-cabundle-cert\") pod \"5757cc18-3321-427d-9177-16c0987bdb2e\" (UID: \"5757cc18-3321-427d-9177-16c0987bdb2e\") "
Sep 17 18:08:05.232184 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:08:05.232040    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/5757cc18-3321-427d-9177-16c0987bdb2e-dshm\") pod \"5757cc18-3321-427d-9177-16c0987bdb2e\" (UID: \"5757cc18-3321-427d-9177-16c0987bdb2e\") "
Sep 17 18:08:05.232184 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:08:05.232086    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/5757cc18-3321-427d-9177-16c0987bdb2e-tls-certs\") pod \"5757cc18-3321-427d-9177-16c0987bdb2e\" (UID: \"5757cc18-3321-427d-9177-16c0987bdb2e\") "
Sep 17 18:08:05.232184 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:08:05.232125    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/5757cc18-3321-427d-9177-16c0987bdb2e-model-cache\") pod \"5757cc18-3321-427d-9177-16c0987bdb2e\" (UID: \"5757cc18-3321-427d-9177-16c0987bdb2e\") "
Sep 17 18:08:05.232184 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:08:05.232157    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/5757cc18-3321-427d-9177-16c0987bdb2e-home\") pod \"5757cc18-3321-427d-9177-16c0987bdb2e\" (UID: \"5757cc18-3321-427d-9177-16c0987bdb2e\") "
Sep 17 18:08:05.232516 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:08:05.232213    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/5757cc18-3321-427d-9177-16c0987bdb2e-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "5757cc18-3321-427d-9177-16c0987bdb2e" (UID: "5757cc18-3321-427d-9177-16c0987bdb2e"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 17 18:08:05.232581 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:08:05.232509    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/5757cc18-3321-427d-9177-16c0987bdb2e-cabundle-cert" (OuterVolumeSpecName: "cabundle-cert") pod "5757cc18-3321-427d-9177-16c0987bdb2e" (UID: "5757cc18-3321-427d-9177-16c0987bdb2e"). InnerVolumeSpecName "cabundle-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 17 18:08:05.232581 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:08:05.232523    2575 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/5757cc18-3321-427d-9177-16c0987bdb2e-tmp-dir\") on node \"ip-10-0-143-74.ec2.internal\" DevicePath \"\""
Sep 17 18:08:05.232581 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:08:05.232569    2575 reconciler_common.go:299] "Volume detached for volume \"cabundle-cert\" (UniqueName: \"kubernetes.io/configmap/4cec6973-4ada-4405-84d2-c41e264d234a-cabundle-cert\") on node \"ip-10-0-143-74.ec2.internal\" DevicePath \"\""
Sep 17 18:08:05.232755 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:08:05.232570    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/5757cc18-3321-427d-9177-16c0987bdb2e-model-cache" (OuterVolumeSpecName: "model-cache") pod "5757cc18-3321-427d-9177-16c0987bdb2e" (UID: "5757cc18-3321-427d-9177-16c0987bdb2e"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 17 18:08:05.232755 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:08:05.232584    2575 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/4cec6973-4ada-4405-84d2-c41e264d234a-tls-certs\") on node \"ip-10-0-143-74.ec2.internal\" DevicePath \"\""
Sep 17 18:08:05.232755 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:08:05.232622    2575 reconciler_common.go:299] "Volume detached for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/4cec6973-4ada-4405-84d2-c41e264d234a-kserve-provision-location\") on node \"ip-10-0-143-74.ec2.internal\" DevicePath \"\""
Sep 17 18:08:05.232755 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:08:05.232639    2575 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/4cec6973-4ada-4405-84d2-c41e264d234a-tmp-dir\") on node \"ip-10-0-143-74.ec2.internal\" DevicePath \"\""
Sep 17 18:08:05.232755 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:08:05.232736    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/5757cc18-3321-427d-9177-16c0987bdb2e-home" (OuterVolumeSpecName: "home") pod "5757cc18-3321-427d-9177-16c0987bdb2e" (UID: "5757cc18-3321-427d-9177-16c0987bdb2e"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 17 18:08:05.234278 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:08:05.234255    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/5757cc18-3321-427d-9177-16c0987bdb2e-dshm" (OuterVolumeSpecName: "dshm") pod "5757cc18-3321-427d-9177-16c0987bdb2e" (UID: "5757cc18-3321-427d-9177-16c0987bdb2e"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 17 18:08:05.234278 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:08:05.234247    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/5757cc18-3321-427d-9177-16c0987bdb2e-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "5757cc18-3321-427d-9177-16c0987bdb2e" (UID: "5757cc18-3321-427d-9177-16c0987bdb2e"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 17 18:08:05.234412 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:08:05.234294    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/5757cc18-3321-427d-9177-16c0987bdb2e-kube-api-access-tw8tj" (OuterVolumeSpecName: "kube-api-access-tw8tj") pod "5757cc18-3321-427d-9177-16c0987bdb2e" (UID: "5757cc18-3321-427d-9177-16c0987bdb2e"). InnerVolumeSpecName "kube-api-access-tw8tj". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 17 18:08:05.255589 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:08:05.255557    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/5757cc18-3321-427d-9177-16c0987bdb2e-kserve-provision-location" (OuterVolumeSpecName: "kserve-provision-location") pod "5757cc18-3321-427d-9177-16c0987bdb2e" (UID: "5757cc18-3321-427d-9177-16c0987bdb2e"). InnerVolumeSpecName "kserve-provision-location". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 17 18:08:05.333902 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:08:05.333812    2575 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/5757cc18-3321-427d-9177-16c0987bdb2e-model-cache\") on node \"ip-10-0-143-74.ec2.internal\" DevicePath \"\""
Sep 17 18:08:05.333902 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:08:05.333844    2575 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/5757cc18-3321-427d-9177-16c0987bdb2e-home\") on node \"ip-10-0-143-74.ec2.internal\" DevicePath \"\""
Sep 17 18:08:05.333902 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:08:05.333853    2575 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-tw8tj\" (UniqueName: \"kubernetes.io/projected/5757cc18-3321-427d-9177-16c0987bdb2e-kube-api-access-tw8tj\") on node \"ip-10-0-143-74.ec2.internal\" DevicePath \"\""
Sep 17 18:08:05.333902 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:08:05.333863    2575 reconciler_common.go:299] "Volume detached for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/5757cc18-3321-427d-9177-16c0987bdb2e-kserve-provision-location\") on node \"ip-10-0-143-74.ec2.internal\" DevicePath \"\""
Sep 17 18:08:05.333902 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:08:05.333872    2575 reconciler_common.go:299] "Volume detached for volume \"cabundle-cert\" (UniqueName: \"kubernetes.io/configmap/5757cc18-3321-427d-9177-16c0987bdb2e-cabundle-cert\") on node \"ip-10-0-143-74.ec2.internal\" DevicePath \"\""
Sep 17 18:08:05.333902 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:08:05.333879    2575 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/5757cc18-3321-427d-9177-16c0987bdb2e-dshm\") on node \"ip-10-0-143-74.ec2.internal\" DevicePath \"\""
Sep 17 18:08:05.333902 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:08:05.333887    2575 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/5757cc18-3321-427d-9177-16c0987bdb2e-tls-certs\") on node \"ip-10-0-143-74.ec2.internal\" DevicePath \"\""
Sep 17 18:08:05.972127 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:08:05.972088    2575 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-test-llm-inference-service-312caf2d/tokenizer-migration-path-test-tokenizer-7944b6fdb6-mqrn6"
Sep 17 18:08:05.972127 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:08:05.972110    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-test-llm-inference-service-312caf2d/tokenizer-migration-path-test-tokenizer-7944b6fdb6-mqrn6" event={"ID":"4cec6973-4ada-4405-84d2-c41e264d234a","Type":"ContainerDied","Data":"6e47a4f20ff45601956e603e3e0e2381d42ae3c8cf7eb57e6b2b0de780f1c0d6"}
Sep 17 18:08:05.972619 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:08:05.972178    2575 scope.go:117] "RemoveContainer" containerID="b32246b6e112b36441ecde73b21f6ce04bdd6f0bd2b2f9a8937feaed14529475"
Sep 17 18:08:05.973880 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:08:05.973847    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-test-llm-inference-service-312caf2d/tokenizer-migration-path-test-kserve-85f864cf88-nzrvs" event={"ID":"5757cc18-3321-427d-9177-16c0987bdb2e","Type":"ContainerDied","Data":"2aed918a7f7cc437476a70cf51c25a3bc395ac535084dbe79dae789c736a6825"}
Sep 17 18:08:05.973880 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:08:05.973874    2575 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-test-llm-inference-service-312caf2d/tokenizer-migration-path-test-kserve-85f864cf88-nzrvs"
Sep 17 18:08:05.982742 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:08:05.982724    2575 scope.go:117] "RemoveContainer" containerID="e5e08b0377f6d0ab71535aba26896f4233ea9b4b3f3da3d9305fae778e4c10e2"
Sep 17 18:08:05.991104 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:08:05.991085    2575 scope.go:117] "RemoveContainer" containerID="e1c1d8301b44bf42588551c2d55a26f16467f1794cc26cab3a7a1406fcc58c70"
Sep 17 18:08:05.996696 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:08:05.996670    2575 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["e2e-test-llm-inference-service-312caf2d/tokenizer-migration-path-test-tokenizer-7944b6fdb6-mqrn6"]
Sep 17 18:08:06.000696 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:08:06.000681    2575 scope.go:117] "RemoveContainer" containerID="a756fd6583e80a08bfb2672ce6cb2fb4e95f1caab37d1630f90d77d17a562cf2"
Sep 17 18:08:06.001982 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:08:06.001961    2575 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["e2e-test-llm-inference-service-312caf2d/tokenizer-migration-path-test-tokenizer-7944b6fdb6-mqrn6"]
Sep 17 18:08:06.013699 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:08:06.013481    2575 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["e2e-test-llm-inference-service-312caf2d/tokenizer-migration-path-test-kserve-85f864cf88-nzrvs"]
Sep 17 18:08:06.017413 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:08:06.017385    2575 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["e2e-test-llm-inference-service-312caf2d/tokenizer-migration-path-test-kserve-85f864cf88-nzrvs"]
Sep 17 18:08:06.968259 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:08:06.968220    2575 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="4cec6973-4ada-4405-84d2-c41e264d234a" path="/var/lib/kubelet/pods/4cec6973-4ada-4405-84d2-c41e264d234a/volumes"
Sep 17 18:08:06.968791 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:08:06.968771    2575 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="5757cc18-3321-427d-9177-16c0987bdb2e" path="/var/lib/kubelet/pods/5757cc18-3321-427d-9177-16c0987bdb2e/volumes"
Sep 17 18:08:44.066211 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:08:44.066162    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-djvzt_1d07e68d-9943-420f-a6a0-5f477792fabd/konnectivity-agent/0.log" limit=1024
Sep 17 18:08:44.270684 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:08:44.270652    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-djvzt_1d07e68d-9943-420f-a6a0-5f477792fabd/konnectivity-agent/0.log" limit=1024
Sep 17 18:08:54.169278 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:08:54.169244    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-djvzt_1d07e68d-9943-420f-a6a0-5f477792fabd/konnectivity-agent/0.log" limit=1024
Sep 17 18:08:54.267138 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:08:54.267094    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-djvzt_1d07e68d-9943-420f-a6a0-5f477792fabd/konnectivity-agent/0.log" limit=1024
Sep 17 18:08:59.614274 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:08:59.614240    2575 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["e2e-test-llm-inference-service-0eb24714/tokenizer-template-args-test-kserve-79dbd77788-tr755"]
Sep 17 18:08:59.614669 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:08:59.614644    2575 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="5757cc18-3321-427d-9177-16c0987bdb2e" containerName="main"
Sep 17 18:08:59.614669 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:08:59.614657    2575 state_mem.go:107] "Deleted CPUSet assignment" podUID="5757cc18-3321-427d-9177-16c0987bdb2e" containerName="main"
Sep 17 18:08:59.614669 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:08:59.614669    2575 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="4cec6973-4ada-4405-84d2-c41e264d234a" containerName="storage-initializer"
Sep 17 18:08:59.614776 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:08:59.614675    2575 state_mem.go:107] "Deleted CPUSet assignment" podUID="4cec6973-4ada-4405-84d2-c41e264d234a" containerName="storage-initializer"
Sep 17 18:08:59.614776 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:08:59.614682    2575 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="4cec6973-4ada-4405-84d2-c41e264d234a" containerName="main"
Sep 17 18:08:59.614776 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:08:59.614687    2575 state_mem.go:107] "Deleted CPUSet assignment" podUID="4cec6973-4ada-4405-84d2-c41e264d234a" containerName="main"
Sep 17 18:08:59.614776 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:08:59.614700    2575 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="5757cc18-3321-427d-9177-16c0987bdb2e" containerName="storage-initializer"
Sep 17 18:08:59.614776 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:08:59.614706    2575 state_mem.go:107] "Deleted CPUSet assignment" podUID="5757cc18-3321-427d-9177-16c0987bdb2e" containerName="storage-initializer"
Sep 17 18:08:59.614776 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:08:59.614767    2575 memory_manager.go:368] "RemoveStaleState removing state" podUID="4cec6973-4ada-4405-84d2-c41e264d234a" containerName="main"
Sep 17 18:08:59.614776 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:08:59.614776    2575 memory_manager.go:368] "RemoveStaleState removing state" podUID="5757cc18-3321-427d-9177-16c0987bdb2e" containerName="main"
Sep 17 18:08:59.619048 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:08:59.619029    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-test-llm-inference-service-0eb24714/tokenizer-template-args-test-kserve-79dbd77788-tr755"
Sep 17 18:08:59.621697 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:08:59.621674    2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-test-llm-inference-service-0eb24714\"/\"openshift-service-ca.crt\""
Sep 17 18:08:59.621825 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:08:59.621711    2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-test-llm-inference-service-0eb24714\"/\"odh-kserve-custom-ca-bundle\""
Sep 17 18:08:59.621825 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:08:59.621755    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-test-llm-inference-service-0eb24714\"/\"seaweedfs-s3-creds\""
Sep 17 18:08:59.622659 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:08:59.622635    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-test-llm-inference-service-0eb24714\"/\"default-dockercfg-dzp2d\""
Sep 17 18:08:59.622764 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:08:59.622703    2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-test-llm-inference-service-0eb24714\"/\"kube-root-ca.crt\""
Sep 17 18:08:59.622764 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:08:59.622735    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-test-llm-inference-service-0eb24714\"/\"tokenizer-template-args-test-kserve-self-signed-certs\""
Sep 17 18:08:59.634361 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:08:59.634324    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-djvzt_1d07e68d-9943-420f-a6a0-5f477792fabd/konnectivity-agent/0.log" limit=1024
Sep 17 18:08:59.635536 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:08:59.635512    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["e2e-test-llm-inference-service-0eb24714/tokenizer-template-args-test-kserve-79dbd77788-tr755"]
Sep 17 18:08:59.704843 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:08:59.704811    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/13ea8edb-65c4-469c-a4fd-b26c6a217485-tls-certs\") pod \"tokenizer-template-args-test-kserve-79dbd77788-tr755\" (UID: \"13ea8edb-65c4-469c-a4fd-b26c6a217485\") " pod="e2e-test-llm-inference-service-0eb24714/tokenizer-template-args-test-kserve-79dbd77788-tr755"
Sep 17 18:08:59.704843 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:08:59.704845    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/13ea8edb-65c4-469c-a4fd-b26c6a217485-kserve-provision-location\") pod \"tokenizer-template-args-test-kserve-79dbd77788-tr755\" (UID: \"13ea8edb-65c4-469c-a4fd-b26c6a217485\") " pod="e2e-test-llm-inference-service-0eb24714/tokenizer-template-args-test-kserve-79dbd77788-tr755"
Sep 17 18:08:59.705030 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:08:59.704874    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/13ea8edb-65c4-469c-a4fd-b26c6a217485-home\") pod \"tokenizer-template-args-test-kserve-79dbd77788-tr755\" (UID: \"13ea8edb-65c4-469c-a4fd-b26c6a217485\") " pod="e2e-test-llm-inference-service-0eb24714/tokenizer-template-args-test-kserve-79dbd77788-tr755"
Sep 17 18:08:59.705030 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:08:59.704945    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/13ea8edb-65c4-469c-a4fd-b26c6a217485-tmp-dir\") pod \"tokenizer-template-args-test-kserve-79dbd77788-tr755\" (UID: \"13ea8edb-65c4-469c-a4fd-b26c6a217485\") " pod="e2e-test-llm-inference-service-0eb24714/tokenizer-template-args-test-kserve-79dbd77788-tr755"
Sep 17 18:08:59.705030 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:08:59.704973    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/13ea8edb-65c4-469c-a4fd-b26c6a217485-model-cache\") pod \"tokenizer-template-args-test-kserve-79dbd77788-tr755\" (UID: \"13ea8edb-65c4-469c-a4fd-b26c6a217485\") " pod="e2e-test-llm-inference-service-0eb24714/tokenizer-template-args-test-kserve-79dbd77788-tr755"
Sep 17 18:08:59.705030 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:08:59.704995    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zgdx5\" (UniqueName: \"kubernetes.io/projected/13ea8edb-65c4-469c-a4fd-b26c6a217485-kube-api-access-zgdx5\") pod \"tokenizer-template-args-test-kserve-79dbd77788-tr755\" (UID: \"13ea8edb-65c4-469c-a4fd-b26c6a217485\") " pod="e2e-test-llm-inference-service-0eb24714/tokenizer-template-args-test-kserve-79dbd77788-tr755"
Sep 17 18:08:59.705165 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:08:59.705053    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cabundle-cert\" (UniqueName: \"kubernetes.io/configmap/13ea8edb-65c4-469c-a4fd-b26c6a217485-cabundle-cert\") pod \"tokenizer-template-args-test-kserve-79dbd77788-tr755\" (UID: \"13ea8edb-65c4-469c-a4fd-b26c6a217485\") " pod="e2e-test-llm-inference-service-0eb24714/tokenizer-template-args-test-kserve-79dbd77788-tr755"
Sep 17 18:08:59.705165 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:08:59.705111    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/13ea8edb-65c4-469c-a4fd-b26c6a217485-dshm\") pod \"tokenizer-template-args-test-kserve-79dbd77788-tr755\" (UID: \"13ea8edb-65c4-469c-a4fd-b26c6a217485\") " pod="e2e-test-llm-inference-service-0eb24714/tokenizer-template-args-test-kserve-79dbd77788-tr755"
Sep 17 18:08:59.805924 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:08:59.805885    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/13ea8edb-65c4-469c-a4fd-b26c6a217485-tls-certs\") pod \"tokenizer-template-args-test-kserve-79dbd77788-tr755\" (UID: \"13ea8edb-65c4-469c-a4fd-b26c6a217485\") " pod="e2e-test-llm-inference-service-0eb24714/tokenizer-template-args-test-kserve-79dbd77788-tr755"
Sep 17 18:08:59.805924 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:08:59.805925    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/13ea8edb-65c4-469c-a4fd-b26c6a217485-kserve-provision-location\") pod \"tokenizer-template-args-test-kserve-79dbd77788-tr755\" (UID: \"13ea8edb-65c4-469c-a4fd-b26c6a217485\") " pod="e2e-test-llm-inference-service-0eb24714/tokenizer-template-args-test-kserve-79dbd77788-tr755"
Sep 17 18:08:59.806166 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:08:59.805970    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/13ea8edb-65c4-469c-a4fd-b26c6a217485-home\") pod \"tokenizer-template-args-test-kserve-79dbd77788-tr755\" (UID: \"13ea8edb-65c4-469c-a4fd-b26c6a217485\") " pod="e2e-test-llm-inference-service-0eb24714/tokenizer-template-args-test-kserve-79dbd77788-tr755"
Sep 17 18:08:59.806166 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:08:59.805999    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/13ea8edb-65c4-469c-a4fd-b26c6a217485-tmp-dir\") pod \"tokenizer-template-args-test-kserve-79dbd77788-tr755\" (UID: \"13ea8edb-65c4-469c-a4fd-b26c6a217485\") " pod="e2e-test-llm-inference-service-0eb24714/tokenizer-template-args-test-kserve-79dbd77788-tr755"
Sep 17 18:08:59.806166 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:08:59.806025    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/13ea8edb-65c4-469c-a4fd-b26c6a217485-model-cache\") pod \"tokenizer-template-args-test-kserve-79dbd77788-tr755\" (UID: \"13ea8edb-65c4-469c-a4fd-b26c6a217485\") " pod="e2e-test-llm-inference-service-0eb24714/tokenizer-template-args-test-kserve-79dbd77788-tr755"
Sep 17 18:08:59.806166 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:08:59.806052    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zgdx5\" (UniqueName: \"kubernetes.io/projected/13ea8edb-65c4-469c-a4fd-b26c6a217485-kube-api-access-zgdx5\") pod \"tokenizer-template-args-test-kserve-79dbd77788-tr755\" (UID: \"13ea8edb-65c4-469c-a4fd-b26c6a217485\") " pod="e2e-test-llm-inference-service-0eb24714/tokenizer-template-args-test-kserve-79dbd77788-tr755"
Sep 17 18:08:59.806166 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:08:59.806089    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cabundle-cert\" (UniqueName: \"kubernetes.io/configmap/13ea8edb-65c4-469c-a4fd-b26c6a217485-cabundle-cert\") pod \"tokenizer-template-args-test-kserve-79dbd77788-tr755\" (UID: \"13ea8edb-65c4-469c-a4fd-b26c6a217485\") " pod="e2e-test-llm-inference-service-0eb24714/tokenizer-template-args-test-kserve-79dbd77788-tr755"
Sep 17 18:08:59.806166 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:08:59.806136    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/13ea8edb-65c4-469c-a4fd-b26c6a217485-dshm\") pod \"tokenizer-template-args-test-kserve-79dbd77788-tr755\" (UID: \"13ea8edb-65c4-469c-a4fd-b26c6a217485\") " pod="e2e-test-llm-inference-service-0eb24714/tokenizer-template-args-test-kserve-79dbd77788-tr755"
Sep 17 18:08:59.806520 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:08:59.806498    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/13ea8edb-65c4-469c-a4fd-b26c6a217485-kserve-provision-location\") pod \"tokenizer-template-args-test-kserve-79dbd77788-tr755\" (UID: \"13ea8edb-65c4-469c-a4fd-b26c6a217485\") " pod="e2e-test-llm-inference-service-0eb24714/tokenizer-template-args-test-kserve-79dbd77788-tr755"
Sep 17 18:08:59.806634 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:08:59.806604    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/13ea8edb-65c4-469c-a4fd-b26c6a217485-home\") pod \"tokenizer-template-args-test-kserve-79dbd77788-tr755\" (UID: \"13ea8edb-65c4-469c-a4fd-b26c6a217485\") " pod="e2e-test-llm-inference-service-0eb24714/tokenizer-template-args-test-kserve-79dbd77788-tr755"
Sep 17 18:08:59.806700 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:08:59.806616    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/13ea8edb-65c4-469c-a4fd-b26c6a217485-tmp-dir\") pod \"tokenizer-template-args-test-kserve-79dbd77788-tr755\" (UID: \"13ea8edb-65c4-469c-a4fd-b26c6a217485\") " pod="e2e-test-llm-inference-service-0eb24714/tokenizer-template-args-test-kserve-79dbd77788-tr755"
Sep 17 18:08:59.806700 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:08:59.806672    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/13ea8edb-65c4-469c-a4fd-b26c6a217485-model-cache\") pod \"tokenizer-template-args-test-kserve-79dbd77788-tr755\" (UID: \"13ea8edb-65c4-469c-a4fd-b26c6a217485\") " pod="e2e-test-llm-inference-service-0eb24714/tokenizer-template-args-test-kserve-79dbd77788-tr755"
Sep 17 18:08:59.806883 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:08:59.806863    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cabundle-cert\" (UniqueName: \"kubernetes.io/configmap/13ea8edb-65c4-469c-a4fd-b26c6a217485-cabundle-cert\") pod \"tokenizer-template-args-test-kserve-79dbd77788-tr755\" (UID: \"13ea8edb-65c4-469c-a4fd-b26c6a217485\") " pod="e2e-test-llm-inference-service-0eb24714/tokenizer-template-args-test-kserve-79dbd77788-tr755"
Sep 17 18:08:59.808307 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:08:59.808285    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/13ea8edb-65c4-469c-a4fd-b26c6a217485-dshm\") pod \"tokenizer-template-args-test-kserve-79dbd77788-tr755\" (UID: \"13ea8edb-65c4-469c-a4fd-b26c6a217485\") " pod="e2e-test-llm-inference-service-0eb24714/tokenizer-template-args-test-kserve-79dbd77788-tr755"
Sep 17 18:08:59.808543 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:08:59.808524    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/13ea8edb-65c4-469c-a4fd-b26c6a217485-tls-certs\") pod \"tokenizer-template-args-test-kserve-79dbd77788-tr755\" (UID: \"13ea8edb-65c4-469c-a4fd-b26c6a217485\") " pod="e2e-test-llm-inference-service-0eb24714/tokenizer-template-args-test-kserve-79dbd77788-tr755"
Sep 17 18:08:59.818909 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:08:59.818887    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-zgdx5\" (UniqueName: \"kubernetes.io/projected/13ea8edb-65c4-469c-a4fd-b26c6a217485-kube-api-access-zgdx5\") pod \"tokenizer-template-args-test-kserve-79dbd77788-tr755\" (UID: \"13ea8edb-65c4-469c-a4fd-b26c6a217485\") " pod="e2e-test-llm-inference-service-0eb24714/tokenizer-template-args-test-kserve-79dbd77788-tr755"
Sep 17 18:08:59.930710 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:08:59.930673    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-test-llm-inference-service-0eb24714/tokenizer-template-args-test-kserve-79dbd77788-tr755"
Sep 17 18:08:59.986795 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:08:59.985547    2575 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["e2e-test-llm-inference-service-0eb24714/tokenizer-template-args-test-tokenizer-799dfc5746-4krxk"]
Sep 17 18:08:59.991503 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:08:59.991478    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-test-llm-inference-service-0eb24714/tokenizer-template-args-test-tokenizer-799dfc5746-4krxk"
Sep 17 18:09:00.011393 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:09:00.011321    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["e2e-test-llm-inference-service-0eb24714/tokenizer-template-args-test-tokenizer-799dfc5746-4krxk"]
Sep 17 18:09:00.025425 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:09:00.025395    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-djvzt_1d07e68d-9943-420f-a6a0-5f477792fabd/konnectivity-agent/0.log" limit=1024
Sep 17 18:09:00.108707 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:09:00.108662    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/59a1daa3-faab-4e8d-9fb3-225b89ef86bc-tls-certs\") pod \"tokenizer-template-args-test-tokenizer-799dfc5746-4krxk\" (UID: \"59a1daa3-faab-4e8d-9fb3-225b89ef86bc\") " pod="e2e-test-llm-inference-service-0eb24714/tokenizer-template-args-test-tokenizer-799dfc5746-4krxk"
Sep 17 18:09:00.108927 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:09:00.108778    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/59a1daa3-faab-4e8d-9fb3-225b89ef86bc-tmp-dir\") pod \"tokenizer-template-args-test-tokenizer-799dfc5746-4krxk\" (UID: \"59a1daa3-faab-4e8d-9fb3-225b89ef86bc\") " pod="e2e-test-llm-inference-service-0eb24714/tokenizer-template-args-test-tokenizer-799dfc5746-4krxk"
Sep 17 18:09:00.108927 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:09:00.108804    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cabundle-cert\" (UniqueName: \"kubernetes.io/configmap/59a1daa3-faab-4e8d-9fb3-225b89ef86bc-cabundle-cert\") pod \"tokenizer-template-args-test-tokenizer-799dfc5746-4krxk\" (UID: \"59a1daa3-faab-4e8d-9fb3-225b89ef86bc\") " pod="e2e-test-llm-inference-service-0eb24714/tokenizer-template-args-test-tokenizer-799dfc5746-4krxk"
Sep 17 18:09:00.108927 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:09:00.108851    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/59a1daa3-faab-4e8d-9fb3-225b89ef86bc-kserve-provision-location\") pod \"tokenizer-template-args-test-tokenizer-799dfc5746-4krxk\" (UID: \"59a1daa3-faab-4e8d-9fb3-225b89ef86bc\") " pod="e2e-test-llm-inference-service-0eb24714/tokenizer-template-args-test-tokenizer-799dfc5746-4krxk"
Sep 17 18:09:00.109470 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:09:00.109433    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-djvzt_1d07e68d-9943-420f-a6a0-5f477792fabd/konnectivity-agent/0.log" limit=1024
Sep 17 18:09:00.209871 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:09:00.209787    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/59a1daa3-faab-4e8d-9fb3-225b89ef86bc-tls-certs\") pod \"tokenizer-template-args-test-tokenizer-799dfc5746-4krxk\" (UID: \"59a1daa3-faab-4e8d-9fb3-225b89ef86bc\") " pod="e2e-test-llm-inference-service-0eb24714/tokenizer-template-args-test-tokenizer-799dfc5746-4krxk"
Sep 17 18:09:00.209871 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:09:00.209852    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/59a1daa3-faab-4e8d-9fb3-225b89ef86bc-tmp-dir\") pod \"tokenizer-template-args-test-tokenizer-799dfc5746-4krxk\" (UID: \"59a1daa3-faab-4e8d-9fb3-225b89ef86bc\") " pod="e2e-test-llm-inference-service-0eb24714/tokenizer-template-args-test-tokenizer-799dfc5746-4krxk"
Sep 17 18:09:00.209871 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:09:00.209869    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cabundle-cert\" (UniqueName: \"kubernetes.io/configmap/59a1daa3-faab-4e8d-9fb3-225b89ef86bc-cabundle-cert\") pod \"tokenizer-template-args-test-tokenizer-799dfc5746-4krxk\" (UID: \"59a1daa3-faab-4e8d-9fb3-225b89ef86bc\") " pod="e2e-test-llm-inference-service-0eb24714/tokenizer-template-args-test-tokenizer-799dfc5746-4krxk"
Sep 17 18:09:00.210101 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:09:00.209891    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/59a1daa3-faab-4e8d-9fb3-225b89ef86bc-kserve-provision-location\") pod \"tokenizer-template-args-test-tokenizer-799dfc5746-4krxk\" (UID: \"59a1daa3-faab-4e8d-9fb3-225b89ef86bc\") " pod="e2e-test-llm-inference-service-0eb24714/tokenizer-template-args-test-tokenizer-799dfc5746-4krxk"
Sep 17 18:09:00.210314 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:09:00.210298    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/59a1daa3-faab-4e8d-9fb3-225b89ef86bc-kserve-provision-location\") pod \"tokenizer-template-args-test-tokenizer-799dfc5746-4krxk\" (UID: \"59a1daa3-faab-4e8d-9fb3-225b89ef86bc\") " pod="e2e-test-llm-inference-service-0eb24714/tokenizer-template-args-test-tokenizer-799dfc5746-4krxk"
Sep 17 18:09:00.210351 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:09:00.210295    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/59a1daa3-faab-4e8d-9fb3-225b89ef86bc-tmp-dir\") pod \"tokenizer-template-args-test-tokenizer-799dfc5746-4krxk\" (UID: \"59a1daa3-faab-4e8d-9fb3-225b89ef86bc\") " pod="e2e-test-llm-inference-service-0eb24714/tokenizer-template-args-test-tokenizer-799dfc5746-4krxk"
Sep 17 18:09:00.210647 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:09:00.210628    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cabundle-cert\" (UniqueName: \"kubernetes.io/configmap/59a1daa3-faab-4e8d-9fb3-225b89ef86bc-cabundle-cert\") pod \"tokenizer-template-args-test-tokenizer-799dfc5746-4krxk\" (UID: \"59a1daa3-faab-4e8d-9fb3-225b89ef86bc\") " pod="e2e-test-llm-inference-service-0eb24714/tokenizer-template-args-test-tokenizer-799dfc5746-4krxk"
Sep 17 18:09:00.212341 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:09:00.212324    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/59a1daa3-faab-4e8d-9fb3-225b89ef86bc-tls-certs\") pod \"tokenizer-template-args-test-tokenizer-799dfc5746-4krxk\" (UID: \"59a1daa3-faab-4e8d-9fb3-225b89ef86bc\") " pod="e2e-test-llm-inference-service-0eb24714/tokenizer-template-args-test-tokenizer-799dfc5746-4krxk"
Sep 17 18:09:00.223082 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:09:00.223053    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["e2e-test-llm-inference-service-0eb24714/tokenizer-template-args-test-kserve-79dbd77788-tr755"]
Sep 17 18:09:00.224574 ip-10-0-143-74 kubenswrapper[2575]: W0917 18:09:00.224552    2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod13ea8edb_65c4_469c_a4fd_b26c6a217485.slice/crio-1fa805b43066c0f6c3bd4dd66bd7d122486146d32f0435c4214b855aa59cc312 WatchSource:0}: Error finding container 1fa805b43066c0f6c3bd4dd66bd7d122486146d32f0435c4214b855aa59cc312: Status 404 returned error can't find the container with id 1fa805b43066c0f6c3bd4dd66bd7d122486146d32f0435c4214b855aa59cc312
Sep 17 18:09:00.259642 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:09:00.259614    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-test-llm-inference-service-0eb24714/tokenizer-template-args-test-kserve-79dbd77788-tr755" event={"ID":"13ea8edb-65c4-469c-a4fd-b26c6a217485","Type":"ContainerStarted","Data":"1fa805b43066c0f6c3bd4dd66bd7d122486146d32f0435c4214b855aa59cc312"}
Sep 17 18:09:00.311834 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:09:00.311793    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-test-llm-inference-service-0eb24714/tokenizer-template-args-test-tokenizer-799dfc5746-4krxk"
Sep 17 18:09:00.600461 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:09:00.600341    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["e2e-test-llm-inference-service-0eb24714/tokenizer-template-args-test-tokenizer-799dfc5746-4krxk"]
Sep 17 18:09:00.603017 ip-10-0-143-74 kubenswrapper[2575]: W0917 18:09:00.602982    2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod59a1daa3_faab_4e8d_9fb3_225b89ef86bc.slice/crio-1a55eb5ca5d46d0482bdb00e4df1d0efab83198e0b4b95fd30efe9c89e42bea0 WatchSource:0}: Error finding container 1a55eb5ca5d46d0482bdb00e4df1d0efab83198e0b4b95fd30efe9c89e42bea0: Status 404 returned error can't find the container with id 1a55eb5ca5d46d0482bdb00e4df1d0efab83198e0b4b95fd30efe9c89e42bea0
Sep 17 18:09:01.268692 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:09:01.268652    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-test-llm-inference-service-0eb24714/tokenizer-template-args-test-kserve-79dbd77788-tr755" event={"ID":"13ea8edb-65c4-469c-a4fd-b26c6a217485","Type":"ContainerStarted","Data":"1e018d0e84c4aac5a12d670c2ea6636a7316627d39a0b247ca05c179a1141dc8"}
Sep 17 18:09:01.270268 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:09:01.270243    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-test-llm-inference-service-0eb24714/tokenizer-template-args-test-tokenizer-799dfc5746-4krxk" event={"ID":"59a1daa3-faab-4e8d-9fb3-225b89ef86bc","Type":"ContainerStarted","Data":"c3499c26281c65931fc540c0bf082eb7534cc777820e41ac330797ad985bc613"}
Sep 17 18:09:01.270391 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:09:01.270295    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-test-llm-inference-service-0eb24714/tokenizer-template-args-test-tokenizer-799dfc5746-4krxk" event={"ID":"59a1daa3-faab-4e8d-9fb3-225b89ef86bc","Type":"ContainerStarted","Data":"1a55eb5ca5d46d0482bdb00e4df1d0efab83198e0b4b95fd30efe9c89e42bea0"}
Sep 17 18:09:02.160741 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:09:02.160698    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-djvzt_1d07e68d-9943-420f-a6a0-5f477792fabd/konnectivity-agent/0.log" limit=1024
Sep 17 18:09:03.285775 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:09:03.285747    2575 generic.go:356] "Generic (PLEG): container finished" podID="13ea8edb-65c4-469c-a4fd-b26c6a217485" containerID="1e018d0e84c4aac5a12d670c2ea6636a7316627d39a0b247ca05c179a1141dc8" exitCode=0
Sep 17 18:09:03.286116 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:09:03.285815    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-test-llm-inference-service-0eb24714/tokenizer-template-args-test-kserve-79dbd77788-tr755" event={"ID":"13ea8edb-65c4-469c-a4fd-b26c6a217485","Type":"ContainerDied","Data":"1e018d0e84c4aac5a12d670c2ea6636a7316627d39a0b247ca05c179a1141dc8"}
Sep 17 18:09:03.287436 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:09:03.287418    2575 generic.go:356] "Generic (PLEG): container finished" podID="59a1daa3-faab-4e8d-9fb3-225b89ef86bc" containerID="c3499c26281c65931fc540c0bf082eb7534cc777820e41ac330797ad985bc613" exitCode=0
Sep 17 18:09:03.287557 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:09:03.287486    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-test-llm-inference-service-0eb24714/tokenizer-template-args-test-tokenizer-799dfc5746-4krxk" event={"ID":"59a1daa3-faab-4e8d-9fb3-225b89ef86bc","Type":"ContainerDied","Data":"c3499c26281c65931fc540c0bf082eb7534cc777820e41ac330797ad985bc613"}
Sep 17 18:09:04.295864 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:09:04.295827    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-test-llm-inference-service-0eb24714/tokenizer-template-args-test-kserve-79dbd77788-tr755" event={"ID":"13ea8edb-65c4-469c-a4fd-b26c6a217485","Type":"ContainerStarted","Data":"6ab8d7346507fca1bd6dea7ebc5ec9d8b698d0c62eeadd183351d869189df26d"}
Sep 17 18:09:04.297792 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:09:04.297757    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-test-llm-inference-service-0eb24714/tokenizer-template-args-test-tokenizer-799dfc5746-4krxk" event={"ID":"59a1daa3-faab-4e8d-9fb3-225b89ef86bc","Type":"ContainerStarted","Data":"3bc788e62cd86d850c5f991ccdf88113ce4dcf890e6713fb2e948895c00e9a8f"}
Sep 17 18:09:04.297989 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:09:04.297969    2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-test-llm-inference-service-0eb24714/tokenizer-template-args-test-tokenizer-799dfc5746-4krxk"
Sep 17 18:09:04.321783 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:09:04.321731    2575 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="e2e-test-llm-inference-service-0eb24714/tokenizer-template-args-test-kserve-79dbd77788-tr755" podStartSLOduration=2.708004943 podStartE2EDuration="5.321713642s" totalImagesPullingTime="0s" totalInitContainerRuntime="2.613708699s" isStatefulPod=false podCreationTimestamp="2026-09-17 18:08:59 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-17 18:09:04.320404177 +0000 UTC m=+2063.950734151" watchObservedRunningTime="2026-09-17 18:09:04.321713642 +0000 UTC m=+2063.952043619"
Sep 17 18:09:04.346679 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:09:04.346624    2575 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="e2e-test-llm-inference-service-0eb24714/tokenizer-template-args-test-tokenizer-799dfc5746-4krxk" podStartSLOduration=2.750305321 podStartE2EDuration="5.34660547s" totalImagesPullingTime="0s" totalInitContainerRuntime="2.596300149s" isStatefulPod=false podCreationTimestamp="2026-09-17 18:08:59 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-17 18:09:04.344677555 +0000 UTC m=+2063.975007531" watchObservedRunningTime="2026-09-17 18:09:04.34660547 +0000 UTC m=+2063.976935444"
Sep 17 18:09:04.931429 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:09:04.931393    2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-test-llm-inference-service-0eb24714/tokenizer-template-args-test-kserve-79dbd77788-tr755"
Sep 17 18:09:07.809049 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:09:07.809005    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-djvzt_1d07e68d-9943-420f-a6a0-5f477792fabd/konnectivity-agent/0.log" limit=1024
Sep 17 18:09:08.612628 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:09:08.612588    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-djvzt_1d07e68d-9943-420f-a6a0-5f477792fabd/konnectivity-agent/0.log" limit=1024
Sep 17 18:09:08.740799 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:09:08.740764    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-djvzt_1d07e68d-9943-420f-a6a0-5f477792fabd/konnectivity-agent/0.log" limit=1024
Sep 17 18:09:09.930924 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:09:09.930878    2575 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="e2e-test-llm-inference-service-0eb24714/tokenizer-template-args-test-kserve-79dbd77788-tr755"
Sep 17 18:09:09.943853 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:09:09.943818    2575 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="e2e-test-llm-inference-service-0eb24714/tokenizer-template-args-test-kserve-79dbd77788-tr755"
Sep 17 18:09:10.343291 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:09:10.343207    2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-test-llm-inference-service-0eb24714/tokenizer-template-args-test-kserve-79dbd77788-tr755"
Sep 17 18:09:33.799416 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:09:33.799383    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-djvzt_1d07e68d-9943-420f-a6a0-5f477792fabd/konnectivity-agent/0.log" limit=1024
Sep 17 18:09:33.896967 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:09:33.896931    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-djvzt_1d07e68d-9943-420f-a6a0-5f477792fabd/konnectivity-agent/0.log" limit=1024
Sep 17 18:09:34.436912 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:09:34.436877    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-djvzt_1d07e68d-9943-420f-a6a0-5f477792fabd/konnectivity-agent/0.log" limit=1024
Sep 17 18:09:35.314203 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:09:35.314163    2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-test-llm-inference-service-0eb24714/tokenizer-template-args-test-tokenizer-799dfc5746-4krxk"
Sep 17 18:09:40.397212 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:09:40.397154    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-djvzt_1d07e68d-9943-420f-a6a0-5f477792fabd/konnectivity-agent/0.log" limit=1024
Sep 17 18:09:45.105472 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:09:45.105400    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-djvzt_1d07e68d-9943-420f-a6a0-5f477792fabd/konnectivity-agent/0.log" limit=1024
Sep 17 18:09:45.303470 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:09:45.303411    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-djvzt_1d07e68d-9943-420f-a6a0-5f477792fabd/konnectivity-agent/0.log" limit=1024
Sep 17 18:09:47.244414 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:09:47.244374    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-djvzt_1d07e68d-9943-420f-a6a0-5f477792fabd/konnectivity-agent/0.log" limit=1024
Sep 17 18:09:47.342902 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:09:47.342861    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-djvzt_1d07e68d-9943-420f-a6a0-5f477792fabd/konnectivity-agent/0.log" limit=1024
Sep 17 18:10:02.815830 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:10:02.815789    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-djvzt_1d07e68d-9943-420f-a6a0-5f477792fabd/konnectivity-agent/0.log" limit=1024
Sep 17 18:10:05.923164 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:10:05.923124    2575 log.go:25] "Finished parsing log file" path="/var/log/pods/e2e-test-llm-inference-service-0eb24714_tokenizer-template-args-test-tokenizer-799dfc5746-4krxk_59a1daa3-faab-4e8d-9fb3-225b89ef86bc/main/0.log"
Sep 17 18:10:07.480184 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:10:07.480148    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-djvzt_1d07e68d-9943-420f-a6a0-5f477792fabd/konnectivity-agent/0.log" limit=1024
Sep 17 18:10:07.496532 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:10:07.496495    2575 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["e2e-test-llm-inference-service-0eb24714/tokenizer-template-args-test-kserve-79dbd77788-tr755"]
Sep 17 18:10:07.507663 ip-10-0-143-74 kubenswrapper[2575]: E0917 18:10:07.507622    2575 log.go:32] "ExecSync cmd from runtime service failed" err=<
Sep 17 18:10:07.507663 ip-10-0-143-74 kubenswrapper[2575]:         rpc error: code = Unknown desc = command error: executable file `/bin/sleep` not found: No such file or directory
Sep 17 18:10:07.507663 ip-10-0-143-74 kubenswrapper[2575]:         , stdout: , stderr: , exit code -1
Sep 17 18:10:07.507663 ip-10-0-143-74 kubenswrapper[2575]:  > containerID="6ab8d7346507fca1bd6dea7ebc5ec9d8b698d0c62eeadd183351d869189df26d" cmd=["/bin/sleep","15"]
Sep 17 18:10:07.507978 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:10:07.507686    2575 handlers.go:82] "Exec lifecycle hook for Container in Pod failed" execCommand=["/bin/sleep","15"] containerName="main" pod="e2e-test-llm-inference-service-0eb24714/tokenizer-template-args-test-kserve-79dbd77788-tr755" message="" err=<
Sep 17 18:10:07.507978 ip-10-0-143-74 kubenswrapper[2575]:         rpc error: code = Unknown desc = command error: executable file `/bin/sleep` not found: No such file or directory
Sep 17 18:10:07.507978 ip-10-0-143-74 kubenswrapper[2575]:         , stdout: , stderr: , exit code -1
Sep 17 18:10:07.507978 ip-10-0-143-74 kubenswrapper[2575]:  >
Sep 17 18:10:07.507978 ip-10-0-143-74 kubenswrapper[2575]: E0917 18:10:07.507706    2575 kuberuntime_container.go:755] "PreStop hook failed" err=<
Sep 17 18:10:07.507978 ip-10-0-143-74 kubenswrapper[2575]:         rpc error: code = Unknown desc = command error: executable file `/bin/sleep` not found: No such file or directory
Sep 17 18:10:07.507978 ip-10-0-143-74 kubenswrapper[2575]:         , stdout: , stderr: , exit code -1
Sep 17 18:10:07.507978 ip-10-0-143-74 kubenswrapper[2575]:  > pod="e2e-test-llm-inference-service-0eb24714/tokenizer-template-args-test-kserve-79dbd77788-tr755" podUID="13ea8edb-65c4-469c-a4fd-b26c6a217485" containerName="main" containerID="cri-o://6ab8d7346507fca1bd6dea7ebc5ec9d8b698d0c62eeadd183351d869189df26d"
Sep 17 18:10:07.507978 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:10:07.507781    2575 kuberuntime_container.go:873] "Killing container with a grace period" pod="e2e-test-llm-inference-service-0eb24714/tokenizer-template-args-test-kserve-79dbd77788-tr755" podUID="13ea8edb-65c4-469c-a4fd-b26c6a217485" containerName="main" containerID="cri-o://6ab8d7346507fca1bd6dea7ebc5ec9d8b698d0c62eeadd183351d869189df26d" gracePeriod=60
Sep 17 18:10:07.510625 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:10:07.510596    2575 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["e2e-test-llm-inference-service-0eb24714/tokenizer-template-args-test-tokenizer-799dfc5746-4krxk"]
Sep 17 18:10:07.510962 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:10:07.510926    2575 kuberuntime_container.go:873] "Killing container with a grace period" pod="e2e-test-llm-inference-service-0eb24714/tokenizer-template-args-test-tokenizer-799dfc5746-4krxk" podUID="59a1daa3-faab-4e8d-9fb3-225b89ef86bc" containerName="main" containerID="cri-o://3bc788e62cd86d850c5f991ccdf88113ce4dcf890e6713fb2e948895c00e9a8f" gracePeriod=30
Sep 17 18:10:07.574490 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:10:07.574433    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-djvzt_1d07e68d-9943-420f-a6a0-5f477792fabd/konnectivity-agent/0.log" limit=1024
Sep 17 18:10:07.658561 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:10:07.658521    2575 generic.go:356] "Generic (PLEG): container finished" podID="13ea8edb-65c4-469c-a4fd-b26c6a217485" containerID="6ab8d7346507fca1bd6dea7ebc5ec9d8b698d0c62eeadd183351d869189df26d" exitCode=0
Sep 17 18:10:07.658720 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:10:07.658590    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-test-llm-inference-service-0eb24714/tokenizer-template-args-test-kserve-79dbd77788-tr755" event={"ID":"13ea8edb-65c4-469c-a4fd-b26c6a217485","Type":"ContainerDied","Data":"6ab8d7346507fca1bd6dea7ebc5ec9d8b698d0c62eeadd183351d869189df26d"}
Sep 17 18:10:07.778003 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:10:07.777972    2575 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-test-llm-inference-service-0eb24714/tokenizer-template-args-test-kserve-79dbd77788-tr755"
Sep 17 18:10:07.939869 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:10:07.939819    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/13ea8edb-65c4-469c-a4fd-b26c6a217485-dshm\") pod \"13ea8edb-65c4-469c-a4fd-b26c6a217485\" (UID: \"13ea8edb-65c4-469c-a4fd-b26c6a217485\") "
Sep 17 18:10:07.940071 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:10:07.939878    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/13ea8edb-65c4-469c-a4fd-b26c6a217485-model-cache\") pod \"13ea8edb-65c4-469c-a4fd-b26c6a217485\" (UID: \"13ea8edb-65c4-469c-a4fd-b26c6a217485\") "
Sep 17 18:10:07.940071 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:10:07.939908    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-zgdx5\" (UniqueName: \"kubernetes.io/projected/13ea8edb-65c4-469c-a4fd-b26c6a217485-kube-api-access-zgdx5\") pod \"13ea8edb-65c4-469c-a4fd-b26c6a217485\" (UID: \"13ea8edb-65c4-469c-a4fd-b26c6a217485\") "
Sep 17 18:10:07.940071 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:10:07.939967    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"cabundle-cert\" (UniqueName: \"kubernetes.io/configmap/13ea8edb-65c4-469c-a4fd-b26c6a217485-cabundle-cert\") pod \"13ea8edb-65c4-469c-a4fd-b26c6a217485\" (UID: \"13ea8edb-65c4-469c-a4fd-b26c6a217485\") "
Sep 17 18:10:07.940071 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:10:07.939995    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/13ea8edb-65c4-469c-a4fd-b26c6a217485-home\") pod \"13ea8edb-65c4-469c-a4fd-b26c6a217485\" (UID: \"13ea8edb-65c4-469c-a4fd-b26c6a217485\") "
Sep 17 18:10:07.940071 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:10:07.940022    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/13ea8edb-65c4-469c-a4fd-b26c6a217485-tls-certs\") pod \"13ea8edb-65c4-469c-a4fd-b26c6a217485\" (UID: \"13ea8edb-65c4-469c-a4fd-b26c6a217485\") "
Sep 17 18:10:07.940368 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:10:07.940091    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/13ea8edb-65c4-469c-a4fd-b26c6a217485-kserve-provision-location\") pod \"13ea8edb-65c4-469c-a4fd-b26c6a217485\" (UID: \"13ea8edb-65c4-469c-a4fd-b26c6a217485\") "
Sep 17 18:10:07.940368 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:10:07.940137    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/13ea8edb-65c4-469c-a4fd-b26c6a217485-tmp-dir\") pod \"13ea8edb-65c4-469c-a4fd-b26c6a217485\" (UID: \"13ea8edb-65c4-469c-a4fd-b26c6a217485\") "
Sep 17 18:10:07.940495 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:10:07.940367    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/13ea8edb-65c4-469c-a4fd-b26c6a217485-cabundle-cert" (OuterVolumeSpecName: "cabundle-cert") pod "13ea8edb-65c4-469c-a4fd-b26c6a217485" (UID: "13ea8edb-65c4-469c-a4fd-b26c6a217485"). InnerVolumeSpecName "cabundle-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 17 18:10:07.942625 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:10:07.940818    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/13ea8edb-65c4-469c-a4fd-b26c6a217485-home" (OuterVolumeSpecName: "home") pod "13ea8edb-65c4-469c-a4fd-b26c6a217485" (UID: "13ea8edb-65c4-469c-a4fd-b26c6a217485"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 17 18:10:07.942625 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:10:07.940986    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/13ea8edb-65c4-469c-a4fd-b26c6a217485-model-cache" (OuterVolumeSpecName: "model-cache") pod "13ea8edb-65c4-469c-a4fd-b26c6a217485" (UID: "13ea8edb-65c4-469c-a4fd-b26c6a217485"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 17 18:10:07.942812 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:10:07.942779    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/13ea8edb-65c4-469c-a4fd-b26c6a217485-dshm" (OuterVolumeSpecName: "dshm") pod "13ea8edb-65c4-469c-a4fd-b26c6a217485" (UID: "13ea8edb-65c4-469c-a4fd-b26c6a217485"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 17 18:10:07.943260 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:10:07.943226    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/13ea8edb-65c4-469c-a4fd-b26c6a217485-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "13ea8edb-65c4-469c-a4fd-b26c6a217485" (UID: "13ea8edb-65c4-469c-a4fd-b26c6a217485"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 17 18:10:07.943611 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:10:07.943577    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/13ea8edb-65c4-469c-a4fd-b26c6a217485-kube-api-access-zgdx5" (OuterVolumeSpecName: "kube-api-access-zgdx5") pod "13ea8edb-65c4-469c-a4fd-b26c6a217485" (UID: "13ea8edb-65c4-469c-a4fd-b26c6a217485"). InnerVolumeSpecName "kube-api-access-zgdx5". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 17 18:10:07.944653 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:10:07.944627    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/13ea8edb-65c4-469c-a4fd-b26c6a217485-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "13ea8edb-65c4-469c-a4fd-b26c6a217485" (UID: "13ea8edb-65c4-469c-a4fd-b26c6a217485"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 17 18:10:07.964618 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:10:07.964586    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/13ea8edb-65c4-469c-a4fd-b26c6a217485-kserve-provision-location" (OuterVolumeSpecName: "kserve-provision-location") pod "13ea8edb-65c4-469c-a4fd-b26c6a217485" (UID: "13ea8edb-65c4-469c-a4fd-b26c6a217485"). InnerVolumeSpecName "kserve-provision-location". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 17 18:10:08.041150 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:10:08.041115    2575 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/13ea8edb-65c4-469c-a4fd-b26c6a217485-tmp-dir\") on node \"ip-10-0-143-74.ec2.internal\" DevicePath \"\""
Sep 17 18:10:08.041150 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:10:08.041150    2575 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/13ea8edb-65c4-469c-a4fd-b26c6a217485-dshm\") on node \"ip-10-0-143-74.ec2.internal\" DevicePath \"\""
Sep 17 18:10:08.041321 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:10:08.041163    2575 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/13ea8edb-65c4-469c-a4fd-b26c6a217485-model-cache\") on node \"ip-10-0-143-74.ec2.internal\" DevicePath \"\""
Sep 17 18:10:08.041321 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:10:08.041177    2575 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-zgdx5\" (UniqueName: \"kubernetes.io/projected/13ea8edb-65c4-469c-a4fd-b26c6a217485-kube-api-access-zgdx5\") on node \"ip-10-0-143-74.ec2.internal\" DevicePath \"\""
Sep 17 18:10:08.041321 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:10:08.041189    2575 reconciler_common.go:299] "Volume detached for volume \"cabundle-cert\" (UniqueName: \"kubernetes.io/configmap/13ea8edb-65c4-469c-a4fd-b26c6a217485-cabundle-cert\") on node \"ip-10-0-143-74.ec2.internal\" DevicePath \"\""
Sep 17 18:10:08.041321 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:10:08.041201    2575 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/13ea8edb-65c4-469c-a4fd-b26c6a217485-home\") on node \"ip-10-0-143-74.ec2.internal\" DevicePath \"\""
Sep 17 18:10:08.041321 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:10:08.041212    2575 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/13ea8edb-65c4-469c-a4fd-b26c6a217485-tls-certs\") on node \"ip-10-0-143-74.ec2.internal\" DevicePath \"\""
Sep 17 18:10:08.041321 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:10:08.041224    2575 reconciler_common.go:299] "Volume detached for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/13ea8edb-65c4-469c-a4fd-b26c6a217485-kserve-provision-location\") on node \"ip-10-0-143-74.ec2.internal\" DevicePath \"\""
Sep 17 18:10:08.162637 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:10:08.162609    2575 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-test-llm-inference-service-0eb24714/tokenizer-template-args-test-tokenizer-799dfc5746-4krxk"
Sep 17 18:10:08.343232 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:10:08.343123    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/59a1daa3-faab-4e8d-9fb3-225b89ef86bc-tls-certs\") pod \"59a1daa3-faab-4e8d-9fb3-225b89ef86bc\" (UID: \"59a1daa3-faab-4e8d-9fb3-225b89ef86bc\") "
Sep 17 18:10:08.343409 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:10:08.343242    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/59a1daa3-faab-4e8d-9fb3-225b89ef86bc-tmp-dir\") pod \"59a1daa3-faab-4e8d-9fb3-225b89ef86bc\" (UID: \"59a1daa3-faab-4e8d-9fb3-225b89ef86bc\") "
Sep 17 18:10:08.343409 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:10:08.343301    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"cabundle-cert\" (UniqueName: \"kubernetes.io/configmap/59a1daa3-faab-4e8d-9fb3-225b89ef86bc-cabundle-cert\") pod \"59a1daa3-faab-4e8d-9fb3-225b89ef86bc\" (UID: \"59a1daa3-faab-4e8d-9fb3-225b89ef86bc\") "
Sep 17 18:10:08.343409 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:10:08.343334    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/59a1daa3-faab-4e8d-9fb3-225b89ef86bc-kserve-provision-location\") pod \"59a1daa3-faab-4e8d-9fb3-225b89ef86bc\" (UID: \"59a1daa3-faab-4e8d-9fb3-225b89ef86bc\") "
Sep 17 18:10:08.343774 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:10:08.343727    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/59a1daa3-faab-4e8d-9fb3-225b89ef86bc-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "59a1daa3-faab-4e8d-9fb3-225b89ef86bc" (UID: "59a1daa3-faab-4e8d-9fb3-225b89ef86bc"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 17 18:10:08.343774 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:10:08.343738    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/59a1daa3-faab-4e8d-9fb3-225b89ef86bc-cabundle-cert" (OuterVolumeSpecName: "cabundle-cert") pod "59a1daa3-faab-4e8d-9fb3-225b89ef86bc" (UID: "59a1daa3-faab-4e8d-9fb3-225b89ef86bc"). InnerVolumeSpecName "cabundle-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 17 18:10:08.345395 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:10:08.345368    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/59a1daa3-faab-4e8d-9fb3-225b89ef86bc-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "59a1daa3-faab-4e8d-9fb3-225b89ef86bc" (UID: "59a1daa3-faab-4e8d-9fb3-225b89ef86bc"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 17 18:10:08.368069 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:10:08.368018    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/59a1daa3-faab-4e8d-9fb3-225b89ef86bc-kserve-provision-location" (OuterVolumeSpecName: "kserve-provision-location") pod "59a1daa3-faab-4e8d-9fb3-225b89ef86bc" (UID: "59a1daa3-faab-4e8d-9fb3-225b89ef86bc"). InnerVolumeSpecName "kserve-provision-location". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 17 18:10:08.444784 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:10:08.444747    2575 reconciler_common.go:299] "Volume detached for volume \"cabundle-cert\" (UniqueName: \"kubernetes.io/configmap/59a1daa3-faab-4e8d-9fb3-225b89ef86bc-cabundle-cert\") on node \"ip-10-0-143-74.ec2.internal\" DevicePath \"\""
Sep 17 18:10:08.444784 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:10:08.444780    2575 reconciler_common.go:299] "Volume detached for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/59a1daa3-faab-4e8d-9fb3-225b89ef86bc-kserve-provision-location\") on node \"ip-10-0-143-74.ec2.internal\" DevicePath \"\""
Sep 17 18:10:08.444784 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:10:08.444791    2575 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/59a1daa3-faab-4e8d-9fb3-225b89ef86bc-tls-certs\") on node \"ip-10-0-143-74.ec2.internal\" DevicePath \"\""
Sep 17 18:10:08.445023 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:10:08.444802    2575 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/59a1daa3-faab-4e8d-9fb3-225b89ef86bc-tmp-dir\") on node \"ip-10-0-143-74.ec2.internal\" DevicePath \"\""
Sep 17 18:10:08.665503 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:10:08.665466    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-test-llm-inference-service-0eb24714/tokenizer-template-args-test-kserve-79dbd77788-tr755" event={"ID":"13ea8edb-65c4-469c-a4fd-b26c6a217485","Type":"ContainerDied","Data":"1fa805b43066c0f6c3bd4dd66bd7d122486146d32f0435c4214b855aa59cc312"}
Sep 17 18:10:08.665984 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:10:08.665530    2575 scope.go:117] "RemoveContainer" containerID="6ab8d7346507fca1bd6dea7ebc5ec9d8b698d0c62eeadd183351d869189df26d"
Sep 17 18:10:08.665984 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:10:08.665549    2575 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-test-llm-inference-service-0eb24714/tokenizer-template-args-test-kserve-79dbd77788-tr755"
Sep 17 18:10:08.667165 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:10:08.667131    2575 generic.go:356] "Generic (PLEG): container finished" podID="59a1daa3-faab-4e8d-9fb3-225b89ef86bc" containerID="3bc788e62cd86d850c5f991ccdf88113ce4dcf890e6713fb2e948895c00e9a8f" exitCode=0
Sep 17 18:10:08.667231 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:10:08.667189    2575 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-test-llm-inference-service-0eb24714/tokenizer-template-args-test-tokenizer-799dfc5746-4krxk"
Sep 17 18:10:08.667292 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:10:08.667268    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-test-llm-inference-service-0eb24714/tokenizer-template-args-test-tokenizer-799dfc5746-4krxk" event={"ID":"59a1daa3-faab-4e8d-9fb3-225b89ef86bc","Type":"ContainerDied","Data":"3bc788e62cd86d850c5f991ccdf88113ce4dcf890e6713fb2e948895c00e9a8f"}
Sep 17 18:10:08.667346 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:10:08.667323    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-test-llm-inference-service-0eb24714/tokenizer-template-args-test-tokenizer-799dfc5746-4krxk" event={"ID":"59a1daa3-faab-4e8d-9fb3-225b89ef86bc","Type":"ContainerDied","Data":"1a55eb5ca5d46d0482bdb00e4df1d0efab83198e0b4b95fd30efe9c89e42bea0"}
Sep 17 18:10:08.678272 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:10:08.678212    2575 scope.go:117] "RemoveContainer" containerID="1e018d0e84c4aac5a12d670c2ea6636a7316627d39a0b247ca05c179a1141dc8"
Sep 17 18:10:08.687470 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:10:08.687434    2575 scope.go:117] "RemoveContainer" containerID="3bc788e62cd86d850c5f991ccdf88113ce4dcf890e6713fb2e948895c00e9a8f"
Sep 17 18:10:08.694456 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:10:08.694419    2575 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["e2e-test-llm-inference-service-0eb24714/tokenizer-template-args-test-tokenizer-799dfc5746-4krxk"]
Sep 17 18:10:08.696133 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:10:08.696088    2575 scope.go:117] "RemoveContainer" containerID="c3499c26281c65931fc540c0bf082eb7534cc777820e41ac330797ad985bc613"
Sep 17 18:10:08.698747 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:10:08.698722    2575 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["e2e-test-llm-inference-service-0eb24714/tokenizer-template-args-test-tokenizer-799dfc5746-4krxk"]
Sep 17 18:10:08.704357 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:10:08.704338    2575 scope.go:117] "RemoveContainer" containerID="3bc788e62cd86d850c5f991ccdf88113ce4dcf890e6713fb2e948895c00e9a8f"
Sep 17 18:10:08.704676 ip-10-0-143-74 kubenswrapper[2575]: E0917 18:10:08.704652    2575 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"3bc788e62cd86d850c5f991ccdf88113ce4dcf890e6713fb2e948895c00e9a8f\": container with ID starting with 3bc788e62cd86d850c5f991ccdf88113ce4dcf890e6713fb2e948895c00e9a8f not found: ID does not exist" containerID="3bc788e62cd86d850c5f991ccdf88113ce4dcf890e6713fb2e948895c00e9a8f"
Sep 17 18:10:08.704748 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:10:08.704686    2575 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"3bc788e62cd86d850c5f991ccdf88113ce4dcf890e6713fb2e948895c00e9a8f"} err="failed to get container status \"3bc788e62cd86d850c5f991ccdf88113ce4dcf890e6713fb2e948895c00e9a8f\": rpc error: code = NotFound desc = could not find container \"3bc788e62cd86d850c5f991ccdf88113ce4dcf890e6713fb2e948895c00e9a8f\": container with ID starting with 3bc788e62cd86d850c5f991ccdf88113ce4dcf890e6713fb2e948895c00e9a8f not found: ID does not exist"
Sep 17 18:10:08.704748 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:10:08.704707    2575 scope.go:117] "RemoveContainer" containerID="c3499c26281c65931fc540c0bf082eb7534cc777820e41ac330797ad985bc613"
Sep 17 18:10:08.704997 ip-10-0-143-74 kubenswrapper[2575]: E0917 18:10:08.704974    2575 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"c3499c26281c65931fc540c0bf082eb7534cc777820e41ac330797ad985bc613\": container with ID starting with c3499c26281c65931fc540c0bf082eb7534cc777820e41ac330797ad985bc613 not found: ID does not exist" containerID="c3499c26281c65931fc540c0bf082eb7534cc777820e41ac330797ad985bc613"
Sep 17 18:10:08.705037 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:10:08.705008    2575 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"c3499c26281c65931fc540c0bf082eb7534cc777820e41ac330797ad985bc613"} err="failed to get container status \"c3499c26281c65931fc540c0bf082eb7534cc777820e41ac330797ad985bc613\": rpc error: code = NotFound desc = could not find container \"c3499c26281c65931fc540c0bf082eb7534cc777820e41ac330797ad985bc613\": container with ID starting with c3499c26281c65931fc540c0bf082eb7534cc777820e41ac330797ad985bc613 not found: ID does not exist"
Sep 17 18:10:08.711347 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:10:08.711326    2575 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["e2e-test-llm-inference-service-0eb24714/tokenizer-template-args-test-kserve-79dbd77788-tr755"]
Sep 17 18:10:08.715948 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:10:08.715925    2575 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["e2e-test-llm-inference-service-0eb24714/tokenizer-template-args-test-kserve-79dbd77788-tr755"]
Sep 17 18:10:08.967789 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:10:08.967714    2575 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="13ea8edb-65c4-469c-a4fd-b26c6a217485" path="/var/lib/kubelet/pods/13ea8edb-65c4-469c-a4fd-b26c6a217485/volumes"
Sep 17 18:10:08.968184 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:10:08.968164    2575 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="59a1daa3-faab-4e8d-9fb3-225b89ef86bc" path="/var/lib/kubelet/pods/59a1daa3-faab-4e8d-9fb3-225b89ef86bc/volumes"
Sep 17 18:10:56.239430 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:10:56.239345    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-djvzt_1d07e68d-9943-420f-a6a0-5f477792fabd/konnectivity-agent/0.log" limit=1024
Sep 17 18:10:57.736662 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:10:57.736623    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-djvzt_1d07e68d-9943-420f-a6a0-5f477792fabd/konnectivity-agent/0.log" limit=1024
Sep 17 18:11:02.444001 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:11:02.443960    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-djvzt_1d07e68d-9943-420f-a6a0-5f477792fabd/konnectivity-agent/0.log" limit=1024
Sep 17 18:11:08.706259 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:11:08.706220    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-djvzt_1d07e68d-9943-420f-a6a0-5f477792fabd/konnectivity-agent/0.log" limit=1024
Sep 17 18:11:08.834517 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:11:08.834481    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-djvzt_1d07e68d-9943-420f-a6a0-5f477792fabd/konnectivity-agent/0.log" limit=1024
Sep 17 18:11:13.225124 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:11:13.225084    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-djvzt_1d07e68d-9943-420f-a6a0-5f477792fabd/konnectivity-agent/0.log" limit=1024
Sep 17 18:11:13.428273 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:11:13.428215    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-djvzt_1d07e68d-9943-420f-a6a0-5f477792fabd/konnectivity-agent/0.log" limit=1024
Sep 17 18:11:46.487155 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:11:46.487107    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-djvzt_1d07e68d-9943-420f-a6a0-5f477792fabd/konnectivity-agent/0.log" limit=1024
Sep 17 18:11:49.825402 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:11:49.825361    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-djvzt_1d07e68d-9943-420f-a6a0-5f477792fabd/konnectivity-agent/0.log" limit=1024
Sep 17 18:11:49.925953 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:11:49.925883    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-djvzt_1d07e68d-9943-420f-a6a0-5f477792fabd/konnectivity-agent/0.log" limit=1024
Sep 17 18:11:52.690020 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:11:52.689975    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-djvzt_1d07e68d-9943-420f-a6a0-5f477792fabd/konnectivity-agent/0.log" limit=1024
Sep 17 18:11:52.755681 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:11:52.755636    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-djvzt_1d07e68d-9943-420f-a6a0-5f477792fabd/konnectivity-agent/0.log" limit=1024
Sep 17 18:12:39.728491 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:12:39.728454    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-djvzt_1d07e68d-9943-420f-a6a0-5f477792fabd/konnectivity-agent/0.log" limit=1024
Sep 17 18:12:39.853316 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:12:39.853279    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-djvzt_1d07e68d-9943-420f-a6a0-5f477792fabd/konnectivity-agent/0.log" limit=1024
Sep 17 18:12:42.797571 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:12:42.797535    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-djvzt_1d07e68d-9943-420f-a6a0-5f477792fabd/konnectivity-agent/0.log" limit=1024
Sep 17 18:12:42.910210 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:12:42.910174    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-djvzt_1d07e68d-9943-420f-a6a0-5f477792fabd/konnectivity-agent/0.log" limit=1024
Sep 17 18:12:43.187429 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:12:43.187394    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-djvzt_1d07e68d-9943-420f-a6a0-5f477792fabd/konnectivity-agent/0.log" limit=1024
Sep 17 18:12:43.297463 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:12:43.297418    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-djvzt_1d07e68d-9943-420f-a6a0-5f477792fabd/konnectivity-agent/0.log" limit=1024
Sep 17 18:12:46.376656 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:12:46.376619    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-djvzt_1d07e68d-9943-420f-a6a0-5f477792fabd/konnectivity-agent/0.log" limit=1024
Sep 17 18:12:46.589402 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:12:46.589357    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-djvzt_1d07e68d-9943-420f-a6a0-5f477792fabd/konnectivity-agent/0.log" limit=1024
Sep 17 18:12:49.059952 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:12:49.059904    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-djvzt_1d07e68d-9943-420f-a6a0-5f477792fabd/konnectivity-agent/0.log" limit=1024
Sep 17 18:12:52.043754 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:12:52.043716    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-djvzt_1d07e68d-9943-420f-a6a0-5f477792fabd/konnectivity-agent/0.log" limit=1024
Sep 17 18:12:52.166262 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:12:52.166225    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-djvzt_1d07e68d-9943-420f-a6a0-5f477792fabd/konnectivity-agent/0.log" limit=1024
Sep 17 18:12:54.838952 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:12:54.838910    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-djvzt_1d07e68d-9943-420f-a6a0-5f477792fabd/konnectivity-agent/0.log" limit=1024
Sep 17 18:12:54.945593 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:12:54.945554    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-djvzt_1d07e68d-9943-420f-a6a0-5f477792fabd/konnectivity-agent/0.log" limit=1024
Sep 17 18:12:55.945383 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:12:55.945352    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-djvzt_1d07e68d-9943-420f-a6a0-5f477792fabd/konnectivity-agent/0.log" limit=1024
Sep 17 18:12:56.122642 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:12:56.122610    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-djvzt_1d07e68d-9943-420f-a6a0-5f477792fabd/konnectivity-agent/0.log" limit=1024
Sep 17 18:13:05.316293 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:13:05.316254    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-djvzt_1d07e68d-9943-420f-a6a0-5f477792fabd/konnectivity-agent/0.log" limit=1024
Sep 17 18:13:30.145596 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:13:30.145557    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-djvzt_1d07e68d-9943-420f-a6a0-5f477792fabd/konnectivity-agent/0.log" limit=1024
Sep 17 18:13:51.387938 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:13:51.387899    2575 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["e2e-test-llm-inference-service-84370a53/llmisvc-model-qwen2-5-0-5b-rout-a50492e9-kserve-5cf996c9955kpnw"]
Sep 17 18:13:51.388336 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:13:51.388281    2575 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="59a1daa3-faab-4e8d-9fb3-225b89ef86bc" containerName="main"
Sep 17 18:13:51.388336 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:13:51.388292    2575 state_mem.go:107] "Deleted CPUSet assignment" podUID="59a1daa3-faab-4e8d-9fb3-225b89ef86bc" containerName="main"
Sep 17 18:13:51.388336 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:13:51.388311    2575 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="13ea8edb-65c4-469c-a4fd-b26c6a217485" containerName="main"
Sep 17 18:13:51.388336 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:13:51.388316    2575 state_mem.go:107] "Deleted CPUSet assignment" podUID="13ea8edb-65c4-469c-a4fd-b26c6a217485" containerName="main"
Sep 17 18:13:51.388336 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:13:51.388330    2575 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="13ea8edb-65c4-469c-a4fd-b26c6a217485" containerName="storage-initializer"
Sep 17 18:13:51.388336 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:13:51.388336    2575 state_mem.go:107] "Deleted CPUSet assignment" podUID="13ea8edb-65c4-469c-a4fd-b26c6a217485" containerName="storage-initializer"
Sep 17 18:13:51.388585 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:13:51.388346    2575 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="59a1daa3-faab-4e8d-9fb3-225b89ef86bc" containerName="storage-initializer"
Sep 17 18:13:51.388585 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:13:51.388351    2575 state_mem.go:107] "Deleted CPUSet assignment" podUID="59a1daa3-faab-4e8d-9fb3-225b89ef86bc" containerName="storage-initializer"
Sep 17 18:13:51.388585 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:13:51.388408    2575 memory_manager.go:368] "RemoveStaleState removing state" podUID="59a1daa3-faab-4e8d-9fb3-225b89ef86bc" containerName="main"
Sep 17 18:13:51.388585 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:13:51.388416    2575 memory_manager.go:368] "RemoveStaleState removing state" podUID="13ea8edb-65c4-469c-a4fd-b26c6a217485" containerName="main"
Sep 17 18:13:51.391485 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:13:51.391458    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-test-llm-inference-service-84370a53/llmisvc-model-qwen2-5-0-5b-rout-a50492e9-kserve-5cf996c9955kpnw"
Sep 17 18:13:51.395273 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:13:51.395240    2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-test-llm-inference-service-84370a53\"/\"kube-root-ca.crt\""
Sep 17 18:13:51.395431 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:13:51.395339    2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-test-llm-inference-service-84370a53\"/\"openshift-service-ca.crt\""
Sep 17 18:13:51.395431 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:13:51.395241    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-test-llm-inference-service-84370a53\"/\"default-dockercfg-m748j\""
Sep 17 18:13:51.395583 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:13:51.395529    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-test-llm-inference-service-84370a53\"/\"llmisvb11a9c9832b99b016bc8f8e0ea095712-kserve-self-signed-certs\""
Sep 17 18:13:51.403134 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:13:51.403103    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["e2e-test-llm-inference-service-84370a53/llmisvc-model-qwen2-5-0-5b-rout-a50492e9-kserve-5cf996c9955kpnw"]
Sep 17 18:13:51.419389 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:13:51.419356    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-djvzt_1d07e68d-9943-420f-a6a0-5f477792fabd/konnectivity-agent/0.log" limit=1024
Sep 17 18:13:51.526066 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:13:51.526017    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/43c47688-adb1-4545-b4e6-f180e5f552e6-model-cache\") pod \"llmisvc-model-qwen2-5-0-5b-rout-a50492e9-kserve-5cf996c9955kpnw\" (UID: \"43c47688-adb1-4545-b4e6-f180e5f552e6\") " pod="e2e-test-llm-inference-service-84370a53/llmisvc-model-qwen2-5-0-5b-rout-a50492e9-kserve-5cf996c9955kpnw"
Sep 17 18:13:51.526286 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:13:51.526116    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qwklv\" (UniqueName: \"kubernetes.io/projected/43c47688-adb1-4545-b4e6-f180e5f552e6-kube-api-access-qwklv\") pod \"llmisvc-model-qwen2-5-0-5b-rout-a50492e9-kserve-5cf996c9955kpnw\" (UID: \"43c47688-adb1-4545-b4e6-f180e5f552e6\") " pod="e2e-test-llm-inference-service-84370a53/llmisvc-model-qwen2-5-0-5b-rout-a50492e9-kserve-5cf996c9955kpnw"
Sep 17 18:13:51.526286 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:13:51.526195    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/43c47688-adb1-4545-b4e6-f180e5f552e6-tmp-dir\") pod \"llmisvc-model-qwen2-5-0-5b-rout-a50492e9-kserve-5cf996c9955kpnw\" (UID: \"43c47688-adb1-4545-b4e6-f180e5f552e6\") " pod="e2e-test-llm-inference-service-84370a53/llmisvc-model-qwen2-5-0-5b-rout-a50492e9-kserve-5cf996c9955kpnw"
Sep 17 18:13:51.526414 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:13:51.526286    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/43c47688-adb1-4545-b4e6-f180e5f552e6-home\") pod \"llmisvc-model-qwen2-5-0-5b-rout-a50492e9-kserve-5cf996c9955kpnw\" (UID: \"43c47688-adb1-4545-b4e6-f180e5f552e6\") " pod="e2e-test-llm-inference-service-84370a53/llmisvc-model-qwen2-5-0-5b-rout-a50492e9-kserve-5cf996c9955kpnw"
Sep 17 18:13:51.526414 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:13:51.526341    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/43c47688-adb1-4545-b4e6-f180e5f552e6-tls-certs\") pod \"llmisvc-model-qwen2-5-0-5b-rout-a50492e9-kserve-5cf996c9955kpnw\" (UID: \"43c47688-adb1-4545-b4e6-f180e5f552e6\") " pod="e2e-test-llm-inference-service-84370a53/llmisvc-model-qwen2-5-0-5b-rout-a50492e9-kserve-5cf996c9955kpnw"
Sep 17 18:13:51.526414 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:13:51.526380    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/43c47688-adb1-4545-b4e6-f180e5f552e6-dshm\") pod \"llmisvc-model-qwen2-5-0-5b-rout-a50492e9-kserve-5cf996c9955kpnw\" (UID: \"43c47688-adb1-4545-b4e6-f180e5f552e6\") " pod="e2e-test-llm-inference-service-84370a53/llmisvc-model-qwen2-5-0-5b-rout-a50492e9-kserve-5cf996c9955kpnw"
Sep 17 18:13:51.624176 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:13:51.624132    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-djvzt_1d07e68d-9943-420f-a6a0-5f477792fabd/konnectivity-agent/0.log" limit=1024
Sep 17 18:13:51.627287 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:13:51.627262    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qwklv\" (UniqueName: \"kubernetes.io/projected/43c47688-adb1-4545-b4e6-f180e5f552e6-kube-api-access-qwklv\") pod \"llmisvc-model-qwen2-5-0-5b-rout-a50492e9-kserve-5cf996c9955kpnw\" (UID: \"43c47688-adb1-4545-b4e6-f180e5f552e6\") " pod="e2e-test-llm-inference-service-84370a53/llmisvc-model-qwen2-5-0-5b-rout-a50492e9-kserve-5cf996c9955kpnw"
Sep 17 18:13:51.627367 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:13:51.627316    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/43c47688-adb1-4545-b4e6-f180e5f552e6-tmp-dir\") pod \"llmisvc-model-qwen2-5-0-5b-rout-a50492e9-kserve-5cf996c9955kpnw\" (UID: \"43c47688-adb1-4545-b4e6-f180e5f552e6\") " pod="e2e-test-llm-inference-service-84370a53/llmisvc-model-qwen2-5-0-5b-rout-a50492e9-kserve-5cf996c9955kpnw"
Sep 17 18:13:51.627406 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:13:51.627388    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/43c47688-adb1-4545-b4e6-f180e5f552e6-home\") pod \"llmisvc-model-qwen2-5-0-5b-rout-a50492e9-kserve-5cf996c9955kpnw\" (UID: \"43c47688-adb1-4545-b4e6-f180e5f552e6\") " pod="e2e-test-llm-inference-service-84370a53/llmisvc-model-qwen2-5-0-5b-rout-a50492e9-kserve-5cf996c9955kpnw"
Sep 17 18:13:51.627455 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:13:51.627415    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/43c47688-adb1-4545-b4e6-f180e5f552e6-tls-certs\") pod \"llmisvc-model-qwen2-5-0-5b-rout-a50492e9-kserve-5cf996c9955kpnw\" (UID: \"43c47688-adb1-4545-b4e6-f180e5f552e6\") " pod="e2e-test-llm-inference-service-84370a53/llmisvc-model-qwen2-5-0-5b-rout-a50492e9-kserve-5cf996c9955kpnw"
Sep 17 18:13:51.627508 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:13:51.627463    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/43c47688-adb1-4545-b4e6-f180e5f552e6-dshm\") pod \"llmisvc-model-qwen2-5-0-5b-rout-a50492e9-kserve-5cf996c9955kpnw\" (UID: \"43c47688-adb1-4545-b4e6-f180e5f552e6\") " pod="e2e-test-llm-inference-service-84370a53/llmisvc-model-qwen2-5-0-5b-rout-a50492e9-kserve-5cf996c9955kpnw"
Sep 17 18:13:51.627553 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:13:51.627511    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/43c47688-adb1-4545-b4e6-f180e5f552e6-model-cache\") pod \"llmisvc-model-qwen2-5-0-5b-rout-a50492e9-kserve-5cf996c9955kpnw\" (UID: \"43c47688-adb1-4545-b4e6-f180e5f552e6\") " pod="e2e-test-llm-inference-service-84370a53/llmisvc-model-qwen2-5-0-5b-rout-a50492e9-kserve-5cf996c9955kpnw"
Sep 17 18:13:51.627756 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:13:51.627735    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/43c47688-adb1-4545-b4e6-f180e5f552e6-tmp-dir\") pod \"llmisvc-model-qwen2-5-0-5b-rout-a50492e9-kserve-5cf996c9955kpnw\" (UID: \"43c47688-adb1-4545-b4e6-f180e5f552e6\") " pod="e2e-test-llm-inference-service-84370a53/llmisvc-model-qwen2-5-0-5b-rout-a50492e9-kserve-5cf996c9955kpnw"
Sep 17 18:13:51.627838 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:13:51.627784    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/43c47688-adb1-4545-b4e6-f180e5f552e6-home\") pod \"llmisvc-model-qwen2-5-0-5b-rout-a50492e9-kserve-5cf996c9955kpnw\" (UID: \"43c47688-adb1-4545-b4e6-f180e5f552e6\") " pod="e2e-test-llm-inference-service-84370a53/llmisvc-model-qwen2-5-0-5b-rout-a50492e9-kserve-5cf996c9955kpnw"
Sep 17 18:13:51.627897 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:13:51.627878    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/43c47688-adb1-4545-b4e6-f180e5f552e6-model-cache\") pod \"llmisvc-model-qwen2-5-0-5b-rout-a50492e9-kserve-5cf996c9955kpnw\" (UID: \"43c47688-adb1-4545-b4e6-f180e5f552e6\") " pod="e2e-test-llm-inference-service-84370a53/llmisvc-model-qwen2-5-0-5b-rout-a50492e9-kserve-5cf996c9955kpnw"
Sep 17 18:13:51.629801 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:13:51.629771    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/43c47688-adb1-4545-b4e6-f180e5f552e6-dshm\") pod \"llmisvc-model-qwen2-5-0-5b-rout-a50492e9-kserve-5cf996c9955kpnw\" (UID: \"43c47688-adb1-4545-b4e6-f180e5f552e6\") " pod="e2e-test-llm-inference-service-84370a53/llmisvc-model-qwen2-5-0-5b-rout-a50492e9-kserve-5cf996c9955kpnw"
Sep 17 18:13:51.630092 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:13:51.630072    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/43c47688-adb1-4545-b4e6-f180e5f552e6-tls-certs\") pod \"llmisvc-model-qwen2-5-0-5b-rout-a50492e9-kserve-5cf996c9955kpnw\" (UID: \"43c47688-adb1-4545-b4e6-f180e5f552e6\") " pod="e2e-test-llm-inference-service-84370a53/llmisvc-model-qwen2-5-0-5b-rout-a50492e9-kserve-5cf996c9955kpnw"
Sep 17 18:13:51.635821 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:13:51.635797    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-qwklv\" (UniqueName: \"kubernetes.io/projected/43c47688-adb1-4545-b4e6-f180e5f552e6-kube-api-access-qwklv\") pod \"llmisvc-model-qwen2-5-0-5b-rout-a50492e9-kserve-5cf996c9955kpnw\" (UID: \"43c47688-adb1-4545-b4e6-f180e5f552e6\") " pod="e2e-test-llm-inference-service-84370a53/llmisvc-model-qwen2-5-0-5b-rout-a50492e9-kserve-5cf996c9955kpnw"
Sep 17 18:13:51.708168 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:13:51.708077    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-test-llm-inference-service-84370a53/llmisvc-model-qwen2-5-0-5b-rout-a50492e9-kserve-5cf996c9955kpnw"
Sep 17 18:13:51.981904 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:13:51.981875    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["e2e-test-llm-inference-service-84370a53/llmisvc-model-qwen2-5-0-5b-rout-a50492e9-kserve-5cf996c9955kpnw"]
Sep 17 18:13:51.984199 ip-10-0-143-74 kubenswrapper[2575]: W0917 18:13:51.984170    2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod43c47688_adb1_4545_b4e6_f180e5f552e6.slice/crio-238d8d1d146f1a52f5550ee34a14e85d9f4d02294290859480c7c77760e0fc55 WatchSource:0}: Error finding container 238d8d1d146f1a52f5550ee34a14e85d9f4d02294290859480c7c77760e0fc55: Status 404 returned error can't find the container with id 238d8d1d146f1a52f5550ee34a14e85d9f4d02294290859480c7c77760e0fc55
Sep 17 18:13:52.885488 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:13:52.885431    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-test-llm-inference-service-84370a53/llmisvc-model-qwen2-5-0-5b-rout-a50492e9-kserve-5cf996c9955kpnw" event={"ID":"43c47688-adb1-4545-b4e6-f180e5f552e6","Type":"ContainerStarted","Data":"11c1916fe782d3ffcdb0bbbdab3c4fe49d97dbe22bd5edc29ee938d24039ed88"}
Sep 17 18:13:52.885884 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:13:52.885504    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-test-llm-inference-service-84370a53/llmisvc-model-qwen2-5-0-5b-rout-a50492e9-kserve-5cf996c9955kpnw" event={"ID":"43c47688-adb1-4545-b4e6-f180e5f552e6","Type":"ContainerStarted","Data":"238d8d1d146f1a52f5550ee34a14e85d9f4d02294290859480c7c77760e0fc55"}
Sep 17 18:13:52.906879 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:13:52.906819    2575 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="e2e-test-llm-inference-service-84370a53/llmisvc-model-qwen2-5-0-5b-rout-a50492e9-kserve-5cf996c9955kpnw" podStartSLOduration=1.906800523 podStartE2EDuration="1.906800523s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-17 18:13:51 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-17 18:13:52.905630646 +0000 UTC m=+2352.535960614" watchObservedRunningTime="2026-09-17 18:13:52.906800523 +0000 UTC m=+2352.537130497"
Sep 17 18:13:53.708999 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:13:53.708951    2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-test-llm-inference-service-84370a53/llmisvc-model-qwen2-5-0-5b-rout-a50492e9-kserve-5cf996c9955kpnw"
Sep 17 18:13:54.625376 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:13:54.625343    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-djvzt_1d07e68d-9943-420f-a6a0-5f477792fabd/konnectivity-agent/0.log" limit=1024
Sep 17 18:13:58.007876 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:13:58.007840    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-djvzt_1d07e68d-9943-420f-a6a0-5f477792fabd/konnectivity-agent/0.log" limit=1024
Sep 17 18:13:58.098480 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:13:58.098434    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-djvzt_1d07e68d-9943-420f-a6a0-5f477792fabd/konnectivity-agent/0.log" limit=1024
Sep 17 18:14:01.708526 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:14:01.708484    2575 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="e2e-test-llm-inference-service-84370a53/llmisvc-model-qwen2-5-0-5b-rout-a50492e9-kserve-5cf996c9955kpnw"
Sep 17 18:14:01.721378 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:14:01.721349    2575 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="e2e-test-llm-inference-service-84370a53/llmisvc-model-qwen2-5-0-5b-rout-a50492e9-kserve-5cf996c9955kpnw"
Sep 17 18:14:01.947537 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:14:01.947504    2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-test-llm-inference-service-84370a53/llmisvc-model-qwen2-5-0-5b-rout-a50492e9-kserve-5cf996c9955kpnw"
Sep 17 18:14:03.013505 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:14:03.013468    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-djvzt_1d07e68d-9943-420f-a6a0-5f477792fabd/konnectivity-agent/0.log" limit=1024
Sep 17 18:14:07.010229 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:14:07.010190    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-djvzt_1d07e68d-9943-420f-a6a0-5f477792fabd/konnectivity-agent/0.log" limit=1024
Sep 17 18:14:19.038896 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:14:19.038860    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-djvzt_1d07e68d-9943-420f-a6a0-5f477792fabd/konnectivity-agent/0.log" limit=1024
Sep 17 18:14:19.584177 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:14:19.584141    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-djvzt_1d07e68d-9943-420f-a6a0-5f477792fabd/konnectivity-agent/0.log" limit=1024
Sep 17 18:14:24.380471 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:14:24.380383    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-djvzt_1d07e68d-9943-420f-a6a0-5f477792fabd/konnectivity-agent/0.log" limit=1024
Sep 17 18:14:26.097852 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:14:26.097811    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-djvzt_1d07e68d-9943-420f-a6a0-5f477792fabd/konnectivity-agent/0.log" limit=1024
Sep 17 18:14:31.175966 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:14:31.175931    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-djvzt_1d07e68d-9943-420f-a6a0-5f477792fabd/konnectivity-agent/0.log" limit=1024
Sep 17 18:14:32.032210 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:14:32.032173    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-djvzt_1d07e68d-9943-420f-a6a0-5f477792fabd/konnectivity-agent/0.log" limit=1024
Sep 17 18:14:36.572822 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:14:36.572780    2575 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["e2e-test-llm-inference-service-84370a53/llmisvc-model-qwen2-5-0-5b-rout-a50492e9-kserve-5cf996c9955kpnw"]
Sep 17 18:14:36.578107 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:14:36.578072    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-djvzt_1d07e68d-9943-420f-a6a0-5f477792fabd/konnectivity-agent/0.log" limit=1024
Sep 17 18:14:36.583226 ip-10-0-143-74 kubenswrapper[2575]: E0917 18:14:36.583176    2575 log.go:32] "ExecSync cmd from runtime service failed" err=<
Sep 17 18:14:36.583226 ip-10-0-143-74 kubenswrapper[2575]:         rpc error: code = Unknown desc = command error: executable file `/bin/sleep` not found: No such file or directory
Sep 17 18:14:36.583226 ip-10-0-143-74 kubenswrapper[2575]:         , stdout: , stderr: , exit code -1
Sep 17 18:14:36.583226 ip-10-0-143-74 kubenswrapper[2575]:  > containerID="11c1916fe782d3ffcdb0bbbdab3c4fe49d97dbe22bd5edc29ee938d24039ed88" cmd=["/bin/sleep","15"]
Sep 17 18:14:36.583623 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:14:36.583255    2575 handlers.go:82] "Exec lifecycle hook for Container in Pod failed" execCommand=["/bin/sleep","15"] containerName="main" pod="e2e-test-llm-inference-service-84370a53/llmisvc-model-qwen2-5-0-5b-rout-a50492e9-kserve-5cf996c9955kpnw" message="" err=<
Sep 17 18:14:36.583623 ip-10-0-143-74 kubenswrapper[2575]:         rpc error: code = Unknown desc = command error: executable file `/bin/sleep` not found: No such file or directory
Sep 17 18:14:36.583623 ip-10-0-143-74 kubenswrapper[2575]:         , stdout: , stderr: , exit code -1
Sep 17 18:14:36.583623 ip-10-0-143-74 kubenswrapper[2575]:  >
Sep 17 18:14:36.583623 ip-10-0-143-74 kubenswrapper[2575]: E0917 18:14:36.583272    2575 kuberuntime_container.go:755] "PreStop hook failed" err=<
Sep 17 18:14:36.583623 ip-10-0-143-74 kubenswrapper[2575]:         rpc error: code = Unknown desc = command error: executable file `/bin/sleep` not found: No such file or directory
Sep 17 18:14:36.583623 ip-10-0-143-74 kubenswrapper[2575]:         , stdout: , stderr: , exit code -1
Sep 17 18:14:36.583623 ip-10-0-143-74 kubenswrapper[2575]:  > pod="e2e-test-llm-inference-service-84370a53/llmisvc-model-qwen2-5-0-5b-rout-a50492e9-kserve-5cf996c9955kpnw" podUID="43c47688-adb1-4545-b4e6-f180e5f552e6" containerName="main" containerID="cri-o://11c1916fe782d3ffcdb0bbbdab3c4fe49d97dbe22bd5edc29ee938d24039ed88"
Sep 17 18:14:36.583623 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:14:36.583342    2575 kuberuntime_container.go:873] "Killing container with a grace period" pod="e2e-test-llm-inference-service-84370a53/llmisvc-model-qwen2-5-0-5b-rout-a50492e9-kserve-5cf996c9955kpnw" podUID="43c47688-adb1-4545-b4e6-f180e5f552e6" containerName="main" containerID="cri-o://11c1916fe782d3ffcdb0bbbdab3c4fe49d97dbe22bd5edc29ee938d24039ed88" gracePeriod=60
Sep 17 18:14:36.654483 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:14:36.654428    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-djvzt_1d07e68d-9943-420f-a6a0-5f477792fabd/konnectivity-agent/0.log" limit=1024
Sep 17 18:14:36.944823 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:14:36.944790    2575 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-test-llm-inference-service-84370a53/llmisvc-model-qwen2-5-0-5b-rout-a50492e9-kserve-5cf996c9955kpnw"
Sep 17 18:14:37.019105 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:14:37.019066    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/43c47688-adb1-4545-b4e6-f180e5f552e6-model-cache\") pod \"43c47688-adb1-4545-b4e6-f180e5f552e6\" (UID: \"43c47688-adb1-4545-b4e6-f180e5f552e6\") "
Sep 17 18:14:37.019105 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:14:37.019108    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/43c47688-adb1-4545-b4e6-f180e5f552e6-tmp-dir\") pod \"43c47688-adb1-4545-b4e6-f180e5f552e6\" (UID: \"43c47688-adb1-4545-b4e6-f180e5f552e6\") "
Sep 17 18:14:37.019347 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:14:37.019157    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/43c47688-adb1-4545-b4e6-f180e5f552e6-tls-certs\") pod \"43c47688-adb1-4545-b4e6-f180e5f552e6\" (UID: \"43c47688-adb1-4545-b4e6-f180e5f552e6\") "
Sep 17 18:14:37.019347 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:14:37.019182    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/43c47688-adb1-4545-b4e6-f180e5f552e6-dshm\") pod \"43c47688-adb1-4545-b4e6-f180e5f552e6\" (UID: \"43c47688-adb1-4545-b4e6-f180e5f552e6\") "
Sep 17 18:14:37.019347 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:14:37.019247    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-qwklv\" (UniqueName: \"kubernetes.io/projected/43c47688-adb1-4545-b4e6-f180e5f552e6-kube-api-access-qwklv\") pod \"43c47688-adb1-4545-b4e6-f180e5f552e6\" (UID: \"43c47688-adb1-4545-b4e6-f180e5f552e6\") "
Sep 17 18:14:37.019347 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:14:37.019267    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/43c47688-adb1-4545-b4e6-f180e5f552e6-home\") pod \"43c47688-adb1-4545-b4e6-f180e5f552e6\" (UID: \"43c47688-adb1-4545-b4e6-f180e5f552e6\") "
Sep 17 18:14:37.019582 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:14:37.019419    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/43c47688-adb1-4545-b4e6-f180e5f552e6-model-cache" (OuterVolumeSpecName: "model-cache") pod "43c47688-adb1-4545-b4e6-f180e5f552e6" (UID: "43c47688-adb1-4545-b4e6-f180e5f552e6"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 17 18:14:37.019582 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:14:37.019437    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/43c47688-adb1-4545-b4e6-f180e5f552e6-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "43c47688-adb1-4545-b4e6-f180e5f552e6" (UID: "43c47688-adb1-4545-b4e6-f180e5f552e6"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 17 18:14:37.019582 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:14:37.019531    2575 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/43c47688-adb1-4545-b4e6-f180e5f552e6-model-cache\") on node \"ip-10-0-143-74.ec2.internal\" DevicePath \"\""
Sep 17 18:14:37.019709 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:14:37.019603    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/43c47688-adb1-4545-b4e6-f180e5f552e6-home" (OuterVolumeSpecName: "home") pod "43c47688-adb1-4545-b4e6-f180e5f552e6" (UID: "43c47688-adb1-4545-b4e6-f180e5f552e6"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 17 18:14:37.021494 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:14:37.021471    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/43c47688-adb1-4545-b4e6-f180e5f552e6-dshm" (OuterVolumeSpecName: "dshm") pod "43c47688-adb1-4545-b4e6-f180e5f552e6" (UID: "43c47688-adb1-4545-b4e6-f180e5f552e6"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 17 18:14:37.021611 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:14:37.021591    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/43c47688-adb1-4545-b4e6-f180e5f552e6-kube-api-access-qwklv" (OuterVolumeSpecName: "kube-api-access-qwklv") pod "43c47688-adb1-4545-b4e6-f180e5f552e6" (UID: "43c47688-adb1-4545-b4e6-f180e5f552e6"). InnerVolumeSpecName "kube-api-access-qwklv". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 17 18:14:37.021611 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:14:37.021599    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/43c47688-adb1-4545-b4e6-f180e5f552e6-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "43c47688-adb1-4545-b4e6-f180e5f552e6" (UID: "43c47688-adb1-4545-b4e6-f180e5f552e6"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 17 18:14:37.120251 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:14:37.120214    2575 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-qwklv\" (UniqueName: \"kubernetes.io/projected/43c47688-adb1-4545-b4e6-f180e5f552e6-kube-api-access-qwklv\") on node \"ip-10-0-143-74.ec2.internal\" DevicePath \"\""
Sep 17 18:14:37.120251 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:14:37.120250    2575 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/43c47688-adb1-4545-b4e6-f180e5f552e6-home\") on node \"ip-10-0-143-74.ec2.internal\" DevicePath \"\""
Sep 17 18:14:37.120488 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:14:37.120261    2575 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/43c47688-adb1-4545-b4e6-f180e5f552e6-tmp-dir\") on node \"ip-10-0-143-74.ec2.internal\" DevicePath \"\""
Sep 17 18:14:37.120488 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:14:37.120269    2575 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/43c47688-adb1-4545-b4e6-f180e5f552e6-tls-certs\") on node \"ip-10-0-143-74.ec2.internal\" DevicePath \"\""
Sep 17 18:14:37.120488 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:14:37.120279    2575 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/43c47688-adb1-4545-b4e6-f180e5f552e6-dshm\") on node \"ip-10-0-143-74.ec2.internal\" DevicePath \"\""
Sep 17 18:14:37.131454 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:14:37.131410    2575 generic.go:356] "Generic (PLEG): container finished" podID="43c47688-adb1-4545-b4e6-f180e5f552e6" containerID="11c1916fe782d3ffcdb0bbbdab3c4fe49d97dbe22bd5edc29ee938d24039ed88" exitCode=0
Sep 17 18:14:37.131619 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:14:37.131498    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-test-llm-inference-service-84370a53/llmisvc-model-qwen2-5-0-5b-rout-a50492e9-kserve-5cf996c9955kpnw" event={"ID":"43c47688-adb1-4545-b4e6-f180e5f552e6","Type":"ContainerDied","Data":"11c1916fe782d3ffcdb0bbbdab3c4fe49d97dbe22bd5edc29ee938d24039ed88"}
Sep 17 18:14:37.131619 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:14:37.131556    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-test-llm-inference-service-84370a53/llmisvc-model-qwen2-5-0-5b-rout-a50492e9-kserve-5cf996c9955kpnw" event={"ID":"43c47688-adb1-4545-b4e6-f180e5f552e6","Type":"ContainerDied","Data":"238d8d1d146f1a52f5550ee34a14e85d9f4d02294290859480c7c77760e0fc55"}
Sep 17 18:14:37.131619 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:14:37.131580    2575 scope.go:117] "RemoveContainer" containerID="11c1916fe782d3ffcdb0bbbdab3c4fe49d97dbe22bd5edc29ee938d24039ed88"
Sep 17 18:14:37.131619 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:14:37.131504    2575 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-test-llm-inference-service-84370a53/llmisvc-model-qwen2-5-0-5b-rout-a50492e9-kserve-5cf996c9955kpnw"
Sep 17 18:14:37.142189 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:14:37.142161    2575 scope.go:117] "RemoveContainer" containerID="11c1916fe782d3ffcdb0bbbdab3c4fe49d97dbe22bd5edc29ee938d24039ed88"
Sep 17 18:14:37.142539 ip-10-0-143-74 kubenswrapper[2575]: E0917 18:14:37.142519    2575 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"11c1916fe782d3ffcdb0bbbdab3c4fe49d97dbe22bd5edc29ee938d24039ed88\": container with ID starting with 11c1916fe782d3ffcdb0bbbdab3c4fe49d97dbe22bd5edc29ee938d24039ed88 not found: ID does not exist" containerID="11c1916fe782d3ffcdb0bbbdab3c4fe49d97dbe22bd5edc29ee938d24039ed88"
Sep 17 18:14:37.142611 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:14:37.142545    2575 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"11c1916fe782d3ffcdb0bbbdab3c4fe49d97dbe22bd5edc29ee938d24039ed88"} err="failed to get container status \"11c1916fe782d3ffcdb0bbbdab3c4fe49d97dbe22bd5edc29ee938d24039ed88\": rpc error: code = NotFound desc = could not find container \"11c1916fe782d3ffcdb0bbbdab3c4fe49d97dbe22bd5edc29ee938d24039ed88\": container with ID starting with 11c1916fe782d3ffcdb0bbbdab3c4fe49d97dbe22bd5edc29ee938d24039ed88 not found: ID does not exist"
Sep 17 18:14:37.155218 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:14:37.155189    2575 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["e2e-test-llm-inference-service-84370a53/llmisvc-model-qwen2-5-0-5b-rout-a50492e9-kserve-5cf996c9955kpnw"]
Sep 17 18:14:37.161694 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:14:37.161663    2575 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["e2e-test-llm-inference-service-84370a53/llmisvc-model-qwen2-5-0-5b-rout-a50492e9-kserve-5cf996c9955kpnw"]
Sep 17 18:14:38.967618 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:14:38.967578    2575 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="43c47688-adb1-4545-b4e6-f180e5f552e6" path="/var/lib/kubelet/pods/43c47688-adb1-4545-b4e6-f180e5f552e6/volumes"
Sep 17 18:14:45.878390 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:14:45.878355    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-djvzt_1d07e68d-9943-420f-a6a0-5f477792fabd/konnectivity-agent/0.log" limit=1024
Sep 17 18:14:46.985566 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:14:46.985519    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-djvzt_1d07e68d-9943-420f-a6a0-5f477792fabd/konnectivity-agent/0.log" limit=1024
Sep 17 18:14:47.086421 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:14:47.086389    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-djvzt_1d07e68d-9943-420f-a6a0-5f477792fabd/konnectivity-agent/0.log" limit=1024
Sep 17 18:15:03.462213 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:15:03.462120    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-djvzt_1d07e68d-9943-420f-a6a0-5f477792fabd/konnectivity-agent/0.log" limit=1024
Sep 17 18:15:03.563363 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:15:03.563314    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-djvzt_1d07e68d-9943-420f-a6a0-5f477792fabd/konnectivity-agent/0.log" limit=1024
Sep 17 18:15:04.568048 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:15:04.568008    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-djvzt_1d07e68d-9943-420f-a6a0-5f477792fabd/konnectivity-agent/0.log" limit=1024
Sep 17 18:15:25.324627 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:15:25.324593    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-djvzt_1d07e68d-9943-420f-a6a0-5f477792fabd/konnectivity-agent/0.log" limit=1024
Sep 17 18:15:25.421817 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:15:25.421759    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-djvzt_1d07e68d-9943-420f-a6a0-5f477792fabd/konnectivity-agent/0.log" limit=1024
Sep 17 18:15:26.090739 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:15:26.090699    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-djvzt_1d07e68d-9943-420f-a6a0-5f477792fabd/konnectivity-agent/0.log" limit=1024
Sep 17 18:16:15.402604 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:16:15.402562    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-djvzt_1d07e68d-9943-420f-a6a0-5f477792fabd/konnectivity-agent/0.log" limit=1024
Sep 17 18:16:16.618769 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:16:16.618728    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-djvzt_1d07e68d-9943-420f-a6a0-5f477792fabd/konnectivity-agent/0.log" limit=1024
Sep 17 18:16:20.295548 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:16:20.295431    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-djvzt_1d07e68d-9943-420f-a6a0-5f477792fabd/konnectivity-agent/0.log" limit=1024
Sep 17 18:16:20.489140 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:16:20.489102    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-djvzt_1d07e68d-9943-420f-a6a0-5f477792fabd/konnectivity-agent/0.log" limit=1024
Sep 17 18:16:22.503764 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:16:22.503721    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-djvzt_1d07e68d-9943-420f-a6a0-5f477792fabd/konnectivity-agent/0.log" limit=1024
Sep 17 18:16:25.819062 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:16:25.819029    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-djvzt_1d07e68d-9943-420f-a6a0-5f477792fabd/konnectivity-agent/0.log" limit=1024
Sep 17 18:16:28.315631 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:16:28.315585    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-djvzt_1d07e68d-9943-420f-a6a0-5f477792fabd/konnectivity-agent/0.log" limit=1024
Sep 17 18:16:28.386256 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:16:28.386222    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-djvzt_1d07e68d-9943-420f-a6a0-5f477792fabd/konnectivity-agent/0.log" limit=1024
Sep 17 18:16:29.831140 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:16:29.831087    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-djvzt_1d07e68d-9943-420f-a6a0-5f477792fabd/konnectivity-agent/0.log" limit=1024
Sep 17 18:16:29.931125 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:16:29.931074    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-djvzt_1d07e68d-9943-420f-a6a0-5f477792fabd/konnectivity-agent/0.log" limit=1024
Sep 17 18:16:32.503323 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:16:32.503287    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-djvzt_1d07e68d-9943-420f-a6a0-5f477792fabd/konnectivity-agent/0.log" limit=1024
Sep 17 18:16:32.601927 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:16:32.601894    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-djvzt_1d07e68d-9943-420f-a6a0-5f477792fabd/konnectivity-agent/0.log" limit=1024
Sep 17 18:16:35.963767 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:16:35.963684    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-djvzt_1d07e68d-9943-420f-a6a0-5f477792fabd/konnectivity-agent/0.log" limit=1024
Sep 17 18:17:00.078666 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:17:00.078627    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-djvzt_1d07e68d-9943-420f-a6a0-5f477792fabd/konnectivity-agent/0.log" limit=1024
Sep 17 18:17:06.205218 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:17:06.205181    2575 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["e2e-test-llm-inference-service-62fead0b/llmisvc-model-qwen2-5-0-5b-rout-4f8c0978-kserve-bb749b474-b846v"]
Sep 17 18:17:06.205685 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:17:06.205589    2575 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="43c47688-adb1-4545-b4e6-f180e5f552e6" containerName="main"
Sep 17 18:17:06.205685 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:17:06.205601    2575 state_mem.go:107] "Deleted CPUSet assignment" podUID="43c47688-adb1-4545-b4e6-f180e5f552e6" containerName="main"
Sep 17 18:17:06.205685 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:17:06.205682    2575 memory_manager.go:368] "RemoveStaleState removing state" podUID="43c47688-adb1-4545-b4e6-f180e5f552e6" containerName="main"
Sep 17 18:17:06.208656 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:17:06.208635    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-test-llm-inference-service-62fead0b/llmisvc-model-qwen2-5-0-5b-rout-4f8c0978-kserve-bb749b474-b846v"
Sep 17 18:17:06.211268 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:17:06.211244    2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-test-llm-inference-service-62fead0b\"/\"kube-root-ca.crt\""
Sep 17 18:17:06.212312 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:17:06.212291    2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-test-llm-inference-service-62fead0b\"/\"openshift-service-ca.crt\""
Sep 17 18:17:06.212431 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:17:06.212309    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-test-llm-inference-service-62fead0b\"/\"llmisv08544b88a8d963ffd553cc1f3ed82d16-kserve-self-signed-certs\""
Sep 17 18:17:06.212431 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:17:06.212320    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-test-llm-inference-service-62fead0b\"/\"default-dockercfg-pg56v\""
Sep 17 18:17:06.220459 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:17:06.220416    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["e2e-test-llm-inference-service-62fead0b/llmisvc-model-qwen2-5-0-5b-rout-4f8c0978-kserve-bb749b474-b846v"]
Sep 17 18:17:06.241867 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:17:06.241834    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-djvzt_1d07e68d-9943-420f-a6a0-5f477792fabd/konnectivity-agent/0.log" limit=1024
Sep 17 18:17:06.324069 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:17:06.324034    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/b55011b4-5278-4dc1-bb0e-391e4644d87a-tls-certs\") pod \"llmisvc-model-qwen2-5-0-5b-rout-4f8c0978-kserve-bb749b474-b846v\" (UID: \"b55011b4-5278-4dc1-bb0e-391e4644d87a\") " pod="e2e-test-llm-inference-service-62fead0b/llmisvc-model-qwen2-5-0-5b-rout-4f8c0978-kserve-bb749b474-b846v"
Sep 17 18:17:06.324069 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:17:06.324074    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/b55011b4-5278-4dc1-bb0e-391e4644d87a-model-cache\") pod \"llmisvc-model-qwen2-5-0-5b-rout-4f8c0978-kserve-bb749b474-b846v\" (UID: \"b55011b4-5278-4dc1-bb0e-391e4644d87a\") " pod="e2e-test-llm-inference-service-62fead0b/llmisvc-model-qwen2-5-0-5b-rout-4f8c0978-kserve-bb749b474-b846v"
Sep 17 18:17:06.324273 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:17:06.324141    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/b55011b4-5278-4dc1-bb0e-391e4644d87a-dshm\") pod \"llmisvc-model-qwen2-5-0-5b-rout-4f8c0978-kserve-bb749b474-b846v\" (UID: \"b55011b4-5278-4dc1-bb0e-391e4644d87a\") " pod="e2e-test-llm-inference-service-62fead0b/llmisvc-model-qwen2-5-0-5b-rout-4f8c0978-kserve-bb749b474-b846v"
Sep 17 18:17:06.324273 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:17:06.324169    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-nc55q\" (UniqueName: \"kubernetes.io/projected/b55011b4-5278-4dc1-bb0e-391e4644d87a-kube-api-access-nc55q\") pod \"llmisvc-model-qwen2-5-0-5b-rout-4f8c0978-kserve-bb749b474-b846v\" (UID: \"b55011b4-5278-4dc1-bb0e-391e4644d87a\") " pod="e2e-test-llm-inference-service-62fead0b/llmisvc-model-qwen2-5-0-5b-rout-4f8c0978-kserve-bb749b474-b846v"
Sep 17 18:17:06.324273 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:17:06.324257    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/b55011b4-5278-4dc1-bb0e-391e4644d87a-tmp-dir\") pod \"llmisvc-model-qwen2-5-0-5b-rout-4f8c0978-kserve-bb749b474-b846v\" (UID: \"b55011b4-5278-4dc1-bb0e-391e4644d87a\") " pod="e2e-test-llm-inference-service-62fead0b/llmisvc-model-qwen2-5-0-5b-rout-4f8c0978-kserve-bb749b474-b846v"
Sep 17 18:17:06.324381 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:17:06.324298    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/b55011b4-5278-4dc1-bb0e-391e4644d87a-home\") pod \"llmisvc-model-qwen2-5-0-5b-rout-4f8c0978-kserve-bb749b474-b846v\" (UID: \"b55011b4-5278-4dc1-bb0e-391e4644d87a\") " pod="e2e-test-llm-inference-service-62fead0b/llmisvc-model-qwen2-5-0-5b-rout-4f8c0978-kserve-bb749b474-b846v"
Sep 17 18:17:06.424874 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:17:06.424831    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-nc55q\" (UniqueName: \"kubernetes.io/projected/b55011b4-5278-4dc1-bb0e-391e4644d87a-kube-api-access-nc55q\") pod \"llmisvc-model-qwen2-5-0-5b-rout-4f8c0978-kserve-bb749b474-b846v\" (UID: \"b55011b4-5278-4dc1-bb0e-391e4644d87a\") " pod="e2e-test-llm-inference-service-62fead0b/llmisvc-model-qwen2-5-0-5b-rout-4f8c0978-kserve-bb749b474-b846v"
Sep 17 18:17:06.425034 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:17:06.424901    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/b55011b4-5278-4dc1-bb0e-391e4644d87a-tmp-dir\") pod \"llmisvc-model-qwen2-5-0-5b-rout-4f8c0978-kserve-bb749b474-b846v\" (UID: \"b55011b4-5278-4dc1-bb0e-391e4644d87a\") " pod="e2e-test-llm-inference-service-62fead0b/llmisvc-model-qwen2-5-0-5b-rout-4f8c0978-kserve-bb749b474-b846v"
Sep 17 18:17:06.425034 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:17:06.424938    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/b55011b4-5278-4dc1-bb0e-391e4644d87a-home\") pod \"llmisvc-model-qwen2-5-0-5b-rout-4f8c0978-kserve-bb749b474-b846v\" (UID: \"b55011b4-5278-4dc1-bb0e-391e4644d87a\") " pod="e2e-test-llm-inference-service-62fead0b/llmisvc-model-qwen2-5-0-5b-rout-4f8c0978-kserve-bb749b474-b846v"
Sep 17 18:17:06.425034 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:17:06.424996    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/b55011b4-5278-4dc1-bb0e-391e4644d87a-tls-certs\") pod \"llmisvc-model-qwen2-5-0-5b-rout-4f8c0978-kserve-bb749b474-b846v\" (UID: \"b55011b4-5278-4dc1-bb0e-391e4644d87a\") " pod="e2e-test-llm-inference-service-62fead0b/llmisvc-model-qwen2-5-0-5b-rout-4f8c0978-kserve-bb749b474-b846v"
Sep 17 18:17:06.425034 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:17:06.425021    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/b55011b4-5278-4dc1-bb0e-391e4644d87a-model-cache\") pod \"llmisvc-model-qwen2-5-0-5b-rout-4f8c0978-kserve-bb749b474-b846v\" (UID: \"b55011b4-5278-4dc1-bb0e-391e4644d87a\") " pod="e2e-test-llm-inference-service-62fead0b/llmisvc-model-qwen2-5-0-5b-rout-4f8c0978-kserve-bb749b474-b846v"
Sep 17 18:17:06.425266 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:17:06.425078    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/b55011b4-5278-4dc1-bb0e-391e4644d87a-dshm\") pod \"llmisvc-model-qwen2-5-0-5b-rout-4f8c0978-kserve-bb749b474-b846v\" (UID: \"b55011b4-5278-4dc1-bb0e-391e4644d87a\") " pod="e2e-test-llm-inference-service-62fead0b/llmisvc-model-qwen2-5-0-5b-rout-4f8c0978-kserve-bb749b474-b846v"
Sep 17 18:17:06.425420 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:17:06.425392    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/b55011b4-5278-4dc1-bb0e-391e4644d87a-tmp-dir\") pod \"llmisvc-model-qwen2-5-0-5b-rout-4f8c0978-kserve-bb749b474-b846v\" (UID: \"b55011b4-5278-4dc1-bb0e-391e4644d87a\") " pod="e2e-test-llm-inference-service-62fead0b/llmisvc-model-qwen2-5-0-5b-rout-4f8c0978-kserve-bb749b474-b846v"
Sep 17 18:17:06.425546 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:17:06.425414    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/b55011b4-5278-4dc1-bb0e-391e4644d87a-home\") pod \"llmisvc-model-qwen2-5-0-5b-rout-4f8c0978-kserve-bb749b474-b846v\" (UID: \"b55011b4-5278-4dc1-bb0e-391e4644d87a\") " pod="e2e-test-llm-inference-service-62fead0b/llmisvc-model-qwen2-5-0-5b-rout-4f8c0978-kserve-bb749b474-b846v"
Sep 17 18:17:06.425546 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:17:06.425482    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/b55011b4-5278-4dc1-bb0e-391e4644d87a-model-cache\") pod \"llmisvc-model-qwen2-5-0-5b-rout-4f8c0978-kserve-bb749b474-b846v\" (UID: \"b55011b4-5278-4dc1-bb0e-391e4644d87a\") " pod="e2e-test-llm-inference-service-62fead0b/llmisvc-model-qwen2-5-0-5b-rout-4f8c0978-kserve-bb749b474-b846v"
Sep 17 18:17:06.427346 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:17:06.427322    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/b55011b4-5278-4dc1-bb0e-391e4644d87a-dshm\") pod \"llmisvc-model-qwen2-5-0-5b-rout-4f8c0978-kserve-bb749b474-b846v\" (UID: \"b55011b4-5278-4dc1-bb0e-391e4644d87a\") " pod="e2e-test-llm-inference-service-62fead0b/llmisvc-model-qwen2-5-0-5b-rout-4f8c0978-kserve-bb749b474-b846v"
Sep 17 18:17:06.427506 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:17:06.427489    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/b55011b4-5278-4dc1-bb0e-391e4644d87a-tls-certs\") pod \"llmisvc-model-qwen2-5-0-5b-rout-4f8c0978-kserve-bb749b474-b846v\" (UID: \"b55011b4-5278-4dc1-bb0e-391e4644d87a\") " pod="e2e-test-llm-inference-service-62fead0b/llmisvc-model-qwen2-5-0-5b-rout-4f8c0978-kserve-bb749b474-b846v"
Sep 17 18:17:06.435070 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:17:06.435046    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-nc55q\" (UniqueName: \"kubernetes.io/projected/b55011b4-5278-4dc1-bb0e-391e4644d87a-kube-api-access-nc55q\") pod \"llmisvc-model-qwen2-5-0-5b-rout-4f8c0978-kserve-bb749b474-b846v\" (UID: \"b55011b4-5278-4dc1-bb0e-391e4644d87a\") " pod="e2e-test-llm-inference-service-62fead0b/llmisvc-model-qwen2-5-0-5b-rout-4f8c0978-kserve-bb749b474-b846v"
Sep 17 18:17:06.519969 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:17:06.519885    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-test-llm-inference-service-62fead0b/llmisvc-model-qwen2-5-0-5b-rout-4f8c0978-kserve-bb749b474-b846v"
Sep 17 18:17:06.535380 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:17:06.535351    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-djvzt_1d07e68d-9943-420f-a6a0-5f477792fabd/konnectivity-agent/0.log" limit=1024
Sep 17 18:17:06.795657 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:17:06.795633    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["e2e-test-llm-inference-service-62fead0b/llmisvc-model-qwen2-5-0-5b-rout-4f8c0978-kserve-bb749b474-b846v"]
Sep 17 18:17:06.797919 ip-10-0-143-74 kubenswrapper[2575]: W0917 18:17:06.797892    2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb55011b4_5278_4dc1_bb0e_391e4644d87a.slice/crio-9b5b363505faedd26c732424f3a7df9df02e0dc7e0cb726fc24f7a93fc528754 WatchSource:0}: Error finding container 9b5b363505faedd26c732424f3a7df9df02e0dc7e0cb726fc24f7a93fc528754: Status 404 returned error can't find the container with id 9b5b363505faedd26c732424f3a7df9df02e0dc7e0cb726fc24f7a93fc528754
Sep 17 18:17:06.943944 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:17:06.943905    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-test-llm-inference-service-62fead0b/llmisvc-model-qwen2-5-0-5b-rout-4f8c0978-kserve-bb749b474-b846v" event={"ID":"b55011b4-5278-4dc1-bb0e-391e4644d87a","Type":"ContainerStarted","Data":"6ec9e8c69930c9fd0adf9c3855ff8b23cf6cae8b44809f3e493ece9346f54c6d"}
Sep 17 18:17:06.944104 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:17:06.943959    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-test-llm-inference-service-62fead0b/llmisvc-model-qwen2-5-0-5b-rout-4f8c0978-kserve-bb749b474-b846v" event={"ID":"b55011b4-5278-4dc1-bb0e-391e4644d87a","Type":"ContainerStarted","Data":"9b5b363505faedd26c732424f3a7df9df02e0dc7e0cb726fc24f7a93fc528754"}
Sep 17 18:17:06.961390 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:17:06.961315    2575 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="e2e-test-llm-inference-service-62fead0b/llmisvc-model-qwen2-5-0-5b-rout-4f8c0978-kserve-bb749b474-b846v" podStartSLOduration=0.961300292 podStartE2EDuration="961.300292ms" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-17 18:17:06 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-17 18:17:06.96028405 +0000 UTC m=+2546.590614028" watchObservedRunningTime="2026-09-17 18:17:06.961300292 +0000 UTC m=+2546.591630400"
Sep 17 18:17:07.520867 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:17:07.520833    2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-test-llm-inference-service-62fead0b/llmisvc-model-qwen2-5-0-5b-rout-4f8c0978-kserve-bb749b474-b846v"
Sep 17 18:17:11.084605 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:17:11.084563    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-djvzt_1d07e68d-9943-420f-a6a0-5f477792fabd/konnectivity-agent/0.log" limit=1024
Sep 17 18:17:13.273121 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:17:13.273087    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-djvzt_1d07e68d-9943-420f-a6a0-5f477792fabd/konnectivity-agent/0.log" limit=1024
Sep 17 18:17:16.520913 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:17:16.520872    2575 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="e2e-test-llm-inference-service-62fead0b/llmisvc-model-qwen2-5-0-5b-rout-4f8c0978-kserve-bb749b474-b846v"
Sep 17 18:17:16.533272 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:17:16.533249    2575 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="e2e-test-llm-inference-service-62fead0b/llmisvc-model-qwen2-5-0-5b-rout-4f8c0978-kserve-bb749b474-b846v"
Sep 17 18:17:17.006843 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:17:17.006813    2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-test-llm-inference-service-62fead0b/llmisvc-model-qwen2-5-0-5b-rout-4f8c0978-kserve-bb749b474-b846v"
Sep 17 18:17:22.500762 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:17:22.500731    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-djvzt_1d07e68d-9943-420f-a6a0-5f477792fabd/konnectivity-agent/0.log" limit=1024
Sep 17 18:17:22.595355 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:17:22.595307    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-djvzt_1d07e68d-9943-420f-a6a0-5f477792fabd/konnectivity-agent/0.log" limit=1024
Sep 17 18:17:28.752872 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:17:28.752830    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-djvzt_1d07e68d-9943-420f-a6a0-5f477792fabd/konnectivity-agent/0.log" limit=1024
Sep 17 18:17:31.103026 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:17:31.102979    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-djvzt_1d07e68d-9943-420f-a6a0-5f477792fabd/konnectivity-agent/0.log" limit=1024
Sep 17 18:17:31.363478 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:17:31.363353    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-djvzt_1d07e68d-9943-420f-a6a0-5f477792fabd/konnectivity-agent/0.log" limit=1024
Sep 17 18:17:34.427630 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:17:34.427577    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-djvzt_1d07e68d-9943-420f-a6a0-5f477792fabd/konnectivity-agent/0.log" limit=1024
Sep 17 18:17:34.529423 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:17:34.529364    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-djvzt_1d07e68d-9943-420f-a6a0-5f477792fabd/konnectivity-agent/0.log" limit=1024
Sep 17 18:17:43.840669 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:17:43.840628    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-djvzt_1d07e68d-9943-420f-a6a0-5f477792fabd/konnectivity-agent/0.log" limit=1024
Sep 17 18:17:54.157537 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:17:54.157493    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-djvzt_1d07e68d-9943-420f-a6a0-5f477792fabd/konnectivity-agent/0.log" limit=1024
Sep 17 18:17:56.954492 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:17:56.954431    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-djvzt_1d07e68d-9943-420f-a6a0-5f477792fabd/konnectivity-agent/0.log" limit=1024
Sep 17 18:17:56.955278 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:17:56.955250    2575 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["e2e-test-llm-inference-service-62fead0b/llmisvc-model-qwen2-5-0-5b-rout-4f8c0978-kserve-bb749b474-b846v"]
Sep 17 18:17:56.964511 ip-10-0-143-74 kubenswrapper[2575]: E0917 18:17:56.964478    2575 log.go:32] "ExecSync cmd from runtime service failed" err=<
Sep 17 18:17:56.964511 ip-10-0-143-74 kubenswrapper[2575]:         rpc error: code = Unknown desc = command error: executable file `/bin/sleep` not found: No such file or directory
Sep 17 18:17:56.964511 ip-10-0-143-74 kubenswrapper[2575]:         , stdout: , stderr: , exit code -1
Sep 17 18:17:56.964511 ip-10-0-143-74 kubenswrapper[2575]:  > containerID="6ec9e8c69930c9fd0adf9c3855ff8b23cf6cae8b44809f3e493ece9346f54c6d" cmd=["/bin/sleep","15"]
Sep 17 18:17:56.964759 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:17:56.964526    2575 handlers.go:82] "Exec lifecycle hook for Container in Pod failed" execCommand=["/bin/sleep","15"] containerName="main" pod="e2e-test-llm-inference-service-62fead0b/llmisvc-model-qwen2-5-0-5b-rout-4f8c0978-kserve-bb749b474-b846v" message="" err=<
Sep 17 18:17:56.964759 ip-10-0-143-74 kubenswrapper[2575]:         rpc error: code = Unknown desc = command error: executable file `/bin/sleep` not found: No such file or directory
Sep 17 18:17:56.964759 ip-10-0-143-74 kubenswrapper[2575]:         , stdout: , stderr: , exit code -1
Sep 17 18:17:56.964759 ip-10-0-143-74 kubenswrapper[2575]:  >
Sep 17 18:17:56.964759 ip-10-0-143-74 kubenswrapper[2575]: E0917 18:17:56.964539    2575 kuberuntime_container.go:755] "PreStop hook failed" err=<
Sep 17 18:17:56.964759 ip-10-0-143-74 kubenswrapper[2575]:         rpc error: code = Unknown desc = command error: executable file `/bin/sleep` not found: No such file or directory
Sep 17 18:17:56.964759 ip-10-0-143-74 kubenswrapper[2575]:         , stdout: , stderr: , exit code -1
Sep 17 18:17:56.964759 ip-10-0-143-74 kubenswrapper[2575]:  > pod="e2e-test-llm-inference-service-62fead0b/llmisvc-model-qwen2-5-0-5b-rout-4f8c0978-kserve-bb749b474-b846v" podUID="b55011b4-5278-4dc1-bb0e-391e4644d87a" containerName="main" containerID="cri-o://6ec9e8c69930c9fd0adf9c3855ff8b23cf6cae8b44809f3e493ece9346f54c6d"
Sep 17 18:17:56.964759 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:17:56.964589    2575 kuberuntime_container.go:873] "Killing container with a grace period" pod="e2e-test-llm-inference-service-62fead0b/llmisvc-model-qwen2-5-0-5b-rout-4f8c0978-kserve-bb749b474-b846v" podUID="b55011b4-5278-4dc1-bb0e-391e4644d87a" containerName="main" containerID="cri-o://6ec9e8c69930c9fd0adf9c3855ff8b23cf6cae8b44809f3e493ece9346f54c6d" gracePeriod=60
Sep 17 18:17:56.997228 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:17:56.997195    2575 prober.go:120] "Probe failed" probeType="Readiness" pod="e2e-test-llm-inference-service-62fead0b/llmisvc-model-qwen2-5-0-5b-rout-4f8c0978-kserve-bb749b474-b846v" podUID="b55011b4-5278-4dc1-bb0e-391e4644d87a" containerName="main" probeResult="failure" output="Get \"https://10.133.0.57:8000/health\": dial tcp 10.133.0.57:8000: connect: connection refused"
Sep 17 18:17:57.052251 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:17:57.052215    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-djvzt_1d07e68d-9943-420f-a6a0-5f477792fabd/konnectivity-agent/0.log" limit=1024
Sep 17 18:17:57.210682 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:17:57.210585    2575 generic.go:356] "Generic (PLEG): container finished" podID="b55011b4-5278-4dc1-bb0e-391e4644d87a" containerID="6ec9e8c69930c9fd0adf9c3855ff8b23cf6cae8b44809f3e493ece9346f54c6d" exitCode=0
Sep 17 18:17:57.210682 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:17:57.210608    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-test-llm-inference-service-62fead0b/llmisvc-model-qwen2-5-0-5b-rout-4f8c0978-kserve-bb749b474-b846v" event={"ID":"b55011b4-5278-4dc1-bb0e-391e4644d87a","Type":"ContainerDied","Data":"6ec9e8c69930c9fd0adf9c3855ff8b23cf6cae8b44809f3e493ece9346f54c6d"}
Sep 17 18:17:57.311603 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:17:57.311580    2575 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-test-llm-inference-service-62fead0b/llmisvc-model-qwen2-5-0-5b-rout-4f8c0978-kserve-bb749b474-b846v"
Sep 17 18:17:57.398828 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:17:57.398782    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/b55011b4-5278-4dc1-bb0e-391e4644d87a-tls-certs\") pod \"b55011b4-5278-4dc1-bb0e-391e4644d87a\" (UID: \"b55011b4-5278-4dc1-bb0e-391e4644d87a\") "
Sep 17 18:17:57.398828 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:17:57.398833    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/b55011b4-5278-4dc1-bb0e-391e4644d87a-model-cache\") pod \"b55011b4-5278-4dc1-bb0e-391e4644d87a\" (UID: \"b55011b4-5278-4dc1-bb0e-391e4644d87a\") "
Sep 17 18:17:57.399081 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:17:57.398864    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-nc55q\" (UniqueName: \"kubernetes.io/projected/b55011b4-5278-4dc1-bb0e-391e4644d87a-kube-api-access-nc55q\") pod \"b55011b4-5278-4dc1-bb0e-391e4644d87a\" (UID: \"b55011b4-5278-4dc1-bb0e-391e4644d87a\") "
Sep 17 18:17:57.399081 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:17:57.398882    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/b55011b4-5278-4dc1-bb0e-391e4644d87a-home\") pod \"b55011b4-5278-4dc1-bb0e-391e4644d87a\" (UID: \"b55011b4-5278-4dc1-bb0e-391e4644d87a\") "
Sep 17 18:17:57.399081 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:17:57.398919    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/b55011b4-5278-4dc1-bb0e-391e4644d87a-tmp-dir\") pod \"b55011b4-5278-4dc1-bb0e-391e4644d87a\" (UID: \"b55011b4-5278-4dc1-bb0e-391e4644d87a\") "
Sep 17 18:17:57.399081 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:17:57.398988    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/b55011b4-5278-4dc1-bb0e-391e4644d87a-dshm\") pod \"b55011b4-5278-4dc1-bb0e-391e4644d87a\" (UID: \"b55011b4-5278-4dc1-bb0e-391e4644d87a\") "
Sep 17 18:17:57.399320 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:17:57.399180    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b55011b4-5278-4dc1-bb0e-391e4644d87a-home" (OuterVolumeSpecName: "home") pod "b55011b4-5278-4dc1-bb0e-391e4644d87a" (UID: "b55011b4-5278-4dc1-bb0e-391e4644d87a"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 17 18:17:57.399320 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:17:57.399195    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b55011b4-5278-4dc1-bb0e-391e4644d87a-model-cache" (OuterVolumeSpecName: "model-cache") pod "b55011b4-5278-4dc1-bb0e-391e4644d87a" (UID: "b55011b4-5278-4dc1-bb0e-391e4644d87a"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 17 18:17:57.399320 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:17:57.399279    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b55011b4-5278-4dc1-bb0e-391e4644d87a-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "b55011b4-5278-4dc1-bb0e-391e4644d87a" (UID: "b55011b4-5278-4dc1-bb0e-391e4644d87a"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 17 18:17:57.401152 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:17:57.401125    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b55011b4-5278-4dc1-bb0e-391e4644d87a-dshm" (OuterVolumeSpecName: "dshm") pod "b55011b4-5278-4dc1-bb0e-391e4644d87a" (UID: "b55011b4-5278-4dc1-bb0e-391e4644d87a"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 17 18:17:57.401152 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:17:57.401143    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/b55011b4-5278-4dc1-bb0e-391e4644d87a-kube-api-access-nc55q" (OuterVolumeSpecName: "kube-api-access-nc55q") pod "b55011b4-5278-4dc1-bb0e-391e4644d87a" (UID: "b55011b4-5278-4dc1-bb0e-391e4644d87a"). InnerVolumeSpecName "kube-api-access-nc55q". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 17 18:17:57.401269 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:17:57.401130    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/b55011b4-5278-4dc1-bb0e-391e4644d87a-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "b55011b4-5278-4dc1-bb0e-391e4644d87a" (UID: "b55011b4-5278-4dc1-bb0e-391e4644d87a"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 17 18:17:57.500330 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:17:57.500243    2575 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/b55011b4-5278-4dc1-bb0e-391e4644d87a-tmp-dir\") on node \"ip-10-0-143-74.ec2.internal\" DevicePath \"\""
Sep 17 18:17:57.500330 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:17:57.500282    2575 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/b55011b4-5278-4dc1-bb0e-391e4644d87a-dshm\") on node \"ip-10-0-143-74.ec2.internal\" DevicePath \"\""
Sep 17 18:17:57.500330 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:17:57.500291    2575 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/b55011b4-5278-4dc1-bb0e-391e4644d87a-tls-certs\") on node \"ip-10-0-143-74.ec2.internal\" DevicePath \"\""
Sep 17 18:17:57.500330 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:17:57.500299    2575 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/b55011b4-5278-4dc1-bb0e-391e4644d87a-model-cache\") on node \"ip-10-0-143-74.ec2.internal\" DevicePath \"\""
Sep 17 18:17:57.500330 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:17:57.500307    2575 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-nc55q\" (UniqueName: \"kubernetes.io/projected/b55011b4-5278-4dc1-bb0e-391e4644d87a-kube-api-access-nc55q\") on node \"ip-10-0-143-74.ec2.internal\" DevicePath \"\""
Sep 17 18:17:57.500330 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:17:57.500315    2575 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/b55011b4-5278-4dc1-bb0e-391e4644d87a-home\") on node \"ip-10-0-143-74.ec2.internal\" DevicePath \"\""
Sep 17 18:17:58.216937 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:17:58.216910    2575 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-test-llm-inference-service-62fead0b/llmisvc-model-qwen2-5-0-5b-rout-4f8c0978-kserve-bb749b474-b846v"
Sep 17 18:17:58.216937 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:17:58.216918    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-test-llm-inference-service-62fead0b/llmisvc-model-qwen2-5-0-5b-rout-4f8c0978-kserve-bb749b474-b846v" event={"ID":"b55011b4-5278-4dc1-bb0e-391e4644d87a","Type":"ContainerDied","Data":"9b5b363505faedd26c732424f3a7df9df02e0dc7e0cb726fc24f7a93fc528754"}
Sep 17 18:17:58.217501 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:17:58.216979    2575 scope.go:117] "RemoveContainer" containerID="6ec9e8c69930c9fd0adf9c3855ff8b23cf6cae8b44809f3e493ece9346f54c6d"
Sep 17 18:17:58.239283 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:17:58.239252    2575 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["e2e-test-llm-inference-service-62fead0b/llmisvc-model-qwen2-5-0-5b-rout-4f8c0978-kserve-bb749b474-b846v"]
Sep 17 18:17:58.241577 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:17:58.241550    2575 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["e2e-test-llm-inference-service-62fead0b/llmisvc-model-qwen2-5-0-5b-rout-4f8c0978-kserve-bb749b474-b846v"]
Sep 17 18:17:58.969651 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:17:58.969608    2575 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="b55011b4-5278-4dc1-bb0e-391e4644d87a" path="/var/lib/kubelet/pods/b55011b4-5278-4dc1-bb0e-391e4644d87a/volumes"
Sep 17 18:18:47.557740 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:18:47.557697    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-djvzt_1d07e68d-9943-420f-a6a0-5f477792fabd/konnectivity-agent/0.log" limit=1024
Sep 17 18:18:47.617128 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:18:47.617088    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-djvzt_1d07e68d-9943-420f-a6a0-5f477792fabd/konnectivity-agent/0.log" limit=1024
Sep 17 18:19:37.651800 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:19:37.651722    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-djvzt_1d07e68d-9943-420f-a6a0-5f477792fabd/konnectivity-agent/0.log" limit=1024
Sep 17 18:19:37.751975 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:19:37.751937    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-djvzt_1d07e68d-9943-420f-a6a0-5f477792fabd/konnectivity-agent/0.log" limit=1024
Sep 17 18:19:44.360793 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:19:44.360750    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-djvzt_1d07e68d-9943-420f-a6a0-5f477792fabd/konnectivity-agent/0.log" limit=1024
Sep 17 18:19:49.165184 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:19:49.165136    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-djvzt_1d07e68d-9943-420f-a6a0-5f477792fabd/konnectivity-agent/0.log" limit=1024
Sep 17 18:19:49.421429 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:19:49.421349    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-djvzt_1d07e68d-9943-420f-a6a0-5f477792fabd/konnectivity-agent/0.log" limit=1024
Sep 17 18:19:50.693346 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:19:50.693307    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-djvzt_1d07e68d-9943-420f-a6a0-5f477792fabd/konnectivity-agent/0.log" limit=1024
Sep 17 18:19:50.766533 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:19:50.766500    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-djvzt_1d07e68d-9943-420f-a6a0-5f477792fabd/konnectivity-agent/0.log" limit=1024
Sep 17 18:20:03.044504 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:20:03.044429    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-djvzt_1d07e68d-9943-420f-a6a0-5f477792fabd/konnectivity-agent/0.log" limit=1024
Sep 17 18:20:09.161011 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:20:09.160970    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-djvzt_1d07e68d-9943-420f-a6a0-5f477792fabd/konnectivity-agent/0.log" limit=1024
Sep 17 18:20:09.231081 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:20:09.231044    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-djvzt_1d07e68d-9943-420f-a6a0-5f477792fabd/konnectivity-agent/0.log" limit=1024
Sep 17 18:20:59.286190 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:20:59.286112    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-djvzt_1d07e68d-9943-420f-a6a0-5f477792fabd/konnectivity-agent/0.log" limit=1024
Sep 17 18:20:59.388011 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:20:59.387977    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-djvzt_1d07e68d-9943-420f-a6a0-5f477792fabd/konnectivity-agent/0.log" limit=1024
Sep 17 18:21:04.311361 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:21:04.311288    2575 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["e2e-test-llm-inference-service-caba0b4a/custom-route-timeout-test-kserve-797b6c779d-zqv6c"]
Sep 17 18:21:04.311748 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:21:04.311732    2575 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b55011b4-5278-4dc1-bb0e-391e4644d87a" containerName="main"
Sep 17 18:21:04.311790 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:21:04.311749    2575 state_mem.go:107] "Deleted CPUSet assignment" podUID="b55011b4-5278-4dc1-bb0e-391e4644d87a" containerName="main"
Sep 17 18:21:04.311845 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:21:04.311834    2575 memory_manager.go:368] "RemoveStaleState removing state" podUID="b55011b4-5278-4dc1-bb0e-391e4644d87a" containerName="main"
Sep 17 18:21:04.315037 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:21:04.315018    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-test-llm-inference-service-caba0b4a/custom-route-timeout-test-kserve-797b6c779d-zqv6c"
Sep 17 18:21:04.317610 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:21:04.317586    2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-test-llm-inference-service-caba0b4a\"/\"openshift-service-ca.crt\""
Sep 17 18:21:04.317743 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:21:04.317610    2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-test-llm-inference-service-caba0b4a\"/\"odh-kserve-custom-ca-bundle\""
Sep 17 18:21:04.317743 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:21:04.317625    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-test-llm-inference-service-caba0b4a\"/\"custom-route-timeout-test-kserve-self-signed-certs\""
Sep 17 18:21:04.317743 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:21:04.317642    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-test-llm-inference-service-caba0b4a\"/\"seaweedfs-s3-creds\""
Sep 17 18:21:04.317912 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:21:04.317748    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-test-llm-inference-service-caba0b4a\"/\"default-dockercfg-2wxbb\""
Sep 17 18:21:04.317912 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:21:04.317754    2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-test-llm-inference-service-caba0b4a\"/\"kube-root-ca.crt\""
Sep 17 18:21:04.329153 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:21:04.329132    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["e2e-test-llm-inference-service-caba0b4a/custom-route-timeout-test-kserve-797b6c779d-zqv6c"]
Sep 17 18:21:04.343397 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:21:04.343368    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-djvzt_1d07e68d-9943-420f-a6a0-5f477792fabd/konnectivity-agent/0.log" limit=1024
Sep 17 18:21:04.463559 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:21:04.463520    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/849412bd-0fdf-4a9c-86c0-058bcc3103e8-home\") pod \"custom-route-timeout-test-kserve-797b6c779d-zqv6c\" (UID: \"849412bd-0fdf-4a9c-86c0-058bcc3103e8\") " pod="e2e-test-llm-inference-service-caba0b4a/custom-route-timeout-test-kserve-797b6c779d-zqv6c"
Sep 17 18:21:04.463739 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:21:04.463574    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cabundle-cert\" (UniqueName: \"kubernetes.io/configmap/849412bd-0fdf-4a9c-86c0-058bcc3103e8-cabundle-cert\") pod \"custom-route-timeout-test-kserve-797b6c779d-zqv6c\" (UID: \"849412bd-0fdf-4a9c-86c0-058bcc3103e8\") " pod="e2e-test-llm-inference-service-caba0b4a/custom-route-timeout-test-kserve-797b6c779d-zqv6c"
Sep 17 18:21:04.463739 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:21:04.463605    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/849412bd-0fdf-4a9c-86c0-058bcc3103e8-dshm\") pod \"custom-route-timeout-test-kserve-797b6c779d-zqv6c\" (UID: \"849412bd-0fdf-4a9c-86c0-058bcc3103e8\") " pod="e2e-test-llm-inference-service-caba0b4a/custom-route-timeout-test-kserve-797b6c779d-zqv6c"
Sep 17 18:21:04.463739 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:21:04.463628    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/849412bd-0fdf-4a9c-86c0-058bcc3103e8-tmp-dir\") pod \"custom-route-timeout-test-kserve-797b6c779d-zqv6c\" (UID: \"849412bd-0fdf-4a9c-86c0-058bcc3103e8\") " pod="e2e-test-llm-inference-service-caba0b4a/custom-route-timeout-test-kserve-797b6c779d-zqv6c"
Sep 17 18:21:04.463739 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:21:04.463652    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/849412bd-0fdf-4a9c-86c0-058bcc3103e8-tls-certs\") pod \"custom-route-timeout-test-kserve-797b6c779d-zqv6c\" (UID: \"849412bd-0fdf-4a9c-86c0-058bcc3103e8\") " pod="e2e-test-llm-inference-service-caba0b4a/custom-route-timeout-test-kserve-797b6c779d-zqv6c"
Sep 17 18:21:04.463739 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:21:04.463732    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/849412bd-0fdf-4a9c-86c0-058bcc3103e8-kserve-provision-location\") pod \"custom-route-timeout-test-kserve-797b6c779d-zqv6c\" (UID: \"849412bd-0fdf-4a9c-86c0-058bcc3103e8\") " pod="e2e-test-llm-inference-service-caba0b4a/custom-route-timeout-test-kserve-797b6c779d-zqv6c"
Sep 17 18:21:04.463923 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:21:04.463781    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-j9x8h\" (UniqueName: \"kubernetes.io/projected/849412bd-0fdf-4a9c-86c0-058bcc3103e8-kube-api-access-j9x8h\") pod \"custom-route-timeout-test-kserve-797b6c779d-zqv6c\" (UID: \"849412bd-0fdf-4a9c-86c0-058bcc3103e8\") " pod="e2e-test-llm-inference-service-caba0b4a/custom-route-timeout-test-kserve-797b6c779d-zqv6c"
Sep 17 18:21:04.463923 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:21:04.463832    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/849412bd-0fdf-4a9c-86c0-058bcc3103e8-model-cache\") pod \"custom-route-timeout-test-kserve-797b6c779d-zqv6c\" (UID: \"849412bd-0fdf-4a9c-86c0-058bcc3103e8\") " pod="e2e-test-llm-inference-service-caba0b4a/custom-route-timeout-test-kserve-797b6c779d-zqv6c"
Sep 17 18:21:04.533157 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:21:04.533115    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-djvzt_1d07e68d-9943-420f-a6a0-5f477792fabd/konnectivity-agent/0.log" limit=1024
Sep 17 18:21:04.565325 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:21:04.565243    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/849412bd-0fdf-4a9c-86c0-058bcc3103e8-kserve-provision-location\") pod \"custom-route-timeout-test-kserve-797b6c779d-zqv6c\" (UID: \"849412bd-0fdf-4a9c-86c0-058bcc3103e8\") " pod="e2e-test-llm-inference-service-caba0b4a/custom-route-timeout-test-kserve-797b6c779d-zqv6c"
Sep 17 18:21:04.565325 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:21:04.565321    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-j9x8h\" (UniqueName: \"kubernetes.io/projected/849412bd-0fdf-4a9c-86c0-058bcc3103e8-kube-api-access-j9x8h\") pod \"custom-route-timeout-test-kserve-797b6c779d-zqv6c\" (UID: \"849412bd-0fdf-4a9c-86c0-058bcc3103e8\") " pod="e2e-test-llm-inference-service-caba0b4a/custom-route-timeout-test-kserve-797b6c779d-zqv6c"
Sep 17 18:21:04.565680 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:21:04.565385    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/849412bd-0fdf-4a9c-86c0-058bcc3103e8-model-cache\") pod \"custom-route-timeout-test-kserve-797b6c779d-zqv6c\" (UID: \"849412bd-0fdf-4a9c-86c0-058bcc3103e8\") " pod="e2e-test-llm-inference-service-caba0b4a/custom-route-timeout-test-kserve-797b6c779d-zqv6c"
Sep 17 18:21:04.565680 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:21:04.565459    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/849412bd-0fdf-4a9c-86c0-058bcc3103e8-home\") pod \"custom-route-timeout-test-kserve-797b6c779d-zqv6c\" (UID: \"849412bd-0fdf-4a9c-86c0-058bcc3103e8\") " pod="e2e-test-llm-inference-service-caba0b4a/custom-route-timeout-test-kserve-797b6c779d-zqv6c"
Sep 17 18:21:04.565680 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:21:04.565503    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cabundle-cert\" (UniqueName: \"kubernetes.io/configmap/849412bd-0fdf-4a9c-86c0-058bcc3103e8-cabundle-cert\") pod \"custom-route-timeout-test-kserve-797b6c779d-zqv6c\" (UID: \"849412bd-0fdf-4a9c-86c0-058bcc3103e8\") " pod="e2e-test-llm-inference-service-caba0b4a/custom-route-timeout-test-kserve-797b6c779d-zqv6c"
Sep 17 18:21:04.565680 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:21:04.565538    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/849412bd-0fdf-4a9c-86c0-058bcc3103e8-dshm\") pod \"custom-route-timeout-test-kserve-797b6c779d-zqv6c\" (UID: \"849412bd-0fdf-4a9c-86c0-058bcc3103e8\") " pod="e2e-test-llm-inference-service-caba0b4a/custom-route-timeout-test-kserve-797b6c779d-zqv6c"
Sep 17 18:21:04.565680 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:21:04.565565    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/849412bd-0fdf-4a9c-86c0-058bcc3103e8-tmp-dir\") pod \"custom-route-timeout-test-kserve-797b6c779d-zqv6c\" (UID: \"849412bd-0fdf-4a9c-86c0-058bcc3103e8\") " pod="e2e-test-llm-inference-service-caba0b4a/custom-route-timeout-test-kserve-797b6c779d-zqv6c"
Sep 17 18:21:04.565680 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:21:04.565594    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/849412bd-0fdf-4a9c-86c0-058bcc3103e8-tls-certs\") pod \"custom-route-timeout-test-kserve-797b6c779d-zqv6c\" (UID: \"849412bd-0fdf-4a9c-86c0-058bcc3103e8\") " pod="e2e-test-llm-inference-service-caba0b4a/custom-route-timeout-test-kserve-797b6c779d-zqv6c"
Sep 17 18:21:04.565680 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:21:04.565668    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/849412bd-0fdf-4a9c-86c0-058bcc3103e8-kserve-provision-location\") pod \"custom-route-timeout-test-kserve-797b6c779d-zqv6c\" (UID: \"849412bd-0fdf-4a9c-86c0-058bcc3103e8\") " pod="e2e-test-llm-inference-service-caba0b4a/custom-route-timeout-test-kserve-797b6c779d-zqv6c"
Sep 17 18:21:04.566000 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:21:04.565744    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/849412bd-0fdf-4a9c-86c0-058bcc3103e8-model-cache\") pod \"custom-route-timeout-test-kserve-797b6c779d-zqv6c\" (UID: \"849412bd-0fdf-4a9c-86c0-058bcc3103e8\") " pod="e2e-test-llm-inference-service-caba0b4a/custom-route-timeout-test-kserve-797b6c779d-zqv6c"
Sep 17 18:21:04.566000 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:21:04.565798    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/849412bd-0fdf-4a9c-86c0-058bcc3103e8-home\") pod \"custom-route-timeout-test-kserve-797b6c779d-zqv6c\" (UID: \"849412bd-0fdf-4a9c-86c0-058bcc3103e8\") " pod="e2e-test-llm-inference-service-caba0b4a/custom-route-timeout-test-kserve-797b6c779d-zqv6c"
Sep 17 18:21:04.566000 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:21:04.565913    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/849412bd-0fdf-4a9c-86c0-058bcc3103e8-tmp-dir\") pod \"custom-route-timeout-test-kserve-797b6c779d-zqv6c\" (UID: \"849412bd-0fdf-4a9c-86c0-058bcc3103e8\") " pod="e2e-test-llm-inference-service-caba0b4a/custom-route-timeout-test-kserve-797b6c779d-zqv6c"
Sep 17 18:21:04.566186 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:21:04.566167    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cabundle-cert\" (UniqueName: \"kubernetes.io/configmap/849412bd-0fdf-4a9c-86c0-058bcc3103e8-cabundle-cert\") pod \"custom-route-timeout-test-kserve-797b6c779d-zqv6c\" (UID: \"849412bd-0fdf-4a9c-86c0-058bcc3103e8\") " pod="e2e-test-llm-inference-service-caba0b4a/custom-route-timeout-test-kserve-797b6c779d-zqv6c"
Sep 17 18:21:04.568097 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:21:04.568071    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/849412bd-0fdf-4a9c-86c0-058bcc3103e8-dshm\") pod \"custom-route-timeout-test-kserve-797b6c779d-zqv6c\" (UID: \"849412bd-0fdf-4a9c-86c0-058bcc3103e8\") " pod="e2e-test-llm-inference-service-caba0b4a/custom-route-timeout-test-kserve-797b6c779d-zqv6c"
Sep 17 18:21:04.568220 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:21:04.568152    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/849412bd-0fdf-4a9c-86c0-058bcc3103e8-tls-certs\") pod \"custom-route-timeout-test-kserve-797b6c779d-zqv6c\" (UID: \"849412bd-0fdf-4a9c-86c0-058bcc3103e8\") " pod="e2e-test-llm-inference-service-caba0b4a/custom-route-timeout-test-kserve-797b6c779d-zqv6c"
Sep 17 18:21:04.573884 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:21:04.573864    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-j9x8h\" (UniqueName: \"kubernetes.io/projected/849412bd-0fdf-4a9c-86c0-058bcc3103e8-kube-api-access-j9x8h\") pod \"custom-route-timeout-test-kserve-797b6c779d-zqv6c\" (UID: \"849412bd-0fdf-4a9c-86c0-058bcc3103e8\") " pod="e2e-test-llm-inference-service-caba0b4a/custom-route-timeout-test-kserve-797b6c779d-zqv6c"
Sep 17 18:21:04.625195 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:21:04.625164    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-test-llm-inference-service-caba0b4a/custom-route-timeout-test-kserve-797b6c779d-zqv6c"
Sep 17 18:21:04.906903 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:21:04.906877    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["e2e-test-llm-inference-service-caba0b4a/custom-route-timeout-test-kserve-797b6c779d-zqv6c"]
Sep 17 18:21:04.909190 ip-10-0-143-74 kubenswrapper[2575]: W0917 18:21:04.909162    2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod849412bd_0fdf_4a9c_86c0_058bcc3103e8.slice/crio-2e285aacf1247bbb715b061514660147c1553ca6ac8398fe70fa905939dc7de1 WatchSource:0}: Error finding container 2e285aacf1247bbb715b061514660147c1553ca6ac8398fe70fa905939dc7de1: Status 404 returned error can't find the container with id 2e285aacf1247bbb715b061514660147c1553ca6ac8398fe70fa905939dc7de1
Sep 17 18:21:05.203014 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:21:05.202926    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-test-llm-inference-service-caba0b4a/custom-route-timeout-test-kserve-797b6c779d-zqv6c" event={"ID":"849412bd-0fdf-4a9c-86c0-058bcc3103e8","Type":"ContainerStarted","Data":"4a7bec6b866aca762e7ecf5f9c882ae836289763f15816e4ebce65137e500aa4"}
Sep 17 18:21:05.203014 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:21:05.203004    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-test-llm-inference-service-caba0b4a/custom-route-timeout-test-kserve-797b6c779d-zqv6c" event={"ID":"849412bd-0fdf-4a9c-86c0-058bcc3103e8","Type":"ContainerStarted","Data":"2e285aacf1247bbb715b061514660147c1553ca6ac8398fe70fa905939dc7de1"}
Sep 17 18:21:06.206347 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:21:06.206314    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-djvzt_1d07e68d-9943-420f-a6a0-5f477792fabd/konnectivity-agent/0.log" limit=1024
Sep 17 18:21:08.220609 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:21:08.220570    2575 generic.go:356] "Generic (PLEG): container finished" podID="849412bd-0fdf-4a9c-86c0-058bcc3103e8" containerID="4a7bec6b866aca762e7ecf5f9c882ae836289763f15816e4ebce65137e500aa4" exitCode=0
Sep 17 18:21:08.221011 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:21:08.220643    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-test-llm-inference-service-caba0b4a/custom-route-timeout-test-kserve-797b6c779d-zqv6c" event={"ID":"849412bd-0fdf-4a9c-86c0-058bcc3103e8","Type":"ContainerDied","Data":"4a7bec6b866aca762e7ecf5f9c882ae836289763f15816e4ebce65137e500aa4"}
Sep 17 18:21:09.228155 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:21:09.228122    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-test-llm-inference-service-caba0b4a/custom-route-timeout-test-kserve-797b6c779d-zqv6c" event={"ID":"849412bd-0fdf-4a9c-86c0-058bcc3103e8","Type":"ContainerStarted","Data":"7859e704110b3101677851b5e721793d62656bbf8429be51fe92fb5d34047cc9"}
Sep 17 18:21:09.256634 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:21:09.256580    2575 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="e2e-test-llm-inference-service-caba0b4a/custom-route-timeout-test-kserve-797b6c779d-zqv6c" podStartSLOduration=2.978651079 podStartE2EDuration="5.256562274s" totalImagesPullingTime="0s" totalInitContainerRuntime="2.277911195s" isStatefulPod=false podCreationTimestamp="2026-09-17 18:21:04 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-17 18:21:09.254218099 +0000 UTC m=+2788.884548073" watchObservedRunningTime="2026-09-17 18:21:09.256562274 +0000 UTC m=+2788.886892248"
Sep 17 18:21:09.625690 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:21:09.625659    2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-test-llm-inference-service-caba0b4a/custom-route-timeout-test-kserve-797b6c779d-zqv6c"
Sep 17 18:21:10.886131 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:21:10.886098    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-djvzt_1d07e68d-9943-420f-a6a0-5f477792fabd/konnectivity-agent/0.log" limit=1024
Sep 17 18:21:11.995487 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:21:11.995455    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-djvzt_1d07e68d-9943-420f-a6a0-5f477792fabd/konnectivity-agent/0.log" limit=1024
Sep 17 18:21:12.095785 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:21:12.095755    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-djvzt_1d07e68d-9943-420f-a6a0-5f477792fabd/konnectivity-agent/0.log" limit=1024
Sep 17 18:21:14.626045 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:21:14.626004    2575 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="e2e-test-llm-inference-service-caba0b4a/custom-route-timeout-test-kserve-797b6c779d-zqv6c"
Sep 17 18:21:14.627801 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:21:14.627772    2575 prober.go:120] "Probe failed" probeType="Startup" pod="e2e-test-llm-inference-service-caba0b4a/custom-route-timeout-test-kserve-797b6c779d-zqv6c" podUID="849412bd-0fdf-4a9c-86c0-058bcc3103e8" containerName="main" probeResult="failure" output="Get \"https://10.133.0.58:8000/health\": dial tcp 10.133.0.58:8000: connect: connection refused"
Sep 17 18:21:24.626194 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:21:24.626143    2575 prober.go:120] "Probe failed" probeType="Startup" pod="e2e-test-llm-inference-service-caba0b4a/custom-route-timeout-test-kserve-797b6c779d-zqv6c" podUID="849412bd-0fdf-4a9c-86c0-058bcc3103e8" containerName="main" probeResult="failure" output="Get \"https://10.133.0.58:8000/health\": dial tcp 10.133.0.58:8000: connect: connection refused"
Sep 17 18:21:26.383912 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:21:26.383881    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-djvzt_1d07e68d-9943-420f-a6a0-5f477792fabd/konnectivity-agent/0.log" limit=1024
Sep 17 18:21:34.625644 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:21:34.625589    2575 prober.go:120] "Probe failed" probeType="Startup" pod="e2e-test-llm-inference-service-caba0b4a/custom-route-timeout-test-kserve-797b6c779d-zqv6c" podUID="849412bd-0fdf-4a9c-86c0-058bcc3103e8" containerName="main" probeResult="failure" output="Get \"https://10.133.0.58:8000/health\": dial tcp 10.133.0.58:8000: connect: connection refused"
Sep 17 18:21:36.851541 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:21:36.851498    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-djvzt_1d07e68d-9943-420f-a6a0-5f477792fabd/konnectivity-agent/0.log" limit=1024
Sep 17 18:21:36.920153 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:21:36.920120    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-djvzt_1d07e68d-9943-420f-a6a0-5f477792fabd/konnectivity-agent/0.log" limit=1024
Sep 17 18:21:44.626627 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:21:44.626574    2575 prober.go:120] "Probe failed" probeType="Startup" pod="e2e-test-llm-inference-service-caba0b4a/custom-route-timeout-test-kserve-797b6c779d-zqv6c" podUID="849412bd-0fdf-4a9c-86c0-058bcc3103e8" containerName="main" probeResult="failure" output="Get \"https://10.133.0.58:8000/health\": dial tcp 10.133.0.58:8000: connect: connection refused"
Sep 17 18:21:54.626655 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:21:54.626610    2575 prober.go:120] "Probe failed" probeType="Startup" pod="e2e-test-llm-inference-service-caba0b4a/custom-route-timeout-test-kserve-797b6c779d-zqv6c" podUID="849412bd-0fdf-4a9c-86c0-058bcc3103e8" containerName="main" probeResult="failure" output="Get \"https://10.133.0.58:8000/health\": dial tcp 10.133.0.58:8000: connect: connection refused"
Sep 17 18:22:04.626104 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:22:04.626051    2575 prober.go:120] "Probe failed" probeType="Startup" pod="e2e-test-llm-inference-service-caba0b4a/custom-route-timeout-test-kserve-797b6c779d-zqv6c" podUID="849412bd-0fdf-4a9c-86c0-058bcc3103e8" containerName="main" probeResult="failure" output="Get \"https://10.133.0.58:8000/health\": dial tcp 10.133.0.58:8000: connect: connection refused"
Sep 17 18:22:14.626206 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:22:14.626151    2575 prober.go:120] "Probe failed" probeType="Startup" pod="e2e-test-llm-inference-service-caba0b4a/custom-route-timeout-test-kserve-797b6c779d-zqv6c" podUID="849412bd-0fdf-4a9c-86c0-058bcc3103e8" containerName="main" probeResult="failure" output="Get \"https://10.133.0.58:8000/health\": dial tcp 10.133.0.58:8000: connect: connection refused"
Sep 17 18:22:24.626159 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:22:24.626101    2575 prober.go:120] "Probe failed" probeType="Startup" pod="e2e-test-llm-inference-service-caba0b4a/custom-route-timeout-test-kserve-797b6c779d-zqv6c" podUID="849412bd-0fdf-4a9c-86c0-058bcc3103e8" containerName="main" probeResult="failure" output="Get \"https://10.133.0.58:8000/health\": dial tcp 10.133.0.58:8000: connect: connection refused"
Sep 17 18:22:26.797154 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:22:26.797113    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-djvzt_1d07e68d-9943-420f-a6a0-5f477792fabd/konnectivity-agent/0.log" limit=1024
Sep 17 18:22:26.886902 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:22:26.886872    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-djvzt_1d07e68d-9943-420f-a6a0-5f477792fabd/konnectivity-agent/0.log" limit=1024
Sep 17 18:22:33.023505 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:22:33.023467    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-djvzt_1d07e68d-9943-420f-a6a0-5f477792fabd/konnectivity-agent/0.log" limit=1024
Sep 17 18:22:34.626153 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:22:34.626109    2575 prober.go:120] "Probe failed" probeType="Startup" pod="e2e-test-llm-inference-service-caba0b4a/custom-route-timeout-test-kserve-797b6c779d-zqv6c" podUID="849412bd-0fdf-4a9c-86c0-058bcc3103e8" containerName="main" probeResult="failure" output="Get \"https://10.133.0.58:8000/health\": dial tcp 10.133.0.58:8000: connect: connection refused"
Sep 17 18:22:35.517695 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:22:35.517657    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-djvzt_1d07e68d-9943-420f-a6a0-5f477792fabd/konnectivity-agent/0.log" limit=1024
Sep 17 18:22:35.816238 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:22:35.816153    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-djvzt_1d07e68d-9943-420f-a6a0-5f477792fabd/konnectivity-agent/0.log" limit=1024
Sep 17 18:22:39.491200 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:22:39.491164    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-djvzt_1d07e68d-9943-420f-a6a0-5f477792fabd/konnectivity-agent/0.log" limit=1024
Sep 17 18:22:39.551188 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:22:39.551155    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-djvzt_1d07e68d-9943-420f-a6a0-5f477792fabd/konnectivity-agent/0.log" limit=1024
Sep 17 18:22:44.626648 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:22:44.626603    2575 prober.go:120] "Probe failed" probeType="Startup" pod="e2e-test-llm-inference-service-caba0b4a/custom-route-timeout-test-kserve-797b6c779d-zqv6c" podUID="849412bd-0fdf-4a9c-86c0-058bcc3103e8" containerName="main" probeResult="failure" output="Get \"https://10.133.0.58:8000/health\": dial tcp 10.133.0.58:8000: connect: connection refused"
Sep 17 18:22:46.555392 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:22:46.555354    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-djvzt_1d07e68d-9943-420f-a6a0-5f477792fabd/konnectivity-agent/0.log" limit=1024
Sep 17 18:22:54.626570 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:22:54.626505    2575 prober.go:120] "Probe failed" probeType="Startup" pod="e2e-test-llm-inference-service-caba0b4a/custom-route-timeout-test-kserve-797b6c779d-zqv6c" podUID="849412bd-0fdf-4a9c-86c0-058bcc3103e8" containerName="main" probeResult="failure" output="Get \"https://10.133.0.58:8000/health\": dial tcp 10.133.0.58:8000: connect: connection refused"
Sep 17 18:23:04.636129 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:23:04.636095    2575 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="e2e-test-llm-inference-service-caba0b4a/custom-route-timeout-test-kserve-797b6c779d-zqv6c"
Sep 17 18:23:04.643566 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:23:04.643544    2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-test-llm-inference-service-caba0b4a/custom-route-timeout-test-kserve-797b6c779d-zqv6c"
Sep 17 18:23:09.165330 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:23:09.165288    2575 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["e2e-test-llm-inference-service-caba0b4a/custom-route-timeout-test-kserve-797b6c779d-zqv6c"]
Sep 17 18:23:09.167161 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:23:09.167118    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-djvzt_1d07e68d-9943-420f-a6a0-5f477792fabd/konnectivity-agent/0.log" limit=1024
Sep 17 18:23:09.230360 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:23:09.230327    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-djvzt_1d07e68d-9943-420f-a6a0-5f477792fabd/konnectivity-agent/0.log" limit=1024
Sep 17 18:23:24.187927 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:23:24.187876    2575 kuberuntime_container.go:873] "Killing container with a grace period" pod="e2e-test-llm-inference-service-caba0b4a/custom-route-timeout-test-kserve-797b6c779d-zqv6c" podUID="849412bd-0fdf-4a9c-86c0-058bcc3103e8" containerName="main" containerID="cri-o://7859e704110b3101677851b5e721793d62656bbf8429be51fe92fb5d34047cc9" gracePeriod=45
Sep 17 18:23:24.644377 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:23:24.637401    2575 prober.go:120] "Probe failed" probeType="Readiness" pod="e2e-test-llm-inference-service-caba0b4a/custom-route-timeout-test-kserve-797b6c779d-zqv6c" podUID="849412bd-0fdf-4a9c-86c0-058bcc3103e8" containerName="main" probeResult="failure" output="Get \"https://10.133.0.58:8000/health\": dial tcp 10.133.0.58:8000: connect: connection refused"
Sep 17 18:23:24.953711 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:23:24.953687    2575 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-test-llm-inference-service-caba0b4a/custom-route-timeout-test-kserve-797b6c779d-zqv6c"
Sep 17 18:23:24.998524 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:23:24.998484    2575 generic.go:356] "Generic (PLEG): container finished" podID="849412bd-0fdf-4a9c-86c0-058bcc3103e8" containerID="7859e704110b3101677851b5e721793d62656bbf8429be51fe92fb5d34047cc9" exitCode=0
Sep 17 18:23:24.998678 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:23:24.998550    2575 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-test-llm-inference-service-caba0b4a/custom-route-timeout-test-kserve-797b6c779d-zqv6c"
Sep 17 18:23:24.998678 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:23:24.998568    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-test-llm-inference-service-caba0b4a/custom-route-timeout-test-kserve-797b6c779d-zqv6c" event={"ID":"849412bd-0fdf-4a9c-86c0-058bcc3103e8","Type":"ContainerDied","Data":"7859e704110b3101677851b5e721793d62656bbf8429be51fe92fb5d34047cc9"}
Sep 17 18:23:24.998678 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:23:24.998619    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-test-llm-inference-service-caba0b4a/custom-route-timeout-test-kserve-797b6c779d-zqv6c" event={"ID":"849412bd-0fdf-4a9c-86c0-058bcc3103e8","Type":"ContainerDied","Data":"2e285aacf1247bbb715b061514660147c1553ca6ac8398fe70fa905939dc7de1"}
Sep 17 18:23:24.998678 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:23:24.998646    2575 scope.go:117] "RemoveContainer" containerID="7859e704110b3101677851b5e721793d62656bbf8429be51fe92fb5d34047cc9"
Sep 17 18:23:25.007881 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:23:25.007865    2575 scope.go:117] "RemoveContainer" containerID="4a7bec6b866aca762e7ecf5f9c882ae836289763f15816e4ebce65137e500aa4"
Sep 17 18:23:25.015739 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:23:25.015724    2575 scope.go:117] "RemoveContainer" containerID="7859e704110b3101677851b5e721793d62656bbf8429be51fe92fb5d34047cc9"
Sep 17 18:23:25.016019 ip-10-0-143-74 kubenswrapper[2575]: E0917 18:23:25.015984    2575 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"7859e704110b3101677851b5e721793d62656bbf8429be51fe92fb5d34047cc9\": container with ID starting with 7859e704110b3101677851b5e721793d62656bbf8429be51fe92fb5d34047cc9 not found: ID does not exist" containerID="7859e704110b3101677851b5e721793d62656bbf8429be51fe92fb5d34047cc9"
Sep 17 18:23:25.016070 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:23:25.016026    2575 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"7859e704110b3101677851b5e721793d62656bbf8429be51fe92fb5d34047cc9"} err="failed to get container status \"7859e704110b3101677851b5e721793d62656bbf8429be51fe92fb5d34047cc9\": rpc error: code = NotFound desc = could not find container \"7859e704110b3101677851b5e721793d62656bbf8429be51fe92fb5d34047cc9\": container with ID starting with 7859e704110b3101677851b5e721793d62656bbf8429be51fe92fb5d34047cc9 not found: ID does not exist"
Sep 17 18:23:25.016070 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:23:25.016041    2575 scope.go:117] "RemoveContainer" containerID="4a7bec6b866aca762e7ecf5f9c882ae836289763f15816e4ebce65137e500aa4"
Sep 17 18:23:25.016270 ip-10-0-143-74 kubenswrapper[2575]: E0917 18:23:25.016254    2575 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"4a7bec6b866aca762e7ecf5f9c882ae836289763f15816e4ebce65137e500aa4\": container with ID starting with 4a7bec6b866aca762e7ecf5f9c882ae836289763f15816e4ebce65137e500aa4 not found: ID does not exist" containerID="4a7bec6b866aca762e7ecf5f9c882ae836289763f15816e4ebce65137e500aa4"
Sep 17 18:23:25.016321 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:23:25.016275    2575 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"4a7bec6b866aca762e7ecf5f9c882ae836289763f15816e4ebce65137e500aa4"} err="failed to get container status \"4a7bec6b866aca762e7ecf5f9c882ae836289763f15816e4ebce65137e500aa4\": rpc error: code = NotFound desc = could not find container \"4a7bec6b866aca762e7ecf5f9c882ae836289763f15816e4ebce65137e500aa4\": container with ID starting with 4a7bec6b866aca762e7ecf5f9c882ae836289763f15816e4ebce65137e500aa4 not found: ID does not exist"
Sep 17 18:23:25.035615 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:23:25.035587    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"cabundle-cert\" (UniqueName: \"kubernetes.io/configmap/849412bd-0fdf-4a9c-86c0-058bcc3103e8-cabundle-cert\") pod \"849412bd-0fdf-4a9c-86c0-058bcc3103e8\" (UID: \"849412bd-0fdf-4a9c-86c0-058bcc3103e8\") "
Sep 17 18:23:25.035727 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:23:25.035691    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/849412bd-0fdf-4a9c-86c0-058bcc3103e8-tls-certs\") pod \"849412bd-0fdf-4a9c-86c0-058bcc3103e8\" (UID: \"849412bd-0fdf-4a9c-86c0-058bcc3103e8\") "
Sep 17 18:23:25.035727 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:23:25.035718    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/849412bd-0fdf-4a9c-86c0-058bcc3103e8-kserve-provision-location\") pod \"849412bd-0fdf-4a9c-86c0-058bcc3103e8\" (UID: \"849412bd-0fdf-4a9c-86c0-058bcc3103e8\") "
Sep 17 18:23:25.035839 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:23:25.035762    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/849412bd-0fdf-4a9c-86c0-058bcc3103e8-tmp-dir\") pod \"849412bd-0fdf-4a9c-86c0-058bcc3103e8\" (UID: \"849412bd-0fdf-4a9c-86c0-058bcc3103e8\") "
Sep 17 18:23:25.035839 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:23:25.035809    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-j9x8h\" (UniqueName: \"kubernetes.io/projected/849412bd-0fdf-4a9c-86c0-058bcc3103e8-kube-api-access-j9x8h\") pod \"849412bd-0fdf-4a9c-86c0-058bcc3103e8\" (UID: \"849412bd-0fdf-4a9c-86c0-058bcc3103e8\") "
Sep 17 18:23:25.035940 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:23:25.035848    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/849412bd-0fdf-4a9c-86c0-058bcc3103e8-model-cache\") pod \"849412bd-0fdf-4a9c-86c0-058bcc3103e8\" (UID: \"849412bd-0fdf-4a9c-86c0-058bcc3103e8\") "
Sep 17 18:23:25.035940 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:23:25.035886    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/849412bd-0fdf-4a9c-86c0-058bcc3103e8-dshm\") pod \"849412bd-0fdf-4a9c-86c0-058bcc3103e8\" (UID: \"849412bd-0fdf-4a9c-86c0-058bcc3103e8\") "
Sep 17 18:23:25.035940 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:23:25.035914    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/849412bd-0fdf-4a9c-86c0-058bcc3103e8-home\") pod \"849412bd-0fdf-4a9c-86c0-058bcc3103e8\" (UID: \"849412bd-0fdf-4a9c-86c0-058bcc3103e8\") "
Sep 17 18:23:25.036081 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:23:25.035949    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/849412bd-0fdf-4a9c-86c0-058bcc3103e8-cabundle-cert" (OuterVolumeSpecName: "cabundle-cert") pod "849412bd-0fdf-4a9c-86c0-058bcc3103e8" (UID: "849412bd-0fdf-4a9c-86c0-058bcc3103e8"). InnerVolumeSpecName "cabundle-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 17 18:23:25.036201 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:23:25.036177    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/849412bd-0fdf-4a9c-86c0-058bcc3103e8-model-cache" (OuterVolumeSpecName: "model-cache") pod "849412bd-0fdf-4a9c-86c0-058bcc3103e8" (UID: "849412bd-0fdf-4a9c-86c0-058bcc3103e8"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 17 18:23:25.036318 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:23:25.036214    2575 reconciler_common.go:299] "Volume detached for volume \"cabundle-cert\" (UniqueName: \"kubernetes.io/configmap/849412bd-0fdf-4a9c-86c0-058bcc3103e8-cabundle-cert\") on node \"ip-10-0-143-74.ec2.internal\" DevicePath \"\""
Sep 17 18:23:25.036806 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:23:25.036787    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/849412bd-0fdf-4a9c-86c0-058bcc3103e8-home" (OuterVolumeSpecName: "home") pod "849412bd-0fdf-4a9c-86c0-058bcc3103e8" (UID: "849412bd-0fdf-4a9c-86c0-058bcc3103e8"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 17 18:23:25.040138 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:23:25.040108    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/849412bd-0fdf-4a9c-86c0-058bcc3103e8-kube-api-access-j9x8h" (OuterVolumeSpecName: "kube-api-access-j9x8h") pod "849412bd-0fdf-4a9c-86c0-058bcc3103e8" (UID: "849412bd-0fdf-4a9c-86c0-058bcc3103e8"). InnerVolumeSpecName "kube-api-access-j9x8h". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 17 18:23:25.040477 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:23:25.040432    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/849412bd-0fdf-4a9c-86c0-058bcc3103e8-dshm" (OuterVolumeSpecName: "dshm") pod "849412bd-0fdf-4a9c-86c0-058bcc3103e8" (UID: "849412bd-0fdf-4a9c-86c0-058bcc3103e8"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 17 18:23:25.040477 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:23:25.040432    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/849412bd-0fdf-4a9c-86c0-058bcc3103e8-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "849412bd-0fdf-4a9c-86c0-058bcc3103e8" (UID: "849412bd-0fdf-4a9c-86c0-058bcc3103e8"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 17 18:23:25.051498 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:23:25.051470    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/849412bd-0fdf-4a9c-86c0-058bcc3103e8-kserve-provision-location" (OuterVolumeSpecName: "kserve-provision-location") pod "849412bd-0fdf-4a9c-86c0-058bcc3103e8" (UID: "849412bd-0fdf-4a9c-86c0-058bcc3103e8"). InnerVolumeSpecName "kserve-provision-location". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 17 18:23:25.053240 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:23:25.053218    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/849412bd-0fdf-4a9c-86c0-058bcc3103e8-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "849412bd-0fdf-4a9c-86c0-058bcc3103e8" (UID: "849412bd-0fdf-4a9c-86c0-058bcc3103e8"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 17 18:23:25.137556 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:23:25.137527    2575 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/849412bd-0fdf-4a9c-86c0-058bcc3103e8-model-cache\") on node \"ip-10-0-143-74.ec2.internal\" DevicePath \"\""
Sep 17 18:23:25.137556 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:23:25.137552    2575 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/849412bd-0fdf-4a9c-86c0-058bcc3103e8-dshm\") on node \"ip-10-0-143-74.ec2.internal\" DevicePath \"\""
Sep 17 18:23:25.137556 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:23:25.137561    2575 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/849412bd-0fdf-4a9c-86c0-058bcc3103e8-home\") on node \"ip-10-0-143-74.ec2.internal\" DevicePath \"\""
Sep 17 18:23:25.137750 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:23:25.137569    2575 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/849412bd-0fdf-4a9c-86c0-058bcc3103e8-tls-certs\") on node \"ip-10-0-143-74.ec2.internal\" DevicePath \"\""
Sep 17 18:23:25.137750 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:23:25.137578    2575 reconciler_common.go:299] "Volume detached for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/849412bd-0fdf-4a9c-86c0-058bcc3103e8-kserve-provision-location\") on node \"ip-10-0-143-74.ec2.internal\" DevicePath \"\""
Sep 17 18:23:25.137750 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:23:25.137586    2575 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/849412bd-0fdf-4a9c-86c0-058bcc3103e8-tmp-dir\") on node \"ip-10-0-143-74.ec2.internal\" DevicePath \"\""
Sep 17 18:23:25.137750 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:23:25.137598    2575 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-j9x8h\" (UniqueName: \"kubernetes.io/projected/849412bd-0fdf-4a9c-86c0-058bcc3103e8-kube-api-access-j9x8h\") on node \"ip-10-0-143-74.ec2.internal\" DevicePath \"\""
Sep 17 18:23:25.323819 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:23:25.323783    2575 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["e2e-test-llm-inference-service-caba0b4a/custom-route-timeout-test-kserve-797b6c779d-zqv6c"]
Sep 17 18:23:25.327853 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:23:25.327825    2575 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["e2e-test-llm-inference-service-caba0b4a/custom-route-timeout-test-kserve-797b6c779d-zqv6c"]
Sep 17 18:23:26.967753 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:23:26.967718    2575 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="849412bd-0fdf-4a9c-86c0-058bcc3103e8" path="/var/lib/kubelet/pods/849412bd-0fdf-4a9c-86c0-058bcc3103e8/volumes"
Sep 17 18:23:52.219185 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:23:52.219152    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-djvzt_1d07e68d-9943-420f-a6a0-5f477792fabd/konnectivity-agent/0.log" limit=1024
Sep 17 18:23:57.671953 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:23:57.671913    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-djvzt_1d07e68d-9943-420f-a6a0-5f477792fabd/konnectivity-agent/0.log" limit=1024
Sep 17 18:23:57.735229 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:23:57.735196    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-djvzt_1d07e68d-9943-420f-a6a0-5f477792fabd/konnectivity-agent/0.log" limit=1024
Sep 17 18:23:59.304104 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:23:59.304072    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-djvzt_1d07e68d-9943-420f-a6a0-5f477792fabd/konnectivity-agent/0.log" limit=1024
Sep 17 18:23:59.392782 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:23:59.392752    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-djvzt_1d07e68d-9943-420f-a6a0-5f477792fabd/konnectivity-agent/0.log" limit=1024
Sep 17 18:23:59.528027 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:23:59.527992    2575 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["e2e-test-llm-inference-service-028f7809/router-gateway-1-openshift-default-cc7cb45dd-srd8v"]
Sep 17 18:23:59.528586 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:23:59.528567    2575 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="849412bd-0fdf-4a9c-86c0-058bcc3103e8" containerName="main"
Sep 17 18:23:59.528648 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:23:59.528591    2575 state_mem.go:107] "Deleted CPUSet assignment" podUID="849412bd-0fdf-4a9c-86c0-058bcc3103e8" containerName="main"
Sep 17 18:23:59.528648 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:23:59.528605    2575 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="849412bd-0fdf-4a9c-86c0-058bcc3103e8" containerName="storage-initializer"
Sep 17 18:23:59.528648 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:23:59.528614    2575 state_mem.go:107] "Deleted CPUSet assignment" podUID="849412bd-0fdf-4a9c-86c0-058bcc3103e8" containerName="storage-initializer"
Sep 17 18:23:59.528749 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:23:59.528720    2575 memory_manager.go:368] "RemoveStaleState removing state" podUID="849412bd-0fdf-4a9c-86c0-058bcc3103e8" containerName="main"
Sep 17 18:23:59.533142 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:23:59.533121    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-test-llm-inference-service-028f7809/router-gateway-1-openshift-default-cc7cb45dd-srd8v"
Sep 17 18:23:59.535818 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:23:59.535794    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-test-llm-inference-service-028f7809\"/\"router-gateway-1-openshift-default-dockercfg-twn5c\""
Sep 17 18:23:59.535944 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:23:59.535794    2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-test-llm-inference-service-028f7809\"/\"openshift-service-ca.crt\""
Sep 17 18:23:59.536188 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:23:59.536164    2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-test-llm-inference-service-028f7809\"/\"openshift-gw-ca-root-cert\""
Sep 17 18:23:59.536289 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:23:59.536191    2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-test-llm-inference-service-028f7809\"/\"kube-root-ca.crt\""
Sep 17 18:23:59.543577 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:23:59.543551    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["e2e-test-llm-inference-service-028f7809/router-gateway-1-openshift-default-cc7cb45dd-srd8v"]
Sep 17 18:23:59.563827 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:23:59.563765    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-djvzt_1d07e68d-9943-420f-a6a0-5f477792fabd/konnectivity-agent/0.log" limit=1024
Sep 17 18:23:59.621471 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:23:59.621427    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/83113b83-f4a3-4836-a6a2-18f5e825be12-istio-podinfo\") pod \"router-gateway-1-openshift-default-cc7cb45dd-srd8v\" (UID: \"83113b83-f4a3-4836-a6a2-18f5e825be12\") " pod="e2e-test-llm-inference-service-028f7809/router-gateway-1-openshift-default-cc7cb45dd-srd8v"
Sep 17 18:23:59.621637 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:23:59.621492    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/83113b83-f4a3-4836-a6a2-18f5e825be12-istio-envoy\") pod \"router-gateway-1-openshift-default-cc7cb45dd-srd8v\" (UID: \"83113b83-f4a3-4836-a6a2-18f5e825be12\") " pod="e2e-test-llm-inference-service-028f7809/router-gateway-1-openshift-default-cc7cb45dd-srd8v"
Sep 17 18:23:59.621637 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:23:59.621543    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/83113b83-f4a3-4836-a6a2-18f5e825be12-credential-socket\") pod \"router-gateway-1-openshift-default-cc7cb45dd-srd8v\" (UID: \"83113b83-f4a3-4836-a6a2-18f5e825be12\") " pod="e2e-test-llm-inference-service-028f7809/router-gateway-1-openshift-default-cc7cb45dd-srd8v"
Sep 17 18:23:59.621637 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:23:59.621582    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/83113b83-f4a3-4836-a6a2-18f5e825be12-istio-data\") pod \"router-gateway-1-openshift-default-cc7cb45dd-srd8v\" (UID: \"83113b83-f4a3-4836-a6a2-18f5e825be12\") " pod="e2e-test-llm-inference-service-028f7809/router-gateway-1-openshift-default-cc7cb45dd-srd8v"
Sep 17 18:23:59.621637 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:23:59.621632    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/83113b83-f4a3-4836-a6a2-18f5e825be12-workload-socket\") pod \"router-gateway-1-openshift-default-cc7cb45dd-srd8v\" (UID: \"83113b83-f4a3-4836-a6a2-18f5e825be12\") " pod="e2e-test-llm-inference-service-028f7809/router-gateway-1-openshift-default-cc7cb45dd-srd8v"
Sep 17 18:23:59.621777 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:23:59.621654    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/83113b83-f4a3-4836-a6a2-18f5e825be12-workload-certs\") pod \"router-gateway-1-openshift-default-cc7cb45dd-srd8v\" (UID: \"83113b83-f4a3-4836-a6a2-18f5e825be12\") " pod="e2e-test-llm-inference-service-028f7809/router-gateway-1-openshift-default-cc7cb45dd-srd8v"
Sep 17 18:23:59.621777 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:23:59.621670    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/83113b83-f4a3-4836-a6a2-18f5e825be12-istio-token\") pod \"router-gateway-1-openshift-default-cc7cb45dd-srd8v\" (UID: \"83113b83-f4a3-4836-a6a2-18f5e825be12\") " pod="e2e-test-llm-inference-service-028f7809/router-gateway-1-openshift-default-cc7cb45dd-srd8v"
Sep 17 18:23:59.621777 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:23:59.621688    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/83113b83-f4a3-4836-a6a2-18f5e825be12-istiod-ca-cert\") pod \"router-gateway-1-openshift-default-cc7cb45dd-srd8v\" (UID: \"83113b83-f4a3-4836-a6a2-18f5e825be12\") " pod="e2e-test-llm-inference-service-028f7809/router-gateway-1-openshift-default-cc7cb45dd-srd8v"
Sep 17 18:23:59.621777 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:23:59.621747    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mspjg\" (UniqueName: \"kubernetes.io/projected/83113b83-f4a3-4836-a6a2-18f5e825be12-kube-api-access-mspjg\") pod \"router-gateway-1-openshift-default-cc7cb45dd-srd8v\" (UID: \"83113b83-f4a3-4836-a6a2-18f5e825be12\") " pod="e2e-test-llm-inference-service-028f7809/router-gateway-1-openshift-default-cc7cb45dd-srd8v"
Sep 17 18:23:59.662841 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:23:59.662799    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-djvzt_1d07e68d-9943-420f-a6a0-5f477792fabd/konnectivity-agent/0.log" limit=1024
Sep 17 18:23:59.722741 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:23:59.722704    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/83113b83-f4a3-4836-a6a2-18f5e825be12-istio-podinfo\") pod \"router-gateway-1-openshift-default-cc7cb45dd-srd8v\" (UID: \"83113b83-f4a3-4836-a6a2-18f5e825be12\") " pod="e2e-test-llm-inference-service-028f7809/router-gateway-1-openshift-default-cc7cb45dd-srd8v"
Sep 17 18:23:59.722910 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:23:59.722749    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/83113b83-f4a3-4836-a6a2-18f5e825be12-istio-envoy\") pod \"router-gateway-1-openshift-default-cc7cb45dd-srd8v\" (UID: \"83113b83-f4a3-4836-a6a2-18f5e825be12\") " pod="e2e-test-llm-inference-service-028f7809/router-gateway-1-openshift-default-cc7cb45dd-srd8v"
Sep 17 18:23:59.722910 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:23:59.722799    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/83113b83-f4a3-4836-a6a2-18f5e825be12-credential-socket\") pod \"router-gateway-1-openshift-default-cc7cb45dd-srd8v\" (UID: \"83113b83-f4a3-4836-a6a2-18f5e825be12\") " pod="e2e-test-llm-inference-service-028f7809/router-gateway-1-openshift-default-cc7cb45dd-srd8v"
Sep 17 18:23:59.722910 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:23:59.722818    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/83113b83-f4a3-4836-a6a2-18f5e825be12-istio-data\") pod \"router-gateway-1-openshift-default-cc7cb45dd-srd8v\" (UID: \"83113b83-f4a3-4836-a6a2-18f5e825be12\") " pod="e2e-test-llm-inference-service-028f7809/router-gateway-1-openshift-default-cc7cb45dd-srd8v"
Sep 17 18:23:59.723045 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:23:59.722951    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/83113b83-f4a3-4836-a6a2-18f5e825be12-workload-socket\") pod \"router-gateway-1-openshift-default-cc7cb45dd-srd8v\" (UID: \"83113b83-f4a3-4836-a6a2-18f5e825be12\") " pod="e2e-test-llm-inference-service-028f7809/router-gateway-1-openshift-default-cc7cb45dd-srd8v"
Sep 17 18:23:59.723600 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:23:59.723573    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/83113b83-f4a3-4836-a6a2-18f5e825be12-credential-socket\") pod \"router-gateway-1-openshift-default-cc7cb45dd-srd8v\" (UID: \"83113b83-f4a3-4836-a6a2-18f5e825be12\") " pod="e2e-test-llm-inference-service-028f7809/router-gateway-1-openshift-default-cc7cb45dd-srd8v"
Sep 17 18:23:59.723719 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:23:59.722997    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/83113b83-f4a3-4836-a6a2-18f5e825be12-workload-certs\") pod \"router-gateway-1-openshift-default-cc7cb45dd-srd8v\" (UID: \"83113b83-f4a3-4836-a6a2-18f5e825be12\") " pod="e2e-test-llm-inference-service-028f7809/router-gateway-1-openshift-default-cc7cb45dd-srd8v"
Sep 17 18:23:59.723719 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:23:59.723657    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/83113b83-f4a3-4836-a6a2-18f5e825be12-istio-token\") pod \"router-gateway-1-openshift-default-cc7cb45dd-srd8v\" (UID: \"83113b83-f4a3-4836-a6a2-18f5e825be12\") " pod="e2e-test-llm-inference-service-028f7809/router-gateway-1-openshift-default-cc7cb45dd-srd8v"
Sep 17 18:23:59.723719 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:23:59.723670    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/83113b83-f4a3-4836-a6a2-18f5e825be12-istio-data\") pod \"router-gateway-1-openshift-default-cc7cb45dd-srd8v\" (UID: \"83113b83-f4a3-4836-a6a2-18f5e825be12\") " pod="e2e-test-llm-inference-service-028f7809/router-gateway-1-openshift-default-cc7cb45dd-srd8v"
Sep 17 18:23:59.723992 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:23:59.723696    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/83113b83-f4a3-4836-a6a2-18f5e825be12-istiod-ca-cert\") pod \"router-gateway-1-openshift-default-cc7cb45dd-srd8v\" (UID: \"83113b83-f4a3-4836-a6a2-18f5e825be12\") " pod="e2e-test-llm-inference-service-028f7809/router-gateway-1-openshift-default-cc7cb45dd-srd8v"
Sep 17 18:23:59.724328 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:23:59.724301    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/83113b83-f4a3-4836-a6a2-18f5e825be12-workload-certs\") pod \"router-gateway-1-openshift-default-cc7cb45dd-srd8v\" (UID: \"83113b83-f4a3-4836-a6a2-18f5e825be12\") " pod="e2e-test-llm-inference-service-028f7809/router-gateway-1-openshift-default-cc7cb45dd-srd8v"
Sep 17 18:23:59.724430 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:23:59.724388    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/83113b83-f4a3-4836-a6a2-18f5e825be12-workload-socket\") pod \"router-gateway-1-openshift-default-cc7cb45dd-srd8v\" (UID: \"83113b83-f4a3-4836-a6a2-18f5e825be12\") " pod="e2e-test-llm-inference-service-028f7809/router-gateway-1-openshift-default-cc7cb45dd-srd8v"
Sep 17 18:23:59.724816 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:23:59.724793    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/83113b83-f4a3-4836-a6a2-18f5e825be12-istiod-ca-cert\") pod \"router-gateway-1-openshift-default-cc7cb45dd-srd8v\" (UID: \"83113b83-f4a3-4836-a6a2-18f5e825be12\") " pod="e2e-test-llm-inference-service-028f7809/router-gateway-1-openshift-default-cc7cb45dd-srd8v"
Sep 17 18:23:59.726038 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:23:59.726013    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/83113b83-f4a3-4836-a6a2-18f5e825be12-istio-envoy\") pod \"router-gateway-1-openshift-default-cc7cb45dd-srd8v\" (UID: \"83113b83-f4a3-4836-a6a2-18f5e825be12\") " pod="e2e-test-llm-inference-service-028f7809/router-gateway-1-openshift-default-cc7cb45dd-srd8v"
Sep 17 18:23:59.726238 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:23:59.726216    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/83113b83-f4a3-4836-a6a2-18f5e825be12-istio-podinfo\") pod \"router-gateway-1-openshift-default-cc7cb45dd-srd8v\" (UID: \"83113b83-f4a3-4836-a6a2-18f5e825be12\") " pod="e2e-test-llm-inference-service-028f7809/router-gateway-1-openshift-default-cc7cb45dd-srd8v"
Sep 17 18:23:59.731881 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:23:59.731851    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mspjg\" (UniqueName: \"kubernetes.io/projected/83113b83-f4a3-4836-a6a2-18f5e825be12-kube-api-access-mspjg\") pod \"router-gateway-1-openshift-default-cc7cb45dd-srd8v\" (UID: \"83113b83-f4a3-4836-a6a2-18f5e825be12\") " pod="e2e-test-llm-inference-service-028f7809/router-gateway-1-openshift-default-cc7cb45dd-srd8v"
Sep 17 18:23:59.733421 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:23:59.733401    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/83113b83-f4a3-4836-a6a2-18f5e825be12-istio-token\") pod \"router-gateway-1-openshift-default-cc7cb45dd-srd8v\" (UID: \"83113b83-f4a3-4836-a6a2-18f5e825be12\") " pod="e2e-test-llm-inference-service-028f7809/router-gateway-1-openshift-default-cc7cb45dd-srd8v"
Sep 17 18:23:59.739893 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:23:59.739872    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-mspjg\" (UniqueName: \"kubernetes.io/projected/83113b83-f4a3-4836-a6a2-18f5e825be12-kube-api-access-mspjg\") pod \"router-gateway-1-openshift-default-cc7cb45dd-srd8v\" (UID: \"83113b83-f4a3-4836-a6a2-18f5e825be12\") " pod="e2e-test-llm-inference-service-028f7809/router-gateway-1-openshift-default-cc7cb45dd-srd8v"
Sep 17 18:23:59.846744 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:23:59.846665    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-test-llm-inference-service-028f7809/router-gateway-1-openshift-default-cc7cb45dd-srd8v"
Sep 17 18:24:00.129234 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:24:00.129208    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["e2e-test-llm-inference-service-028f7809/router-gateway-1-openshift-default-cc7cb45dd-srd8v"]
Sep 17 18:24:00.131117 ip-10-0-143-74 kubenswrapper[2575]: W0917 18:24:00.131094    2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod83113b83_f4a3_4836_a6a2_18f5e825be12.slice/crio-710c0ccca51eca82d007e8747e0c4ef0d0b4ec6aa6920d9a2cb4ac50b9593eeb WatchSource:0}: Error finding container 710c0ccca51eca82d007e8747e0c4ef0d0b4ec6aa6920d9a2cb4ac50b9593eeb: Status 404 returned error can't find the container with id 710c0ccca51eca82d007e8747e0c4ef0d0b4ec6aa6920d9a2cb4ac50b9593eeb
Sep 17 18:24:00.133040 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:24:00.133022    2575 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Sep 17 18:24:00.191034 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:24:00.190986    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-test-llm-inference-service-028f7809/router-gateway-1-openshift-default-cc7cb45dd-srd8v" event={"ID":"83113b83-f4a3-4836-a6a2-18f5e825be12","Type":"ContainerStarted","Data":"710c0ccca51eca82d007e8747e0c4ef0d0b4ec6aa6920d9a2cb4ac50b9593eeb"}
Sep 17 18:24:03.192324 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:24:03.192279    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-djvzt_1d07e68d-9943-420f-a6a0-5f477792fabd/konnectivity-agent/0.log" limit=1024
Sep 17 18:24:03.391196 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:24:03.391152    2575 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236216Ki","pods":"250"}
Sep 17 18:24:03.391268 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:24:03.391234    2575 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236216Ki","pods":"250"}
Sep 17 18:24:03.391268 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:24:03.391261    2575 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236216Ki","pods":"250"}
Sep 17 18:24:04.215058 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:24:04.215021    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-test-llm-inference-service-028f7809/router-gateway-1-openshift-default-cc7cb45dd-srd8v" event={"ID":"83113b83-f4a3-4836-a6a2-18f5e825be12","Type":"ContainerStarted","Data":"dd645fd08d7b50a41ac57c2807cd00cf76fdf816123f397268c581dc6ceaede7"}
Sep 17 18:24:04.236499 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:24:04.236436    2575 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="e2e-test-llm-inference-service-028f7809/router-gateway-1-openshift-default-cc7cb45dd-srd8v" podStartSLOduration=1.978647188 podStartE2EDuration="5.236420575s" totalImagesPullingTime="3.257773387s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-17 18:23:59 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-17 18:24:04.234968721 +0000 UTC m=+2963.865298696" watchObservedRunningTime="2026-09-17 18:24:04.236420575 +0000 UTC m=+2963.866750550"
Sep 17 18:24:04.849278 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:24:04.849236    2575 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="e2e-test-llm-inference-service-028f7809/router-gateway-1-openshift-default-cc7cb45dd-srd8v"
Sep 17 18:24:04.849488 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:24:04.849394    2575 prober.go:120] "Probe failed" probeType="Startup" pod="e2e-test-llm-inference-service-028f7809/router-gateway-1-openshift-default-cc7cb45dd-srd8v" podUID="83113b83-f4a3-4836-a6a2-18f5e825be12" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.59:15021/healthz/ready\": dial tcp 10.133.0.59:15021: connect: connection refused"
Sep 17 18:24:05.448874 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:24:05.448839    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-djvzt_1d07e68d-9943-420f-a6a0-5f477792fabd/konnectivity-agent/0.log" limit=1024
Sep 17 18:24:05.847242 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:24:05.847131    2575 prober.go:120] "Probe failed" probeType="Startup" pod="e2e-test-llm-inference-service-028f7809/router-gateway-1-openshift-default-cc7cb45dd-srd8v" podUID="83113b83-f4a3-4836-a6a2-18f5e825be12" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.59:15021/healthz/ready\": dial tcp 10.133.0.59:15021: connect: connection refused"
Sep 17 18:24:06.847801 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:24:06.847754    2575 prober.go:120] "Probe failed" probeType="Startup" pod="e2e-test-llm-inference-service-028f7809/router-gateway-1-openshift-default-cc7cb45dd-srd8v" podUID="83113b83-f4a3-4836-a6a2-18f5e825be12" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.59:15021/healthz/ready\": dial tcp 10.133.0.59:15021: connect: connection refused"
Sep 17 18:24:07.848070 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:24:07.848021    2575 prober.go:120] "Probe failed" probeType="Startup" pod="e2e-test-llm-inference-service-028f7809/router-gateway-1-openshift-default-cc7cb45dd-srd8v" podUID="83113b83-f4a3-4836-a6a2-18f5e825be12" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.59:15021/healthz/ready\": dial tcp 10.133.0.59:15021: connect: connection refused"
Sep 17 18:24:08.847815 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:24:08.847765    2575 prober.go:120] "Probe failed" probeType="Startup" pod="e2e-test-llm-inference-service-028f7809/router-gateway-1-openshift-default-cc7cb45dd-srd8v" podUID="83113b83-f4a3-4836-a6a2-18f5e825be12" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.59:15021/healthz/ready\": dial tcp 10.133.0.59:15021: connect: connection refused"
Sep 17 18:24:09.851599 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:24:09.851568    2575 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="e2e-test-llm-inference-service-028f7809/router-gateway-1-openshift-default-cc7cb45dd-srd8v"
Sep 17 18:24:09.851979 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:24:09.851959    2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-test-llm-inference-service-028f7809/router-gateway-1-openshift-default-cc7cb45dd-srd8v"
Sep 17 18:24:09.852683 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:24:09.852664    2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-test-llm-inference-service-028f7809/router-gateway-1-openshift-default-cc7cb45dd-srd8v"
Sep 17 18:24:11.157298 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:24:11.157264    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-djvzt_1d07e68d-9943-420f-a6a0-5f477792fabd/konnectivity-agent/0.log" limit=1024
Sep 17 18:24:11.217280 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:24:11.217251    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-djvzt_1d07e68d-9943-420f-a6a0-5f477792fabd/konnectivity-agent/0.log" limit=1024
Sep 17 18:24:19.369422 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:24:19.369387    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-djvzt_1d07e68d-9943-420f-a6a0-5f477792fabd/konnectivity-agent/0.log" limit=1024
Sep 17 18:24:19.810708 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:24:19.810615    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-djvzt_1d07e68d-9943-420f-a6a0-5f477792fabd/konnectivity-agent/0.log" limit=1024
Sep 17 18:24:19.910602 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:24:19.910569    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-djvzt_1d07e68d-9943-420f-a6a0-5f477792fabd/konnectivity-agent/0.log" limit=1024
Sep 17 18:24:32.318677 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:24:32.318638    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-djvzt_1d07e68d-9943-420f-a6a0-5f477792fabd/konnectivity-agent/0.log" limit=1024
Sep 17 18:24:47.233783 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:24:47.233742    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-djvzt_1d07e68d-9943-420f-a6a0-5f477792fabd/konnectivity-agent/0.log" limit=1024
Sep 17 18:24:47.326342 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:24:47.326304    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-djvzt_1d07e68d-9943-420f-a6a0-5f477792fabd/konnectivity-agent/0.log" limit=1024
Sep 17 18:24:51.505886 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:24:51.505853    2575 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["e2e-test-llm-stop-feature-d9eef2a1/stop-feature-test-kserve-c54f4856d-gr2xm"]
Sep 17 18:24:51.512015 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:24:51.511990    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-test-llm-stop-feature-d9eef2a1/stop-feature-test-kserve-c54f4856d-gr2xm"
Sep 17 18:24:51.515038 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:24:51.515007    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-test-llm-stop-feature-d9eef2a1\"/\"default-dockercfg-jhjnz\""
Sep 17 18:24:51.515038 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:24:51.515009    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-test-llm-stop-feature-d9eef2a1\"/\"seaweedfs-s3-creds\""
Sep 17 18:24:51.515245 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:24:51.515046    2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-test-llm-stop-feature-d9eef2a1\"/\"odh-kserve-custom-ca-bundle\""
Sep 17 18:24:51.515302 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:24:51.515244    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-test-llm-stop-feature-d9eef2a1\"/\"stop-feature-test-kserve-self-signed-certs\""
Sep 17 18:24:51.515302 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:24:51.515272    2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-test-llm-stop-feature-d9eef2a1\"/\"openshift-service-ca.crt\""
Sep 17 18:24:51.515302 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:24:51.515298    2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-test-llm-stop-feature-d9eef2a1\"/\"kube-root-ca.crt\""
Sep 17 18:24:51.522585 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:24:51.522562    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["e2e-test-llm-stop-feature-d9eef2a1/stop-feature-test-kserve-c54f4856d-gr2xm"]
Sep 17 18:24:51.534059 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:24:51.534032    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-djvzt_1d07e68d-9943-420f-a6a0-5f477792fabd/konnectivity-agent/0.log" limit=1024
Sep 17 18:24:51.585986 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:24:51.585952    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/696ad0c4-b80c-4c49-a1af-7d35979c6b40-tmp-dir\") pod \"stop-feature-test-kserve-c54f4856d-gr2xm\" (UID: \"696ad0c4-b80c-4c49-a1af-7d35979c6b40\") " pod="e2e-test-llm-stop-feature-d9eef2a1/stop-feature-test-kserve-c54f4856d-gr2xm"
Sep 17 18:24:51.585986 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:24:51.585989    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/696ad0c4-b80c-4c49-a1af-7d35979c6b40-tls-certs\") pod \"stop-feature-test-kserve-c54f4856d-gr2xm\" (UID: \"696ad0c4-b80c-4c49-a1af-7d35979c6b40\") " pod="e2e-test-llm-stop-feature-d9eef2a1/stop-feature-test-kserve-c54f4856d-gr2xm"
Sep 17 18:24:51.586205 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:24:51.586025    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cabundle-cert\" (UniqueName: \"kubernetes.io/configmap/696ad0c4-b80c-4c49-a1af-7d35979c6b40-cabundle-cert\") pod \"stop-feature-test-kserve-c54f4856d-gr2xm\" (UID: \"696ad0c4-b80c-4c49-a1af-7d35979c6b40\") " pod="e2e-test-llm-stop-feature-d9eef2a1/stop-feature-test-kserve-c54f4856d-gr2xm"
Sep 17 18:24:51.586205 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:24:51.586054    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/696ad0c4-b80c-4c49-a1af-7d35979c6b40-dshm\") pod \"stop-feature-test-kserve-c54f4856d-gr2xm\" (UID: \"696ad0c4-b80c-4c49-a1af-7d35979c6b40\") " pod="e2e-test-llm-stop-feature-d9eef2a1/stop-feature-test-kserve-c54f4856d-gr2xm"
Sep 17 18:24:51.586205 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:24:51.586069    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/696ad0c4-b80c-4c49-a1af-7d35979c6b40-model-cache\") pod \"stop-feature-test-kserve-c54f4856d-gr2xm\" (UID: \"696ad0c4-b80c-4c49-a1af-7d35979c6b40\") " pod="e2e-test-llm-stop-feature-d9eef2a1/stop-feature-test-kserve-c54f4856d-gr2xm"
Sep 17 18:24:51.586205 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:24:51.586085    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/696ad0c4-b80c-4c49-a1af-7d35979c6b40-home\") pod \"stop-feature-test-kserve-c54f4856d-gr2xm\" (UID: \"696ad0c4-b80c-4c49-a1af-7d35979c6b40\") " pod="e2e-test-llm-stop-feature-d9eef2a1/stop-feature-test-kserve-c54f4856d-gr2xm"
Sep 17 18:24:51.586205 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:24:51.586102    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-p5hqf\" (UniqueName: \"kubernetes.io/projected/696ad0c4-b80c-4c49-a1af-7d35979c6b40-kube-api-access-p5hqf\") pod \"stop-feature-test-kserve-c54f4856d-gr2xm\" (UID: \"696ad0c4-b80c-4c49-a1af-7d35979c6b40\") " pod="e2e-test-llm-stop-feature-d9eef2a1/stop-feature-test-kserve-c54f4856d-gr2xm"
Sep 17 18:24:51.586205 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:24:51.586158    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/696ad0c4-b80c-4c49-a1af-7d35979c6b40-kserve-provision-location\") pod \"stop-feature-test-kserve-c54f4856d-gr2xm\" (UID: \"696ad0c4-b80c-4c49-a1af-7d35979c6b40\") " pod="e2e-test-llm-stop-feature-d9eef2a1/stop-feature-test-kserve-c54f4856d-gr2xm"
Sep 17 18:24:51.686840 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:24:51.686803    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/696ad0c4-b80c-4c49-a1af-7d35979c6b40-kserve-provision-location\") pod \"stop-feature-test-kserve-c54f4856d-gr2xm\" (UID: \"696ad0c4-b80c-4c49-a1af-7d35979c6b40\") " pod="e2e-test-llm-stop-feature-d9eef2a1/stop-feature-test-kserve-c54f4856d-gr2xm"
Sep 17 18:24:51.687025 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:24:51.686884    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/696ad0c4-b80c-4c49-a1af-7d35979c6b40-tmp-dir\") pod \"stop-feature-test-kserve-c54f4856d-gr2xm\" (UID: \"696ad0c4-b80c-4c49-a1af-7d35979c6b40\") " pod="e2e-test-llm-stop-feature-d9eef2a1/stop-feature-test-kserve-c54f4856d-gr2xm"
Sep 17 18:24:51.687025 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:24:51.686920    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/696ad0c4-b80c-4c49-a1af-7d35979c6b40-tls-certs\") pod \"stop-feature-test-kserve-c54f4856d-gr2xm\" (UID: \"696ad0c4-b80c-4c49-a1af-7d35979c6b40\") " pod="e2e-test-llm-stop-feature-d9eef2a1/stop-feature-test-kserve-c54f4856d-gr2xm"
Sep 17 18:24:51.687025 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:24:51.686973    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cabundle-cert\" (UniqueName: \"kubernetes.io/configmap/696ad0c4-b80c-4c49-a1af-7d35979c6b40-cabundle-cert\") pod \"stop-feature-test-kserve-c54f4856d-gr2xm\" (UID: \"696ad0c4-b80c-4c49-a1af-7d35979c6b40\") " pod="e2e-test-llm-stop-feature-d9eef2a1/stop-feature-test-kserve-c54f4856d-gr2xm"
Sep 17 18:24:51.687025 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:24:51.687003    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/696ad0c4-b80c-4c49-a1af-7d35979c6b40-dshm\") pod \"stop-feature-test-kserve-c54f4856d-gr2xm\" (UID: \"696ad0c4-b80c-4c49-a1af-7d35979c6b40\") " pod="e2e-test-llm-stop-feature-d9eef2a1/stop-feature-test-kserve-c54f4856d-gr2xm"
Sep 17 18:24:51.687025 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:24:51.687021    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/696ad0c4-b80c-4c49-a1af-7d35979c6b40-model-cache\") pod \"stop-feature-test-kserve-c54f4856d-gr2xm\" (UID: \"696ad0c4-b80c-4c49-a1af-7d35979c6b40\") " pod="e2e-test-llm-stop-feature-d9eef2a1/stop-feature-test-kserve-c54f4856d-gr2xm"
Sep 17 18:24:51.687291 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:24:51.687045    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/696ad0c4-b80c-4c49-a1af-7d35979c6b40-home\") pod \"stop-feature-test-kserve-c54f4856d-gr2xm\" (UID: \"696ad0c4-b80c-4c49-a1af-7d35979c6b40\") " pod="e2e-test-llm-stop-feature-d9eef2a1/stop-feature-test-kserve-c54f4856d-gr2xm"
Sep 17 18:24:51.687291 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:24:51.687078    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-p5hqf\" (UniqueName: \"kubernetes.io/projected/696ad0c4-b80c-4c49-a1af-7d35979c6b40-kube-api-access-p5hqf\") pod \"stop-feature-test-kserve-c54f4856d-gr2xm\" (UID: \"696ad0c4-b80c-4c49-a1af-7d35979c6b40\") " pod="e2e-test-llm-stop-feature-d9eef2a1/stop-feature-test-kserve-c54f4856d-gr2xm"
Sep 17 18:24:51.687392 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:24:51.687313    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/696ad0c4-b80c-4c49-a1af-7d35979c6b40-tmp-dir\") pod \"stop-feature-test-kserve-c54f4856d-gr2xm\" (UID: \"696ad0c4-b80c-4c49-a1af-7d35979c6b40\") " pod="e2e-test-llm-stop-feature-d9eef2a1/stop-feature-test-kserve-c54f4856d-gr2xm"
Sep 17 18:24:51.687392 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:24:51.687348    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/696ad0c4-b80c-4c49-a1af-7d35979c6b40-model-cache\") pod \"stop-feature-test-kserve-c54f4856d-gr2xm\" (UID: \"696ad0c4-b80c-4c49-a1af-7d35979c6b40\") " pod="e2e-test-llm-stop-feature-d9eef2a1/stop-feature-test-kserve-c54f4856d-gr2xm"
Sep 17 18:24:51.687513 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:24:51.687431    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/696ad0c4-b80c-4c49-a1af-7d35979c6b40-home\") pod \"stop-feature-test-kserve-c54f4856d-gr2xm\" (UID: \"696ad0c4-b80c-4c49-a1af-7d35979c6b40\") " pod="e2e-test-llm-stop-feature-d9eef2a1/stop-feature-test-kserve-c54f4856d-gr2xm"
Sep 17 18:24:51.687513 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:24:51.687503    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/696ad0c4-b80c-4c49-a1af-7d35979c6b40-kserve-provision-location\") pod \"stop-feature-test-kserve-c54f4856d-gr2xm\" (UID: \"696ad0c4-b80c-4c49-a1af-7d35979c6b40\") " pod="e2e-test-llm-stop-feature-d9eef2a1/stop-feature-test-kserve-c54f4856d-gr2xm"
Sep 17 18:24:51.687766 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:24:51.687747    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cabundle-cert\" (UniqueName: \"kubernetes.io/configmap/696ad0c4-b80c-4c49-a1af-7d35979c6b40-cabundle-cert\") pod \"stop-feature-test-kserve-c54f4856d-gr2xm\" (UID: \"696ad0c4-b80c-4c49-a1af-7d35979c6b40\") " pod="e2e-test-llm-stop-feature-d9eef2a1/stop-feature-test-kserve-c54f4856d-gr2xm"
Sep 17 18:24:51.689764 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:24:51.689737    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/696ad0c4-b80c-4c49-a1af-7d35979c6b40-dshm\") pod \"stop-feature-test-kserve-c54f4856d-gr2xm\" (UID: \"696ad0c4-b80c-4c49-a1af-7d35979c6b40\") " pod="e2e-test-llm-stop-feature-d9eef2a1/stop-feature-test-kserve-c54f4856d-gr2xm"
Sep 17 18:24:51.689938 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:24:51.689920    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/696ad0c4-b80c-4c49-a1af-7d35979c6b40-tls-certs\") pod \"stop-feature-test-kserve-c54f4856d-gr2xm\" (UID: \"696ad0c4-b80c-4c49-a1af-7d35979c6b40\") " pod="e2e-test-llm-stop-feature-d9eef2a1/stop-feature-test-kserve-c54f4856d-gr2xm"
Sep 17 18:24:51.695483 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:24:51.695456    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-p5hqf\" (UniqueName: \"kubernetes.io/projected/696ad0c4-b80c-4c49-a1af-7d35979c6b40-kube-api-access-p5hqf\") pod \"stop-feature-test-kserve-c54f4856d-gr2xm\" (UID: \"696ad0c4-b80c-4c49-a1af-7d35979c6b40\") " pod="e2e-test-llm-stop-feature-d9eef2a1/stop-feature-test-kserve-c54f4856d-gr2xm"
Sep 17 18:24:51.824094 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:24:51.824005    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-test-llm-stop-feature-d9eef2a1/stop-feature-test-kserve-c54f4856d-gr2xm"
Sep 17 18:24:51.850213 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:24:51.850176    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-djvzt_1d07e68d-9943-420f-a6a0-5f477792fabd/konnectivity-agent/0.log" limit=1024
Sep 17 18:24:52.124786 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:24:52.124756    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["e2e-test-llm-stop-feature-d9eef2a1/stop-feature-test-kserve-c54f4856d-gr2xm"]
Sep 17 18:24:52.127540 ip-10-0-143-74 kubenswrapper[2575]: W0917 18:24:52.127515    2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod696ad0c4_b80c_4c49_a1af_7d35979c6b40.slice/crio-7644d5cf59c8a6d257a8b1e8b2ff3bf3b4f98cbce3c429f27f12458df6617490 WatchSource:0}: Error finding container 7644d5cf59c8a6d257a8b1e8b2ff3bf3b4f98cbce3c429f27f12458df6617490: Status 404 returned error can't find the container with id 7644d5cf59c8a6d257a8b1e8b2ff3bf3b4f98cbce3c429f27f12458df6617490
Sep 17 18:24:52.472669 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:24:52.472586    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-test-llm-stop-feature-d9eef2a1/stop-feature-test-kserve-c54f4856d-gr2xm" event={"ID":"696ad0c4-b80c-4c49-a1af-7d35979c6b40","Type":"ContainerStarted","Data":"21cd053421b561dc1fe8600c014254de1746a7cf9f7b721e20a73f1ac757b390"}
Sep 17 18:24:52.472669 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:24:52.472643    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-test-llm-stop-feature-d9eef2a1/stop-feature-test-kserve-c54f4856d-gr2xm" event={"ID":"696ad0c4-b80c-4c49-a1af-7d35979c6b40","Type":"ContainerStarted","Data":"7644d5cf59c8a6d257a8b1e8b2ff3bf3b4f98cbce3c429f27f12458df6617490"}
Sep 17 18:24:54.954588 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:24:54.954554    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-djvzt_1d07e68d-9943-420f-a6a0-5f477792fabd/konnectivity-agent/0.log" limit=1024
Sep 17 18:24:55.489286 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:24:55.489256    2575 generic.go:356] "Generic (PLEG): container finished" podID="696ad0c4-b80c-4c49-a1af-7d35979c6b40" containerID="21cd053421b561dc1fe8600c014254de1746a7cf9f7b721e20a73f1ac757b390" exitCode=0
Sep 17 18:24:55.489486 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:24:55.489295    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-test-llm-stop-feature-d9eef2a1/stop-feature-test-kserve-c54f4856d-gr2xm" event={"ID":"696ad0c4-b80c-4c49-a1af-7d35979c6b40","Type":"ContainerDied","Data":"21cd053421b561dc1fe8600c014254de1746a7cf9f7b721e20a73f1ac757b390"}
Sep 17 18:24:56.496226 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:24:56.496192    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-test-llm-stop-feature-d9eef2a1/stop-feature-test-kserve-c54f4856d-gr2xm" event={"ID":"696ad0c4-b80c-4c49-a1af-7d35979c6b40","Type":"ContainerStarted","Data":"ce5ca0ecc837e5d0d06ceb45ecafcb4fdb26c0ce5638f0164213bac5cd573c18"}
Sep 17 18:24:56.522991 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:24:56.522922    2575 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="e2e-test-llm-stop-feature-d9eef2a1/stop-feature-test-kserve-c54f4856d-gr2xm" podStartSLOduration=3.10950581 podStartE2EDuration="5.522903641s" totalImagesPullingTime="0s" totalInitContainerRuntime="2.413397831s" isStatefulPod=false podCreationTimestamp="2026-09-17 18:24:51 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-17 18:24:56.519603873 +0000 UTC m=+3016.149933846" watchObservedRunningTime="2026-09-17 18:24:56.522903641 +0000 UTC m=+3016.153233615"
Sep 17 18:24:56.825163 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:24:56.825073    2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-test-llm-stop-feature-d9eef2a1/stop-feature-test-kserve-c54f4856d-gr2xm"
Sep 17 18:25:00.640732 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:25:00.640697    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-djvzt_1d07e68d-9943-420f-a6a0-5f477792fabd/konnectivity-agent/0.log" limit=1024
Sep 17 18:25:00.741058 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:25:00.741026    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-djvzt_1d07e68d-9943-420f-a6a0-5f477792fabd/konnectivity-agent/0.log" limit=1024
Sep 17 18:25:01.824859 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:25:01.824823    2575 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="e2e-test-llm-stop-feature-d9eef2a1/stop-feature-test-kserve-c54f4856d-gr2xm"
Sep 17 18:25:01.826319 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:25:01.826291    2575 prober.go:120] "Probe failed" probeType="Startup" pod="e2e-test-llm-stop-feature-d9eef2a1/stop-feature-test-kserve-c54f4856d-gr2xm" podUID="696ad0c4-b80c-4c49-a1af-7d35979c6b40" containerName="main" probeResult="failure" output="Get \"https://10.133.0.60:8000/health\": dial tcp 10.133.0.60:8000: connect: connection refused"
Sep 17 18:25:02.132847 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:25:02.132817    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-djvzt_1d07e68d-9943-420f-a6a0-5f477792fabd/konnectivity-agent/0.log" limit=1024
Sep 17 18:25:11.824386 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:25:11.824346    2575 prober.go:120] "Probe failed" probeType="Startup" pod="e2e-test-llm-stop-feature-d9eef2a1/stop-feature-test-kserve-c54f4856d-gr2xm" podUID="696ad0c4-b80c-4c49-a1af-7d35979c6b40" containerName="main" probeResult="failure" output="Get \"https://10.133.0.60:8000/health\": dial tcp 10.133.0.60:8000: connect: connection refused"
Sep 17 18:25:21.824460 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:25:21.824406    2575 prober.go:120] "Probe failed" probeType="Startup" pod="e2e-test-llm-stop-feature-d9eef2a1/stop-feature-test-kserve-c54f4856d-gr2xm" podUID="696ad0c4-b80c-4c49-a1af-7d35979c6b40" containerName="main" probeResult="failure" output="Get \"https://10.133.0.60:8000/health\": dial tcp 10.133.0.60:8000: connect: connection refused"
Sep 17 18:25:31.824391 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:25:31.824346    2575 prober.go:120] "Probe failed" probeType="Startup" pod="e2e-test-llm-stop-feature-d9eef2a1/stop-feature-test-kserve-c54f4856d-gr2xm" podUID="696ad0c4-b80c-4c49-a1af-7d35979c6b40" containerName="main" probeResult="failure" output="Get \"https://10.133.0.60:8000/health\": dial tcp 10.133.0.60:8000: connect: connection refused"
Sep 17 18:25:41.824487 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:25:41.824376    2575 prober.go:120] "Probe failed" probeType="Startup" pod="e2e-test-llm-stop-feature-d9eef2a1/stop-feature-test-kserve-c54f4856d-gr2xm" podUID="696ad0c4-b80c-4c49-a1af-7d35979c6b40" containerName="main" probeResult="failure" output="Get \"https://10.133.0.60:8000/health\": dial tcp 10.133.0.60:8000: connect: connection refused"
Sep 17 18:25:51.824434 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:25:51.824394    2575 prober.go:120] "Probe failed" probeType="Startup" pod="e2e-test-llm-stop-feature-d9eef2a1/stop-feature-test-kserve-c54f4856d-gr2xm" podUID="696ad0c4-b80c-4c49-a1af-7d35979c6b40" containerName="main" probeResult="failure" output="Get \"https://10.133.0.60:8000/health\": dial tcp 10.133.0.60:8000: connect: connection refused"
Sep 17 18:25:56.298061 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:25:56.298024    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-djvzt_1d07e68d-9943-420f-a6a0-5f477792fabd/konnectivity-agent/0.log" limit=1024
Sep 17 18:25:56.363158 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:25:56.363127    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-djvzt_1d07e68d-9943-420f-a6a0-5f477792fabd/konnectivity-agent/0.log" limit=1024
Sep 17 18:26:01.824457 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:26:01.824395    2575 prober.go:120] "Probe failed" probeType="Startup" pod="e2e-test-llm-stop-feature-d9eef2a1/stop-feature-test-kserve-c54f4856d-gr2xm" podUID="696ad0c4-b80c-4c49-a1af-7d35979c6b40" containerName="main" probeResult="failure" output="Get \"https://10.133.0.60:8000/health\": dial tcp 10.133.0.60:8000: connect: connection refused"
Sep 17 18:26:11.824840 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:26:11.824782    2575 prober.go:120] "Probe failed" probeType="Startup" pod="e2e-test-llm-stop-feature-d9eef2a1/stop-feature-test-kserve-c54f4856d-gr2xm" podUID="696ad0c4-b80c-4c49-a1af-7d35979c6b40" containerName="main" probeResult="failure" output="Get \"https://10.133.0.60:8000/health\": dial tcp 10.133.0.60:8000: connect: connection refused"
Sep 17 18:26:21.824414 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:26:21.824373    2575 prober.go:120] "Probe failed" probeType="Startup" pod="e2e-test-llm-stop-feature-d9eef2a1/stop-feature-test-kserve-c54f4856d-gr2xm" podUID="696ad0c4-b80c-4c49-a1af-7d35979c6b40" containerName="main" probeResult="failure" output="Get \"https://10.133.0.60:8000/health\": dial tcp 10.133.0.60:8000: connect: connection refused"
Sep 17 18:26:24.567063 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:26:24.567020    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-djvzt_1d07e68d-9943-420f-a6a0-5f477792fabd/konnectivity-agent/0.log" limit=1024
Sep 17 18:26:24.636694 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:26:24.636662    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-djvzt_1d07e68d-9943-420f-a6a0-5f477792fabd/konnectivity-agent/0.log" limit=1024
Sep 17 18:26:31.824783 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:26:31.824736    2575 prober.go:120] "Probe failed" probeType="Startup" pod="e2e-test-llm-stop-feature-d9eef2a1/stop-feature-test-kserve-c54f4856d-gr2xm" podUID="696ad0c4-b80c-4c49-a1af-7d35979c6b40" containerName="main" probeResult="failure" output="Get \"https://10.133.0.60:8000/health\": dial tcp 10.133.0.60:8000: connect: connection refused"
Sep 17 18:26:41.824606 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:26:41.824549    2575 prober.go:120] "Probe failed" probeType="Startup" pod="e2e-test-llm-stop-feature-d9eef2a1/stop-feature-test-kserve-c54f4856d-gr2xm" podUID="696ad0c4-b80c-4c49-a1af-7d35979c6b40" containerName="main" probeResult="failure" output="Get \"https://10.133.0.60:8000/health\": dial tcp 10.133.0.60:8000: connect: connection refused"
Sep 17 18:26:51.834574 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:26:51.834541    2575 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="e2e-test-llm-stop-feature-d9eef2a1/stop-feature-test-kserve-c54f4856d-gr2xm"
Sep 17 18:26:51.844221 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:26:51.844200    2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-test-llm-stop-feature-d9eef2a1/stop-feature-test-kserve-c54f4856d-gr2xm"
Sep 17 18:26:53.582928 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:26:53.582893    2575 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["e2e-test-llm-stop-feature-d9eef2a1/stop-feature-test-kserve-c54f4856d-gr2xm"]
Sep 17 18:26:53.615593 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:26:53.615563    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-djvzt_1d07e68d-9943-420f-a6a0-5f477792fabd/konnectivity-agent/0.log" limit=1024
Sep 17 18:26:54.556927 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:26:54.556889    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-djvzt_1d07e68d-9943-420f-a6a0-5f477792fabd/konnectivity-agent/0.log" limit=1024
Sep 17 18:27:08.607568 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:27:08.607508    2575 kuberuntime_container.go:873] "Killing container with a grace period" pod="e2e-test-llm-stop-feature-d9eef2a1/stop-feature-test-kserve-c54f4856d-gr2xm" podUID="696ad0c4-b80c-4c49-a1af-7d35979c6b40" containerName="main" containerID="cri-o://ce5ca0ecc837e5d0d06ceb45ecafcb4fdb26c0ce5638f0164213bac5cd573c18" gracePeriod=45
Sep 17 18:27:08.835784 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:27:08.835742    2575 prober.go:120] "Probe failed" probeType="Readiness" pod="e2e-test-llm-stop-feature-d9eef2a1/stop-feature-test-kserve-c54f4856d-gr2xm" podUID="696ad0c4-b80c-4c49-a1af-7d35979c6b40" containerName="main" probeResult="failure" output="Get \"https://10.133.0.60:8000/health\": dial tcp 10.133.0.60:8000: connect: connection refused"
Sep 17 18:27:09.246316 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:27:09.246179    2575 generic.go:356] "Generic (PLEG): container finished" podID="696ad0c4-b80c-4c49-a1af-7d35979c6b40" containerID="ce5ca0ecc837e5d0d06ceb45ecafcb4fdb26c0ce5638f0164213bac5cd573c18" exitCode=0
Sep 17 18:27:09.246316 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:27:09.246218    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-test-llm-stop-feature-d9eef2a1/stop-feature-test-kserve-c54f4856d-gr2xm" event={"ID":"696ad0c4-b80c-4c49-a1af-7d35979c6b40","Type":"ContainerDied","Data":"ce5ca0ecc837e5d0d06ceb45ecafcb4fdb26c0ce5638f0164213bac5cd573c18"}
Sep 17 18:27:09.262664 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:27:09.262632    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-djvzt_1d07e68d-9943-420f-a6a0-5f477792fabd/konnectivity-agent/0.log" limit=1024
Sep 17 18:27:09.351701 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:27:09.351676    2575 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-test-llm-stop-feature-d9eef2a1/stop-feature-test-kserve-c54f4856d-gr2xm"
Sep 17 18:27:09.426054 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:27:09.426019    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-djvzt_1d07e68d-9943-420f-a6a0-5f477792fabd/konnectivity-agent/0.log" limit=1024
Sep 17 18:27:09.443775 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:27:09.443747    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/696ad0c4-b80c-4c49-a1af-7d35979c6b40-tls-certs\") pod \"696ad0c4-b80c-4c49-a1af-7d35979c6b40\" (UID: \"696ad0c4-b80c-4c49-a1af-7d35979c6b40\") "
Sep 17 18:27:09.443775 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:27:09.443777    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/696ad0c4-b80c-4c49-a1af-7d35979c6b40-kserve-provision-location\") pod \"696ad0c4-b80c-4c49-a1af-7d35979c6b40\" (UID: \"696ad0c4-b80c-4c49-a1af-7d35979c6b40\") "
Sep 17 18:27:09.444010 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:27:09.443802    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/696ad0c4-b80c-4c49-a1af-7d35979c6b40-model-cache\") pod \"696ad0c4-b80c-4c49-a1af-7d35979c6b40\" (UID: \"696ad0c4-b80c-4c49-a1af-7d35979c6b40\") "
Sep 17 18:27:09.444010 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:27:09.443817    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/696ad0c4-b80c-4c49-a1af-7d35979c6b40-home\") pod \"696ad0c4-b80c-4c49-a1af-7d35979c6b40\" (UID: \"696ad0c4-b80c-4c49-a1af-7d35979c6b40\") "
Sep 17 18:27:09.444010 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:27:09.443858    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/696ad0c4-b80c-4c49-a1af-7d35979c6b40-dshm\") pod \"696ad0c4-b80c-4c49-a1af-7d35979c6b40\" (UID: \"696ad0c4-b80c-4c49-a1af-7d35979c6b40\") "
Sep 17 18:27:09.444010 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:27:09.443902    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/696ad0c4-b80c-4c49-a1af-7d35979c6b40-tmp-dir\") pod \"696ad0c4-b80c-4c49-a1af-7d35979c6b40\" (UID: \"696ad0c4-b80c-4c49-a1af-7d35979c6b40\") "
Sep 17 18:27:09.444010 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:27:09.443938    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-p5hqf\" (UniqueName: \"kubernetes.io/projected/696ad0c4-b80c-4c49-a1af-7d35979c6b40-kube-api-access-p5hqf\") pod \"696ad0c4-b80c-4c49-a1af-7d35979c6b40\" (UID: \"696ad0c4-b80c-4c49-a1af-7d35979c6b40\") "
Sep 17 18:27:09.444010 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:27:09.443979    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"cabundle-cert\" (UniqueName: \"kubernetes.io/configmap/696ad0c4-b80c-4c49-a1af-7d35979c6b40-cabundle-cert\") pod \"696ad0c4-b80c-4c49-a1af-7d35979c6b40\" (UID: \"696ad0c4-b80c-4c49-a1af-7d35979c6b40\") "
Sep 17 18:27:09.444312 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:27:09.444096    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/696ad0c4-b80c-4c49-a1af-7d35979c6b40-model-cache" (OuterVolumeSpecName: "model-cache") pod "696ad0c4-b80c-4c49-a1af-7d35979c6b40" (UID: "696ad0c4-b80c-4c49-a1af-7d35979c6b40"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 17 18:27:09.444312 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:27:09.444256    2575 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/696ad0c4-b80c-4c49-a1af-7d35979c6b40-model-cache\") on node \"ip-10-0-143-74.ec2.internal\" DevicePath \"\""
Sep 17 18:27:09.444799 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:27:09.444769    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/696ad0c4-b80c-4c49-a1af-7d35979c6b40-home" (OuterVolumeSpecName: "home") pod "696ad0c4-b80c-4c49-a1af-7d35979c6b40" (UID: "696ad0c4-b80c-4c49-a1af-7d35979c6b40"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 17 18:27:09.444799 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:27:09.444783    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/696ad0c4-b80c-4c49-a1af-7d35979c6b40-cabundle-cert" (OuterVolumeSpecName: "cabundle-cert") pod "696ad0c4-b80c-4c49-a1af-7d35979c6b40" (UID: "696ad0c4-b80c-4c49-a1af-7d35979c6b40"). InnerVolumeSpecName "cabundle-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 17 18:27:09.446230 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:27:09.446198    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/696ad0c4-b80c-4c49-a1af-7d35979c6b40-dshm" (OuterVolumeSpecName: "dshm") pod "696ad0c4-b80c-4c49-a1af-7d35979c6b40" (UID: "696ad0c4-b80c-4c49-a1af-7d35979c6b40"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 17 18:27:09.446361 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:27:09.446342    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/696ad0c4-b80c-4c49-a1af-7d35979c6b40-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "696ad0c4-b80c-4c49-a1af-7d35979c6b40" (UID: "696ad0c4-b80c-4c49-a1af-7d35979c6b40"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 17 18:27:09.446616 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:27:09.446596    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/696ad0c4-b80c-4c49-a1af-7d35979c6b40-kube-api-access-p5hqf" (OuterVolumeSpecName: "kube-api-access-p5hqf") pod "696ad0c4-b80c-4c49-a1af-7d35979c6b40" (UID: "696ad0c4-b80c-4c49-a1af-7d35979c6b40"). InnerVolumeSpecName "kube-api-access-p5hqf". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 17 18:27:09.456837 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:27:09.456810    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/696ad0c4-b80c-4c49-a1af-7d35979c6b40-kserve-provision-location" (OuterVolumeSpecName: "kserve-provision-location") pod "696ad0c4-b80c-4c49-a1af-7d35979c6b40" (UID: "696ad0c4-b80c-4c49-a1af-7d35979c6b40"). InnerVolumeSpecName "kserve-provision-location". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 17 18:27:09.460815 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:27:09.460792    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/696ad0c4-b80c-4c49-a1af-7d35979c6b40-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "696ad0c4-b80c-4c49-a1af-7d35979c6b40" (UID: "696ad0c4-b80c-4c49-a1af-7d35979c6b40"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 17 18:27:09.544885 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:27:09.544859    2575 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/696ad0c4-b80c-4c49-a1af-7d35979c6b40-dshm\") on node \"ip-10-0-143-74.ec2.internal\" DevicePath \"\""
Sep 17 18:27:09.544885 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:27:09.544882    2575 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/696ad0c4-b80c-4c49-a1af-7d35979c6b40-tmp-dir\") on node \"ip-10-0-143-74.ec2.internal\" DevicePath \"\""
Sep 17 18:27:09.545038 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:27:09.544895    2575 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-p5hqf\" (UniqueName: \"kubernetes.io/projected/696ad0c4-b80c-4c49-a1af-7d35979c6b40-kube-api-access-p5hqf\") on node \"ip-10-0-143-74.ec2.internal\" DevicePath \"\""
Sep 17 18:27:09.545038 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:27:09.544908    2575 reconciler_common.go:299] "Volume detached for volume \"cabundle-cert\" (UniqueName: \"kubernetes.io/configmap/696ad0c4-b80c-4c49-a1af-7d35979c6b40-cabundle-cert\") on node \"ip-10-0-143-74.ec2.internal\" DevicePath \"\""
Sep 17 18:27:09.545038 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:27:09.544920    2575 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/696ad0c4-b80c-4c49-a1af-7d35979c6b40-tls-certs\") on node \"ip-10-0-143-74.ec2.internal\" DevicePath \"\""
Sep 17 18:27:09.545038 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:27:09.544931    2575 reconciler_common.go:299] "Volume detached for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/696ad0c4-b80c-4c49-a1af-7d35979c6b40-kserve-provision-location\") on node \"ip-10-0-143-74.ec2.internal\" DevicePath \"\""
Sep 17 18:27:09.545038 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:27:09.544943    2575 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/696ad0c4-b80c-4c49-a1af-7d35979c6b40-home\") on node \"ip-10-0-143-74.ec2.internal\" DevicePath \"\""
Sep 17 18:27:10.254719 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:27:10.254693    2575 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-test-llm-stop-feature-d9eef2a1/stop-feature-test-kserve-c54f4856d-gr2xm"
Sep 17 18:27:10.254719 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:27:10.254701    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-test-llm-stop-feature-d9eef2a1/stop-feature-test-kserve-c54f4856d-gr2xm" event={"ID":"696ad0c4-b80c-4c49-a1af-7d35979c6b40","Type":"ContainerDied","Data":"7644d5cf59c8a6d257a8b1e8b2ff3bf3b4f98cbce3c429f27f12458df6617490"}
Sep 17 18:27:10.255298 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:27:10.254755    2575 scope.go:117] "RemoveContainer" containerID="ce5ca0ecc837e5d0d06ceb45ecafcb4fdb26c0ce5638f0164213bac5cd573c18"
Sep 17 18:27:10.264375 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:27:10.264357    2575 scope.go:117] "RemoveContainer" containerID="21cd053421b561dc1fe8600c014254de1746a7cf9f7b721e20a73f1ac757b390"
Sep 17 18:27:10.284347 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:27:10.284274    2575 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["e2e-test-llm-stop-feature-d9eef2a1/stop-feature-test-kserve-c54f4856d-gr2xm"]
Sep 17 18:27:10.290050 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:27:10.290025    2575 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["e2e-test-llm-stop-feature-d9eef2a1/stop-feature-test-kserve-c54f4856d-gr2xm"]
Sep 17 18:27:10.967809 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:27:10.967779    2575 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="696ad0c4-b80c-4c49-a1af-7d35979c6b40" path="/var/lib/kubelet/pods/696ad0c4-b80c-4c49-a1af-7d35979c6b40/volumes"
Sep 17 18:27:14.695157 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:27:14.695047    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-djvzt_1d07e68d-9943-420f-a6a0-5f477792fabd/konnectivity-agent/0.log" limit=1024
Sep 17 18:27:14.790313 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:27:14.790279    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-djvzt_1d07e68d-9943-420f-a6a0-5f477792fabd/konnectivity-agent/0.log" limit=1024
Sep 17 18:27:19.641779 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:27:19.641738    2575 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["e2e-test-llm-inference-service-028f7809/router-gateway-1-openshift-default-cc7cb45dd-srd8v"]
Sep 17 18:27:19.642170 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:27:19.641983    2575 kuberuntime_container.go:873] "Killing container with a grace period" pod="e2e-test-llm-inference-service-028f7809/router-gateway-1-openshift-default-cc7cb45dd-srd8v" podUID="83113b83-f4a3-4836-a6a2-18f5e825be12" containerName="istio-proxy" containerID="cri-o://dd645fd08d7b50a41ac57c2807cd00cf76fdf816123f397268c581dc6ceaede7" gracePeriod=30
Sep 17 18:27:19.694607 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:27:19.694558    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-djvzt_1d07e68d-9943-420f-a6a0-5f477792fabd/konnectivity-agent/0.log" limit=1024
Sep 17 18:27:24.889286 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:27:24.889262    2575 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-test-llm-inference-service-028f7809/router-gateway-1-openshift-default-cc7cb45dd-srd8v"
Sep 17 18:27:24.976728 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:27:24.976702    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/83113b83-f4a3-4836-a6a2-18f5e825be12-istiod-ca-cert\") pod \"83113b83-f4a3-4836-a6a2-18f5e825be12\" (UID: \"83113b83-f4a3-4836-a6a2-18f5e825be12\") "
Sep 17 18:27:24.976908 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:27:24.976760    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/83113b83-f4a3-4836-a6a2-18f5e825be12-credential-socket\") pod \"83113b83-f4a3-4836-a6a2-18f5e825be12\" (UID: \"83113b83-f4a3-4836-a6a2-18f5e825be12\") "
Sep 17 18:27:24.976908 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:27:24.976781    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/83113b83-f4a3-4836-a6a2-18f5e825be12-istio-token\") pod \"83113b83-f4a3-4836-a6a2-18f5e825be12\" (UID: \"83113b83-f4a3-4836-a6a2-18f5e825be12\") "
Sep 17 18:27:24.977116 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:27:24.976949    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/83113b83-f4a3-4836-a6a2-18f5e825be12-workload-socket\") pod \"83113b83-f4a3-4836-a6a2-18f5e825be12\" (UID: \"83113b83-f4a3-4836-a6a2-18f5e825be12\") "
Sep 17 18:27:24.977116 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:27:24.976999    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-mspjg\" (UniqueName: \"kubernetes.io/projected/83113b83-f4a3-4836-a6a2-18f5e825be12-kube-api-access-mspjg\") pod \"83113b83-f4a3-4836-a6a2-18f5e825be12\" (UID: \"83113b83-f4a3-4836-a6a2-18f5e825be12\") "
Sep 17 18:27:24.977116 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:27:24.977049    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/83113b83-f4a3-4836-a6a2-18f5e825be12-istio-data\") pod \"83113b83-f4a3-4836-a6a2-18f5e825be12\" (UID: \"83113b83-f4a3-4836-a6a2-18f5e825be12\") "
Sep 17 18:27:24.977116 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:27:24.977075    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/83113b83-f4a3-4836-a6a2-18f5e825be12-workload-certs\") pod \"83113b83-f4a3-4836-a6a2-18f5e825be12\" (UID: \"83113b83-f4a3-4836-a6a2-18f5e825be12\") "
Sep 17 18:27:24.977116 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:27:24.977096    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/83113b83-f4a3-4836-a6a2-18f5e825be12-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "83113b83-f4a3-4836-a6a2-18f5e825be12" (UID: "83113b83-f4a3-4836-a6a2-18f5e825be12"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 17 18:27:24.977553 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:27:24.977113    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/83113b83-f4a3-4836-a6a2-18f5e825be12-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "83113b83-f4a3-4836-a6a2-18f5e825be12" (UID: "83113b83-f4a3-4836-a6a2-18f5e825be12"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 17 18:27:24.977553 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:27:24.977201    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/83113b83-f4a3-4836-a6a2-18f5e825be12-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "83113b83-f4a3-4836-a6a2-18f5e825be12" (UID: "83113b83-f4a3-4836-a6a2-18f5e825be12"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 17 18:27:24.977553 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:27:24.977122    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/83113b83-f4a3-4836-a6a2-18f5e825be12-istio-podinfo\") pod \"83113b83-f4a3-4836-a6a2-18f5e825be12\" (UID: \"83113b83-f4a3-4836-a6a2-18f5e825be12\") "
Sep 17 18:27:24.977553 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:27:24.977264    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/83113b83-f4a3-4836-a6a2-18f5e825be12-istio-envoy\") pod \"83113b83-f4a3-4836-a6a2-18f5e825be12\" (UID: \"83113b83-f4a3-4836-a6a2-18f5e825be12\") "
Sep 17 18:27:24.977553 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:27:24.977406    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/83113b83-f4a3-4836-a6a2-18f5e825be12-istio-data" (OuterVolumeSpecName: "istio-data") pod "83113b83-f4a3-4836-a6a2-18f5e825be12" (UID: "83113b83-f4a3-4836-a6a2-18f5e825be12"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 17 18:27:24.977817 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:27:24.977653    2575 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/83113b83-f4a3-4836-a6a2-18f5e825be12-workload-socket\") on node \"ip-10-0-143-74.ec2.internal\" DevicePath \"\""
Sep 17 18:27:24.977817 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:27:24.977675    2575 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/83113b83-f4a3-4836-a6a2-18f5e825be12-istio-data\") on node \"ip-10-0-143-74.ec2.internal\" DevicePath \"\""
Sep 17 18:27:24.977817 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:27:24.977689    2575 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/83113b83-f4a3-4836-a6a2-18f5e825be12-istiod-ca-cert\") on node \"ip-10-0-143-74.ec2.internal\" DevicePath \"\""
Sep 17 18:27:24.977817 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:27:24.977701    2575 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/83113b83-f4a3-4836-a6a2-18f5e825be12-credential-socket\") on node \"ip-10-0-143-74.ec2.internal\" DevicePath \"\""
Sep 17 18:27:24.978098 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:27:24.978076    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/83113b83-f4a3-4836-a6a2-18f5e825be12-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "83113b83-f4a3-4836-a6a2-18f5e825be12" (UID: "83113b83-f4a3-4836-a6a2-18f5e825be12"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 17 18:27:24.979558 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:27:24.979528    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/83113b83-f4a3-4836-a6a2-18f5e825be12-istio-token" (OuterVolumeSpecName: "istio-token") pod "83113b83-f4a3-4836-a6a2-18f5e825be12" (UID: "83113b83-f4a3-4836-a6a2-18f5e825be12"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 17 18:27:24.979663 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:27:24.979573    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/83113b83-f4a3-4836-a6a2-18f5e825be12-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "83113b83-f4a3-4836-a6a2-18f5e825be12" (UID: "83113b83-f4a3-4836-a6a2-18f5e825be12"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 17 18:27:24.979927 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:27:24.979901    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/83113b83-f4a3-4836-a6a2-18f5e825be12-kube-api-access-mspjg" (OuterVolumeSpecName: "kube-api-access-mspjg") pod "83113b83-f4a3-4836-a6a2-18f5e825be12" (UID: "83113b83-f4a3-4836-a6a2-18f5e825be12"). InnerVolumeSpecName "kube-api-access-mspjg". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 17 18:27:24.980042 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:27:24.980027    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/83113b83-f4a3-4836-a6a2-18f5e825be12-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "83113b83-f4a3-4836-a6a2-18f5e825be12" (UID: "83113b83-f4a3-4836-a6a2-18f5e825be12"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Sep 17 18:27:25.078262 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:27:25.078228    2575 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/83113b83-f4a3-4836-a6a2-18f5e825be12-istio-token\") on node \"ip-10-0-143-74.ec2.internal\" DevicePath \"\""
Sep 17 18:27:25.078262 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:27:25.078257    2575 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-mspjg\" (UniqueName: \"kubernetes.io/projected/83113b83-f4a3-4836-a6a2-18f5e825be12-kube-api-access-mspjg\") on node \"ip-10-0-143-74.ec2.internal\" DevicePath \"\""
Sep 17 18:27:25.078262 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:27:25.078270    2575 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/83113b83-f4a3-4836-a6a2-18f5e825be12-workload-certs\") on node \"ip-10-0-143-74.ec2.internal\" DevicePath \"\""
Sep 17 18:27:25.078526 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:27:25.078283    2575 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/83113b83-f4a3-4836-a6a2-18f5e825be12-istio-podinfo\") on node \"ip-10-0-143-74.ec2.internal\" DevicePath \"\""
Sep 17 18:27:25.078526 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:27:25.078293    2575 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/83113b83-f4a3-4836-a6a2-18f5e825be12-istio-envoy\") on node \"ip-10-0-143-74.ec2.internal\" DevicePath \"\""
Sep 17 18:27:25.335746 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:27:25.335650    2575 generic.go:356] "Generic (PLEG): container finished" podID="83113b83-f4a3-4836-a6a2-18f5e825be12" containerID="dd645fd08d7b50a41ac57c2807cd00cf76fdf816123f397268c581dc6ceaede7" exitCode=0
Sep 17 18:27:25.335746 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:27:25.335703    2575 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-test-llm-inference-service-028f7809/router-gateway-1-openshift-default-cc7cb45dd-srd8v"
Sep 17 18:27:25.335746 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:27:25.335733    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-test-llm-inference-service-028f7809/router-gateway-1-openshift-default-cc7cb45dd-srd8v" event={"ID":"83113b83-f4a3-4836-a6a2-18f5e825be12","Type":"ContainerDied","Data":"dd645fd08d7b50a41ac57c2807cd00cf76fdf816123f397268c581dc6ceaede7"}
Sep 17 18:27:25.336010 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:27:25.335787    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-test-llm-inference-service-028f7809/router-gateway-1-openshift-default-cc7cb45dd-srd8v" event={"ID":"83113b83-f4a3-4836-a6a2-18f5e825be12","Type":"ContainerDied","Data":"710c0ccca51eca82d007e8747e0c4ef0d0b4ec6aa6920d9a2cb4ac50b9593eeb"}
Sep 17 18:27:25.336010 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:27:25.335809    2575 scope.go:117] "RemoveContainer" containerID="dd645fd08d7b50a41ac57c2807cd00cf76fdf816123f397268c581dc6ceaede7"
Sep 17 18:27:25.345232 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:27:25.345215    2575 scope.go:117] "RemoveContainer" containerID="dd645fd08d7b50a41ac57c2807cd00cf76fdf816123f397268c581dc6ceaede7"
Sep 17 18:27:25.345536 ip-10-0-143-74 kubenswrapper[2575]: E0917 18:27:25.345516    2575 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"dd645fd08d7b50a41ac57c2807cd00cf76fdf816123f397268c581dc6ceaede7\": container with ID starting with dd645fd08d7b50a41ac57c2807cd00cf76fdf816123f397268c581dc6ceaede7 not found: ID does not exist" containerID="dd645fd08d7b50a41ac57c2807cd00cf76fdf816123f397268c581dc6ceaede7"
Sep 17 18:27:25.345616 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:27:25.345544    2575 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"dd645fd08d7b50a41ac57c2807cd00cf76fdf816123f397268c581dc6ceaede7"} err="failed to get container status \"dd645fd08d7b50a41ac57c2807cd00cf76fdf816123f397268c581dc6ceaede7\": rpc error: code = NotFound desc = could not find container \"dd645fd08d7b50a41ac57c2807cd00cf76fdf816123f397268c581dc6ceaede7\": container with ID starting with dd645fd08d7b50a41ac57c2807cd00cf76fdf816123f397268c581dc6ceaede7 not found: ID does not exist"
Sep 17 18:27:25.358200 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:27:25.358176    2575 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["e2e-test-llm-inference-service-028f7809/router-gateway-1-openshift-default-cc7cb45dd-srd8v"]
Sep 17 18:27:25.363914 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:27:25.363887    2575 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["e2e-test-llm-inference-service-028f7809/router-gateway-1-openshift-default-cc7cb45dd-srd8v"]
Sep 17 18:27:25.853412 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:27:25.853366    2575 prober.go:120] "Probe failed" probeType="Readiness" pod="e2e-test-llm-inference-service-028f7809/router-gateway-1-openshift-default-cc7cb45dd-srd8v" podUID="83113b83-f4a3-4836-a6a2-18f5e825be12" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.59:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Sep 17 18:27:26.966990 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:27:26.966957    2575 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="83113b83-f4a3-4836-a6a2-18f5e825be12" path="/var/lib/kubelet/pods/83113b83-f4a3-4836-a6a2-18f5e825be12/volumes"
Sep 17 18:27:30.431425 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:27:30.431391    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-djvzt_1d07e68d-9943-420f-a6a0-5f477792fabd/konnectivity-agent/0.log" limit=1024
Sep 17 18:27:34.850141 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:27:34.850108    2575 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["e2e-test-llm-inference-service-62876e81/llmisvc-model-fb-opt-125m-route-50bc673d-kserve-5bf645b766dzg4n"]
Sep 17 18:27:34.850618 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:27:34.850503    2575 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="696ad0c4-b80c-4c49-a1af-7d35979c6b40" containerName="main"
Sep 17 18:27:34.850618 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:27:34.850516    2575 state_mem.go:107] "Deleted CPUSet assignment" podUID="696ad0c4-b80c-4c49-a1af-7d35979c6b40" containerName="main"
Sep 17 18:27:34.850618 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:27:34.850532    2575 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="83113b83-f4a3-4836-a6a2-18f5e825be12" containerName="istio-proxy"
Sep 17 18:27:34.850618 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:27:34.850537    2575 state_mem.go:107] "Deleted CPUSet assignment" podUID="83113b83-f4a3-4836-a6a2-18f5e825be12" containerName="istio-proxy"
Sep 17 18:27:34.850618 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:27:34.850543    2575 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="696ad0c4-b80c-4c49-a1af-7d35979c6b40" containerName="storage-initializer"
Sep 17 18:27:34.850618 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:27:34.850549    2575 state_mem.go:107] "Deleted CPUSet assignment" podUID="696ad0c4-b80c-4c49-a1af-7d35979c6b40" containerName="storage-initializer"
Sep 17 18:27:34.850618 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:27:34.850622    2575 memory_manager.go:368] "RemoveStaleState removing state" podUID="696ad0c4-b80c-4c49-a1af-7d35979c6b40" containerName="main"
Sep 17 18:27:34.850983 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:27:34.850630    2575 memory_manager.go:368] "RemoveStaleState removing state" podUID="83113b83-f4a3-4836-a6a2-18f5e825be12" containerName="istio-proxy"
Sep 17 18:27:34.855072 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:27:34.855046    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-test-llm-inference-service-62876e81/llmisvc-model-fb-opt-125m-route-50bc673d-kserve-5bf645b766dzg4n"
Sep 17 18:27:34.858659 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:27:34.858146    2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-test-llm-inference-service-62876e81\"/\"openshift-service-ca.crt\""
Sep 17 18:27:34.858659 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:27:34.858302    2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-test-llm-inference-service-62876e81\"/\"odh-kserve-custom-ca-bundle\""
Sep 17 18:27:34.858659 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:27:34.858540    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-test-llm-inference-service-62876e81\"/\"seaweedfs-s3-creds\""
Sep 17 18:27:34.859518 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:27:34.859263    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-test-llm-inference-service-62876e81\"/\"llmisv44d181485fad85e662eb092f3749502f-kserve-self-signed-certs\""
Sep 17 18:27:34.859518 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:27:34.859358    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-test-llm-inference-service-62876e81\"/\"default-dockercfg-v6v88\""
Sep 17 18:27:34.860546 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:27:34.860353    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-test-llm-inference-service-62876e81\"/\"llmisvc-model-fb-opt-125m-route-50bc673d-kserve-dockercfg-276mz\""
Sep 17 18:27:34.860991 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:27:34.860968    2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-test-llm-inference-service-62876e81\"/\"kube-root-ca.crt\""
Sep 17 18:27:34.870343 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:27:34.870317    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["e2e-test-llm-inference-service-62876e81/llmisvc-model-fb-opt-125m-route-50bc673d-kserve-5bf645b766dzg4n"]
Sep 17 18:27:34.888625 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:27:34.888585    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-djvzt_1d07e68d-9943-420f-a6a0-5f477792fabd/konnectivity-agent/0.log" limit=1024
Sep 17 18:27:34.963091 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:27:34.963059    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7pj7b\" (UniqueName: \"kubernetes.io/projected/a443051c-645a-4acd-b004-bd7e26d33944-kube-api-access-7pj7b\") pod \"llmisvc-model-fb-opt-125m-route-50bc673d-kserve-5bf645b766dzg4n\" (UID: \"a443051c-645a-4acd-b004-bd7e26d33944\") " pod="e2e-test-llm-inference-service-62876e81/llmisvc-model-fb-opt-125m-route-50bc673d-kserve-5bf645b766dzg4n"
Sep 17 18:27:34.963255 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:27:34.963101    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/a443051c-645a-4acd-b004-bd7e26d33944-model-cache\") pod \"llmisvc-model-fb-opt-125m-route-50bc673d-kserve-5bf645b766dzg4n\" (UID: \"a443051c-645a-4acd-b004-bd7e26d33944\") " pod="e2e-test-llm-inference-service-62876e81/llmisvc-model-fb-opt-125m-route-50bc673d-kserve-5bf645b766dzg4n"
Sep 17 18:27:34.963255 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:27:34.963129    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cabundle-cert\" (UniqueName: \"kubernetes.io/configmap/a443051c-645a-4acd-b004-bd7e26d33944-cabundle-cert\") pod \"llmisvc-model-fb-opt-125m-route-50bc673d-kserve-5bf645b766dzg4n\" (UID: \"a443051c-645a-4acd-b004-bd7e26d33944\") " pod="e2e-test-llm-inference-service-62876e81/llmisvc-model-fb-opt-125m-route-50bc673d-kserve-5bf645b766dzg4n"
Sep 17 18:27:34.963255 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:27:34.963196    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/a443051c-645a-4acd-b004-bd7e26d33944-dshm\") pod \"llmisvc-model-fb-opt-125m-route-50bc673d-kserve-5bf645b766dzg4n\" (UID: \"a443051c-645a-4acd-b004-bd7e26d33944\") " pod="e2e-test-llm-inference-service-62876e81/llmisvc-model-fb-opt-125m-route-50bc673d-kserve-5bf645b766dzg4n"
Sep 17 18:27:34.963255 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:27:34.963226    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/a443051c-645a-4acd-b004-bd7e26d33944-home\") pod \"llmisvc-model-fb-opt-125m-route-50bc673d-kserve-5bf645b766dzg4n\" (UID: \"a443051c-645a-4acd-b004-bd7e26d33944\") " pod="e2e-test-llm-inference-service-62876e81/llmisvc-model-fb-opt-125m-route-50bc673d-kserve-5bf645b766dzg4n"
Sep 17 18:27:34.963255 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:27:34.963247    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/a443051c-645a-4acd-b004-bd7e26d33944-tmp-dir\") pod \"llmisvc-model-fb-opt-125m-route-50bc673d-kserve-5bf645b766dzg4n\" (UID: \"a443051c-645a-4acd-b004-bd7e26d33944\") " pod="e2e-test-llm-inference-service-62876e81/llmisvc-model-fb-opt-125m-route-50bc673d-kserve-5bf645b766dzg4n"
Sep 17 18:27:34.963468 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:27:34.963264    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/a443051c-645a-4acd-b004-bd7e26d33944-kserve-provision-location\") pod \"llmisvc-model-fb-opt-125m-route-50bc673d-kserve-5bf645b766dzg4n\" (UID: \"a443051c-645a-4acd-b004-bd7e26d33944\") " pod="e2e-test-llm-inference-service-62876e81/llmisvc-model-fb-opt-125m-route-50bc673d-kserve-5bf645b766dzg4n"
Sep 17 18:27:34.963468 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:27:34.963323    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/a443051c-645a-4acd-b004-bd7e26d33944-tls-certs\") pod \"llmisvc-model-fb-opt-125m-route-50bc673d-kserve-5bf645b766dzg4n\" (UID: \"a443051c-645a-4acd-b004-bd7e26d33944\") " pod="e2e-test-llm-inference-service-62876e81/llmisvc-model-fb-opt-125m-route-50bc673d-kserve-5bf645b766dzg4n"
Sep 17 18:27:35.064780 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:27:35.064735    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/a443051c-645a-4acd-b004-bd7e26d33944-home\") pod \"llmisvc-model-fb-opt-125m-route-50bc673d-kserve-5bf645b766dzg4n\" (UID: \"a443051c-645a-4acd-b004-bd7e26d33944\") " pod="e2e-test-llm-inference-service-62876e81/llmisvc-model-fb-opt-125m-route-50bc673d-kserve-5bf645b766dzg4n"
Sep 17 18:27:35.064972 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:27:35.064800    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/a443051c-645a-4acd-b004-bd7e26d33944-tmp-dir\") pod \"llmisvc-model-fb-opt-125m-route-50bc673d-kserve-5bf645b766dzg4n\" (UID: \"a443051c-645a-4acd-b004-bd7e26d33944\") " pod="e2e-test-llm-inference-service-62876e81/llmisvc-model-fb-opt-125m-route-50bc673d-kserve-5bf645b766dzg4n"
Sep 17 18:27:35.064972 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:27:35.064832    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/a443051c-645a-4acd-b004-bd7e26d33944-kserve-provision-location\") pod \"llmisvc-model-fb-opt-125m-route-50bc673d-kserve-5bf645b766dzg4n\" (UID: \"a443051c-645a-4acd-b004-bd7e26d33944\") " pod="e2e-test-llm-inference-service-62876e81/llmisvc-model-fb-opt-125m-route-50bc673d-kserve-5bf645b766dzg4n"
Sep 17 18:27:35.064972 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:27:35.064893    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/a443051c-645a-4acd-b004-bd7e26d33944-tls-certs\") pod \"llmisvc-model-fb-opt-125m-route-50bc673d-kserve-5bf645b766dzg4n\" (UID: \"a443051c-645a-4acd-b004-bd7e26d33944\") " pod="e2e-test-llm-inference-service-62876e81/llmisvc-model-fb-opt-125m-route-50bc673d-kserve-5bf645b766dzg4n"
Sep 17 18:27:35.064972 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:27:35.064920    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7pj7b\" (UniqueName: \"kubernetes.io/projected/a443051c-645a-4acd-b004-bd7e26d33944-kube-api-access-7pj7b\") pod \"llmisvc-model-fb-opt-125m-route-50bc673d-kserve-5bf645b766dzg4n\" (UID: \"a443051c-645a-4acd-b004-bd7e26d33944\") " pod="e2e-test-llm-inference-service-62876e81/llmisvc-model-fb-opt-125m-route-50bc673d-kserve-5bf645b766dzg4n"
Sep 17 18:27:35.065191 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:27:35.064977    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/a443051c-645a-4acd-b004-bd7e26d33944-model-cache\") pod \"llmisvc-model-fb-opt-125m-route-50bc673d-kserve-5bf645b766dzg4n\" (UID: \"a443051c-645a-4acd-b004-bd7e26d33944\") " pod="e2e-test-llm-inference-service-62876e81/llmisvc-model-fb-opt-125m-route-50bc673d-kserve-5bf645b766dzg4n"
Sep 17 18:27:35.065191 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:27:35.065031    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cabundle-cert\" (UniqueName: \"kubernetes.io/configmap/a443051c-645a-4acd-b004-bd7e26d33944-cabundle-cert\") pod \"llmisvc-model-fb-opt-125m-route-50bc673d-kserve-5bf645b766dzg4n\" (UID: \"a443051c-645a-4acd-b004-bd7e26d33944\") " pod="e2e-test-llm-inference-service-62876e81/llmisvc-model-fb-opt-125m-route-50bc673d-kserve-5bf645b766dzg4n"
Sep 17 18:27:35.065191 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:27:35.065108    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/a443051c-645a-4acd-b004-bd7e26d33944-dshm\") pod \"llmisvc-model-fb-opt-125m-route-50bc673d-kserve-5bf645b766dzg4n\" (UID: \"a443051c-645a-4acd-b004-bd7e26d33944\") " pod="e2e-test-llm-inference-service-62876e81/llmisvc-model-fb-opt-125m-route-50bc673d-kserve-5bf645b766dzg4n"
Sep 17 18:27:35.065802 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:27:35.065430    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/a443051c-645a-4acd-b004-bd7e26d33944-home\") pod \"llmisvc-model-fb-opt-125m-route-50bc673d-kserve-5bf645b766dzg4n\" (UID: \"a443051c-645a-4acd-b004-bd7e26d33944\") " pod="e2e-test-llm-inference-service-62876e81/llmisvc-model-fb-opt-125m-route-50bc673d-kserve-5bf645b766dzg4n"
Sep 17 18:27:35.065802 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:27:35.065432    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/a443051c-645a-4acd-b004-bd7e26d33944-tmp-dir\") pod \"llmisvc-model-fb-opt-125m-route-50bc673d-kserve-5bf645b766dzg4n\" (UID: \"a443051c-645a-4acd-b004-bd7e26d33944\") " pod="e2e-test-llm-inference-service-62876e81/llmisvc-model-fb-opt-125m-route-50bc673d-kserve-5bf645b766dzg4n"
Sep 17 18:27:35.065802 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:27:35.065631    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/a443051c-645a-4acd-b004-bd7e26d33944-kserve-provision-location\") pod \"llmisvc-model-fb-opt-125m-route-50bc673d-kserve-5bf645b766dzg4n\" (UID: \"a443051c-645a-4acd-b004-bd7e26d33944\") " pod="e2e-test-llm-inference-service-62876e81/llmisvc-model-fb-opt-125m-route-50bc673d-kserve-5bf645b766dzg4n"
Sep 17 18:27:35.066220 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:27:35.066201    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/a443051c-645a-4acd-b004-bd7e26d33944-model-cache\") pod \"llmisvc-model-fb-opt-125m-route-50bc673d-kserve-5bf645b766dzg4n\" (UID: \"a443051c-645a-4acd-b004-bd7e26d33944\") " pod="e2e-test-llm-inference-service-62876e81/llmisvc-model-fb-opt-125m-route-50bc673d-kserve-5bf645b766dzg4n"
Sep 17 18:27:35.066549 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:27:35.066524    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cabundle-cert\" (UniqueName: \"kubernetes.io/configmap/a443051c-645a-4acd-b004-bd7e26d33944-cabundle-cert\") pod \"llmisvc-model-fb-opt-125m-route-50bc673d-kserve-5bf645b766dzg4n\" (UID: \"a443051c-645a-4acd-b004-bd7e26d33944\") " pod="e2e-test-llm-inference-service-62876e81/llmisvc-model-fb-opt-125m-route-50bc673d-kserve-5bf645b766dzg4n"
Sep 17 18:27:35.068005 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:27:35.067986    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/a443051c-645a-4acd-b004-bd7e26d33944-dshm\") pod \"llmisvc-model-fb-opt-125m-route-50bc673d-kserve-5bf645b766dzg4n\" (UID: \"a443051c-645a-4acd-b004-bd7e26d33944\") " pod="e2e-test-llm-inference-service-62876e81/llmisvc-model-fb-opt-125m-route-50bc673d-kserve-5bf645b766dzg4n"
Sep 17 18:27:35.068537 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:27:35.068517    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/a443051c-645a-4acd-b004-bd7e26d33944-tls-certs\") pod \"llmisvc-model-fb-opt-125m-route-50bc673d-kserve-5bf645b766dzg4n\" (UID: \"a443051c-645a-4acd-b004-bd7e26d33944\") " pod="e2e-test-llm-inference-service-62876e81/llmisvc-model-fb-opt-125m-route-50bc673d-kserve-5bf645b766dzg4n"
Sep 17 18:27:35.073038 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:27:35.073019    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-7pj7b\" (UniqueName: \"kubernetes.io/projected/a443051c-645a-4acd-b004-bd7e26d33944-kube-api-access-7pj7b\") pod \"llmisvc-model-fb-opt-125m-route-50bc673d-kserve-5bf645b766dzg4n\" (UID: \"a443051c-645a-4acd-b004-bd7e26d33944\") " pod="e2e-test-llm-inference-service-62876e81/llmisvc-model-fb-opt-125m-route-50bc673d-kserve-5bf645b766dzg4n"
Sep 17 18:27:35.137526 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:27:35.137496    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-djvzt_1d07e68d-9943-420f-a6a0-5f477792fabd/konnectivity-agent/0.log" limit=1024
Sep 17 18:27:35.172648 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:27:35.172604    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-test-llm-inference-service-62876e81/llmisvc-model-fb-opt-125m-route-50bc673d-kserve-5bf645b766dzg4n"
Sep 17 18:27:35.461599 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:27:35.461564    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["e2e-test-llm-inference-service-62876e81/llmisvc-model-fb-opt-125m-route-50bc673d-kserve-5bf645b766dzg4n"]
Sep 17 18:27:35.462990 ip-10-0-143-74 kubenswrapper[2575]: W0917 18:27:35.462959    2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda443051c_645a_4acd_b004_bd7e26d33944.slice/crio-bdddf3c360436bac1ac93821dafdfa4e22dbdef5873b4f08814a51bb4092d63f WatchSource:0}: Error finding container bdddf3c360436bac1ac93821dafdfa4e22dbdef5873b4f08814a51bb4092d63f: Status 404 returned error can't find the container with id bdddf3c360436bac1ac93821dafdfa4e22dbdef5873b4f08814a51bb4092d63f
Sep 17 18:27:36.397888 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:27:36.397846    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-test-llm-inference-service-62876e81/llmisvc-model-fb-opt-125m-route-50bc673d-kserve-5bf645b766dzg4n" event={"ID":"a443051c-645a-4acd-b004-bd7e26d33944","Type":"ContainerStarted","Data":"bdddf3c360436bac1ac93821dafdfa4e22dbdef5873b4f08814a51bb4092d63f"}
Sep 17 18:27:38.411602 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:27:38.411563    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-test-llm-inference-service-62876e81/llmisvc-model-fb-opt-125m-route-50bc673d-kserve-5bf645b766dzg4n" event={"ID":"a443051c-645a-4acd-b004-bd7e26d33944","Type":"ContainerStarted","Data":"2411e3d8b0878bb567d3209003bb7c85d6979760705e753ec12968be48b49ae3"}
Sep 17 18:27:38.411976 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:27:38.411637    2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-test-llm-inference-service-62876e81/llmisvc-model-fb-opt-125m-route-50bc673d-kserve-5bf645b766dzg4n"
Sep 17 18:27:39.419094 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:27:39.419054    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-test-llm-inference-service-62876e81/llmisvc-model-fb-opt-125m-route-50bc673d-kserve-5bf645b766dzg4n" event={"ID":"a443051c-645a-4acd-b004-bd7e26d33944","Type":"ContainerStarted","Data":"8a1396e13838eb8c534c08672cbe649d1f141bef0667d774481d01b39de2381e"}
Sep 17 18:27:41.328323 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:27:41.328235    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-djvzt_1d07e68d-9943-420f-a6a0-5f477792fabd/konnectivity-agent/0.log" limit=1024
Sep 17 18:27:41.432083 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:27:41.432051    2575 generic.go:356] "Generic (PLEG): container finished" podID="a443051c-645a-4acd-b004-bd7e26d33944" containerID="8a1396e13838eb8c534c08672cbe649d1f141bef0667d774481d01b39de2381e" exitCode=0
Sep 17 18:27:41.432238 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:27:41.432114    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-test-llm-inference-service-62876e81/llmisvc-model-fb-opt-125m-route-50bc673d-kserve-5bf645b766dzg4n" event={"ID":"a443051c-645a-4acd-b004-bd7e26d33944","Type":"ContainerDied","Data":"8a1396e13838eb8c534c08672cbe649d1f141bef0667d774481d01b39de2381e"}
Sep 17 18:27:42.095514 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:27:42.095476    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-djvzt_1d07e68d-9943-420f-a6a0-5f477792fabd/konnectivity-agent/0.log" limit=1024
Sep 17 18:27:42.441968 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:27:42.441931    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-test-llm-inference-service-62876e81/llmisvc-model-fb-opt-125m-route-50bc673d-kserve-5bf645b766dzg4n" event={"ID":"a443051c-645a-4acd-b004-bd7e26d33944","Type":"ContainerStarted","Data":"4aff41aefaea9955515a5d84928ec4012e03afc9d2789e3a2094effb72f9f7a8"}
Sep 17 18:27:42.470810 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:27:42.470754    2575 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="e2e-test-llm-inference-service-62876e81/llmisvc-model-fb-opt-125m-route-50bc673d-kserve-5bf645b766dzg4n" podStartSLOduration=4.094689442 podStartE2EDuration="8.470739462s" totalImagesPullingTime="2.262276619s" totalInitContainerRuntime="2.113773401s" isStatefulPod=false podCreationTimestamp="2026-09-17 18:27:34 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-17 18:27:42.467674947 +0000 UTC m=+3182.098004922" watchObservedRunningTime="2026-09-17 18:27:42.470739462 +0000 UTC m=+3182.101069485"
Sep 17 18:27:45.172780 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:27:45.172740    2575 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="e2e-test-llm-inference-service-62876e81/llmisvc-model-fb-opt-125m-route-50bc673d-kserve-5bf645b766dzg4n"
Sep 17 18:27:45.173193 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:27:45.172805    2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-test-llm-inference-service-62876e81/llmisvc-model-fb-opt-125m-route-50bc673d-kserve-5bf645b766dzg4n"
Sep 17 18:27:45.174235 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:27:45.174211    2575 prober.go:120] "Probe failed" probeType="Startup" pod="e2e-test-llm-inference-service-62876e81/llmisvc-model-fb-opt-125m-route-50bc673d-kserve-5bf645b766dzg4n" podUID="a443051c-645a-4acd-b004-bd7e26d33944" containerName="main" probeResult="failure" output="Get \"https://10.133.0.61:8001/health\": dial tcp 10.133.0.61:8001: connect: connection refused"
Sep 17 18:27:47.758478 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:27:47.758430    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-djvzt_1d07e68d-9943-420f-a6a0-5f477792fabd/konnectivity-agent/0.log" limit=1024
Sep 17 18:27:55.173144 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:27:55.173094    2575 prober.go:120] "Probe failed" probeType="Startup" pod="e2e-test-llm-inference-service-62876e81/llmisvc-model-fb-opt-125m-route-50bc673d-kserve-5bf645b766dzg4n" podUID="a443051c-645a-4acd-b004-bd7e26d33944" containerName="main" probeResult="failure" output="Get \"https://10.133.0.61:8001/health\": dial tcp 10.133.0.61:8001: connect: connection refused"
Sep 17 18:27:55.185022 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:27:55.184997    2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-test-llm-inference-service-62876e81/llmisvc-model-fb-opt-125m-route-50bc673d-kserve-5bf645b766dzg4n"
Sep 17 18:28:02.323501 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:28:02.323463    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-djvzt_1d07e68d-9943-420f-a6a0-5f477792fabd/konnectivity-agent/0.log" limit=1024
Sep 17 18:28:05.173152 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:28:05.173104    2575 prober.go:120] "Probe failed" probeType="Startup" pod="e2e-test-llm-inference-service-62876e81/llmisvc-model-fb-opt-125m-route-50bc673d-kserve-5bf645b766dzg4n" podUID="a443051c-645a-4acd-b004-bd7e26d33944" containerName="main" probeResult="failure" output="Get \"https://10.133.0.61:8001/health\": dial tcp 10.133.0.61:8001: connect: connection refused"
Sep 17 18:28:12.820198 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:28:12.820158    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-djvzt_1d07e68d-9943-420f-a6a0-5f477792fabd/konnectivity-agent/0.log" limit=1024
Sep 17 18:28:15.173532 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:28:15.173485    2575 prober.go:120] "Probe failed" probeType="Startup" pod="e2e-test-llm-inference-service-62876e81/llmisvc-model-fb-opt-125m-route-50bc673d-kserve-5bf645b766dzg4n" podUID="a443051c-645a-4acd-b004-bd7e26d33944" containerName="main" probeResult="failure" output="Get \"https://10.133.0.61:8001/health\": dial tcp 10.133.0.61:8001: connect: connection refused"
Sep 17 18:28:19.252572 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:28:19.252538    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-djvzt_1d07e68d-9943-420f-a6a0-5f477792fabd/konnectivity-agent/0.log" limit=1024
Sep 17 18:28:19.353701 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:28:19.353662    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-djvzt_1d07e68d-9943-420f-a6a0-5f477792fabd/konnectivity-agent/0.log" limit=1024
Sep 17 18:28:25.173956 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:28:25.173901    2575 prober.go:120] "Probe failed" probeType="Startup" pod="e2e-test-llm-inference-service-62876e81/llmisvc-model-fb-opt-125m-route-50bc673d-kserve-5bf645b766dzg4n" podUID="a443051c-645a-4acd-b004-bd7e26d33944" containerName="main" probeResult="failure" output="Get \"https://10.133.0.61:8001/health\": dial tcp 10.133.0.61:8001: connect: connection refused"
Sep 17 18:28:35.173244 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:28:35.173133    2575 prober.go:120] "Probe failed" probeType="Startup" pod="e2e-test-llm-inference-service-62876e81/llmisvc-model-fb-opt-125m-route-50bc673d-kserve-5bf645b766dzg4n" podUID="a443051c-645a-4acd-b004-bd7e26d33944" containerName="main" probeResult="failure" output="Get \"https://10.133.0.61:8001/health\": dial tcp 10.133.0.61:8001: connect: connection refused"
Sep 17 18:28:45.173982 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:28:45.173915    2575 prober.go:120] "Probe failed" probeType="Startup" pod="e2e-test-llm-inference-service-62876e81/llmisvc-model-fb-opt-125m-route-50bc673d-kserve-5bf645b766dzg4n" podUID="a443051c-645a-4acd-b004-bd7e26d33944" containerName="main" probeResult="failure" output="Get \"https://10.133.0.61:8001/health\": dial tcp 10.133.0.61:8001: connect: connection refused"
Sep 17 18:28:55.173827 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:28:55.173779    2575 prober.go:120] "Probe failed" probeType="Startup" pod="e2e-test-llm-inference-service-62876e81/llmisvc-model-fb-opt-125m-route-50bc673d-kserve-5bf645b766dzg4n" podUID="a443051c-645a-4acd-b004-bd7e26d33944" containerName="main" probeResult="failure" output="Get \"https://10.133.0.61:8001/health\": dial tcp 10.133.0.61:8001: connect: connection refused"
Sep 17 18:29:05.173115 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:29:05.173065    2575 prober.go:120] "Probe failed" probeType="Startup" pod="e2e-test-llm-inference-service-62876e81/llmisvc-model-fb-opt-125m-route-50bc673d-kserve-5bf645b766dzg4n" podUID="a443051c-645a-4acd-b004-bd7e26d33944" containerName="main" probeResult="failure" output="Get \"https://10.133.0.61:8001/health\": dial tcp 10.133.0.61:8001: connect: connection refused"
Sep 17 18:29:15.173606 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:29:15.173557    2575 prober.go:120] "Probe failed" probeType="Startup" pod="e2e-test-llm-inference-service-62876e81/llmisvc-model-fb-opt-125m-route-50bc673d-kserve-5bf645b766dzg4n" podUID="a443051c-645a-4acd-b004-bd7e26d33944" containerName="main" probeResult="failure" output="Get \"https://10.133.0.61:8001/health\": dial tcp 10.133.0.61:8001: connect: connection refused"
Sep 17 18:29:25.174017 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:29:25.173970    2575 prober.go:120] "Probe failed" probeType="Startup" pod="e2e-test-llm-inference-service-62876e81/llmisvc-model-fb-opt-125m-route-50bc673d-kserve-5bf645b766dzg4n" podUID="a443051c-645a-4acd-b004-bd7e26d33944" containerName="main" probeResult="failure" output="Get \"https://10.133.0.61:8001/health\": dial tcp 10.133.0.61:8001: connect: connection refused"
Sep 17 18:29:35.182750 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:29:35.182718    2575 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="e2e-test-llm-inference-service-62876e81/llmisvc-model-fb-opt-125m-route-50bc673d-kserve-5bf645b766dzg4n"
Sep 17 18:29:35.194208 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:29:35.194182    2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-test-llm-inference-service-62876e81/llmisvc-model-fb-opt-125m-route-50bc673d-kserve-5bf645b766dzg4n"
Sep 17 18:29:40.946272 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:29:40.946235    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-djvzt_1d07e68d-9943-420f-a6a0-5f477792fabd/konnectivity-agent/0.log" limit=1024
Sep 17 18:29:41.038698 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:29:41.038668    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-djvzt_1d07e68d-9943-420f-a6a0-5f477792fabd/konnectivity-agent/0.log" limit=1024
Sep 17 18:29:49.617708 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:29:49.617671    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-djvzt_1d07e68d-9943-420f-a6a0-5f477792fabd/konnectivity-agent/0.log" limit=1024
Sep 17 18:29:53.849803 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:29:53.849773    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-djvzt_1d07e68d-9943-420f-a6a0-5f477792fabd/konnectivity-agent/0.log" limit=1024
Sep 17 18:29:53.862113 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:29:53.862082    2575 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["e2e-test-llm-inference-service-62876e81/llmisvc-model-fb-opt-125m-route-50bc673d-kserve-5bf645b766dzg4n"]
Sep 17 18:29:53.920923 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:29:53.920885    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-djvzt_1d07e68d-9943-420f-a6a0-5f477792fabd/konnectivity-agent/0.log" limit=1024
Sep 17 18:30:08.889080 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:30:08.889017    2575 kuberuntime_container.go:873] "Killing container with a grace period" pod="e2e-test-llm-inference-service-62876e81/llmisvc-model-fb-opt-125m-route-50bc673d-kserve-5bf645b766dzg4n" podUID="a443051c-645a-4acd-b004-bd7e26d33944" containerName="main" containerID="cri-o://4aff41aefaea9955515a5d84928ec4012e03afc9d2789e3a2094effb72f9f7a8" gracePeriod=45
Sep 17 18:30:09.505390 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:30:09.505359    2575 kuberuntime_container.go:873] "Killing container with a grace period" pod="e2e-test-llm-inference-service-62876e81/llmisvc-model-fb-opt-125m-route-50bc673d-kserve-5bf645b766dzg4n" podUID="a443051c-645a-4acd-b004-bd7e26d33944" containerName="llm-d-routing-sidecar" containerID="cri-o://2411e3d8b0878bb567d3209003bb7c85d6979760705e753ec12968be48b49ae3" gracePeriod=45
Sep 17 18:30:09.754109 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:30:09.754087    2575 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-test-llm-inference-service-62876e81/llmisvc-model-fb-opt-125m-route-50bc673d-kserve-5bf645b766dzg4n"
Sep 17 18:30:09.774022 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:30:09.773998    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/a443051c-645a-4acd-b004-bd7e26d33944-model-cache\") pod \"a443051c-645a-4acd-b004-bd7e26d33944\" (UID: \"a443051c-645a-4acd-b004-bd7e26d33944\") "
Sep 17 18:30:09.774117 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:30:09.774036    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-7pj7b\" (UniqueName: \"kubernetes.io/projected/a443051c-645a-4acd-b004-bd7e26d33944-kube-api-access-7pj7b\") pod \"a443051c-645a-4acd-b004-bd7e26d33944\" (UID: \"a443051c-645a-4acd-b004-bd7e26d33944\") "
Sep 17 18:30:09.774117 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:30:09.774080    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"cabundle-cert\" (UniqueName: \"kubernetes.io/configmap/a443051c-645a-4acd-b004-bd7e26d33944-cabundle-cert\") pod \"a443051c-645a-4acd-b004-bd7e26d33944\" (UID: \"a443051c-645a-4acd-b004-bd7e26d33944\") "
Sep 17 18:30:09.774204 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:30:09.774118    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/a443051c-645a-4acd-b004-bd7e26d33944-home\") pod \"a443051c-645a-4acd-b004-bd7e26d33944\" (UID: \"a443051c-645a-4acd-b004-bd7e26d33944\") "
Sep 17 18:30:09.774204 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:30:09.774138    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/a443051c-645a-4acd-b004-bd7e26d33944-tmp-dir\") pod \"a443051c-645a-4acd-b004-bd7e26d33944\" (UID: \"a443051c-645a-4acd-b004-bd7e26d33944\") "
Sep 17 18:30:09.774300 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:30:09.774258    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a443051c-645a-4acd-b004-bd7e26d33944-model-cache" (OuterVolumeSpecName: "model-cache") pod "a443051c-645a-4acd-b004-bd7e26d33944" (UID: "a443051c-645a-4acd-b004-bd7e26d33944"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 17 18:30:09.774355 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:30:09.774306    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/a443051c-645a-4acd-b004-bd7e26d33944-kserve-provision-location\") pod \"a443051c-645a-4acd-b004-bd7e26d33944\" (UID: \"a443051c-645a-4acd-b004-bd7e26d33944\") "
Sep 17 18:30:09.774408 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:30:09.774371    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/a443051c-645a-4acd-b004-bd7e26d33944-dshm\") pod \"a443051c-645a-4acd-b004-bd7e26d33944\" (UID: \"a443051c-645a-4acd-b004-bd7e26d33944\") "
Sep 17 18:30:09.774408 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:30:09.774399    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/a443051c-645a-4acd-b004-bd7e26d33944-tls-certs\") pod \"a443051c-645a-4acd-b004-bd7e26d33944\" (UID: \"a443051c-645a-4acd-b004-bd7e26d33944\") "
Sep 17 18:30:09.774595 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:30:09.774565    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/a443051c-645a-4acd-b004-bd7e26d33944-cabundle-cert" (OuterVolumeSpecName: "cabundle-cert") pod "a443051c-645a-4acd-b004-bd7e26d33944" (UID: "a443051c-645a-4acd-b004-bd7e26d33944"). InnerVolumeSpecName "cabundle-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 17 18:30:09.774747 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:30:09.774706    2575 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/a443051c-645a-4acd-b004-bd7e26d33944-model-cache\") on node \"ip-10-0-143-74.ec2.internal\" DevicePath \"\""
Sep 17 18:30:09.774747 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:30:09.774728    2575 reconciler_common.go:299] "Volume detached for volume \"cabundle-cert\" (UniqueName: \"kubernetes.io/configmap/a443051c-645a-4acd-b004-bd7e26d33944-cabundle-cert\") on node \"ip-10-0-143-74.ec2.internal\" DevicePath \"\""
Sep 17 18:30:09.774866 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:30:09.774789    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a443051c-645a-4acd-b004-bd7e26d33944-home" (OuterVolumeSpecName: "home") pod "a443051c-645a-4acd-b004-bd7e26d33944" (UID: "a443051c-645a-4acd-b004-bd7e26d33944"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 17 18:30:09.776808 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:30:09.776781    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a443051c-645a-4acd-b004-bd7e26d33944-kube-api-access-7pj7b" (OuterVolumeSpecName: "kube-api-access-7pj7b") pod "a443051c-645a-4acd-b004-bd7e26d33944" (UID: "a443051c-645a-4acd-b004-bd7e26d33944"). InnerVolumeSpecName "kube-api-access-7pj7b". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 17 18:30:09.776943 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:30:09.776923    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/a443051c-645a-4acd-b004-bd7e26d33944-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "a443051c-645a-4acd-b004-bd7e26d33944" (UID: "a443051c-645a-4acd-b004-bd7e26d33944"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 17 18:30:09.777009 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:30:09.776947    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a443051c-645a-4acd-b004-bd7e26d33944-dshm" (OuterVolumeSpecName: "dshm") pod "a443051c-645a-4acd-b004-bd7e26d33944" (UID: "a443051c-645a-4acd-b004-bd7e26d33944"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 17 18:30:09.790223 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:30:09.790191    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a443051c-645a-4acd-b004-bd7e26d33944-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "a443051c-645a-4acd-b004-bd7e26d33944" (UID: "a443051c-645a-4acd-b004-bd7e26d33944"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 17 18:30:09.793875 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:30:09.793852    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a443051c-645a-4acd-b004-bd7e26d33944-kserve-provision-location" (OuterVolumeSpecName: "kserve-provision-location") pod "a443051c-645a-4acd-b004-bd7e26d33944" (UID: "a443051c-645a-4acd-b004-bd7e26d33944"). InnerVolumeSpecName "kserve-provision-location". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 17 18:30:09.875135 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:30:09.875103    2575 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/a443051c-645a-4acd-b004-bd7e26d33944-home\") on node \"ip-10-0-143-74.ec2.internal\" DevicePath \"\""
Sep 17 18:30:09.875135 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:30:09.875126    2575 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/a443051c-645a-4acd-b004-bd7e26d33944-tmp-dir\") on node \"ip-10-0-143-74.ec2.internal\" DevicePath \"\""
Sep 17 18:30:09.875310 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:30:09.875142    2575 reconciler_common.go:299] "Volume detached for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/a443051c-645a-4acd-b004-bd7e26d33944-kserve-provision-location\") on node \"ip-10-0-143-74.ec2.internal\" DevicePath \"\""
Sep 17 18:30:09.875310 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:30:09.875152    2575 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/a443051c-645a-4acd-b004-bd7e26d33944-dshm\") on node \"ip-10-0-143-74.ec2.internal\" DevicePath \"\""
Sep 17 18:30:09.875310 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:30:09.875160    2575 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/a443051c-645a-4acd-b004-bd7e26d33944-tls-certs\") on node \"ip-10-0-143-74.ec2.internal\" DevicePath \"\""
Sep 17 18:30:09.875310 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:30:09.875168    2575 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-7pj7b\" (UniqueName: \"kubernetes.io/projected/a443051c-645a-4acd-b004-bd7e26d33944-kube-api-access-7pj7b\") on node \"ip-10-0-143-74.ec2.internal\" DevicePath \"\""
Sep 17 18:30:10.296304 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:30:10.296266    2575 generic.go:356] "Generic (PLEG): container finished" podID="a443051c-645a-4acd-b004-bd7e26d33944" containerID="4aff41aefaea9955515a5d84928ec4012e03afc9d2789e3a2094effb72f9f7a8" exitCode=0
Sep 17 18:30:10.296304 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:30:10.296294    2575 generic.go:356] "Generic (PLEG): container finished" podID="a443051c-645a-4acd-b004-bd7e26d33944" containerID="2411e3d8b0878bb567d3209003bb7c85d6979760705e753ec12968be48b49ae3" exitCode=0
Sep 17 18:30:10.296761 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:30:10.296325    2575 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-test-llm-inference-service-62876e81/llmisvc-model-fb-opt-125m-route-50bc673d-kserve-5bf645b766dzg4n"
Sep 17 18:30:10.296761 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:30:10.296349    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-test-llm-inference-service-62876e81/llmisvc-model-fb-opt-125m-route-50bc673d-kserve-5bf645b766dzg4n" event={"ID":"a443051c-645a-4acd-b004-bd7e26d33944","Type":"ContainerDied","Data":"4aff41aefaea9955515a5d84928ec4012e03afc9d2789e3a2094effb72f9f7a8"}
Sep 17 18:30:10.296761 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:30:10.296401    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-test-llm-inference-service-62876e81/llmisvc-model-fb-opt-125m-route-50bc673d-kserve-5bf645b766dzg4n" event={"ID":"a443051c-645a-4acd-b004-bd7e26d33944","Type":"ContainerDied","Data":"2411e3d8b0878bb567d3209003bb7c85d6979760705e753ec12968be48b49ae3"}
Sep 17 18:30:10.296761 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:30:10.296421    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-test-llm-inference-service-62876e81/llmisvc-model-fb-opt-125m-route-50bc673d-kserve-5bf645b766dzg4n" event={"ID":"a443051c-645a-4acd-b004-bd7e26d33944","Type":"ContainerDied","Data":"bdddf3c360436bac1ac93821dafdfa4e22dbdef5873b4f08814a51bb4092d63f"}
Sep 17 18:30:10.296761 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:30:10.296460    2575 scope.go:117] "RemoveContainer" containerID="4aff41aefaea9955515a5d84928ec4012e03afc9d2789e3a2094effb72f9f7a8"
Sep 17 18:30:10.305720 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:30:10.305701    2575 scope.go:117] "RemoveContainer" containerID="8a1396e13838eb8c534c08672cbe649d1f141bef0667d774481d01b39de2381e"
Sep 17 18:30:10.313634 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:30:10.313617    2575 scope.go:117] "RemoveContainer" containerID="2411e3d8b0878bb567d3209003bb7c85d6979760705e753ec12968be48b49ae3"
Sep 17 18:30:10.321591 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:30:10.321568    2575 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["e2e-test-llm-inference-service-62876e81/llmisvc-model-fb-opt-125m-route-50bc673d-kserve-5bf645b766dzg4n"]
Sep 17 18:30:10.322007 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:30:10.321995    2575 scope.go:117] "RemoveContainer" containerID="4aff41aefaea9955515a5d84928ec4012e03afc9d2789e3a2094effb72f9f7a8"
Sep 17 18:30:10.322276 ip-10-0-143-74 kubenswrapper[2575]: E0917 18:30:10.322255    2575 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"4aff41aefaea9955515a5d84928ec4012e03afc9d2789e3a2094effb72f9f7a8\": container with ID starting with 4aff41aefaea9955515a5d84928ec4012e03afc9d2789e3a2094effb72f9f7a8 not found: ID does not exist" containerID="4aff41aefaea9955515a5d84928ec4012e03afc9d2789e3a2094effb72f9f7a8"
Sep 17 18:30:10.322337 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:30:10.322283    2575 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"4aff41aefaea9955515a5d84928ec4012e03afc9d2789e3a2094effb72f9f7a8"} err="failed to get container status \"4aff41aefaea9955515a5d84928ec4012e03afc9d2789e3a2094effb72f9f7a8\": rpc error: code = NotFound desc = could not find container \"4aff41aefaea9955515a5d84928ec4012e03afc9d2789e3a2094effb72f9f7a8\": container with ID starting with 4aff41aefaea9955515a5d84928ec4012e03afc9d2789e3a2094effb72f9f7a8 not found: ID does not exist"
Sep 17 18:30:10.322337 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:30:10.322303    2575 scope.go:117] "RemoveContainer" containerID="8a1396e13838eb8c534c08672cbe649d1f141bef0667d774481d01b39de2381e"
Sep 17 18:30:10.322556 ip-10-0-143-74 kubenswrapper[2575]: E0917 18:30:10.322539    2575 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"8a1396e13838eb8c534c08672cbe649d1f141bef0667d774481d01b39de2381e\": container with ID starting with 8a1396e13838eb8c534c08672cbe649d1f141bef0667d774481d01b39de2381e not found: ID does not exist" containerID="8a1396e13838eb8c534c08672cbe649d1f141bef0667d774481d01b39de2381e"
Sep 17 18:30:10.322615 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:30:10.322560    2575 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"8a1396e13838eb8c534c08672cbe649d1f141bef0667d774481d01b39de2381e"} err="failed to get container status \"8a1396e13838eb8c534c08672cbe649d1f141bef0667d774481d01b39de2381e\": rpc error: code = NotFound desc = could not find container \"8a1396e13838eb8c534c08672cbe649d1f141bef0667d774481d01b39de2381e\": container with ID starting with 8a1396e13838eb8c534c08672cbe649d1f141bef0667d774481d01b39de2381e not found: ID does not exist"
Sep 17 18:30:10.322615 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:30:10.322574    2575 scope.go:117] "RemoveContainer" containerID="2411e3d8b0878bb567d3209003bb7c85d6979760705e753ec12968be48b49ae3"
Sep 17 18:30:10.322839 ip-10-0-143-74 kubenswrapper[2575]: E0917 18:30:10.322822    2575 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"2411e3d8b0878bb567d3209003bb7c85d6979760705e753ec12968be48b49ae3\": container with ID starting with 2411e3d8b0878bb567d3209003bb7c85d6979760705e753ec12968be48b49ae3 not found: ID does not exist" containerID="2411e3d8b0878bb567d3209003bb7c85d6979760705e753ec12968be48b49ae3"
Sep 17 18:30:10.322888 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:30:10.322841    2575 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"2411e3d8b0878bb567d3209003bb7c85d6979760705e753ec12968be48b49ae3"} err="failed to get container status \"2411e3d8b0878bb567d3209003bb7c85d6979760705e753ec12968be48b49ae3\": rpc error: code = NotFound desc = could not find container \"2411e3d8b0878bb567d3209003bb7c85d6979760705e753ec12968be48b49ae3\": container with ID starting with 2411e3d8b0878bb567d3209003bb7c85d6979760705e753ec12968be48b49ae3 not found: ID does not exist"
Sep 17 18:30:10.322888 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:30:10.322855    2575 scope.go:117] "RemoveContainer" containerID="4aff41aefaea9955515a5d84928ec4012e03afc9d2789e3a2094effb72f9f7a8"
Sep 17 18:30:10.323071 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:30:10.323053    2575 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"4aff41aefaea9955515a5d84928ec4012e03afc9d2789e3a2094effb72f9f7a8"} err="failed to get container status \"4aff41aefaea9955515a5d84928ec4012e03afc9d2789e3a2094effb72f9f7a8\": rpc error: code = NotFound desc = could not find container \"4aff41aefaea9955515a5d84928ec4012e03afc9d2789e3a2094effb72f9f7a8\": container with ID starting with 4aff41aefaea9955515a5d84928ec4012e03afc9d2789e3a2094effb72f9f7a8 not found: ID does not exist"
Sep 17 18:30:10.323117 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:30:10.323072    2575 scope.go:117] "RemoveContainer" containerID="8a1396e13838eb8c534c08672cbe649d1f141bef0667d774481d01b39de2381e"
Sep 17 18:30:10.323301 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:30:10.323285    2575 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"8a1396e13838eb8c534c08672cbe649d1f141bef0667d774481d01b39de2381e"} err="failed to get container status \"8a1396e13838eb8c534c08672cbe649d1f141bef0667d774481d01b39de2381e\": rpc error: code = NotFound desc = could not find container \"8a1396e13838eb8c534c08672cbe649d1f141bef0667d774481d01b39de2381e\": container with ID starting with 8a1396e13838eb8c534c08672cbe649d1f141bef0667d774481d01b39de2381e not found: ID does not exist"
Sep 17 18:30:10.323343 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:30:10.323300    2575 scope.go:117] "RemoveContainer" containerID="2411e3d8b0878bb567d3209003bb7c85d6979760705e753ec12968be48b49ae3"
Sep 17 18:30:10.323529 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:30:10.323504    2575 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"2411e3d8b0878bb567d3209003bb7c85d6979760705e753ec12968be48b49ae3"} err="failed to get container status \"2411e3d8b0878bb567d3209003bb7c85d6979760705e753ec12968be48b49ae3\": rpc error: code = NotFound desc = could not find container \"2411e3d8b0878bb567d3209003bb7c85d6979760705e753ec12968be48b49ae3\": container with ID starting with 2411e3d8b0878bb567d3209003bb7c85d6979760705e753ec12968be48b49ae3 not found: ID does not exist"
Sep 17 18:30:10.326608 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:30:10.326585    2575 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["e2e-test-llm-inference-service-62876e81/llmisvc-model-fb-opt-125m-route-50bc673d-kserve-5bf645b766dzg4n"]
Sep 17 18:30:10.967744 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:30:10.967711    2575 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="a443051c-645a-4acd-b004-bd7e26d33944" path="/var/lib/kubelet/pods/a443051c-645a-4acd-b004-bd7e26d33944/volumes"
Sep 17 18:30:13.263858 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:30:13.263813    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-djvzt_1d07e68d-9943-420f-a6a0-5f477792fabd/konnectivity-agent/0.log" limit=1024
Sep 17 18:30:30.892187 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:30:30.892156    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-djvzt_1d07e68d-9943-420f-a6a0-5f477792fabd/konnectivity-agent/0.log" limit=1024
Sep 17 18:30:30.991029 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:30:30.991001    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-djvzt_1d07e68d-9943-420f-a6a0-5f477792fabd/konnectivity-agent/0.log" limit=1024
Sep 17 18:30:37.265257 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:30:37.265215    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-djvzt_1d07e68d-9943-420f-a6a0-5f477792fabd/konnectivity-agent/0.log" limit=1024
Sep 17 18:30:42.934524 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:30:42.934488    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-djvzt_1d07e68d-9943-420f-a6a0-5f477792fabd/konnectivity-agent/0.log" limit=1024
Sep 17 18:30:43.046813 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:30:43.046776    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-djvzt_1d07e68d-9943-420f-a6a0-5f477792fabd/konnectivity-agent/0.log" limit=1024
Sep 17 18:30:43.641475 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:30:43.641407    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-djvzt_1d07e68d-9943-420f-a6a0-5f477792fabd/konnectivity-agent/0.log" limit=1024
Sep 17 18:30:44.019798 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:30:44.019696    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-djvzt_1d07e68d-9943-420f-a6a0-5f477792fabd/konnectivity-agent/0.log" limit=1024
Sep 17 18:30:44.111175 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:30:44.111141    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-djvzt_1d07e68d-9943-420f-a6a0-5f477792fabd/konnectivity-agent/0.log" limit=1024
Sep 17 18:30:50.622798 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:30:50.622765    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-djvzt_1d07e68d-9943-420f-a6a0-5f477792fabd/konnectivity-agent/0.log" limit=1024
Sep 17 18:30:57.521426 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:30:57.521387    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-djvzt_1d07e68d-9943-420f-a6a0-5f477792fabd/konnectivity-agent/0.log" limit=1024
Sep 17 18:30:57.595303 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:30:57.595273    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-djvzt_1d07e68d-9943-420f-a6a0-5f477792fabd/konnectivity-agent/0.log" limit=1024
Sep 17 18:31:11.295388 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:31:11.295355    2575 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["e2e-test-llm-inference-service-d73d44f4/custom-route-timeout-pd-test-kserve-prefill-549595b7df-qxqsh"]
Sep 17 18:31:11.295822 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:31:11.295747    2575 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a443051c-645a-4acd-b004-bd7e26d33944" containerName="llm-d-routing-sidecar"
Sep 17 18:31:11.295822 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:31:11.295764    2575 state_mem.go:107] "Deleted CPUSet assignment" podUID="a443051c-645a-4acd-b004-bd7e26d33944" containerName="llm-d-routing-sidecar"
Sep 17 18:31:11.295822 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:31:11.295778    2575 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a443051c-645a-4acd-b004-bd7e26d33944" containerName="storage-initializer"
Sep 17 18:31:11.295822 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:31:11.295788    2575 state_mem.go:107] "Deleted CPUSet assignment" podUID="a443051c-645a-4acd-b004-bd7e26d33944" containerName="storage-initializer"
Sep 17 18:31:11.295822 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:31:11.295817    2575 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a443051c-645a-4acd-b004-bd7e26d33944" containerName="main"
Sep 17 18:31:11.295992 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:31:11.295825    2575 state_mem.go:107] "Deleted CPUSet assignment" podUID="a443051c-645a-4acd-b004-bd7e26d33944" containerName="main"
Sep 17 18:31:11.295992 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:31:11.295908    2575 memory_manager.go:368] "RemoveStaleState removing state" podUID="a443051c-645a-4acd-b004-bd7e26d33944" containerName="main"
Sep 17 18:31:11.295992 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:31:11.295919    2575 memory_manager.go:368] "RemoveStaleState removing state" podUID="a443051c-645a-4acd-b004-bd7e26d33944" containerName="llm-d-routing-sidecar"
Sep 17 18:31:11.300533 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:31:11.300510    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-test-llm-inference-service-d73d44f4/custom-route-timeout-pd-test-kserve-prefill-549595b7df-qxqsh"
Sep 17 18:31:11.303901 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:31:11.303876    2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-test-llm-inference-service-d73d44f4\"/\"kube-root-ca.crt\""
Sep 17 18:31:11.304030 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:31:11.303949    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-test-llm-inference-service-d73d44f4\"/\"seaweedfs-s3-creds\""
Sep 17 18:31:11.304315 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:31:11.304292    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-test-llm-inference-service-d73d44f4\"/\"custom-route-timeout-pd-test-kserve-self-signed-certs\""
Sep 17 18:31:11.304315 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:31:11.304308    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-test-llm-inference-service-d73d44f4\"/\"default-dockercfg-gcrsm\""
Sep 17 18:31:11.304491 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:31:11.304318    2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-test-llm-inference-service-d73d44f4\"/\"openshift-service-ca.crt\""
Sep 17 18:31:11.305400 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:31:11.305382    2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-test-llm-inference-service-d73d44f4\"/\"odh-kserve-custom-ca-bundle\""
Sep 17 18:31:11.312504 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:31:11.312484    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["e2e-test-llm-inference-service-d73d44f4/custom-route-timeout-pd-test-kserve-prefill-549595b7df-qxqsh"]
Sep 17 18:31:11.331708 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:31:11.331684    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-djvzt_1d07e68d-9943-420f-a6a0-5f477792fabd/konnectivity-agent/0.log" limit=1024
Sep 17 18:31:11.396078 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:31:11.396043    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/ccb9097d-7605-4e8f-acf9-ca75f4ecc720-model-cache\") pod \"custom-route-timeout-pd-test-kserve-prefill-549595b7df-qxqsh\" (UID: \"ccb9097d-7605-4e8f-acf9-ca75f4ecc720\") " pod="e2e-test-llm-inference-service-d73d44f4/custom-route-timeout-pd-test-kserve-prefill-549595b7df-qxqsh"
Sep 17 18:31:11.396078 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:31:11.396081    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/ccb9097d-7605-4e8f-acf9-ca75f4ecc720-kserve-provision-location\") pod \"custom-route-timeout-pd-test-kserve-prefill-549595b7df-qxqsh\" (UID: \"ccb9097d-7605-4e8f-acf9-ca75f4ecc720\") " pod="e2e-test-llm-inference-service-d73d44f4/custom-route-timeout-pd-test-kserve-prefill-549595b7df-qxqsh"
Sep 17 18:31:11.396290 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:31:11.396106    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cabundle-cert\" (UniqueName: \"kubernetes.io/configmap/ccb9097d-7605-4e8f-acf9-ca75f4ecc720-cabundle-cert\") pod \"custom-route-timeout-pd-test-kserve-prefill-549595b7df-qxqsh\" (UID: \"ccb9097d-7605-4e8f-acf9-ca75f4ecc720\") " pod="e2e-test-llm-inference-service-d73d44f4/custom-route-timeout-pd-test-kserve-prefill-549595b7df-qxqsh"
Sep 17 18:31:11.396290 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:31:11.396155    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/ccb9097d-7605-4e8f-acf9-ca75f4ecc720-home\") pod \"custom-route-timeout-pd-test-kserve-prefill-549595b7df-qxqsh\" (UID: \"ccb9097d-7605-4e8f-acf9-ca75f4ecc720\") " pod="e2e-test-llm-inference-service-d73d44f4/custom-route-timeout-pd-test-kserve-prefill-549595b7df-qxqsh"
Sep 17 18:31:11.396290 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:31:11.396201    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/ccb9097d-7605-4e8f-acf9-ca75f4ecc720-tmp-dir\") pod \"custom-route-timeout-pd-test-kserve-prefill-549595b7df-qxqsh\" (UID: \"ccb9097d-7605-4e8f-acf9-ca75f4ecc720\") " pod="e2e-test-llm-inference-service-d73d44f4/custom-route-timeout-pd-test-kserve-prefill-549595b7df-qxqsh"
Sep 17 18:31:11.396290 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:31:11.396227    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/ccb9097d-7605-4e8f-acf9-ca75f4ecc720-dshm\") pod \"custom-route-timeout-pd-test-kserve-prefill-549595b7df-qxqsh\" (UID: \"ccb9097d-7605-4e8f-acf9-ca75f4ecc720\") " pod="e2e-test-llm-inference-service-d73d44f4/custom-route-timeout-pd-test-kserve-prefill-549595b7df-qxqsh"
Sep 17 18:31:11.396290 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:31:11.396246    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vplss\" (UniqueName: \"kubernetes.io/projected/ccb9097d-7605-4e8f-acf9-ca75f4ecc720-kube-api-access-vplss\") pod \"custom-route-timeout-pd-test-kserve-prefill-549595b7df-qxqsh\" (UID: \"ccb9097d-7605-4e8f-acf9-ca75f4ecc720\") " pod="e2e-test-llm-inference-service-d73d44f4/custom-route-timeout-pd-test-kserve-prefill-549595b7df-qxqsh"
Sep 17 18:31:11.396290 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:31:11.396266    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/ccb9097d-7605-4e8f-acf9-ca75f4ecc720-tls-certs\") pod \"custom-route-timeout-pd-test-kserve-prefill-549595b7df-qxqsh\" (UID: \"ccb9097d-7605-4e8f-acf9-ca75f4ecc720\") " pod="e2e-test-llm-inference-service-d73d44f4/custom-route-timeout-pd-test-kserve-prefill-549595b7df-qxqsh"
Sep 17 18:31:11.497573 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:31:11.497539    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/ccb9097d-7605-4e8f-acf9-ca75f4ecc720-kserve-provision-location\") pod \"custom-route-timeout-pd-test-kserve-prefill-549595b7df-qxqsh\" (UID: \"ccb9097d-7605-4e8f-acf9-ca75f4ecc720\") " pod="e2e-test-llm-inference-service-d73d44f4/custom-route-timeout-pd-test-kserve-prefill-549595b7df-qxqsh"
Sep 17 18:31:11.497758 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:31:11.497586    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cabundle-cert\" (UniqueName: \"kubernetes.io/configmap/ccb9097d-7605-4e8f-acf9-ca75f4ecc720-cabundle-cert\") pod \"custom-route-timeout-pd-test-kserve-prefill-549595b7df-qxqsh\" (UID: \"ccb9097d-7605-4e8f-acf9-ca75f4ecc720\") " pod="e2e-test-llm-inference-service-d73d44f4/custom-route-timeout-pd-test-kserve-prefill-549595b7df-qxqsh"
Sep 17 18:31:11.497758 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:31:11.497620    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/ccb9097d-7605-4e8f-acf9-ca75f4ecc720-home\") pod \"custom-route-timeout-pd-test-kserve-prefill-549595b7df-qxqsh\" (UID: \"ccb9097d-7605-4e8f-acf9-ca75f4ecc720\") " pod="e2e-test-llm-inference-service-d73d44f4/custom-route-timeout-pd-test-kserve-prefill-549595b7df-qxqsh"
Sep 17 18:31:11.497758 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:31:11.497661    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/ccb9097d-7605-4e8f-acf9-ca75f4ecc720-tmp-dir\") pod \"custom-route-timeout-pd-test-kserve-prefill-549595b7df-qxqsh\" (UID: \"ccb9097d-7605-4e8f-acf9-ca75f4ecc720\") " pod="e2e-test-llm-inference-service-d73d44f4/custom-route-timeout-pd-test-kserve-prefill-549595b7df-qxqsh"
Sep 17 18:31:11.497758 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:31:11.497697    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/ccb9097d-7605-4e8f-acf9-ca75f4ecc720-dshm\") pod \"custom-route-timeout-pd-test-kserve-prefill-549595b7df-qxqsh\" (UID: \"ccb9097d-7605-4e8f-acf9-ca75f4ecc720\") " pod="e2e-test-llm-inference-service-d73d44f4/custom-route-timeout-pd-test-kserve-prefill-549595b7df-qxqsh"
Sep 17 18:31:11.497758 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:31:11.497724    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vplss\" (UniqueName: \"kubernetes.io/projected/ccb9097d-7605-4e8f-acf9-ca75f4ecc720-kube-api-access-vplss\") pod \"custom-route-timeout-pd-test-kserve-prefill-549595b7df-qxqsh\" (UID: \"ccb9097d-7605-4e8f-acf9-ca75f4ecc720\") " pod="e2e-test-llm-inference-service-d73d44f4/custom-route-timeout-pd-test-kserve-prefill-549595b7df-qxqsh"
Sep 17 18:31:11.497758 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:31:11.497749    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/ccb9097d-7605-4e8f-acf9-ca75f4ecc720-tls-certs\") pod \"custom-route-timeout-pd-test-kserve-prefill-549595b7df-qxqsh\" (UID: \"ccb9097d-7605-4e8f-acf9-ca75f4ecc720\") " pod="e2e-test-llm-inference-service-d73d44f4/custom-route-timeout-pd-test-kserve-prefill-549595b7df-qxqsh"
Sep 17 18:31:11.498078 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:31:11.497804    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/ccb9097d-7605-4e8f-acf9-ca75f4ecc720-model-cache\") pod \"custom-route-timeout-pd-test-kserve-prefill-549595b7df-qxqsh\" (UID: \"ccb9097d-7605-4e8f-acf9-ca75f4ecc720\") " pod="e2e-test-llm-inference-service-d73d44f4/custom-route-timeout-pd-test-kserve-prefill-549595b7df-qxqsh"
Sep 17 18:31:11.498078 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:31:11.498002    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/ccb9097d-7605-4e8f-acf9-ca75f4ecc720-home\") pod \"custom-route-timeout-pd-test-kserve-prefill-549595b7df-qxqsh\" (UID: \"ccb9097d-7605-4e8f-acf9-ca75f4ecc720\") " pod="e2e-test-llm-inference-service-d73d44f4/custom-route-timeout-pd-test-kserve-prefill-549595b7df-qxqsh"
Sep 17 18:31:11.498189 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:31:11.498138    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/ccb9097d-7605-4e8f-acf9-ca75f4ecc720-kserve-provision-location\") pod \"custom-route-timeout-pd-test-kserve-prefill-549595b7df-qxqsh\" (UID: \"ccb9097d-7605-4e8f-acf9-ca75f4ecc720\") " pod="e2e-test-llm-inference-service-d73d44f4/custom-route-timeout-pd-test-kserve-prefill-549595b7df-qxqsh"
Sep 17 18:31:11.498255 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:31:11.498193    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/ccb9097d-7605-4e8f-acf9-ca75f4ecc720-tmp-dir\") pod \"custom-route-timeout-pd-test-kserve-prefill-549595b7df-qxqsh\" (UID: \"ccb9097d-7605-4e8f-acf9-ca75f4ecc720\") " pod="e2e-test-llm-inference-service-d73d44f4/custom-route-timeout-pd-test-kserve-prefill-549595b7df-qxqsh"
Sep 17 18:31:11.498255 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:31:11.498197    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/ccb9097d-7605-4e8f-acf9-ca75f4ecc720-model-cache\") pod \"custom-route-timeout-pd-test-kserve-prefill-549595b7df-qxqsh\" (UID: \"ccb9097d-7605-4e8f-acf9-ca75f4ecc720\") " pod="e2e-test-llm-inference-service-d73d44f4/custom-route-timeout-pd-test-kserve-prefill-549595b7df-qxqsh"
Sep 17 18:31:11.498487 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:31:11.498433    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cabundle-cert\" (UniqueName: \"kubernetes.io/configmap/ccb9097d-7605-4e8f-acf9-ca75f4ecc720-cabundle-cert\") pod \"custom-route-timeout-pd-test-kserve-prefill-549595b7df-qxqsh\" (UID: \"ccb9097d-7605-4e8f-acf9-ca75f4ecc720\") " pod="e2e-test-llm-inference-service-d73d44f4/custom-route-timeout-pd-test-kserve-prefill-549595b7df-qxqsh"
Sep 17 18:31:11.500000 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:31:11.499981    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/ccb9097d-7605-4e8f-acf9-ca75f4ecc720-dshm\") pod \"custom-route-timeout-pd-test-kserve-prefill-549595b7df-qxqsh\" (UID: \"ccb9097d-7605-4e8f-acf9-ca75f4ecc720\") " pod="e2e-test-llm-inference-service-d73d44f4/custom-route-timeout-pd-test-kserve-prefill-549595b7df-qxqsh"
Sep 17 18:31:11.500503 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:31:11.500471    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/ccb9097d-7605-4e8f-acf9-ca75f4ecc720-tls-certs\") pod \"custom-route-timeout-pd-test-kserve-prefill-549595b7df-qxqsh\" (UID: \"ccb9097d-7605-4e8f-acf9-ca75f4ecc720\") " pod="e2e-test-llm-inference-service-d73d44f4/custom-route-timeout-pd-test-kserve-prefill-549595b7df-qxqsh"
Sep 17 18:31:11.505876 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:31:11.505851    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-vplss\" (UniqueName: \"kubernetes.io/projected/ccb9097d-7605-4e8f-acf9-ca75f4ecc720-kube-api-access-vplss\") pod \"custom-route-timeout-pd-test-kserve-prefill-549595b7df-qxqsh\" (UID: \"ccb9097d-7605-4e8f-acf9-ca75f4ecc720\") " pod="e2e-test-llm-inference-service-d73d44f4/custom-route-timeout-pd-test-kserve-prefill-549595b7df-qxqsh"
Sep 17 18:31:11.611517 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:31:11.611389    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-test-llm-inference-service-d73d44f4/custom-route-timeout-pd-test-kserve-prefill-549595b7df-qxqsh"
Sep 17 18:31:11.665689 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:31:11.665651    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-djvzt_1d07e68d-9943-420f-a6a0-5f477792fabd/konnectivity-agent/0.log" limit=1024
Sep 17 18:31:11.894506 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:31:11.894478    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["e2e-test-llm-inference-service-d73d44f4/custom-route-timeout-pd-test-kserve-prefill-549595b7df-qxqsh"]
Sep 17 18:31:11.896208 ip-10-0-143-74 kubenswrapper[2575]: W0917 18:31:11.896164    2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podccb9097d_7605_4e8f_acf9_ca75f4ecc720.slice/crio-e75f4e731672e42b312cde6450f654a6bb992f46e0e30268bd22e1655390316f WatchSource:0}: Error finding container e75f4e731672e42b312cde6450f654a6bb992f46e0e30268bd22e1655390316f: Status 404 returned error can't find the container with id e75f4e731672e42b312cde6450f654a6bb992f46e0e30268bd22e1655390316f
Sep 17 18:31:12.630273 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:31:12.630237    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-test-llm-inference-service-d73d44f4/custom-route-timeout-pd-test-kserve-prefill-549595b7df-qxqsh" event={"ID":"ccb9097d-7605-4e8f-acf9-ca75f4ecc720","Type":"ContainerStarted","Data":"2074abe8b1662eea5f7ba63509827b3a0c5dd390cac6f0727e0c05b89706bc32"}
Sep 17 18:31:12.630723 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:31:12.630291    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-test-llm-inference-service-d73d44f4/custom-route-timeout-pd-test-kserve-prefill-549595b7df-qxqsh" event={"ID":"ccb9097d-7605-4e8f-acf9-ca75f4ecc720","Type":"ContainerStarted","Data":"e75f4e731672e42b312cde6450f654a6bb992f46e0e30268bd22e1655390316f"}
Sep 17 18:31:14.643377 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:31:14.643343    2575 generic.go:356] "Generic (PLEG): container finished" podID="ccb9097d-7605-4e8f-acf9-ca75f4ecc720" containerID="2074abe8b1662eea5f7ba63509827b3a0c5dd390cac6f0727e0c05b89706bc32" exitCode=0
Sep 17 18:31:14.643757 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:31:14.643390    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-test-llm-inference-service-d73d44f4/custom-route-timeout-pd-test-kserve-prefill-549595b7df-qxqsh" event={"ID":"ccb9097d-7605-4e8f-acf9-ca75f4ecc720","Type":"ContainerDied","Data":"2074abe8b1662eea5f7ba63509827b3a0c5dd390cac6f0727e0c05b89706bc32"}
Sep 17 18:31:15.650618 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:31:15.650581    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-test-llm-inference-service-d73d44f4/custom-route-timeout-pd-test-kserve-prefill-549595b7df-qxqsh" event={"ID":"ccb9097d-7605-4e8f-acf9-ca75f4ecc720","Type":"ContainerStarted","Data":"5532852c70a3c48177bec8d65d09c9b666c75688741e13b012d9f52b1ec865c4"}
Sep 17 18:31:15.680328 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:31:15.680270    2575 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="e2e-test-llm-inference-service-d73d44f4/custom-route-timeout-pd-test-kserve-prefill-549595b7df-qxqsh" podStartSLOduration=2.468199065 podStartE2EDuration="4.680253902s" totalImagesPullingTime="0s" totalInitContainerRuntime="2.212054837s" isStatefulPod=false podCreationTimestamp="2026-09-17 18:31:11 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-17 18:31:15.676768458 +0000 UTC m=+3395.307098432" watchObservedRunningTime="2026-09-17 18:31:15.680253902 +0000 UTC m=+3395.310583876"
Sep 17 18:31:21.612493 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:31:21.612458    2575 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="e2e-test-llm-inference-service-d73d44f4/custom-route-timeout-pd-test-kserve-prefill-549595b7df-qxqsh"
Sep 17 18:31:21.612957 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:31:21.612516    2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-test-llm-inference-service-d73d44f4/custom-route-timeout-pd-test-kserve-prefill-549595b7df-qxqsh"
Sep 17 18:31:21.614147 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:31:21.614119    2575 prober.go:120] "Probe failed" probeType="Startup" pod="e2e-test-llm-inference-service-d73d44f4/custom-route-timeout-pd-test-kserve-prefill-549595b7df-qxqsh" podUID="ccb9097d-7605-4e8f-acf9-ca75f4ecc720" containerName="main" probeResult="failure" output="Get \"https://10.133.0.62:8000/health\": dial tcp 10.133.0.62:8000: connect: connection refused"
Sep 17 18:31:21.720666 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:31:21.720630    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-djvzt_1d07e68d-9943-420f-a6a0-5f477792fabd/konnectivity-agent/0.log" limit=1024
Sep 17 18:31:31.612144 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:31:31.612103    2575 prober.go:120] "Probe failed" probeType="Startup" pod="e2e-test-llm-inference-service-d73d44f4/custom-route-timeout-pd-test-kserve-prefill-549595b7df-qxqsh" podUID="ccb9097d-7605-4e8f-acf9-ca75f4ecc720" containerName="main" probeResult="failure" output="Get \"https://10.133.0.62:8000/health\": dial tcp 10.133.0.62:8000: connect: connection refused"
Sep 17 18:31:39.044173 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:31:39.044087    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-djvzt_1d07e68d-9943-420f-a6a0-5f477792fabd/konnectivity-agent/0.log" limit=1024
Sep 17 18:31:39.145832 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:31:39.145791    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-djvzt_1d07e68d-9943-420f-a6a0-5f477792fabd/konnectivity-agent/0.log" limit=1024
Sep 17 18:31:41.612494 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:31:41.612437    2575 prober.go:120] "Probe failed" probeType="Startup" pod="e2e-test-llm-inference-service-d73d44f4/custom-route-timeout-pd-test-kserve-prefill-549595b7df-qxqsh" podUID="ccb9097d-7605-4e8f-acf9-ca75f4ecc720" containerName="main" probeResult="failure" output="Get \"https://10.133.0.62:8000/health\": dial tcp 10.133.0.62:8000: connect: connection refused"
Sep 17 18:31:44.513436 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:31:44.513405    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-djvzt_1d07e68d-9943-420f-a6a0-5f477792fabd/konnectivity-agent/0.log" limit=1024
Sep 17 18:31:44.614711 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:31:44.614676    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-djvzt_1d07e68d-9943-420f-a6a0-5f477792fabd/konnectivity-agent/0.log" limit=1024
Sep 17 18:31:51.612206 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:31:51.612161    2575 prober.go:120] "Probe failed" probeType="Startup" pod="e2e-test-llm-inference-service-d73d44f4/custom-route-timeout-pd-test-kserve-prefill-549595b7df-qxqsh" podUID="ccb9097d-7605-4e8f-acf9-ca75f4ecc720" containerName="main" probeResult="failure" output="Get \"https://10.133.0.62:8000/health\": dial tcp 10.133.0.62:8000: connect: connection refused"
Sep 17 18:32:01.612360 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:32:01.612305    2575 prober.go:120] "Probe failed" probeType="Startup" pod="e2e-test-llm-inference-service-d73d44f4/custom-route-timeout-pd-test-kserve-prefill-549595b7df-qxqsh" podUID="ccb9097d-7605-4e8f-acf9-ca75f4ecc720" containerName="main" probeResult="failure" output="Get \"https://10.133.0.62:8000/health\": dial tcp 10.133.0.62:8000: connect: connection refused"
Sep 17 18:32:10.115050 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:32:10.115012    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-djvzt_1d07e68d-9943-420f-a6a0-5f477792fabd/konnectivity-agent/0.log" limit=1024
Sep 17 18:32:11.612564 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:32:11.612506    2575 prober.go:120] "Probe failed" probeType="Startup" pod="e2e-test-llm-inference-service-d73d44f4/custom-route-timeout-pd-test-kserve-prefill-549595b7df-qxqsh" podUID="ccb9097d-7605-4e8f-acf9-ca75f4ecc720" containerName="main" probeResult="failure" output="Get \"https://10.133.0.62:8000/health\": dial tcp 10.133.0.62:8000: connect: connection refused"
Sep 17 18:32:21.612205 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:32:21.612158    2575 prober.go:120] "Probe failed" probeType="Startup" pod="e2e-test-llm-inference-service-d73d44f4/custom-route-timeout-pd-test-kserve-prefill-549595b7df-qxqsh" podUID="ccb9097d-7605-4e8f-acf9-ca75f4ecc720" containerName="main" probeResult="failure" output="Get \"https://10.133.0.62:8000/health\": dial tcp 10.133.0.62:8000: connect: connection refused"
Sep 17 18:32:30.225814 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:32:30.225772    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-djvzt_1d07e68d-9943-420f-a6a0-5f477792fabd/konnectivity-agent/0.log" limit=1024
Sep 17 18:32:31.612859 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:32:31.612808    2575 prober.go:120] "Probe failed" probeType="Startup" pod="e2e-test-llm-inference-service-d73d44f4/custom-route-timeout-pd-test-kserve-prefill-549595b7df-qxqsh" podUID="ccb9097d-7605-4e8f-acf9-ca75f4ecc720" containerName="main" probeResult="failure" output="Get \"https://10.133.0.62:8000/health\": dial tcp 10.133.0.62:8000: connect: connection refused"
Sep 17 18:32:41.612373 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:32:41.612330    2575 prober.go:120] "Probe failed" probeType="Startup" pod="e2e-test-llm-inference-service-d73d44f4/custom-route-timeout-pd-test-kserve-prefill-549595b7df-qxqsh" podUID="ccb9097d-7605-4e8f-acf9-ca75f4ecc720" containerName="main" probeResult="failure" output="Get \"https://10.133.0.62:8000/health\": dial tcp 10.133.0.62:8000: connect: connection refused"
Sep 17 18:32:50.335921 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:32:50.335881    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-djvzt_1d07e68d-9943-420f-a6a0-5f477792fabd/konnectivity-agent/0.log" limit=1024
Sep 17 18:32:50.431146 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:32:50.431112    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-djvzt_1d07e68d-9943-420f-a6a0-5f477792fabd/konnectivity-agent/0.log" limit=1024
Sep 17 18:32:50.534254 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:32:50.534215    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-djvzt_1d07e68d-9943-420f-a6a0-5f477792fabd/konnectivity-agent/0.log" limit=1024
Sep 17 18:32:51.612810 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:32:51.612767    2575 prober.go:120] "Probe failed" probeType="Startup" pod="e2e-test-llm-inference-service-d73d44f4/custom-route-timeout-pd-test-kserve-prefill-549595b7df-qxqsh" podUID="ccb9097d-7605-4e8f-acf9-ca75f4ecc720" containerName="main" probeResult="failure" output="Get \"https://10.133.0.62:8000/health\": dial tcp 10.133.0.62:8000: connect: connection refused"
Sep 17 18:32:56.756402 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:32:56.756358    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-djvzt_1d07e68d-9943-420f-a6a0-5f477792fabd/konnectivity-agent/0.log" limit=1024
Sep 17 18:32:56.856841 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:32:56.856811    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-djvzt_1d07e68d-9943-420f-a6a0-5f477792fabd/konnectivity-agent/0.log" limit=1024
Sep 17 18:32:59.147212 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:32:59.147164    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-djvzt_1d07e68d-9943-420f-a6a0-5f477792fabd/konnectivity-agent/0.log" limit=1024
Sep 17 18:33:01.626037 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:33:01.626002    2575 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="e2e-test-llm-inference-service-d73d44f4/custom-route-timeout-pd-test-kserve-prefill-549595b7df-qxqsh"
Sep 17 18:33:01.635403 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:33:01.635373    2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-test-llm-inference-service-d73d44f4/custom-route-timeout-pd-test-kserve-prefill-549595b7df-qxqsh"
Sep 17 18:33:22.014116 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:33:22.014081    2575 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["e2e-test-llm-inference-service-d73d44f4/custom-route-timeout-pd-test-kserve-prefill-549595b7df-qxqsh"]
Sep 17 18:33:22.023824 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:33:22.023781    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-djvzt_1d07e68d-9943-420f-a6a0-5f477792fabd/konnectivity-agent/0.log" limit=1024
Sep 17 18:33:22.097953 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:33:22.097916    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-djvzt_1d07e68d-9943-420f-a6a0-5f477792fabd/konnectivity-agent/0.log" limit=1024
Sep 17 18:33:27.522897 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:33:27.522864    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-djvzt_1d07e68d-9943-420f-a6a0-5f477792fabd/konnectivity-agent/0.log" limit=1024
Sep 17 18:33:27.634804 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:33:27.634753    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-djvzt_1d07e68d-9943-420f-a6a0-5f477792fabd/konnectivity-agent/0.log" limit=1024
Sep 17 18:33:37.036248 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:33:37.036192    2575 kuberuntime_container.go:873] "Killing container with a grace period" pod="e2e-test-llm-inference-service-d73d44f4/custom-route-timeout-pd-test-kserve-prefill-549595b7df-qxqsh" podUID="ccb9097d-7605-4e8f-acf9-ca75f4ecc720" containerName="main" containerID="cri-o://5532852c70a3c48177bec8d65d09c9b666c75688741e13b012d9f52b1ec865c4" gracePeriod=45
Sep 17 18:33:37.784398 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:33:37.784370    2575 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-test-llm-inference-service-d73d44f4/custom-route-timeout-pd-test-kserve-prefill-549595b7df-qxqsh"
Sep 17 18:33:37.836082 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:33:37.835993    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"cabundle-cert\" (UniqueName: \"kubernetes.io/configmap/ccb9097d-7605-4e8f-acf9-ca75f4ecc720-cabundle-cert\") pod \"ccb9097d-7605-4e8f-acf9-ca75f4ecc720\" (UID: \"ccb9097d-7605-4e8f-acf9-ca75f4ecc720\") "
Sep 17 18:33:37.836082 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:33:37.836037    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/ccb9097d-7605-4e8f-acf9-ca75f4ecc720-tmp-dir\") pod \"ccb9097d-7605-4e8f-acf9-ca75f4ecc720\" (UID: \"ccb9097d-7605-4e8f-acf9-ca75f4ecc720\") "
Sep 17 18:33:37.836307 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:33:37.836093    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/ccb9097d-7605-4e8f-acf9-ca75f4ecc720-kserve-provision-location\") pod \"ccb9097d-7605-4e8f-acf9-ca75f4ecc720\" (UID: \"ccb9097d-7605-4e8f-acf9-ca75f4ecc720\") "
Sep 17 18:33:37.836307 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:33:37.836121    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/ccb9097d-7605-4e8f-acf9-ca75f4ecc720-tls-certs\") pod \"ccb9097d-7605-4e8f-acf9-ca75f4ecc720\" (UID: \"ccb9097d-7605-4e8f-acf9-ca75f4ecc720\") "
Sep 17 18:33:37.836307 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:33:37.836180    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/ccb9097d-7605-4e8f-acf9-ca75f4ecc720-model-cache\") pod \"ccb9097d-7605-4e8f-acf9-ca75f4ecc720\" (UID: \"ccb9097d-7605-4e8f-acf9-ca75f4ecc720\") "
Sep 17 18:33:37.836307 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:33:37.836200    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/ccb9097d-7605-4e8f-acf9-ca75f4ecc720-home\") pod \"ccb9097d-7605-4e8f-acf9-ca75f4ecc720\" (UID: \"ccb9097d-7605-4e8f-acf9-ca75f4ecc720\") "
Sep 17 18:33:37.836307 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:33:37.836223    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-vplss\" (UniqueName: \"kubernetes.io/projected/ccb9097d-7605-4e8f-acf9-ca75f4ecc720-kube-api-access-vplss\") pod \"ccb9097d-7605-4e8f-acf9-ca75f4ecc720\" (UID: \"ccb9097d-7605-4e8f-acf9-ca75f4ecc720\") "
Sep 17 18:33:37.836307 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:33:37.836253    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/ccb9097d-7605-4e8f-acf9-ca75f4ecc720-dshm\") pod \"ccb9097d-7605-4e8f-acf9-ca75f4ecc720\" (UID: \"ccb9097d-7605-4e8f-acf9-ca75f4ecc720\") "
Sep 17 18:33:37.836631 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:33:37.836575    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ccb9097d-7605-4e8f-acf9-ca75f4ecc720-model-cache" (OuterVolumeSpecName: "model-cache") pod "ccb9097d-7605-4e8f-acf9-ca75f4ecc720" (UID: "ccb9097d-7605-4e8f-acf9-ca75f4ecc720"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 17 18:33:37.837033 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:33:37.836983    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/ccb9097d-7605-4e8f-acf9-ca75f4ecc720-cabundle-cert" (OuterVolumeSpecName: "cabundle-cert") pod "ccb9097d-7605-4e8f-acf9-ca75f4ecc720" (UID: "ccb9097d-7605-4e8f-acf9-ca75f4ecc720"). InnerVolumeSpecName "cabundle-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 17 18:33:37.837175 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:33:37.837059    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ccb9097d-7605-4e8f-acf9-ca75f4ecc720-home" (OuterVolumeSpecName: "home") pod "ccb9097d-7605-4e8f-acf9-ca75f4ecc720" (UID: "ccb9097d-7605-4e8f-acf9-ca75f4ecc720"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 17 18:33:37.839229 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:33:37.839205    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ccb9097d-7605-4e8f-acf9-ca75f4ecc720-dshm" (OuterVolumeSpecName: "dshm") pod "ccb9097d-7605-4e8f-acf9-ca75f4ecc720" (UID: "ccb9097d-7605-4e8f-acf9-ca75f4ecc720"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 17 18:33:37.839320 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:33:37.839221    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/ccb9097d-7605-4e8f-acf9-ca75f4ecc720-kube-api-access-vplss" (OuterVolumeSpecName: "kube-api-access-vplss") pod "ccb9097d-7605-4e8f-acf9-ca75f4ecc720" (UID: "ccb9097d-7605-4e8f-acf9-ca75f4ecc720"). InnerVolumeSpecName "kube-api-access-vplss". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 17 18:33:37.839320 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:33:37.839307    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/ccb9097d-7605-4e8f-acf9-ca75f4ecc720-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "ccb9097d-7605-4e8f-acf9-ca75f4ecc720" (UID: "ccb9097d-7605-4e8f-acf9-ca75f4ecc720"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 17 18:33:37.848579 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:33:37.848545    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ccb9097d-7605-4e8f-acf9-ca75f4ecc720-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "ccb9097d-7605-4e8f-acf9-ca75f4ecc720" (UID: "ccb9097d-7605-4e8f-acf9-ca75f4ecc720"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 17 18:33:37.860020 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:33:37.859987    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ccb9097d-7605-4e8f-acf9-ca75f4ecc720-kserve-provision-location" (OuterVolumeSpecName: "kserve-provision-location") pod "ccb9097d-7605-4e8f-acf9-ca75f4ecc720" (UID: "ccb9097d-7605-4e8f-acf9-ca75f4ecc720"). InnerVolumeSpecName "kserve-provision-location". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 17 18:33:37.937572 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:33:37.937535    2575 reconciler_common.go:299] "Volume detached for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/ccb9097d-7605-4e8f-acf9-ca75f4ecc720-kserve-provision-location\") on node \"ip-10-0-143-74.ec2.internal\" DevicePath \"\""
Sep 17 18:33:37.937572 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:33:37.937567    2575 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/ccb9097d-7605-4e8f-acf9-ca75f4ecc720-tls-certs\") on node \"ip-10-0-143-74.ec2.internal\" DevicePath \"\""
Sep 17 18:33:37.937572 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:33:37.937578    2575 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/ccb9097d-7605-4e8f-acf9-ca75f4ecc720-model-cache\") on node \"ip-10-0-143-74.ec2.internal\" DevicePath \"\""
Sep 17 18:33:37.937572 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:33:37.937585    2575 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/ccb9097d-7605-4e8f-acf9-ca75f4ecc720-home\") on node \"ip-10-0-143-74.ec2.internal\" DevicePath \"\""
Sep 17 18:33:37.937572 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:33:37.937595    2575 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-vplss\" (UniqueName: \"kubernetes.io/projected/ccb9097d-7605-4e8f-acf9-ca75f4ecc720-kube-api-access-vplss\") on node \"ip-10-0-143-74.ec2.internal\" DevicePath \"\""
Sep 17 18:33:37.937893 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:33:37.937604    2575 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/ccb9097d-7605-4e8f-acf9-ca75f4ecc720-dshm\") on node \"ip-10-0-143-74.ec2.internal\" DevicePath \"\""
Sep 17 18:33:37.937893 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:33:37.937612    2575 reconciler_common.go:299] "Volume detached for volume \"cabundle-cert\" (UniqueName: \"kubernetes.io/configmap/ccb9097d-7605-4e8f-acf9-ca75f4ecc720-cabundle-cert\") on node \"ip-10-0-143-74.ec2.internal\" DevicePath \"\""
Sep 17 18:33:37.937893 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:33:37.937620    2575 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/ccb9097d-7605-4e8f-acf9-ca75f4ecc720-tmp-dir\") on node \"ip-10-0-143-74.ec2.internal\" DevicePath \"\""
Sep 17 18:33:38.433830 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:33:38.433790    2575 generic.go:356] "Generic (PLEG): container finished" podID="ccb9097d-7605-4e8f-acf9-ca75f4ecc720" containerID="5532852c70a3c48177bec8d65d09c9b666c75688741e13b012d9f52b1ec865c4" exitCode=0
Sep 17 18:33:38.434246 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:33:38.433848    2575 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-test-llm-inference-service-d73d44f4/custom-route-timeout-pd-test-kserve-prefill-549595b7df-qxqsh"
Sep 17 18:33:38.434246 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:33:38.433848    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-test-llm-inference-service-d73d44f4/custom-route-timeout-pd-test-kserve-prefill-549595b7df-qxqsh" event={"ID":"ccb9097d-7605-4e8f-acf9-ca75f4ecc720","Type":"ContainerDied","Data":"5532852c70a3c48177bec8d65d09c9b666c75688741e13b012d9f52b1ec865c4"}
Sep 17 18:33:38.434246 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:33:38.433966    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-test-llm-inference-service-d73d44f4/custom-route-timeout-pd-test-kserve-prefill-549595b7df-qxqsh" event={"ID":"ccb9097d-7605-4e8f-acf9-ca75f4ecc720","Type":"ContainerDied","Data":"e75f4e731672e42b312cde6450f654a6bb992f46e0e30268bd22e1655390316f"}
Sep 17 18:33:38.434246 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:33:38.433996    2575 scope.go:117] "RemoveContainer" containerID="5532852c70a3c48177bec8d65d09c9b666c75688741e13b012d9f52b1ec865c4"
Sep 17 18:33:38.445058 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:33:38.444962    2575 scope.go:117] "RemoveContainer" containerID="2074abe8b1662eea5f7ba63509827b3a0c5dd390cac6f0727e0c05b89706bc32"
Sep 17 18:33:38.453460 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:33:38.453430    2575 scope.go:117] "RemoveContainer" containerID="5532852c70a3c48177bec8d65d09c9b666c75688741e13b012d9f52b1ec865c4"
Sep 17 18:33:38.453758 ip-10-0-143-74 kubenswrapper[2575]: E0917 18:33:38.453739    2575 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"5532852c70a3c48177bec8d65d09c9b666c75688741e13b012d9f52b1ec865c4\": container with ID starting with 5532852c70a3c48177bec8d65d09c9b666c75688741e13b012d9f52b1ec865c4 not found: ID does not exist" containerID="5532852c70a3c48177bec8d65d09c9b666c75688741e13b012d9f52b1ec865c4"
Sep 17 18:33:38.453808 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:33:38.453767    2575 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"5532852c70a3c48177bec8d65d09c9b666c75688741e13b012d9f52b1ec865c4"} err="failed to get container status \"5532852c70a3c48177bec8d65d09c9b666c75688741e13b012d9f52b1ec865c4\": rpc error: code = NotFound desc = could not find container \"5532852c70a3c48177bec8d65d09c9b666c75688741e13b012d9f52b1ec865c4\": container with ID starting with 5532852c70a3c48177bec8d65d09c9b666c75688741e13b012d9f52b1ec865c4 not found: ID does not exist"
Sep 17 18:33:38.453808 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:33:38.453784    2575 scope.go:117] "RemoveContainer" containerID="2074abe8b1662eea5f7ba63509827b3a0c5dd390cac6f0727e0c05b89706bc32"
Sep 17 18:33:38.454018 ip-10-0-143-74 kubenswrapper[2575]: E0917 18:33:38.454000    2575 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"2074abe8b1662eea5f7ba63509827b3a0c5dd390cac6f0727e0c05b89706bc32\": container with ID starting with 2074abe8b1662eea5f7ba63509827b3a0c5dd390cac6f0727e0c05b89706bc32 not found: ID does not exist" containerID="2074abe8b1662eea5f7ba63509827b3a0c5dd390cac6f0727e0c05b89706bc32"
Sep 17 18:33:38.454067 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:33:38.454024    2575 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"2074abe8b1662eea5f7ba63509827b3a0c5dd390cac6f0727e0c05b89706bc32"} err="failed to get container status \"2074abe8b1662eea5f7ba63509827b3a0c5dd390cac6f0727e0c05b89706bc32\": rpc error: code = NotFound desc = could not find container \"2074abe8b1662eea5f7ba63509827b3a0c5dd390cac6f0727e0c05b89706bc32\": container with ID starting with 2074abe8b1662eea5f7ba63509827b3a0c5dd390cac6f0727e0c05b89706bc32 not found: ID does not exist"
Sep 17 18:33:38.460283 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:33:38.460258    2575 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["e2e-test-llm-inference-service-d73d44f4/custom-route-timeout-pd-test-kserve-prefill-549595b7df-qxqsh"]
Sep 17 18:33:38.465259 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:33:38.465237    2575 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["e2e-test-llm-inference-service-d73d44f4/custom-route-timeout-pd-test-kserve-prefill-549595b7df-qxqsh"]
Sep 17 18:33:38.968011 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:33:38.967979    2575 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="ccb9097d-7605-4e8f-acf9-ca75f4ecc720" path="/var/lib/kubelet/pods/ccb9097d-7605-4e8f-acf9-ca75f4ecc720/volumes"
Sep 17 18:33:41.437597 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:33:41.437563    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-djvzt_1d07e68d-9943-420f-a6a0-5f477792fabd/konnectivity-agent/0.log" limit=1024
Sep 17 18:34:12.237913 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:34:12.237880    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-djvzt_1d07e68d-9943-420f-a6a0-5f477792fabd/konnectivity-agent/0.log" limit=1024
Sep 17 18:34:12.329910 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:34:12.329874    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-djvzt_1d07e68d-9943-420f-a6a0-5f477792fabd/konnectivity-agent/0.log" limit=1024
Sep 17 18:34:12.518952 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:34:12.518870    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-djvzt_1d07e68d-9943-420f-a6a0-5f477792fabd/konnectivity-agent/0.log" limit=1024
Sep 17 18:34:12.806971 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:34:12.806883    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-djvzt_1d07e68d-9943-420f-a6a0-5f477792fabd/konnectivity-agent/0.log" limit=1024
Sep 17 18:34:16.044814 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:34:16.044774    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-djvzt_1d07e68d-9943-420f-a6a0-5f477792fabd/konnectivity-agent/0.log" limit=1024
Sep 17 18:34:19.116077 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:34:19.116018    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-djvzt_1d07e68d-9943-420f-a6a0-5f477792fabd/konnectivity-agent/0.log" limit=1024
Sep 17 18:34:25.230343 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:34:25.230302    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-djvzt_1d07e68d-9943-420f-a6a0-5f477792fabd/konnectivity-agent/0.log" limit=1024
Sep 17 18:34:25.331204 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:34:25.331172    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-djvzt_1d07e68d-9943-420f-a6a0-5f477792fabd/konnectivity-agent/0.log" limit=1024
Sep 17 18:34:38.158322 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:34:38.158232    2575 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["e2e-test-llm-inference-service-b5dd93e6/router-with-refs-pd-test-kserve-6757c56889-fpdpb"]
Sep 17 18:34:38.158910 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:34:38.158816    2575 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ccb9097d-7605-4e8f-acf9-ca75f4ecc720" containerName="main"
Sep 17 18:34:38.158910 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:34:38.158836    2575 state_mem.go:107] "Deleted CPUSet assignment" podUID="ccb9097d-7605-4e8f-acf9-ca75f4ecc720" containerName="main"
Sep 17 18:34:38.158910 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:34:38.158866    2575 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ccb9097d-7605-4e8f-acf9-ca75f4ecc720" containerName="storage-initializer"
Sep 17 18:34:38.158910 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:34:38.158875    2575 state_mem.go:107] "Deleted CPUSet assignment" podUID="ccb9097d-7605-4e8f-acf9-ca75f4ecc720" containerName="storage-initializer"
Sep 17 18:34:38.159133 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:34:38.158986    2575 memory_manager.go:368] "RemoveStaleState removing state" podUID="ccb9097d-7605-4e8f-acf9-ca75f4ecc720" containerName="main"
Sep 17 18:34:38.162537 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:34:38.162511    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-test-llm-inference-service-b5dd93e6/router-with-refs-pd-test-kserve-6757c56889-fpdpb"
Sep 17 18:34:38.165347 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:34:38.165324    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-test-llm-inference-service-b5dd93e6\"/\"seaweedfs-s3-creds\""
Sep 17 18:34:38.165528 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:34:38.165324    2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-test-llm-inference-service-b5dd93e6\"/\"openshift-service-ca.crt\""
Sep 17 18:34:38.165528 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:34:38.165369    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-test-llm-inference-service-b5dd93e6\"/\"default-dockercfg-bwhqj\""
Sep 17 18:34:38.165528 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:34:38.165436    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-test-llm-inference-service-b5dd93e6\"/\"router-with-refs-pd-test-kserve-self-signed-certs\""
Sep 17 18:34:38.165795 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:34:38.165772    2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-test-llm-inference-service-b5dd93e6\"/\"kube-root-ca.crt\""
Sep 17 18:34:38.165941 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:34:38.165796    2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-test-llm-inference-service-b5dd93e6\"/\"odh-kserve-custom-ca-bundle\""
Sep 17 18:34:38.166042 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:34:38.166027    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-test-llm-inference-service-b5dd93e6\"/\"router-with-refs-pd-test-kserve-dockercfg-zq6hb\""
Sep 17 18:34:38.179561 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:34:38.179524    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["e2e-test-llm-inference-service-b5dd93e6/router-with-refs-pd-test-kserve-6757c56889-fpdpb"]
Sep 17 18:34:38.212727 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:34:38.212687    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-djvzt_1d07e68d-9943-420f-a6a0-5f477792fabd/konnectivity-agent/0.log" limit=1024
Sep 17 18:34:38.280603 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:34:38.280562    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/1c9c57cc-6b68-42ac-a52c-fcb64e122231-tls-certs\") pod \"router-with-refs-pd-test-kserve-6757c56889-fpdpb\" (UID: \"1c9c57cc-6b68-42ac-a52c-fcb64e122231\") " pod="e2e-test-llm-inference-service-b5dd93e6/router-with-refs-pd-test-kserve-6757c56889-fpdpb"
Sep 17 18:34:38.280603 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:34:38.280602    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cabundle-cert\" (UniqueName: \"kubernetes.io/configmap/1c9c57cc-6b68-42ac-a52c-fcb64e122231-cabundle-cert\") pod \"router-with-refs-pd-test-kserve-6757c56889-fpdpb\" (UID: \"1c9c57cc-6b68-42ac-a52c-fcb64e122231\") " pod="e2e-test-llm-inference-service-b5dd93e6/router-with-refs-pd-test-kserve-6757c56889-fpdpb"
Sep 17 18:34:38.280865 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:34:38.280635    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/1c9c57cc-6b68-42ac-a52c-fcb64e122231-tmp-dir\") pod \"router-with-refs-pd-test-kserve-6757c56889-fpdpb\" (UID: \"1c9c57cc-6b68-42ac-a52c-fcb64e122231\") " pod="e2e-test-llm-inference-service-b5dd93e6/router-with-refs-pd-test-kserve-6757c56889-fpdpb"
Sep 17 18:34:38.280865 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:34:38.280657    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-42mn8\" (UniqueName: \"kubernetes.io/projected/1c9c57cc-6b68-42ac-a52c-fcb64e122231-kube-api-access-42mn8\") pod \"router-with-refs-pd-test-kserve-6757c56889-fpdpb\" (UID: \"1c9c57cc-6b68-42ac-a52c-fcb64e122231\") " pod="e2e-test-llm-inference-service-b5dd93e6/router-with-refs-pd-test-kserve-6757c56889-fpdpb"
Sep 17 18:34:38.280865 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:34:38.280694    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/1c9c57cc-6b68-42ac-a52c-fcb64e122231-home\") pod \"router-with-refs-pd-test-kserve-6757c56889-fpdpb\" (UID: \"1c9c57cc-6b68-42ac-a52c-fcb64e122231\") " pod="e2e-test-llm-inference-service-b5dd93e6/router-with-refs-pd-test-kserve-6757c56889-fpdpb"
Sep 17 18:34:38.280865 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:34:38.280758    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/1c9c57cc-6b68-42ac-a52c-fcb64e122231-kserve-provision-location\") pod \"router-with-refs-pd-test-kserve-6757c56889-fpdpb\" (UID: \"1c9c57cc-6b68-42ac-a52c-fcb64e122231\") " pod="e2e-test-llm-inference-service-b5dd93e6/router-with-refs-pd-test-kserve-6757c56889-fpdpb"
Sep 17 18:34:38.280865 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:34:38.280835    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/1c9c57cc-6b68-42ac-a52c-fcb64e122231-dshm\") pod \"router-with-refs-pd-test-kserve-6757c56889-fpdpb\" (UID: \"1c9c57cc-6b68-42ac-a52c-fcb64e122231\") " pod="e2e-test-llm-inference-service-b5dd93e6/router-with-refs-pd-test-kserve-6757c56889-fpdpb"
Sep 17 18:34:38.280865 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:34:38.280861    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/1c9c57cc-6b68-42ac-a52c-fcb64e122231-model-cache\") pod \"router-with-refs-pd-test-kserve-6757c56889-fpdpb\" (UID: \"1c9c57cc-6b68-42ac-a52c-fcb64e122231\") " pod="e2e-test-llm-inference-service-b5dd93e6/router-with-refs-pd-test-kserve-6757c56889-fpdpb"
Sep 17 18:34:38.382307 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:34:38.382262    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/1c9c57cc-6b68-42ac-a52c-fcb64e122231-dshm\") pod \"router-with-refs-pd-test-kserve-6757c56889-fpdpb\" (UID: \"1c9c57cc-6b68-42ac-a52c-fcb64e122231\") " pod="e2e-test-llm-inference-service-b5dd93e6/router-with-refs-pd-test-kserve-6757c56889-fpdpb"
Sep 17 18:34:38.382307 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:34:38.382307    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/1c9c57cc-6b68-42ac-a52c-fcb64e122231-model-cache\") pod \"router-with-refs-pd-test-kserve-6757c56889-fpdpb\" (UID: \"1c9c57cc-6b68-42ac-a52c-fcb64e122231\") " pod="e2e-test-llm-inference-service-b5dd93e6/router-with-refs-pd-test-kserve-6757c56889-fpdpb"
Sep 17 18:34:38.382611 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:34:38.382427    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/1c9c57cc-6b68-42ac-a52c-fcb64e122231-tls-certs\") pod \"router-with-refs-pd-test-kserve-6757c56889-fpdpb\" (UID: \"1c9c57cc-6b68-42ac-a52c-fcb64e122231\") " pod="e2e-test-llm-inference-service-b5dd93e6/router-with-refs-pd-test-kserve-6757c56889-fpdpb"
Sep 17 18:34:38.382611 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:34:38.382486    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cabundle-cert\" (UniqueName: \"kubernetes.io/configmap/1c9c57cc-6b68-42ac-a52c-fcb64e122231-cabundle-cert\") pod \"router-with-refs-pd-test-kserve-6757c56889-fpdpb\" (UID: \"1c9c57cc-6b68-42ac-a52c-fcb64e122231\") " pod="e2e-test-llm-inference-service-b5dd93e6/router-with-refs-pd-test-kserve-6757c56889-fpdpb"
Sep 17 18:34:38.382611 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:34:38.382521    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/1c9c57cc-6b68-42ac-a52c-fcb64e122231-tmp-dir\") pod \"router-with-refs-pd-test-kserve-6757c56889-fpdpb\" (UID: \"1c9c57cc-6b68-42ac-a52c-fcb64e122231\") " pod="e2e-test-llm-inference-service-b5dd93e6/router-with-refs-pd-test-kserve-6757c56889-fpdpb"
Sep 17 18:34:38.382611 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:34:38.382539    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-42mn8\" (UniqueName: \"kubernetes.io/projected/1c9c57cc-6b68-42ac-a52c-fcb64e122231-kube-api-access-42mn8\") pod \"router-with-refs-pd-test-kserve-6757c56889-fpdpb\" (UID: \"1c9c57cc-6b68-42ac-a52c-fcb64e122231\") " pod="e2e-test-llm-inference-service-b5dd93e6/router-with-refs-pd-test-kserve-6757c56889-fpdpb"
Sep 17 18:34:38.382611 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:34:38.382580    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/1c9c57cc-6b68-42ac-a52c-fcb64e122231-home\") pod \"router-with-refs-pd-test-kserve-6757c56889-fpdpb\" (UID: \"1c9c57cc-6b68-42ac-a52c-fcb64e122231\") " pod="e2e-test-llm-inference-service-b5dd93e6/router-with-refs-pd-test-kserve-6757c56889-fpdpb"
Sep 17 18:34:38.382904 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:34:38.382641    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/1c9c57cc-6b68-42ac-a52c-fcb64e122231-kserve-provision-location\") pod \"router-with-refs-pd-test-kserve-6757c56889-fpdpb\" (UID: \"1c9c57cc-6b68-42ac-a52c-fcb64e122231\") " pod="e2e-test-llm-inference-service-b5dd93e6/router-with-refs-pd-test-kserve-6757c56889-fpdpb"
Sep 17 18:34:38.382904 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:34:38.382767    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/1c9c57cc-6b68-42ac-a52c-fcb64e122231-model-cache\") pod \"router-with-refs-pd-test-kserve-6757c56889-fpdpb\" (UID: \"1c9c57cc-6b68-42ac-a52c-fcb64e122231\") " pod="e2e-test-llm-inference-service-b5dd93e6/router-with-refs-pd-test-kserve-6757c56889-fpdpb"
Sep 17 18:34:38.383029 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:34:38.383007    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/1c9c57cc-6b68-42ac-a52c-fcb64e122231-home\") pod \"router-with-refs-pd-test-kserve-6757c56889-fpdpb\" (UID: \"1c9c57cc-6b68-42ac-a52c-fcb64e122231\") " pod="e2e-test-llm-inference-service-b5dd93e6/router-with-refs-pd-test-kserve-6757c56889-fpdpb"
Sep 17 18:34:38.383075 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:34:38.383021    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/1c9c57cc-6b68-42ac-a52c-fcb64e122231-kserve-provision-location\") pod \"router-with-refs-pd-test-kserve-6757c56889-fpdpb\" (UID: \"1c9c57cc-6b68-42ac-a52c-fcb64e122231\") " pod="e2e-test-llm-inference-service-b5dd93e6/router-with-refs-pd-test-kserve-6757c56889-fpdpb"
Sep 17 18:34:38.383235 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:34:38.383217    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/1c9c57cc-6b68-42ac-a52c-fcb64e122231-tmp-dir\") pod \"router-with-refs-pd-test-kserve-6757c56889-fpdpb\" (UID: \"1c9c57cc-6b68-42ac-a52c-fcb64e122231\") " pod="e2e-test-llm-inference-service-b5dd93e6/router-with-refs-pd-test-kserve-6757c56889-fpdpb"
Sep 17 18:34:38.383274 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:34:38.383254    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cabundle-cert\" (UniqueName: \"kubernetes.io/configmap/1c9c57cc-6b68-42ac-a52c-fcb64e122231-cabundle-cert\") pod \"router-with-refs-pd-test-kserve-6757c56889-fpdpb\" (UID: \"1c9c57cc-6b68-42ac-a52c-fcb64e122231\") " pod="e2e-test-llm-inference-service-b5dd93e6/router-with-refs-pd-test-kserve-6757c56889-fpdpb"
Sep 17 18:34:38.384711 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:34:38.384690    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/1c9c57cc-6b68-42ac-a52c-fcb64e122231-dshm\") pod \"router-with-refs-pd-test-kserve-6757c56889-fpdpb\" (UID: \"1c9c57cc-6b68-42ac-a52c-fcb64e122231\") " pod="e2e-test-llm-inference-service-b5dd93e6/router-with-refs-pd-test-kserve-6757c56889-fpdpb"
Sep 17 18:34:38.384960 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:34:38.384944    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/1c9c57cc-6b68-42ac-a52c-fcb64e122231-tls-certs\") pod \"router-with-refs-pd-test-kserve-6757c56889-fpdpb\" (UID: \"1c9c57cc-6b68-42ac-a52c-fcb64e122231\") " pod="e2e-test-llm-inference-service-b5dd93e6/router-with-refs-pd-test-kserve-6757c56889-fpdpb"
Sep 17 18:34:38.391203 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:34:38.391181    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-42mn8\" (UniqueName: \"kubernetes.io/projected/1c9c57cc-6b68-42ac-a52c-fcb64e122231-kube-api-access-42mn8\") pod \"router-with-refs-pd-test-kserve-6757c56889-fpdpb\" (UID: \"1c9c57cc-6b68-42ac-a52c-fcb64e122231\") " pod="e2e-test-llm-inference-service-b5dd93e6/router-with-refs-pd-test-kserve-6757c56889-fpdpb"
Sep 17 18:34:38.474034 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:34:38.473944    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-test-llm-inference-service-b5dd93e6/router-with-refs-pd-test-kserve-6757c56889-fpdpb"
Sep 17 18:34:38.620506 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:34:38.620466    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-djvzt_1d07e68d-9943-420f-a6a0-5f477792fabd/konnectivity-agent/0.log" limit=1024
Sep 17 18:34:38.702948 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:34:38.702907    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-djvzt_1d07e68d-9943-420f-a6a0-5f477792fabd/konnectivity-agent/0.log" limit=1024
Sep 17 18:34:38.759081 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:34:38.759054    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["e2e-test-llm-inference-service-b5dd93e6/router-with-refs-pd-test-kserve-6757c56889-fpdpb"]
Sep 17 18:34:38.760769 ip-10-0-143-74 kubenswrapper[2575]: W0917 18:34:38.760735    2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1c9c57cc_6b68_42ac_a52c_fcb64e122231.slice/crio-2f006142ed366e298657fae5b0f85bdfd0b8058b2c0e4737b896d2046a4ea775 WatchSource:0}: Error finding container 2f006142ed366e298657fae5b0f85bdfd0b8058b2c0e4737b896d2046a4ea775: Status 404 returned error can't find the container with id 2f006142ed366e298657fae5b0f85bdfd0b8058b2c0e4737b896d2046a4ea775
Sep 17 18:34:39.768128 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:34:39.768092    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-test-llm-inference-service-b5dd93e6/router-with-refs-pd-test-kserve-6757c56889-fpdpb" event={"ID":"1c9c57cc-6b68-42ac-a52c-fcb64e122231","Type":"ContainerStarted","Data":"3de708cc05cc3ec1e34c4c0bd04da0c21dee854833cade8c01c7487eb88e9df6"}
Sep 17 18:34:39.768540 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:34:39.768146    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-test-llm-inference-service-b5dd93e6/router-with-refs-pd-test-kserve-6757c56889-fpdpb" event={"ID":"1c9c57cc-6b68-42ac-a52c-fcb64e122231","Type":"ContainerStarted","Data":"2f006142ed366e298657fae5b0f85bdfd0b8058b2c0e4737b896d2046a4ea775"}
Sep 17 18:34:39.768540 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:34:39.768183    2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-test-llm-inference-service-b5dd93e6/router-with-refs-pd-test-kserve-6757c56889-fpdpb"
Sep 17 18:34:40.777326 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:34:40.777271    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-test-llm-inference-service-b5dd93e6/router-with-refs-pd-test-kserve-6757c56889-fpdpb" event={"ID":"1c9c57cc-6b68-42ac-a52c-fcb64e122231","Type":"ContainerStarted","Data":"1c3507a8c08c663bcfa4b0f6d1e526b35d0a9051abc492a0944b99b1ec369a2c"}
Sep 17 18:34:42.790556 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:34:42.790519    2575 generic.go:356] "Generic (PLEG): container finished" podID="1c9c57cc-6b68-42ac-a52c-fcb64e122231" containerID="1c3507a8c08c663bcfa4b0f6d1e526b35d0a9051abc492a0944b99b1ec369a2c" exitCode=0
Sep 17 18:34:42.791049 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:34:42.790594    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-test-llm-inference-service-b5dd93e6/router-with-refs-pd-test-kserve-6757c56889-fpdpb" event={"ID":"1c9c57cc-6b68-42ac-a52c-fcb64e122231","Type":"ContainerDied","Data":"1c3507a8c08c663bcfa4b0f6d1e526b35d0a9051abc492a0944b99b1ec369a2c"}
Sep 17 18:34:43.798595 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:34:43.798554    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-test-llm-inference-service-b5dd93e6/router-with-refs-pd-test-kserve-6757c56889-fpdpb" event={"ID":"1c9c57cc-6b68-42ac-a52c-fcb64e122231","Type":"ContainerStarted","Data":"926bc684892e23e964c42a699ef0c256ca4ee274bab4b16a3dc824e1528f892b"}
Sep 17 18:34:43.845135 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:34:43.845057    2575 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="e2e-test-llm-inference-service-b5dd93e6/router-with-refs-pd-test-kserve-6757c56889-fpdpb" podStartSLOduration=3.517127935 podStartE2EDuration="5.845032954s" totalImagesPullingTime="0s" totalInitContainerRuntime="2.327905019s" isStatefulPod=false podCreationTimestamp="2026-09-17 18:34:38 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-17 18:34:43.84198505 +0000 UTC m=+3603.472315025" watchObservedRunningTime="2026-09-17 18:34:43.845032954 +0000 UTC m=+3603.475362932"
Sep 17 18:34:48.474925 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:34:48.474878    2575 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="e2e-test-llm-inference-service-b5dd93e6/router-with-refs-pd-test-kserve-6757c56889-fpdpb"
Sep 17 18:34:48.475405 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:34:48.474958    2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-test-llm-inference-service-b5dd93e6/router-with-refs-pd-test-kserve-6757c56889-fpdpb"
Sep 17 18:34:48.476495 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:34:48.476427    2575 prober.go:120] "Probe failed" probeType="Startup" pod="e2e-test-llm-inference-service-b5dd93e6/router-with-refs-pd-test-kserve-6757c56889-fpdpb" podUID="1c9c57cc-6b68-42ac-a52c-fcb64e122231" containerName="main" probeResult="failure" output="Get \"https://10.133.0.63:8001/health\": dial tcp 10.133.0.63:8001: connect: connection refused"
Sep 17 18:34:48.487809 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:34:48.487782    2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-test-llm-inference-service-b5dd93e6/router-with-refs-pd-test-kserve-6757c56889-fpdpb"
Sep 17 18:34:58.474637 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:34:58.474589    2575 prober.go:120] "Probe failed" probeType="Startup" pod="e2e-test-llm-inference-service-b5dd93e6/router-with-refs-pd-test-kserve-6757c56889-fpdpb" podUID="1c9c57cc-6b68-42ac-a52c-fcb64e122231" containerName="main" probeResult="failure" output="Get \"https://10.133.0.63:8001/health\": dial tcp 10.133.0.63:8001: connect: connection refused"
Sep 17 18:35:08.474343 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:35:08.474300    2575 prober.go:120] "Probe failed" probeType="Startup" pod="e2e-test-llm-inference-service-b5dd93e6/router-with-refs-pd-test-kserve-6757c56889-fpdpb" podUID="1c9c57cc-6b68-42ac-a52c-fcb64e122231" containerName="main" probeResult="failure" output="Get \"https://10.133.0.63:8001/health\": dial tcp 10.133.0.63:8001: connect: connection refused"
Sep 17 18:35:18.474532 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:35:18.474486    2575 prober.go:120] "Probe failed" probeType="Startup" pod="e2e-test-llm-inference-service-b5dd93e6/router-with-refs-pd-test-kserve-6757c56889-fpdpb" podUID="1c9c57cc-6b68-42ac-a52c-fcb64e122231" containerName="main" probeResult="failure" output="Get \"https://10.133.0.63:8001/health\": dial tcp 10.133.0.63:8001: connect: connection refused"
Sep 17 18:35:28.066671 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:35:28.066636    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-djvzt_1d07e68d-9943-420f-a6a0-5f477792fabd/konnectivity-agent/0.log" limit=1024
Sep 17 18:35:28.474405 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:35:28.474358    2575 prober.go:120] "Probe failed" probeType="Startup" pod="e2e-test-llm-inference-service-b5dd93e6/router-with-refs-pd-test-kserve-6757c56889-fpdpb" podUID="1c9c57cc-6b68-42ac-a52c-fcb64e122231" containerName="main" probeResult="failure" output="Get \"https://10.133.0.63:8001/health\": dial tcp 10.133.0.63:8001: connect: connection refused"
Sep 17 18:35:38.475291 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:35:38.475251    2575 prober.go:120] "Probe failed" probeType="Startup" pod="e2e-test-llm-inference-service-b5dd93e6/router-with-refs-pd-test-kserve-6757c56889-fpdpb" podUID="1c9c57cc-6b68-42ac-a52c-fcb64e122231" containerName="main" probeResult="failure" output="Get \"https://10.133.0.63:8001/health\": dial tcp 10.133.0.63:8001: connect: connection refused"
Sep 17 18:35:48.328725 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:35:48.328680    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-djvzt_1d07e68d-9943-420f-a6a0-5f477792fabd/konnectivity-agent/0.log" limit=1024
Sep 17 18:35:48.429620 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:35:48.429574    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-djvzt_1d07e68d-9943-420f-a6a0-5f477792fabd/konnectivity-agent/0.log" limit=1024
Sep 17 18:35:48.475359 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:35:48.475313    2575 prober.go:120] "Probe failed" probeType="Startup" pod="e2e-test-llm-inference-service-b5dd93e6/router-with-refs-pd-test-kserve-6757c56889-fpdpb" podUID="1c9c57cc-6b68-42ac-a52c-fcb64e122231" containerName="main" probeResult="failure" output="Get \"https://10.133.0.63:8001/health\": dial tcp 10.133.0.63:8001: connect: connection refused"
Sep 17 18:35:48.534024 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:35:48.533982    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-djvzt_1d07e68d-9943-420f-a6a0-5f477792fabd/konnectivity-agent/0.log" limit=1024
Sep 17 18:35:48.612383 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:35:48.612288    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-djvzt_1d07e68d-9943-420f-a6a0-5f477792fabd/konnectivity-agent/0.log" limit=1024
Sep 17 18:35:48.673260 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:35:48.673217    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-djvzt_1d07e68d-9943-420f-a6a0-5f477792fabd/konnectivity-agent/0.log" limit=1024
Sep 17 18:35:51.042378 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:35:51.042343    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-djvzt_1d07e68d-9943-420f-a6a0-5f477792fabd/konnectivity-agent/0.log" limit=1024
Sep 17 18:35:51.144338 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:35:51.144298    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-djvzt_1d07e68d-9943-420f-a6a0-5f477792fabd/konnectivity-agent/0.log" limit=1024
Sep 17 18:35:51.768324 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:35:51.768291    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-djvzt_1d07e68d-9943-420f-a6a0-5f477792fabd/konnectivity-agent/0.log" limit=1024
Sep 17 18:35:54.480947 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:35:54.480905    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-djvzt_1d07e68d-9943-420f-a6a0-5f477792fabd/konnectivity-agent/0.log" limit=1024
Sep 17 18:35:54.559304 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:35:54.559272    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-djvzt_1d07e68d-9943-420f-a6a0-5f477792fabd/konnectivity-agent/0.log" limit=1024
Sep 17 18:35:58.474918 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:35:58.474864    2575 prober.go:120] "Probe failed" probeType="Startup" pod="e2e-test-llm-inference-service-b5dd93e6/router-with-refs-pd-test-kserve-6757c56889-fpdpb" podUID="1c9c57cc-6b68-42ac-a52c-fcb64e122231" containerName="main" probeResult="failure" output="Get \"https://10.133.0.63:8001/health\": dial tcp 10.133.0.63:8001: connect: connection refused"
Sep 17 18:36:08.474496 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:36:08.474408    2575 prober.go:120] "Probe failed" probeType="Startup" pod="e2e-test-llm-inference-service-b5dd93e6/router-with-refs-pd-test-kserve-6757c56889-fpdpb" podUID="1c9c57cc-6b68-42ac-a52c-fcb64e122231" containerName="main" probeResult="failure" output="Get \"https://10.133.0.63:8001/health\": dial tcp 10.133.0.63:8001: connect: connection refused"
Sep 17 18:36:18.474983 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:36:18.474922    2575 prober.go:120] "Probe failed" probeType="Startup" pod="e2e-test-llm-inference-service-b5dd93e6/router-with-refs-pd-test-kserve-6757c56889-fpdpb" podUID="1c9c57cc-6b68-42ac-a52c-fcb64e122231" containerName="main" probeResult="failure" output="Get \"https://10.133.0.63:8001/health\": dial tcp 10.133.0.63:8001: connect: connection refused"
Sep 17 18:36:21.452834 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:36:21.452788    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-djvzt_1d07e68d-9943-420f-a6a0-5f477792fabd/konnectivity-agent/0.log" limit=1024
Sep 17 18:36:28.475029 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:36:28.474979    2575 prober.go:120] "Probe failed" probeType="Startup" pod="e2e-test-llm-inference-service-b5dd93e6/router-with-refs-pd-test-kserve-6757c56889-fpdpb" podUID="1c9c57cc-6b68-42ac-a52c-fcb64e122231" containerName="main" probeResult="failure" output="Get \"https://10.133.0.63:8001/health\": dial tcp 10.133.0.63:8001: connect: connection refused"
Sep 17 18:36:38.489627 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:36:38.489595    2575 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="e2e-test-llm-inference-service-b5dd93e6/router-with-refs-pd-test-kserve-6757c56889-fpdpb"
Sep 17 18:36:38.505057 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:36:38.505033    2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-test-llm-inference-service-b5dd93e6/router-with-refs-pd-test-kserve-6757c56889-fpdpb"
Sep 17 18:36:57.216178 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:36:57.216144    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-djvzt_1d07e68d-9943-420f-a6a0-5f477792fabd/konnectivity-agent/0.log" limit=1024
Sep 17 18:36:57.231971 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:36:57.231935    2575 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["e2e-test-llm-inference-service-b5dd93e6/router-with-refs-pd-test-kserve-6757c56889-fpdpb"]
Sep 17 18:36:57.285405 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:36:57.285371    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-djvzt_1d07e68d-9943-420f-a6a0-5f477792fabd/konnectivity-agent/0.log" limit=1024
Sep 17 18:37:12.254568 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:37:12.254509    2575 kuberuntime_container.go:873] "Killing container with a grace period" pod="e2e-test-llm-inference-service-b5dd93e6/router-with-refs-pd-test-kserve-6757c56889-fpdpb" podUID="1c9c57cc-6b68-42ac-a52c-fcb64e122231" containerName="main" containerID="cri-o://926bc684892e23e964c42a699ef0c256ca4ee274bab4b16a3dc824e1528f892b" gracePeriod=45
Sep 17 18:37:12.871431 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:37:12.871383    2575 kuberuntime_container.go:873] "Killing container with a grace period" pod="e2e-test-llm-inference-service-b5dd93e6/router-with-refs-pd-test-kserve-6757c56889-fpdpb" podUID="1c9c57cc-6b68-42ac-a52c-fcb64e122231" containerName="llm-d-routing-sidecar" containerID="cri-o://3de708cc05cc3ec1e34c4c0bd04da0c21dee854833cade8c01c7487eb88e9df6" gracePeriod=45
Sep 17 18:37:13.122219 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:37:13.122194    2575 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-test-llm-inference-service-b5dd93e6/router-with-refs-pd-test-kserve-6757c56889-fpdpb"
Sep 17 18:37:13.199325 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:37:13.199293    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/1c9c57cc-6b68-42ac-a52c-fcb64e122231-dshm\") pod \"1c9c57cc-6b68-42ac-a52c-fcb64e122231\" (UID: \"1c9c57cc-6b68-42ac-a52c-fcb64e122231\") "
Sep 17 18:37:13.199506 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:37:13.199345    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-42mn8\" (UniqueName: \"kubernetes.io/projected/1c9c57cc-6b68-42ac-a52c-fcb64e122231-kube-api-access-42mn8\") pod \"1c9c57cc-6b68-42ac-a52c-fcb64e122231\" (UID: \"1c9c57cc-6b68-42ac-a52c-fcb64e122231\") "
Sep 17 18:37:13.199506 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:37:13.199361    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/1c9c57cc-6b68-42ac-a52c-fcb64e122231-model-cache\") pod \"1c9c57cc-6b68-42ac-a52c-fcb64e122231\" (UID: \"1c9c57cc-6b68-42ac-a52c-fcb64e122231\") "
Sep 17 18:37:13.199506 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:37:13.199388    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"cabundle-cert\" (UniqueName: \"kubernetes.io/configmap/1c9c57cc-6b68-42ac-a52c-fcb64e122231-cabundle-cert\") pod \"1c9c57cc-6b68-42ac-a52c-fcb64e122231\" (UID: \"1c9c57cc-6b68-42ac-a52c-fcb64e122231\") "
Sep 17 18:37:13.199506 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:37:13.199407    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/1c9c57cc-6b68-42ac-a52c-fcb64e122231-tls-certs\") pod \"1c9c57cc-6b68-42ac-a52c-fcb64e122231\" (UID: \"1c9c57cc-6b68-42ac-a52c-fcb64e122231\") "
Sep 17 18:37:13.199506 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:37:13.199422    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/1c9c57cc-6b68-42ac-a52c-fcb64e122231-home\") pod \"1c9c57cc-6b68-42ac-a52c-fcb64e122231\" (UID: \"1c9c57cc-6b68-42ac-a52c-fcb64e122231\") "
Sep 17 18:37:13.199506 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:37:13.199490    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/1c9c57cc-6b68-42ac-a52c-fcb64e122231-tmp-dir\") pod \"1c9c57cc-6b68-42ac-a52c-fcb64e122231\" (UID: \"1c9c57cc-6b68-42ac-a52c-fcb64e122231\") "
Sep 17 18:37:13.199830 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:37:13.199534    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/1c9c57cc-6b68-42ac-a52c-fcb64e122231-kserve-provision-location\") pod \"1c9c57cc-6b68-42ac-a52c-fcb64e122231\" (UID: \"1c9c57cc-6b68-42ac-a52c-fcb64e122231\") "
Sep 17 18:37:13.199977 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:37:13.199927    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/1c9c57cc-6b68-42ac-a52c-fcb64e122231-cabundle-cert" (OuterVolumeSpecName: "cabundle-cert") pod "1c9c57cc-6b68-42ac-a52c-fcb64e122231" (UID: "1c9c57cc-6b68-42ac-a52c-fcb64e122231"). InnerVolumeSpecName "cabundle-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 17 18:37:13.200230 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:37:13.200198    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/1c9c57cc-6b68-42ac-a52c-fcb64e122231-home" (OuterVolumeSpecName: "home") pod "1c9c57cc-6b68-42ac-a52c-fcb64e122231" (UID: "1c9c57cc-6b68-42ac-a52c-fcb64e122231"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 17 18:37:13.200305 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:37:13.200288    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/1c9c57cc-6b68-42ac-a52c-fcb64e122231-model-cache" (OuterVolumeSpecName: "model-cache") pod "1c9c57cc-6b68-42ac-a52c-fcb64e122231" (UID: "1c9c57cc-6b68-42ac-a52c-fcb64e122231"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 17 18:37:13.201736 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:37:13.201706    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/1c9c57cc-6b68-42ac-a52c-fcb64e122231-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "1c9c57cc-6b68-42ac-a52c-fcb64e122231" (UID: "1c9c57cc-6b68-42ac-a52c-fcb64e122231"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 17 18:37:13.201884 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:37:13.201866    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/1c9c57cc-6b68-42ac-a52c-fcb64e122231-dshm" (OuterVolumeSpecName: "dshm") pod "1c9c57cc-6b68-42ac-a52c-fcb64e122231" (UID: "1c9c57cc-6b68-42ac-a52c-fcb64e122231"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 17 18:37:13.201947 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:37:13.201892    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/1c9c57cc-6b68-42ac-a52c-fcb64e122231-kube-api-access-42mn8" (OuterVolumeSpecName: "kube-api-access-42mn8") pod "1c9c57cc-6b68-42ac-a52c-fcb64e122231" (UID: "1c9c57cc-6b68-42ac-a52c-fcb64e122231"). InnerVolumeSpecName "kube-api-access-42mn8". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 17 18:37:13.210656 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:37:13.210628    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/1c9c57cc-6b68-42ac-a52c-fcb64e122231-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "1c9c57cc-6b68-42ac-a52c-fcb64e122231" (UID: "1c9c57cc-6b68-42ac-a52c-fcb64e122231"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 17 18:37:13.223721 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:37:13.223691    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/1c9c57cc-6b68-42ac-a52c-fcb64e122231-kserve-provision-location" (OuterVolumeSpecName: "kserve-provision-location") pod "1c9c57cc-6b68-42ac-a52c-fcb64e122231" (UID: "1c9c57cc-6b68-42ac-a52c-fcb64e122231"). InnerVolumeSpecName "kserve-provision-location". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 17 18:37:13.300636 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:37:13.300546    2575 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/1c9c57cc-6b68-42ac-a52c-fcb64e122231-dshm\") on node \"ip-10-0-143-74.ec2.internal\" DevicePath \"\""
Sep 17 18:37:13.300636 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:37:13.300581    2575 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-42mn8\" (UniqueName: \"kubernetes.io/projected/1c9c57cc-6b68-42ac-a52c-fcb64e122231-kube-api-access-42mn8\") on node \"ip-10-0-143-74.ec2.internal\" DevicePath \"\""
Sep 17 18:37:13.300636 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:37:13.300595    2575 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/1c9c57cc-6b68-42ac-a52c-fcb64e122231-model-cache\") on node \"ip-10-0-143-74.ec2.internal\" DevicePath \"\""
Sep 17 18:37:13.300636 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:37:13.300610    2575 reconciler_common.go:299] "Volume detached for volume \"cabundle-cert\" (UniqueName: \"kubernetes.io/configmap/1c9c57cc-6b68-42ac-a52c-fcb64e122231-cabundle-cert\") on node \"ip-10-0-143-74.ec2.internal\" DevicePath \"\""
Sep 17 18:37:13.300636 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:37:13.300620    2575 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/1c9c57cc-6b68-42ac-a52c-fcb64e122231-tls-certs\") on node \"ip-10-0-143-74.ec2.internal\" DevicePath \"\""
Sep 17 18:37:13.300636 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:37:13.300631    2575 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/1c9c57cc-6b68-42ac-a52c-fcb64e122231-home\") on node \"ip-10-0-143-74.ec2.internal\" DevicePath \"\""
Sep 17 18:37:13.301132 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:37:13.300667    2575 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/1c9c57cc-6b68-42ac-a52c-fcb64e122231-tmp-dir\") on node \"ip-10-0-143-74.ec2.internal\" DevicePath \"\""
Sep 17 18:37:13.301132 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:37:13.300679    2575 reconciler_common.go:299] "Volume detached for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/1c9c57cc-6b68-42ac-a52c-fcb64e122231-kserve-provision-location\") on node \"ip-10-0-143-74.ec2.internal\" DevicePath \"\""
Sep 17 18:37:13.667652 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:37:13.667618    2575 generic.go:356] "Generic (PLEG): container finished" podID="1c9c57cc-6b68-42ac-a52c-fcb64e122231" containerID="926bc684892e23e964c42a699ef0c256ca4ee274bab4b16a3dc824e1528f892b" exitCode=0
Sep 17 18:37:13.667652 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:37:13.667646    2575 generic.go:356] "Generic (PLEG): container finished" podID="1c9c57cc-6b68-42ac-a52c-fcb64e122231" containerID="3de708cc05cc3ec1e34c4c0bd04da0c21dee854833cade8c01c7487eb88e9df6" exitCode=0
Sep 17 18:37:13.667871 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:37:13.667678    2575 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-test-llm-inference-service-b5dd93e6/router-with-refs-pd-test-kserve-6757c56889-fpdpb"
Sep 17 18:37:13.667871 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:37:13.667694    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-test-llm-inference-service-b5dd93e6/router-with-refs-pd-test-kserve-6757c56889-fpdpb" event={"ID":"1c9c57cc-6b68-42ac-a52c-fcb64e122231","Type":"ContainerDied","Data":"926bc684892e23e964c42a699ef0c256ca4ee274bab4b16a3dc824e1528f892b"}
Sep 17 18:37:13.667871 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:37:13.667746    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-test-llm-inference-service-b5dd93e6/router-with-refs-pd-test-kserve-6757c56889-fpdpb" event={"ID":"1c9c57cc-6b68-42ac-a52c-fcb64e122231","Type":"ContainerDied","Data":"3de708cc05cc3ec1e34c4c0bd04da0c21dee854833cade8c01c7487eb88e9df6"}
Sep 17 18:37:13.667871 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:37:13.667765    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-test-llm-inference-service-b5dd93e6/router-with-refs-pd-test-kserve-6757c56889-fpdpb" event={"ID":"1c9c57cc-6b68-42ac-a52c-fcb64e122231","Type":"ContainerDied","Data":"2f006142ed366e298657fae5b0f85bdfd0b8058b2c0e4737b896d2046a4ea775"}
Sep 17 18:37:13.667871 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:37:13.667786    2575 scope.go:117] "RemoveContainer" containerID="926bc684892e23e964c42a699ef0c256ca4ee274bab4b16a3dc824e1528f892b"
Sep 17 18:37:13.676889 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:37:13.676870    2575 scope.go:117] "RemoveContainer" containerID="1c3507a8c08c663bcfa4b0f6d1e526b35d0a9051abc492a0944b99b1ec369a2c"
Sep 17 18:37:13.684900 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:37:13.684882    2575 scope.go:117] "RemoveContainer" containerID="3de708cc05cc3ec1e34c4c0bd04da0c21dee854833cade8c01c7487eb88e9df6"
Sep 17 18:37:13.692978 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:37:13.692956    2575 scope.go:117] "RemoveContainer" containerID="926bc684892e23e964c42a699ef0c256ca4ee274bab4b16a3dc824e1528f892b"
Sep 17 18:37:13.693251 ip-10-0-143-74 kubenswrapper[2575]: E0917 18:37:13.693226    2575 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"926bc684892e23e964c42a699ef0c256ca4ee274bab4b16a3dc824e1528f892b\": container with ID starting with 926bc684892e23e964c42a699ef0c256ca4ee274bab4b16a3dc824e1528f892b not found: ID does not exist" containerID="926bc684892e23e964c42a699ef0c256ca4ee274bab4b16a3dc824e1528f892b"
Sep 17 18:37:13.693334 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:37:13.693257    2575 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"926bc684892e23e964c42a699ef0c256ca4ee274bab4b16a3dc824e1528f892b"} err="failed to get container status \"926bc684892e23e964c42a699ef0c256ca4ee274bab4b16a3dc824e1528f892b\": rpc error: code = NotFound desc = could not find container \"926bc684892e23e964c42a699ef0c256ca4ee274bab4b16a3dc824e1528f892b\": container with ID starting with 926bc684892e23e964c42a699ef0c256ca4ee274bab4b16a3dc824e1528f892b not found: ID does not exist"
Sep 17 18:37:13.693334 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:37:13.693280    2575 scope.go:117] "RemoveContainer" containerID="1c3507a8c08c663bcfa4b0f6d1e526b35d0a9051abc492a0944b99b1ec369a2c"
Sep 17 18:37:13.693569 ip-10-0-143-74 kubenswrapper[2575]: E0917 18:37:13.693546    2575 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"1c3507a8c08c663bcfa4b0f6d1e526b35d0a9051abc492a0944b99b1ec369a2c\": container with ID starting with 1c3507a8c08c663bcfa4b0f6d1e526b35d0a9051abc492a0944b99b1ec369a2c not found: ID does not exist" containerID="1c3507a8c08c663bcfa4b0f6d1e526b35d0a9051abc492a0944b99b1ec369a2c"
Sep 17 18:37:13.693642 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:37:13.693576    2575 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"1c3507a8c08c663bcfa4b0f6d1e526b35d0a9051abc492a0944b99b1ec369a2c"} err="failed to get container status \"1c3507a8c08c663bcfa4b0f6d1e526b35d0a9051abc492a0944b99b1ec369a2c\": rpc error: code = NotFound desc = could not find container \"1c3507a8c08c663bcfa4b0f6d1e526b35d0a9051abc492a0944b99b1ec369a2c\": container with ID starting with 1c3507a8c08c663bcfa4b0f6d1e526b35d0a9051abc492a0944b99b1ec369a2c not found: ID does not exist"
Sep 17 18:37:13.693642 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:37:13.693593    2575 scope.go:117] "RemoveContainer" containerID="3de708cc05cc3ec1e34c4c0bd04da0c21dee854833cade8c01c7487eb88e9df6"
Sep 17 18:37:13.693863 ip-10-0-143-74 kubenswrapper[2575]: E0917 18:37:13.693844    2575 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"3de708cc05cc3ec1e34c4c0bd04da0c21dee854833cade8c01c7487eb88e9df6\": container with ID starting with 3de708cc05cc3ec1e34c4c0bd04da0c21dee854833cade8c01c7487eb88e9df6 not found: ID does not exist" containerID="3de708cc05cc3ec1e34c4c0bd04da0c21dee854833cade8c01c7487eb88e9df6"
Sep 17 18:37:13.693920 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:37:13.693871    2575 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"3de708cc05cc3ec1e34c4c0bd04da0c21dee854833cade8c01c7487eb88e9df6"} err="failed to get container status \"3de708cc05cc3ec1e34c4c0bd04da0c21dee854833cade8c01c7487eb88e9df6\": rpc error: code = NotFound desc = could not find container \"3de708cc05cc3ec1e34c4c0bd04da0c21dee854833cade8c01c7487eb88e9df6\": container with ID starting with 3de708cc05cc3ec1e34c4c0bd04da0c21dee854833cade8c01c7487eb88e9df6 not found: ID does not exist"
Sep 17 18:37:13.693920 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:37:13.693887    2575 scope.go:117] "RemoveContainer" containerID="926bc684892e23e964c42a699ef0c256ca4ee274bab4b16a3dc824e1528f892b"
Sep 17 18:37:13.694139 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:37:13.694121    2575 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"926bc684892e23e964c42a699ef0c256ca4ee274bab4b16a3dc824e1528f892b"} err="failed to get container status \"926bc684892e23e964c42a699ef0c256ca4ee274bab4b16a3dc824e1528f892b\": rpc error: code = NotFound desc = could not find container \"926bc684892e23e964c42a699ef0c256ca4ee274bab4b16a3dc824e1528f892b\": container with ID starting with 926bc684892e23e964c42a699ef0c256ca4ee274bab4b16a3dc824e1528f892b not found: ID does not exist"
Sep 17 18:37:13.694200 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:37:13.694140    2575 scope.go:117] "RemoveContainer" containerID="1c3507a8c08c663bcfa4b0f6d1e526b35d0a9051abc492a0944b99b1ec369a2c"
Sep 17 18:37:13.694393 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:37:13.694356    2575 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"1c3507a8c08c663bcfa4b0f6d1e526b35d0a9051abc492a0944b99b1ec369a2c"} err="failed to get container status \"1c3507a8c08c663bcfa4b0f6d1e526b35d0a9051abc492a0944b99b1ec369a2c\": rpc error: code = NotFound desc = could not find container \"1c3507a8c08c663bcfa4b0f6d1e526b35d0a9051abc492a0944b99b1ec369a2c\": container with ID starting with 1c3507a8c08c663bcfa4b0f6d1e526b35d0a9051abc492a0944b99b1ec369a2c not found: ID does not exist"
Sep 17 18:37:13.694479 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:37:13.694396    2575 scope.go:117] "RemoveContainer" containerID="3de708cc05cc3ec1e34c4c0bd04da0c21dee854833cade8c01c7487eb88e9df6"
Sep 17 18:37:13.694677 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:37:13.694657    2575 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"3de708cc05cc3ec1e34c4c0bd04da0c21dee854833cade8c01c7487eb88e9df6"} err="failed to get container status \"3de708cc05cc3ec1e34c4c0bd04da0c21dee854833cade8c01c7487eb88e9df6\": rpc error: code = NotFound desc = could not find container \"3de708cc05cc3ec1e34c4c0bd04da0c21dee854833cade8c01c7487eb88e9df6\": container with ID starting with 3de708cc05cc3ec1e34c4c0bd04da0c21dee854833cade8c01c7487eb88e9df6 not found: ID does not exist"
Sep 17 18:37:13.694739 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:37:13.694682    2575 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["e2e-test-llm-inference-service-b5dd93e6/router-with-refs-pd-test-kserve-6757c56889-fpdpb"]
Sep 17 18:37:13.699432 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:37:13.699411    2575 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["e2e-test-llm-inference-service-b5dd93e6/router-with-refs-pd-test-kserve-6757c56889-fpdpb"]
Sep 17 18:37:14.967518 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:37:14.967484    2575 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="1c9c57cc-6b68-42ac-a52c-fcb64e122231" path="/var/lib/kubelet/pods/1c9c57cc-6b68-42ac-a52c-fcb64e122231/volumes"
Sep 17 18:37:47.211844 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:37:47.211768    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-djvzt_1d07e68d-9943-420f-a6a0-5f477792fabd/konnectivity-agent/0.log" limit=1024
Sep 17 18:37:48.575591 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:37:48.575559    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-djvzt_1d07e68d-9943-420f-a6a0-5f477792fabd/konnectivity-agent/0.log" limit=1024
Sep 17 18:37:52.212028 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:37:52.211992    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-djvzt_1d07e68d-9943-420f-a6a0-5f477792fabd/konnectivity-agent/0.log" limit=1024
Sep 17 18:37:57.327774 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:37:57.327738    2575 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["e2e-test-llm-inference-service-fac1c4ae/llmisvc-model-fb-opt-125m-route-3c960099-kserve-789bbb5b587wnv5"]
Sep 17 18:37:57.328234 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:37:57.328213    2575 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="1c9c57cc-6b68-42ac-a52c-fcb64e122231" containerName="storage-initializer"
Sep 17 18:37:57.328345 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:37:57.328238    2575 state_mem.go:107] "Deleted CPUSet assignment" podUID="1c9c57cc-6b68-42ac-a52c-fcb64e122231" containerName="storage-initializer"
Sep 17 18:37:57.328345 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:37:57.328258    2575 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="1c9c57cc-6b68-42ac-a52c-fcb64e122231" containerName="main"
Sep 17 18:37:57.328345 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:37:57.328267    2575 state_mem.go:107] "Deleted CPUSet assignment" podUID="1c9c57cc-6b68-42ac-a52c-fcb64e122231" containerName="main"
Sep 17 18:37:57.328345 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:37:57.328283    2575 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="1c9c57cc-6b68-42ac-a52c-fcb64e122231" containerName="llm-d-routing-sidecar"
Sep 17 18:37:57.328345 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:37:57.328292    2575 state_mem.go:107] "Deleted CPUSet assignment" podUID="1c9c57cc-6b68-42ac-a52c-fcb64e122231" containerName="llm-d-routing-sidecar"
Sep 17 18:37:57.328631 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:37:57.328396    2575 memory_manager.go:368] "RemoveStaleState removing state" podUID="1c9c57cc-6b68-42ac-a52c-fcb64e122231" containerName="llm-d-routing-sidecar"
Sep 17 18:37:57.328631 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:37:57.328415    2575 memory_manager.go:368] "RemoveStaleState removing state" podUID="1c9c57cc-6b68-42ac-a52c-fcb64e122231" containerName="main"
Sep 17 18:37:57.331635 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:37:57.331614    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-test-llm-inference-service-fac1c4ae/llmisvc-model-fb-opt-125m-route-3c960099-kserve-789bbb5b587wnv5"
Sep 17 18:37:57.334390 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:37:57.334368    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-test-llm-inference-service-fac1c4ae\"/\"seaweedfs-s3-creds\""
Sep 17 18:37:57.334523 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:37:57.334385    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-test-llm-inference-service-fac1c4ae\"/\"default-dockercfg-2wpjc\""
Sep 17 18:37:57.334725 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:37:57.334708    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-test-llm-inference-service-fac1c4ae\"/\"llmisvd10b93c8eba12f87c3a350a5cae2ee0c-kserve-self-signed-certs\""
Sep 17 18:37:57.334798 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:37:57.334710    2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-test-llm-inference-service-fac1c4ae\"/\"odh-kserve-custom-ca-bundle\""
Sep 17 18:37:57.334798 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:37:57.334713    2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-test-llm-inference-service-fac1c4ae\"/\"openshift-service-ca.crt\""
Sep 17 18:37:57.334881 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:37:57.334712    2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-test-llm-inference-service-fac1c4ae\"/\"kube-root-ca.crt\""
Sep 17 18:37:57.344608 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:37:57.344583    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["e2e-test-llm-inference-service-fac1c4ae/llmisvc-model-fb-opt-125m-route-3c960099-kserve-789bbb5b587wnv5"]
Sep 17 18:37:57.347698 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:37:57.347675    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-djvzt_1d07e68d-9943-420f-a6a0-5f477792fabd/konnectivity-agent/0.log" limit=1024
Sep 17 18:37:57.374637 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:37:57.374604    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/9341934b-0015-404b-81e3-9ef24cc0086a-home\") pod \"llmisvc-model-fb-opt-125m-route-3c960099-kserve-789bbb5b587wnv5\" (UID: \"9341934b-0015-404b-81e3-9ef24cc0086a\") " pod="e2e-test-llm-inference-service-fac1c4ae/llmisvc-model-fb-opt-125m-route-3c960099-kserve-789bbb5b587wnv5"
Sep 17 18:37:57.374637 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:37:57.374637    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/9341934b-0015-404b-81e3-9ef24cc0086a-kserve-provision-location\") pod \"llmisvc-model-fb-opt-125m-route-3c960099-kserve-789bbb5b587wnv5\" (UID: \"9341934b-0015-404b-81e3-9ef24cc0086a\") " pod="e2e-test-llm-inference-service-fac1c4ae/llmisvc-model-fb-opt-125m-route-3c960099-kserve-789bbb5b587wnv5"
Sep 17 18:37:57.374858 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:37:57.374674    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/9341934b-0015-404b-81e3-9ef24cc0086a-model-cache\") pod \"llmisvc-model-fb-opt-125m-route-3c960099-kserve-789bbb5b587wnv5\" (UID: \"9341934b-0015-404b-81e3-9ef24cc0086a\") " pod="e2e-test-llm-inference-service-fac1c4ae/llmisvc-model-fb-opt-125m-route-3c960099-kserve-789bbb5b587wnv5"
Sep 17 18:37:57.374858 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:37:57.374756    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/9341934b-0015-404b-81e3-9ef24cc0086a-tls-certs\") pod \"llmisvc-model-fb-opt-125m-route-3c960099-kserve-789bbb5b587wnv5\" (UID: \"9341934b-0015-404b-81e3-9ef24cc0086a\") " pod="e2e-test-llm-inference-service-fac1c4ae/llmisvc-model-fb-opt-125m-route-3c960099-kserve-789bbb5b587wnv5"
Sep 17 18:37:57.374858 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:37:57.374792    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/9341934b-0015-404b-81e3-9ef24cc0086a-tmp-dir\") pod \"llmisvc-model-fb-opt-125m-route-3c960099-kserve-789bbb5b587wnv5\" (UID: \"9341934b-0015-404b-81e3-9ef24cc0086a\") " pod="e2e-test-llm-inference-service-fac1c4ae/llmisvc-model-fb-opt-125m-route-3c960099-kserve-789bbb5b587wnv5"
Sep 17 18:37:57.374858 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:37:57.374818    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-cssrh\" (UniqueName: \"kubernetes.io/projected/9341934b-0015-404b-81e3-9ef24cc0086a-kube-api-access-cssrh\") pod \"llmisvc-model-fb-opt-125m-route-3c960099-kserve-789bbb5b587wnv5\" (UID: \"9341934b-0015-404b-81e3-9ef24cc0086a\") " pod="e2e-test-llm-inference-service-fac1c4ae/llmisvc-model-fb-opt-125m-route-3c960099-kserve-789bbb5b587wnv5"
Sep 17 18:37:57.375054 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:37:57.374859    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/9341934b-0015-404b-81e3-9ef24cc0086a-dshm\") pod \"llmisvc-model-fb-opt-125m-route-3c960099-kserve-789bbb5b587wnv5\" (UID: \"9341934b-0015-404b-81e3-9ef24cc0086a\") " pod="e2e-test-llm-inference-service-fac1c4ae/llmisvc-model-fb-opt-125m-route-3c960099-kserve-789bbb5b587wnv5"
Sep 17 18:37:57.375054 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:37:57.374895    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cabundle-cert\" (UniqueName: \"kubernetes.io/configmap/9341934b-0015-404b-81e3-9ef24cc0086a-cabundle-cert\") pod \"llmisvc-model-fb-opt-125m-route-3c960099-kserve-789bbb5b587wnv5\" (UID: \"9341934b-0015-404b-81e3-9ef24cc0086a\") " pod="e2e-test-llm-inference-service-fac1c4ae/llmisvc-model-fb-opt-125m-route-3c960099-kserve-789bbb5b587wnv5"
Sep 17 18:37:57.475639 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:37:57.475612    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/9341934b-0015-404b-81e3-9ef24cc0086a-model-cache\") pod \"llmisvc-model-fb-opt-125m-route-3c960099-kserve-789bbb5b587wnv5\" (UID: \"9341934b-0015-404b-81e3-9ef24cc0086a\") " pod="e2e-test-llm-inference-service-fac1c4ae/llmisvc-model-fb-opt-125m-route-3c960099-kserve-789bbb5b587wnv5"
Sep 17 18:37:57.475819 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:37:57.475647    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/9341934b-0015-404b-81e3-9ef24cc0086a-tls-certs\") pod \"llmisvc-model-fb-opt-125m-route-3c960099-kserve-789bbb5b587wnv5\" (UID: \"9341934b-0015-404b-81e3-9ef24cc0086a\") " pod="e2e-test-llm-inference-service-fac1c4ae/llmisvc-model-fb-opt-125m-route-3c960099-kserve-789bbb5b587wnv5"
Sep 17 18:37:57.475819 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:37:57.475668    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/9341934b-0015-404b-81e3-9ef24cc0086a-tmp-dir\") pod \"llmisvc-model-fb-opt-125m-route-3c960099-kserve-789bbb5b587wnv5\" (UID: \"9341934b-0015-404b-81e3-9ef24cc0086a\") " pod="e2e-test-llm-inference-service-fac1c4ae/llmisvc-model-fb-opt-125m-route-3c960099-kserve-789bbb5b587wnv5"
Sep 17 18:37:57.475819 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:37:57.475690    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-cssrh\" (UniqueName: \"kubernetes.io/projected/9341934b-0015-404b-81e3-9ef24cc0086a-kube-api-access-cssrh\") pod \"llmisvc-model-fb-opt-125m-route-3c960099-kserve-789bbb5b587wnv5\" (UID: \"9341934b-0015-404b-81e3-9ef24cc0086a\") " pod="e2e-test-llm-inference-service-fac1c4ae/llmisvc-model-fb-opt-125m-route-3c960099-kserve-789bbb5b587wnv5"
Sep 17 18:37:57.475819 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:37:57.475723    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/9341934b-0015-404b-81e3-9ef24cc0086a-dshm\") pod \"llmisvc-model-fb-opt-125m-route-3c960099-kserve-789bbb5b587wnv5\" (UID: \"9341934b-0015-404b-81e3-9ef24cc0086a\") " pod="e2e-test-llm-inference-service-fac1c4ae/llmisvc-model-fb-opt-125m-route-3c960099-kserve-789bbb5b587wnv5"
Sep 17 18:37:57.475819 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:37:57.475754    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cabundle-cert\" (UniqueName: \"kubernetes.io/configmap/9341934b-0015-404b-81e3-9ef24cc0086a-cabundle-cert\") pod \"llmisvc-model-fb-opt-125m-route-3c960099-kserve-789bbb5b587wnv5\" (UID: \"9341934b-0015-404b-81e3-9ef24cc0086a\") " pod="e2e-test-llm-inference-service-fac1c4ae/llmisvc-model-fb-opt-125m-route-3c960099-kserve-789bbb5b587wnv5"
Sep 17 18:37:57.476101 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:37:57.475826    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/9341934b-0015-404b-81e3-9ef24cc0086a-home\") pod \"llmisvc-model-fb-opt-125m-route-3c960099-kserve-789bbb5b587wnv5\" (UID: \"9341934b-0015-404b-81e3-9ef24cc0086a\") " pod="e2e-test-llm-inference-service-fac1c4ae/llmisvc-model-fb-opt-125m-route-3c960099-kserve-789bbb5b587wnv5"
Sep 17 18:37:57.476101 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:37:57.475848    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/9341934b-0015-404b-81e3-9ef24cc0086a-kserve-provision-location\") pod \"llmisvc-model-fb-opt-125m-route-3c960099-kserve-789bbb5b587wnv5\" (UID: \"9341934b-0015-404b-81e3-9ef24cc0086a\") " pod="e2e-test-llm-inference-service-fac1c4ae/llmisvc-model-fb-opt-125m-route-3c960099-kserve-789bbb5b587wnv5"
Sep 17 18:37:57.476380 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:37:57.476163    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/9341934b-0015-404b-81e3-9ef24cc0086a-model-cache\") pod \"llmisvc-model-fb-opt-125m-route-3c960099-kserve-789bbb5b587wnv5\" (UID: \"9341934b-0015-404b-81e3-9ef24cc0086a\") " pod="e2e-test-llm-inference-service-fac1c4ae/llmisvc-model-fb-opt-125m-route-3c960099-kserve-789bbb5b587wnv5"
Sep 17 18:37:57.476380 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:37:57.476182    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/9341934b-0015-404b-81e3-9ef24cc0086a-tmp-dir\") pod \"llmisvc-model-fb-opt-125m-route-3c960099-kserve-789bbb5b587wnv5\" (UID: \"9341934b-0015-404b-81e3-9ef24cc0086a\") " pod="e2e-test-llm-inference-service-fac1c4ae/llmisvc-model-fb-opt-125m-route-3c960099-kserve-789bbb5b587wnv5"
Sep 17 18:37:57.476380 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:37:57.476260    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/9341934b-0015-404b-81e3-9ef24cc0086a-home\") pod \"llmisvc-model-fb-opt-125m-route-3c960099-kserve-789bbb5b587wnv5\" (UID: \"9341934b-0015-404b-81e3-9ef24cc0086a\") " pod="e2e-test-llm-inference-service-fac1c4ae/llmisvc-model-fb-opt-125m-route-3c960099-kserve-789bbb5b587wnv5"
Sep 17 18:37:57.476582 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:37:57.476468    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/9341934b-0015-404b-81e3-9ef24cc0086a-kserve-provision-location\") pod \"llmisvc-model-fb-opt-125m-route-3c960099-kserve-789bbb5b587wnv5\" (UID: \"9341934b-0015-404b-81e3-9ef24cc0086a\") " pod="e2e-test-llm-inference-service-fac1c4ae/llmisvc-model-fb-opt-125m-route-3c960099-kserve-789bbb5b587wnv5"
Sep 17 18:37:57.476901 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:37:57.476880    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cabundle-cert\" (UniqueName: \"kubernetes.io/configmap/9341934b-0015-404b-81e3-9ef24cc0086a-cabundle-cert\") pod \"llmisvc-model-fb-opt-125m-route-3c960099-kserve-789bbb5b587wnv5\" (UID: \"9341934b-0015-404b-81e3-9ef24cc0086a\") " pod="e2e-test-llm-inference-service-fac1c4ae/llmisvc-model-fb-opt-125m-route-3c960099-kserve-789bbb5b587wnv5"
Sep 17 18:37:57.478169 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:37:57.478141    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/9341934b-0015-404b-81e3-9ef24cc0086a-dshm\") pod \"llmisvc-model-fb-opt-125m-route-3c960099-kserve-789bbb5b587wnv5\" (UID: \"9341934b-0015-404b-81e3-9ef24cc0086a\") " pod="e2e-test-llm-inference-service-fac1c4ae/llmisvc-model-fb-opt-125m-route-3c960099-kserve-789bbb5b587wnv5"
Sep 17 18:37:57.478261 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:37:57.478182    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/9341934b-0015-404b-81e3-9ef24cc0086a-tls-certs\") pod \"llmisvc-model-fb-opt-125m-route-3c960099-kserve-789bbb5b587wnv5\" (UID: \"9341934b-0015-404b-81e3-9ef24cc0086a\") " pod="e2e-test-llm-inference-service-fac1c4ae/llmisvc-model-fb-opt-125m-route-3c960099-kserve-789bbb5b587wnv5"
Sep 17 18:37:57.485849 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:37:57.485811    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-cssrh\" (UniqueName: \"kubernetes.io/projected/9341934b-0015-404b-81e3-9ef24cc0086a-kube-api-access-cssrh\") pod \"llmisvc-model-fb-opt-125m-route-3c960099-kserve-789bbb5b587wnv5\" (UID: \"9341934b-0015-404b-81e3-9ef24cc0086a\") " pod="e2e-test-llm-inference-service-fac1c4ae/llmisvc-model-fb-opt-125m-route-3c960099-kserve-789bbb5b587wnv5"
Sep 17 18:37:57.644175 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:37:57.644133    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-test-llm-inference-service-fac1c4ae/llmisvc-model-fb-opt-125m-route-3c960099-kserve-789bbb5b587wnv5"
Sep 17 18:37:57.930657 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:37:57.930626    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["e2e-test-llm-inference-service-fac1c4ae/llmisvc-model-fb-opt-125m-route-3c960099-kserve-789bbb5b587wnv5"]
Sep 17 18:37:57.931937 ip-10-0-143-74 kubenswrapper[2575]: W0917 18:37:57.931910    2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9341934b_0015_404b_81e3_9ef24cc0086a.slice/crio-06900a967624bbdd9661ad8a9e236c39d3d4db62ebbb48504ed846e52176757c WatchSource:0}: Error finding container 06900a967624bbdd9661ad8a9e236c39d3d4db62ebbb48504ed846e52176757c: Status 404 returned error can't find the container with id 06900a967624bbdd9661ad8a9e236c39d3d4db62ebbb48504ed846e52176757c
Sep 17 18:37:58.900249 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:37:58.900209    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-test-llm-inference-service-fac1c4ae/llmisvc-model-fb-opt-125m-route-3c960099-kserve-789bbb5b587wnv5" event={"ID":"9341934b-0015-404b-81e3-9ef24cc0086a","Type":"ContainerStarted","Data":"dd28598bad3b5cc561008b5effe960032355ce2121ae57ed9583374b51409cb3"}
Sep 17 18:37:58.900698 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:37:58.900272    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-test-llm-inference-service-fac1c4ae/llmisvc-model-fb-opt-125m-route-3c960099-kserve-789bbb5b587wnv5" event={"ID":"9341934b-0015-404b-81e3-9ef24cc0086a","Type":"ContainerStarted","Data":"06900a967624bbdd9661ad8a9e236c39d3d4db62ebbb48504ed846e52176757c"}
Sep 17 18:38:00.912960 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:38:00.912922    2575 generic.go:356] "Generic (PLEG): container finished" podID="9341934b-0015-404b-81e3-9ef24cc0086a" containerID="dd28598bad3b5cc561008b5effe960032355ce2121ae57ed9583374b51409cb3" exitCode=0
Sep 17 18:38:00.913419 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:38:00.912967    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-test-llm-inference-service-fac1c4ae/llmisvc-model-fb-opt-125m-route-3c960099-kserve-789bbb5b587wnv5" event={"ID":"9341934b-0015-404b-81e3-9ef24cc0086a","Type":"ContainerDied","Data":"dd28598bad3b5cc561008b5effe960032355ce2121ae57ed9583374b51409cb3"}
Sep 17 18:38:01.920733 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:38:01.920696    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-test-llm-inference-service-fac1c4ae/llmisvc-model-fb-opt-125m-route-3c960099-kserve-789bbb5b587wnv5" event={"ID":"9341934b-0015-404b-81e3-9ef24cc0086a","Type":"ContainerStarted","Data":"f6fa6dc2547be1338be37f7ea0de225f0a5e9277f38502d026623d6c46d905d2"}
Sep 17 18:38:01.945027 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:38:01.944970    2575 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="e2e-test-llm-inference-service-fac1c4ae/llmisvc-model-fb-opt-125m-route-3c960099-kserve-789bbb5b587wnv5" podStartSLOduration=2.621665241 podStartE2EDuration="4.944953461s" totalImagesPullingTime="0s" totalInitContainerRuntime="2.32328822s" isStatefulPod=false podCreationTimestamp="2026-09-17 18:37:57 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-17 18:38:01.942267227 +0000 UTC m=+3801.572597201" watchObservedRunningTime="2026-09-17 18:38:01.944953461 +0000 UTC m=+3801.575283434"
Sep 17 18:38:02.645067 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:38:02.645029    2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-test-llm-inference-service-fac1c4ae/llmisvc-model-fb-opt-125m-route-3c960099-kserve-789bbb5b587wnv5"
Sep 17 18:38:07.644812 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:38:07.644751    2575 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="e2e-test-llm-inference-service-fac1c4ae/llmisvc-model-fb-opt-125m-route-3c960099-kserve-789bbb5b587wnv5"
Sep 17 18:38:07.646694 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:38:07.646659    2575 prober.go:120] "Probe failed" probeType="Startup" pod="e2e-test-llm-inference-service-fac1c4ae/llmisvc-model-fb-opt-125m-route-3c960099-kserve-789bbb5b587wnv5" podUID="9341934b-0015-404b-81e3-9ef24cc0086a" containerName="main" probeResult="failure" output="Get \"https://10.133.0.64:8000/health\": dial tcp 10.133.0.64:8000: connect: connection refused"
Sep 17 18:38:14.143142 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:38:14.143097    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-djvzt_1d07e68d-9943-420f-a6a0-5f477792fabd/konnectivity-agent/0.log" limit=1024
Sep 17 18:38:14.557983 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:38:14.557948    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-djvzt_1d07e68d-9943-420f-a6a0-5f477792fabd/konnectivity-agent/0.log" limit=1024
Sep 17 18:38:17.645229 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:38:17.645186    2575 prober.go:120] "Probe failed" probeType="Startup" pod="e2e-test-llm-inference-service-fac1c4ae/llmisvc-model-fb-opt-125m-route-3c960099-kserve-789bbb5b587wnv5" podUID="9341934b-0015-404b-81e3-9ef24cc0086a" containerName="main" probeResult="failure" output="Get \"https://10.133.0.64:8000/health\": dial tcp 10.133.0.64:8000: connect: connection refused"
Sep 17 18:38:20.150568 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:38:20.150523    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-djvzt_1d07e68d-9943-420f-a6a0-5f477792fabd/konnectivity-agent/0.log" limit=1024
Sep 17 18:38:23.401708 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:38:23.401658    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-djvzt_1d07e68d-9943-420f-a6a0-5f477792fabd/konnectivity-agent/0.log" limit=1024
Sep 17 18:38:23.504662 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:38:23.504629    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-djvzt_1d07e68d-9943-420f-a6a0-5f477792fabd/konnectivity-agent/0.log" limit=1024
Sep 17 18:38:27.645051 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:38:27.644992    2575 prober.go:120] "Probe failed" probeType="Startup" pod="e2e-test-llm-inference-service-fac1c4ae/llmisvc-model-fb-opt-125m-route-3c960099-kserve-789bbb5b587wnv5" podUID="9341934b-0015-404b-81e3-9ef24cc0086a" containerName="main" probeResult="failure" output="Get \"https://10.133.0.64:8000/health\": dial tcp 10.133.0.64:8000: connect: connection refused"
Sep 17 18:38:27.971299 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:38:27.971207    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-djvzt_1d07e68d-9943-420f-a6a0-5f477792fabd/konnectivity-agent/0.log" limit=1024
Sep 17 18:38:35.376771 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:38:35.376730    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-djvzt_1d07e68d-9943-420f-a6a0-5f477792fabd/konnectivity-agent/0.log" limit=1024
Sep 17 18:38:37.644844 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:38:37.644798    2575 prober.go:120] "Probe failed" probeType="Startup" pod="e2e-test-llm-inference-service-fac1c4ae/llmisvc-model-fb-opt-125m-route-3c960099-kserve-789bbb5b587wnv5" podUID="9341934b-0015-404b-81e3-9ef24cc0086a" containerName="main" probeResult="failure" output="Get \"https://10.133.0.64:8000/health\": dial tcp 10.133.0.64:8000: connect: connection refused"
Sep 17 18:38:43.112084 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:38:43.112035    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-djvzt_1d07e68d-9943-420f-a6a0-5f477792fabd/konnectivity-agent/0.log" limit=1024
Sep 17 18:38:43.198947 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:38:43.198906    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-djvzt_1d07e68d-9943-420f-a6a0-5f477792fabd/konnectivity-agent/0.log" limit=1024
Sep 17 18:38:45.814457 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:38:45.814417    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-djvzt_1d07e68d-9943-420f-a6a0-5f477792fabd/konnectivity-agent/0.log" limit=1024
Sep 17 18:38:45.914288 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:38:45.914257    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-djvzt_1d07e68d-9943-420f-a6a0-5f477792fabd/konnectivity-agent/0.log" limit=1024
Sep 17 18:38:46.014941 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:38:46.014907    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-djvzt_1d07e68d-9943-420f-a6a0-5f477792fabd/konnectivity-agent/0.log" limit=1024
Sep 17 18:38:46.116843 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:38:46.116760    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-djvzt_1d07e68d-9943-420f-a6a0-5f477792fabd/konnectivity-agent/0.log" limit=1024
Sep 17 18:38:46.223855 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:38:46.223820    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-djvzt_1d07e68d-9943-420f-a6a0-5f477792fabd/konnectivity-agent/0.log" limit=1024
Sep 17 18:38:47.644823 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:38:47.644784    2575 prober.go:120] "Probe failed" probeType="Startup" pod="e2e-test-llm-inference-service-fac1c4ae/llmisvc-model-fb-opt-125m-route-3c960099-kserve-789bbb5b587wnv5" podUID="9341934b-0015-404b-81e3-9ef24cc0086a" containerName="main" probeResult="failure" output="Get \"https://10.133.0.64:8000/health\": dial tcp 10.133.0.64:8000: connect: connection refused"
Sep 17 18:38:48.494238 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:38:48.494197    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-djvzt_1d07e68d-9943-420f-a6a0-5f477792fabd/konnectivity-agent/0.log" limit=1024
Sep 17 18:38:48.570244 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:38:48.570206    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-djvzt_1d07e68d-9943-420f-a6a0-5f477792fabd/konnectivity-agent/0.log" limit=1024
Sep 17 18:38:49.295257 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:38:49.295218    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-djvzt_1d07e68d-9943-420f-a6a0-5f477792fabd/konnectivity-agent/0.log" limit=1024
Sep 17 18:38:51.145240 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:38:51.145202    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-djvzt_1d07e68d-9943-420f-a6a0-5f477792fabd/konnectivity-agent/0.log" limit=1024
Sep 17 18:38:51.218764 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:38:51.218731    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-djvzt_1d07e68d-9943-420f-a6a0-5f477792fabd/konnectivity-agent/0.log" limit=1024
Sep 17 18:38:51.315538 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:38:51.315506    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-djvzt_1d07e68d-9943-420f-a6a0-5f477792fabd/konnectivity-agent/0.log" limit=1024
Sep 17 18:38:51.415050 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:38:51.414964    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-djvzt_1d07e68d-9943-420f-a6a0-5f477792fabd/konnectivity-agent/0.log" limit=1024
Sep 17 18:38:51.968237 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:38:51.968207    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-djvzt_1d07e68d-9943-420f-a6a0-5f477792fabd/konnectivity-agent/0.log" limit=1024
Sep 17 18:38:57.644632 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:38:57.644579    2575 prober.go:120] "Probe failed" probeType="Startup" pod="e2e-test-llm-inference-service-fac1c4ae/llmisvc-model-fb-opt-125m-route-3c960099-kserve-789bbb5b587wnv5" podUID="9341934b-0015-404b-81e3-9ef24cc0086a" containerName="main" probeResult="failure" output="Get \"https://10.133.0.64:8000/health\": dial tcp 10.133.0.64:8000: connect: connection refused"
Sep 17 18:38:57.692514 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:38:57.692475    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-djvzt_1d07e68d-9943-420f-a6a0-5f477792fabd/konnectivity-agent/0.log" limit=1024
Sep 17 18:38:57.842419 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:38:57.842385    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-djvzt_1d07e68d-9943-420f-a6a0-5f477792fabd/konnectivity-agent/0.log" limit=1024
Sep 17 18:39:07.645219 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:39:07.645115    2575 prober.go:120] "Probe failed" probeType="Startup" pod="e2e-test-llm-inference-service-fac1c4ae/llmisvc-model-fb-opt-125m-route-3c960099-kserve-789bbb5b587wnv5" podUID="9341934b-0015-404b-81e3-9ef24cc0086a" containerName="main" probeResult="failure" output="Get \"https://10.133.0.64:8000/health\": dial tcp 10.133.0.64:8000: connect: connection refused"
Sep 17 18:39:14.517969 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:39:14.517929    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-djvzt_1d07e68d-9943-420f-a6a0-5f477792fabd/konnectivity-agent/0.log" limit=1024
Sep 17 18:39:17.645626 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:39:17.645573    2575 prober.go:120] "Probe failed" probeType="Startup" pod="e2e-test-llm-inference-service-fac1c4ae/llmisvc-model-fb-opt-125m-route-3c960099-kserve-789bbb5b587wnv5" podUID="9341934b-0015-404b-81e3-9ef24cc0086a" containerName="main" probeResult="failure" output="Get \"https://10.133.0.64:8000/health\": dial tcp 10.133.0.64:8000: connect: connection refused"
Sep 17 18:39:27.645410 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:39:27.645356    2575 prober.go:120] "Probe failed" probeType="Startup" pod="e2e-test-llm-inference-service-fac1c4ae/llmisvc-model-fb-opt-125m-route-3c960099-kserve-789bbb5b587wnv5" podUID="9341934b-0015-404b-81e3-9ef24cc0086a" containerName="main" probeResult="failure" output="Get \"https://10.133.0.64:8000/health\": dial tcp 10.133.0.64:8000: connect: connection refused"
Sep 17 18:39:36.710581 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:39:36.710536    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-djvzt_1d07e68d-9943-420f-a6a0-5f477792fabd/konnectivity-agent/0.log" limit=1024
Sep 17 18:39:37.644980 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:39:37.644932    2575 prober.go:120] "Probe failed" probeType="Startup" pod="e2e-test-llm-inference-service-fac1c4ae/llmisvc-model-fb-opt-125m-route-3c960099-kserve-789bbb5b587wnv5" podUID="9341934b-0015-404b-81e3-9ef24cc0086a" containerName="main" probeResult="failure" output="Get \"https://10.133.0.64:8000/health\": dial tcp 10.133.0.64:8000: connect: connection refused"
Sep 17 18:39:47.645018 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:39:47.644962    2575 prober.go:120] "Probe failed" probeType="Startup" pod="e2e-test-llm-inference-service-fac1c4ae/llmisvc-model-fb-opt-125m-route-3c960099-kserve-789bbb5b587wnv5" podUID="9341934b-0015-404b-81e3-9ef24cc0086a" containerName="main" probeResult="failure" output="Get \"https://10.133.0.64:8000/health\": dial tcp 10.133.0.64:8000: connect: connection refused"
Sep 17 18:39:57.655226 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:39:57.655195    2575 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="e2e-test-llm-inference-service-fac1c4ae/llmisvc-model-fb-opt-125m-route-3c960099-kserve-789bbb5b587wnv5"
Sep 17 18:39:57.663235 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:39:57.663208    2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-test-llm-inference-service-fac1c4ae/llmisvc-model-fb-opt-125m-route-3c960099-kserve-789bbb5b587wnv5"
Sep 17 18:40:02.445866 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:40:02.445829    2575 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["e2e-test-llm-inference-service-fac1c4ae/llmisvc-model-fb-opt-125m-route-3c960099-kserve-789bbb5b587wnv5"]
Sep 17 18:40:02.468378 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:40:02.468332    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-djvzt_1d07e68d-9943-420f-a6a0-5f477792fabd/konnectivity-agent/0.log" limit=1024
Sep 17 18:40:06.801656 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:40:06.801616    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-djvzt_1d07e68d-9943-420f-a6a0-5f477792fabd/konnectivity-agent/0.log" limit=1024
Sep 17 18:40:06.903595 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:40:06.903535    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-djvzt_1d07e68d-9943-420f-a6a0-5f477792fabd/konnectivity-agent/0.log" limit=1024
Sep 17 18:40:17.473135 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:40:17.473093    2575 kuberuntime_container.go:873] "Killing container with a grace period" pod="e2e-test-llm-inference-service-fac1c4ae/llmisvc-model-fb-opt-125m-route-3c960099-kserve-789bbb5b587wnv5" podUID="9341934b-0015-404b-81e3-9ef24cc0086a" containerName="main" containerID="cri-o://f6fa6dc2547be1338be37f7ea0de225f0a5e9277f38502d026623d6c46d905d2" gracePeriod=45
Sep 17 18:40:17.656421 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:40:17.656370    2575 prober.go:120] "Probe failed" probeType="Readiness" pod="e2e-test-llm-inference-service-fac1c4ae/llmisvc-model-fb-opt-125m-route-3c960099-kserve-789bbb5b587wnv5" podUID="9341934b-0015-404b-81e3-9ef24cc0086a" containerName="main" probeResult="failure" output="Get \"https://10.133.0.64:8000/health\": dial tcp 10.133.0.64:8000: connect: connection refused"
Sep 17 18:40:18.230500 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:40:18.230475    2575 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-test-llm-inference-service-fac1c4ae/llmisvc-model-fb-opt-125m-route-3c960099-kserve-789bbb5b587wnv5"
Sep 17 18:40:18.361590 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:40:18.361505    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"cabundle-cert\" (UniqueName: \"kubernetes.io/configmap/9341934b-0015-404b-81e3-9ef24cc0086a-cabundle-cert\") pod \"9341934b-0015-404b-81e3-9ef24cc0086a\" (UID: \"9341934b-0015-404b-81e3-9ef24cc0086a\") "
Sep 17 18:40:18.361590 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:40:18.361550    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/9341934b-0015-404b-81e3-9ef24cc0086a-tmp-dir\") pod \"9341934b-0015-404b-81e3-9ef24cc0086a\" (UID: \"9341934b-0015-404b-81e3-9ef24cc0086a\") "
Sep 17 18:40:18.361816 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:40:18.361607    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/9341934b-0015-404b-81e3-9ef24cc0086a-home\") pod \"9341934b-0015-404b-81e3-9ef24cc0086a\" (UID: \"9341934b-0015-404b-81e3-9ef24cc0086a\") "
Sep 17 18:40:18.361816 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:40:18.361658    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/9341934b-0015-404b-81e3-9ef24cc0086a-tls-certs\") pod \"9341934b-0015-404b-81e3-9ef24cc0086a\" (UID: \"9341934b-0015-404b-81e3-9ef24cc0086a\") "
Sep 17 18:40:18.361816 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:40:18.361678    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/9341934b-0015-404b-81e3-9ef24cc0086a-dshm\") pod \"9341934b-0015-404b-81e3-9ef24cc0086a\" (UID: \"9341934b-0015-404b-81e3-9ef24cc0086a\") "
Sep 17 18:40:18.361816 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:40:18.361706    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/9341934b-0015-404b-81e3-9ef24cc0086a-kserve-provision-location\") pod \"9341934b-0015-404b-81e3-9ef24cc0086a\" (UID: \"9341934b-0015-404b-81e3-9ef24cc0086a\") "
Sep 17 18:40:18.361816 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:40:18.361736    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-cssrh\" (UniqueName: \"kubernetes.io/projected/9341934b-0015-404b-81e3-9ef24cc0086a-kube-api-access-cssrh\") pod \"9341934b-0015-404b-81e3-9ef24cc0086a\" (UID: \"9341934b-0015-404b-81e3-9ef24cc0086a\") "
Sep 17 18:40:18.361816 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:40:18.361775    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/9341934b-0015-404b-81e3-9ef24cc0086a-model-cache\") pod \"9341934b-0015-404b-81e3-9ef24cc0086a\" (UID: \"9341934b-0015-404b-81e3-9ef24cc0086a\") "
Sep 17 18:40:18.362245 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:40:18.362198    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/9341934b-0015-404b-81e3-9ef24cc0086a-model-cache" (OuterVolumeSpecName: "model-cache") pod "9341934b-0015-404b-81e3-9ef24cc0086a" (UID: "9341934b-0015-404b-81e3-9ef24cc0086a"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 17 18:40:18.362245 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:40:18.362225    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/9341934b-0015-404b-81e3-9ef24cc0086a-cabundle-cert" (OuterVolumeSpecName: "cabundle-cert") pod "9341934b-0015-404b-81e3-9ef24cc0086a" (UID: "9341934b-0015-404b-81e3-9ef24cc0086a"). InnerVolumeSpecName "cabundle-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 17 18:40:18.362469 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:40:18.362414    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/9341934b-0015-404b-81e3-9ef24cc0086a-home" (OuterVolumeSpecName: "home") pod "9341934b-0015-404b-81e3-9ef24cc0086a" (UID: "9341934b-0015-404b-81e3-9ef24cc0086a"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 17 18:40:18.364135 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:40:18.364109    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/9341934b-0015-404b-81e3-9ef24cc0086a-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "9341934b-0015-404b-81e3-9ef24cc0086a" (UID: "9341934b-0015-404b-81e3-9ef24cc0086a"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 17 18:40:18.364324 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:40:18.364295    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/9341934b-0015-404b-81e3-9ef24cc0086a-dshm" (OuterVolumeSpecName: "dshm") pod "9341934b-0015-404b-81e3-9ef24cc0086a" (UID: "9341934b-0015-404b-81e3-9ef24cc0086a"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 17 18:40:18.364463 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:40:18.364342    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/9341934b-0015-404b-81e3-9ef24cc0086a-kube-api-access-cssrh" (OuterVolumeSpecName: "kube-api-access-cssrh") pod "9341934b-0015-404b-81e3-9ef24cc0086a" (UID: "9341934b-0015-404b-81e3-9ef24cc0086a"). InnerVolumeSpecName "kube-api-access-cssrh". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 17 18:40:18.373064 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:40:18.373040    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/9341934b-0015-404b-81e3-9ef24cc0086a-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "9341934b-0015-404b-81e3-9ef24cc0086a" (UID: "9341934b-0015-404b-81e3-9ef24cc0086a"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 17 18:40:18.386770 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:40:18.386739    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/9341934b-0015-404b-81e3-9ef24cc0086a-kserve-provision-location" (OuterVolumeSpecName: "kserve-provision-location") pod "9341934b-0015-404b-81e3-9ef24cc0086a" (UID: "9341934b-0015-404b-81e3-9ef24cc0086a"). InnerVolumeSpecName "kserve-provision-location". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 17 18:40:18.462674 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:40:18.462642    2575 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/9341934b-0015-404b-81e3-9ef24cc0086a-home\") on node \"ip-10-0-143-74.ec2.internal\" DevicePath \"\""
Sep 17 18:40:18.462674 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:40:18.462668    2575 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/9341934b-0015-404b-81e3-9ef24cc0086a-tls-certs\") on node \"ip-10-0-143-74.ec2.internal\" DevicePath \"\""
Sep 17 18:40:18.462674 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:40:18.462678    2575 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/9341934b-0015-404b-81e3-9ef24cc0086a-dshm\") on node \"ip-10-0-143-74.ec2.internal\" DevicePath \"\""
Sep 17 18:40:18.462894 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:40:18.462686    2575 reconciler_common.go:299] "Volume detached for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/9341934b-0015-404b-81e3-9ef24cc0086a-kserve-provision-location\") on node \"ip-10-0-143-74.ec2.internal\" DevicePath \"\""
Sep 17 18:40:18.462894 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:40:18.462695    2575 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-cssrh\" (UniqueName: \"kubernetes.io/projected/9341934b-0015-404b-81e3-9ef24cc0086a-kube-api-access-cssrh\") on node \"ip-10-0-143-74.ec2.internal\" DevicePath \"\""
Sep 17 18:40:18.462894 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:40:18.462703    2575 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/9341934b-0015-404b-81e3-9ef24cc0086a-model-cache\") on node \"ip-10-0-143-74.ec2.internal\" DevicePath \"\""
Sep 17 18:40:18.462894 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:40:18.462711    2575 reconciler_common.go:299] "Volume detached for volume \"cabundle-cert\" (UniqueName: \"kubernetes.io/configmap/9341934b-0015-404b-81e3-9ef24cc0086a-cabundle-cert\") on node \"ip-10-0-143-74.ec2.internal\" DevicePath \"\""
Sep 17 18:40:18.462894 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:40:18.462718    2575 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/9341934b-0015-404b-81e3-9ef24cc0086a-tmp-dir\") on node \"ip-10-0-143-74.ec2.internal\" DevicePath \"\""
Sep 17 18:40:18.705611 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:40:18.705573    2575 generic.go:356] "Generic (PLEG): container finished" podID="9341934b-0015-404b-81e3-9ef24cc0086a" containerID="f6fa6dc2547be1338be37f7ea0de225f0a5e9277f38502d026623d6c46d905d2" exitCode=0
Sep 17 18:40:18.706031 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:40:18.705622    2575 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-test-llm-inference-service-fac1c4ae/llmisvc-model-fb-opt-125m-route-3c960099-kserve-789bbb5b587wnv5"
Sep 17 18:40:18.706031 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:40:18.705649    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-test-llm-inference-service-fac1c4ae/llmisvc-model-fb-opt-125m-route-3c960099-kserve-789bbb5b587wnv5" event={"ID":"9341934b-0015-404b-81e3-9ef24cc0086a","Type":"ContainerDied","Data":"f6fa6dc2547be1338be37f7ea0de225f0a5e9277f38502d026623d6c46d905d2"}
Sep 17 18:40:18.706031 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:40:18.705701    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-test-llm-inference-service-fac1c4ae/llmisvc-model-fb-opt-125m-route-3c960099-kserve-789bbb5b587wnv5" event={"ID":"9341934b-0015-404b-81e3-9ef24cc0086a","Type":"ContainerDied","Data":"06900a967624bbdd9661ad8a9e236c39d3d4db62ebbb48504ed846e52176757c"}
Sep 17 18:40:18.706031 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:40:18.705724    2575 scope.go:117] "RemoveContainer" containerID="f6fa6dc2547be1338be37f7ea0de225f0a5e9277f38502d026623d6c46d905d2"
Sep 17 18:40:18.715508 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:40:18.715486    2575 scope.go:117] "RemoveContainer" containerID="dd28598bad3b5cc561008b5effe960032355ce2121ae57ed9583374b51409cb3"
Sep 17 18:40:18.723620 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:40:18.723602    2575 scope.go:117] "RemoveContainer" containerID="f6fa6dc2547be1338be37f7ea0de225f0a5e9277f38502d026623d6c46d905d2"
Sep 17 18:40:18.723885 ip-10-0-143-74 kubenswrapper[2575]: E0917 18:40:18.723865    2575 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"f6fa6dc2547be1338be37f7ea0de225f0a5e9277f38502d026623d6c46d905d2\": container with ID starting with f6fa6dc2547be1338be37f7ea0de225f0a5e9277f38502d026623d6c46d905d2 not found: ID does not exist" containerID="f6fa6dc2547be1338be37f7ea0de225f0a5e9277f38502d026623d6c46d905d2"
Sep 17 18:40:18.723933 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:40:18.723894    2575 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"f6fa6dc2547be1338be37f7ea0de225f0a5e9277f38502d026623d6c46d905d2"} err="failed to get container status \"f6fa6dc2547be1338be37f7ea0de225f0a5e9277f38502d026623d6c46d905d2\": rpc error: code = NotFound desc = could not find container \"f6fa6dc2547be1338be37f7ea0de225f0a5e9277f38502d026623d6c46d905d2\": container with ID starting with f6fa6dc2547be1338be37f7ea0de225f0a5e9277f38502d026623d6c46d905d2 not found: ID does not exist"
Sep 17 18:40:18.723933 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:40:18.723912    2575 scope.go:117] "RemoveContainer" containerID="dd28598bad3b5cc561008b5effe960032355ce2121ae57ed9583374b51409cb3"
Sep 17 18:40:18.724130 ip-10-0-143-74 kubenswrapper[2575]: E0917 18:40:18.724114    2575 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"dd28598bad3b5cc561008b5effe960032355ce2121ae57ed9583374b51409cb3\": container with ID starting with dd28598bad3b5cc561008b5effe960032355ce2121ae57ed9583374b51409cb3 not found: ID does not exist" containerID="dd28598bad3b5cc561008b5effe960032355ce2121ae57ed9583374b51409cb3"
Sep 17 18:40:18.724176 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:40:18.724133    2575 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"dd28598bad3b5cc561008b5effe960032355ce2121ae57ed9583374b51409cb3"} err="failed to get container status \"dd28598bad3b5cc561008b5effe960032355ce2121ae57ed9583374b51409cb3\": rpc error: code = NotFound desc = could not find container \"dd28598bad3b5cc561008b5effe960032355ce2121ae57ed9583374b51409cb3\": container with ID starting with dd28598bad3b5cc561008b5effe960032355ce2121ae57ed9583374b51409cb3 not found: ID does not exist"
Sep 17 18:40:18.731757 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:40:18.731734    2575 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["e2e-test-llm-inference-service-fac1c4ae/llmisvc-model-fb-opt-125m-route-3c960099-kserve-789bbb5b587wnv5"]
Sep 17 18:40:18.735379 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:40:18.735357    2575 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["e2e-test-llm-inference-service-fac1c4ae/llmisvc-model-fb-opt-125m-route-3c960099-kserve-789bbb5b587wnv5"]
Sep 17 18:40:18.967558 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:40:18.967469    2575 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="9341934b-0015-404b-81e3-9ef24cc0086a" path="/var/lib/kubelet/pods/9341934b-0015-404b-81e3-9ef24cc0086a/volumes"
Sep 17 18:40:22.525868 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:40:22.525832    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-djvzt_1d07e68d-9943-420f-a6a0-5f477792fabd/konnectivity-agent/0.log" limit=1024
Sep 17 18:40:22.624590 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:40:22.624548    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-djvzt_1d07e68d-9943-420f-a6a0-5f477792fabd/konnectivity-agent/0.log" limit=1024
Sep 17 18:40:29.150670 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:40:29.150638    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-djvzt_1d07e68d-9943-420f-a6a0-5f477792fabd/konnectivity-agent/0.log" limit=1024
Sep 17 18:40:34.494911 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:40:34.494830    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-djvzt_1d07e68d-9943-420f-a6a0-5f477792fabd/konnectivity-agent/0.log" limit=1024
Sep 17 18:40:34.620490 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:40:34.620455    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-djvzt_1d07e68d-9943-420f-a6a0-5f477792fabd/konnectivity-agent/0.log" limit=1024
Sep 17 18:40:34.745658 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:40:34.745576    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-djvzt_1d07e68d-9943-420f-a6a0-5f477792fabd/konnectivity-agent/0.log" limit=1024
Sep 17 18:40:34.794667 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:40:34.794631    2575 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["e2e-test-llm-inference-service-63a03ef4/llmisvc-model-fb-opt-125m-route-dc21cb14-kserve-mn-0"]
Sep 17 18:40:34.795772 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:40:34.795745    2575 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="9341934b-0015-404b-81e3-9ef24cc0086a" containerName="storage-initializer"
Sep 17 18:40:34.795920 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:40:34.795908    2575 state_mem.go:107] "Deleted CPUSet assignment" podUID="9341934b-0015-404b-81e3-9ef24cc0086a" containerName="storage-initializer"
Sep 17 18:40:34.796019 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:40:34.796010    2575 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="9341934b-0015-404b-81e3-9ef24cc0086a" containerName="main"
Sep 17 18:40:34.796091 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:40:34.796083    2575 state_mem.go:107] "Deleted CPUSet assignment" podUID="9341934b-0015-404b-81e3-9ef24cc0086a" containerName="main"
Sep 17 18:40:34.796348 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:40:34.796333    2575 memory_manager.go:368] "RemoveStaleState removing state" podUID="9341934b-0015-404b-81e3-9ef24cc0086a" containerName="main"
Sep 17 18:40:34.800701 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:40:34.800675    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-test-llm-inference-service-63a03ef4/llmisvc-model-fb-opt-125m-route-dc21cb14-kserve-mn-0"
Sep 17 18:40:34.804943 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:40:34.804492    2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-test-llm-inference-service-63a03ef4\"/\"kube-root-ca.crt\""
Sep 17 18:40:34.804943 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:40:34.804601    2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-test-llm-inference-service-63a03ef4\"/\"openshift-service-ca.crt\""
Sep 17 18:40:34.804943 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:40:34.804751    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-test-llm-inference-service-63a03ef4\"/\"seaweedfs-s3-creds\""
Sep 17 18:40:34.804943 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:40:34.804817    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-test-llm-inference-service-63a03ef4\"/\"default-dockercfg-wfdbf\""
Sep 17 18:40:34.804943 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:40:34.804838    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-test-llm-inference-service-63a03ef4\"/\"llmisv32fd5468e8138ec3cd54cf7b9c18cfb0-kserve-self-signed-certs\""
Sep 17 18:40:34.804943 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:40:34.804885    2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-test-llm-inference-service-63a03ef4\"/\"odh-kserve-custom-ca-bundle\""
Sep 17 18:40:34.805477 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:40:34.805026    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-test-llm-inference-service-63a03ef4\"/\"llmisvc-model-fb-opt-125m-route-dc21cb-57cf7736-dockercfg-tpc9j\""
Sep 17 18:40:34.806692 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:40:34.806671    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["e2e-test-llm-inference-service-63a03ef4/llmisvc-model-fb-opt-125m-route-dc21cb14-kserve-mn-0"]
Sep 17 18:40:34.823019 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:40:34.822989    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-djvzt_1d07e68d-9943-420f-a6a0-5f477792fabd/konnectivity-agent/0.log" limit=1024
Sep 17 18:40:34.899493 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:40:34.899438    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/8fe22b46-e7d7-4925-8d01-756f99948d81-dshm\") pod \"llmisvc-model-fb-opt-125m-route-dc21cb14-kserve-mn-0\" (UID: \"8fe22b46-e7d7-4925-8d01-756f99948d81\") " pod="e2e-test-llm-inference-service-63a03ef4/llmisvc-model-fb-opt-125m-route-dc21cb14-kserve-mn-0"
Sep 17 18:40:34.899653 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:40:34.899501    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/8fe22b46-e7d7-4925-8d01-756f99948d81-kserve-provision-location\") pod \"llmisvc-model-fb-opt-125m-route-dc21cb14-kserve-mn-0\" (UID: \"8fe22b46-e7d7-4925-8d01-756f99948d81\") " pod="e2e-test-llm-inference-service-63a03ef4/llmisvc-model-fb-opt-125m-route-dc21cb14-kserve-mn-0"
Sep 17 18:40:34.899653 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:40:34.899536    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cabundle-cert\" (UniqueName: \"kubernetes.io/configmap/8fe22b46-e7d7-4925-8d01-756f99948d81-cabundle-cert\") pod \"llmisvc-model-fb-opt-125m-route-dc21cb14-kserve-mn-0\" (UID: \"8fe22b46-e7d7-4925-8d01-756f99948d81\") " pod="e2e-test-llm-inference-service-63a03ef4/llmisvc-model-fb-opt-125m-route-dc21cb14-kserve-mn-0"
Sep 17 18:40:34.899653 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:40:34.899562    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/8fe22b46-e7d7-4925-8d01-756f99948d81-home\") pod \"llmisvc-model-fb-opt-125m-route-dc21cb14-kserve-mn-0\" (UID: \"8fe22b46-e7d7-4925-8d01-756f99948d81\") " pod="e2e-test-llm-inference-service-63a03ef4/llmisvc-model-fb-opt-125m-route-dc21cb14-kserve-mn-0"
Sep 17 18:40:34.899653 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:40:34.899585    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-b4d9f\" (UniqueName: \"kubernetes.io/projected/8fe22b46-e7d7-4925-8d01-756f99948d81-kube-api-access-b4d9f\") pod \"llmisvc-model-fb-opt-125m-route-dc21cb14-kserve-mn-0\" (UID: \"8fe22b46-e7d7-4925-8d01-756f99948d81\") " pod="e2e-test-llm-inference-service-63a03ef4/llmisvc-model-fb-opt-125m-route-dc21cb14-kserve-mn-0"
Sep 17 18:40:34.899653 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:40:34.899644    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/8fe22b46-e7d7-4925-8d01-756f99948d81-tls-certs\") pod \"llmisvc-model-fb-opt-125m-route-dc21cb14-kserve-mn-0\" (UID: \"8fe22b46-e7d7-4925-8d01-756f99948d81\") " pod="e2e-test-llm-inference-service-63a03ef4/llmisvc-model-fb-opt-125m-route-dc21cb14-kserve-mn-0"
Sep 17 18:40:34.899862 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:40:34.899665    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/8fe22b46-e7d7-4925-8d01-756f99948d81-model-cache\") pod \"llmisvc-model-fb-opt-125m-route-dc21cb14-kserve-mn-0\" (UID: \"8fe22b46-e7d7-4925-8d01-756f99948d81\") " pod="e2e-test-llm-inference-service-63a03ef4/llmisvc-model-fb-opt-125m-route-dc21cb14-kserve-mn-0"
Sep 17 18:40:34.899862 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:40:34.899707    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/8fe22b46-e7d7-4925-8d01-756f99948d81-tmp-dir\") pod \"llmisvc-model-fb-opt-125m-route-dc21cb14-kserve-mn-0\" (UID: \"8fe22b46-e7d7-4925-8d01-756f99948d81\") " pod="e2e-test-llm-inference-service-63a03ef4/llmisvc-model-fb-opt-125m-route-dc21cb14-kserve-mn-0"
Sep 17 18:40:35.000404 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:40:35.000318    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/8fe22b46-e7d7-4925-8d01-756f99948d81-tls-certs\") pod \"llmisvc-model-fb-opt-125m-route-dc21cb14-kserve-mn-0\" (UID: \"8fe22b46-e7d7-4925-8d01-756f99948d81\") " pod="e2e-test-llm-inference-service-63a03ef4/llmisvc-model-fb-opt-125m-route-dc21cb14-kserve-mn-0"
Sep 17 18:40:35.000404 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:40:35.000357    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/8fe22b46-e7d7-4925-8d01-756f99948d81-model-cache\") pod \"llmisvc-model-fb-opt-125m-route-dc21cb14-kserve-mn-0\" (UID: \"8fe22b46-e7d7-4925-8d01-756f99948d81\") " pod="e2e-test-llm-inference-service-63a03ef4/llmisvc-model-fb-opt-125m-route-dc21cb14-kserve-mn-0"
Sep 17 18:40:35.000404 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:40:35.000406    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/8fe22b46-e7d7-4925-8d01-756f99948d81-tmp-dir\") pod \"llmisvc-model-fb-opt-125m-route-dc21cb14-kserve-mn-0\" (UID: \"8fe22b46-e7d7-4925-8d01-756f99948d81\") " pod="e2e-test-llm-inference-service-63a03ef4/llmisvc-model-fb-opt-125m-route-dc21cb14-kserve-mn-0"
Sep 17 18:40:35.000730 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:40:35.000471    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/8fe22b46-e7d7-4925-8d01-756f99948d81-dshm\") pod \"llmisvc-model-fb-opt-125m-route-dc21cb14-kserve-mn-0\" (UID: \"8fe22b46-e7d7-4925-8d01-756f99948d81\") " pod="e2e-test-llm-inference-service-63a03ef4/llmisvc-model-fb-opt-125m-route-dc21cb14-kserve-mn-0"
Sep 17 18:40:35.000730 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:40:35.000497    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/8fe22b46-e7d7-4925-8d01-756f99948d81-kserve-provision-location\") pod \"llmisvc-model-fb-opt-125m-route-dc21cb14-kserve-mn-0\" (UID: \"8fe22b46-e7d7-4925-8d01-756f99948d81\") " pod="e2e-test-llm-inference-service-63a03ef4/llmisvc-model-fb-opt-125m-route-dc21cb14-kserve-mn-0"
Sep 17 18:40:35.000730 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:40:35.000518    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cabundle-cert\" (UniqueName: \"kubernetes.io/configmap/8fe22b46-e7d7-4925-8d01-756f99948d81-cabundle-cert\") pod \"llmisvc-model-fb-opt-125m-route-dc21cb14-kserve-mn-0\" (UID: \"8fe22b46-e7d7-4925-8d01-756f99948d81\") " pod="e2e-test-llm-inference-service-63a03ef4/llmisvc-model-fb-opt-125m-route-dc21cb14-kserve-mn-0"
Sep 17 18:40:35.000730 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:40:35.000537    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/8fe22b46-e7d7-4925-8d01-756f99948d81-home\") pod \"llmisvc-model-fb-opt-125m-route-dc21cb14-kserve-mn-0\" (UID: \"8fe22b46-e7d7-4925-8d01-756f99948d81\") " pod="e2e-test-llm-inference-service-63a03ef4/llmisvc-model-fb-opt-125m-route-dc21cb14-kserve-mn-0"
Sep 17 18:40:35.000730 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:40:35.000623    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-b4d9f\" (UniqueName: \"kubernetes.io/projected/8fe22b46-e7d7-4925-8d01-756f99948d81-kube-api-access-b4d9f\") pod \"llmisvc-model-fb-opt-125m-route-dc21cb14-kserve-mn-0\" (UID: \"8fe22b46-e7d7-4925-8d01-756f99948d81\") " pod="e2e-test-llm-inference-service-63a03ef4/llmisvc-model-fb-opt-125m-route-dc21cb14-kserve-mn-0"
Sep 17 18:40:35.001000 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:40:35.000774    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/8fe22b46-e7d7-4925-8d01-756f99948d81-model-cache\") pod \"llmisvc-model-fb-opt-125m-route-dc21cb14-kserve-mn-0\" (UID: \"8fe22b46-e7d7-4925-8d01-756f99948d81\") " pod="e2e-test-llm-inference-service-63a03ef4/llmisvc-model-fb-opt-125m-route-dc21cb14-kserve-mn-0"
Sep 17 18:40:35.001000 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:40:35.000829    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/8fe22b46-e7d7-4925-8d01-756f99948d81-tmp-dir\") pod \"llmisvc-model-fb-opt-125m-route-dc21cb14-kserve-mn-0\" (UID: \"8fe22b46-e7d7-4925-8d01-756f99948d81\") " pod="e2e-test-llm-inference-service-63a03ef4/llmisvc-model-fb-opt-125m-route-dc21cb14-kserve-mn-0"
Sep 17 18:40:35.001094 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:40:35.000997    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/8fe22b46-e7d7-4925-8d01-756f99948d81-home\") pod \"llmisvc-model-fb-opt-125m-route-dc21cb14-kserve-mn-0\" (UID: \"8fe22b46-e7d7-4925-8d01-756f99948d81\") " pod="e2e-test-llm-inference-service-63a03ef4/llmisvc-model-fb-opt-125m-route-dc21cb14-kserve-mn-0"
Sep 17 18:40:35.001141 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:40:35.001104    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cabundle-cert\" (UniqueName: \"kubernetes.io/configmap/8fe22b46-e7d7-4925-8d01-756f99948d81-cabundle-cert\") pod \"llmisvc-model-fb-opt-125m-route-dc21cb14-kserve-mn-0\" (UID: \"8fe22b46-e7d7-4925-8d01-756f99948d81\") " pod="e2e-test-llm-inference-service-63a03ef4/llmisvc-model-fb-opt-125m-route-dc21cb14-kserve-mn-0"
Sep 17 18:40:35.001186 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:40:35.001148    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/8fe22b46-e7d7-4925-8d01-756f99948d81-kserve-provision-location\") pod \"llmisvc-model-fb-opt-125m-route-dc21cb14-kserve-mn-0\" (UID: \"8fe22b46-e7d7-4925-8d01-756f99948d81\") " pod="e2e-test-llm-inference-service-63a03ef4/llmisvc-model-fb-opt-125m-route-dc21cb14-kserve-mn-0"
Sep 17 18:40:35.002810 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:40:35.002789    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/8fe22b46-e7d7-4925-8d01-756f99948d81-dshm\") pod \"llmisvc-model-fb-opt-125m-route-dc21cb14-kserve-mn-0\" (UID: \"8fe22b46-e7d7-4925-8d01-756f99948d81\") " pod="e2e-test-llm-inference-service-63a03ef4/llmisvc-model-fb-opt-125m-route-dc21cb14-kserve-mn-0"
Sep 17 18:40:35.002926 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:40:35.002826    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/8fe22b46-e7d7-4925-8d01-756f99948d81-tls-certs\") pod \"llmisvc-model-fb-opt-125m-route-dc21cb14-kserve-mn-0\" (UID: \"8fe22b46-e7d7-4925-8d01-756f99948d81\") " pod="e2e-test-llm-inference-service-63a03ef4/llmisvc-model-fb-opt-125m-route-dc21cb14-kserve-mn-0"
Sep 17 18:40:35.008310 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:40:35.008288    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-b4d9f\" (UniqueName: \"kubernetes.io/projected/8fe22b46-e7d7-4925-8d01-756f99948d81-kube-api-access-b4d9f\") pod \"llmisvc-model-fb-opt-125m-route-dc21cb14-kserve-mn-0\" (UID: \"8fe22b46-e7d7-4925-8d01-756f99948d81\") " pod="e2e-test-llm-inference-service-63a03ef4/llmisvc-model-fb-opt-125m-route-dc21cb14-kserve-mn-0"
Sep 17 18:40:35.112504 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:40:35.112429    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-test-llm-inference-service-63a03ef4/llmisvc-model-fb-opt-125m-route-dc21cb14-kserve-mn-0"
Sep 17 18:40:35.398297 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:40:35.398266    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["e2e-test-llm-inference-service-63a03ef4/llmisvc-model-fb-opt-125m-route-dc21cb14-kserve-mn-0"]
Sep 17 18:40:35.399626 ip-10-0-143-74 kubenswrapper[2575]: W0917 18:40:35.399598    2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8fe22b46_e7d7_4925_8d01_756f99948d81.slice/crio-fe25c478bb412c5346d53b5b553f53523e3af2c5aa286f566dbde5890f656e1c WatchSource:0}: Error finding container fe25c478bb412c5346d53b5b553f53523e3af2c5aa286f566dbde5890f656e1c: Status 404 returned error can't find the container with id fe25c478bb412c5346d53b5b553f53523e3af2c5aa286f566dbde5890f656e1c
Sep 17 18:40:35.715727 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:40:35.715638    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-djvzt_1d07e68d-9943-420f-a6a0-5f477792fabd/konnectivity-agent/0.log" limit=1024
Sep 17 18:40:35.813805 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:40:35.813764    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-test-llm-inference-service-63a03ef4/llmisvc-model-fb-opt-125m-route-dc21cb14-kserve-mn-0" event={"ID":"8fe22b46-e7d7-4925-8d01-756f99948d81","Type":"ContainerStarted","Data":"22c8ef9b74a361e4451f9f7096b91a75781d03434c4be57ff2e84dc93cb1d814"}
Sep 17 18:40:35.813970 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:40:35.813828    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-test-llm-inference-service-63a03ef4/llmisvc-model-fb-opt-125m-route-dc21cb14-kserve-mn-0" event={"ID":"8fe22b46-e7d7-4925-8d01-756f99948d81","Type":"ContainerStarted","Data":"fe25c478bb412c5346d53b5b553f53523e3af2c5aa286f566dbde5890f656e1c"}
Sep 17 18:40:35.815474 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:40:35.815426    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-djvzt_1d07e68d-9943-420f-a6a0-5f477792fabd/konnectivity-agent/0.log" limit=1024
Sep 17 18:40:37.826807 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:40:37.826770    2575 generic.go:356] "Generic (PLEG): container finished" podID="8fe22b46-e7d7-4925-8d01-756f99948d81" containerID="22c8ef9b74a361e4451f9f7096b91a75781d03434c4be57ff2e84dc93cb1d814" exitCode=0
Sep 17 18:40:37.827221 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:40:37.826840    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-test-llm-inference-service-63a03ef4/llmisvc-model-fb-opt-125m-route-dc21cb14-kserve-mn-0" event={"ID":"8fe22b46-e7d7-4925-8d01-756f99948d81","Type":"ContainerDied","Data":"22c8ef9b74a361e4451f9f7096b91a75781d03434c4be57ff2e84dc93cb1d814"}
Sep 17 18:40:38.834107 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:40:38.834074    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-test-llm-inference-service-63a03ef4/llmisvc-model-fb-opt-125m-route-dc21cb14-kserve-mn-0" event={"ID":"8fe22b46-e7d7-4925-8d01-756f99948d81","Type":"ContainerStarted","Data":"c97a7a3b069f61850752d529df2c9ea69507b5b304040e62f72f90c457f605da"}
Sep 17 18:40:38.857519 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:40:38.857456    2575 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="e2e-test-llm-inference-service-63a03ef4/llmisvc-model-fb-opt-125m-route-dc21cb14-kserve-mn-0" podStartSLOduration=2.586573095 podStartE2EDuration="4.857419901s" totalImagesPullingTime="0s" totalInitContainerRuntime="2.270846806s" isStatefulPod=false podCreationTimestamp="2026-09-17 18:40:34 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-17 18:40:38.857321346 +0000 UTC m=+3958.487651321" watchObservedRunningTime="2026-09-17 18:40:38.857419901 +0000 UTC m=+3958.487749876"
Sep 17 18:40:39.112914 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:40:39.112836    2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-test-llm-inference-service-63a03ef4/llmisvc-model-fb-opt-125m-route-dc21cb14-kserve-mn-0"
Sep 17 18:40:45.113118 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:40:45.113077    2575 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="e2e-test-llm-inference-service-63a03ef4/llmisvc-model-fb-opt-125m-route-dc21cb14-kserve-mn-0"
Sep 17 18:40:45.114940 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:40:45.114907    2575 prober.go:120] "Probe failed" probeType="Startup" pod="e2e-test-llm-inference-service-63a03ef4/llmisvc-model-fb-opt-125m-route-dc21cb14-kserve-mn-0" podUID="8fe22b46-e7d7-4925-8d01-756f99948d81" containerName="main" probeResult="failure" output="Get \"https://10.133.0.65:8000/health\": dial tcp 10.133.0.65:8000: connect: connection refused"
Sep 17 18:40:47.495572 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:40:47.495539    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-djvzt_1d07e68d-9943-420f-a6a0-5f477792fabd/konnectivity-agent/0.log" limit=1024
Sep 17 18:40:55.113184 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:40:55.113139    2575 prober.go:120] "Probe failed" probeType="Startup" pod="e2e-test-llm-inference-service-63a03ef4/llmisvc-model-fb-opt-125m-route-dc21cb14-kserve-mn-0" podUID="8fe22b46-e7d7-4925-8d01-756f99948d81" containerName="main" probeResult="failure" output="Get \"https://10.133.0.65:8000/health\": dial tcp 10.133.0.65:8000: connect: connection refused"
Sep 17 18:41:05.112999 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:41:05.112949    2575 prober.go:120] "Probe failed" probeType="Startup" pod="e2e-test-llm-inference-service-63a03ef4/llmisvc-model-fb-opt-125m-route-dc21cb14-kserve-mn-0" podUID="8fe22b46-e7d7-4925-8d01-756f99948d81" containerName="main" probeResult="failure" output="Get \"https://10.133.0.65:8000/health\": dial tcp 10.133.0.65:8000: connect: connection refused"
Sep 17 18:41:15.113569 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:41:15.113521    2575 prober.go:120] "Probe failed" probeType="Startup" pod="e2e-test-llm-inference-service-63a03ef4/llmisvc-model-fb-opt-125m-route-dc21cb14-kserve-mn-0" podUID="8fe22b46-e7d7-4925-8d01-756f99948d81" containerName="main" probeResult="failure" output="Get \"https://10.133.0.65:8000/health\": dial tcp 10.133.0.65:8000: connect: connection refused"
Sep 17 18:41:16.840509 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:41:16.840471    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-djvzt_1d07e68d-9943-420f-a6a0-5f477792fabd/konnectivity-agent/0.log" limit=1024
Sep 17 18:41:25.113780 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:41:25.113735    2575 prober.go:120] "Probe failed" probeType="Startup" pod="e2e-test-llm-inference-service-63a03ef4/llmisvc-model-fb-opt-125m-route-dc21cb14-kserve-mn-0" podUID="8fe22b46-e7d7-4925-8d01-756f99948d81" containerName="main" probeResult="failure" output="Get \"https://10.133.0.65:8000/health\": dial tcp 10.133.0.65:8000: connect: connection refused"
Sep 17 18:41:35.113285 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:41:35.113228    2575 prober.go:120] "Probe failed" probeType="Startup" pod="e2e-test-llm-inference-service-63a03ef4/llmisvc-model-fb-opt-125m-route-dc21cb14-kserve-mn-0" podUID="8fe22b46-e7d7-4925-8d01-756f99948d81" containerName="main" probeResult="failure" output="Get \"https://10.133.0.65:8000/health\": dial tcp 10.133.0.65:8000: connect: connection refused"
Sep 17 18:41:36.926168 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:41:36.926130    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-djvzt_1d07e68d-9943-420f-a6a0-5f477792fabd/konnectivity-agent/0.log" limit=1024
Sep 17 18:41:37.030056 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:41:37.030006    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-djvzt_1d07e68d-9943-420f-a6a0-5f477792fabd/konnectivity-agent/0.log" limit=1024
Sep 17 18:41:37.130713 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:41:37.130663    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-djvzt_1d07e68d-9943-420f-a6a0-5f477792fabd/konnectivity-agent/0.log" limit=1024
Sep 17 18:41:38.932013 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:41:38.931972    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-djvzt_1d07e68d-9943-420f-a6a0-5f477792fabd/konnectivity-agent/0.log" limit=1024
Sep 17 18:41:39.185210 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:41:39.185120    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-djvzt_1d07e68d-9943-420f-a6a0-5f477792fabd/konnectivity-agent/0.log" limit=1024
Sep 17 18:41:42.380282 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:41:42.380224    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-djvzt_1d07e68d-9943-420f-a6a0-5f477792fabd/konnectivity-agent/0.log" limit=1024
Sep 17 18:41:42.480770 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:41:42.480731    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-djvzt_1d07e68d-9943-420f-a6a0-5f477792fabd/konnectivity-agent/0.log" limit=1024
Sep 17 18:41:43.209791 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:41:43.209755    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-djvzt_1d07e68d-9943-420f-a6a0-5f477792fabd/konnectivity-agent/0.log" limit=1024
Sep 17 18:41:45.113745 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:41:45.113693    2575 prober.go:120] "Probe failed" probeType="Startup" pod="e2e-test-llm-inference-service-63a03ef4/llmisvc-model-fb-opt-125m-route-dc21cb14-kserve-mn-0" podUID="8fe22b46-e7d7-4925-8d01-756f99948d81" containerName="main" probeResult="failure" output="Get \"https://10.133.0.65:8000/health\": dial tcp 10.133.0.65:8000: connect: connection refused"
Sep 17 18:41:45.585345 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:41:45.585309    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-djvzt_1d07e68d-9943-420f-a6a0-5f477792fabd/konnectivity-agent/0.log" limit=1024
Sep 17 18:41:45.665343 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:41:45.665290    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-djvzt_1d07e68d-9943-420f-a6a0-5f477792fabd/konnectivity-agent/0.log" limit=1024
Sep 17 18:41:48.988998 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:41:48.988963    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-djvzt_1d07e68d-9943-420f-a6a0-5f477792fabd/konnectivity-agent/0.log" limit=1024
Sep 17 18:41:49.089772 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:41:49.089735    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-djvzt_1d07e68d-9943-420f-a6a0-5f477792fabd/konnectivity-agent/0.log" limit=1024
Sep 17 18:41:55.039580 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:41:55.039515    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-djvzt_1d07e68d-9943-420f-a6a0-5f477792fabd/konnectivity-agent/0.log" limit=1024
Sep 17 18:41:55.113207 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:41:55.113164    2575 prober.go:120] "Probe failed" probeType="Startup" pod="e2e-test-llm-inference-service-63a03ef4/llmisvc-model-fb-opt-125m-route-dc21cb14-kserve-mn-0" podUID="8fe22b46-e7d7-4925-8d01-756f99948d81" containerName="main" probeResult="failure" output="Get \"https://10.133.0.65:8000/health\": dial tcp 10.133.0.65:8000: connect: connection refused"
Sep 17 18:42:05.113727 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:42:05.113624    2575 prober.go:120] "Probe failed" probeType="Startup" pod="e2e-test-llm-inference-service-63a03ef4/llmisvc-model-fb-opt-125m-route-dc21cb14-kserve-mn-0" podUID="8fe22b46-e7d7-4925-8d01-756f99948d81" containerName="main" probeResult="failure" output="Get \"https://10.133.0.65:8000/health\": dial tcp 10.133.0.65:8000: connect: connection refused"
Sep 17 18:42:15.122917 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:42:15.122877    2575 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="e2e-test-llm-inference-service-63a03ef4/llmisvc-model-fb-opt-125m-route-dc21cb14-kserve-mn-0"
Sep 17 18:42:15.130599 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:42:15.130576    2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-test-llm-inference-service-63a03ef4/llmisvc-model-fb-opt-125m-route-dc21cb14-kserve-mn-0"
Sep 17 18:42:21.071798 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:42:21.071763    2575 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["e2e-test-llm-inference-service-63a03ef4/llmisvc-model-fb-opt-125m-route-dc21cb14-kserve-mn-0"]
Sep 17 18:42:21.087939 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:42:21.087908    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-djvzt_1d07e68d-9943-420f-a6a0-5f477792fabd/konnectivity-agent/0.log" limit=1024
Sep 17 18:42:21.188568 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:42:21.188535    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-djvzt_1d07e68d-9943-420f-a6a0-5f477792fabd/konnectivity-agent/0.log" limit=1024
Sep 17 18:42:36.094709 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:42:36.094657    2575 kuberuntime_container.go:873] "Killing container with a grace period" pod="e2e-test-llm-inference-service-63a03ef4/llmisvc-model-fb-opt-125m-route-dc21cb14-kserve-mn-0" podUID="8fe22b46-e7d7-4925-8d01-756f99948d81" containerName="main" containerID="cri-o://c97a7a3b069f61850752d529df2c9ea69507b5b304040e62f72f90c457f605da" gracePeriod=45
Sep 17 18:42:36.846927 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:42:36.846894    2575 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-test-llm-inference-service-63a03ef4/llmisvc-model-fb-opt-125m-route-dc21cb14-kserve-mn-0"
Sep 17 18:42:36.993680 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:42:36.993598    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-b4d9f\" (UniqueName: \"kubernetes.io/projected/8fe22b46-e7d7-4925-8d01-756f99948d81-kube-api-access-b4d9f\") pod \"8fe22b46-e7d7-4925-8d01-756f99948d81\" (UID: \"8fe22b46-e7d7-4925-8d01-756f99948d81\") "
Sep 17 18:42:36.993680 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:42:36.993631    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"cabundle-cert\" (UniqueName: \"kubernetes.io/configmap/8fe22b46-e7d7-4925-8d01-756f99948d81-cabundle-cert\") pod \"8fe22b46-e7d7-4925-8d01-756f99948d81\" (UID: \"8fe22b46-e7d7-4925-8d01-756f99948d81\") "
Sep 17 18:42:36.993858 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:42:36.993689    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/8fe22b46-e7d7-4925-8d01-756f99948d81-dshm\") pod \"8fe22b46-e7d7-4925-8d01-756f99948d81\" (UID: \"8fe22b46-e7d7-4925-8d01-756f99948d81\") "
Sep 17 18:42:36.993858 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:42:36.993713    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/8fe22b46-e7d7-4925-8d01-756f99948d81-tls-certs\") pod \"8fe22b46-e7d7-4925-8d01-756f99948d81\" (UID: \"8fe22b46-e7d7-4925-8d01-756f99948d81\") "
Sep 17 18:42:36.993858 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:42:36.993748    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/8fe22b46-e7d7-4925-8d01-756f99948d81-tmp-dir\") pod \"8fe22b46-e7d7-4925-8d01-756f99948d81\" (UID: \"8fe22b46-e7d7-4925-8d01-756f99948d81\") "
Sep 17 18:42:36.993858 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:42:36.993825    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/8fe22b46-e7d7-4925-8d01-756f99948d81-home\") pod \"8fe22b46-e7d7-4925-8d01-756f99948d81\" (UID: \"8fe22b46-e7d7-4925-8d01-756f99948d81\") "
Sep 17 18:42:36.993858 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:42:36.993850    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/8fe22b46-e7d7-4925-8d01-756f99948d81-kserve-provision-location\") pod \"8fe22b46-e7d7-4925-8d01-756f99948d81\" (UID: \"8fe22b46-e7d7-4925-8d01-756f99948d81\") "
Sep 17 18:42:36.994103 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:42:36.993874    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/8fe22b46-e7d7-4925-8d01-756f99948d81-model-cache\") pod \"8fe22b46-e7d7-4925-8d01-756f99948d81\" (UID: \"8fe22b46-e7d7-4925-8d01-756f99948d81\") "
Sep 17 18:42:36.994196 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:42:36.994081    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/8fe22b46-e7d7-4925-8d01-756f99948d81-cabundle-cert" (OuterVolumeSpecName: "cabundle-cert") pod "8fe22b46-e7d7-4925-8d01-756f99948d81" (UID: "8fe22b46-e7d7-4925-8d01-756f99948d81"). InnerVolumeSpecName "cabundle-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 17 18:42:36.994353 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:42:36.994325    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/8fe22b46-e7d7-4925-8d01-756f99948d81-model-cache" (OuterVolumeSpecName: "model-cache") pod "8fe22b46-e7d7-4925-8d01-756f99948d81" (UID: "8fe22b46-e7d7-4925-8d01-756f99948d81"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 17 18:42:36.994783 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:42:36.994755    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/8fe22b46-e7d7-4925-8d01-756f99948d81-home" (OuterVolumeSpecName: "home") pod "8fe22b46-e7d7-4925-8d01-756f99948d81" (UID: "8fe22b46-e7d7-4925-8d01-756f99948d81"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 17 18:42:36.996051 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:42:36.996026    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/8fe22b46-e7d7-4925-8d01-756f99948d81-kube-api-access-b4d9f" (OuterVolumeSpecName: "kube-api-access-b4d9f") pod "8fe22b46-e7d7-4925-8d01-756f99948d81" (UID: "8fe22b46-e7d7-4925-8d01-756f99948d81"). InnerVolumeSpecName "kube-api-access-b4d9f". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 17 18:42:36.996051 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:42:36.996036    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/8fe22b46-e7d7-4925-8d01-756f99948d81-dshm" (OuterVolumeSpecName: "dshm") pod "8fe22b46-e7d7-4925-8d01-756f99948d81" (UID: "8fe22b46-e7d7-4925-8d01-756f99948d81"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 17 18:42:36.996198 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:42:36.996127    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/8fe22b46-e7d7-4925-8d01-756f99948d81-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "8fe22b46-e7d7-4925-8d01-756f99948d81" (UID: "8fe22b46-e7d7-4925-8d01-756f99948d81"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 17 18:42:37.005487 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:42:37.005461    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/8fe22b46-e7d7-4925-8d01-756f99948d81-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "8fe22b46-e7d7-4925-8d01-756f99948d81" (UID: "8fe22b46-e7d7-4925-8d01-756f99948d81"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 17 18:42:37.025206 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:42:37.025173    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/8fe22b46-e7d7-4925-8d01-756f99948d81-kserve-provision-location" (OuterVolumeSpecName: "kserve-provision-location") pod "8fe22b46-e7d7-4925-8d01-756f99948d81" (UID: "8fe22b46-e7d7-4925-8d01-756f99948d81"). InnerVolumeSpecName "kserve-provision-location". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 17 18:42:37.095331 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:42:37.095305    2575 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/8fe22b46-e7d7-4925-8d01-756f99948d81-home\") on node \"ip-10-0-143-74.ec2.internal\" DevicePath \"\""
Sep 17 18:42:37.095331 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:42:37.095332    2575 reconciler_common.go:299] "Volume detached for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/8fe22b46-e7d7-4925-8d01-756f99948d81-kserve-provision-location\") on node \"ip-10-0-143-74.ec2.internal\" DevicePath \"\""
Sep 17 18:42:37.095687 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:42:37.095342    2575 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/8fe22b46-e7d7-4925-8d01-756f99948d81-model-cache\") on node \"ip-10-0-143-74.ec2.internal\" DevicePath \"\""
Sep 17 18:42:37.095687 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:42:37.095351    2575 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-b4d9f\" (UniqueName: \"kubernetes.io/projected/8fe22b46-e7d7-4925-8d01-756f99948d81-kube-api-access-b4d9f\") on node \"ip-10-0-143-74.ec2.internal\" DevicePath \"\""
Sep 17 18:42:37.095687 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:42:37.095360    2575 reconciler_common.go:299] "Volume detached for volume \"cabundle-cert\" (UniqueName: \"kubernetes.io/configmap/8fe22b46-e7d7-4925-8d01-756f99948d81-cabundle-cert\") on node \"ip-10-0-143-74.ec2.internal\" DevicePath \"\""
Sep 17 18:42:37.095687 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:42:37.095368    2575 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/8fe22b46-e7d7-4925-8d01-756f99948d81-dshm\") on node \"ip-10-0-143-74.ec2.internal\" DevicePath \"\""
Sep 17 18:42:37.095687 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:42:37.095376    2575 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/8fe22b46-e7d7-4925-8d01-756f99948d81-tls-certs\") on node \"ip-10-0-143-74.ec2.internal\" DevicePath \"\""
Sep 17 18:42:37.095687 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:42:37.095383    2575 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/8fe22b46-e7d7-4925-8d01-756f99948d81-tmp-dir\") on node \"ip-10-0-143-74.ec2.internal\" DevicePath \"\""
Sep 17 18:42:37.545813 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:42:37.545774    2575 generic.go:356] "Generic (PLEG): container finished" podID="8fe22b46-e7d7-4925-8d01-756f99948d81" containerID="c97a7a3b069f61850752d529df2c9ea69507b5b304040e62f72f90c457f605da" exitCode=0
Sep 17 18:42:37.545984 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:42:37.545831    2575 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-test-llm-inference-service-63a03ef4/llmisvc-model-fb-opt-125m-route-dc21cb14-kserve-mn-0"
Sep 17 18:42:37.545984 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:42:37.545849    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-test-llm-inference-service-63a03ef4/llmisvc-model-fb-opt-125m-route-dc21cb14-kserve-mn-0" event={"ID":"8fe22b46-e7d7-4925-8d01-756f99948d81","Type":"ContainerDied","Data":"c97a7a3b069f61850752d529df2c9ea69507b5b304040e62f72f90c457f605da"}
Sep 17 18:42:37.545984 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:42:37.545908    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-test-llm-inference-service-63a03ef4/llmisvc-model-fb-opt-125m-route-dc21cb14-kserve-mn-0" event={"ID":"8fe22b46-e7d7-4925-8d01-756f99948d81","Type":"ContainerDied","Data":"fe25c478bb412c5346d53b5b553f53523e3af2c5aa286f566dbde5890f656e1c"}
Sep 17 18:42:37.545984 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:42:37.545930    2575 scope.go:117] "RemoveContainer" containerID="c97a7a3b069f61850752d529df2c9ea69507b5b304040e62f72f90c457f605da"
Sep 17 18:42:37.558117 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:42:37.558097    2575 scope.go:117] "RemoveContainer" containerID="22c8ef9b74a361e4451f9f7096b91a75781d03434c4be57ff2e84dc93cb1d814"
Sep 17 18:42:37.566232 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:42:37.566210    2575 scope.go:117] "RemoveContainer" containerID="c97a7a3b069f61850752d529df2c9ea69507b5b304040e62f72f90c457f605da"
Sep 17 18:42:37.566491 ip-10-0-143-74 kubenswrapper[2575]: E0917 18:42:37.566470    2575 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"c97a7a3b069f61850752d529df2c9ea69507b5b304040e62f72f90c457f605da\": container with ID starting with c97a7a3b069f61850752d529df2c9ea69507b5b304040e62f72f90c457f605da not found: ID does not exist" containerID="c97a7a3b069f61850752d529df2c9ea69507b5b304040e62f72f90c457f605da"
Sep 17 18:42:37.566541 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:42:37.566499    2575 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"c97a7a3b069f61850752d529df2c9ea69507b5b304040e62f72f90c457f605da"} err="failed to get container status \"c97a7a3b069f61850752d529df2c9ea69507b5b304040e62f72f90c457f605da\": rpc error: code = NotFound desc = could not find container \"c97a7a3b069f61850752d529df2c9ea69507b5b304040e62f72f90c457f605da\": container with ID starting with c97a7a3b069f61850752d529df2c9ea69507b5b304040e62f72f90c457f605da not found: ID does not exist"
Sep 17 18:42:37.566541 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:42:37.566516    2575 scope.go:117] "RemoveContainer" containerID="22c8ef9b74a361e4451f9f7096b91a75781d03434c4be57ff2e84dc93cb1d814"
Sep 17 18:42:37.566730 ip-10-0-143-74 kubenswrapper[2575]: E0917 18:42:37.566715    2575 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"22c8ef9b74a361e4451f9f7096b91a75781d03434c4be57ff2e84dc93cb1d814\": container with ID starting with 22c8ef9b74a361e4451f9f7096b91a75781d03434c4be57ff2e84dc93cb1d814 not found: ID does not exist" containerID="22c8ef9b74a361e4451f9f7096b91a75781d03434c4be57ff2e84dc93cb1d814"
Sep 17 18:42:37.566764 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:42:37.566733    2575 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"22c8ef9b74a361e4451f9f7096b91a75781d03434c4be57ff2e84dc93cb1d814"} err="failed to get container status \"22c8ef9b74a361e4451f9f7096b91a75781d03434c4be57ff2e84dc93cb1d814\": rpc error: code = NotFound desc = could not find container \"22c8ef9b74a361e4451f9f7096b91a75781d03434c4be57ff2e84dc93cb1d814\": container with ID starting with 22c8ef9b74a361e4451f9f7096b91a75781d03434c4be57ff2e84dc93cb1d814 not found: ID does not exist"
Sep 17 18:42:37.571424 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:42:37.571401    2575 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["e2e-test-llm-inference-service-63a03ef4/llmisvc-model-fb-opt-125m-route-dc21cb14-kserve-mn-0"]
Sep 17 18:42:37.575268 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:42:37.575242    2575 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["e2e-test-llm-inference-service-63a03ef4/llmisvc-model-fb-opt-125m-route-dc21cb14-kserve-mn-0"]
Sep 17 18:42:38.967403 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:42:38.967369    2575 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="8fe22b46-e7d7-4925-8d01-756f99948d81" path="/var/lib/kubelet/pods/8fe22b46-e7d7-4925-8d01-756f99948d81/volumes"
Sep 17 18:43:03.190516 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:43:03.190476    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-djvzt_1d07e68d-9943-420f-a6a0-5f477792fabd/konnectivity-agent/0.log" limit=1024
Sep 17 18:43:11.055942 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:43:11.055908    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-djvzt_1d07e68d-9943-420f-a6a0-5f477792fabd/konnectivity-agent/0.log" limit=1024
Sep 17 18:43:11.146348 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:43:11.146296    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-djvzt_1d07e68d-9943-420f-a6a0-5f477792fabd/konnectivity-agent/0.log" limit=1024
Sep 17 18:43:14.092554 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:43:14.092519    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-djvzt_1d07e68d-9943-420f-a6a0-5f477792fabd/konnectivity-agent/0.log" limit=1024
Sep 17 18:43:17.496091 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:43:17.496051    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-djvzt_1d07e68d-9943-420f-a6a0-5f477792fabd/konnectivity-agent/0.log" limit=1024
Sep 17 18:43:20.695301 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:43:20.695267    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-djvzt_1d07e68d-9943-420f-a6a0-5f477792fabd/konnectivity-agent/0.log" limit=1024
Sep 17 18:43:20.883876 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:43:20.883845    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-djvzt_1d07e68d-9943-420f-a6a0-5f477792fabd/konnectivity-agent/0.log" limit=1024
Sep 17 18:43:23.629666 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:43:23.629627    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-djvzt_1d07e68d-9943-420f-a6a0-5f477792fabd/konnectivity-agent/0.log" limit=1024
Sep 17 18:43:23.730720 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:43:23.730682    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-djvzt_1d07e68d-9943-420f-a6a0-5f477792fabd/konnectivity-agent/0.log" limit=1024
Sep 17 18:43:36.880272 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:43:36.880185    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-djvzt_1d07e68d-9943-420f-a6a0-5f477792fabd/konnectivity-agent/0.log" limit=1024
Sep 17 18:43:44.083453 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:43:44.083400    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-djvzt_1d07e68d-9943-420f-a6a0-5f477792fabd/konnectivity-agent/0.log" limit=1024
Sep 17 18:43:44.152857 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:43:44.152824    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-djvzt_1d07e68d-9943-420f-a6a0-5f477792fabd/konnectivity-agent/0.log" limit=1024
Sep 17 18:43:54.930644 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:43:54.930605    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-djvzt_1d07e68d-9943-420f-a6a0-5f477792fabd/konnectivity-agent/0.log" limit=1024
Sep 17 18:43:55.025207 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:43:55.025176    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-djvzt_1d07e68d-9943-420f-a6a0-5f477792fabd/konnectivity-agent/0.log" limit=1024
Sep 17 18:44:34.188813 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:44:34.188775    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-djvzt_1d07e68d-9943-420f-a6a0-5f477792fabd/konnectivity-agent/0.log" limit=1024
Sep 17 18:44:34.292778 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:44:34.292741    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-djvzt_1d07e68d-9943-420f-a6a0-5f477792fabd/konnectivity-agent/0.log" limit=1024
Sep 17 18:44:40.384633 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:44:40.384589    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-djvzt_1d07e68d-9943-420f-a6a0-5f477792fabd/konnectivity-agent/0.log" limit=1024
Sep 17 18:44:44.309749 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:44:44.309712    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-djvzt_1d07e68d-9943-420f-a6a0-5f477792fabd/konnectivity-agent/0.log" limit=1024
Sep 17 18:44:44.409808 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:44:44.409772    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-djvzt_1d07e68d-9943-420f-a6a0-5f477792fabd/konnectivity-agent/0.log" limit=1024
Sep 17 18:44:45.502433 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:44:45.502399    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-djvzt_1d07e68d-9943-420f-a6a0-5f477792fabd/konnectivity-agent/0.log" limit=1024
Sep 17 18:44:46.104151 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:44:46.104112    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-djvzt_1d07e68d-9943-420f-a6a0-5f477792fabd/konnectivity-agent/0.log" limit=1024
Sep 17 18:44:48.399099 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:44:48.399060    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-djvzt_1d07e68d-9943-420f-a6a0-5f477792fabd/konnectivity-agent/0.log" limit=1024
Sep 17 18:44:48.499529 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:44:48.499499    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-djvzt_1d07e68d-9943-420f-a6a0-5f477792fabd/konnectivity-agent/0.log" limit=1024
Sep 17 18:44:50.881825 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:44:50.881793    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-djvzt_1d07e68d-9943-420f-a6a0-5f477792fabd/konnectivity-agent/0.log" limit=1024
Sep 17 18:44:53.842338 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:44:53.842287    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-djvzt_1d07e68d-9943-420f-a6a0-5f477792fabd/konnectivity-agent/0.log" limit=1024
Sep 17 18:44:56.502687 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:44:56.502643    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-djvzt_1d07e68d-9943-420f-a6a0-5f477792fabd/konnectivity-agent/0.log" limit=1024
Sep 17 18:44:56.574264 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:44:56.574236    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-djvzt_1d07e68d-9943-420f-a6a0-5f477792fabd/konnectivity-agent/0.log" limit=1024
Sep 17 18:44:56.930657 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:44:56.930617    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-djvzt_1d07e68d-9943-420f-a6a0-5f477792fabd/konnectivity-agent/0.log" limit=1024
Sep 17 18:44:57.133398 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:44:57.133363    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-djvzt_1d07e68d-9943-420f-a6a0-5f477792fabd/konnectivity-agent/0.log" limit=1024
Sep 17 18:45:08.226717 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:45:08.226680    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-djvzt_1d07e68d-9943-420f-a6a0-5f477792fabd/konnectivity-agent/0.log" limit=1024
Sep 17 18:45:30.232190 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:45:30.232151    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-djvzt_1d07e68d-9943-420f-a6a0-5f477792fabd/konnectivity-agent/0.log" limit=1024
Sep 17 18:45:30.327745 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:45:30.327710    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-djvzt_1d07e68d-9943-420f-a6a0-5f477792fabd/konnectivity-agent/0.log" limit=1024
Sep 17 18:45:38.307071 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:45:38.307032    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-djvzt_1d07e68d-9943-420f-a6a0-5f477792fabd/konnectivity-agent/0.log" limit=1024
Sep 17 18:46:20.310068 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:46:20.310030    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-djvzt_1d07e68d-9943-420f-a6a0-5f477792fabd/konnectivity-agent/0.log" limit=1024
Sep 17 18:46:20.405347 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:46:20.405313    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-djvzt_1d07e68d-9943-420f-a6a0-5f477792fabd/konnectivity-agent/0.log" limit=1024
Sep 17 18:46:20.940973 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:46:20.940938    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-djvzt_1d07e68d-9943-420f-a6a0-5f477792fabd/konnectivity-agent/0.log" limit=1024
Sep 17 18:46:24.686330 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:46:24.686295    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-djvzt_1d07e68d-9943-420f-a6a0-5f477792fabd/konnectivity-agent/0.log" limit=1024
Sep 17 18:46:24.786728 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:46:24.786698    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-djvzt_1d07e68d-9943-420f-a6a0-5f477792fabd/konnectivity-agent/0.log" limit=1024
Sep 17 18:46:27.122483 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:46:27.122427    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-djvzt_1d07e68d-9943-420f-a6a0-5f477792fabd/konnectivity-agent/0.log" limit=1024
Sep 17 18:46:33.100014 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:46:33.099979    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-djvzt_1d07e68d-9943-420f-a6a0-5f477792fabd/konnectivity-agent/0.log" limit=1024
Sep 17 18:46:33.160515 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:46:33.160485    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-djvzt_1d07e68d-9943-420f-a6a0-5f477792fabd/konnectivity-agent/0.log" limit=1024
Sep 17 18:46:41.899796 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:46:41.899755    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-djvzt_1d07e68d-9943-420f-a6a0-5f477792fabd/konnectivity-agent/0.log" limit=1024
Sep 17 18:46:42.087358 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:46:42.087325    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-djvzt_1d07e68d-9943-420f-a6a0-5f477792fabd/konnectivity-agent/0.log" limit=1024
Sep 17 18:46:49.936185 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:46:49.936146    2575 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["e2e-test-prestop-hook-22727e61/prestop-hook-test-kserve-5f6b9b8bc6-nmsvm"]
Sep 17 18:46:49.936583 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:46:49.936531    2575 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="8fe22b46-e7d7-4925-8d01-756f99948d81" containerName="main"
Sep 17 18:46:49.936583 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:46:49.936542    2575 state_mem.go:107] "Deleted CPUSet assignment" podUID="8fe22b46-e7d7-4925-8d01-756f99948d81" containerName="main"
Sep 17 18:46:49.936583 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:46:49.936557    2575 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="8fe22b46-e7d7-4925-8d01-756f99948d81" containerName="storage-initializer"
Sep 17 18:46:49.936583 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:46:49.936563    2575 state_mem.go:107] "Deleted CPUSet assignment" podUID="8fe22b46-e7d7-4925-8d01-756f99948d81" containerName="storage-initializer"
Sep 17 18:46:49.936719 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:46:49.936622    2575 memory_manager.go:368] "RemoveStaleState removing state" podUID="8fe22b46-e7d7-4925-8d01-756f99948d81" containerName="main"
Sep 17 18:46:49.939649 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:46:49.939631    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-test-prestop-hook-22727e61/prestop-hook-test-kserve-5f6b9b8bc6-nmsvm"
Sep 17 18:46:49.942623 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:46:49.942601    2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-test-prestop-hook-22727e61\"/\"kube-root-ca.crt\""
Sep 17 18:46:49.942729 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:46:49.942639    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-test-prestop-hook-22727e61\"/\"prestop-hook-test-kserve-self-signed-certs\""
Sep 17 18:46:49.943025 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:46:49.943005    2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-test-prestop-hook-22727e61\"/\"openshift-service-ca.crt\""
Sep 17 18:46:49.943118 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:46:49.943007    2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-test-prestop-hook-22727e61\"/\"odh-kserve-custom-ca-bundle\""
Sep 17 18:46:49.943118 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:46:49.943009    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-test-prestop-hook-22727e61\"/\"seaweedfs-s3-creds\""
Sep 17 18:46:49.943836 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:46:49.943819    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-test-prestop-hook-22727e61\"/\"default-dockercfg-65nmv\""
Sep 17 18:46:49.950736 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:46:49.950703    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["e2e-test-prestop-hook-22727e61/prestop-hook-test-kserve-5f6b9b8bc6-nmsvm"]
Sep 17 18:46:50.063363 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:46:50.063329    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cabundle-cert\" (UniqueName: \"kubernetes.io/configmap/7a9d2ee5-9aa4-4778-a7ea-3e46ebf9cf8d-cabundle-cert\") pod \"prestop-hook-test-kserve-5f6b9b8bc6-nmsvm\" (UID: \"7a9d2ee5-9aa4-4778-a7ea-3e46ebf9cf8d\") " pod="e2e-test-prestop-hook-22727e61/prestop-hook-test-kserve-5f6b9b8bc6-nmsvm"
Sep 17 18:46:50.063592 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:46:50.063388    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/7a9d2ee5-9aa4-4778-a7ea-3e46ebf9cf8d-home\") pod \"prestop-hook-test-kserve-5f6b9b8bc6-nmsvm\" (UID: \"7a9d2ee5-9aa4-4778-a7ea-3e46ebf9cf8d\") " pod="e2e-test-prestop-hook-22727e61/prestop-hook-test-kserve-5f6b9b8bc6-nmsvm"
Sep 17 18:46:50.063592 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:46:50.063415    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/7a9d2ee5-9aa4-4778-a7ea-3e46ebf9cf8d-dshm\") pod \"prestop-hook-test-kserve-5f6b9b8bc6-nmsvm\" (UID: \"7a9d2ee5-9aa4-4778-a7ea-3e46ebf9cf8d\") " pod="e2e-test-prestop-hook-22727e61/prestop-hook-test-kserve-5f6b9b8bc6-nmsvm"
Sep 17 18:46:50.063592 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:46:50.063436    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-nhsrf\" (UniqueName: \"kubernetes.io/projected/7a9d2ee5-9aa4-4778-a7ea-3e46ebf9cf8d-kube-api-access-nhsrf\") pod \"prestop-hook-test-kserve-5f6b9b8bc6-nmsvm\" (UID: \"7a9d2ee5-9aa4-4778-a7ea-3e46ebf9cf8d\") " pod="e2e-test-prestop-hook-22727e61/prestop-hook-test-kserve-5f6b9b8bc6-nmsvm"
Sep 17 18:46:50.063592 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:46:50.063556    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/7a9d2ee5-9aa4-4778-a7ea-3e46ebf9cf8d-tls-certs\") pod \"prestop-hook-test-kserve-5f6b9b8bc6-nmsvm\" (UID: \"7a9d2ee5-9aa4-4778-a7ea-3e46ebf9cf8d\") " pod="e2e-test-prestop-hook-22727e61/prestop-hook-test-kserve-5f6b9b8bc6-nmsvm"
Sep 17 18:46:50.063813 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:46:50.063605    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/7a9d2ee5-9aa4-4778-a7ea-3e46ebf9cf8d-kserve-provision-location\") pod \"prestop-hook-test-kserve-5f6b9b8bc6-nmsvm\" (UID: \"7a9d2ee5-9aa4-4778-a7ea-3e46ebf9cf8d\") " pod="e2e-test-prestop-hook-22727e61/prestop-hook-test-kserve-5f6b9b8bc6-nmsvm"
Sep 17 18:46:50.063813 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:46:50.063637    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/7a9d2ee5-9aa4-4778-a7ea-3e46ebf9cf8d-model-cache\") pod \"prestop-hook-test-kserve-5f6b9b8bc6-nmsvm\" (UID: \"7a9d2ee5-9aa4-4778-a7ea-3e46ebf9cf8d\") " pod="e2e-test-prestop-hook-22727e61/prestop-hook-test-kserve-5f6b9b8bc6-nmsvm"
Sep 17 18:46:50.063813 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:46:50.063719    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/7a9d2ee5-9aa4-4778-a7ea-3e46ebf9cf8d-tmp-dir\") pod \"prestop-hook-test-kserve-5f6b9b8bc6-nmsvm\" (UID: \"7a9d2ee5-9aa4-4778-a7ea-3e46ebf9cf8d\") " pod="e2e-test-prestop-hook-22727e61/prestop-hook-test-kserve-5f6b9b8bc6-nmsvm"
Sep 17 18:46:50.164286 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:46:50.164249    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/7a9d2ee5-9aa4-4778-a7ea-3e46ebf9cf8d-home\") pod \"prestop-hook-test-kserve-5f6b9b8bc6-nmsvm\" (UID: \"7a9d2ee5-9aa4-4778-a7ea-3e46ebf9cf8d\") " pod="e2e-test-prestop-hook-22727e61/prestop-hook-test-kserve-5f6b9b8bc6-nmsvm"
Sep 17 18:46:50.164286 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:46:50.164286    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/7a9d2ee5-9aa4-4778-a7ea-3e46ebf9cf8d-dshm\") pod \"prestop-hook-test-kserve-5f6b9b8bc6-nmsvm\" (UID: \"7a9d2ee5-9aa4-4778-a7ea-3e46ebf9cf8d\") " pod="e2e-test-prestop-hook-22727e61/prestop-hook-test-kserve-5f6b9b8bc6-nmsvm"
Sep 17 18:46:50.164556 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:46:50.164316    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-nhsrf\" (UniqueName: \"kubernetes.io/projected/7a9d2ee5-9aa4-4778-a7ea-3e46ebf9cf8d-kube-api-access-nhsrf\") pod \"prestop-hook-test-kserve-5f6b9b8bc6-nmsvm\" (UID: \"7a9d2ee5-9aa4-4778-a7ea-3e46ebf9cf8d\") " pod="e2e-test-prestop-hook-22727e61/prestop-hook-test-kserve-5f6b9b8bc6-nmsvm"
Sep 17 18:46:50.164556 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:46:50.164356    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/7a9d2ee5-9aa4-4778-a7ea-3e46ebf9cf8d-tls-certs\") pod \"prestop-hook-test-kserve-5f6b9b8bc6-nmsvm\" (UID: \"7a9d2ee5-9aa4-4778-a7ea-3e46ebf9cf8d\") " pod="e2e-test-prestop-hook-22727e61/prestop-hook-test-kserve-5f6b9b8bc6-nmsvm"
Sep 17 18:46:50.164556 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:46:50.164376    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/7a9d2ee5-9aa4-4778-a7ea-3e46ebf9cf8d-kserve-provision-location\") pod \"prestop-hook-test-kserve-5f6b9b8bc6-nmsvm\" (UID: \"7a9d2ee5-9aa4-4778-a7ea-3e46ebf9cf8d\") " pod="e2e-test-prestop-hook-22727e61/prestop-hook-test-kserve-5f6b9b8bc6-nmsvm"
Sep 17 18:46:50.164556 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:46:50.164406    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/7a9d2ee5-9aa4-4778-a7ea-3e46ebf9cf8d-model-cache\") pod \"prestop-hook-test-kserve-5f6b9b8bc6-nmsvm\" (UID: \"7a9d2ee5-9aa4-4778-a7ea-3e46ebf9cf8d\") " pod="e2e-test-prestop-hook-22727e61/prestop-hook-test-kserve-5f6b9b8bc6-nmsvm"
Sep 17 18:46:50.164556 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:46:50.164483    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/7a9d2ee5-9aa4-4778-a7ea-3e46ebf9cf8d-tmp-dir\") pod \"prestop-hook-test-kserve-5f6b9b8bc6-nmsvm\" (UID: \"7a9d2ee5-9aa4-4778-a7ea-3e46ebf9cf8d\") " pod="e2e-test-prestop-hook-22727e61/prestop-hook-test-kserve-5f6b9b8bc6-nmsvm"
Sep 17 18:46:50.164556 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:46:50.164523    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cabundle-cert\" (UniqueName: \"kubernetes.io/configmap/7a9d2ee5-9aa4-4778-a7ea-3e46ebf9cf8d-cabundle-cert\") pod \"prestop-hook-test-kserve-5f6b9b8bc6-nmsvm\" (UID: \"7a9d2ee5-9aa4-4778-a7ea-3e46ebf9cf8d\") " pod="e2e-test-prestop-hook-22727e61/prestop-hook-test-kserve-5f6b9b8bc6-nmsvm"
Sep 17 18:46:50.164868 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:46:50.164774    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/7a9d2ee5-9aa4-4778-a7ea-3e46ebf9cf8d-home\") pod \"prestop-hook-test-kserve-5f6b9b8bc6-nmsvm\" (UID: \"7a9d2ee5-9aa4-4778-a7ea-3e46ebf9cf8d\") " pod="e2e-test-prestop-hook-22727e61/prestop-hook-test-kserve-5f6b9b8bc6-nmsvm"
Sep 17 18:46:50.164910 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:46:50.164881    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/7a9d2ee5-9aa4-4778-a7ea-3e46ebf9cf8d-model-cache\") pod \"prestop-hook-test-kserve-5f6b9b8bc6-nmsvm\" (UID: \"7a9d2ee5-9aa4-4778-a7ea-3e46ebf9cf8d\") " pod="e2e-test-prestop-hook-22727e61/prestop-hook-test-kserve-5f6b9b8bc6-nmsvm"
Sep 17 18:46:50.164984 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:46:50.164965    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/7a9d2ee5-9aa4-4778-a7ea-3e46ebf9cf8d-kserve-provision-location\") pod \"prestop-hook-test-kserve-5f6b9b8bc6-nmsvm\" (UID: \"7a9d2ee5-9aa4-4778-a7ea-3e46ebf9cf8d\") " pod="e2e-test-prestop-hook-22727e61/prestop-hook-test-kserve-5f6b9b8bc6-nmsvm"
Sep 17 18:46:50.165037 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:46:50.165018    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/7a9d2ee5-9aa4-4778-a7ea-3e46ebf9cf8d-tmp-dir\") pod \"prestop-hook-test-kserve-5f6b9b8bc6-nmsvm\" (UID: \"7a9d2ee5-9aa4-4778-a7ea-3e46ebf9cf8d\") " pod="e2e-test-prestop-hook-22727e61/prestop-hook-test-kserve-5f6b9b8bc6-nmsvm"
Sep 17 18:46:50.165256 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:46:50.165239    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cabundle-cert\" (UniqueName: \"kubernetes.io/configmap/7a9d2ee5-9aa4-4778-a7ea-3e46ebf9cf8d-cabundle-cert\") pod \"prestop-hook-test-kserve-5f6b9b8bc6-nmsvm\" (UID: \"7a9d2ee5-9aa4-4778-a7ea-3e46ebf9cf8d\") " pod="e2e-test-prestop-hook-22727e61/prestop-hook-test-kserve-5f6b9b8bc6-nmsvm"
Sep 17 18:46:50.166698 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:46:50.166677    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/7a9d2ee5-9aa4-4778-a7ea-3e46ebf9cf8d-dshm\") pod \"prestop-hook-test-kserve-5f6b9b8bc6-nmsvm\" (UID: \"7a9d2ee5-9aa4-4778-a7ea-3e46ebf9cf8d\") " pod="e2e-test-prestop-hook-22727e61/prestop-hook-test-kserve-5f6b9b8bc6-nmsvm"
Sep 17 18:46:50.166991 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:46:50.166973    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/7a9d2ee5-9aa4-4778-a7ea-3e46ebf9cf8d-tls-certs\") pod \"prestop-hook-test-kserve-5f6b9b8bc6-nmsvm\" (UID: \"7a9d2ee5-9aa4-4778-a7ea-3e46ebf9cf8d\") " pod="e2e-test-prestop-hook-22727e61/prestop-hook-test-kserve-5f6b9b8bc6-nmsvm"
Sep 17 18:46:50.172778 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:46:50.172756    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-nhsrf\" (UniqueName: \"kubernetes.io/projected/7a9d2ee5-9aa4-4778-a7ea-3e46ebf9cf8d-kube-api-access-nhsrf\") pod \"prestop-hook-test-kserve-5f6b9b8bc6-nmsvm\" (UID: \"7a9d2ee5-9aa4-4778-a7ea-3e46ebf9cf8d\") " pod="e2e-test-prestop-hook-22727e61/prestop-hook-test-kserve-5f6b9b8bc6-nmsvm"
Sep 17 18:46:50.251244 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:46:50.251160    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-test-prestop-hook-22727e61/prestop-hook-test-kserve-5f6b9b8bc6-nmsvm"
Sep 17 18:46:50.550684 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:46:50.550656    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["e2e-test-prestop-hook-22727e61/prestop-hook-test-kserve-5f6b9b8bc6-nmsvm"]
Sep 17 18:46:50.553149 ip-10-0-143-74 kubenswrapper[2575]: W0917 18:46:50.553120    2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7a9d2ee5_9aa4_4778_a7ea_3e46ebf9cf8d.slice/crio-e59ec8f4064decc2f4822357d6732989fb5d15beabf83ad1f77111f7718fb609 WatchSource:0}: Error finding container e59ec8f4064decc2f4822357d6732989fb5d15beabf83ad1f77111f7718fb609: Status 404 returned error can't find the container with id e59ec8f4064decc2f4822357d6732989fb5d15beabf83ad1f77111f7718fb609
Sep 17 18:46:50.878784 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:46:50.878749    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-test-prestop-hook-22727e61/prestop-hook-test-kserve-5f6b9b8bc6-nmsvm" event={"ID":"7a9d2ee5-9aa4-4778-a7ea-3e46ebf9cf8d","Type":"ContainerStarted","Data":"fcd5baa93713cc3ac9f170d48e3bfbacef962e9eb01b4b8634255a19d9a1a791"}
Sep 17 18:46:50.878974 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:46:50.878801    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-test-prestop-hook-22727e61/prestop-hook-test-kserve-5f6b9b8bc6-nmsvm" event={"ID":"7a9d2ee5-9aa4-4778-a7ea-3e46ebf9cf8d","Type":"ContainerStarted","Data":"e59ec8f4064decc2f4822357d6732989fb5d15beabf83ad1f77111f7718fb609"}
Sep 17 18:46:53.896405 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:46:53.896368    2575 generic.go:356] "Generic (PLEG): container finished" podID="7a9d2ee5-9aa4-4778-a7ea-3e46ebf9cf8d" containerID="fcd5baa93713cc3ac9f170d48e3bfbacef962e9eb01b4b8634255a19d9a1a791" exitCode=0
Sep 17 18:46:53.896834 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:46:53.896457    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-test-prestop-hook-22727e61/prestop-hook-test-kserve-5f6b9b8bc6-nmsvm" event={"ID":"7a9d2ee5-9aa4-4778-a7ea-3e46ebf9cf8d","Type":"ContainerDied","Data":"fcd5baa93713cc3ac9f170d48e3bfbacef962e9eb01b4b8634255a19d9a1a791"}
Sep 17 18:46:54.905242 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:46:54.905207    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-test-prestop-hook-22727e61/prestop-hook-test-kserve-5f6b9b8bc6-nmsvm" event={"ID":"7a9d2ee5-9aa4-4778-a7ea-3e46ebf9cf8d","Type":"ContainerStarted","Data":"631c518770261f1483cc3ab4940d988a30e97392d85d6f7c76b23ee24f916b59"}
Sep 17 18:46:54.933329 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:46:54.933262    2575 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="e2e-test-prestop-hook-22727e61/prestop-hook-test-kserve-5f6b9b8bc6-nmsvm" podStartSLOduration=3.665870274 podStartE2EDuration="5.933240378s" totalImagesPullingTime="0s" totalInitContainerRuntime="2.267370104s" isStatefulPod=false podCreationTimestamp="2026-09-17 18:46:49 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-17 18:46:54.928939044 +0000 UTC m=+4334.559269017" watchObservedRunningTime="2026-09-17 18:46:54.933240378 +0000 UTC m=+4334.563570353"
Sep 17 18:46:55.251314 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:46:55.251230    2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-test-prestop-hook-22727e61/prestop-hook-test-kserve-5f6b9b8bc6-nmsvm"
Sep 17 18:47:00.251398 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:47:00.251343    2575 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="e2e-test-prestop-hook-22727e61/prestop-hook-test-kserve-5f6b9b8bc6-nmsvm"
Sep 17 18:47:00.253073 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:47:00.253042    2575 prober.go:120] "Probe failed" probeType="Startup" pod="e2e-test-prestop-hook-22727e61/prestop-hook-test-kserve-5f6b9b8bc6-nmsvm" podUID="7a9d2ee5-9aa4-4778-a7ea-3e46ebf9cf8d" containerName="main" probeResult="failure" output="Get \"https://10.133.0.66:8000/health\": dial tcp 10.133.0.66:8000: connect: connection refused"
Sep 17 18:47:00.415401 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:47:00.415357    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-djvzt_1d07e68d-9943-420f-a6a0-5f477792fabd/konnectivity-agent/0.log" limit=1024
Sep 17 18:47:10.252388 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:47:10.252347    2575 prober.go:120] "Probe failed" probeType="Startup" pod="e2e-test-prestop-hook-22727e61/prestop-hook-test-kserve-5f6b9b8bc6-nmsvm" podUID="7a9d2ee5-9aa4-4778-a7ea-3e46ebf9cf8d" containerName="main" probeResult="failure" output="Get \"https://10.133.0.66:8000/health\": dial tcp 10.133.0.66:8000: connect: connection refused"
Sep 17 18:47:13.858398 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:47:13.858358    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-djvzt_1d07e68d-9943-420f-a6a0-5f477792fabd/konnectivity-agent/0.log" limit=1024
Sep 17 18:47:13.866910 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:47:13.866877    2575 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["e2e-test-prestop-hook-22727e61/prestop-hook-test-kserve-5f6b9b8bc6-nmsvm"]
Sep 17 18:47:13.938342 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:47:13.938300    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-djvzt_1d07e68d-9943-420f-a6a0-5f477792fabd/konnectivity-agent/0.log" limit=1024
Sep 17 18:47:17.163548 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:47:17.163517    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-djvzt_1d07e68d-9943-420f-a6a0-5f477792fabd/konnectivity-agent/0.log" limit=1024
Sep 17 18:47:17.235479 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:47:17.235406    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-djvzt_1d07e68d-9943-420f-a6a0-5f477792fabd/konnectivity-agent/0.log" limit=1024
Sep 17 18:47:28.890902 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:47:28.890830    2575 kuberuntime_container.go:873] "Killing container with a grace period" pod="e2e-test-prestop-hook-22727e61/prestop-hook-test-kserve-5f6b9b8bc6-nmsvm" podUID="7a9d2ee5-9aa4-4778-a7ea-3e46ebf9cf8d" containerName="main" containerID="cri-o://631c518770261f1483cc3ab4940d988a30e97392d85d6f7c76b23ee24f916b59" gracePeriod=45
Sep 17 18:48:06.722230 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:48:06.722148    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-djvzt_1d07e68d-9943-420f-a6a0-5f477792fabd/konnectivity-agent/0.log" limit=1024
Sep 17 18:48:12.846048 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:48:12.846005    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-djvzt_1d07e68d-9943-420f-a6a0-5f477792fabd/konnectivity-agent/0.log" limit=1024
Sep 17 18:48:14.370828 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:48:14.370799    2575 log.go:25] "Finished parsing log file" path="/var/log/pods/e2e-test-prestop-hook-22727e61_prestop-hook-test-kserve-5f6b9b8bc6-nmsvm_7a9d2ee5-9aa4-4778-a7ea-3e46ebf9cf8d/main/0.log"
Sep 17 18:48:14.371267 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:48:14.371190    2575 generic.go:356] "Generic (PLEG): container finished" podID="7a9d2ee5-9aa4-4778-a7ea-3e46ebf9cf8d" containerID="631c518770261f1483cc3ab4940d988a30e97392d85d6f7c76b23ee24f916b59" exitCode=137
Sep 17 18:48:14.371336 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:48:14.371275    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-test-prestop-hook-22727e61/prestop-hook-test-kserve-5f6b9b8bc6-nmsvm" event={"ID":"7a9d2ee5-9aa4-4778-a7ea-3e46ebf9cf8d","Type":"ContainerDied","Data":"631c518770261f1483cc3ab4940d988a30e97392d85d6f7c76b23ee24f916b59"}
Sep 17 18:48:14.539390 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:48:14.539365    2575 log.go:25] "Finished parsing log file" path="/var/log/pods/e2e-test-prestop-hook-22727e61_prestop-hook-test-kserve-5f6b9b8bc6-nmsvm_7a9d2ee5-9aa4-4778-a7ea-3e46ebf9cf8d/main/0.log"
Sep 17 18:48:14.541968 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:48:14.541945    2575 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-test-prestop-hook-22727e61/prestop-hook-test-kserve-5f6b9b8bc6-nmsvm"
Sep 17 18:48:14.727324 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:48:14.727228    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/7a9d2ee5-9aa4-4778-a7ea-3e46ebf9cf8d-kserve-provision-location\") pod \"7a9d2ee5-9aa4-4778-a7ea-3e46ebf9cf8d\" (UID: \"7a9d2ee5-9aa4-4778-a7ea-3e46ebf9cf8d\") "
Sep 17 18:48:14.727324 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:48:14.727271    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/7a9d2ee5-9aa4-4778-a7ea-3e46ebf9cf8d-model-cache\") pod \"7a9d2ee5-9aa4-4778-a7ea-3e46ebf9cf8d\" (UID: \"7a9d2ee5-9aa4-4778-a7ea-3e46ebf9cf8d\") "
Sep 17 18:48:14.727324 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:48:14.727307    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/7a9d2ee5-9aa4-4778-a7ea-3e46ebf9cf8d-tls-certs\") pod \"7a9d2ee5-9aa4-4778-a7ea-3e46ebf9cf8d\" (UID: \"7a9d2ee5-9aa4-4778-a7ea-3e46ebf9cf8d\") "
Sep 17 18:48:14.727649 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:48:14.727335    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/7a9d2ee5-9aa4-4778-a7ea-3e46ebf9cf8d-dshm\") pod \"7a9d2ee5-9aa4-4778-a7ea-3e46ebf9cf8d\" (UID: \"7a9d2ee5-9aa4-4778-a7ea-3e46ebf9cf8d\") "
Sep 17 18:48:14.727649 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:48:14.727378    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"cabundle-cert\" (UniqueName: \"kubernetes.io/configmap/7a9d2ee5-9aa4-4778-a7ea-3e46ebf9cf8d-cabundle-cert\") pod \"7a9d2ee5-9aa4-4778-a7ea-3e46ebf9cf8d\" (UID: \"7a9d2ee5-9aa4-4778-a7ea-3e46ebf9cf8d\") "
Sep 17 18:48:14.727649 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:48:14.727400    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-nhsrf\" (UniqueName: \"kubernetes.io/projected/7a9d2ee5-9aa4-4778-a7ea-3e46ebf9cf8d-kube-api-access-nhsrf\") pod \"7a9d2ee5-9aa4-4778-a7ea-3e46ebf9cf8d\" (UID: \"7a9d2ee5-9aa4-4778-a7ea-3e46ebf9cf8d\") "
Sep 17 18:48:14.727649 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:48:14.727506    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/7a9d2ee5-9aa4-4778-a7ea-3e46ebf9cf8d-tmp-dir\") pod \"7a9d2ee5-9aa4-4778-a7ea-3e46ebf9cf8d\" (UID: \"7a9d2ee5-9aa4-4778-a7ea-3e46ebf9cf8d\") "
Sep 17 18:48:14.727649 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:48:14.727563    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/7a9d2ee5-9aa4-4778-a7ea-3e46ebf9cf8d-home\") pod \"7a9d2ee5-9aa4-4778-a7ea-3e46ebf9cf8d\" (UID: \"7a9d2ee5-9aa4-4778-a7ea-3e46ebf9cf8d\") "
Sep 17 18:48:14.727649 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:48:14.727598    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/7a9d2ee5-9aa4-4778-a7ea-3e46ebf9cf8d-model-cache" (OuterVolumeSpecName: "model-cache") pod "7a9d2ee5-9aa4-4778-a7ea-3e46ebf9cf8d" (UID: "7a9d2ee5-9aa4-4778-a7ea-3e46ebf9cf8d"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 17 18:48:14.727972 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:48:14.727850    2575 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/7a9d2ee5-9aa4-4778-a7ea-3e46ebf9cf8d-model-cache\") on node \"ip-10-0-143-74.ec2.internal\" DevicePath \"\""
Sep 17 18:48:14.727972 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:48:14.727869    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/7a9d2ee5-9aa4-4778-a7ea-3e46ebf9cf8d-cabundle-cert" (OuterVolumeSpecName: "cabundle-cert") pod "7a9d2ee5-9aa4-4778-a7ea-3e46ebf9cf8d" (UID: "7a9d2ee5-9aa4-4778-a7ea-3e46ebf9cf8d"). InnerVolumeSpecName "cabundle-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 17 18:48:14.728237 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:48:14.728204    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/7a9d2ee5-9aa4-4778-a7ea-3e46ebf9cf8d-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "7a9d2ee5-9aa4-4778-a7ea-3e46ebf9cf8d" (UID: "7a9d2ee5-9aa4-4778-a7ea-3e46ebf9cf8d"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 17 18:48:14.728237 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:48:14.728214    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/7a9d2ee5-9aa4-4778-a7ea-3e46ebf9cf8d-home" (OuterVolumeSpecName: "home") pod "7a9d2ee5-9aa4-4778-a7ea-3e46ebf9cf8d" (UID: "7a9d2ee5-9aa4-4778-a7ea-3e46ebf9cf8d"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 17 18:48:14.729764 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:48:14.729741    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/7a9d2ee5-9aa4-4778-a7ea-3e46ebf9cf8d-kube-api-access-nhsrf" (OuterVolumeSpecName: "kube-api-access-nhsrf") pod "7a9d2ee5-9aa4-4778-a7ea-3e46ebf9cf8d" (UID: "7a9d2ee5-9aa4-4778-a7ea-3e46ebf9cf8d"). InnerVolumeSpecName "kube-api-access-nhsrf". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 17 18:48:14.729879 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:48:14.729786    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/7a9d2ee5-9aa4-4778-a7ea-3e46ebf9cf8d-dshm" (OuterVolumeSpecName: "dshm") pod "7a9d2ee5-9aa4-4778-a7ea-3e46ebf9cf8d" (UID: "7a9d2ee5-9aa4-4778-a7ea-3e46ebf9cf8d"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 17 18:48:14.729879 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:48:14.729842    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/7a9d2ee5-9aa4-4778-a7ea-3e46ebf9cf8d-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "7a9d2ee5-9aa4-4778-a7ea-3e46ebf9cf8d" (UID: "7a9d2ee5-9aa4-4778-a7ea-3e46ebf9cf8d"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 17 18:48:14.751162 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:48:14.751111    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/7a9d2ee5-9aa4-4778-a7ea-3e46ebf9cf8d-kserve-provision-location" (OuterVolumeSpecName: "kserve-provision-location") pod "7a9d2ee5-9aa4-4778-a7ea-3e46ebf9cf8d" (UID: "7a9d2ee5-9aa4-4778-a7ea-3e46ebf9cf8d"). InnerVolumeSpecName "kserve-provision-location". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 17 18:48:14.828541 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:48:14.828503    2575 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/7a9d2ee5-9aa4-4778-a7ea-3e46ebf9cf8d-tls-certs\") on node \"ip-10-0-143-74.ec2.internal\" DevicePath \"\""
Sep 17 18:48:14.828541 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:48:14.828534    2575 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/7a9d2ee5-9aa4-4778-a7ea-3e46ebf9cf8d-dshm\") on node \"ip-10-0-143-74.ec2.internal\" DevicePath \"\""
Sep 17 18:48:14.828541 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:48:14.828544    2575 reconciler_common.go:299] "Volume detached for volume \"cabundle-cert\" (UniqueName: \"kubernetes.io/configmap/7a9d2ee5-9aa4-4778-a7ea-3e46ebf9cf8d-cabundle-cert\") on node \"ip-10-0-143-74.ec2.internal\" DevicePath \"\""
Sep 17 18:48:14.828769 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:48:14.828553    2575 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-nhsrf\" (UniqueName: \"kubernetes.io/projected/7a9d2ee5-9aa4-4778-a7ea-3e46ebf9cf8d-kube-api-access-nhsrf\") on node \"ip-10-0-143-74.ec2.internal\" DevicePath \"\""
Sep 17 18:48:14.828769 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:48:14.828562    2575 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/7a9d2ee5-9aa4-4778-a7ea-3e46ebf9cf8d-tmp-dir\") on node \"ip-10-0-143-74.ec2.internal\" DevicePath \"\""
Sep 17 18:48:14.828769 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:48:14.828570    2575 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/7a9d2ee5-9aa4-4778-a7ea-3e46ebf9cf8d-home\") on node \"ip-10-0-143-74.ec2.internal\" DevicePath \"\""
Sep 17 18:48:14.828769 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:48:14.828577    2575 reconciler_common.go:299] "Volume detached for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/7a9d2ee5-9aa4-4778-a7ea-3e46ebf9cf8d-kserve-provision-location\") on node \"ip-10-0-143-74.ec2.internal\" DevicePath \"\""
Sep 17 18:48:15.377850 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:48:15.377820    2575 log.go:25] "Finished parsing log file" path="/var/log/pods/e2e-test-prestop-hook-22727e61_prestop-hook-test-kserve-5f6b9b8bc6-nmsvm_7a9d2ee5-9aa4-4778-a7ea-3e46ebf9cf8d/main/0.log"
Sep 17 18:48:15.378293 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:48:15.378265    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-test-prestop-hook-22727e61/prestop-hook-test-kserve-5f6b9b8bc6-nmsvm" event={"ID":"7a9d2ee5-9aa4-4778-a7ea-3e46ebf9cf8d","Type":"ContainerDied","Data":"e59ec8f4064decc2f4822357d6732989fb5d15beabf83ad1f77111f7718fb609"}
Sep 17 18:48:15.378367 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:48:15.378322    2575 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-test-prestop-hook-22727e61/prestop-hook-test-kserve-5f6b9b8bc6-nmsvm"
Sep 17 18:48:15.378426 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:48:15.378376    2575 scope.go:117] "RemoveContainer" containerID="631c518770261f1483cc3ab4940d988a30e97392d85d6f7c76b23ee24f916b59"
Sep 17 18:48:15.387539 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:48:15.387522    2575 scope.go:117] "RemoveContainer" containerID="fcd5baa93713cc3ac9f170d48e3bfbacef962e9eb01b4b8634255a19d9a1a791"
Sep 17 18:48:15.399903 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:48:15.399854    2575 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["e2e-test-prestop-hook-22727e61/prestop-hook-test-kserve-5f6b9b8bc6-nmsvm"]
Sep 17 18:48:15.406772 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:48:15.406749    2575 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["e2e-test-prestop-hook-22727e61/prestop-hook-test-kserve-5f6b9b8bc6-nmsvm"]
Sep 17 18:48:16.968213 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:48:16.968179    2575 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="7a9d2ee5-9aa4-4778-a7ea-3e46ebf9cf8d" path="/var/lib/kubelet/pods/7a9d2ee5-9aa4-4778-a7ea-3e46ebf9cf8d/volumes"
Sep 17 18:48:18.609904 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:48:18.609869    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-djvzt_1d07e68d-9943-420f-a6a0-5f477792fabd/konnectivity-agent/0.log" limit=1024
Sep 17 18:48:18.707110 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:48:18.707064    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-djvzt_1d07e68d-9943-420f-a6a0-5f477792fabd/konnectivity-agent/0.log" limit=1024
Sep 17 18:48:18.807760 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:48:18.807725    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-djvzt_1d07e68d-9943-420f-a6a0-5f477792fabd/konnectivity-agent/0.log" limit=1024
Sep 17 18:48:23.390402 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:48:23.390363    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-djvzt_1d07e68d-9943-420f-a6a0-5f477792fabd/konnectivity-agent/0.log" limit=1024
Sep 17 18:48:24.965231 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:48:24.965192    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-djvzt_1d07e68d-9943-420f-a6a0-5f477792fabd/konnectivity-agent/0.log" limit=1024
Sep 17 18:48:30.701371 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:48:30.701328    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-djvzt_1d07e68d-9943-420f-a6a0-5f477792fabd/konnectivity-agent/0.log" limit=1024
Sep 17 18:48:30.801267 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:48:30.801225    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-djvzt_1d07e68d-9943-420f-a6a0-5f477792fabd/konnectivity-agent/0.log" limit=1024
Sep 17 18:48:42.450913 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:48:42.450874    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-djvzt_1d07e68d-9943-420f-a6a0-5f477792fabd/konnectivity-agent/0.log" limit=1024
Sep 17 18:48:42.658166 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:48:42.658128    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-djvzt_1d07e68d-9943-420f-a6a0-5f477792fabd/konnectivity-agent/0.log" limit=1024
Sep 17 18:49:01.574125 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:49:01.574068    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-djvzt_1d07e68d-9943-420f-a6a0-5f477792fabd/konnectivity-agent/0.log" limit=1024
Sep 17 18:49:16.538251 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:49:16.538207    2575 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["e2e-test-rolling-upgrade-coordination-a9229638/rolling-upgrade-test-kserve-6dd8dbf85-cvsq9"]
Sep 17 18:49:16.538738 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:49:16.538696    2575 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="7a9d2ee5-9aa4-4778-a7ea-3e46ebf9cf8d" containerName="storage-initializer"
Sep 17 18:49:16.538738 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:49:16.538713    2575 state_mem.go:107] "Deleted CPUSet assignment" podUID="7a9d2ee5-9aa4-4778-a7ea-3e46ebf9cf8d" containerName="storage-initializer"
Sep 17 18:49:16.538738 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:49:16.538723    2575 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="7a9d2ee5-9aa4-4778-a7ea-3e46ebf9cf8d" containerName="main"
Sep 17 18:49:16.538738 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:49:16.538730    2575 state_mem.go:107] "Deleted CPUSet assignment" podUID="7a9d2ee5-9aa4-4778-a7ea-3e46ebf9cf8d" containerName="main"
Sep 17 18:49:16.538884 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:49:16.538807    2575 memory_manager.go:368] "RemoveStaleState removing state" podUID="7a9d2ee5-9aa4-4778-a7ea-3e46ebf9cf8d" containerName="main"
Sep 17 18:49:16.541927 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:49:16.541910    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-test-rolling-upgrade-coordination-a9229638/rolling-upgrade-test-kserve-6dd8dbf85-cvsq9"
Sep 17 18:49:16.545179 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:49:16.545144    2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-test-rolling-upgrade-coordination-a9229638\"/\"kube-root-ca.crt\""
Sep 17 18:49:16.545179 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:49:16.545165    2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-test-rolling-upgrade-coordination-a9229638\"/\"openshift-service-ca.crt\""
Sep 17 18:49:16.545179 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:49:16.545147    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-test-rolling-upgrade-coordination-a9229638\"/\"default-dockercfg-8ttfc\""
Sep 17 18:49:16.545409 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:49:16.545227    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-test-rolling-upgrade-coordination-a9229638\"/\"rolling-upgrade-test-kserve-self-signed-certs\""
Sep 17 18:49:16.550721 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:49:16.550695    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["e2e-test-rolling-upgrade-coordination-a9229638/rolling-upgrade-test-kserve-6dd8dbf85-cvsq9"]
Sep 17 18:49:16.636533 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:49:16.636494    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/291f170d-6485-48c1-bf21-30f3adbd5f6e-tmp-dir\") pod \"rolling-upgrade-test-kserve-6dd8dbf85-cvsq9\" (UID: \"291f170d-6485-48c1-bf21-30f3adbd5f6e\") " pod="e2e-test-rolling-upgrade-coordination-a9229638/rolling-upgrade-test-kserve-6dd8dbf85-cvsq9"
Sep 17 18:49:16.636709 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:49:16.636628    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/291f170d-6485-48c1-bf21-30f3adbd5f6e-home\") pod \"rolling-upgrade-test-kserve-6dd8dbf85-cvsq9\" (UID: \"291f170d-6485-48c1-bf21-30f3adbd5f6e\") " pod="e2e-test-rolling-upgrade-coordination-a9229638/rolling-upgrade-test-kserve-6dd8dbf85-cvsq9"
Sep 17 18:49:16.636709 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:49:16.636661    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/291f170d-6485-48c1-bf21-30f3adbd5f6e-model-cache\") pod \"rolling-upgrade-test-kserve-6dd8dbf85-cvsq9\" (UID: \"291f170d-6485-48c1-bf21-30f3adbd5f6e\") " pod="e2e-test-rolling-upgrade-coordination-a9229638/rolling-upgrade-test-kserve-6dd8dbf85-cvsq9"
Sep 17 18:49:16.636709 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:49:16.636683    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2hmkr\" (UniqueName: \"kubernetes.io/projected/291f170d-6485-48c1-bf21-30f3adbd5f6e-kube-api-access-2hmkr\") pod \"rolling-upgrade-test-kserve-6dd8dbf85-cvsq9\" (UID: \"291f170d-6485-48c1-bf21-30f3adbd5f6e\") " pod="e2e-test-rolling-upgrade-coordination-a9229638/rolling-upgrade-test-kserve-6dd8dbf85-cvsq9"
Sep 17 18:49:16.636709 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:49:16.636707    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/291f170d-6485-48c1-bf21-30f3adbd5f6e-tls-certs\") pod \"rolling-upgrade-test-kserve-6dd8dbf85-cvsq9\" (UID: \"291f170d-6485-48c1-bf21-30f3adbd5f6e\") " pod="e2e-test-rolling-upgrade-coordination-a9229638/rolling-upgrade-test-kserve-6dd8dbf85-cvsq9"
Sep 17 18:49:16.636853 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:49:16.636724    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/291f170d-6485-48c1-bf21-30f3adbd5f6e-dshm\") pod \"rolling-upgrade-test-kserve-6dd8dbf85-cvsq9\" (UID: \"291f170d-6485-48c1-bf21-30f3adbd5f6e\") " pod="e2e-test-rolling-upgrade-coordination-a9229638/rolling-upgrade-test-kserve-6dd8dbf85-cvsq9"
Sep 17 18:49:16.737643 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:49:16.737602    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/291f170d-6485-48c1-bf21-30f3adbd5f6e-home\") pod \"rolling-upgrade-test-kserve-6dd8dbf85-cvsq9\" (UID: \"291f170d-6485-48c1-bf21-30f3adbd5f6e\") " pod="e2e-test-rolling-upgrade-coordination-a9229638/rolling-upgrade-test-kserve-6dd8dbf85-cvsq9"
Sep 17 18:49:16.737833 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:49:16.737651    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/291f170d-6485-48c1-bf21-30f3adbd5f6e-model-cache\") pod \"rolling-upgrade-test-kserve-6dd8dbf85-cvsq9\" (UID: \"291f170d-6485-48c1-bf21-30f3adbd5f6e\") " pod="e2e-test-rolling-upgrade-coordination-a9229638/rolling-upgrade-test-kserve-6dd8dbf85-cvsq9"
Sep 17 18:49:16.737833 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:49:16.737676    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2hmkr\" (UniqueName: \"kubernetes.io/projected/291f170d-6485-48c1-bf21-30f3adbd5f6e-kube-api-access-2hmkr\") pod \"rolling-upgrade-test-kserve-6dd8dbf85-cvsq9\" (UID: \"291f170d-6485-48c1-bf21-30f3adbd5f6e\") " pod="e2e-test-rolling-upgrade-coordination-a9229638/rolling-upgrade-test-kserve-6dd8dbf85-cvsq9"
Sep 17 18:49:16.737833 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:49:16.737712    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/291f170d-6485-48c1-bf21-30f3adbd5f6e-tls-certs\") pod \"rolling-upgrade-test-kserve-6dd8dbf85-cvsq9\" (UID: \"291f170d-6485-48c1-bf21-30f3adbd5f6e\") " pod="e2e-test-rolling-upgrade-coordination-a9229638/rolling-upgrade-test-kserve-6dd8dbf85-cvsq9"
Sep 17 18:49:16.737833 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:49:16.737735    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/291f170d-6485-48c1-bf21-30f3adbd5f6e-dshm\") pod \"rolling-upgrade-test-kserve-6dd8dbf85-cvsq9\" (UID: \"291f170d-6485-48c1-bf21-30f3adbd5f6e\") " pod="e2e-test-rolling-upgrade-coordination-a9229638/rolling-upgrade-test-kserve-6dd8dbf85-cvsq9"
Sep 17 18:49:16.737833 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:49:16.737776    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/291f170d-6485-48c1-bf21-30f3adbd5f6e-tmp-dir\") pod \"rolling-upgrade-test-kserve-6dd8dbf85-cvsq9\" (UID: \"291f170d-6485-48c1-bf21-30f3adbd5f6e\") " pod="e2e-test-rolling-upgrade-coordination-a9229638/rolling-upgrade-test-kserve-6dd8dbf85-cvsq9"
Sep 17 18:49:16.738135 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:49:16.738074    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/291f170d-6485-48c1-bf21-30f3adbd5f6e-model-cache\") pod \"rolling-upgrade-test-kserve-6dd8dbf85-cvsq9\" (UID: \"291f170d-6485-48c1-bf21-30f3adbd5f6e\") " pod="e2e-test-rolling-upgrade-coordination-a9229638/rolling-upgrade-test-kserve-6dd8dbf85-cvsq9"
Sep 17 18:49:16.738135 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:49:16.738115    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/291f170d-6485-48c1-bf21-30f3adbd5f6e-home\") pod \"rolling-upgrade-test-kserve-6dd8dbf85-cvsq9\" (UID: \"291f170d-6485-48c1-bf21-30f3adbd5f6e\") " pod="e2e-test-rolling-upgrade-coordination-a9229638/rolling-upgrade-test-kserve-6dd8dbf85-cvsq9"
Sep 17 18:49:16.738264 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:49:16.738240    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/291f170d-6485-48c1-bf21-30f3adbd5f6e-tmp-dir\") pod \"rolling-upgrade-test-kserve-6dd8dbf85-cvsq9\" (UID: \"291f170d-6485-48c1-bf21-30f3adbd5f6e\") " pod="e2e-test-rolling-upgrade-coordination-a9229638/rolling-upgrade-test-kserve-6dd8dbf85-cvsq9"
Sep 17 18:49:16.740111 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:49:16.740087    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/291f170d-6485-48c1-bf21-30f3adbd5f6e-dshm\") pod \"rolling-upgrade-test-kserve-6dd8dbf85-cvsq9\" (UID: \"291f170d-6485-48c1-bf21-30f3adbd5f6e\") " pod="e2e-test-rolling-upgrade-coordination-a9229638/rolling-upgrade-test-kserve-6dd8dbf85-cvsq9"
Sep 17 18:49:16.740269 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:49:16.740253    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/291f170d-6485-48c1-bf21-30f3adbd5f6e-tls-certs\") pod \"rolling-upgrade-test-kserve-6dd8dbf85-cvsq9\" (UID: \"291f170d-6485-48c1-bf21-30f3adbd5f6e\") " pod="e2e-test-rolling-upgrade-coordination-a9229638/rolling-upgrade-test-kserve-6dd8dbf85-cvsq9"
Sep 17 18:49:16.745526 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:49:16.745504    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-2hmkr\" (UniqueName: \"kubernetes.io/projected/291f170d-6485-48c1-bf21-30f3adbd5f6e-kube-api-access-2hmkr\") pod \"rolling-upgrade-test-kserve-6dd8dbf85-cvsq9\" (UID: \"291f170d-6485-48c1-bf21-30f3adbd5f6e\") " pod="e2e-test-rolling-upgrade-coordination-a9229638/rolling-upgrade-test-kserve-6dd8dbf85-cvsq9"
Sep 17 18:49:16.853290 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:49:16.853202    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-test-rolling-upgrade-coordination-a9229638/rolling-upgrade-test-kserve-6dd8dbf85-cvsq9"
Sep 17 18:49:17.130291 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:49:17.130261    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["e2e-test-rolling-upgrade-coordination-a9229638/rolling-upgrade-test-kserve-6dd8dbf85-cvsq9"]
Sep 17 18:49:17.132260 ip-10-0-143-74 kubenswrapper[2575]: W0917 18:49:17.132234    2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod291f170d_6485_48c1_bf21_30f3adbd5f6e.slice/crio-467fb1bd31940949331851490afaed69665334baf478ed522b13417d013edbe4 WatchSource:0}: Error finding container 467fb1bd31940949331851490afaed69665334baf478ed522b13417d013edbe4: Status 404 returned error can't find the container with id 467fb1bd31940949331851490afaed69665334baf478ed522b13417d013edbe4
Sep 17 18:49:17.716871 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:49:17.716834    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-test-rolling-upgrade-coordination-a9229638/rolling-upgrade-test-kserve-6dd8dbf85-cvsq9" event={"ID":"291f170d-6485-48c1-bf21-30f3adbd5f6e","Type":"ContainerStarted","Data":"1d9be20fe8459fa1bc6fedafbf38549e907127dcf9260df2ae88f04227bd9154"}
Sep 17 18:49:17.717280 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:49:17.716921    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-test-rolling-upgrade-coordination-a9229638/rolling-upgrade-test-kserve-6dd8dbf85-cvsq9" event={"ID":"291f170d-6485-48c1-bf21-30f3adbd5f6e","Type":"ContainerStarted","Data":"467fb1bd31940949331851490afaed69665334baf478ed522b13417d013edbe4"}
Sep 17 18:49:17.736852 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:49:17.736804    2575 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="e2e-test-rolling-upgrade-coordination-a9229638/rolling-upgrade-test-kserve-6dd8dbf85-cvsq9" podStartSLOduration=1.73678903 podStartE2EDuration="1.73678903s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-17 18:49:16 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-17 18:49:17.73365553 +0000 UTC m=+4477.363985501" watchObservedRunningTime="2026-09-17 18:49:17.73678903 +0000 UTC m=+4477.367119005"
Sep 17 18:49:17.853539 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:49:17.853504    2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-test-rolling-upgrade-coordination-a9229638/rolling-upgrade-test-kserve-6dd8dbf85-cvsq9"
Sep 17 18:49:25.780619 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:49:25.780582    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-djvzt_1d07e68d-9943-420f-a6a0-5f477792fabd/konnectivity-agent/0.log" limit=1024
Sep 17 18:49:26.853583 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:49:26.853538    2575 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="e2e-test-rolling-upgrade-coordination-a9229638/rolling-upgrade-test-kserve-6dd8dbf85-cvsq9"
Sep 17 18:49:26.866246 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:49:26.866219    2575 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="e2e-test-rolling-upgrade-coordination-a9229638/rolling-upgrade-test-kserve-6dd8dbf85-cvsq9"
Sep 17 18:49:27.782498 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:49:27.782465    2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-test-rolling-upgrade-coordination-a9229638/rolling-upgrade-test-kserve-6dd8dbf85-cvsq9"
Sep 17 18:49:40.120959 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:49:40.120869    2575 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["e2e-test-rolling-upgrade-coordination-a9229638/rolling-upgrade-test-kserve-6dd8dbf85-cvsq9"]
Sep 17 18:49:40.124694 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:49:40.124647    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-djvzt_1d07e68d-9943-420f-a6a0-5f477792fabd/konnectivity-agent/0.log" limit=1024
Sep 17 18:49:40.131077 ip-10-0-143-74 kubenswrapper[2575]: E0917 18:49:40.131035    2575 log.go:32] "ExecSync cmd from runtime service failed" err=<
Sep 17 18:49:40.131077 ip-10-0-143-74 kubenswrapper[2575]:         rpc error: code = Unknown desc = command error: executable file `/bin/sleep` not found: No such file or directory
Sep 17 18:49:40.131077 ip-10-0-143-74 kubenswrapper[2575]:         , stdout: , stderr: , exit code -1
Sep 17 18:49:40.131077 ip-10-0-143-74 kubenswrapper[2575]:  > containerID="1d9be20fe8459fa1bc6fedafbf38549e907127dcf9260df2ae88f04227bd9154" cmd=["/bin/sleep","15"]
Sep 17 18:49:40.131356 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:49:40.131100    2575 handlers.go:82] "Exec lifecycle hook for Container in Pod failed" execCommand=["/bin/sleep","15"] containerName="main" pod="e2e-test-rolling-upgrade-coordination-a9229638/rolling-upgrade-test-kserve-6dd8dbf85-cvsq9" message="" err=<
Sep 17 18:49:40.131356 ip-10-0-143-74 kubenswrapper[2575]:         rpc error: code = Unknown desc = command error: executable file `/bin/sleep` not found: No such file or directory
Sep 17 18:49:40.131356 ip-10-0-143-74 kubenswrapper[2575]:         , stdout: , stderr: , exit code -1
Sep 17 18:49:40.131356 ip-10-0-143-74 kubenswrapper[2575]:  >
Sep 17 18:49:40.131356 ip-10-0-143-74 kubenswrapper[2575]: E0917 18:49:40.131118    2575 kuberuntime_container.go:755] "PreStop hook failed" err=<
Sep 17 18:49:40.131356 ip-10-0-143-74 kubenswrapper[2575]:         rpc error: code = Unknown desc = command error: executable file `/bin/sleep` not found: No such file or directory
Sep 17 18:49:40.131356 ip-10-0-143-74 kubenswrapper[2575]:         , stdout: , stderr: , exit code -1
Sep 17 18:49:40.131356 ip-10-0-143-74 kubenswrapper[2575]:  > pod="e2e-test-rolling-upgrade-coordination-a9229638/rolling-upgrade-test-kserve-6dd8dbf85-cvsq9" podUID="291f170d-6485-48c1-bf21-30f3adbd5f6e" containerName="main" containerID="cri-o://1d9be20fe8459fa1bc6fedafbf38549e907127dcf9260df2ae88f04227bd9154"
Sep 17 18:49:40.131356 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:49:40.131181    2575 kuberuntime_container.go:873] "Killing container with a grace period" pod="e2e-test-rolling-upgrade-coordination-a9229638/rolling-upgrade-test-kserve-6dd8dbf85-cvsq9" podUID="291f170d-6485-48c1-bf21-30f3adbd5f6e" containerName="main" containerID="cri-o://1d9be20fe8459fa1bc6fedafbf38549e907127dcf9260df2ae88f04227bd9154" gracePeriod=60
Sep 17 18:49:40.223980 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:49:40.223946    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-djvzt_1d07e68d-9943-420f-a6a0-5f477792fabd/konnectivity-agent/0.log" limit=1024
Sep 17 18:49:40.772272 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:49:40.772222    2575 prober.go:120] "Probe failed" probeType="Readiness" pod="e2e-test-rolling-upgrade-coordination-a9229638/rolling-upgrade-test-kserve-6dd8dbf85-cvsq9" podUID="291f170d-6485-48c1-bf21-30f3adbd5f6e" containerName="main" probeResult="failure" output="Get \"https://10.133.0.67:8000/health\": dial tcp 10.133.0.67:8000: connect: connection refused"
Sep 17 18:49:40.846116 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:49:40.846083    2575 generic.go:356] "Generic (PLEG): container finished" podID="291f170d-6485-48c1-bf21-30f3adbd5f6e" containerID="1d9be20fe8459fa1bc6fedafbf38549e907127dcf9260df2ae88f04227bd9154" exitCode=0
Sep 17 18:49:40.846283 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:49:40.846161    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-test-rolling-upgrade-coordination-a9229638/rolling-upgrade-test-kserve-6dd8dbf85-cvsq9" event={"ID":"291f170d-6485-48c1-bf21-30f3adbd5f6e","Type":"ContainerDied","Data":"1d9be20fe8459fa1bc6fedafbf38549e907127dcf9260df2ae88f04227bd9154"}
Sep 17 18:49:40.899109 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:49:40.899077    2575 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-test-rolling-upgrade-coordination-a9229638/rolling-upgrade-test-kserve-6dd8dbf85-cvsq9"
Sep 17 18:49:40.940526 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:49:40.940495    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/291f170d-6485-48c1-bf21-30f3adbd5f6e-tls-certs\") pod \"291f170d-6485-48c1-bf21-30f3adbd5f6e\" (UID: \"291f170d-6485-48c1-bf21-30f3adbd5f6e\") "
Sep 17 18:49:40.940526 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:49:40.940529    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-2hmkr\" (UniqueName: \"kubernetes.io/projected/291f170d-6485-48c1-bf21-30f3adbd5f6e-kube-api-access-2hmkr\") pod \"291f170d-6485-48c1-bf21-30f3adbd5f6e\" (UID: \"291f170d-6485-48c1-bf21-30f3adbd5f6e\") "
Sep 17 18:49:40.940795 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:49:40.940570    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/291f170d-6485-48c1-bf21-30f3adbd5f6e-home\") pod \"291f170d-6485-48c1-bf21-30f3adbd5f6e\" (UID: \"291f170d-6485-48c1-bf21-30f3adbd5f6e\") "
Sep 17 18:49:40.940795 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:49:40.940612    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/291f170d-6485-48c1-bf21-30f3adbd5f6e-tmp-dir\") pod \"291f170d-6485-48c1-bf21-30f3adbd5f6e\" (UID: \"291f170d-6485-48c1-bf21-30f3adbd5f6e\") "
Sep 17 18:49:40.940795 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:49:40.940634    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/291f170d-6485-48c1-bf21-30f3adbd5f6e-dshm\") pod \"291f170d-6485-48c1-bf21-30f3adbd5f6e\" (UID: \"291f170d-6485-48c1-bf21-30f3adbd5f6e\") "
Sep 17 18:49:40.940795 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:49:40.940660    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/291f170d-6485-48c1-bf21-30f3adbd5f6e-model-cache\") pod \"291f170d-6485-48c1-bf21-30f3adbd5f6e\" (UID: \"291f170d-6485-48c1-bf21-30f3adbd5f6e\") "
Sep 17 18:49:40.941033 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:49:40.940918    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/291f170d-6485-48c1-bf21-30f3adbd5f6e-home" (OuterVolumeSpecName: "home") pod "291f170d-6485-48c1-bf21-30f3adbd5f6e" (UID: "291f170d-6485-48c1-bf21-30f3adbd5f6e"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 17 18:49:40.941033 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:49:40.940933    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/291f170d-6485-48c1-bf21-30f3adbd5f6e-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "291f170d-6485-48c1-bf21-30f3adbd5f6e" (UID: "291f170d-6485-48c1-bf21-30f3adbd5f6e"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 17 18:49:40.941147 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:49:40.941051    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/291f170d-6485-48c1-bf21-30f3adbd5f6e-model-cache" (OuterVolumeSpecName: "model-cache") pod "291f170d-6485-48c1-bf21-30f3adbd5f6e" (UID: "291f170d-6485-48c1-bf21-30f3adbd5f6e"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 17 18:49:40.942896 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:49:40.942870    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/291f170d-6485-48c1-bf21-30f3adbd5f6e-kube-api-access-2hmkr" (OuterVolumeSpecName: "kube-api-access-2hmkr") pod "291f170d-6485-48c1-bf21-30f3adbd5f6e" (UID: "291f170d-6485-48c1-bf21-30f3adbd5f6e"). InnerVolumeSpecName "kube-api-access-2hmkr". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 17 18:49:40.943039 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:49:40.943019    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/291f170d-6485-48c1-bf21-30f3adbd5f6e-dshm" (OuterVolumeSpecName: "dshm") pod "291f170d-6485-48c1-bf21-30f3adbd5f6e" (UID: "291f170d-6485-48c1-bf21-30f3adbd5f6e"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 17 18:49:40.943214 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:49:40.943193    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/291f170d-6485-48c1-bf21-30f3adbd5f6e-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "291f170d-6485-48c1-bf21-30f3adbd5f6e" (UID: "291f170d-6485-48c1-bf21-30f3adbd5f6e"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 17 18:49:41.041679 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:49:41.041583    2575 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/291f170d-6485-48c1-bf21-30f3adbd5f6e-tls-certs\") on node \"ip-10-0-143-74.ec2.internal\" DevicePath \"\""
Sep 17 18:49:41.041679 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:49:41.041619    2575 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-2hmkr\" (UniqueName: \"kubernetes.io/projected/291f170d-6485-48c1-bf21-30f3adbd5f6e-kube-api-access-2hmkr\") on node \"ip-10-0-143-74.ec2.internal\" DevicePath \"\""
Sep 17 18:49:41.041679 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:49:41.041629    2575 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/291f170d-6485-48c1-bf21-30f3adbd5f6e-home\") on node \"ip-10-0-143-74.ec2.internal\" DevicePath \"\""
Sep 17 18:49:41.041679 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:49:41.041678    2575 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/291f170d-6485-48c1-bf21-30f3adbd5f6e-tmp-dir\") on node \"ip-10-0-143-74.ec2.internal\" DevicePath \"\""
Sep 17 18:49:41.041679 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:49:41.041687    2575 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/291f170d-6485-48c1-bf21-30f3adbd5f6e-dshm\") on node \"ip-10-0-143-74.ec2.internal\" DevicePath \"\""
Sep 17 18:49:41.041953 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:49:41.041695    2575 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/291f170d-6485-48c1-bf21-30f3adbd5f6e-model-cache\") on node \"ip-10-0-143-74.ec2.internal\" DevicePath \"\""
Sep 17 18:49:41.853224 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:49:41.853189    2575 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-test-rolling-upgrade-coordination-a9229638/rolling-upgrade-test-kserve-6dd8dbf85-cvsq9"
Sep 17 18:49:41.853665 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:49:41.853188    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-test-rolling-upgrade-coordination-a9229638/rolling-upgrade-test-kserve-6dd8dbf85-cvsq9" event={"ID":"291f170d-6485-48c1-bf21-30f3adbd5f6e","Type":"ContainerDied","Data":"467fb1bd31940949331851490afaed69665334baf478ed522b13417d013edbe4"}
Sep 17 18:49:41.853665 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:49:41.853327    2575 scope.go:117] "RemoveContainer" containerID="1d9be20fe8459fa1bc6fedafbf38549e907127dcf9260df2ae88f04227bd9154"
Sep 17 18:49:41.872652 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:49:41.872627    2575 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["e2e-test-rolling-upgrade-coordination-a9229638/rolling-upgrade-test-kserve-6dd8dbf85-cvsq9"]
Sep 17 18:49:41.875936 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:49:41.875899    2575 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["e2e-test-rolling-upgrade-coordination-a9229638/rolling-upgrade-test-kserve-6dd8dbf85-cvsq9"]
Sep 17 18:49:42.967977 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:49:42.967941    2575 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="291f170d-6485-48c1-bf21-30f3adbd5f6e" path="/var/lib/kubelet/pods/291f170d-6485-48c1-bf21-30f3adbd5f6e/volumes"
Sep 17 18:50:28.956004 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:50:28.955965    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-djvzt_1d07e68d-9943-420f-a6a0-5f477792fabd/konnectivity-agent/0.log" limit=1024
Sep 17 18:50:30.705369 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:50:30.705331    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-djvzt_1d07e68d-9943-420f-a6a0-5f477792fabd/konnectivity-agent/0.log" limit=1024
Sep 17 18:50:30.804625 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:50:30.804585    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-djvzt_1d07e68d-9943-420f-a6a0-5f477792fabd/konnectivity-agent/0.log" limit=1024
Sep 17 18:50:35.371863 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:50:35.371826    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-djvzt_1d07e68d-9943-420f-a6a0-5f477792fabd/konnectivity-agent/0.log" limit=1024
Sep 17 18:50:41.345719 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:50:41.345683    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-djvzt_1d07e68d-9943-420f-a6a0-5f477792fabd/konnectivity-agent/0.log" limit=1024
Sep 17 18:50:41.411675 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:50:41.411637    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-djvzt_1d07e68d-9943-420f-a6a0-5f477792fabd/konnectivity-agent/0.log" limit=1024
Sep 17 18:50:53.617011 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:50:53.616975    2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_istiod-openshift-gateway-5677ccb49b-ft57x_7ca824c6-1031-48be-af6a-0d0b5de1aa51/discovery/0.log"
Sep 17 18:50:53.656751 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:50:53.656722    2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_router-default-76f78f8976-kwv2h_3a6e39e9-229a-440e-a565-16ef48e36247/router/0.log"
Sep 17 18:50:54.386022 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:50:54.385981    2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_istiod-openshift-gateway-5677ccb49b-ft57x_7ca824c6-1031-48be-af6a-0d0b5de1aa51/discovery/0.log"
Sep 17 18:50:54.421080 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:50:54.421053    2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_router-default-76f78f8976-kwv2h_3a6e39e9-229a-440e-a565-16ef48e36247/router/0.log"
Sep 17 18:50:55.245334 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:50:55.245309    2575 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_dns-operator-controller-manager-65dc8f76f5-8s7cq_9c5e897b-2d08-490f-b194-ef54c1d56ba3/manager/0.log"
Sep 17 18:50:55.344224 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:50:55.344183    2575 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-operator-controller-manager-6d4cb79c-25rvf_3cb0e8b5-8b54-49a6-aa86-0dcf1b07f4a2/manager/0.log"
Sep 17 18:50:55.369337 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:50:55.369308    2575 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_limitador-operator-controller-manager-5ffb4b4ccb-gn5dk_049f5402-5027-46b4-92c6-36c31dbb26b0/manager/0.log"
Sep 17 18:51:01.088976 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:51:01.088948    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-djvzt_1d07e68d-9943-420f-a6a0-5f477792fabd/konnectivity-agent/0.log" limit=1024
Sep 17 18:51:01.230638 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:51:01.230594    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-djvzt_1d07e68d-9943-420f-a6a0-5f477792fabd/konnectivity-agent/0.log" limit=1024
Sep 17 18:51:01.324459 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:51:01.324402    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-djvzt_1d07e68d-9943-420f-a6a0-5f477792fabd/konnectivity-agent/0.log" limit=1024
Sep 17 18:51:04.148480 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:51:04.148354    2575 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_global-pull-secret-syncer-9s5wl_7892b408-d2f1-4e39-9bf6-e68a452196b1/global-pull-secret-syncer/0.log"
Sep 17 18:51:04.259227 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:51:04.259191    2575 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_konnectivity-agent-djvzt_1d07e68d-9943-420f-a6a0-5f477792fabd/konnectivity-agent/0.log"
Sep 17 18:51:04.411847 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:51:04.411764    2575 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_kube-apiserver-proxy-ip-10-0-143-74.ec2.internal_d937ab292439efa4b2d5e0f9e6f432b8/haproxy/0.log"
Sep 17 18:51:07.265615 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:51:07.265561    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-djvzt_1d07e68d-9943-420f-a6a0-5f477792fabd/konnectivity-agent/0.log" limit=1024
Sep 17 18:51:07.365637 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:51:07.365595    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-djvzt_1d07e68d-9943-420f-a6a0-5f477792fabd/konnectivity-agent/0.log" limit=1024
Sep 17 18:51:09.475400 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:51:09.475364    2575 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_dns-operator-controller-manager-65dc8f76f5-8s7cq_9c5e897b-2d08-490f-b194-ef54c1d56ba3/manager/0.log"
Sep 17 18:51:09.583165 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:51:09.583122    2575 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-operator-controller-manager-6d4cb79c-25rvf_3cb0e8b5-8b54-49a6-aa86-0dcf1b07f4a2/manager/0.log"
Sep 17 18:51:09.642765 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:51:09.642731    2575 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_limitador-operator-controller-manager-5ffb4b4ccb-gn5dk_049f5402-5027-46b4-92c6-36c31dbb26b0/manager/0.log"
Sep 17 18:51:10.928164 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:51:10.928130    2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_26063eb2-0a4f-460f-a6ba-6d19ab96af70/alertmanager/0.log"
Sep 17 18:51:10.953437 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:51:10.953410    2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_26063eb2-0a4f-460f-a6ba-6d19ab96af70/config-reloader/0.log"
Sep 17 18:51:10.977839 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:51:10.977811    2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_26063eb2-0a4f-460f-a6ba-6d19ab96af70/kube-rbac-proxy-web/0.log"
Sep 17 18:51:11.001276 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:51:11.001243    2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_26063eb2-0a4f-460f-a6ba-6d19ab96af70/kube-rbac-proxy/0.log"
Sep 17 18:51:11.029343 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:51:11.029316    2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_26063eb2-0a4f-460f-a6ba-6d19ab96af70/kube-rbac-proxy-metric/0.log"
Sep 17 18:51:11.059221 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:51:11.059191    2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_26063eb2-0a4f-460f-a6ba-6d19ab96af70/prom-label-proxy/0.log"
Sep 17 18:51:11.080315 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:51:11.080291    2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_26063eb2-0a4f-460f-a6ba-6d19ab96af70/init-config-reloader/0.log"
Sep 17 18:51:11.389993 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:51:11.389956    2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-ksdnc_748f74c6-2a2e-4d13-898e-662858625d39/node-exporter/0.log"
Sep 17 18:51:11.413053 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:51:11.413010    2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-ksdnc_748f74c6-2a2e-4d13-898e-662858625d39/kube-rbac-proxy/0.log"
Sep 17 18:51:11.438293 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:51:11.438266    2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-ksdnc_748f74c6-2a2e-4d13-898e-662858625d39/init-textfile/0.log"
Sep 17 18:51:11.588268 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:51:11.588241    2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_openshift-state-metrics-86c5bc79d6-hqncr_8dfa0270-d733-4438-b1e6-dde2f283d99b/kube-rbac-proxy-main/0.log"
Sep 17 18:51:11.622823 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:51:11.622799    2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_openshift-state-metrics-86c5bc79d6-hqncr_8dfa0270-d733-4438-b1e6-dde2f283d99b/kube-rbac-proxy-self/0.log"
Sep 17 18:51:11.657489 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:51:11.657433    2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_openshift-state-metrics-86c5bc79d6-hqncr_8dfa0270-d733-4438-b1e6-dde2f283d99b/openshift-state-metrics/0.log"
Sep 17 18:51:12.175526 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:51:12.175493    2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-569d74d969-7c5rn_fccda717-3dbe-4bb3-8bd0-d4c8bcb2855f/thanos-query/0.log"
Sep 17 18:51:12.198929 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:51:12.198903    2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-569d74d969-7c5rn_fccda717-3dbe-4bb3-8bd0-d4c8bcb2855f/kube-rbac-proxy-web/0.log"
Sep 17 18:51:12.220383 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:51:12.220350    2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-569d74d969-7c5rn_fccda717-3dbe-4bb3-8bd0-d4c8bcb2855f/kube-rbac-proxy/0.log"
Sep 17 18:51:12.242939 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:51:12.242915    2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-569d74d969-7c5rn_fccda717-3dbe-4bb3-8bd0-d4c8bcb2855f/prom-label-proxy/0.log"
Sep 17 18:51:12.270032 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:51:12.270010    2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-569d74d969-7c5rn_fccda717-3dbe-4bb3-8bd0-d4c8bcb2855f/kube-rbac-proxy-rules/0.log"
Sep 17 18:51:12.293327 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:51:12.293305    2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-569d74d969-7c5rn_fccda717-3dbe-4bb3-8bd0-d4c8bcb2855f/kube-rbac-proxy-metrics/0.log"
Sep 17 18:51:14.022935 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:51:14.022894    2575 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-must-gather-fdhq6/perf-node-gather-daemonset-hmrtt"]
Sep 17 18:51:14.023542 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:51:14.023497    2575 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="291f170d-6485-48c1-bf21-30f3adbd5f6e" containerName="main"
Sep 17 18:51:14.023542 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:51:14.023516    2575 state_mem.go:107] "Deleted CPUSet assignment" podUID="291f170d-6485-48c1-bf21-30f3adbd5f6e" containerName="main"
Sep 17 18:51:14.023669 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:51:14.023626    2575 memory_manager.go:368] "RemoveStaleState removing state" podUID="291f170d-6485-48c1-bf21-30f3adbd5f6e" containerName="main"
Sep 17 18:51:14.026841 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:51:14.026820    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-fdhq6/perf-node-gather-daemonset-hmrtt"
Sep 17 18:51:14.029626 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:51:14.029598    2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-fdhq6\"/\"kube-root-ca.crt\""
Sep 17 18:51:14.030339 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:51:14.030313    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-must-gather-fdhq6\"/\"default-dockercfg-2qc86\""
Sep 17 18:51:14.030431 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:51:14.030371    2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-fdhq6\"/\"openshift-service-ca.crt\""
Sep 17 18:51:14.038371 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:51:14.038342    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-fdhq6/perf-node-gather-daemonset-hmrtt"]
Sep 17 18:51:14.073034 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:51:14.072997    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/e100b5bb-0cf5-4e17-8bff-14ee824c806e-podres\") pod \"perf-node-gather-daemonset-hmrtt\" (UID: \"e100b5bb-0cf5-4e17-8bff-14ee824c806e\") " pod="openshift-must-gather-fdhq6/perf-node-gather-daemonset-hmrtt"
Sep 17 18:51:14.073197 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:51:14.073051    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/e100b5bb-0cf5-4e17-8bff-14ee824c806e-lib-modules\") pod \"perf-node-gather-daemonset-hmrtt\" (UID: \"e100b5bb-0cf5-4e17-8bff-14ee824c806e\") " pod="openshift-must-gather-fdhq6/perf-node-gather-daemonset-hmrtt"
Sep 17 18:51:14.073197 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:51:14.073135    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/e100b5bb-0cf5-4e17-8bff-14ee824c806e-sys\") pod \"perf-node-gather-daemonset-hmrtt\" (UID: \"e100b5bb-0cf5-4e17-8bff-14ee824c806e\") " pod="openshift-must-gather-fdhq6/perf-node-gather-daemonset-hmrtt"
Sep 17 18:51:14.073197 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:51:14.073179    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/e100b5bb-0cf5-4e17-8bff-14ee824c806e-proc\") pod \"perf-node-gather-daemonset-hmrtt\" (UID: \"e100b5bb-0cf5-4e17-8bff-14ee824c806e\") " pod="openshift-must-gather-fdhq6/perf-node-gather-daemonset-hmrtt"
Sep 17 18:51:14.073299 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:51:14.073238    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-b9lts\" (UniqueName: \"kubernetes.io/projected/e100b5bb-0cf5-4e17-8bff-14ee824c806e-kube-api-access-b9lts\") pod \"perf-node-gather-daemonset-hmrtt\" (UID: \"e100b5bb-0cf5-4e17-8bff-14ee824c806e\") " pod="openshift-must-gather-fdhq6/perf-node-gather-daemonset-hmrtt"
Sep 17 18:51:14.174569 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:51:14.174532    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/e100b5bb-0cf5-4e17-8bff-14ee824c806e-proc\") pod \"perf-node-gather-daemonset-hmrtt\" (UID: \"e100b5bb-0cf5-4e17-8bff-14ee824c806e\") " pod="openshift-must-gather-fdhq6/perf-node-gather-daemonset-hmrtt"
Sep 17 18:51:14.174717 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:51:14.174579    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-b9lts\" (UniqueName: \"kubernetes.io/projected/e100b5bb-0cf5-4e17-8bff-14ee824c806e-kube-api-access-b9lts\") pod \"perf-node-gather-daemonset-hmrtt\" (UID: \"e100b5bb-0cf5-4e17-8bff-14ee824c806e\") " pod="openshift-must-gather-fdhq6/perf-node-gather-daemonset-hmrtt"
Sep 17 18:51:14.174717 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:51:14.174618    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/e100b5bb-0cf5-4e17-8bff-14ee824c806e-podres\") pod \"perf-node-gather-daemonset-hmrtt\" (UID: \"e100b5bb-0cf5-4e17-8bff-14ee824c806e\") " pod="openshift-must-gather-fdhq6/perf-node-gather-daemonset-hmrtt"
Sep 17 18:51:14.174717 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:51:14.174649    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/e100b5bb-0cf5-4e17-8bff-14ee824c806e-lib-modules\") pod \"perf-node-gather-daemonset-hmrtt\" (UID: \"e100b5bb-0cf5-4e17-8bff-14ee824c806e\") " pod="openshift-must-gather-fdhq6/perf-node-gather-daemonset-hmrtt"
Sep 17 18:51:14.174717 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:51:14.174663    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/e100b5bb-0cf5-4e17-8bff-14ee824c806e-proc\") pod \"perf-node-gather-daemonset-hmrtt\" (UID: \"e100b5bb-0cf5-4e17-8bff-14ee824c806e\") " pod="openshift-must-gather-fdhq6/perf-node-gather-daemonset-hmrtt"
Sep 17 18:51:14.174717 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:51:14.174699    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/e100b5bb-0cf5-4e17-8bff-14ee824c806e-sys\") pod \"perf-node-gather-daemonset-hmrtt\" (UID: \"e100b5bb-0cf5-4e17-8bff-14ee824c806e\") " pod="openshift-must-gather-fdhq6/perf-node-gather-daemonset-hmrtt"
Sep 17 18:51:14.174885 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:51:14.174782    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/e100b5bb-0cf5-4e17-8bff-14ee824c806e-lib-modules\") pod \"perf-node-gather-daemonset-hmrtt\" (UID: \"e100b5bb-0cf5-4e17-8bff-14ee824c806e\") " pod="openshift-must-gather-fdhq6/perf-node-gather-daemonset-hmrtt"
Sep 17 18:51:14.174885 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:51:14.174786    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/e100b5bb-0cf5-4e17-8bff-14ee824c806e-sys\") pod \"perf-node-gather-daemonset-hmrtt\" (UID: \"e100b5bb-0cf5-4e17-8bff-14ee824c806e\") " pod="openshift-must-gather-fdhq6/perf-node-gather-daemonset-hmrtt"
Sep 17 18:51:14.174885 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:51:14.174786    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/e100b5bb-0cf5-4e17-8bff-14ee824c806e-podres\") pod \"perf-node-gather-daemonset-hmrtt\" (UID: \"e100b5bb-0cf5-4e17-8bff-14ee824c806e\") " pod="openshift-must-gather-fdhq6/perf-node-gather-daemonset-hmrtt"
Sep 17 18:51:14.183930 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:51:14.183907    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-b9lts\" (UniqueName: \"kubernetes.io/projected/e100b5bb-0cf5-4e17-8bff-14ee824c806e-kube-api-access-b9lts\") pod \"perf-node-gather-daemonset-hmrtt\" (UID: \"e100b5bb-0cf5-4e17-8bff-14ee824c806e\") " pod="openshift-must-gather-fdhq6/perf-node-gather-daemonset-hmrtt"
Sep 17 18:51:14.337046 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:51:14.336955    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-fdhq6/perf-node-gather-daemonset-hmrtt"
Sep 17 18:51:14.518725 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:51:14.518691    2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console_downloads-6594f4c75-lblbf_13f90145-45b6-49b9-a30e-bb2064b55386/download-server/0.log"
Sep 17 18:51:14.610983 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:51:14.610948    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-fdhq6/perf-node-gather-daemonset-hmrtt"]
Sep 17 18:51:14.613271 ip-10-0-143-74 kubenswrapper[2575]: W0917 18:51:14.613242    2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-pode100b5bb_0cf5_4e17_8bff_14ee824c806e.slice/crio-e50fecefd336342adc66c84ca0eb49edebbac1b86627a04089149fef5b08b3a7 WatchSource:0}: Error finding container e50fecefd336342adc66c84ca0eb49edebbac1b86627a04089149fef5b08b3a7: Status 404 returned error can't find the container with id e50fecefd336342adc66c84ca0eb49edebbac1b86627a04089149fef5b08b3a7
Sep 17 18:51:15.349375 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:51:15.349333    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-fdhq6/perf-node-gather-daemonset-hmrtt" event={"ID":"e100b5bb-0cf5-4e17-8bff-14ee824c806e","Type":"ContainerStarted","Data":"8ddc93df5e60d36fed8305ccba7f754eb0a781c44cc89e43036d4e0787537b5f"}
Sep 17 18:51:15.349781 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:51:15.349396    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-fdhq6/perf-node-gather-daemonset-hmrtt" event={"ID":"e100b5bb-0cf5-4e17-8bff-14ee824c806e","Type":"ContainerStarted","Data":"e50fecefd336342adc66c84ca0eb49edebbac1b86627a04089149fef5b08b3a7"}
Sep 17 18:51:15.349781 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:51:15.349475    2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-must-gather-fdhq6/perf-node-gather-daemonset-hmrtt"
Sep 17 18:51:15.368083 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:51:15.368031    2575 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-must-gather-fdhq6/perf-node-gather-daemonset-hmrtt" podStartSLOduration=1.36801573 podStartE2EDuration="1.36801573s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-17 18:51:14 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-17 18:51:15.367282733 +0000 UTC m=+4594.997612708" watchObservedRunningTime="2026-09-17 18:51:15.36801573 +0000 UTC m=+4594.998345742"
Sep 17 18:51:16.136396 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:51:16.136361    2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-tl8ds_b8b3d730-8877-45f1-8500-451ee8ac53a3/dns/0.log"
Sep 17 18:51:16.157487 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:51:16.157438    2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-tl8ds_b8b3d730-8877-45f1-8500-451ee8ac53a3/kube-rbac-proxy/0.log"
Sep 17 18:51:16.232317 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:51:16.232286    2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-5549b_93fd2156-b185-4995-8e49-d6d9c0eda255/dns-node-resolver/0.log"
Sep 17 18:51:16.801083 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:51:16.801049    2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_image-registry-567fd9f7c4-z54rn_9476f49c-219d-4d93-8ac2-b98df60c6b22/registry/0.log"
Sep 17 18:51:16.817132 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:51:16.817101    2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_image-registry-567fd9f7c4-z54rn_9476f49c-219d-4d93-8ac2-b98df60c6b22/registry/1.log"
Sep 17 18:51:16.840501 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:51:16.840474    2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-2fp6z_173c8902-414c-4212-8d93-ce15853e8ca8/node-ca/0.log"
Sep 17 18:51:17.963788 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:51:17.963752    2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_istiod-openshift-gateway-5677ccb49b-ft57x_7ca824c6-1031-48be-af6a-0d0b5de1aa51/discovery/0.log"
Sep 17 18:51:18.014620 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:51:18.014588    2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_router-default-76f78f8976-kwv2h_3a6e39e9-229a-440e-a565-16ef48e36247/router/0.log"
Sep 17 18:51:18.505311 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:51:18.505274    2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-4p2w5_cacfc0cf-465f-43c5-bfe6-dd4934eb4a6f/serve-healthcheck-canary/0.log"
Sep 17 18:51:18.996763 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:51:18.996731    2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-operator-6f9c644dd9-dllkw_b253f0e2-6a5b-4fc4-b686-db5ae716f81b/insights-operator/0.log"
Sep 17 18:51:18.997249 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:51:18.997231    2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-operator-6f9c644dd9-dllkw_b253f0e2-6a5b-4fc4-b686-db5ae716f81b/insights-operator/1.log"
Sep 17 18:51:19.094128 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:51:19.094098    2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-gspvq_1e261d7f-9daa-43ed-84f0-8d4d2aea69d7/kube-rbac-proxy/0.log"
Sep 17 18:51:19.118260 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:51:19.118232    2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-gspvq_1e261d7f-9daa-43ed-84f0-8d4d2aea69d7/exporter/0.log"
Sep 17 18:51:19.141577 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:51:19.141552    2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-gspvq_1e261d7f-9daa-43ed-84f0-8d4d2aea69d7/extractor/0.log"
Sep 17 18:51:21.365232 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:51:21.365204    2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-must-gather-fdhq6/perf-node-gather-daemonset-hmrtt"
Sep 17 18:51:22.873913 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:51:22.873885    2575 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve_kserve-controller-manager-9b487cb79-x9vqw_1b0ee150-8018-4525-9a50-8f2c70195c99/manager/0.log"
Sep 17 18:51:23.145159 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:51:23.145123    2575 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve_llmisvc-controller-manager-6b54c95c95-m69ds_473587de-f311-4519-9e1c-10154aa00c2b/manager/0.log"
Sep 17 18:51:23.689682 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:51:23.689597    2575 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve_odh-model-controller-7b966d7984-5cnqv_8dd98cfc-69b0-4814-beca-bee51a1fdaf8/manager/0.log"
Sep 17 18:51:23.780503 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:51:23.780472    2575 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve_seaweedfs-64568bcd49-6mnw4_29930da5-fff7-42a4-bf2c-30f5d52ed31e/seaweedfs/0.log"
Sep 17 18:51:23.867027 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:51:23.866988    2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_istiod-openshift-gateway-5677ccb49b-ft57x_7ca824c6-1031-48be-af6a-0d0b5de1aa51/discovery/0.log"
Sep 17 18:51:23.909184 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:51:23.909159    2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_router-default-76f78f8976-kwv2h_3a6e39e9-229a-440e-a565-16ef48e36247/router/0.log"
Sep 17 18:51:24.979336 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:51:24.979305    2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_istiod-openshift-gateway-5677ccb49b-ft57x_7ca824c6-1031-48be-af6a-0d0b5de1aa51/discovery/0.log"
Sep 17 18:51:25.010684 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:51:25.010657    2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_router-default-76f78f8976-kwv2h_3a6e39e9-229a-440e-a565-16ef48e36247/router/0.log"
Sep 17 18:51:28.711184 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:51:28.711150    2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-kube-storage-version-migrator_migrator-76794cbd6c-mm4dm_66bf1996-7788-4880-a36d-4efa474cdb85/migrator/0.log"
Sep 17 18:51:28.732298 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:51:28.732270    2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-kube-storage-version-migrator_migrator-76794cbd6c-mm4dm_66bf1996-7788-4880-a36d-4efa474cdb85/graceful-termination/0.log"
Sep 17 18:51:30.234730 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:51:30.234700    2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-gtxcq_d52f9540-79e9-4df7-b3a9-e8c7cc6d48b0/kube-multus-additional-cni-plugins/0.log"
Sep 17 18:51:30.257751 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:51:30.257728    2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-gtxcq_d52f9540-79e9-4df7-b3a9-e8c7cc6d48b0/egress-router-binary-copy/0.log"
Sep 17 18:51:30.282354 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:51:30.282322    2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-gtxcq_d52f9540-79e9-4df7-b3a9-e8c7cc6d48b0/cni-plugins/0.log"
Sep 17 18:51:30.336400 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:51:30.336374    2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-gtxcq_d52f9540-79e9-4df7-b3a9-e8c7cc6d48b0/bond-cni-plugin/0.log"
Sep 17 18:51:30.355126 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:51:30.355098    2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-gtxcq_d52f9540-79e9-4df7-b3a9-e8c7cc6d48b0/routeoverride-cni/0.log"
Sep 17 18:51:30.375768 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:51:30.375733    2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-gtxcq_d52f9540-79e9-4df7-b3a9-e8c7cc6d48b0/whereabouts-cni-bincopy/0.log"
Sep 17 18:51:30.400041 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:51:30.400015    2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-gtxcq_d52f9540-79e9-4df7-b3a9-e8c7cc6d48b0/whereabouts-cni/0.log"
Sep 17 18:51:30.626652 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:51:30.626619    2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-srdtd_fda64577-6753-498e-ab9e-128a567424ca/kube-multus/0.log"
Sep 17 18:51:30.747003 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:51:30.746971    2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_network-metrics-daemon-wdp4h_27866e17-c231-49a6-921f-a83a01b8e173/network-metrics-daemon/0.log"
Sep 17 18:51:30.765115 ip-10-0-143-74 kubenswrapper[2575]: I0917 18:51:30.765088    2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_network-metrics-daemon-wdp4h_27866e17-c231-49a6-921f-a83a01b8e173/kube-rbac-proxy/0.log"