Apr 24 14:21:30.429509 ip-10-0-136-132 systemd[1]: kubelet.service: Failed to load environment files: No such file or directory
Apr 24 14:21:30.429643 ip-10-0-136-132 systemd[1]: kubelet.service: Failed to run 'start-pre' task: No such file or directory
Apr 24 14:21:30.429730 ip-10-0-136-132 systemd[1]: kubelet.service: Failed with result 'resources'.
Apr 24 14:21:30.430244 ip-10-0-136-132 systemd[1]: Failed to start Kubernetes Kubelet.
Apr 24 14:21:41.741142 ip-10-0-136-132 systemd[1]: kubelet.service: Failed to schedule restart job: Unit crio.service not found.
Apr 24 14:21:41.741155 ip-10-0-136-132 systemd[1]: kubelet.service: Failed with result 'resources'.
-- Boot 0fd73915336b4aa8aa288e7f22ab03dc --
Apr 24 14:23:57.164631 ip-10-0-136-132 systemd[1]: Starting Kubernetes Kubelet...
Apr 24 14:23:57.557297 ip-10-0-136-132 kubenswrapper[2574]: Flag --container-runtime-endpoint has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Apr 24 14:23:57.557297 ip-10-0-136-132 kubenswrapper[2574]: Flag --minimum-container-ttl-duration has been deprecated, Use --eviction-hard or --eviction-soft instead. Will be removed in a future version.
Apr 24 14:23:57.557297 ip-10-0-136-132 kubenswrapper[2574]: Flag --volume-plugin-dir has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Apr 24 14:23:57.557297 ip-10-0-136-132 kubenswrapper[2574]: Flag --pod-infra-container-image has been deprecated, will be removed in 1.35. Image garbage collector will get sandbox image information from CRI.
Apr 24 14:23:57.557297 ip-10-0-136-132 kubenswrapper[2574]: Flag --system-reserved has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Apr 24 14:23:57.559715 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:23:57.559617    2574 server.go:212] "--pod-infra-container-image will not be pruned by the image garbage collector in kubelet and should also be set in the remote runtime"
Apr 24 14:23:57.566029 ip-10-0-136-132 kubenswrapper[2574]: W0424 14:23:57.566005    2574 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Apr 24 14:23:57.566029 ip-10-0-136-132 kubenswrapper[2574]: W0424 14:23:57.566025    2574 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Apr 24 14:23:57.566029 ip-10-0-136-132 kubenswrapper[2574]: W0424 14:23:57.566033    2574 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Apr 24 14:23:57.566029 ip-10-0-136-132 kubenswrapper[2574]: W0424 14:23:57.566038    2574 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Apr 24 14:23:57.566284 ip-10-0-136-132 kubenswrapper[2574]: W0424 14:23:57.566042    2574 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Apr 24 14:23:57.566284 ip-10-0-136-132 kubenswrapper[2574]: W0424 14:23:57.566047    2574 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Apr 24 14:23:57.566284 ip-10-0-136-132 kubenswrapper[2574]: W0424 14:23:57.566051    2574 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Apr 24 14:23:57.566284 ip-10-0-136-132 kubenswrapper[2574]: W0424 14:23:57.566056    2574 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Apr 24 14:23:57.566284 ip-10-0-136-132 kubenswrapper[2574]: W0424 14:23:57.566060    2574 feature_gate.go:328] unrecognized feature gate: OVNObservability
Apr 24 14:23:57.566284 ip-10-0-136-132 kubenswrapper[2574]: W0424 14:23:57.566064    2574 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Apr 24 14:23:57.566284 ip-10-0-136-132 kubenswrapper[2574]: W0424 14:23:57.566068    2574 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Apr 24 14:23:57.566284 ip-10-0-136-132 kubenswrapper[2574]: W0424 14:23:57.566071    2574 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Apr 24 14:23:57.566284 ip-10-0-136-132 kubenswrapper[2574]: W0424 14:23:57.566075    2574 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Apr 24 14:23:57.566284 ip-10-0-136-132 kubenswrapper[2574]: W0424 14:23:57.566079    2574 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Apr 24 14:23:57.566284 ip-10-0-136-132 kubenswrapper[2574]: W0424 14:23:57.566083    2574 feature_gate.go:328] unrecognized feature gate: IngressControllerLBSubnetsAWS
Apr 24 14:23:57.566284 ip-10-0-136-132 kubenswrapper[2574]: W0424 14:23:57.566087    2574 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Apr 24 14:23:57.566284 ip-10-0-136-132 kubenswrapper[2574]: W0424 14:23:57.566091    2574 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Apr 24 14:23:57.566284 ip-10-0-136-132 kubenswrapper[2574]: W0424 14:23:57.566095    2574 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Apr 24 14:23:57.566284 ip-10-0-136-132 kubenswrapper[2574]: W0424 14:23:57.566101    2574 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Apr 24 14:23:57.566284 ip-10-0-136-132 kubenswrapper[2574]: W0424 14:23:57.566107    2574 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Apr 24 14:23:57.566284 ip-10-0-136-132 kubenswrapper[2574]: W0424 14:23:57.566113    2574 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Apr 24 14:23:57.566284 ip-10-0-136-132 kubenswrapper[2574]: W0424 14:23:57.566126    2574 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Apr 24 14:23:57.566284 ip-10-0-136-132 kubenswrapper[2574]: W0424 14:23:57.566130    2574 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Apr 24 14:23:57.567051 ip-10-0-136-132 kubenswrapper[2574]: W0424 14:23:57.566134    2574 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Apr 24 14:23:57.567051 ip-10-0-136-132 kubenswrapper[2574]: W0424 14:23:57.566138    2574 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Apr 24 14:23:57.567051 ip-10-0-136-132 kubenswrapper[2574]: W0424 14:23:57.566142    2574 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Apr 24 14:23:57.567051 ip-10-0-136-132 kubenswrapper[2574]: W0424 14:23:57.566146    2574 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Apr 24 14:23:57.567051 ip-10-0-136-132 kubenswrapper[2574]: W0424 14:23:57.566151    2574 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Apr 24 14:23:57.567051 ip-10-0-136-132 kubenswrapper[2574]: W0424 14:23:57.566154    2574 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Apr 24 14:23:57.567051 ip-10-0-136-132 kubenswrapper[2574]: W0424 14:23:57.566158    2574 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Apr 24 14:23:57.567051 ip-10-0-136-132 kubenswrapper[2574]: W0424 14:23:57.566164    2574 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Apr 24 14:23:57.567051 ip-10-0-136-132 kubenswrapper[2574]: W0424 14:23:57.566168    2574 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterOperations
Apr 24 14:23:57.567051 ip-10-0-136-132 kubenswrapper[2574]: W0424 14:23:57.566172    2574 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Apr 24 14:23:57.567051 ip-10-0-136-132 kubenswrapper[2574]: W0424 14:23:57.566177    2574 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Apr 24 14:23:57.567051 ip-10-0-136-132 kubenswrapper[2574]: W0424 14:23:57.566181    2574 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Apr 24 14:23:57.567051 ip-10-0-136-132 kubenswrapper[2574]: W0424 14:23:57.566185    2574 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Apr 24 14:23:57.567051 ip-10-0-136-132 kubenswrapper[2574]: W0424 14:23:57.566189    2574 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Apr 24 14:23:57.567051 ip-10-0-136-132 kubenswrapper[2574]: W0424 14:23:57.566196    2574 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Apr 24 14:23:57.567051 ip-10-0-136-132 kubenswrapper[2574]: W0424 14:23:57.566200    2574 feature_gate.go:328] unrecognized feature gate: Example
Apr 24 14:23:57.567051 ip-10-0-136-132 kubenswrapper[2574]: W0424 14:23:57.566238    2574 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Apr 24 14:23:57.567051 ip-10-0-136-132 kubenswrapper[2574]: W0424 14:23:57.566245    2574 feature_gate.go:328] unrecognized feature gate: SignatureStores
Apr 24 14:23:57.567051 ip-10-0-136-132 kubenswrapper[2574]: W0424 14:23:57.566251    2574 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Apr 24 14:23:57.567051 ip-10-0-136-132 kubenswrapper[2574]: W0424 14:23:57.566256    2574 feature_gate.go:328] unrecognized feature gate: PinnedImages
Apr 24 14:23:57.567849 ip-10-0-136-132 kubenswrapper[2574]: W0424 14:23:57.566260    2574 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Apr 24 14:23:57.567849 ip-10-0-136-132 kubenswrapper[2574]: W0424 14:23:57.566265    2574 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Apr 24 14:23:57.567849 ip-10-0-136-132 kubenswrapper[2574]: W0424 14:23:57.566270    2574 feature_gate.go:328] unrecognized feature gate: DualReplica
Apr 24 14:23:57.567849 ip-10-0-136-132 kubenswrapper[2574]: W0424 14:23:57.566274    2574 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Apr 24 14:23:57.567849 ip-10-0-136-132 kubenswrapper[2574]: W0424 14:23:57.566279    2574 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Apr 24 14:23:57.567849 ip-10-0-136-132 kubenswrapper[2574]: W0424 14:23:57.566283    2574 feature_gate.go:328] unrecognized feature gate: NewOLM
Apr 24 14:23:57.567849 ip-10-0-136-132 kubenswrapper[2574]: W0424 14:23:57.566287    2574 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Apr 24 14:23:57.567849 ip-10-0-136-132 kubenswrapper[2574]: W0424 14:23:57.566292    2574 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Apr 24 14:23:57.567849 ip-10-0-136-132 kubenswrapper[2574]: W0424 14:23:57.566296    2574 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Apr 24 14:23:57.567849 ip-10-0-136-132 kubenswrapper[2574]: W0424 14:23:57.566302    2574 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Apr 24 14:23:57.567849 ip-10-0-136-132 kubenswrapper[2574]: W0424 14:23:57.566306    2574 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Apr 24 14:23:57.567849 ip-10-0-136-132 kubenswrapper[2574]: W0424 14:23:57.566311    2574 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Apr 24 14:23:57.567849 ip-10-0-136-132 kubenswrapper[2574]: W0424 14:23:57.566315    2574 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Apr 24 14:23:57.567849 ip-10-0-136-132 kubenswrapper[2574]: W0424 14:23:57.566319    2574 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Apr 24 14:23:57.567849 ip-10-0-136-132 kubenswrapper[2574]: W0424 14:23:57.566323    2574 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Apr 24 14:23:57.567849 ip-10-0-136-132 kubenswrapper[2574]: W0424 14:23:57.566328    2574 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Apr 24 14:23:57.567849 ip-10-0-136-132 kubenswrapper[2574]: W0424 14:23:57.566349    2574 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Apr 24 14:23:57.567849 ip-10-0-136-132 kubenswrapper[2574]: W0424 14:23:57.566354    2574 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Apr 24 14:23:57.567849 ip-10-0-136-132 kubenswrapper[2574]: W0424 14:23:57.566359    2574 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Apr 24 14:23:57.567849 ip-10-0-136-132 kubenswrapper[2574]: W0424 14:23:57.566363    2574 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Apr 24 14:23:57.568474 ip-10-0-136-132 kubenswrapper[2574]: W0424 14:23:57.566369    2574 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Apr 24 14:23:57.568474 ip-10-0-136-132 kubenswrapper[2574]: W0424 14:23:57.566373    2574 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Apr 24 14:23:57.568474 ip-10-0-136-132 kubenswrapper[2574]: W0424 14:23:57.566377    2574 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Apr 24 14:23:57.568474 ip-10-0-136-132 kubenswrapper[2574]: W0424 14:23:57.566382    2574 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Apr 24 14:23:57.568474 ip-10-0-136-132 kubenswrapper[2574]: W0424 14:23:57.566386    2574 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Apr 24 14:23:57.568474 ip-10-0-136-132 kubenswrapper[2574]: W0424 14:23:57.566390    2574 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Apr 24 14:23:57.568474 ip-10-0-136-132 kubenswrapper[2574]: W0424 14:23:57.566396    2574 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Apr 24 14:23:57.568474 ip-10-0-136-132 kubenswrapper[2574]: W0424 14:23:57.566400    2574 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Apr 24 14:23:57.568474 ip-10-0-136-132 kubenswrapper[2574]: W0424 14:23:57.566404    2574 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Apr 24 14:23:57.568474 ip-10-0-136-132 kubenswrapper[2574]: W0424 14:23:57.566408    2574 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Apr 24 14:23:57.568474 ip-10-0-136-132 kubenswrapper[2574]: W0424 14:23:57.566412    2574 feature_gate.go:328] unrecognized feature gate: SetEIPForNLBIngressController
Apr 24 14:23:57.568474 ip-10-0-136-132 kubenswrapper[2574]: W0424 14:23:57.566417    2574 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Apr 24 14:23:57.568474 ip-10-0-136-132 kubenswrapper[2574]: W0424 14:23:57.566421    2574 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Apr 24 14:23:57.568474 ip-10-0-136-132 kubenswrapper[2574]: W0424 14:23:57.566425    2574 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Apr 24 14:23:57.568474 ip-10-0-136-132 kubenswrapper[2574]: W0424 14:23:57.566429    2574 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Apr 24 14:23:57.568474 ip-10-0-136-132 kubenswrapper[2574]: W0424 14:23:57.566433    2574 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Apr 24 14:23:57.568474 ip-10-0-136-132 kubenswrapper[2574]: W0424 14:23:57.566437    2574 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Apr 24 14:23:57.568474 ip-10-0-136-132 kubenswrapper[2574]: W0424 14:23:57.566441    2574 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Apr 24 14:23:57.568474 ip-10-0-136-132 kubenswrapper[2574]: W0424 14:23:57.566445    2574 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Apr 24 14:23:57.569250 ip-10-0-136-132 kubenswrapper[2574]: W0424 14:23:57.566449    2574 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Apr 24 14:23:57.569250 ip-10-0-136-132 kubenswrapper[2574]: W0424 14:23:57.566452    2574 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Apr 24 14:23:57.569250 ip-10-0-136-132 kubenswrapper[2574]: W0424 14:23:57.566457    2574 feature_gate.go:328] unrecognized feature gate: Example2
Apr 24 14:23:57.569250 ip-10-0-136-132 kubenswrapper[2574]: W0424 14:23:57.566461    2574 feature_gate.go:328] unrecognized feature gate: InsightsConfigAPI
Apr 24 14:23:57.569250 ip-10-0-136-132 kubenswrapper[2574]: W0424 14:23:57.567110    2574 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Apr 24 14:23:57.569250 ip-10-0-136-132 kubenswrapper[2574]: W0424 14:23:57.567120    2574 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Apr 24 14:23:57.569250 ip-10-0-136-132 kubenswrapper[2574]: W0424 14:23:57.567125    2574 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Apr 24 14:23:57.569250 ip-10-0-136-132 kubenswrapper[2574]: W0424 14:23:57.567130    2574 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Apr 24 14:23:57.569250 ip-10-0-136-132 kubenswrapper[2574]: W0424 14:23:57.567135    2574 feature_gate.go:328] unrecognized feature gate: Example2
Apr 24 14:23:57.569250 ip-10-0-136-132 kubenswrapper[2574]: W0424 14:23:57.567140    2574 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Apr 24 14:23:57.569250 ip-10-0-136-132 kubenswrapper[2574]: W0424 14:23:57.567145    2574 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Apr 24 14:23:57.569250 ip-10-0-136-132 kubenswrapper[2574]: W0424 14:23:57.567149    2574 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Apr 24 14:23:57.569250 ip-10-0-136-132 kubenswrapper[2574]: W0424 14:23:57.567154    2574 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Apr 24 14:23:57.569250 ip-10-0-136-132 kubenswrapper[2574]: W0424 14:23:57.567160    2574 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Apr 24 14:23:57.569250 ip-10-0-136-132 kubenswrapper[2574]: W0424 14:23:57.567165    2574 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Apr 24 14:23:57.569250 ip-10-0-136-132 kubenswrapper[2574]: W0424 14:23:57.567169    2574 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Apr 24 14:23:57.569250 ip-10-0-136-132 kubenswrapper[2574]: W0424 14:23:57.567173    2574 feature_gate.go:328] unrecognized feature gate: IngressControllerLBSubnetsAWS
Apr 24 14:23:57.569250 ip-10-0-136-132 kubenswrapper[2574]: W0424 14:23:57.567178    2574 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Apr 24 14:23:57.569250 ip-10-0-136-132 kubenswrapper[2574]: W0424 14:23:57.567182    2574 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Apr 24 14:23:57.569250 ip-10-0-136-132 kubenswrapper[2574]: W0424 14:23:57.567187    2574 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Apr 24 14:23:57.569829 ip-10-0-136-132 kubenswrapper[2574]: W0424 14:23:57.567191    2574 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Apr 24 14:23:57.569829 ip-10-0-136-132 kubenswrapper[2574]: W0424 14:23:57.567195    2574 feature_gate.go:328] unrecognized feature gate: Example
Apr 24 14:23:57.569829 ip-10-0-136-132 kubenswrapper[2574]: W0424 14:23:57.567200    2574 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Apr 24 14:23:57.569829 ip-10-0-136-132 kubenswrapper[2574]: W0424 14:23:57.567204    2574 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Apr 24 14:23:57.569829 ip-10-0-136-132 kubenswrapper[2574]: W0424 14:23:57.567208    2574 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Apr 24 14:23:57.569829 ip-10-0-136-132 kubenswrapper[2574]: W0424 14:23:57.567212    2574 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterOperations
Apr 24 14:23:57.569829 ip-10-0-136-132 kubenswrapper[2574]: W0424 14:23:57.567217    2574 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Apr 24 14:23:57.569829 ip-10-0-136-132 kubenswrapper[2574]: W0424 14:23:57.567221    2574 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Apr 24 14:23:57.569829 ip-10-0-136-132 kubenswrapper[2574]: W0424 14:23:57.567225    2574 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Apr 24 14:23:57.569829 ip-10-0-136-132 kubenswrapper[2574]: W0424 14:23:57.567229    2574 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Apr 24 14:23:57.569829 ip-10-0-136-132 kubenswrapper[2574]: W0424 14:23:57.567234    2574 feature_gate.go:328] unrecognized feature gate: NewOLM
Apr 24 14:23:57.569829 ip-10-0-136-132 kubenswrapper[2574]: W0424 14:23:57.567238    2574 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Apr 24 14:23:57.569829 ip-10-0-136-132 kubenswrapper[2574]: W0424 14:23:57.567242    2574 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Apr 24 14:23:57.569829 ip-10-0-136-132 kubenswrapper[2574]: W0424 14:23:57.567246    2574 feature_gate.go:328] unrecognized feature gate: OVNObservability
Apr 24 14:23:57.569829 ip-10-0-136-132 kubenswrapper[2574]: W0424 14:23:57.567251    2574 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Apr 24 14:23:57.569829 ip-10-0-136-132 kubenswrapper[2574]: W0424 14:23:57.567255    2574 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Apr 24 14:23:57.569829 ip-10-0-136-132 kubenswrapper[2574]: W0424 14:23:57.567259    2574 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Apr 24 14:23:57.569829 ip-10-0-136-132 kubenswrapper[2574]: W0424 14:23:57.567263    2574 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Apr 24 14:23:57.569829 ip-10-0-136-132 kubenswrapper[2574]: W0424 14:23:57.567267    2574 feature_gate.go:328] unrecognized feature gate: SignatureStores
Apr 24 14:23:57.569829 ip-10-0-136-132 kubenswrapper[2574]: W0424 14:23:57.567271    2574 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Apr 24 14:23:57.570322 ip-10-0-136-132 kubenswrapper[2574]: W0424 14:23:57.567275    2574 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Apr 24 14:23:57.570322 ip-10-0-136-132 kubenswrapper[2574]: W0424 14:23:57.567279    2574 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Apr 24 14:23:57.570322 ip-10-0-136-132 kubenswrapper[2574]: W0424 14:23:57.567284    2574 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Apr 24 14:23:57.570322 ip-10-0-136-132 kubenswrapper[2574]: W0424 14:23:57.567288    2574 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Apr 24 14:23:57.570322 ip-10-0-136-132 kubenswrapper[2574]: W0424 14:23:57.567292    2574 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Apr 24 14:23:57.570322 ip-10-0-136-132 kubenswrapper[2574]: W0424 14:23:57.567297    2574 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Apr 24 14:23:57.570322 ip-10-0-136-132 kubenswrapper[2574]: W0424 14:23:57.567302    2574 feature_gate.go:328] unrecognized feature gate: SetEIPForNLBIngressController
Apr 24 14:23:57.570322 ip-10-0-136-132 kubenswrapper[2574]: W0424 14:23:57.567306    2574 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Apr 24 14:23:57.570322 ip-10-0-136-132 kubenswrapper[2574]: W0424 14:23:57.567310    2574 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Apr 24 14:23:57.570322 ip-10-0-136-132 kubenswrapper[2574]: W0424 14:23:57.567314    2574 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Apr 24 14:23:57.570322 ip-10-0-136-132 kubenswrapper[2574]: W0424 14:23:57.567319    2574 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Apr 24 14:23:57.570322 ip-10-0-136-132 kubenswrapper[2574]: W0424 14:23:57.567329    2574 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Apr 24 14:23:57.570322 ip-10-0-136-132 kubenswrapper[2574]: W0424 14:23:57.567351    2574 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Apr 24 14:23:57.570322 ip-10-0-136-132 kubenswrapper[2574]: W0424 14:23:57.567356    2574 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Apr 24 14:23:57.570322 ip-10-0-136-132 kubenswrapper[2574]: W0424 14:23:57.567360    2574 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Apr 24 14:23:57.570322 ip-10-0-136-132 kubenswrapper[2574]: W0424 14:23:57.567364    2574 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Apr 24 14:23:57.570322 ip-10-0-136-132 kubenswrapper[2574]: W0424 14:23:57.567368    2574 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Apr 24 14:23:57.570322 ip-10-0-136-132 kubenswrapper[2574]: W0424 14:23:57.567372    2574 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Apr 24 14:23:57.570322 ip-10-0-136-132 kubenswrapper[2574]: W0424 14:23:57.567376    2574 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Apr 24 14:23:57.570815 ip-10-0-136-132 kubenswrapper[2574]: W0424 14:23:57.567380    2574 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Apr 24 14:23:57.570815 ip-10-0-136-132 kubenswrapper[2574]: W0424 14:23:57.567384    2574 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Apr 24 14:23:57.570815 ip-10-0-136-132 kubenswrapper[2574]: W0424 14:23:57.567388    2574 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Apr 24 14:23:57.570815 ip-10-0-136-132 kubenswrapper[2574]: W0424 14:23:57.567392    2574 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Apr 24 14:23:57.570815 ip-10-0-136-132 kubenswrapper[2574]: W0424 14:23:57.567396    2574 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Apr 24 14:23:57.570815 ip-10-0-136-132 kubenswrapper[2574]: W0424 14:23:57.567401    2574 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Apr 24 14:23:57.570815 ip-10-0-136-132 kubenswrapper[2574]: W0424 14:23:57.567405    2574 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Apr 24 14:23:57.570815 ip-10-0-136-132 kubenswrapper[2574]: W0424 14:23:57.567409    2574 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Apr 24 14:23:57.570815 ip-10-0-136-132 kubenswrapper[2574]: W0424 14:23:57.567413    2574 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Apr 24 14:23:57.570815 ip-10-0-136-132 kubenswrapper[2574]: W0424 14:23:57.567417    2574 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Apr 24 14:23:57.570815 ip-10-0-136-132 kubenswrapper[2574]: W0424 14:23:57.567421    2574 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Apr 24 14:23:57.570815 ip-10-0-136-132 kubenswrapper[2574]: W0424 14:23:57.567425    2574 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Apr 24 14:23:57.570815 ip-10-0-136-132 kubenswrapper[2574]: W0424 14:23:57.567432    2574 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Apr 24 14:23:57.570815 ip-10-0-136-132 kubenswrapper[2574]: W0424 14:23:57.567438    2574 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Apr 24 14:23:57.570815 ip-10-0-136-132 kubenswrapper[2574]: W0424 14:23:57.567443    2574 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Apr 24 14:23:57.570815 ip-10-0-136-132 kubenswrapper[2574]: W0424 14:23:57.567448    2574 feature_gate.go:328] unrecognized feature gate: PinnedImages
Apr 24 14:23:57.570815 ip-10-0-136-132 kubenswrapper[2574]: W0424 14:23:57.567453    2574 feature_gate.go:328] unrecognized feature gate: InsightsConfigAPI
Apr 24 14:23:57.570815 ip-10-0-136-132 kubenswrapper[2574]: W0424 14:23:57.567458    2574 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Apr 24 14:23:57.570815 ip-10-0-136-132 kubenswrapper[2574]: W0424 14:23:57.567463    2574 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Apr 24 14:23:57.570815 ip-10-0-136-132 kubenswrapper[2574]: W0424 14:23:57.567469    2574 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Apr 24 14:23:57.571303 ip-10-0-136-132 kubenswrapper[2574]: W0424 14:23:57.567474    2574 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Apr 24 14:23:57.571303 ip-10-0-136-132 kubenswrapper[2574]: W0424 14:23:57.567479    2574 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Apr 24 14:23:57.571303 ip-10-0-136-132 kubenswrapper[2574]: W0424 14:23:57.567484    2574 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Apr 24 14:23:57.571303 ip-10-0-136-132 kubenswrapper[2574]: W0424 14:23:57.567488    2574 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Apr 24 14:23:57.571303 ip-10-0-136-132 kubenswrapper[2574]: W0424 14:23:57.567498    2574 feature_gate.go:328] unrecognized feature gate: DualReplica
Apr 24 14:23:57.571303 ip-10-0-136-132 kubenswrapper[2574]: W0424 14:23:57.567503    2574 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Apr 24 14:23:57.571303 ip-10-0-136-132 kubenswrapper[2574]: W0424 14:23:57.567511    2574 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Apr 24 14:23:57.571303 ip-10-0-136-132 kubenswrapper[2574]: W0424 14:23:57.567516    2574 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Apr 24 14:23:57.571303 ip-10-0-136-132 kubenswrapper[2574]: W0424 14:23:57.567521    2574 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Apr 24 14:23:57.571303 ip-10-0-136-132 kubenswrapper[2574]: W0424 14:23:57.567525    2574 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Apr 24 14:23:57.571303 ip-10-0-136-132 kubenswrapper[2574]: W0424 14:23:57.567530    2574 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Apr 24 14:23:57.571303 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:23:57.568232    2574 flags.go:64] FLAG: --address="0.0.0.0"
Apr 24 14:23:57.571303 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:23:57.568249    2574 flags.go:64] FLAG: --allowed-unsafe-sysctls="[]"
Apr 24 14:23:57.571303 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:23:57.568261    2574 flags.go:64] FLAG: --anonymous-auth="true"
Apr 24 14:23:57.571303 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:23:57.568269    2574 flags.go:64] FLAG: --application-metrics-count-limit="100"
Apr 24 14:23:57.571303 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:23:57.568276    2574 flags.go:64] FLAG: --authentication-token-webhook="false"
Apr 24 14:23:57.571303 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:23:57.568281    2574 flags.go:64] FLAG: --authentication-token-webhook-cache-ttl="2m0s"
Apr 24 14:23:57.571303 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:23:57.568288    2574 flags.go:64] FLAG: --authorization-mode="AlwaysAllow"
Apr 24 14:23:57.571303 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:23:57.568296    2574 flags.go:64] FLAG: --authorization-webhook-cache-authorized-ttl="5m0s"
Apr 24 14:23:57.571303 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:23:57.568301    2574 flags.go:64] FLAG: --authorization-webhook-cache-unauthorized-ttl="30s"
Apr 24 14:23:57.571823 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:23:57.568307    2574 flags.go:64] FLAG: --boot-id-file="/proc/sys/kernel/random/boot_id"
Apr 24 14:23:57.571823 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:23:57.568312    2574 flags.go:64] FLAG: --bootstrap-kubeconfig="/etc/kubernetes/kubeconfig"
Apr 24 14:23:57.571823 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:23:57.568318    2574 flags.go:64] FLAG: --cert-dir="/var/lib/kubelet/pki"
Apr 24 14:23:57.571823 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:23:57.568323    2574 flags.go:64] FLAG: --cgroup-driver="cgroupfs"
Apr 24 14:23:57.571823 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:23:57.568328    2574 flags.go:64] FLAG: --cgroup-root=""
Apr 24 14:23:57.571823 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:23:57.568353    2574 flags.go:64] FLAG: --cgroups-per-qos="true"
Apr 24 14:23:57.571823 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:23:57.568360    2574 flags.go:64] FLAG: --client-ca-file=""
Apr 24 14:23:57.571823 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:23:57.568364    2574 flags.go:64] FLAG: --cloud-config=""
Apr 24 14:23:57.571823 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:23:57.568369    2574 flags.go:64] FLAG: --cloud-provider="external"
Apr 24 14:23:57.571823 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:23:57.568374    2574 flags.go:64] FLAG: --cluster-dns="[]"
Apr 24 14:23:57.571823 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:23:57.568381    2574 flags.go:64] FLAG: --cluster-domain=""
Apr 24 14:23:57.571823 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:23:57.568386    2574 flags.go:64] FLAG: --config="/etc/kubernetes/kubelet.conf"
Apr 24 14:23:57.571823 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:23:57.568391    2574 flags.go:64] FLAG: --config-dir=""
Apr 24 14:23:57.571823 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:23:57.568396    2574 flags.go:64] FLAG: --container-hints="/etc/cadvisor/container_hints.json"
Apr 24 14:23:57.571823 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:23:57.568403    2574 flags.go:64] FLAG: --container-log-max-files="5"
Apr 24 14:23:57.571823 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:23:57.568410    2574 flags.go:64] FLAG: --container-log-max-size="10Mi"
Apr 24 14:23:57.571823 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:23:57.568418    2574 flags.go:64] FLAG: --container-runtime-endpoint="/var/run/crio/crio.sock"
Apr 24 14:23:57.571823 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:23:57.568423    2574 flags.go:64] FLAG: --containerd="/run/containerd/containerd.sock"
Apr 24 14:23:57.571823 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:23:57.568428    2574 flags.go:64] FLAG: --containerd-namespace="k8s.io"
Apr 24 14:23:57.571823 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:23:57.568433    2574 flags.go:64] FLAG: --contention-profiling="false"
Apr 24 14:23:57.571823 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:23:57.568439    2574 flags.go:64] FLAG: --cpu-cfs-quota="true"
Apr 24 14:23:57.571823 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:23:57.568444    2574 flags.go:64] FLAG: --cpu-cfs-quota-period="100ms"
Apr 24 14:23:57.571823 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:23:57.568449    2574 flags.go:64] FLAG: --cpu-manager-policy="none"
Apr 24 14:23:57.571823 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:23:57.568454    2574 flags.go:64] FLAG: --cpu-manager-policy-options=""
Apr 24 14:23:57.571823 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:23:57.568461    2574 flags.go:64] FLAG: --cpu-manager-reconcile-period="10s"
Apr 24 14:23:57.572443 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:23:57.568466    2574 flags.go:64] FLAG: --enable-controller-attach-detach="true"
Apr 24 14:23:57.572443 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:23:57.568470    2574 flags.go:64] FLAG: --enable-debugging-handlers="true"
Apr 24 14:23:57.572443 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:23:57.568475    2574 flags.go:64] FLAG: --enable-load-reader="false"
Apr 24 14:23:57.572443 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:23:57.568480    2574 flags.go:64] FLAG: --enable-server="true"
Apr 24 14:23:57.572443 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:23:57.568485    2574 flags.go:64] FLAG: --enforce-node-allocatable="[pods]"
Apr 24 14:23:57.572443 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:23:57.568492    2574 flags.go:64] FLAG: --event-burst="100"
Apr 24 14:23:57.572443 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:23:57.568498    2574 flags.go:64] FLAG: --event-qps="50"
Apr 24 14:23:57.572443 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:23:57.568502    2574 flags.go:64] FLAG: --event-storage-age-limit="default=0"
Apr 24 14:23:57.572443 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:23:57.568508    2574 flags.go:64] FLAG: --event-storage-event-limit="default=0"
Apr 24 14:23:57.572443 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:23:57.568513    2574 flags.go:64] FLAG: --eviction-hard=""
Apr 24 14:23:57.572443 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:23:57.568519    2574 flags.go:64] FLAG: --eviction-max-pod-grace-period="0"
Apr 24 14:23:57.572443 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:23:57.568524    2574 flags.go:64] FLAG: --eviction-minimum-reclaim=""
Apr 24 14:23:57.572443 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:23:57.568529    2574 flags.go:64] FLAG: --eviction-pressure-transition-period="5m0s"
Apr 24 14:23:57.572443 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:23:57.568535    2574 flags.go:64] FLAG: --eviction-soft=""
Apr 24 14:23:57.572443 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:23:57.568539    2574 flags.go:64] FLAG: --eviction-soft-grace-period=""
Apr 24 14:23:57.572443 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:23:57.568545    2574 flags.go:64] FLAG: --exit-on-lock-contention="false"
Apr 24 14:23:57.572443 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:23:57.568550    2574 flags.go:64] FLAG: --experimental-allocatable-ignore-eviction="false"
Apr 24 14:23:57.572443 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:23:57.568555    2574 flags.go:64] FLAG: --experimental-mounter-path=""
Apr 24 14:23:57.572443 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:23:57.568560    2574 flags.go:64] FLAG: --fail-cgroupv1="false"
Apr 24 14:23:57.572443 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:23:57.568565    2574 flags.go:64] FLAG: --fail-swap-on="true"
Apr 24 14:23:57.572443 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:23:57.568570    2574 flags.go:64] FLAG: --feature-gates=""
Apr 24 14:23:57.572443 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:23:57.568577    2574 flags.go:64] FLAG: --file-check-frequency="20s"
Apr 24 14:23:57.572443 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:23:57.568583    2574 flags.go:64] FLAG: --global-housekeeping-interval="1m0s"
Apr 24 14:23:57.572443 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:23:57.568588    2574 flags.go:64] FLAG: --hairpin-mode="promiscuous-bridge"
Apr 24 14:23:57.572443 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:23:57.568594    2574 flags.go:64] FLAG: --healthz-bind-address="127.0.0.1"
Apr 24 14:23:57.573054 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:23:57.568599    2574 flags.go:64] FLAG: --healthz-port="10248"
Apr 24 14:23:57.573054 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:23:57.568604    2574 flags.go:64] FLAG: --help="false"
Apr 24 14:23:57.573054 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:23:57.568609    2574 flags.go:64] FLAG: --hostname-override="ip-10-0-136-132.ec2.internal"
Apr 24 14:23:57.573054 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:23:57.568615    2574 flags.go:64] FLAG: --housekeeping-interval="10s"
Apr 24 14:23:57.573054 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:23:57.568620    2574 flags.go:64] FLAG: --http-check-frequency="20s"
Apr 24 14:23:57.573054 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:23:57.568625    2574 flags.go:64] FLAG: --image-credential-provider-bin-dir="/usr/libexec/kubelet-image-credential-provider-plugins"
Apr 24 14:23:57.573054 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:23:57.568631    2574 flags.go:64] FLAG: --image-credential-provider-config="/etc/kubernetes/credential-providers/ecr-credential-provider.yaml"
Apr 24 14:23:57.573054 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:23:57.568636    2574 flags.go:64] FLAG: --image-gc-high-threshold="85"
Apr 24 14:23:57.573054 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:23:57.568642    2574 flags.go:64] FLAG: --image-gc-low-threshold="80"
Apr 24 14:23:57.573054 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:23:57.568646    2574 flags.go:64] FLAG: --image-service-endpoint=""
Apr 24 14:23:57.573054 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:23:57.568651    2574 flags.go:64] FLAG: --kernel-memcg-notification="false"
Apr 24 14:23:57.573054 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:23:57.568656    2574 flags.go:64] FLAG: --kube-api-burst="100"
Apr 24 14:23:57.573054 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:23:57.568660    2574 flags.go:64] FLAG: --kube-api-content-type="application/vnd.kubernetes.protobuf"
Apr 24 14:23:57.573054 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:23:57.568666    2574 flags.go:64] FLAG: --kube-api-qps="50"
Apr 24 14:23:57.573054 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:23:57.568671    2574 flags.go:64] FLAG: --kube-reserved=""
Apr 24 14:23:57.573054 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:23:57.568677    2574 flags.go:64] FLAG: --kube-reserved-cgroup=""
Apr 24 14:23:57.573054 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:23:57.568682    2574 flags.go:64] FLAG: --kubeconfig="/var/lib/kubelet/kubeconfig"
Apr 24 14:23:57.573054 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:23:57.568687    2574 flags.go:64] FLAG: --kubelet-cgroups=""
Apr 24 14:23:57.573054 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:23:57.568692    2574 flags.go:64] FLAG: --local-storage-capacity-isolation="true"
Apr 24 14:23:57.573054 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:23:57.568697    2574 flags.go:64] FLAG: --lock-file=""
Apr 24 14:23:57.573054 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:23:57.568702    2574 flags.go:64] FLAG: --log-cadvisor-usage="false"
Apr 24 14:23:57.573054 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:23:57.568706    2574 flags.go:64] FLAG: --log-flush-frequency="5s"
Apr 24 14:23:57.573054 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:23:57.568712    2574 flags.go:64] FLAG: --log-json-info-buffer-size="0"
Apr 24 14:23:57.573054 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:23:57.568721    2574 flags.go:64] FLAG: --log-json-split-stream="false"
Apr 24 14:23:57.573679 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:23:57.568726    2574 flags.go:64] FLAG: --log-text-info-buffer-size="0"
Apr 24 14:23:57.573679 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:23:57.568731    2574 flags.go:64] FLAG: --log-text-split-stream="false"
Apr 24 14:23:57.573679 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:23:57.568736    2574 flags.go:64] FLAG: --logging-format="text"
Apr 24 14:23:57.573679 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:23:57.568741    2574 flags.go:64] FLAG: --machine-id-file="/etc/machine-id,/var/lib/dbus/machine-id"
Apr 24 14:23:57.573679 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:23:57.568747    2574 flags.go:64] FLAG: --make-iptables-util-chains="true"
Apr 24 14:23:57.573679 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:23:57.568753    2574 flags.go:64] FLAG: --manifest-url=""
Apr 24 14:23:57.573679 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:23:57.568758    2574 flags.go:64] FLAG: --manifest-url-header=""
Apr 24 14:23:57.573679 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:23:57.568765    2574 flags.go:64] FLAG: --max-housekeeping-interval="15s"
Apr 24 14:23:57.573679 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:23:57.568771    2574 flags.go:64] FLAG: --max-open-files="1000000"
Apr 24 14:23:57.573679 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:23:57.568777    2574 flags.go:64] FLAG: --max-pods="110"
Apr 24 14:23:57.573679 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:23:57.568782    2574 flags.go:64] FLAG: --maximum-dead-containers="-1"
Apr 24 14:23:57.573679 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:23:57.568788    2574 flags.go:64] FLAG: --maximum-dead-containers-per-container="1"
Apr 24 14:23:57.573679 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:23:57.568793    2574 flags.go:64] FLAG: --memory-manager-policy="None"
Apr 24 14:23:57.573679 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:23:57.568797    2574 flags.go:64] FLAG: --minimum-container-ttl-duration="6m0s"
Apr 24 14:23:57.573679 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:23:57.568802    2574 flags.go:64] FLAG: --minimum-image-ttl-duration="2m0s"
Apr 24 14:23:57.573679 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:23:57.568807    2574 flags.go:64] FLAG: --node-ip="0.0.0.0"
Apr 24 14:23:57.573679 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:23:57.568812    2574 flags.go:64] FLAG: --node-labels="node-role.kubernetes.io/worker=,node.openshift.io/os_id=rhel"
Apr 24 14:23:57.573679 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:23:57.568832    2574 flags.go:64] FLAG: --node-status-max-images="50"
Apr 24 14:23:57.573679 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:23:57.568837    2574 flags.go:64] FLAG: --node-status-update-frequency="10s"
Apr 24 14:23:57.573679 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:23:57.568842    2574 flags.go:64] FLAG: --oom-score-adj="-999"
Apr 24 14:23:57.573679 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:23:57.568847    2574 flags.go:64] FLAG: --pod-cidr=""
Apr 24 14:23:57.573679 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:23:57.568852    2574 flags.go:64] FLAG: --pod-infra-container-image="quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:c8cfe89231412ff3ee8cb6207fa0be33cad0f08e88c9c0f1e9f7e8c6f14d6715"
Apr 24 14:23:57.573679 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:23:57.568862    2574 flags.go:64] FLAG: --pod-manifest-path=""
Apr 24 14:23:57.574232 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:23:57.568867    2574 flags.go:64] FLAG: --pod-max-pids="-1"
Apr 24 14:23:57.574232 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:23:57.568872    2574 flags.go:64] FLAG: --pods-per-core="0"
Apr 24 14:23:57.574232 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:23:57.568877    2574 flags.go:64] FLAG: --port="10250"
Apr 24 14:23:57.574232 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:23:57.568882    2574 flags.go:64] FLAG: --protect-kernel-defaults="false"
Apr 24 14:23:57.574232 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:23:57.568886    2574 flags.go:64] FLAG: --provider-id="aws:///us-east-1a/i-018f91fc1851ea4eb"
Apr 24 14:23:57.574232 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:23:57.568892    2574 flags.go:64] FLAG: --qos-reserved=""
Apr 24 14:23:57.574232 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:23:57.568897    2574 flags.go:64] FLAG: --read-only-port="10255"
Apr 24 14:23:57.574232 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:23:57.568902    2574 flags.go:64] FLAG: --register-node="true"
Apr 24 14:23:57.574232 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:23:57.568907    2574 flags.go:64] FLAG: --register-schedulable="true"
Apr 24 14:23:57.574232 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:23:57.568912    2574 flags.go:64] FLAG: --register-with-taints=""
Apr 24 14:23:57.574232 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:23:57.568918    2574 flags.go:64] FLAG: --registry-burst="10"
Apr 24 14:23:57.574232 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:23:57.568923    2574 flags.go:64] FLAG: --registry-qps="5"
Apr 24 14:23:57.574232 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:23:57.568927    2574 flags.go:64] FLAG: --reserved-cpus=""
Apr 24 14:23:57.574232 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:23:57.568932    2574 flags.go:64] FLAG: --reserved-memory=""
Apr 24 14:23:57.574232 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:23:57.568938    2574 flags.go:64] FLAG: --resolv-conf="/etc/resolv.conf"
Apr 24 14:23:57.574232 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:23:57.568944    2574 flags.go:64] FLAG: --root-dir="/var/lib/kubelet"
Apr 24 14:23:57.574232 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:23:57.568950    2574 flags.go:64] FLAG: --rotate-certificates="false"
Apr 24 14:23:57.574232 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:23:57.568955    2574 flags.go:64] FLAG: --rotate-server-certificates="false"
Apr 24 14:23:57.574232 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:23:57.568960    2574 flags.go:64] FLAG: --runonce="false"
Apr 24 14:23:57.574232 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:23:57.568965    2574 flags.go:64] FLAG: --runtime-cgroups="/system.slice/crio.service"
Apr 24 14:23:57.574232 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:23:57.568970    2574 flags.go:64] FLAG: --runtime-request-timeout="2m0s"
Apr 24 14:23:57.574232 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:23:57.568975    2574 flags.go:64] FLAG: --seccomp-default="false"
Apr 24 14:23:57.574232 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:23:57.568980    2574 flags.go:64] FLAG: --serialize-image-pulls="true"
Apr 24 14:23:57.574232 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:23:57.568985    2574 flags.go:64] FLAG: --storage-driver-buffer-duration="1m0s"
Apr 24 14:23:57.574232 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:23:57.568991    2574 flags.go:64] FLAG: --storage-driver-db="cadvisor"
Apr 24 14:23:57.574232 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:23:57.568996    2574 flags.go:64] FLAG: --storage-driver-host="localhost:8086"
Apr 24 14:23:57.574881 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:23:57.569001    2574 flags.go:64] FLAG: --storage-driver-password="root"
Apr 24 14:23:57.574881 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:23:57.569006    2574 flags.go:64] FLAG: --storage-driver-secure="false"
Apr 24 14:23:57.574881 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:23:57.569011    2574 flags.go:64] FLAG: --storage-driver-table="stats"
Apr 24 14:23:57.574881 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:23:57.569016    2574 flags.go:64] FLAG: --storage-driver-user="root"
Apr 24 14:23:57.574881 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:23:57.569021    2574 flags.go:64] FLAG: --streaming-connection-idle-timeout="4h0m0s"
Apr 24 14:23:57.574881 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:23:57.569026    2574 flags.go:64] FLAG: --sync-frequency="1m0s"
Apr 24 14:23:57.574881 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:23:57.569031    2574 flags.go:64] FLAG: --system-cgroups=""
Apr 24 14:23:57.574881 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:23:57.569036    2574 flags.go:64] FLAG: --system-reserved="cpu=500m,ephemeral-storage=1Gi,memory=1Gi"
Apr 24 14:23:57.574881 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:23:57.569045    2574 flags.go:64] FLAG: --system-reserved-cgroup=""
Apr 24 14:23:57.574881 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:23:57.569050    2574 flags.go:64] FLAG: --tls-cert-file=""
Apr 24 14:23:57.574881 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:23:57.569054    2574 flags.go:64] FLAG: --tls-cipher-suites="[]"
Apr 24 14:23:57.574881 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:23:57.569061    2574 flags.go:64] FLAG: --tls-min-version=""
Apr 24 14:23:57.574881 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:23:57.569066    2574 flags.go:64] FLAG: --tls-private-key-file=""
Apr 24 14:23:57.574881 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:23:57.569074    2574 flags.go:64] FLAG: --topology-manager-policy="none"
Apr 24 14:23:57.574881 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:23:57.569079    2574 flags.go:64] FLAG: --topology-manager-policy-options=""
Apr 24 14:23:57.574881 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:23:57.569084    2574 flags.go:64] FLAG: --topology-manager-scope="container"
Apr 24 14:23:57.574881 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:23:57.569088    2574 flags.go:64] FLAG: --v="2"
Apr 24 14:23:57.574881 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:23:57.569095    2574 flags.go:64] FLAG: --version="false"
Apr 24 14:23:57.574881 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:23:57.569102    2574 flags.go:64] FLAG: --vmodule=""
Apr 24 14:23:57.574881 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:23:57.569110    2574 flags.go:64] FLAG: --volume-plugin-dir="/etc/kubernetes/kubelet-plugins/volume/exec"
Apr 24 14:23:57.574881 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:23:57.569115    2574 flags.go:64] FLAG: --volume-stats-agg-period="1m0s"
Apr 24 14:23:57.574881 ip-10-0-136-132 kubenswrapper[2574]: W0424 14:23:57.569265    2574 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Apr 24 14:23:57.574881 ip-10-0-136-132 kubenswrapper[2574]: W0424 14:23:57.569273    2574 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Apr 24 14:23:57.574881 ip-10-0-136-132 kubenswrapper[2574]: W0424 14:23:57.569278    2574 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Apr 24 14:23:57.575471 ip-10-0-136-132 kubenswrapper[2574]: W0424 14:23:57.569283    2574 feature_gate.go:328] unrecognized feature gate: DualReplica
Apr 24 14:23:57.575471 ip-10-0-136-132 kubenswrapper[2574]: W0424 14:23:57.569288    2574 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Apr 24 14:23:57.575471 ip-10-0-136-132 kubenswrapper[2574]: W0424 14:23:57.569293    2574 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Apr 24 14:23:57.575471 ip-10-0-136-132 kubenswrapper[2574]: W0424 14:23:57.569298    2574 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Apr 24 14:23:57.575471 ip-10-0-136-132 kubenswrapper[2574]: W0424 14:23:57.569303    2574 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Apr 24 14:23:57.575471 ip-10-0-136-132 kubenswrapper[2574]: W0424 14:23:57.569307    2574 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Apr 24 14:23:57.575471 ip-10-0-136-132 kubenswrapper[2574]: W0424 14:23:57.569312    2574 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Apr 24 14:23:57.575471 ip-10-0-136-132 kubenswrapper[2574]: W0424 14:23:57.569316    2574 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Apr 24 14:23:57.575471 ip-10-0-136-132 kubenswrapper[2574]: W0424 14:23:57.569320    2574 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Apr 24 14:23:57.575471 ip-10-0-136-132 kubenswrapper[2574]: W0424 14:23:57.569324    2574 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Apr 24 14:23:57.575471 ip-10-0-136-132 kubenswrapper[2574]: W0424 14:23:57.569329    2574 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Apr 24 14:23:57.575471 ip-10-0-136-132 kubenswrapper[2574]: W0424 14:23:57.569348    2574 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Apr 24 14:23:57.575471 ip-10-0-136-132 kubenswrapper[2574]: W0424 14:23:57.569353    2574 feature_gate.go:328] unrecognized feature gate: SignatureStores
Apr 24 14:23:57.575471 ip-10-0-136-132 kubenswrapper[2574]: W0424 14:23:57.569357    2574 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Apr 24 14:23:57.575471 ip-10-0-136-132 kubenswrapper[2574]: W0424 14:23:57.569361    2574 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Apr 24 14:23:57.575471 ip-10-0-136-132 kubenswrapper[2574]: W0424 14:23:57.569366    2574 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Apr 24 14:23:57.575471 ip-10-0-136-132 kubenswrapper[2574]: W0424 14:23:57.569370    2574 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Apr 24 14:23:57.575471 ip-10-0-136-132 kubenswrapper[2574]: W0424 14:23:57.569374    2574 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Apr 24 14:23:57.575471 ip-10-0-136-132 kubenswrapper[2574]: W0424 14:23:57.569378    2574 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Apr 24 14:23:57.575471 ip-10-0-136-132 kubenswrapper[2574]: W0424 14:23:57.569382    2574 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Apr 24 14:23:57.576203 ip-10-0-136-132 kubenswrapper[2574]: W0424 14:23:57.569387    2574 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Apr 24 14:23:57.576203 ip-10-0-136-132 kubenswrapper[2574]: W0424 14:23:57.569394    2574 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Apr 24 14:23:57.576203 ip-10-0-136-132 kubenswrapper[2574]: W0424 14:23:57.569399    2574 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Apr 24 14:23:57.576203 ip-10-0-136-132 kubenswrapper[2574]: W0424 14:23:57.569405    2574 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Apr 24 14:23:57.576203 ip-10-0-136-132 kubenswrapper[2574]: W0424 14:23:57.569412    2574 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Apr 24 14:23:57.576203 ip-10-0-136-132 kubenswrapper[2574]: W0424 14:23:57.569416    2574 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Apr 24 14:23:57.576203 ip-10-0-136-132 kubenswrapper[2574]: W0424 14:23:57.569421    2574 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Apr 24 14:23:57.576203 ip-10-0-136-132 kubenswrapper[2574]: W0424 14:23:57.569425    2574 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Apr 24 14:23:57.576203 ip-10-0-136-132 kubenswrapper[2574]: W0424 14:23:57.569429    2574 feature_gate.go:328] unrecognized feature gate: InsightsConfigAPI
Apr 24 14:23:57.576203 ip-10-0-136-132 kubenswrapper[2574]: W0424 14:23:57.569475    2574 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Apr 24 14:23:57.576203 ip-10-0-136-132 kubenswrapper[2574]: W0424 14:23:57.569481    2574 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Apr 24 14:23:57.576203 ip-10-0-136-132 kubenswrapper[2574]: W0424 14:23:57.569485    2574 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Apr 24 14:23:57.576203 ip-10-0-136-132 kubenswrapper[2574]: W0424 14:23:57.569488    2574 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Apr 24 14:23:57.576203 ip-10-0-136-132 kubenswrapper[2574]: W0424 14:23:57.569493    2574 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Apr 24 14:23:57.576203 ip-10-0-136-132 kubenswrapper[2574]: W0424 14:23:57.569496    2574 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Apr 24 14:23:57.576203 ip-10-0-136-132 kubenswrapper[2574]: W0424 14:23:57.569499    2574 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Apr 24 14:23:57.576203 ip-10-0-136-132 kubenswrapper[2574]: W0424 14:23:57.569502    2574 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Apr 24 14:23:57.576203 ip-10-0-136-132 kubenswrapper[2574]: W0424 14:23:57.569505    2574 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Apr 24 14:23:57.576203 ip-10-0-136-132 kubenswrapper[2574]: W0424 14:23:57.569508    2574 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Apr 24 14:23:57.576914 ip-10-0-136-132 kubenswrapper[2574]: W0424 14:23:57.569511    2574 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Apr 24 14:23:57.576914 ip-10-0-136-132 kubenswrapper[2574]: W0424 14:23:57.569514    2574 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Apr 24 14:23:57.576914 ip-10-0-136-132 kubenswrapper[2574]: W0424 14:23:57.569517    2574 feature_gate.go:328] unrecognized feature gate: OVNObservability
Apr 24 14:23:57.576914 ip-10-0-136-132 kubenswrapper[2574]: W0424 14:23:57.569521    2574 feature_gate.go:328] unrecognized feature gate: Example2
Apr 24 14:23:57.576914 ip-10-0-136-132 kubenswrapper[2574]: W0424 14:23:57.569524    2574 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Apr 24 14:23:57.576914 ip-10-0-136-132 kubenswrapper[2574]: W0424 14:23:57.569527    2574 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Apr 24 14:23:57.576914 ip-10-0-136-132 kubenswrapper[2574]: W0424 14:23:57.569529    2574 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Apr 24 14:23:57.576914 ip-10-0-136-132 kubenswrapper[2574]: W0424 14:23:57.569532    2574 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Apr 24 14:23:57.576914 ip-10-0-136-132 kubenswrapper[2574]: W0424 14:23:57.569535    2574 feature_gate.go:328] unrecognized feature gate: IngressControllerLBSubnetsAWS
Apr 24 14:23:57.576914 ip-10-0-136-132 kubenswrapper[2574]: W0424 14:23:57.569539    2574 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Apr 24 14:23:57.576914 ip-10-0-136-132 kubenswrapper[2574]: W0424 14:23:57.569542    2574 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Apr 24 14:23:57.576914 ip-10-0-136-132 kubenswrapper[2574]: W0424 14:23:57.569545    2574 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Apr 24 14:23:57.576914 ip-10-0-136-132 kubenswrapper[2574]: W0424 14:23:57.569548    2574 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Apr 24 14:23:57.576914 ip-10-0-136-132 kubenswrapper[2574]: W0424 14:23:57.569551    2574 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Apr 24 14:23:57.576914 ip-10-0-136-132 kubenswrapper[2574]: W0424 14:23:57.569553    2574 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Apr 24 14:23:57.576914 ip-10-0-136-132 kubenswrapper[2574]: W0424 14:23:57.569556    2574 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Apr 24 14:23:57.576914 ip-10-0-136-132 kubenswrapper[2574]: W0424 14:23:57.569559    2574 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Apr 24 14:23:57.576914 ip-10-0-136-132 kubenswrapper[2574]: W0424 14:23:57.569562    2574 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Apr 24 14:23:57.576914 ip-10-0-136-132 kubenswrapper[2574]: W0424 14:23:57.569565    2574 feature_gate.go:328] unrecognized feature gate: SetEIPForNLBIngressController
Apr 24 14:23:57.576914 ip-10-0-136-132 kubenswrapper[2574]: W0424 14:23:57.569568    2574 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Apr 24 14:23:57.577465 ip-10-0-136-132 kubenswrapper[2574]: W0424 14:23:57.569571    2574 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterOperations
Apr 24 14:23:57.577465 ip-10-0-136-132 kubenswrapper[2574]: W0424 14:23:57.569574    2574 feature_gate.go:328] unrecognized feature gate: Example
Apr 24 14:23:57.577465 ip-10-0-136-132 kubenswrapper[2574]: W0424 14:23:57.569578    2574 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Apr 24 14:23:57.577465 ip-10-0-136-132 kubenswrapper[2574]: W0424 14:23:57.569582    2574 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Apr 24 14:23:57.577465 ip-10-0-136-132 kubenswrapper[2574]: W0424 14:23:57.569585    2574 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Apr 24 14:23:57.577465 ip-10-0-136-132 kubenswrapper[2574]: W0424 14:23:57.569588    2574 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Apr 24 14:23:57.577465 ip-10-0-136-132 kubenswrapper[2574]: W0424 14:23:57.569590    2574 feature_gate.go:328] unrecognized feature gate: PinnedImages
Apr 24 14:23:57.577465 ip-10-0-136-132 kubenswrapper[2574]: W0424 14:23:57.569595    2574 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Apr 24 14:23:57.577465 ip-10-0-136-132 kubenswrapper[2574]: W0424 14:23:57.569599    2574 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Apr 24 14:23:57.577465 ip-10-0-136-132 kubenswrapper[2574]: W0424 14:23:57.569602    2574 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Apr 24 14:23:57.577465 ip-10-0-136-132 kubenswrapper[2574]: W0424 14:23:57.569605    2574 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Apr 24 14:23:57.577465 ip-10-0-136-132 kubenswrapper[2574]: W0424 14:23:57.569608    2574 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Apr 24 14:23:57.577465 ip-10-0-136-132 kubenswrapper[2574]: W0424 14:23:57.569611    2574 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Apr 24 14:23:57.577465 ip-10-0-136-132 kubenswrapper[2574]: W0424 14:23:57.569614    2574 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Apr 24 14:23:57.577465 ip-10-0-136-132 kubenswrapper[2574]: W0424 14:23:57.569617    2574 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Apr 24 14:23:57.577465 ip-10-0-136-132 kubenswrapper[2574]: W0424 14:23:57.569619    2574 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Apr 24 14:23:57.577465 ip-10-0-136-132 kubenswrapper[2574]: W0424 14:23:57.569622    2574 feature_gate.go:328] unrecognized feature gate: NewOLM
Apr 24 14:23:57.577465 ip-10-0-136-132 kubenswrapper[2574]: W0424 14:23:57.569625    2574 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Apr 24 14:23:57.577465 ip-10-0-136-132 kubenswrapper[2574]: W0424 14:23:57.569628    2574 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Apr 24 14:23:57.577946 ip-10-0-136-132 kubenswrapper[2574]: W0424 14:23:57.569630    2574 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Apr 24 14:23:57.577946 ip-10-0-136-132 kubenswrapper[2574]: W0424 14:23:57.569633    2574 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Apr 24 14:23:57.577946 ip-10-0-136-132 kubenswrapper[2574]: W0424 14:23:57.569636    2574 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Apr 24 14:23:57.577946 ip-10-0-136-132 kubenswrapper[2574]: W0424 14:23:57.569639    2574 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Apr 24 14:23:57.577946 ip-10-0-136-132 kubenswrapper[2574]: W0424 14:23:57.569641    2574 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Apr 24 14:23:57.577946 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:23:57.570319    2574 feature_gate.go:384] feature gates: {map[DynamicResourceAllocation:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutatingAdmissionPolicy:false NodeSwap:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:false]}
Apr 24 14:23:57.579702 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:23:57.579564    2574 server.go:530] "Kubelet version" kubeletVersion="v1.33.9"
Apr 24 14:23:57.579736 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:23:57.579704    2574 server.go:532] "Golang settings" GOGC="" GOMAXPROCS="" GOTRACEBACK=""
Apr 24 14:23:57.579762 ip-10-0-136-132 kubenswrapper[2574]: W0424 14:23:57.579757    2574 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Apr 24 14:23:57.579795 ip-10-0-136-132 kubenswrapper[2574]: W0424 14:23:57.579764    2574 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Apr 24 14:23:57.579795 ip-10-0-136-132 kubenswrapper[2574]: W0424 14:23:57.579767    2574 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Apr 24 14:23:57.579795 ip-10-0-136-132 kubenswrapper[2574]: W0424 14:23:57.579771    2574 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Apr 24 14:23:57.579795 ip-10-0-136-132 kubenswrapper[2574]: W0424 14:23:57.579774    2574 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Apr 24 14:23:57.579795 ip-10-0-136-132 kubenswrapper[2574]: W0424 14:23:57.579777    2574 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Apr 24 14:23:57.579795 ip-10-0-136-132 kubenswrapper[2574]: W0424 14:23:57.579780    2574 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Apr 24 14:23:57.579795 ip-10-0-136-132 kubenswrapper[2574]: W0424 14:23:57.579782    2574 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Apr 24 14:23:57.579795 ip-10-0-136-132 kubenswrapper[2574]: W0424 14:23:57.579785    2574 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Apr 24 14:23:57.579795 ip-10-0-136-132 kubenswrapper[2574]: W0424 14:23:57.579788    2574 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Apr 24 14:23:57.579795 ip-10-0-136-132 kubenswrapper[2574]: W0424 14:23:57.579791    2574 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Apr 24 14:23:57.579795 ip-10-0-136-132 kubenswrapper[2574]: W0424 14:23:57.579794    2574 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Apr 24 14:23:57.579795 ip-10-0-136-132 kubenswrapper[2574]: W0424 14:23:57.579797    2574 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Apr 24 14:23:57.579795 ip-10-0-136-132 kubenswrapper[2574]: W0424 14:23:57.579800    2574 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Apr 24 14:23:57.579795 ip-10-0-136-132 kubenswrapper[2574]: W0424 14:23:57.579802    2574 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Apr 24 14:23:57.580145 ip-10-0-136-132 kubenswrapper[2574]: W0424 14:23:57.579806    2574 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Apr 24 14:23:57.580145 ip-10-0-136-132 kubenswrapper[2574]: W0424 14:23:57.579809    2574 feature_gate.go:328] unrecognized feature gate: PinnedImages
Apr 24 14:23:57.580145 ip-10-0-136-132 kubenswrapper[2574]: W0424 14:23:57.579812    2574 feature_gate.go:328] unrecognized feature gate: IngressControllerLBSubnetsAWS
Apr 24 14:23:57.580145 ip-10-0-136-132 kubenswrapper[2574]: W0424 14:23:57.579815    2574 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Apr 24 14:23:57.580145 ip-10-0-136-132 kubenswrapper[2574]: W0424 14:23:57.579817    2574 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Apr 24 14:23:57.580145 ip-10-0-136-132 kubenswrapper[2574]: W0424 14:23:57.579820    2574 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Apr 24 14:23:57.580145 ip-10-0-136-132 kubenswrapper[2574]: W0424 14:23:57.579823    2574 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Apr 24 14:23:57.580145 ip-10-0-136-132 kubenswrapper[2574]: W0424 14:23:57.579826    2574 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Apr 24 14:23:57.580145 ip-10-0-136-132 kubenswrapper[2574]: W0424 14:23:57.579829    2574 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Apr 24 14:23:57.580145 ip-10-0-136-132 kubenswrapper[2574]: W0424 14:23:57.579831    2574 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Apr 24 14:23:57.580145 ip-10-0-136-132 kubenswrapper[2574]: W0424 14:23:57.579834    2574 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Apr 24 14:23:57.580145 ip-10-0-136-132 kubenswrapper[2574]: W0424 14:23:57.579837    2574 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Apr 24 14:23:57.580145 ip-10-0-136-132 kubenswrapper[2574]: W0424 14:23:57.579840    2574 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Apr 24 14:23:57.580145 ip-10-0-136-132 kubenswrapper[2574]: W0424 14:23:57.579842    2574 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Apr 24 14:23:57.580145 ip-10-0-136-132 kubenswrapper[2574]: W0424 14:23:57.579845    2574 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Apr 24 14:23:57.580145 ip-10-0-136-132 kubenswrapper[2574]: W0424 14:23:57.579848    2574 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Apr 24 14:23:57.580145 ip-10-0-136-132 kubenswrapper[2574]: W0424 14:23:57.579850    2574 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Apr 24 14:23:57.580145 ip-10-0-136-132 kubenswrapper[2574]: W0424 14:23:57.579853    2574 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Apr 24 14:23:57.580145 ip-10-0-136-132 kubenswrapper[2574]: W0424 14:23:57.579856    2574 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Apr 24 14:23:57.580145 ip-10-0-136-132 kubenswrapper[2574]: W0424 14:23:57.579858    2574 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Apr 24 14:23:57.580666 ip-10-0-136-132 kubenswrapper[2574]: W0424 14:23:57.579861    2574 feature_gate.go:328] unrecognized feature gate: OVNObservability
Apr 24 14:23:57.580666 ip-10-0-136-132 kubenswrapper[2574]: W0424 14:23:57.579863    2574 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Apr 24 14:23:57.580666 ip-10-0-136-132 kubenswrapper[2574]: W0424 14:23:57.579867    2574 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Apr 24 14:23:57.580666 ip-10-0-136-132 kubenswrapper[2574]: W0424 14:23:57.579869    2574 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Apr 24 14:23:57.580666 ip-10-0-136-132 kubenswrapper[2574]: W0424 14:23:57.579872    2574 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterOperations
Apr 24 14:23:57.580666 ip-10-0-136-132 kubenswrapper[2574]: W0424 14:23:57.579874    2574 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Apr 24 14:23:57.580666 ip-10-0-136-132 kubenswrapper[2574]: W0424 14:23:57.579877    2574 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Apr 24 14:23:57.580666 ip-10-0-136-132 kubenswrapper[2574]: W0424 14:23:57.579880    2574 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Apr 24 14:23:57.580666 ip-10-0-136-132 kubenswrapper[2574]: W0424 14:23:57.579883    2574 feature_gate.go:328] unrecognized feature gate: DualReplica
Apr 24 14:23:57.580666 ip-10-0-136-132 kubenswrapper[2574]: W0424 14:23:57.579885    2574 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Apr 24 14:23:57.580666 ip-10-0-136-132 kubenswrapper[2574]: W0424 14:23:57.579888    2574 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Apr 24 14:23:57.580666 ip-10-0-136-132 kubenswrapper[2574]: W0424 14:23:57.579890    2574 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Apr 24 14:23:57.580666 ip-10-0-136-132 kubenswrapper[2574]: W0424 14:23:57.579893    2574 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Apr 24 14:23:57.580666 ip-10-0-136-132 kubenswrapper[2574]: W0424 14:23:57.579897    2574 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Apr 24 14:23:57.580666 ip-10-0-136-132 kubenswrapper[2574]: W0424 14:23:57.579900    2574 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Apr 24 14:23:57.580666 ip-10-0-136-132 kubenswrapper[2574]: W0424 14:23:57.579903    2574 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Apr 24 14:23:57.580666 ip-10-0-136-132 kubenswrapper[2574]: W0424 14:23:57.579905    2574 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Apr 24 14:23:57.580666 ip-10-0-136-132 kubenswrapper[2574]: W0424 14:23:57.579909    2574 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Apr 24 14:23:57.580666 ip-10-0-136-132 kubenswrapper[2574]: W0424 14:23:57.579914    2574 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Apr 24 14:23:57.581131 ip-10-0-136-132 kubenswrapper[2574]: W0424 14:23:57.579917    2574 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Apr 24 14:23:57.581131 ip-10-0-136-132 kubenswrapper[2574]: W0424 14:23:57.579920    2574 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Apr 24 14:23:57.581131 ip-10-0-136-132 kubenswrapper[2574]: W0424 14:23:57.579922    2574 feature_gate.go:328] unrecognized feature gate: SetEIPForNLBIngressController
Apr 24 14:23:57.581131 ip-10-0-136-132 kubenswrapper[2574]: W0424 14:23:57.579925    2574 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Apr 24 14:23:57.581131 ip-10-0-136-132 kubenswrapper[2574]: W0424 14:23:57.579928    2574 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Apr 24 14:23:57.581131 ip-10-0-136-132 kubenswrapper[2574]: W0424 14:23:57.579931    2574 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Apr 24 14:23:57.581131 ip-10-0-136-132 kubenswrapper[2574]: W0424 14:23:57.579934    2574 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Apr 24 14:23:57.581131 ip-10-0-136-132 kubenswrapper[2574]: W0424 14:23:57.579936    2574 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Apr 24 14:23:57.581131 ip-10-0-136-132 kubenswrapper[2574]: W0424 14:23:57.579939    2574 feature_gate.go:328] unrecognized feature gate: NewOLM
Apr 24 14:23:57.581131 ip-10-0-136-132 kubenswrapper[2574]: W0424 14:23:57.579941    2574 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Apr 24 14:23:57.581131 ip-10-0-136-132 kubenswrapper[2574]: W0424 14:23:57.579944    2574 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Apr 24 14:23:57.581131 ip-10-0-136-132 kubenswrapper[2574]: W0424 14:23:57.579946    2574 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Apr 24 14:23:57.581131 ip-10-0-136-132 kubenswrapper[2574]: W0424 14:23:57.579949    2574 feature_gate.go:328] unrecognized feature gate: InsightsConfigAPI
Apr 24 14:23:57.581131 ip-10-0-136-132 kubenswrapper[2574]: W0424 14:23:57.579952    2574 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Apr 24 14:23:57.581131 ip-10-0-136-132 kubenswrapper[2574]: W0424 14:23:57.579954    2574 feature_gate.go:328] unrecognized feature gate: SignatureStores
Apr 24 14:23:57.581131 ip-10-0-136-132 kubenswrapper[2574]: W0424 14:23:57.579957    2574 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Apr 24 14:23:57.581131 ip-10-0-136-132 kubenswrapper[2574]: W0424 14:23:57.579960    2574 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Apr 24 14:23:57.581131 ip-10-0-136-132 kubenswrapper[2574]: W0424 14:23:57.579962    2574 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Apr 24 14:23:57.581131 ip-10-0-136-132 kubenswrapper[2574]: W0424 14:23:57.579967    2574 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Apr 24 14:23:57.581131 ip-10-0-136-132 kubenswrapper[2574]: W0424 14:23:57.579971    2574 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Apr 24 14:23:57.581639 ip-10-0-136-132 kubenswrapper[2574]: W0424 14:23:57.579974    2574 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Apr 24 14:23:57.581639 ip-10-0-136-132 kubenswrapper[2574]: W0424 14:23:57.579977    2574 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Apr 24 14:23:57.581639 ip-10-0-136-132 kubenswrapper[2574]: W0424 14:23:57.579981    2574 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Apr 24 14:23:57.581639 ip-10-0-136-132 kubenswrapper[2574]: W0424 14:23:57.579983    2574 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Apr 24 14:23:57.581639 ip-10-0-136-132 kubenswrapper[2574]: W0424 14:23:57.579986    2574 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Apr 24 14:23:57.581639 ip-10-0-136-132 kubenswrapper[2574]: W0424 14:23:57.579989    2574 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Apr 24 14:23:57.581639 ip-10-0-136-132 kubenswrapper[2574]: W0424 14:23:57.579992    2574 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Apr 24 14:23:57.581639 ip-10-0-136-132 kubenswrapper[2574]: W0424 14:23:57.579995    2574 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Apr 24 14:23:57.581639 ip-10-0-136-132 kubenswrapper[2574]: W0424 14:23:57.579998    2574 feature_gate.go:328] unrecognized feature gate: Example2
Apr 24 14:23:57.581639 ip-10-0-136-132 kubenswrapper[2574]: W0424 14:23:57.580001    2574 feature_gate.go:328] unrecognized feature gate: Example
Apr 24 14:23:57.581639 ip-10-0-136-132 kubenswrapper[2574]: W0424 14:23:57.580003    2574 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Apr 24 14:23:57.581639 ip-10-0-136-132 kubenswrapper[2574]: W0424 14:23:57.580006    2574 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Apr 24 14:23:57.581639 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:23:57.580011    2574 feature_gate.go:384] feature gates: {map[DynamicResourceAllocation:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutatingAdmissionPolicy:false NodeSwap:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:false]}
Apr 24 14:23:57.581639 ip-10-0-136-132 kubenswrapper[2574]: W0424 14:23:57.580116    2574 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Apr 24 14:23:57.581639 ip-10-0-136-132 kubenswrapper[2574]: W0424 14:23:57.580121    2574 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Apr 24 14:23:57.581639 ip-10-0-136-132 kubenswrapper[2574]: W0424 14:23:57.580125    2574 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Apr 24 14:23:57.582094 ip-10-0-136-132 kubenswrapper[2574]: W0424 14:23:57.580128    2574 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Apr 24 14:23:57.582094 ip-10-0-136-132 kubenswrapper[2574]: W0424 14:23:57.580131    2574 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Apr 24 14:23:57.582094 ip-10-0-136-132 kubenswrapper[2574]: W0424 14:23:57.580134    2574 feature_gate.go:328] unrecognized feature gate: SignatureStores
Apr 24 14:23:57.582094 ip-10-0-136-132 kubenswrapper[2574]: W0424 14:23:57.580137    2574 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Apr 24 14:23:57.582094 ip-10-0-136-132 kubenswrapper[2574]: W0424 14:23:57.580140    2574 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Apr 24 14:23:57.582094 ip-10-0-136-132 kubenswrapper[2574]: W0424 14:23:57.580142    2574 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Apr 24 14:23:57.582094 ip-10-0-136-132 kubenswrapper[2574]: W0424 14:23:57.580145    2574 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Apr 24 14:23:57.582094 ip-10-0-136-132 kubenswrapper[2574]: W0424 14:23:57.580148    2574 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Apr 24 14:23:57.582094 ip-10-0-136-132 kubenswrapper[2574]: W0424 14:23:57.580151    2574 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Apr 24 14:23:57.582094 ip-10-0-136-132 kubenswrapper[2574]: W0424 14:23:57.580154    2574 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Apr 24 14:23:57.582094 ip-10-0-136-132 kubenswrapper[2574]: W0424 14:23:57.580157    2574 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Apr 24 14:23:57.582094 ip-10-0-136-132 kubenswrapper[2574]: W0424 14:23:57.580160    2574 feature_gate.go:328] unrecognized feature gate: NewOLM
Apr 24 14:23:57.582094 ip-10-0-136-132 kubenswrapper[2574]: W0424 14:23:57.580162    2574 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Apr 24 14:23:57.582094 ip-10-0-136-132 kubenswrapper[2574]: W0424 14:23:57.580165    2574 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Apr 24 14:23:57.582094 ip-10-0-136-132 kubenswrapper[2574]: W0424 14:23:57.580168    2574 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Apr 24 14:23:57.582094 ip-10-0-136-132 kubenswrapper[2574]: W0424 14:23:57.580170    2574 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Apr 24 14:23:57.582094 ip-10-0-136-132 kubenswrapper[2574]: W0424 14:23:57.580173    2574 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Apr 24 14:23:57.582094 ip-10-0-136-132 kubenswrapper[2574]: W0424 14:23:57.580184    2574 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Apr 24 14:23:57.582094 ip-10-0-136-132 kubenswrapper[2574]: W0424 14:23:57.580187    2574 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Apr 24 14:23:57.582094 ip-10-0-136-132 kubenswrapper[2574]: W0424 14:23:57.580191    2574 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Apr 24 14:23:57.582607 ip-10-0-136-132 kubenswrapper[2574]: W0424 14:23:57.580194    2574 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Apr 24 14:23:57.582607 ip-10-0-136-132 kubenswrapper[2574]: W0424 14:23:57.580197    2574 feature_gate.go:328] unrecognized feature gate: InsightsConfigAPI
Apr 24 14:23:57.582607 ip-10-0-136-132 kubenswrapper[2574]: W0424 14:23:57.580200    2574 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Apr 24 14:23:57.582607 ip-10-0-136-132 kubenswrapper[2574]: W0424 14:23:57.580203    2574 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Apr 24 14:23:57.582607 ip-10-0-136-132 kubenswrapper[2574]: W0424 14:23:57.580206    2574 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Apr 24 14:23:57.582607 ip-10-0-136-132 kubenswrapper[2574]: W0424 14:23:57.580209    2574 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Apr 24 14:23:57.582607 ip-10-0-136-132 kubenswrapper[2574]: W0424 14:23:57.580212    2574 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Apr 24 14:23:57.582607 ip-10-0-136-132 kubenswrapper[2574]: W0424 14:23:57.580215    2574 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Apr 24 14:23:57.582607 ip-10-0-136-132 kubenswrapper[2574]: W0424 14:23:57.580217    2574 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Apr 24 14:23:57.582607 ip-10-0-136-132 kubenswrapper[2574]: W0424 14:23:57.580220    2574 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Apr 24 14:23:57.582607 ip-10-0-136-132 kubenswrapper[2574]: W0424 14:23:57.580223    2574 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Apr 24 14:23:57.582607 ip-10-0-136-132 kubenswrapper[2574]: W0424 14:23:57.580225    2574 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Apr 24 14:23:57.582607 ip-10-0-136-132 kubenswrapper[2574]: W0424 14:23:57.580228    2574 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Apr 24 14:23:57.582607 ip-10-0-136-132 kubenswrapper[2574]: W0424 14:23:57.580231    2574 feature_gate.go:328] unrecognized feature gate: DualReplica
Apr 24 14:23:57.582607 ip-10-0-136-132 kubenswrapper[2574]: W0424 14:23:57.580234    2574 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Apr 24 14:23:57.582607 ip-10-0-136-132 kubenswrapper[2574]: W0424 14:23:57.580236    2574 feature_gate.go:328] unrecognized feature gate: OVNObservability
Apr 24 14:23:57.582607 ip-10-0-136-132 kubenswrapper[2574]: W0424 14:23:57.580239    2574 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Apr 24 14:23:57.582607 ip-10-0-136-132 kubenswrapper[2574]: W0424 14:23:57.580242    2574 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Apr 24 14:23:57.582607 ip-10-0-136-132 kubenswrapper[2574]: W0424 14:23:57.580244    2574 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Apr 24 14:23:57.582607 ip-10-0-136-132 kubenswrapper[2574]: W0424 14:23:57.580247    2574 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Apr 24 14:23:57.583078 ip-10-0-136-132 kubenswrapper[2574]: W0424 14:23:57.580250    2574 feature_gate.go:328] unrecognized feature gate: IngressControllerLBSubnetsAWS
Apr 24 14:23:57.583078 ip-10-0-136-132 kubenswrapper[2574]: W0424 14:23:57.580252    2574 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Apr 24 14:23:57.583078 ip-10-0-136-132 kubenswrapper[2574]: W0424 14:23:57.580255    2574 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Apr 24 14:23:57.583078 ip-10-0-136-132 kubenswrapper[2574]: W0424 14:23:57.580257    2574 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Apr 24 14:23:57.583078 ip-10-0-136-132 kubenswrapper[2574]: W0424 14:23:57.580260    2574 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Apr 24 14:23:57.583078 ip-10-0-136-132 kubenswrapper[2574]: W0424 14:23:57.580262    2574 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Apr 24 14:23:57.583078 ip-10-0-136-132 kubenswrapper[2574]: W0424 14:23:57.580265    2574 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Apr 24 14:23:57.583078 ip-10-0-136-132 kubenswrapper[2574]: W0424 14:23:57.580268    2574 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Apr 24 14:23:57.583078 ip-10-0-136-132 kubenswrapper[2574]: W0424 14:23:57.580270    2574 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Apr 24 14:23:57.583078 ip-10-0-136-132 kubenswrapper[2574]: W0424 14:23:57.580273    2574 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Apr 24 14:23:57.583078 ip-10-0-136-132 kubenswrapper[2574]: W0424 14:23:57.580277    2574 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Apr 24 14:23:57.583078 ip-10-0-136-132 kubenswrapper[2574]: W0424 14:23:57.580279    2574 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Apr 24 14:23:57.583078 ip-10-0-136-132 kubenswrapper[2574]: W0424 14:23:57.580283    2574 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Apr 24 14:23:57.583078 ip-10-0-136-132 kubenswrapper[2574]: W0424 14:23:57.580286    2574 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Apr 24 14:23:57.583078 ip-10-0-136-132 kubenswrapper[2574]: W0424 14:23:57.580289    2574 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Apr 24 14:23:57.583078 ip-10-0-136-132 kubenswrapper[2574]: W0424 14:23:57.580292    2574 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Apr 24 14:23:57.583078 ip-10-0-136-132 kubenswrapper[2574]: W0424 14:23:57.580295    2574 feature_gate.go:328] unrecognized feature gate: Example2
Apr 24 14:23:57.583078 ip-10-0-136-132 kubenswrapper[2574]: W0424 14:23:57.580299    2574 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Apr 24 14:23:57.583078 ip-10-0-136-132 kubenswrapper[2574]: W0424 14:23:57.580302    2574 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Apr 24 14:23:57.583663 ip-10-0-136-132 kubenswrapper[2574]: W0424 14:23:57.580305    2574 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Apr 24 14:23:57.583663 ip-10-0-136-132 kubenswrapper[2574]: W0424 14:23:57.580308    2574 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Apr 24 14:23:57.583663 ip-10-0-136-132 kubenswrapper[2574]: W0424 14:23:57.580311    2574 feature_gate.go:328] unrecognized feature gate: Example
Apr 24 14:23:57.583663 ip-10-0-136-132 kubenswrapper[2574]: W0424 14:23:57.580314    2574 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Apr 24 14:23:57.583663 ip-10-0-136-132 kubenswrapper[2574]: W0424 14:23:57.580316    2574 feature_gate.go:328] unrecognized feature gate: PinnedImages
Apr 24 14:23:57.583663 ip-10-0-136-132 kubenswrapper[2574]: W0424 14:23:57.580319    2574 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Apr 24 14:23:57.583663 ip-10-0-136-132 kubenswrapper[2574]: W0424 14:23:57.580322    2574 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Apr 24 14:23:57.583663 ip-10-0-136-132 kubenswrapper[2574]: W0424 14:23:57.580324    2574 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Apr 24 14:23:57.583663 ip-10-0-136-132 kubenswrapper[2574]: W0424 14:23:57.580327    2574 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Apr 24 14:23:57.583663 ip-10-0-136-132 kubenswrapper[2574]: W0424 14:23:57.580329    2574 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Apr 24 14:23:57.583663 ip-10-0-136-132 kubenswrapper[2574]: W0424 14:23:57.580346    2574 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Apr 24 14:23:57.583663 ip-10-0-136-132 kubenswrapper[2574]: W0424 14:23:57.580350    2574 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Apr 24 14:23:57.583663 ip-10-0-136-132 kubenswrapper[2574]: W0424 14:23:57.580352    2574 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Apr 24 14:23:57.583663 ip-10-0-136-132 kubenswrapper[2574]: W0424 14:23:57.580355    2574 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Apr 24 14:23:57.583663 ip-10-0-136-132 kubenswrapper[2574]: W0424 14:23:57.580358    2574 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Apr 24 14:23:57.583663 ip-10-0-136-132 kubenswrapper[2574]: W0424 14:23:57.580360    2574 feature_gate.go:328] unrecognized feature gate: SetEIPForNLBIngressController
Apr 24 14:23:57.583663 ip-10-0-136-132 kubenswrapper[2574]: W0424 14:23:57.580363    2574 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Apr 24 14:23:57.583663 ip-10-0-136-132 kubenswrapper[2574]: W0424 14:23:57.580365    2574 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Apr 24 14:23:57.583663 ip-10-0-136-132 kubenswrapper[2574]: W0424 14:23:57.580369    2574 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Apr 24 14:23:57.583663 ip-10-0-136-132 kubenswrapper[2574]: W0424 14:23:57.580373    2574 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Apr 24 14:23:57.584143 ip-10-0-136-132 kubenswrapper[2574]: W0424 14:23:57.580376    2574 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Apr 24 14:23:57.584143 ip-10-0-136-132 kubenswrapper[2574]: W0424 14:23:57.580379    2574 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Apr 24 14:23:57.584143 ip-10-0-136-132 kubenswrapper[2574]: W0424 14:23:57.580381    2574 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterOperations
Apr 24 14:23:57.584143 ip-10-0-136-132 kubenswrapper[2574]: W0424 14:23:57.580384    2574 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Apr 24 14:23:57.584143 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:23:57.580389    2574 feature_gate.go:384] feature gates: {map[DynamicResourceAllocation:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutatingAdmissionPolicy:false NodeSwap:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:false]}
Apr 24 14:23:57.584143 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:23:57.581134    2574 server.go:962] "Client rotation is on, will bootstrap in background"
Apr 24 14:23:57.584370 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:23:57.584355    2574 bootstrap.go:101] "Use the bootstrap credentials to request a cert, and set kubeconfig to point to the certificate dir"
Apr 24 14:23:57.585236 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:23:57.585225    2574 server.go:1019] "Starting client certificate rotation"
Apr 24 14:23:57.585342 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:23:57.585318    2574 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kube-apiserver-client-kubelet"
Apr 24 14:23:57.585388 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:23:57.585369    2574 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kube-apiserver-client-kubelet"
Apr 24 14:23:57.607860 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:23:57.607836    2574 dynamic_cafile_content.go:123] "Loaded a new CA Bundle and Verifier" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt"
Apr 24 14:23:57.611970 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:23:57.611944    2574 dynamic_cafile_content.go:161] "Starting controller" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt"
Apr 24 14:23:57.624114 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:23:57.624087    2574 log.go:25] "Validated CRI v1 runtime API"
Apr 24 14:23:57.628709 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:23:57.628690    2574 log.go:25] "Validated CRI v1 image API"
Apr 24 14:23:57.630919 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:23:57.630904    2574 server.go:1452] "Using cgroup driver setting received from the CRI runtime" cgroupDriver="systemd"
Apr 24 14:23:57.635004 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:23:57.634977    2574 fs.go:135] Filesystem UUIDs: map[5f078a8d-cc46-4895-9f0a-126071fb180d:/dev/nvme0n1p3 7B77-95E7:/dev/nvme0n1p2 d4ff15c7-66c3-4da8-bf4e-05c1118781f4:/dev/nvme0n1p4]
Apr 24 14:23:57.635102 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:23:57.634999    2574 fs.go:136] Filesystem partitions: map[/dev/nvme0n1p3:{mountpoint:/boot major:259 minor:3 fsType:ext4 blockSize:0} /dev/nvme0n1p4:{mountpoint:/var major:259 minor:4 fsType:xfs blockSize:0} /dev/shm:{mountpoint:/dev/shm major:0 minor:22 fsType:tmpfs blockSize:0} /run:{mountpoint:/run major:0 minor:24 fsType:tmpfs blockSize:0} /tmp:{mountpoint:/tmp major:0 minor:31 fsType:tmpfs blockSize:0} composefs_0-33:{mountpoint:/ major:0 minor:33 fsType:overlay blockSize:0}]
Apr 24 14:23:57.637435 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:23:57.637417    2574 reflector.go:430] "Caches populated" logger="kubernetes.io/kube-apiserver-client-kubelet" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162"
Apr 24 14:23:57.640622 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:23:57.640513    2574 manager.go:217] Machine: {Timestamp:2026-04-24 14:23:57.638791875 +0000 UTC m=+0.370893515 CPUVendorID:GenuineIntel NumCores:8 NumPhysicalCores:4 NumSockets:1 CpuFrequency:3110235 MemoryCapacity:33164492800 SwapCapacity:0 MemoryByType:map[] NVMInfo:{MemoryModeCapacity:0 AppDirectModeCapacity:0 AvgPowerBudget:0} HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] MachineID:ec2393a1e52703f47ede8715af63315e SystemUUID:ec2393a1-e527-03f4-7ede-8715af63315e BootID:0fd73915-336b-4aa8-aa28-8e7f22ab03dc Filesystems:[{Device:composefs_0-33 DeviceMajor:0 DeviceMinor:33 Capacity:6103040 Type:vfs Inodes:18446744073709551615 HasInodes:true} {Device:/tmp DeviceMajor:0 DeviceMinor:31 Capacity:16582246400 Type:vfs Inodes:1048576 HasInodes:true} {Device:/dev/nvme0n1p3 DeviceMajor:259 DeviceMinor:3 Capacity:366869504 Type:vfs Inodes:98304 HasInodes:true} {Device:/dev/shm DeviceMajor:0 DeviceMinor:22 Capacity:16582246400 Type:vfs Inodes:4048400 HasInodes:true} {Device:/run DeviceMajor:0 DeviceMinor:24 Capacity:6632898560 Type:vfs Inodes:819200 HasInodes:true} {Device:/dev/nvme0n1p4 DeviceMajor:259 DeviceMinor:4 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true}] DiskMap:map[259:0:{Name:nvme0n1 Major:259 Minor:0 Size:128849018880 Scheduler:none}] NetworkDevices:[{Name:br-ex MacAddress:02:09:6b:f2:74:3d Speed:0 Mtu:9001} {Name:ens5 MacAddress:02:09:6b:f2:74:3d Speed:0 Mtu:9001} {Name:ovs-system MacAddress:82:35:14:df:99:57 Speed:0 Mtu:1500}] Topology:[{Id:0 Memory:33164492800 HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] Cores:[{Id:0 Threads:[0 4] Caches:[{Id:0 Size:32768 Type:Data Level:1} {Id:0 Size:32768 Type:Instruction Level:1} {Id:0 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:1 Threads:[1 5] Caches:[{Id:1 Size:32768 Type:Data Level:1} {Id:1 Size:32768 Type:Instruction Level:1} {Id:1 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:2 Threads:[2 6] Caches:[{Id:2 Size:32768 Type:Data Level:1} {Id:2 Size:32768 Type:Instruction Level:1} {Id:2 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:3 Threads:[3 7] Caches:[{Id:3 Size:32768 Type:Data Level:1} {Id:3 Size:32768 Type:Instruction Level:1} {Id:3 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:}] Caches:[{Id:0 Size:37486592 Type:Unified Level:3}] Distances:[10]}] CloudProvider:Unknown InstanceType:Unknown InstanceID:None}
Apr 24 14:23:57.640622 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:23:57.640616    2574 manager_no_libpfm.go:29] cAdvisor is build without cgo and/or libpfm support. Perf event counters are not available.
Apr 24 14:23:57.640730 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:23:57.640709    2574 manager.go:233] Version: {KernelVersion:5.14.0-570.107.1.el9_6.x86_64 ContainerOsVersion:Red Hat Enterprise Linux CoreOS 9.6.20260414-0 (Plow) DockerVersion: DockerAPIVersion: CadvisorVersion: CadvisorRevision:}
Apr 24 14:23:57.641780 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:23:57.641753    2574 container_manager_linux.go:270] "Container manager verified user specified cgroup-root exists" cgroupRoot=[]
Apr 24 14:23:57.641927 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:23:57.641783    2574 container_manager_linux.go:275] "Creating Container Manager object based on Node Config" nodeConfig={"NodeName":"ip-10-0-136-132.ec2.internal","RuntimeCgroupsName":"/system.slice/crio.service","SystemCgroupsName":"/system.slice","KubeletCgroupsName":"","KubeletOOMScoreAdj":-999,"ContainerRuntime":"","CgroupsPerQOS":true,"CgroupRoot":"/","CgroupDriver":"systemd","KubeletRootDir":"/var/lib/kubelet","ProtectKernelDefaults":true,"KubeReservedCgroupName":"","SystemReservedCgroupName":"","ReservedSystemCPUs":{},"EnforceNodeAllocatable":{"pods":{}},"KubeReserved":null,"SystemReserved":{"cpu":"500m","ephemeral-storage":"1Gi","memory":"1Gi"},"HardEvictionThresholds":[{"Signal":"memory.available","Operator":"LessThan","Value":{"Quantity":"100Mi","Percentage":0},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.1},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null},{"Signal":"imagefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.15},"GracePeriod":0,"MinReclaim":null},{"Signal":"imagefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null}],"QOSReserved":{},"CPUManagerPolicy":"none","CPUManagerPolicyOptions":null,"TopologyManagerScope":"container","CPUManagerReconcilePeriod":10000000000,"MemoryManagerPolicy":"None","MemoryManagerReservedMemory":null,"PodPidsLimit":4096,"EnforceCPULimits":true,"CPUCFSQuotaPeriod":100000000,"TopologyManagerPolicy":"none","TopologyManagerPolicyOptions":null,"CgroupVersion":2}
Apr 24 14:23:57.641969 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:23:57.641936    2574 topology_manager.go:138] "Creating topology manager with none policy"
Apr 24 14:23:57.641969 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:23:57.641945    2574 container_manager_linux.go:306] "Creating device plugin manager"
Apr 24 14:23:57.641969 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:23:57.641964    2574 manager.go:141] "Creating Device Plugin manager" path="/var/lib/kubelet/device-plugins/kubelet.sock"
Apr 24 14:23:57.642663 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:23:57.642652    2574 server.go:72] "Creating device plugin registration server" version="v1beta1" socket="/var/lib/kubelet/device-plugins/kubelet.sock"
Apr 24 14:23:57.643412 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:23:57.643402    2574 state_mem.go:36] "Initialized new in-memory state store"
Apr 24 14:23:57.643548 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:23:57.643539    2574 server.go:1267] "Using root directory" path="/var/lib/kubelet"
Apr 24 14:23:57.645742 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:23:57.645733    2574 kubelet.go:491] "Attempting to sync node with API server"
Apr 24 14:23:57.645785 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:23:57.645746    2574 kubelet.go:386] "Adding static pod path" path="/etc/kubernetes/manifests"
Apr 24 14:23:57.645785 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:23:57.645758    2574 file.go:69] "Watching path" path="/etc/kubernetes/manifests"
Apr 24 14:23:57.645785 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:23:57.645768    2574 kubelet.go:397] "Adding apiserver pod source"
Apr 24 14:23:57.645785 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:23:57.645778    2574 apiserver.go:42] "Waiting for node sync before watching apiserver pods"
Apr 24 14:23:57.647156 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:23:57.647144    2574 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking"
Apr 24 14:23:57.647207 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:23:57.647167    2574 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking"
Apr 24 14:23:57.650310 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:23:57.650294    2574 kuberuntime_manager.go:279] "Container runtime initialized" containerRuntime="cri-o" version="1.33.10-2.rhaos4.20.gita4d0894.el9" apiVersion="v1"
Apr 24 14:23:57.651610 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:23:57.651597    2574 kubelet.go:953] "Not starting ClusterTrustBundle informer because we are in static kubelet mode or the ClusterTrustBundleProjection featuregate is disabled"
Apr 24 14:23:57.653083 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:23:57.653067    2574 plugins.go:616] "Loaded volume plugin" pluginName="kubernetes.io/portworx-volume"
Apr 24 14:23:57.653083 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:23:57.653084    2574 plugins.go:616] "Loaded volume plugin" pluginName="kubernetes.io/empty-dir"
Apr 24 14:23:57.653183 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:23:57.653091    2574 plugins.go:616] "Loaded volume plugin" pluginName="kubernetes.io/git-repo"
Apr 24 14:23:57.653183 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:23:57.653096    2574 plugins.go:616] "Loaded volume plugin" pluginName="kubernetes.io/host-path"
Apr 24 14:23:57.653183 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:23:57.653103    2574 plugins.go:616] "Loaded volume plugin" pluginName="kubernetes.io/nfs"
Apr 24 14:23:57.653183 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:23:57.653109    2574 plugins.go:616] "Loaded volume plugin" pluginName="kubernetes.io/secret"
Apr 24 14:23:57.653183 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:23:57.653123    2574 plugins.go:616] "Loaded volume plugin" pluginName="kubernetes.io/iscsi"
Apr 24 14:23:57.653183 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:23:57.653129    2574 plugins.go:616] "Loaded volume plugin" pluginName="kubernetes.io/downward-api"
Apr 24 14:23:57.653183 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:23:57.653136    2574 plugins.go:616] "Loaded volume plugin" pluginName="kubernetes.io/fc"
Apr 24 14:23:57.653183 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:23:57.653143    2574 plugins.go:616] "Loaded volume plugin" pluginName="kubernetes.io/configmap"
Apr 24 14:23:57.653183 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:23:57.653158    2574 plugins.go:616] "Loaded volume plugin" pluginName="kubernetes.io/projected"
Apr 24 14:23:57.653183 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:23:57.653167    2574 plugins.go:616] "Loaded volume plugin" pluginName="kubernetes.io/local-volume"
Apr 24 14:23:57.654892 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:23:57.654881    2574 plugins.go:616] "Loaded volume plugin" pluginName="kubernetes.io/csi"
Apr 24 14:23:57.654892 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:23:57.654893    2574 plugins.go:616] "Loaded volume plugin" pluginName="kubernetes.io/image"
Apr 24 14:23:57.658063 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:23:57.658035    2574 csi_plugin.go:988] Failed to contact API server when waiting for CSINode publishing: csinodes.storage.k8s.io "ip-10-0-136-132.ec2.internal" is forbidden: User "system:anonymous" cannot get resource "csinodes" in API group "storage.k8s.io" at the cluster scope
Apr 24 14:23:57.658272 ip-10-0-136-132 kubenswrapper[2574]: E0424 14:23:57.658250    2574 reflector.go:200] "Failed to watch" err="failed to list *v1.Node: nodes \"ip-10-0-136-132.ec2.internal\" is forbidden: User \"system:anonymous\" cannot list resource \"nodes\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Node"
Apr 24 14:23:57.658521 ip-10-0-136-132 kubenswrapper[2574]: E0424 14:23:57.658502    2574 reflector.go:200] "Failed to watch" err="failed to list *v1.Service: services is forbidden: User \"system:anonymous\" cannot list resource \"services\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Service"
Apr 24 14:23:57.659400 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:23:57.659386    2574 watchdog_linux.go:99] "Systemd watchdog is not enabled"
Apr 24 14:23:57.659461 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:23:57.659419    2574 server.go:1295] "Started kubelet"
Apr 24 14:23:57.659605 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:23:57.659575    2574 server.go:180] "Starting to listen" address="0.0.0.0" port=10250
Apr 24 14:23:57.659919 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:23:57.659560    2574 ratelimit.go:55] "Setting rate limiting for endpoint" service="podresources" qps=100 burstTokens=10
Apr 24 14:23:57.659990 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:23:57.659939    2574 server_v1.go:47] "podresources" method="list" useActivePods=true
Apr 24 14:23:57.660354 ip-10-0-136-132 systemd[1]: Started Kubernetes Kubelet.
Apr 24 14:23:57.660932 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:23:57.660919    2574 server.go:255] "Starting to serve the podresources API" endpoint="unix:/var/lib/kubelet/pod-resources/kubelet.sock"
Apr 24 14:23:57.662121 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:23:57.662105    2574 server.go:317] "Adding debug handlers to kubelet server"
Apr 24 14:23:57.663801 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:23:57.663782    2574 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-fvzg4"
Apr 24 14:23:57.666939 ip-10-0-136-132 kubenswrapper[2574]: E0424 14:23:57.664100    2574 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-136-132.ec2.internal.18a9510b771f4163  default    0 0001-01-01 00:00:00 +0000 UTC <nil> <nil> map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-136-132.ec2.internal,UID:ip-10-0-136-132.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:Starting,Message:Starting kubelet.,Source:EventSource{Component:kubelet,Host:ip-10-0-136-132.ec2.internal,},FirstTimestamp:2026-04-24 14:23:57.659398499 +0000 UTC m=+0.391500138,LastTimestamp:2026-04-24 14:23:57.659398499 +0000 UTC m=+0.391500138,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-136-132.ec2.internal,}"
Apr 24 14:23:57.668799 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:23:57.668774    2574 fs_resource_analyzer.go:67] "Starting FS ResourceAnalyzer"
Apr 24 14:23:57.668874 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:23:57.668829    2574 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kubelet-serving"
Apr 24 14:23:57.669394 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:23:57.669378    2574 desired_state_of_world_populator.go:150] "Desired state populator starts to run"
Apr 24 14:23:57.669467 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:23:57.669419    2574 volume_manager.go:295] "The desired_state_of_world populator starts"
Apr 24 14:23:57.669467 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:23:57.669438    2574 volume_manager.go:297] "Starting Kubelet Volume Manager"
Apr 24 14:23:57.669626 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:23:57.669614    2574 reconstruct.go:97] "Volume reconstruction finished"
Apr 24 14:23:57.669626 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:23:57.669625    2574 reconciler.go:26] "Reconciler: start to sync state"
Apr 24 14:23:57.669626 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:23:57.669625    2574 factory.go:55] Registering systemd factory
Apr 24 14:23:57.669765 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:23:57.669641    2574 factory.go:223] Registration of the systemd container factory successfully
Apr 24 14:23:57.669898 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:23:57.669884    2574 factory.go:153] Registering CRI-O factory
Apr 24 14:23:57.669898 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:23:57.669897    2574 factory.go:223] Registration of the crio container factory successfully
Apr 24 14:23:57.670023 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:23:57.669953    2574 factory.go:221] Registration of the containerd container factory failed: unable to create containerd client: containerd: cannot unix dial containerd api service: dial unix /run/containerd/containerd.sock: connect: no such file or directory
Apr 24 14:23:57.670023 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:23:57.669973    2574 factory.go:103] Registering Raw factory
Apr 24 14:23:57.670023 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:23:57.669984    2574 manager.go:1196] Started watching for new ooms in manager
Apr 24 14:23:57.670489 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:23:57.670468    2574 manager.go:319] Starting recovery of all containers
Apr 24 14:23:57.670791 ip-10-0-136-132 kubenswrapper[2574]: E0424 14:23:57.670767    2574 kubelet_node_status.go:515] "Error getting the current node from lister" err="node \"ip-10-0-136-132.ec2.internal\" not found"
Apr 24 14:23:57.671740 ip-10-0-136-132 kubenswrapper[2574]: E0424 14:23:57.671715    2574 reflector.go:200] "Failed to watch" err="failed to list *v1.CSIDriver: csidrivers.storage.k8s.io is forbidden: User \"system:anonymous\" cannot list resource \"csidrivers\" in API group \"storage.k8s.io\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.CSIDriver"
Apr 24 14:23:57.672832 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:23:57.672812    2574 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-fvzg4"
Apr 24 14:23:57.673118 ip-10-0-136-132 kubenswrapper[2574]: E0424 14:23:57.673093    2574 kubelet.go:1618] "Image garbage collection failed once. Stats initialization may not have completed yet" err="failed to get imageFs info: unable to find data in memory cache"
Apr 24 14:23:57.680221 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:23:57.680207    2574 manager.go:324] Recovery completed
Apr 24 14:23:57.684199 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:23:57.684184    2574 kubelet_node_status.go:413] "Setting node annotation to enable volume controller attach/detach"
Apr 24 14:23:57.686569 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:23:57.686551    2574 kubelet_node_status.go:736] "Recording event message for node" node="ip-10-0-136-132.ec2.internal" event="NodeHasSufficientMemory"
Apr 24 14:23:57.686649 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:23:57.686583    2574 kubelet_node_status.go:736] "Recording event message for node" node="ip-10-0-136-132.ec2.internal" event="NodeHasNoDiskPressure"
Apr 24 14:23:57.686649 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:23:57.686598    2574 kubelet_node_status.go:736] "Recording event message for node" node="ip-10-0-136-132.ec2.internal" event="NodeHasSufficientPID"
Apr 24 14:23:57.687019 ip-10-0-136-132 kubenswrapper[2574]: E0424 14:23:57.687002    2574 nodelease.go:49] "Failed to get node when trying to set owner ref to the node lease" err="nodes \"ip-10-0-136-132.ec2.internal\" not found" node="ip-10-0-136-132.ec2.internal"
Apr 24 14:23:57.687125 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:23:57.687110    2574 cpu_manager.go:222] "Starting CPU manager" policy="none"
Apr 24 14:23:57.687170 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:23:57.687125    2574 cpu_manager.go:223] "Reconciling" reconcilePeriod="10s"
Apr 24 14:23:57.687170 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:23:57.687147    2574 state_mem.go:36] "Initialized new in-memory state store"
Apr 24 14:23:57.690413 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:23:57.690394    2574 policy_none.go:49] "None policy: Start"
Apr 24 14:23:57.690483 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:23:57.690417    2574 memory_manager.go:186] "Starting memorymanager" policy="None"
Apr 24 14:23:57.690483 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:23:57.690431    2574 state_mem.go:35] "Initializing new in-memory state store"
Apr 24 14:23:57.723072 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:23:57.723056    2574 manager.go:341] "Starting Device Plugin manager"
Apr 24 14:23:57.734964 ip-10-0-136-132 kubenswrapper[2574]: E0424 14:23:57.723095    2574 manager.go:517] "Failed to read data from checkpoint" err="checkpoint is not found" checkpoint="kubelet_internal_checkpoint"
Apr 24 14:23:57.734964 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:23:57.723109    2574 server.go:85] "Starting device plugin registration server"
Apr 24 14:23:57.734964 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:23:57.723388    2574 eviction_manager.go:189] "Eviction manager: starting control loop"
Apr 24 14:23:57.734964 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:23:57.723402    2574 container_log_manager.go:189] "Initializing container log rotate workers" workers=1 monitorPeriod="10s"
Apr 24 14:23:57.734964 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:23:57.723482    2574 plugin_watcher.go:51] "Plugin Watcher Start" path="/var/lib/kubelet/plugins_registry"
Apr 24 14:23:57.734964 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:23:57.723559    2574 plugin_manager.go:116] "The desired_state_of_world populator (plugin watcher) starts"
Apr 24 14:23:57.734964 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:23:57.723566    2574 plugin_manager.go:118] "Starting Kubelet Plugin Manager"
Apr 24 14:23:57.734964 ip-10-0-136-132 kubenswrapper[2574]: E0424 14:23:57.724486    2574 eviction_manager.go:267] "eviction manager: failed to check if we have separate container filesystem. Ignoring." err="non-existent label \"crio-containers\""
Apr 24 14:23:57.734964 ip-10-0-136-132 kubenswrapper[2574]: E0424 14:23:57.724545    2574 eviction_manager.go:292] "Eviction manager: failed to get summary stats" err="failed to get node info: node \"ip-10-0-136-132.ec2.internal\" not found"
Apr 24 14:23:57.763831 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:23:57.763792    2574 kubelet_network_linux.go:49] "Initialized iptables rules." protocol="IPv4"
Apr 24 14:23:57.764982 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:23:57.764968    2574 kubelet_network_linux.go:49] "Initialized iptables rules." protocol="IPv6"
Apr 24 14:23:57.765042 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:23:57.764992    2574 status_manager.go:230] "Starting to sync pod status with apiserver"
Apr 24 14:23:57.765042 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:23:57.765013    2574 watchdog_linux.go:127] "Systemd watchdog is not enabled or the interval is invalid, so health checking will not be started."
Apr 24 14:23:57.765042 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:23:57.765019    2574 kubelet.go:2451] "Starting kubelet main sync loop"
Apr 24 14:23:57.765141 ip-10-0-136-132 kubenswrapper[2574]: E0424 14:23:57.765055    2574 kubelet.go:2475] "Skipping pod synchronization" err="PLEG is not healthy: pleg has yet to be successful"
Apr 24 14:23:57.767842 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:23:57.767821    2574 reflector.go:430] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160"
Apr 24 14:23:57.824430 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:23:57.824350    2574 kubelet_node_status.go:413] "Setting node annotation to enable volume controller attach/detach"
Apr 24 14:23:57.826468 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:23:57.826452    2574 kubelet_node_status.go:736] "Recording event message for node" node="ip-10-0-136-132.ec2.internal" event="NodeHasSufficientMemory"
Apr 24 14:23:57.826539 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:23:57.826484    2574 kubelet_node_status.go:736] "Recording event message for node" node="ip-10-0-136-132.ec2.internal" event="NodeHasNoDiskPressure"
Apr 24 14:23:57.826539 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:23:57.826501    2574 kubelet_node_status.go:736] "Recording event message for node" node="ip-10-0-136-132.ec2.internal" event="NodeHasSufficientPID"
Apr 24 14:23:57.826539 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:23:57.826524    2574 kubelet_node_status.go:78] "Attempting to register node" node="ip-10-0-136-132.ec2.internal"
Apr 24 14:23:57.832898 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:23:57.832881    2574 kubelet_node_status.go:81] "Successfully registered node" node="ip-10-0-136-132.ec2.internal"
Apr 24 14:23:57.832960 ip-10-0-136-132 kubenswrapper[2574]: E0424 14:23:57.832907    2574 kubelet_node_status.go:597] "Error updating node status, will retry" err="error getting node \"ip-10-0-136-132.ec2.internal\": node \"ip-10-0-136-132.ec2.internal\" not found"
Apr 24 14:23:57.851719 ip-10-0-136-132 kubenswrapper[2574]: E0424 14:23:57.851693    2574 kubelet_node_status.go:515] "Error getting the current node from lister" err="node \"ip-10-0-136-132.ec2.internal\" not found"
Apr 24 14:23:57.865704 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:23:57.865670    2574 kubelet.go:2537] "SyncLoop ADD" source="file" pods=["openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-136-132.ec2.internal","kube-system/kube-apiserver-proxy-ip-10-0-136-132.ec2.internal"]
Apr 24 14:23:57.865762 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:23:57.865752    2574 kubelet_node_status.go:413] "Setting node annotation to enable volume controller attach/detach"
Apr 24 14:23:57.866702 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:23:57.866685    2574 kubelet_node_status.go:736] "Recording event message for node" node="ip-10-0-136-132.ec2.internal" event="NodeHasSufficientMemory"
Apr 24 14:23:57.866809 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:23:57.866720    2574 kubelet_node_status.go:736] "Recording event message for node" node="ip-10-0-136-132.ec2.internal" event="NodeHasNoDiskPressure"
Apr 24 14:23:57.866809 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:23:57.866735    2574 kubelet_node_status.go:736] "Recording event message for node" node="ip-10-0-136-132.ec2.internal" event="NodeHasSufficientPID"
Apr 24 14:23:57.868565 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:23:57.868243    2574 kubelet_node_status.go:413] "Setting node annotation to enable volume controller attach/detach"
Apr 24 14:23:57.868682 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:23:57.868639    2574 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-136-132.ec2.internal"
Apr 24 14:23:57.868738 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:23:57.868696    2574 kubelet_node_status.go:413] "Setting node annotation to enable volume controller attach/detach"
Apr 24 14:23:57.869619 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:23:57.869603    2574 kubelet_node_status.go:736] "Recording event message for node" node="ip-10-0-136-132.ec2.internal" event="NodeHasSufficientMemory"
Apr 24 14:23:57.869619 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:23:57.869615    2574 kubelet_node_status.go:736] "Recording event message for node" node="ip-10-0-136-132.ec2.internal" event="NodeHasSufficientMemory"
Apr 24 14:23:57.869725 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:23:57.869633    2574 kubelet_node_status.go:736] "Recording event message for node" node="ip-10-0-136-132.ec2.internal" event="NodeHasNoDiskPressure"
Apr 24 14:23:57.869725 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:23:57.869636    2574 kubelet_node_status.go:736] "Recording event message for node" node="ip-10-0-136-132.ec2.internal" event="NodeHasNoDiskPressure"
Apr 24 14:23:57.869725 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:23:57.869646    2574 kubelet_node_status.go:736] "Recording event message for node" node="ip-10-0-136-132.ec2.internal" event="NodeHasSufficientPID"
Apr 24 14:23:57.869725 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:23:57.869647    2574 kubelet_node_status.go:736] "Recording event message for node" node="ip-10-0-136-132.ec2.internal" event="NodeHasSufficientPID"
Apr 24 14:23:57.870782 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:23:57.870766    2574 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-136-132.ec2.internal"
Apr 24 14:23:57.870845 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:23:57.870797    2574 kubelet_node_status.go:413] "Setting node annotation to enable volume controller attach/detach"
Apr 24 14:23:57.871667 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:23:57.871651    2574 kubelet_node_status.go:736] "Recording event message for node" node="ip-10-0-136-132.ec2.internal" event="NodeHasSufficientMemory"
Apr 24 14:23:57.871745 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:23:57.871681    2574 kubelet_node_status.go:736] "Recording event message for node" node="ip-10-0-136-132.ec2.internal" event="NodeHasNoDiskPressure"
Apr 24 14:23:57.871745 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:23:57.871691    2574 kubelet_node_status.go:736] "Recording event message for node" node="ip-10-0-136-132.ec2.internal" event="NodeHasSufficientPID"
Apr 24 14:23:57.900069 ip-10-0-136-132 kubenswrapper[2574]: E0424 14:23:57.900041    2574 kubelet.go:3336] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-136-132.ec2.internal\" not found" node="ip-10-0-136-132.ec2.internal"
Apr 24 14:23:57.904429 ip-10-0-136-132 kubenswrapper[2574]: E0424 14:23:57.904409    2574 kubelet.go:3336] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-136-132.ec2.internal\" not found" node="ip-10-0-136-132.ec2.internal"
Apr 24 14:23:57.952052 ip-10-0-136-132 kubenswrapper[2574]: E0424 14:23:57.952013    2574 kubelet_node_status.go:515] "Error getting the current node from lister" err="node \"ip-10-0-136-132.ec2.internal\" not found"
Apr 24 14:23:57.970519 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:23:57.970495    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/b5277dfd83a996ad93ff188797af276d-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-136-132.ec2.internal\" (UID: \"b5277dfd83a996ad93ff188797af276d\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-136-132.ec2.internal"
Apr 24 14:23:57.970595 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:23:57.970523    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/b5277dfd83a996ad93ff188797af276d-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-136-132.ec2.internal\" (UID: \"b5277dfd83a996ad93ff188797af276d\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-136-132.ec2.internal"
Apr 24 14:23:57.970595 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:23:57.970541    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/682e7062ab8cd6aae842785f377acfd4-config\") pod \"kube-apiserver-proxy-ip-10-0-136-132.ec2.internal\" (UID: \"682e7062ab8cd6aae842785f377acfd4\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-136-132.ec2.internal"
Apr 24 14:23:58.052961 ip-10-0-136-132 kubenswrapper[2574]: E0424 14:23:58.052923    2574 kubelet_node_status.go:515] "Error getting the current node from lister" err="node \"ip-10-0-136-132.ec2.internal\" not found"
Apr 24 14:23:58.071219 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:23:58.071196    2574 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/b5277dfd83a996ad93ff188797af276d-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-136-132.ec2.internal\" (UID: \"b5277dfd83a996ad93ff188797af276d\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-136-132.ec2.internal"
Apr 24 14:23:58.071279 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:23:58.071225    2574 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/b5277dfd83a996ad93ff188797af276d-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-136-132.ec2.internal\" (UID: \"b5277dfd83a996ad93ff188797af276d\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-136-132.ec2.internal"
Apr 24 14:23:58.071279 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:23:58.071246    2574 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/682e7062ab8cd6aae842785f377acfd4-config\") pod \"kube-apiserver-proxy-ip-10-0-136-132.ec2.internal\" (UID: \"682e7062ab8cd6aae842785f377acfd4\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-136-132.ec2.internal"
Apr 24 14:23:58.071360 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:23:58.071299    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/b5277dfd83a996ad93ff188797af276d-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-136-132.ec2.internal\" (UID: \"b5277dfd83a996ad93ff188797af276d\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-136-132.ec2.internal"
Apr 24 14:23:58.071360 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:23:58.071305    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/b5277dfd83a996ad93ff188797af276d-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-136-132.ec2.internal\" (UID: \"b5277dfd83a996ad93ff188797af276d\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-136-132.ec2.internal"
Apr 24 14:23:58.071422 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:23:58.071364    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/host-path/682e7062ab8cd6aae842785f377acfd4-config\") pod \"kube-apiserver-proxy-ip-10-0-136-132.ec2.internal\" (UID: \"682e7062ab8cd6aae842785f377acfd4\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-136-132.ec2.internal"
Apr 24 14:23:58.153675 ip-10-0-136-132 kubenswrapper[2574]: E0424 14:23:58.153614    2574 kubelet_node_status.go:515] "Error getting the current node from lister" err="node \"ip-10-0-136-132.ec2.internal\" not found"
Apr 24 14:23:58.202133 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:23:58.202096    2574 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-136-132.ec2.internal"
Apr 24 14:23:58.206760 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:23:58.206742    2574 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-136-132.ec2.internal"
Apr 24 14:23:58.254580 ip-10-0-136-132 kubenswrapper[2574]: E0424 14:23:58.254551    2574 kubelet_node_status.go:515] "Error getting the current node from lister" err="node \"ip-10-0-136-132.ec2.internal\" not found"
Apr 24 14:23:58.355105 ip-10-0-136-132 kubenswrapper[2574]: E0424 14:23:58.355071    2574 kubelet_node_status.go:515] "Error getting the current node from lister" err="node \"ip-10-0-136-132.ec2.internal\" not found"
Apr 24 14:23:58.455741 ip-10-0-136-132 kubenswrapper[2574]: E0424 14:23:58.455647    2574 kubelet_node_status.go:515] "Error getting the current node from lister" err="node \"ip-10-0-136-132.ec2.internal\" not found"
Apr 24 14:23:58.556287 ip-10-0-136-132 kubenswrapper[2574]: E0424 14:23:58.556241    2574 kubelet_node_status.go:515] "Error getting the current node from lister" err="node \"ip-10-0-136-132.ec2.internal\" not found"
Apr 24 14:23:58.585695 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:23:58.585664    2574 transport.go:147] "Certificate rotation detected, shutting down client connections to start using new credentials"
Apr 24 14:23:58.586324 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:23:58.585801    2574 reflector.go:556] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.RuntimeClass" err="very short watch: k8s.io/client-go/informers/factory.go:160: Unexpected watch close - watch lasted less than a second and no items received"
Apr 24 14:23:58.634574 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:23:58.634546    2574 reflector.go:430] "Caches populated" type="*v1.Service" reflector="k8s.io/client-go/informers/factory.go:160"
Apr 24 14:23:58.657190 ip-10-0-136-132 kubenswrapper[2574]: E0424 14:23:58.657160    2574 kubelet_node_status.go:515] "Error getting the current node from lister" err="node \"ip-10-0-136-132.ec2.internal\" not found"
Apr 24 14:23:58.669711 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:23:58.669570    2574 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kubelet-serving"
Apr 24 14:23:58.675700 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:23:58.675668    2574 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kube-apiserver-client-kubelet" expiration="2028-04-23 14:18:57 +0000 UTC" deadline="2027-10-10 04:40:32.302538255 +0000 UTC"
Apr 24 14:23:58.675700 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:23:58.675696    2574 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kube-apiserver-client-kubelet" sleep="12806h16m33.626845286s"
Apr 24 14:23:58.679251 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:23:58.679230    2574 reflector.go:430] "Caches populated" logger="kubernetes.io/kubelet-serving" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162"
Apr 24 14:23:58.703465 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:23:58.703440    2574 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kubelet-serving" csr="csr-hbkjj"
Apr 24 14:23:58.711384 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:23:58.711305    2574 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kubelet-serving" csr="csr-hbkjj"
Apr 24 14:23:58.751171 ip-10-0-136-132 kubenswrapper[2574]: W0424 14:23:58.751141    2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb5277dfd83a996ad93ff188797af276d.slice/crio-b22a5bd44b854b4c268f2971d606af9f02e0ab0e0b3c63c900c8e61877fb7a86 WatchSource:0}: Error finding container b22a5bd44b854b4c268f2971d606af9f02e0ab0e0b3c63c900c8e61877fb7a86: Status 404 returned error can't find the container with id b22a5bd44b854b4c268f2971d606af9f02e0ab0e0b3c63c900c8e61877fb7a86
Apr 24 14:23:58.751413 ip-10-0-136-132 kubenswrapper[2574]: W0424 14:23:58.751394    2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod682e7062ab8cd6aae842785f377acfd4.slice/crio-168b74379fdee1977a29fecf491dc6a3f89f972fd31f184b0a50c485aae595eb WatchSource:0}: Error finding container 168b74379fdee1977a29fecf491dc6a3f89f972fd31f184b0a50c485aae595eb: Status 404 returned error can't find the container with id 168b74379fdee1977a29fecf491dc6a3f89f972fd31f184b0a50c485aae595eb
Apr 24 14:23:58.756843 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:23:58.756821    2574 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Apr 24 14:23:58.757244 ip-10-0-136-132 kubenswrapper[2574]: E0424 14:23:58.757220    2574 kubelet_node_status.go:515] "Error getting the current node from lister" err="node \"ip-10-0-136-132.ec2.internal\" not found"
Apr 24 14:23:58.767568 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:23:58.767529    2574 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-136-132.ec2.internal" event={"ID":"b5277dfd83a996ad93ff188797af276d","Type":"ContainerStarted","Data":"b22a5bd44b854b4c268f2971d606af9f02e0ab0e0b3c63c900c8e61877fb7a86"}
Apr 24 14:23:58.768384 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:23:58.768359    2574 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-136-132.ec2.internal" event={"ID":"682e7062ab8cd6aae842785f377acfd4","Type":"ContainerStarted","Data":"168b74379fdee1977a29fecf491dc6a3f89f972fd31f184b0a50c485aae595eb"}
Apr 24 14:23:58.857556 ip-10-0-136-132 kubenswrapper[2574]: E0424 14:23:58.857514    2574 kubelet_node_status.go:515] "Error getting the current node from lister" err="node \"ip-10-0-136-132.ec2.internal\" not found"
Apr 24 14:23:58.958005 ip-10-0-136-132 kubenswrapper[2574]: E0424 14:23:58.957979    2574 kubelet_node_status.go:515] "Error getting the current node from lister" err="node \"ip-10-0-136-132.ec2.internal\" not found"
Apr 24 14:23:58.960842 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:23:58.960829    2574 reflector.go:430] "Caches populated" type="*v1.CSIDriver" reflector="k8s.io/client-go/informers/factory.go:160"
Apr 24 14:23:59.046493 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:23:59.046416    2574 reflector.go:430] "Caches populated" type="*v1.Node" reflector="k8s.io/client-go/informers/factory.go:160"
Apr 24 14:23:59.069655 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:23:59.069624    2574 kubelet.go:3340] "Creating a mirror pod for static pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-136-132.ec2.internal"
Apr 24 14:23:59.080839 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:23:59.080816    2574 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]"
Apr 24 14:23:59.081737 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:23:59.081724    2574 kubelet.go:3340] "Creating a mirror pod for static pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-136-132.ec2.internal"
Apr 24 14:23:59.089160 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:23:59.089144    2574 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]"
Apr 24 14:23:59.488563 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:23:59.488482    2574 reflector.go:430] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160"
Apr 24 14:23:59.647507 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:23:59.647469    2574 apiserver.go:52] "Watching apiserver"
Apr 24 14:23:59.651879 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:23:59.651854    2574 reflector.go:430] "Caches populated" type="*v1.Pod" reflector="pkg/kubelet/config/apiserver.go:66"
Apr 24 14:23:59.652273 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:23:59.652240    2574 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["openshift-network-operator/iptables-alerter-r2pxq","kube-system/konnectivity-agent-zz678","openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-6mgkm","openshift-cluster-node-tuning-operator/tuned-4kqkj","openshift-image-registry/node-ca-lgh9n","openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-136-132.ec2.internal","openshift-multus/network-metrics-daemon-zhkmq","openshift-network-diagnostics/network-check-target-6tsjh","openshift-ovn-kubernetes/ovnkube-node-w5x5x","kube-system/kube-apiserver-proxy-ip-10-0-136-132.ec2.internal","openshift-dns/node-resolver-bt99l","openshift-multus/multus-additional-cni-plugins-j2jc7","openshift-multus/multus-rwdxf"]
Apr 24 14:23:59.656830 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:23:59.656808    2574 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-zhkmq"
Apr 24 14:23:59.656925 ip-10-0-136-132 kubenswrapper[2574]: E0424 14:23:59.656893    2574 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-zhkmq" podUID="3bc44c73-963f-4a9e-8a52-1e8b031a08dc"
Apr 24 14:23:59.658954 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:23:59.658924    2574 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-zz678"
Apr 24 14:23:59.659087 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:23:59.659056    2574 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-6mgkm"
Apr 24 14:23:59.661240 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:23:59.661103    2574 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"konnectivity-agent\""
Apr 24 14:23:59.661240 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:23:59.661214    2574 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-sa-dockercfg-zx5w5\""
Apr 24 14:23:59.661771 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:23:59.661521    2574 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"default-dockercfg-ml6fg\""
Apr 24 14:23:59.661876 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:23:59.661849    2574 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-metrics-serving-cert\""
Apr 24 14:23:59.662001 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:23:59.661979    2574 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"openshift-service-ca.crt\""
Apr 24 14:23:59.662613 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:23:59.662212    2574 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"kube-root-ca.crt\""
Apr 24 14:23:59.662613 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:23:59.662244    2574 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kube-system\"/\"konnectivity-ca-bundle\""
Apr 24 14:23:59.662909 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:23:59.662883    2574 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-4kqkj"
Apr 24 14:23:59.664478 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:23:59.664444    2574 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"openshift-service-ca.crt\""
Apr 24 14:23:59.664608 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:23:59.664588    2574 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"tuned-dockercfg-dqmt8\""
Apr 24 14:23:59.664753 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:23:59.664720    2574 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"kube-root-ca.crt\""
Apr 24 14:23:59.665726 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:23:59.665707    2574 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-lgh9n"
Apr 24 14:23:59.667080 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:23:59.667063    2574 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"openshift-service-ca.crt\""
Apr 24 14:23:59.667242 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:23:59.667227    2574 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"kube-root-ca.crt\""
Apr 24 14:23:59.667321 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:23:59.667299    2574 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"image-registry-certificates\""
Apr 24 14:23:59.667436 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:23:59.667381    2574 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"node-ca-dockercfg-m8lld\""
Apr 24 14:23:59.668157 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:23:59.667940    2574 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-r2pxq"
Apr 24 14:23:59.669412 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:23:59.669391    2574 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"openshift-service-ca.crt\""
Apr 24 14:23:59.669639 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:23:59.669617    2574 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-dockercfg-88gmb\""
Apr 24 14:23:59.669729 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:23:59.669713    2574 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"kube-root-ca.crt\""
Apr 24 14:23:59.669891 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:23:59.669875    2574 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-script\""
Apr 24 14:23:59.670309 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:23:59.670207    2574 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-6tsjh"
Apr 24 14:23:59.670309 ip-10-0-136-132 kubenswrapper[2574]: E0424 14:23:59.670280    2574 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-6tsjh" podUID="3964207c-9c58-4994-a8a9-aa4e4feca773"
Apr 24 14:23:59.673208 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:23:59.673173    2574 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-w5x5x"
Apr 24 14:23:59.674795 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:23:59.674779    2574 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-kubernetes-node-dockercfg-dq7l2\""
Apr 24 14:23:59.674949 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:23:59.674927    2574 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-node-metrics-cert\""
Apr 24 14:23:59.675037 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:23:59.674959    2574 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-config\""
Apr 24 14:23:59.675146 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:23:59.675121    2574 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"kube-root-ca.crt\""
Apr 24 14:23:59.675146 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:23:59.675134    2574 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"env-overrides\""
Apr 24 14:23:59.675259 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:23:59.675171    2574 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-script-lib\""
Apr 24 14:23:59.675314 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:23:59.675258    2574 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"openshift-service-ca.crt\""
Apr 24 14:23:59.675596 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:23:59.675583    2574 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-bt99l"
Apr 24 14:23:59.676928 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:23:59.676907    2574 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"node-resolver-dockercfg-xvfpf\""
Apr 24 14:23:59.677006 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:23:59.676951    2574 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"kube-root-ca.crt\""
Apr 24 14:23:59.677071 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:23:59.677028    2574 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"openshift-service-ca.crt\""
Apr 24 14:23:59.677891 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:23:59.677873    2574 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-j2jc7"
Apr 24 14:23:59.679549 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:23:59.679518    2574 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"default-cni-sysctl-allowlist\""
Apr 24 14:23:59.679638 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:23:59.679586    2574 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"openshift-service-ca.crt\""
Apr 24 14:23:59.679720 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:23:59.679702    2574 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"whereabouts-flatfile-config\""
Apr 24 14:23:59.680024 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:23:59.679886    2574 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"cni-copy-resources\""
Apr 24 14:23:59.680024 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:23:59.679941    2574 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"multus-ancillary-tools-dockercfg-qrfns\""
Apr 24 14:23:59.680155 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:23:59.680087    2574 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-rwdxf"
Apr 24 14:23:59.680211 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:23:59.680087    2574 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"kube-root-ca.crt\""
Apr 24 14:23:59.680355 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:23:59.680311    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/b1844926-4d01-4f6e-8970-a8d662acd7c6-agent-certs\") pod \"konnectivity-agent-zz678\" (UID: \"b1844926-4d01-4f6e-8970-a8d662acd7c6\") " pod="kube-system/konnectivity-agent-zz678"
Apr 24 14:23:59.680440 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:23:59.680366    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/3bc44c73-963f-4a9e-8a52-1e8b031a08dc-metrics-certs\") pod \"network-metrics-daemon-zhkmq\" (UID: \"3bc44c73-963f-4a9e-8a52-1e8b031a08dc\") " pod="openshift-multus/network-metrics-daemon-zhkmq"
Apr 24 14:23:59.680440 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:23:59.680391    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/21ff8454-68c7-440e-9994-e8405b47919a-run\") pod \"tuned-4kqkj\" (UID: \"21ff8454-68c7-440e-9994-e8405b47919a\") " pod="openshift-cluster-node-tuning-operator/tuned-4kqkj"
Apr 24 14:23:59.680440 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:23:59.680409    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-m9qwz\" (UniqueName: \"kubernetes.io/projected/5e4a7a90-d6ce-491b-ad65-52f0b2a127d6-kube-api-access-m9qwz\") pod \"node-ca-lgh9n\" (UID: \"5e4a7a90-d6ce-491b-ad65-52f0b2a127d6\") " pod="openshift-image-registry/node-ca-lgh9n"
Apr 24 14:23:59.680440 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:23:59.680425    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/b1844926-4d01-4f6e-8970-a8d662acd7c6-konnectivity-ca\") pod \"konnectivity-agent-zz678\" (UID: \"b1844926-4d01-4f6e-8970-a8d662acd7c6\") " pod="kube-system/konnectivity-agent-zz678"
Apr 24 14:23:59.680613 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:23:59.680440    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/97869a65-fbd6-4a1b-9eac-7adc13e4c254-device-dir\") pod \"aws-ebs-csi-driver-node-6mgkm\" (UID: \"97869a65-fbd6-4a1b-9eac-7adc13e4c254\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-6mgkm"
Apr 24 14:23:59.680613 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:23:59.680465    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zdh58\" (UniqueName: \"kubernetes.io/projected/97869a65-fbd6-4a1b-9eac-7adc13e4c254-kube-api-access-zdh58\") pod \"aws-ebs-csi-driver-node-6mgkm\" (UID: \"97869a65-fbd6-4a1b-9eac-7adc13e4c254\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-6mgkm"
Apr 24 14:23:59.680613 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:23:59.680485    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/5e4a7a90-d6ce-491b-ad65-52f0b2a127d6-host\") pod \"node-ca-lgh9n\" (UID: \"5e4a7a90-d6ce-491b-ad65-52f0b2a127d6\") " pod="openshift-image-registry/node-ca-lgh9n"
Apr 24 14:23:59.680613 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:23:59.680509    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/97869a65-fbd6-4a1b-9eac-7adc13e4c254-etc-selinux\") pod \"aws-ebs-csi-driver-node-6mgkm\" (UID: \"97869a65-fbd6-4a1b-9eac-7adc13e4c254\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-6mgkm"
Apr 24 14:23:59.680613 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:23:59.680531    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/21ff8454-68c7-440e-9994-e8405b47919a-etc-sysctl-d\") pod \"tuned-4kqkj\" (UID: \"21ff8454-68c7-440e-9994-e8405b47919a\") " pod="openshift-cluster-node-tuning-operator/tuned-4kqkj"
Apr 24 14:23:59.680613 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:23:59.680553    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/95f221a8-f16d-4d5f-9638-29db9dd3a969-iptables-alerter-script\") pod \"iptables-alerter-r2pxq\" (UID: \"95f221a8-f16d-4d5f-9638-29db9dd3a969\") " pod="openshift-network-operator/iptables-alerter-r2pxq"
Apr 24 14:23:59.680613 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:23:59.680575    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/97869a65-fbd6-4a1b-9eac-7adc13e4c254-kubelet-dir\") pod \"aws-ebs-csi-driver-node-6mgkm\" (UID: \"97869a65-fbd6-4a1b-9eac-7adc13e4c254\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-6mgkm"
Apr 24 14:23:59.680613 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:23:59.680597    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/97869a65-fbd6-4a1b-9eac-7adc13e4c254-sys-fs\") pod \"aws-ebs-csi-driver-node-6mgkm\" (UID: \"97869a65-fbd6-4a1b-9eac-7adc13e4c254\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-6mgkm"
Apr 24 14:23:59.680911 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:23:59.680616    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/21ff8454-68c7-440e-9994-e8405b47919a-lib-modules\") pod \"tuned-4kqkj\" (UID: \"21ff8454-68c7-440e-9994-e8405b47919a\") " pod="openshift-cluster-node-tuning-operator/tuned-4kqkj"
Apr 24 14:23:59.680911 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:23:59.680676    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/21ff8454-68c7-440e-9994-e8405b47919a-etc-tuned\") pod \"tuned-4kqkj\" (UID: \"21ff8454-68c7-440e-9994-e8405b47919a\") " pod="openshift-cluster-node-tuning-operator/tuned-4kqkj"
Apr 24 14:23:59.680911 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:23:59.680728    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/97869a65-fbd6-4a1b-9eac-7adc13e4c254-registration-dir\") pod \"aws-ebs-csi-driver-node-6mgkm\" (UID: \"97869a65-fbd6-4a1b-9eac-7adc13e4c254\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-6mgkm"
Apr 24 14:23:59.680911 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:23:59.680759    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-css2f\" (UniqueName: \"kubernetes.io/projected/3964207c-9c58-4994-a8a9-aa4e4feca773-kube-api-access-css2f\") pod \"network-check-target-6tsjh\" (UID: \"3964207c-9c58-4994-a8a9-aa4e4feca773\") " pod="openshift-network-diagnostics/network-check-target-6tsjh"
Apr 24 14:23:59.680911 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:23:59.680783    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/21ff8454-68c7-440e-9994-e8405b47919a-etc-kubernetes\") pod \"tuned-4kqkj\" (UID: \"21ff8454-68c7-440e-9994-e8405b47919a\") " pod="openshift-cluster-node-tuning-operator/tuned-4kqkj"
Apr 24 14:23:59.680911 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:23:59.680808    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/21ff8454-68c7-440e-9994-e8405b47919a-etc-sysctl-conf\") pod \"tuned-4kqkj\" (UID: \"21ff8454-68c7-440e-9994-e8405b47919a\") " pod="openshift-cluster-node-tuning-operator/tuned-4kqkj"
Apr 24 14:23:59.680911 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:23:59.680865    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/95f221a8-f16d-4d5f-9638-29db9dd3a969-host-slash\") pod \"iptables-alerter-r2pxq\" (UID: \"95f221a8-f16d-4d5f-9638-29db9dd3a969\") " pod="openshift-network-operator/iptables-alerter-r2pxq"
Apr 24 14:23:59.680911 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:23:59.680888    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/21ff8454-68c7-440e-9994-e8405b47919a-etc-sysconfig\") pod \"tuned-4kqkj\" (UID: \"21ff8454-68c7-440e-9994-e8405b47919a\") " pod="openshift-cluster-node-tuning-operator/tuned-4kqkj"
Apr 24 14:23:59.681282 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:23:59.680913    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/21ff8454-68c7-440e-9994-e8405b47919a-sys\") pod \"tuned-4kqkj\" (UID: \"21ff8454-68c7-440e-9994-e8405b47919a\") " pod="openshift-cluster-node-tuning-operator/tuned-4kqkj"
Apr 24 14:23:59.681282 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:23:59.680938    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/21ff8454-68c7-440e-9994-e8405b47919a-host\") pod \"tuned-4kqkj\" (UID: \"21ff8454-68c7-440e-9994-e8405b47919a\") " pod="openshift-cluster-node-tuning-operator/tuned-4kqkj"
Apr 24 14:23:59.681282 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:23:59.680962    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/21ff8454-68c7-440e-9994-e8405b47919a-tmp\") pod \"tuned-4kqkj\" (UID: \"21ff8454-68c7-440e-9994-e8405b47919a\") " pod="openshift-cluster-node-tuning-operator/tuned-4kqkj"
Apr 24 14:23:59.681282 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:23:59.681001    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5js85\" (UniqueName: \"kubernetes.io/projected/21ff8454-68c7-440e-9994-e8405b47919a-kube-api-access-5js85\") pod \"tuned-4kqkj\" (UID: \"21ff8454-68c7-440e-9994-e8405b47919a\") " pod="openshift-cluster-node-tuning-operator/tuned-4kqkj"
Apr 24 14:23:59.681282 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:23:59.681050    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/5e4a7a90-d6ce-491b-ad65-52f0b2a127d6-serviceca\") pod \"node-ca-lgh9n\" (UID: \"5e4a7a90-d6ce-491b-ad65-52f0b2a127d6\") " pod="openshift-image-registry/node-ca-lgh9n"
Apr 24 14:23:59.681282 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:23:59.681073    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/21ff8454-68c7-440e-9994-e8405b47919a-var-lib-kubelet\") pod \"tuned-4kqkj\" (UID: \"21ff8454-68c7-440e-9994-e8405b47919a\") " pod="openshift-cluster-node-tuning-operator/tuned-4kqkj"
Apr 24 14:23:59.681282 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:23:59.681116    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-c6kpn\" (UniqueName: \"kubernetes.io/projected/3bc44c73-963f-4a9e-8a52-1e8b031a08dc-kube-api-access-c6kpn\") pod \"network-metrics-daemon-zhkmq\" (UID: \"3bc44c73-963f-4a9e-8a52-1e8b031a08dc\") " pod="openshift-multus/network-metrics-daemon-zhkmq"
Apr 24 14:23:59.681282 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:23:59.681144    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/97869a65-fbd6-4a1b-9eac-7adc13e4c254-socket-dir\") pod \"aws-ebs-csi-driver-node-6mgkm\" (UID: \"97869a65-fbd6-4a1b-9eac-7adc13e4c254\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-6mgkm"
Apr 24 14:23:59.681282 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:23:59.681172    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/21ff8454-68c7-440e-9994-e8405b47919a-etc-modprobe-d\") pod \"tuned-4kqkj\" (UID: \"21ff8454-68c7-440e-9994-e8405b47919a\") " pod="openshift-cluster-node-tuning-operator/tuned-4kqkj"
Apr 24 14:23:59.681282 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:23:59.681195    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/21ff8454-68c7-440e-9994-e8405b47919a-etc-systemd\") pod \"tuned-4kqkj\" (UID: \"21ff8454-68c7-440e-9994-e8405b47919a\") " pod="openshift-cluster-node-tuning-operator/tuned-4kqkj"
Apr 24 14:23:59.681282 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:23:59.681219    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5x7fm\" (UniqueName: \"kubernetes.io/projected/95f221a8-f16d-4d5f-9638-29db9dd3a969-kube-api-access-5x7fm\") pod \"iptables-alerter-r2pxq\" (UID: \"95f221a8-f16d-4d5f-9638-29db9dd3a969\") " pod="openshift-network-operator/iptables-alerter-r2pxq"
Apr 24 14:23:59.681952 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:23:59.681933    2574 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"multus-daemon-config\""
Apr 24 14:23:59.682020 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:23:59.681953    2574 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"default-dockercfg-vc9gg\""
Apr 24 14:23:59.712106 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:23:59.712078    2574 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-04-23 14:18:58 +0000 UTC" deadline="2028-01-15 03:24:37.33862595 +0000 UTC"
Apr 24 14:23:59.712106 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:23:59.712103    2574 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="15133h0m37.626524778s"
Apr 24 14:23:59.770574 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:23:59.770504    2574 desired_state_of_world_populator.go:158] "Finished populating initial desired state of world"
Apr 24 14:23:59.781611 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:23:59.781585    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/c9814dbf-8db1-4d4a-9d40-d7a0fdc96b26-tuning-conf-dir\") pod \"multus-additional-cni-plugins-j2jc7\" (UID: \"c9814dbf-8db1-4d4a-9d40-d7a0fdc96b26\") " pod="openshift-multus/multus-additional-cni-plugins-j2jc7"
Apr 24 14:23:59.781611 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:23:59.781614    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/44036c3e-1fdc-40d4-899c-9009cd4589e3-run-systemd\") pod \"ovnkube-node-w5x5x\" (UID: \"44036c3e-1fdc-40d4-899c-9009cd4589e3\") " pod="openshift-ovn-kubernetes/ovnkube-node-w5x5x"
Apr 24 14:23:59.781839 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:23:59.781632    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/e87608cb-741d-47f5-bb29-fc68c0dc8bc4-hostroot\") pod \"multus-rwdxf\" (UID: \"e87608cb-741d-47f5-bb29-fc68c0dc8bc4\") " pod="openshift-multus/multus-rwdxf"
Apr 24 14:23:59.781839 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:23:59.781658    2574 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/97869a65-fbd6-4a1b-9eac-7adc13e4c254-kubelet-dir\") pod \"aws-ebs-csi-driver-node-6mgkm\" (UID: \"97869a65-fbd6-4a1b-9eac-7adc13e4c254\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-6mgkm"
Apr 24 14:23:59.781839 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:23:59.781704    2574 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/97869a65-fbd6-4a1b-9eac-7adc13e4c254-sys-fs\") pod \"aws-ebs-csi-driver-node-6mgkm\" (UID: \"97869a65-fbd6-4a1b-9eac-7adc13e4c254\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-6mgkm"
Apr 24 14:23:59.781839 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:23:59.781758    2574 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/21ff8454-68c7-440e-9994-e8405b47919a-lib-modules\") pod \"tuned-4kqkj\" (UID: \"21ff8454-68c7-440e-9994-e8405b47919a\") " pod="openshift-cluster-node-tuning-operator/tuned-4kqkj"
Apr 24 14:23:59.781839 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:23:59.781783    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/97869a65-fbd6-4a1b-9eac-7adc13e4c254-kubelet-dir\") pod \"aws-ebs-csi-driver-node-6mgkm\" (UID: \"97869a65-fbd6-4a1b-9eac-7adc13e4c254\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-6mgkm"
Apr 24 14:23:59.781839 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:23:59.781809    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/97869a65-fbd6-4a1b-9eac-7adc13e4c254-sys-fs\") pod \"aws-ebs-csi-driver-node-6mgkm\" (UID: \"97869a65-fbd6-4a1b-9eac-7adc13e4c254\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-6mgkm"
Apr 24 14:23:59.781839 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:23:59.781821    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-n64fl\" (UniqueName: \"kubernetes.io/projected/e87608cb-741d-47f5-bb29-fc68c0dc8bc4-kube-api-access-n64fl\") pod \"multus-rwdxf\" (UID: \"e87608cb-741d-47f5-bb29-fc68c0dc8bc4\") " pod="openshift-multus/multus-rwdxf"
Apr 24 14:23:59.782164 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:23:59.781879    2574 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/97869a65-fbd6-4a1b-9eac-7adc13e4c254-registration-dir\") pod \"aws-ebs-csi-driver-node-6mgkm\" (UID: \"97869a65-fbd6-4a1b-9eac-7adc13e4c254\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-6mgkm"
Apr 24 14:23:59.782164 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:23:59.781902    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/97869a65-fbd6-4a1b-9eac-7adc13e4c254-registration-dir\") pod \"aws-ebs-csi-driver-node-6mgkm\" (UID: \"97869a65-fbd6-4a1b-9eac-7adc13e4c254\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-6mgkm"
Apr 24 14:23:59.782164 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:23:59.781910    2574 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-css2f\" (UniqueName: \"kubernetes.io/projected/3964207c-9c58-4994-a8a9-aa4e4feca773-kube-api-access-css2f\") pod \"network-check-target-6tsjh\" (UID: \"3964207c-9c58-4994-a8a9-aa4e4feca773\") " pod="openshift-network-diagnostics/network-check-target-6tsjh"
Apr 24 14:23:59.782164 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:23:59.781882    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/21ff8454-68c7-440e-9994-e8405b47919a-lib-modules\") pod \"tuned-4kqkj\" (UID: \"21ff8454-68c7-440e-9994-e8405b47919a\") " pod="openshift-cluster-node-tuning-operator/tuned-4kqkj"
Apr 24 14:23:59.782164 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:23:59.781945    2574 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/21ff8454-68c7-440e-9994-e8405b47919a-etc-kubernetes\") pod \"tuned-4kqkj\" (UID: \"21ff8454-68c7-440e-9994-e8405b47919a\") " pod="openshift-cluster-node-tuning-operator/tuned-4kqkj"
Apr 24 14:23:59.782164 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:23:59.781972    2574 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/95f221a8-f16d-4d5f-9638-29db9dd3a969-host-slash\") pod \"iptables-alerter-r2pxq\" (UID: \"95f221a8-f16d-4d5f-9638-29db9dd3a969\") " pod="openshift-network-operator/iptables-alerter-r2pxq"
Apr 24 14:23:59.782164 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:23:59.781993    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/44036c3e-1fdc-40d4-899c-9009cd4589e3-host-run-ovn-kubernetes\") pod \"ovnkube-node-w5x5x\" (UID: \"44036c3e-1fdc-40d4-899c-9009cd4589e3\") " pod="openshift-ovn-kubernetes/ovnkube-node-w5x5x"
Apr 24 14:23:59.782164 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:23:59.782014    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/21ff8454-68c7-440e-9994-e8405b47919a-etc-kubernetes\") pod \"tuned-4kqkj\" (UID: \"21ff8454-68c7-440e-9994-e8405b47919a\") " pod="openshift-cluster-node-tuning-operator/tuned-4kqkj"
Apr 24 14:23:59.782164 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:23:59.782022    2574 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/21ff8454-68c7-440e-9994-e8405b47919a-etc-sysconfig\") pod \"tuned-4kqkj\" (UID: \"21ff8454-68c7-440e-9994-e8405b47919a\") " pod="openshift-cluster-node-tuning-operator/tuned-4kqkj"
Apr 24 14:23:59.782164 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:23:59.782049    2574 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/21ff8454-68c7-440e-9994-e8405b47919a-sys\") pod \"tuned-4kqkj\" (UID: \"21ff8454-68c7-440e-9994-e8405b47919a\") " pod="openshift-cluster-node-tuning-operator/tuned-4kqkj"
Apr 24 14:23:59.782164 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:23:59.782058    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/95f221a8-f16d-4d5f-9638-29db9dd3a969-host-slash\") pod \"iptables-alerter-r2pxq\" (UID: \"95f221a8-f16d-4d5f-9638-29db9dd3a969\") " pod="openshift-network-operator/iptables-alerter-r2pxq"
Apr 24 14:23:59.782164 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:23:59.782088    2574 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-5js85\" (UniqueName: \"kubernetes.io/projected/21ff8454-68c7-440e-9994-e8405b47919a-kube-api-access-5js85\") pod \"tuned-4kqkj\" (UID: \"21ff8454-68c7-440e-9994-e8405b47919a\") " pod="openshift-cluster-node-tuning-operator/tuned-4kqkj"
Apr 24 14:23:59.782164 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:23:59.782097    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/21ff8454-68c7-440e-9994-e8405b47919a-sys\") pod \"tuned-4kqkj\" (UID: \"21ff8454-68c7-440e-9994-e8405b47919a\") " pod="openshift-cluster-node-tuning-operator/tuned-4kqkj"
Apr 24 14:23:59.782164 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:23:59.782113    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/e87608cb-741d-47f5-bb29-fc68c0dc8bc4-system-cni-dir\") pod \"multus-rwdxf\" (UID: \"e87608cb-741d-47f5-bb29-fc68c0dc8bc4\") " pod="openshift-multus/multus-rwdxf"
Apr 24 14:23:59.782164 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:23:59.782133    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/e87608cb-741d-47f5-bb29-fc68c0dc8bc4-host-run-k8s-cni-cncf-io\") pod \"multus-rwdxf\" (UID: \"e87608cb-741d-47f5-bb29-fc68c0dc8bc4\") " pod="openshift-multus/multus-rwdxf"
Apr 24 14:23:59.782164 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:23:59.782141    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/21ff8454-68c7-440e-9994-e8405b47919a-etc-sysconfig\") pod \"tuned-4kqkj\" (UID: \"21ff8454-68c7-440e-9994-e8405b47919a\") " pod="openshift-cluster-node-tuning-operator/tuned-4kqkj"
Apr 24 14:23:59.782796 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:23:59.782178    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/44036c3e-1fdc-40d4-899c-9009cd4589e3-host-slash\") pod \"ovnkube-node-w5x5x\" (UID: \"44036c3e-1fdc-40d4-899c-9009cd4589e3\") " pod="openshift-ovn-kubernetes/ovnkube-node-w5x5x"
Apr 24 14:23:59.782796 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:23:59.782210    2574 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/21ff8454-68c7-440e-9994-e8405b47919a-var-lib-kubelet\") pod \"tuned-4kqkj\" (UID: \"21ff8454-68c7-440e-9994-e8405b47919a\") " pod="openshift-cluster-node-tuning-operator/tuned-4kqkj"
Apr 24 14:23:59.782796 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:23:59.782235    2574 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/21ff8454-68c7-440e-9994-e8405b47919a-etc-systemd\") pod \"tuned-4kqkj\" (UID: \"21ff8454-68c7-440e-9994-e8405b47919a\") " pod="openshift-cluster-node-tuning-operator/tuned-4kqkj"
Apr 24 14:23:59.782796 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:23:59.782260    2574 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-5x7fm\" (UniqueName: \"kubernetes.io/projected/95f221a8-f16d-4d5f-9638-29db9dd3a969-kube-api-access-5x7fm\") pod \"iptables-alerter-r2pxq\" (UID: \"95f221a8-f16d-4d5f-9638-29db9dd3a969\") " pod="openshift-network-operator/iptables-alerter-r2pxq"
Apr 24 14:23:59.782796 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:23:59.782299    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/21ff8454-68c7-440e-9994-e8405b47919a-var-lib-kubelet\") pod \"tuned-4kqkj\" (UID: \"21ff8454-68c7-440e-9994-e8405b47919a\") " pod="openshift-cluster-node-tuning-operator/tuned-4kqkj"
Apr 24 14:23:59.782796 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:23:59.782308    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/c9814dbf-8db1-4d4a-9d40-d7a0fdc96b26-cni-binary-copy\") pod \"multus-additional-cni-plugins-j2jc7\" (UID: \"c9814dbf-8db1-4d4a-9d40-d7a0fdc96b26\") " pod="openshift-multus/multus-additional-cni-plugins-j2jc7"
Apr 24 14:23:59.782796 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:23:59.782331    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/21ff8454-68c7-440e-9994-e8405b47919a-etc-systemd\") pod \"tuned-4kqkj\" (UID: \"21ff8454-68c7-440e-9994-e8405b47919a\") " pod="openshift-cluster-node-tuning-operator/tuned-4kqkj"
Apr 24 14:23:59.782796 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:23:59.782349    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/e87608cb-741d-47f5-bb29-fc68c0dc8bc4-host-var-lib-kubelet\") pod \"multus-rwdxf\" (UID: \"e87608cb-741d-47f5-bb29-fc68c0dc8bc4\") " pod="openshift-multus/multus-rwdxf"
Apr 24 14:23:59.782796 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:23:59.782378    2574 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/b1844926-4d01-4f6e-8970-a8d662acd7c6-agent-certs\") pod \"konnectivity-agent-zz678\" (UID: \"b1844926-4d01-4f6e-8970-a8d662acd7c6\") " pod="kube-system/konnectivity-agent-zz678"
Apr 24 14:23:59.782796 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:23:59.782454    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-kfrc8\" (UniqueName: \"kubernetes.io/projected/c9814dbf-8db1-4d4a-9d40-d7a0fdc96b26-kube-api-access-kfrc8\") pod \"multus-additional-cni-plugins-j2jc7\" (UID: \"c9814dbf-8db1-4d4a-9d40-d7a0fdc96b26\") " pod="openshift-multus/multus-additional-cni-plugins-j2jc7"
Apr 24 14:23:59.782796 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:23:59.782489    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/44036c3e-1fdc-40d4-899c-9009cd4589e3-run-ovn\") pod \"ovnkube-node-w5x5x\" (UID: \"44036c3e-1fdc-40d4-899c-9009cd4589e3\") " pod="openshift-ovn-kubernetes/ovnkube-node-w5x5x"
Apr 24 14:23:59.782796 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:23:59.782534    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/0b8fb5e7-eaa1-4b51-9f84-37bb0eb1a711-hosts-file\") pod \"node-resolver-bt99l\" (UID: \"0b8fb5e7-eaa1-4b51-9f84-37bb0eb1a711\") " pod="openshift-dns/node-resolver-bt99l"
Apr 24 14:23:59.782796 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:23:59.782577    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/e87608cb-741d-47f5-bb29-fc68c0dc8bc4-multus-socket-dir-parent\") pod \"multus-rwdxf\" (UID: \"e87608cb-741d-47f5-bb29-fc68c0dc8bc4\") " pod="openshift-multus/multus-rwdxf"
Apr 24 14:23:59.782796 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:23:59.782615    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/e87608cb-741d-47f5-bb29-fc68c0dc8bc4-host-var-lib-cni-multus\") pod \"multus-rwdxf\" (UID: \"e87608cb-741d-47f5-bb29-fc68c0dc8bc4\") " pod="openshift-multus/multus-rwdxf"
Apr 24 14:23:59.782796 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:23:59.782642    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/e87608cb-741d-47f5-bb29-fc68c0dc8bc4-host-run-multus-certs\") pod \"multus-rwdxf\" (UID: \"e87608cb-741d-47f5-bb29-fc68c0dc8bc4\") " pod="openshift-multus/multus-rwdxf"
Apr 24 14:23:59.782796 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:23:59.782671    2574 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/b1844926-4d01-4f6e-8970-a8d662acd7c6-konnectivity-ca\") pod \"konnectivity-agent-zz678\" (UID: \"b1844926-4d01-4f6e-8970-a8d662acd7c6\") " pod="kube-system/konnectivity-agent-zz678"
Apr 24 14:23:59.782796 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:23:59.782716    2574 swap_util.go:74] "error creating dir to test if tmpfs noswap is enabled. Assuming not supported" mount path="" error="stat /var/lib/kubelet/plugins/kubernetes.io/empty-dir: no such file or directory"
Apr 24 14:23:59.783481 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:23:59.782727    2574 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/97869a65-fbd6-4a1b-9eac-7adc13e4c254-device-dir\") pod \"aws-ebs-csi-driver-node-6mgkm\" (UID: \"97869a65-fbd6-4a1b-9eac-7adc13e4c254\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-6mgkm"
Apr 24 14:23:59.783481 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:23:59.782786    2574 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-zdh58\" (UniqueName: \"kubernetes.io/projected/97869a65-fbd6-4a1b-9eac-7adc13e4c254-kube-api-access-zdh58\") pod \"aws-ebs-csi-driver-node-6mgkm\" (UID: \"97869a65-fbd6-4a1b-9eac-7adc13e4c254\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-6mgkm"
Apr 24 14:23:59.783481 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:23:59.782799    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/97869a65-fbd6-4a1b-9eac-7adc13e4c254-device-dir\") pod \"aws-ebs-csi-driver-node-6mgkm\" (UID: \"97869a65-fbd6-4a1b-9eac-7adc13e4c254\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-6mgkm"
Apr 24 14:23:59.783481 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:23:59.782816    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/44036c3e-1fdc-40d4-899c-9009cd4589e3-ovnkube-config\") pod \"ovnkube-node-w5x5x\" (UID: \"44036c3e-1fdc-40d4-899c-9009cd4589e3\") " pod="openshift-ovn-kubernetes/ovnkube-node-w5x5x"
Apr 24 14:23:59.783481 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:23:59.782840    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/44036c3e-1fdc-40d4-899c-9009cd4589e3-env-overrides\") pod \"ovnkube-node-w5x5x\" (UID: \"44036c3e-1fdc-40d4-899c-9009cd4589e3\") " pod="openshift-ovn-kubernetes/ovnkube-node-w5x5x"
Apr 24 14:23:59.783481 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:23:59.782863    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/0b8fb5e7-eaa1-4b51-9f84-37bb0eb1a711-tmp-dir\") pod \"node-resolver-bt99l\" (UID: \"0b8fb5e7-eaa1-4b51-9f84-37bb0eb1a711\") " pod="openshift-dns/node-resolver-bt99l"
Apr 24 14:23:59.783481 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:23:59.782896    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/e87608cb-741d-47f5-bb29-fc68c0dc8bc4-multus-daemon-config\") pod \"multus-rwdxf\" (UID: \"e87608cb-741d-47f5-bb29-fc68c0dc8bc4\") " pod="openshift-multus/multus-rwdxf"
Apr 24 14:23:59.783481 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:23:59.782912    2574 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/97869a65-fbd6-4a1b-9eac-7adc13e4c254-etc-selinux\") pod \"aws-ebs-csi-driver-node-6mgkm\" (UID: \"97869a65-fbd6-4a1b-9eac-7adc13e4c254\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-6mgkm"
Apr 24 14:23:59.783481 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:23:59.782931    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/44036c3e-1fdc-40d4-899c-9009cd4589e3-run-openvswitch\") pod \"ovnkube-node-w5x5x\" (UID: \"44036c3e-1fdc-40d4-899c-9009cd4589e3\") " pod="openshift-ovn-kubernetes/ovnkube-node-w5x5x"
Apr 24 14:23:59.783481 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:23:59.782949    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/44036c3e-1fdc-40d4-899c-9009cd4589e3-node-log\") pod \"ovnkube-node-w5x5x\" (UID: \"44036c3e-1fdc-40d4-899c-9009cd4589e3\") " pod="openshift-ovn-kubernetes/ovnkube-node-w5x5x"
Apr 24 14:23:59.783481 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:23:59.782974    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/97869a65-fbd6-4a1b-9eac-7adc13e4c254-etc-selinux\") pod \"aws-ebs-csi-driver-node-6mgkm\" (UID: \"97869a65-fbd6-4a1b-9eac-7adc13e4c254\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-6mgkm"
Apr 24 14:23:59.783481 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:23:59.782978    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/e87608cb-741d-47f5-bb29-fc68c0dc8bc4-multus-conf-dir\") pod \"multus-rwdxf\" (UID: \"e87608cb-741d-47f5-bb29-fc68c0dc8bc4\") " pod="openshift-multus/multus-rwdxf"
Apr 24 14:23:59.783481 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:23:59.783013    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/e87608cb-741d-47f5-bb29-fc68c0dc8bc4-etc-kubernetes\") pod \"multus-rwdxf\" (UID: \"e87608cb-741d-47f5-bb29-fc68c0dc8bc4\") " pod="openshift-multus/multus-rwdxf"
Apr 24 14:23:59.783481 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:23:59.783041    2574 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/21ff8454-68c7-440e-9994-e8405b47919a-etc-tuned\") pod \"tuned-4kqkj\" (UID: \"21ff8454-68c7-440e-9994-e8405b47919a\") " pod="openshift-cluster-node-tuning-operator/tuned-4kqkj"
Apr 24 14:23:59.783481 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:23:59.783108    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/44036c3e-1fdc-40d4-899c-9009cd4589e3-log-socket\") pod \"ovnkube-node-w5x5x\" (UID: \"44036c3e-1fdc-40d4-899c-9009cd4589e3\") " pod="openshift-ovn-kubernetes/ovnkube-node-w5x5x"
Apr 24 14:23:59.783481 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:23:59.783134    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-662k2\" (UniqueName: \"kubernetes.io/projected/44036c3e-1fdc-40d4-899c-9009cd4589e3-kube-api-access-662k2\") pod \"ovnkube-node-w5x5x\" (UID: \"44036c3e-1fdc-40d4-899c-9009cd4589e3\") " pod="openshift-ovn-kubernetes/ovnkube-node-w5x5x"
Apr 24 14:23:59.783481 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:23:59.783161    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/e87608cb-741d-47f5-bb29-fc68c0dc8bc4-cnibin\") pod \"multus-rwdxf\" (UID: \"e87608cb-741d-47f5-bb29-fc68c0dc8bc4\") " pod="openshift-multus/multus-rwdxf"
Apr 24 14:23:59.784240 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:23:59.783175    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/b1844926-4d01-4f6e-8970-a8d662acd7c6-konnectivity-ca\") pod \"konnectivity-agent-zz678\" (UID: \"b1844926-4d01-4f6e-8970-a8d662acd7c6\") " pod="kube-system/konnectivity-agent-zz678"
Apr 24 14:23:59.784240 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:23:59.783191    2574 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/21ff8454-68c7-440e-9994-e8405b47919a-etc-sysctl-conf\") pod \"tuned-4kqkj\" (UID: \"21ff8454-68c7-440e-9994-e8405b47919a\") " pod="openshift-cluster-node-tuning-operator/tuned-4kqkj"
Apr 24 14:23:59.784240 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:23:59.783218    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/c9814dbf-8db1-4d4a-9d40-d7a0fdc96b26-os-release\") pod \"multus-additional-cni-plugins-j2jc7\" (UID: \"c9814dbf-8db1-4d4a-9d40-d7a0fdc96b26\") " pod="openshift-multus/multus-additional-cni-plugins-j2jc7"
Apr 24 14:23:59.784240 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:23:59.783241    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/44036c3e-1fdc-40d4-899c-9009cd4589e3-host-run-netns\") pod \"ovnkube-node-w5x5x\" (UID: \"44036c3e-1fdc-40d4-899c-9009cd4589e3\") " pod="openshift-ovn-kubernetes/ovnkube-node-w5x5x"
Apr 24 14:23:59.784240 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:23:59.783264    2574 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/21ff8454-68c7-440e-9994-e8405b47919a-host\") pod \"tuned-4kqkj\" (UID: \"21ff8454-68c7-440e-9994-e8405b47919a\") " pod="openshift-cluster-node-tuning-operator/tuned-4kqkj"
Apr 24 14:23:59.784240 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:23:59.783286    2574 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/21ff8454-68c7-440e-9994-e8405b47919a-tmp\") pod \"tuned-4kqkj\" (UID: \"21ff8454-68c7-440e-9994-e8405b47919a\") " pod="openshift-cluster-node-tuning-operator/tuned-4kqkj"
Apr 24 14:23:59.784240 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:23:59.783325    2574 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/5e4a7a90-d6ce-491b-ad65-52f0b2a127d6-serviceca\") pod \"node-ca-lgh9n\" (UID: \"5e4a7a90-d6ce-491b-ad65-52f0b2a127d6\") " pod="openshift-image-registry/node-ca-lgh9n"
Apr 24 14:23:59.784240 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:23:59.783347    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/21ff8454-68c7-440e-9994-e8405b47919a-etc-sysctl-conf\") pod \"tuned-4kqkj\" (UID: \"21ff8454-68c7-440e-9994-e8405b47919a\") " pod="openshift-cluster-node-tuning-operator/tuned-4kqkj"
Apr 24 14:23:59.784240 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:23:59.783356    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/21ff8454-68c7-440e-9994-e8405b47919a-host\") pod \"tuned-4kqkj\" (UID: \"21ff8454-68c7-440e-9994-e8405b47919a\") " pod="openshift-cluster-node-tuning-operator/tuned-4kqkj"
Apr 24 14:23:59.784240 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:23:59.783385    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/c9814dbf-8db1-4d4a-9d40-d7a0fdc96b26-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-j2jc7\" (UID: \"c9814dbf-8db1-4d4a-9d40-d7a0fdc96b26\") " pod="openshift-multus/multus-additional-cni-plugins-j2jc7"
Apr 24 14:23:59.784240 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:23:59.783423    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/44036c3e-1fdc-40d4-899c-9009cd4589e3-ovnkube-script-lib\") pod \"ovnkube-node-w5x5x\" (UID: \"44036c3e-1fdc-40d4-899c-9009cd4589e3\") " pod="openshift-ovn-kubernetes/ovnkube-node-w5x5x"
Apr 24 14:23:59.784240 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:23:59.783469    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-w8qz8\" (UniqueName: \"kubernetes.io/projected/0b8fb5e7-eaa1-4b51-9f84-37bb0eb1a711-kube-api-access-w8qz8\") pod \"node-resolver-bt99l\" (UID: \"0b8fb5e7-eaa1-4b51-9f84-37bb0eb1a711\") " pod="openshift-dns/node-resolver-bt99l"
Apr 24 14:23:59.784240 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:23:59.783496    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/e87608cb-741d-47f5-bb29-fc68c0dc8bc4-cni-binary-copy\") pod \"multus-rwdxf\" (UID: \"e87608cb-741d-47f5-bb29-fc68c0dc8bc4\") " pod="openshift-multus/multus-rwdxf"
Apr 24 14:23:59.784240 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:23:59.783523    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/44036c3e-1fdc-40d4-899c-9009cd4589e3-etc-openvswitch\") pod \"ovnkube-node-w5x5x\" (UID: \"44036c3e-1fdc-40d4-899c-9009cd4589e3\") " pod="openshift-ovn-kubernetes/ovnkube-node-w5x5x"
Apr 24 14:23:59.784240 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:23:59.783546    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/44036c3e-1fdc-40d4-899c-9009cd4589e3-host-cni-bin\") pod \"ovnkube-node-w5x5x\" (UID: \"44036c3e-1fdc-40d4-899c-9009cd4589e3\") " pod="openshift-ovn-kubernetes/ovnkube-node-w5x5x"
Apr 24 14:23:59.784240 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:23:59.783587    2574 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-c6kpn\" (UniqueName: \"kubernetes.io/projected/3bc44c73-963f-4a9e-8a52-1e8b031a08dc-kube-api-access-c6kpn\") pod \"network-metrics-daemon-zhkmq\" (UID: \"3bc44c73-963f-4a9e-8a52-1e8b031a08dc\") " pod="openshift-multus/network-metrics-daemon-zhkmq"
Apr 24 14:23:59.784240 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:23:59.783620    2574 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/97869a65-fbd6-4a1b-9eac-7adc13e4c254-socket-dir\") pod \"aws-ebs-csi-driver-node-6mgkm\" (UID: \"97869a65-fbd6-4a1b-9eac-7adc13e4c254\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-6mgkm"
Apr 24 14:23:59.785066 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:23:59.783645    2574 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/21ff8454-68c7-440e-9994-e8405b47919a-etc-modprobe-d\") pod \"tuned-4kqkj\" (UID: \"21ff8454-68c7-440e-9994-e8405b47919a\") " pod="openshift-cluster-node-tuning-operator/tuned-4kqkj"
Apr 24 14:23:59.785066 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:23:59.783671    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/44036c3e-1fdc-40d4-899c-9009cd4589e3-host-kubelet\") pod \"ovnkube-node-w5x5x\" (UID: \"44036c3e-1fdc-40d4-899c-9009cd4589e3\") " pod="openshift-ovn-kubernetes/ovnkube-node-w5x5x"
Apr 24 14:23:59.785066 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:23:59.783696    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/44036c3e-1fdc-40d4-899c-9009cd4589e3-systemd-units\") pod \"ovnkube-node-w5x5x\" (UID: \"44036c3e-1fdc-40d4-899c-9009cd4589e3\") " pod="openshift-ovn-kubernetes/ovnkube-node-w5x5x"
Apr 24 14:23:59.785066 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:23:59.783724    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/44036c3e-1fdc-40d4-899c-9009cd4589e3-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-w5x5x\" (UID: \"44036c3e-1fdc-40d4-899c-9009cd4589e3\") " pod="openshift-ovn-kubernetes/ovnkube-node-w5x5x"
Apr 24 14:23:59.785066 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:23:59.783741    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/5e4a7a90-d6ce-491b-ad65-52f0b2a127d6-serviceca\") pod \"node-ca-lgh9n\" (UID: \"5e4a7a90-d6ce-491b-ad65-52f0b2a127d6\") " pod="openshift-image-registry/node-ca-lgh9n"
Apr 24 14:23:59.785066 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:23:59.783750    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/e87608cb-741d-47f5-bb29-fc68c0dc8bc4-host-var-lib-cni-bin\") pod \"multus-rwdxf\" (UID: \"e87608cb-741d-47f5-bb29-fc68c0dc8bc4\") " pod="openshift-multus/multus-rwdxf"
Apr 24 14:23:59.785066 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:23:59.783788    2574 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/3bc44c73-963f-4a9e-8a52-1e8b031a08dc-metrics-certs\") pod \"network-metrics-daemon-zhkmq\" (UID: \"3bc44c73-963f-4a9e-8a52-1e8b031a08dc\") " pod="openshift-multus/network-metrics-daemon-zhkmq"
Apr 24 14:23:59.785066 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:23:59.783845    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/97869a65-fbd6-4a1b-9eac-7adc13e4c254-socket-dir\") pod \"aws-ebs-csi-driver-node-6mgkm\" (UID: \"97869a65-fbd6-4a1b-9eac-7adc13e4c254\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-6mgkm"
Apr 24 14:23:59.785066 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:23:59.783871    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/21ff8454-68c7-440e-9994-e8405b47919a-etc-modprobe-d\") pod \"tuned-4kqkj\" (UID: \"21ff8454-68c7-440e-9994-e8405b47919a\") " pod="openshift-cluster-node-tuning-operator/tuned-4kqkj"
Apr 24 14:23:59.785066 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:23:59.783879    2574 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/21ff8454-68c7-440e-9994-e8405b47919a-run\") pod \"tuned-4kqkj\" (UID: \"21ff8454-68c7-440e-9994-e8405b47919a\") " pod="openshift-cluster-node-tuning-operator/tuned-4kqkj"
Apr 24 14:23:59.785066 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:23:59.783908    2574 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-m9qwz\" (UniqueName: \"kubernetes.io/projected/5e4a7a90-d6ce-491b-ad65-52f0b2a127d6-kube-api-access-m9qwz\") pod \"node-ca-lgh9n\" (UID: \"5e4a7a90-d6ce-491b-ad65-52f0b2a127d6\") " pod="openshift-image-registry/node-ca-lgh9n"
Apr 24 14:23:59.785066 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:23:59.783957    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run\" (UniqueName: \"kubernetes.io/host-path/21ff8454-68c7-440e-9994-e8405b47919a-run\") pod \"tuned-4kqkj\" (UID: \"21ff8454-68c7-440e-9994-e8405b47919a\") " pod="openshift-cluster-node-tuning-operator/tuned-4kqkj"
Apr 24 14:23:59.785066 ip-10-0-136-132 kubenswrapper[2574]: E0424 14:23:59.783970    2574 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Apr 24 14:23:59.785066 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:23:59.784002    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/c9814dbf-8db1-4d4a-9d40-d7a0fdc96b26-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-j2jc7\" (UID: \"c9814dbf-8db1-4d4a-9d40-d7a0fdc96b26\") " pod="openshift-multus/multus-additional-cni-plugins-j2jc7"
Apr 24 14:23:59.785066 ip-10-0-136-132 kubenswrapper[2574]: E0424 14:23:59.784077    2574 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/3bc44c73-963f-4a9e-8a52-1e8b031a08dc-metrics-certs podName:3bc44c73-963f-4a9e-8a52-1e8b031a08dc nodeName:}" failed. No retries permitted until 2026-04-24 14:24:00.284018889 +0000 UTC m=+3.016120537 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/3bc44c73-963f-4a9e-8a52-1e8b031a08dc-metrics-certs") pod "network-metrics-daemon-zhkmq" (UID: "3bc44c73-963f-4a9e-8a52-1e8b031a08dc") : object "openshift-multus"/"metrics-daemon-secret" not registered
Apr 24 14:23:59.785066 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:23:59.784186    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/44036c3e-1fdc-40d4-899c-9009cd4589e3-var-lib-openvswitch\") pod \"ovnkube-node-w5x5x\" (UID: \"44036c3e-1fdc-40d4-899c-9009cd4589e3\") " pod="openshift-ovn-kubernetes/ovnkube-node-w5x5x"
Apr 24 14:23:59.785066 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:23:59.784217    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/44036c3e-1fdc-40d4-899c-9009cd4589e3-ovn-node-metrics-cert\") pod \"ovnkube-node-w5x5x\" (UID: \"44036c3e-1fdc-40d4-899c-9009cd4589e3\") " pod="openshift-ovn-kubernetes/ovnkube-node-w5x5x"
Apr 24 14:23:59.785818 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:23:59.784251    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/e87608cb-741d-47f5-bb29-fc68c0dc8bc4-multus-cni-dir\") pod \"multus-rwdxf\" (UID: \"e87608cb-741d-47f5-bb29-fc68c0dc8bc4\") " pod="openshift-multus/multus-rwdxf"
Apr 24 14:23:59.785818 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:23:59.784288    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/e87608cb-741d-47f5-bb29-fc68c0dc8bc4-host-run-netns\") pod \"multus-rwdxf\" (UID: \"e87608cb-741d-47f5-bb29-fc68c0dc8bc4\") " pod="openshift-multus/multus-rwdxf"
Apr 24 14:23:59.785818 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:23:59.784328    2574 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/5e4a7a90-d6ce-491b-ad65-52f0b2a127d6-host\") pod \"node-ca-lgh9n\" (UID: \"5e4a7a90-d6ce-491b-ad65-52f0b2a127d6\") " pod="openshift-image-registry/node-ca-lgh9n"
Apr 24 14:23:59.785818 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:23:59.784375    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/c9814dbf-8db1-4d4a-9d40-d7a0fdc96b26-system-cni-dir\") pod \"multus-additional-cni-plugins-j2jc7\" (UID: \"c9814dbf-8db1-4d4a-9d40-d7a0fdc96b26\") " pod="openshift-multus/multus-additional-cni-plugins-j2jc7"
Apr 24 14:23:59.785818 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:23:59.784411    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/c9814dbf-8db1-4d4a-9d40-d7a0fdc96b26-cnibin\") pod \"multus-additional-cni-plugins-j2jc7\" (UID: \"c9814dbf-8db1-4d4a-9d40-d7a0fdc96b26\") " pod="openshift-multus/multus-additional-cni-plugins-j2jc7"
Apr 24 14:23:59.785818 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:23:59.784464    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/5e4a7a90-d6ce-491b-ad65-52f0b2a127d6-host\") pod \"node-ca-lgh9n\" (UID: \"5e4a7a90-d6ce-491b-ad65-52f0b2a127d6\") " pod="openshift-image-registry/node-ca-lgh9n"
Apr 24 14:23:59.785818 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:23:59.784500    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/44036c3e-1fdc-40d4-899c-9009cd4589e3-host-cni-netd\") pod \"ovnkube-node-w5x5x\" (UID: \"44036c3e-1fdc-40d4-899c-9009cd4589e3\") " pod="openshift-ovn-kubernetes/ovnkube-node-w5x5x"
Apr 24 14:23:59.785818 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:23:59.784533    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/e87608cb-741d-47f5-bb29-fc68c0dc8bc4-os-release\") pod \"multus-rwdxf\" (UID: \"e87608cb-741d-47f5-bb29-fc68c0dc8bc4\") " pod="openshift-multus/multus-rwdxf"
Apr 24 14:23:59.785818 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:23:59.784574    2574 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/21ff8454-68c7-440e-9994-e8405b47919a-etc-sysctl-d\") pod \"tuned-4kqkj\" (UID: \"21ff8454-68c7-440e-9994-e8405b47919a\") " pod="openshift-cluster-node-tuning-operator/tuned-4kqkj"
Apr 24 14:23:59.785818 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:23:59.784634    2574 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/95f221a8-f16d-4d5f-9638-29db9dd3a969-iptables-alerter-script\") pod \"iptables-alerter-r2pxq\" (UID: \"95f221a8-f16d-4d5f-9638-29db9dd3a969\") " pod="openshift-network-operator/iptables-alerter-r2pxq"
Apr 24 14:23:59.785818 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:23:59.784683    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/21ff8454-68c7-440e-9994-e8405b47919a-etc-sysctl-d\") pod \"tuned-4kqkj\" (UID: \"21ff8454-68c7-440e-9994-e8405b47919a\") " pod="openshift-cluster-node-tuning-operator/tuned-4kqkj"
Apr 24 14:23:59.785818 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:23:59.785111    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/95f221a8-f16d-4d5f-9638-29db9dd3a969-iptables-alerter-script\") pod \"iptables-alerter-r2pxq\" (UID: \"95f221a8-f16d-4d5f-9638-29db9dd3a969\") " pod="openshift-network-operator/iptables-alerter-r2pxq"
Apr 24 14:23:59.786184 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:23:59.785842    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/21ff8454-68c7-440e-9994-e8405b47919a-etc-tuned\") pod \"tuned-4kqkj\" (UID: \"21ff8454-68c7-440e-9994-e8405b47919a\") " pod="openshift-cluster-node-tuning-operator/tuned-4kqkj"
Apr 24 14:23:59.786184 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:23:59.785976    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/21ff8454-68c7-440e-9994-e8405b47919a-tmp\") pod \"tuned-4kqkj\" (UID: \"21ff8454-68c7-440e-9994-e8405b47919a\") " pod="openshift-cluster-node-tuning-operator/tuned-4kqkj"
Apr 24 14:23:59.786184 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:23:59.786150    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/b1844926-4d01-4f6e-8970-a8d662acd7c6-agent-certs\") pod \"konnectivity-agent-zz678\" (UID: \"b1844926-4d01-4f6e-8970-a8d662acd7c6\") " pod="kube-system/konnectivity-agent-zz678"
Apr 24 14:23:59.786498 ip-10-0-136-132 kubenswrapper[2574]: E0424 14:23:59.786481    2574 projected.go:289] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Apr 24 14:23:59.786584 ip-10-0-136-132 kubenswrapper[2574]: E0424 14:23:59.786503    2574 projected.go:289] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Apr 24 14:23:59.786584 ip-10-0-136-132 kubenswrapper[2574]: E0424 14:23:59.786517    2574 projected.go:194] Error preparing data for projected volume kube-api-access-css2f for pod openshift-network-diagnostics/network-check-target-6tsjh: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Apr 24 14:23:59.786584 ip-10-0-136-132 kubenswrapper[2574]: E0424 14:23:59.786567    2574 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/3964207c-9c58-4994-a8a9-aa4e4feca773-kube-api-access-css2f podName:3964207c-9c58-4994-a8a9-aa4e4feca773 nodeName:}" failed. No retries permitted until 2026-04-24 14:24:00.286555921 +0000 UTC m=+3.018657553 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-api-access-css2f" (UniqueName: "kubernetes.io/projected/3964207c-9c58-4994-a8a9-aa4e4feca773-kube-api-access-css2f") pod "network-check-target-6tsjh" (UID: "3964207c-9c58-4994-a8a9-aa4e4feca773") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Apr 24 14:23:59.789558 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:23:59.789445    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-5js85\" (UniqueName: \"kubernetes.io/projected/21ff8454-68c7-440e-9994-e8405b47919a-kube-api-access-5js85\") pod \"tuned-4kqkj\" (UID: \"21ff8454-68c7-440e-9994-e8405b47919a\") " pod="openshift-cluster-node-tuning-operator/tuned-4kqkj"
Apr 24 14:23:59.789558 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:23:59.789460    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-5x7fm\" (UniqueName: \"kubernetes.io/projected/95f221a8-f16d-4d5f-9638-29db9dd3a969-kube-api-access-5x7fm\") pod \"iptables-alerter-r2pxq\" (UID: \"95f221a8-f16d-4d5f-9638-29db9dd3a969\") " pod="openshift-network-operator/iptables-alerter-r2pxq"
Apr 24 14:23:59.789719 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:23:59.789598    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-zdh58\" (UniqueName: \"kubernetes.io/projected/97869a65-fbd6-4a1b-9eac-7adc13e4c254-kube-api-access-zdh58\") pod \"aws-ebs-csi-driver-node-6mgkm\" (UID: \"97869a65-fbd6-4a1b-9eac-7adc13e4c254\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-6mgkm"
Apr 24 14:23:59.789976 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:23:59.789954    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-c6kpn\" (UniqueName: \"kubernetes.io/projected/3bc44c73-963f-4a9e-8a52-1e8b031a08dc-kube-api-access-c6kpn\") pod \"network-metrics-daemon-zhkmq\" (UID: \"3bc44c73-963f-4a9e-8a52-1e8b031a08dc\") " pod="openshift-multus/network-metrics-daemon-zhkmq"
Apr 24 14:23:59.793171 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:23:59.793141    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-m9qwz\" (UniqueName: \"kubernetes.io/projected/5e4a7a90-d6ce-491b-ad65-52f0b2a127d6-kube-api-access-m9qwz\") pod \"node-ca-lgh9n\" (UID: \"5e4a7a90-d6ce-491b-ad65-52f0b2a127d6\") " pod="openshift-image-registry/node-ca-lgh9n"
Apr 24 14:23:59.885912 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:23:59.885878    2574 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/c9814dbf-8db1-4d4a-9d40-d7a0fdc96b26-os-release\") pod \"multus-additional-cni-plugins-j2jc7\" (UID: \"c9814dbf-8db1-4d4a-9d40-d7a0fdc96b26\") " pod="openshift-multus/multus-additional-cni-plugins-j2jc7"
Apr 24 14:23:59.886066 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:23:59.885927    2574 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/44036c3e-1fdc-40d4-899c-9009cd4589e3-host-run-netns\") pod \"ovnkube-node-w5x5x\" (UID: \"44036c3e-1fdc-40d4-899c-9009cd4589e3\") " pod="openshift-ovn-kubernetes/ovnkube-node-w5x5x"
Apr 24 14:23:59.886066 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:23:59.885962    2574 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/c9814dbf-8db1-4d4a-9d40-d7a0fdc96b26-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-j2jc7\" (UID: \"c9814dbf-8db1-4d4a-9d40-d7a0fdc96b26\") " pod="openshift-multus/multus-additional-cni-plugins-j2jc7"
Apr 24 14:23:59.886066 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:23:59.885979    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/c9814dbf-8db1-4d4a-9d40-d7a0fdc96b26-os-release\") pod \"multus-additional-cni-plugins-j2jc7\" (UID: \"c9814dbf-8db1-4d4a-9d40-d7a0fdc96b26\") " pod="openshift-multus/multus-additional-cni-plugins-j2jc7"
Apr 24 14:23:59.886066 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:23:59.885992    2574 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/44036c3e-1fdc-40d4-899c-9009cd4589e3-ovnkube-script-lib\") pod \"ovnkube-node-w5x5x\" (UID: \"44036c3e-1fdc-40d4-899c-9009cd4589e3\") " pod="openshift-ovn-kubernetes/ovnkube-node-w5x5x"
Apr 24 14:23:59.886066 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:23:59.886030    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/44036c3e-1fdc-40d4-899c-9009cd4589e3-host-run-netns\") pod \"ovnkube-node-w5x5x\" (UID: \"44036c3e-1fdc-40d4-899c-9009cd4589e3\") " pod="openshift-ovn-kubernetes/ovnkube-node-w5x5x"
Apr 24 14:23:59.886258 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:23:59.886094    2574 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-w8qz8\" (UniqueName: \"kubernetes.io/projected/0b8fb5e7-eaa1-4b51-9f84-37bb0eb1a711-kube-api-access-w8qz8\") pod \"node-resolver-bt99l\" (UID: \"0b8fb5e7-eaa1-4b51-9f84-37bb0eb1a711\") " pod="openshift-dns/node-resolver-bt99l"
Apr 24 14:23:59.886258 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:23:59.886131    2574 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/e87608cb-741d-47f5-bb29-fc68c0dc8bc4-cni-binary-copy\") pod \"multus-rwdxf\" (UID: \"e87608cb-741d-47f5-bb29-fc68c0dc8bc4\") " pod="openshift-multus/multus-rwdxf"
Apr 24 14:23:59.886258 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:23:59.886156    2574 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/44036c3e-1fdc-40d4-899c-9009cd4589e3-etc-openvswitch\") pod \"ovnkube-node-w5x5x\" (UID: \"44036c3e-1fdc-40d4-899c-9009cd4589e3\") " pod="openshift-ovn-kubernetes/ovnkube-node-w5x5x"
Apr 24 14:23:59.886258 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:23:59.886183    2574 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/44036c3e-1fdc-40d4-899c-9009cd4589e3-host-cni-bin\") pod \"ovnkube-node-w5x5x\" (UID: \"44036c3e-1fdc-40d4-899c-9009cd4589e3\") " pod="openshift-ovn-kubernetes/ovnkube-node-w5x5x"
Apr 24 14:23:59.886258 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:23:59.886211    2574 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/44036c3e-1fdc-40d4-899c-9009cd4589e3-host-kubelet\") pod \"ovnkube-node-w5x5x\" (UID: \"44036c3e-1fdc-40d4-899c-9009cd4589e3\") " pod="openshift-ovn-kubernetes/ovnkube-node-w5x5x"
Apr 24 14:23:59.886258 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:23:59.886234    2574 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/44036c3e-1fdc-40d4-899c-9009cd4589e3-systemd-units\") pod \"ovnkube-node-w5x5x\" (UID: \"44036c3e-1fdc-40d4-899c-9009cd4589e3\") " pod="openshift-ovn-kubernetes/ovnkube-node-w5x5x"
Apr 24 14:23:59.886528 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:23:59.886259    2574 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/44036c3e-1fdc-40d4-899c-9009cd4589e3-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-w5x5x\" (UID: \"44036c3e-1fdc-40d4-899c-9009cd4589e3\") " pod="openshift-ovn-kubernetes/ovnkube-node-w5x5x"
Apr 24 14:23:59.886528 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:23:59.886288    2574 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/e87608cb-741d-47f5-bb29-fc68c0dc8bc4-host-var-lib-cni-bin\") pod \"multus-rwdxf\" (UID: \"e87608cb-741d-47f5-bb29-fc68c0dc8bc4\") " pod="openshift-multus/multus-rwdxf"
Apr 24 14:23:59.886528 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:23:59.886331    2574 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/c9814dbf-8db1-4d4a-9d40-d7a0fdc96b26-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-j2jc7\" (UID: \"c9814dbf-8db1-4d4a-9d40-d7a0fdc96b26\") " pod="openshift-multus/multus-additional-cni-plugins-j2jc7"
Apr 24 14:23:59.886528 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:23:59.886373    2574 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/44036c3e-1fdc-40d4-899c-9009cd4589e3-var-lib-openvswitch\") pod \"ovnkube-node-w5x5x\" (UID: \"44036c3e-1fdc-40d4-899c-9009cd4589e3\") " pod="openshift-ovn-kubernetes/ovnkube-node-w5x5x"
Apr 24 14:23:59.886528 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:23:59.886398    2574 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/44036c3e-1fdc-40d4-899c-9009cd4589e3-ovn-node-metrics-cert\") pod \"ovnkube-node-w5x5x\" (UID: \"44036c3e-1fdc-40d4-899c-9009cd4589e3\") " pod="openshift-ovn-kubernetes/ovnkube-node-w5x5x"
Apr 24 14:23:59.886528 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:23:59.886422    2574 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/e87608cb-741d-47f5-bb29-fc68c0dc8bc4-multus-cni-dir\") pod \"multus-rwdxf\" (UID: \"e87608cb-741d-47f5-bb29-fc68c0dc8bc4\") " pod="openshift-multus/multus-rwdxf"
Apr 24 14:23:59.886528 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:23:59.886439    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/44036c3e-1fdc-40d4-899c-9009cd4589e3-systemd-units\") pod \"ovnkube-node-w5x5x\" (UID: \"44036c3e-1fdc-40d4-899c-9009cd4589e3\") " pod="openshift-ovn-kubernetes/ovnkube-node-w5x5x"
Apr 24 14:23:59.886528 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:23:59.886448    2574 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/e87608cb-741d-47f5-bb29-fc68c0dc8bc4-host-run-netns\") pod \"multus-rwdxf\" (UID: \"e87608cb-741d-47f5-bb29-fc68c0dc8bc4\") " pod="openshift-multus/multus-rwdxf"
Apr 24 14:23:59.886528 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:23:59.886489    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/44036c3e-1fdc-40d4-899c-9009cd4589e3-host-cni-bin\") pod \"ovnkube-node-w5x5x\" (UID: \"44036c3e-1fdc-40d4-899c-9009cd4589e3\") " pod="openshift-ovn-kubernetes/ovnkube-node-w5x5x"
Apr 24 14:23:59.886528 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:23:59.886495    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/44036c3e-1fdc-40d4-899c-9009cd4589e3-etc-openvswitch\") pod \"ovnkube-node-w5x5x\" (UID: \"44036c3e-1fdc-40d4-899c-9009cd4589e3\") " pod="openshift-ovn-kubernetes/ovnkube-node-w5x5x"
Apr 24 14:23:59.886528 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:23:59.886523    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/44036c3e-1fdc-40d4-899c-9009cd4589e3-host-kubelet\") pod \"ovnkube-node-w5x5x\" (UID: \"44036c3e-1fdc-40d4-899c-9009cd4589e3\") " pod="openshift-ovn-kubernetes/ovnkube-node-w5x5x"
Apr 24 14:23:59.887055 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:23:59.886552    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/e87608cb-741d-47f5-bb29-fc68c0dc8bc4-multus-cni-dir\") pod \"multus-rwdxf\" (UID: \"e87608cb-741d-47f5-bb29-fc68c0dc8bc4\") " pod="openshift-multus/multus-rwdxf"
Apr 24 14:23:59.887055 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:23:59.886568    2574 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/c9814dbf-8db1-4d4a-9d40-d7a0fdc96b26-system-cni-dir\") pod \"multus-additional-cni-plugins-j2jc7\" (UID: \"c9814dbf-8db1-4d4a-9d40-d7a0fdc96b26\") " pod="openshift-multus/multus-additional-cni-plugins-j2jc7"
Apr 24 14:23:59.887055 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:23:59.886571    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/c9814dbf-8db1-4d4a-9d40-d7a0fdc96b26-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-j2jc7\" (UID: \"c9814dbf-8db1-4d4a-9d40-d7a0fdc96b26\") " pod="openshift-multus/multus-additional-cni-plugins-j2jc7"
Apr 24 14:23:59.887055 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:23:59.886592    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/e87608cb-741d-47f5-bb29-fc68c0dc8bc4-host-var-lib-cni-bin\") pod \"multus-rwdxf\" (UID: \"e87608cb-741d-47f5-bb29-fc68c0dc8bc4\") " pod="openshift-multus/multus-rwdxf"
Apr 24 14:23:59.887055 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:23:59.886633    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/44036c3e-1fdc-40d4-899c-9009cd4589e3-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-w5x5x\" (UID: \"44036c3e-1fdc-40d4-899c-9009cd4589e3\") " pod="openshift-ovn-kubernetes/ovnkube-node-w5x5x"
Apr 24 14:23:59.887055 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:23:59.886690    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/e87608cb-741d-47f5-bb29-fc68c0dc8bc4-host-run-netns\") pod \"multus-rwdxf\" (UID: \"e87608cb-741d-47f5-bb29-fc68c0dc8bc4\") " pod="openshift-multus/multus-rwdxf"
Apr 24 14:23:59.887055 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:23:59.886691    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/44036c3e-1fdc-40d4-899c-9009cd4589e3-var-lib-openvswitch\") pod \"ovnkube-node-w5x5x\" (UID: \"44036c3e-1fdc-40d4-899c-9009cd4589e3\") " pod="openshift-ovn-kubernetes/ovnkube-node-w5x5x"
Apr 24 14:23:59.887055 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:23:59.886720    2574 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/c9814dbf-8db1-4d4a-9d40-d7a0fdc96b26-cnibin\") pod \"multus-additional-cni-plugins-j2jc7\" (UID: \"c9814dbf-8db1-4d4a-9d40-d7a0fdc96b26\") " pod="openshift-multus/multus-additional-cni-plugins-j2jc7"
Apr 24 14:23:59.887055 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:23:59.886740    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/c9814dbf-8db1-4d4a-9d40-d7a0fdc96b26-system-cni-dir\") pod \"multus-additional-cni-plugins-j2jc7\" (UID: \"c9814dbf-8db1-4d4a-9d40-d7a0fdc96b26\") " pod="openshift-multus/multus-additional-cni-plugins-j2jc7"
Apr 24 14:23:59.887055 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:23:59.886788    2574 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/44036c3e-1fdc-40d4-899c-9009cd4589e3-host-cni-netd\") pod \"ovnkube-node-w5x5x\" (UID: \"44036c3e-1fdc-40d4-899c-9009cd4589e3\") " pod="openshift-ovn-kubernetes/ovnkube-node-w5x5x"
Apr 24 14:23:59.887055 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:23:59.886815    2574 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/e87608cb-741d-47f5-bb29-fc68c0dc8bc4-os-release\") pod \"multus-rwdxf\" (UID: \"e87608cb-741d-47f5-bb29-fc68c0dc8bc4\") " pod="openshift-multus/multus-rwdxf"
Apr 24 14:23:59.887055 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:23:59.886847    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/44036c3e-1fdc-40d4-899c-9009cd4589e3-host-cni-netd\") pod \"ovnkube-node-w5x5x\" (UID: \"44036c3e-1fdc-40d4-899c-9009cd4589e3\") " pod="openshift-ovn-kubernetes/ovnkube-node-w5x5x"
Apr 24 14:23:59.887055 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:23:59.886875    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/e87608cb-741d-47f5-bb29-fc68c0dc8bc4-os-release\") pod \"multus-rwdxf\" (UID: \"e87608cb-741d-47f5-bb29-fc68c0dc8bc4\") " pod="openshift-multus/multus-rwdxf"
Apr 24 14:23:59.887055 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:23:59.886898    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/c9814dbf-8db1-4d4a-9d40-d7a0fdc96b26-cnibin\") pod \"multus-additional-cni-plugins-j2jc7\" (UID: \"c9814dbf-8db1-4d4a-9d40-d7a0fdc96b26\") " pod="openshift-multus/multus-additional-cni-plugins-j2jc7"
Apr 24 14:23:59.887055 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:23:59.886948    2574 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/c9814dbf-8db1-4d4a-9d40-d7a0fdc96b26-tuning-conf-dir\") pod \"multus-additional-cni-plugins-j2jc7\" (UID: \"c9814dbf-8db1-4d4a-9d40-d7a0fdc96b26\") " pod="openshift-multus/multus-additional-cni-plugins-j2jc7"
Apr 24 14:23:59.887055 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:23:59.886987    2574 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/44036c3e-1fdc-40d4-899c-9009cd4589e3-run-systemd\") pod \"ovnkube-node-w5x5x\" (UID: \"44036c3e-1fdc-40d4-899c-9009cd4589e3\") " pod="openshift-ovn-kubernetes/ovnkube-node-w5x5x"
Apr 24 14:23:59.887055 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:23:59.887023    2574 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/e87608cb-741d-47f5-bb29-fc68c0dc8bc4-hostroot\") pod \"multus-rwdxf\" (UID: \"e87608cb-741d-47f5-bb29-fc68c0dc8bc4\") " pod="openshift-multus/multus-rwdxf"
Apr 24 14:23:59.887890 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:23:59.887064    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/44036c3e-1fdc-40d4-899c-9009cd4589e3-run-systemd\") pod \"ovnkube-node-w5x5x\" (UID: \"44036c3e-1fdc-40d4-899c-9009cd4589e3\") " pod="openshift-ovn-kubernetes/ovnkube-node-w5x5x"
Apr 24 14:23:59.887890 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:23:59.887072    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/c9814dbf-8db1-4d4a-9d40-d7a0fdc96b26-tuning-conf-dir\") pod \"multus-additional-cni-plugins-j2jc7\" (UID: \"c9814dbf-8db1-4d4a-9d40-d7a0fdc96b26\") " pod="openshift-multus/multus-additional-cni-plugins-j2jc7"
Apr 24 14:23:59.887890 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:23:59.887081    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/44036c3e-1fdc-40d4-899c-9009cd4589e3-ovnkube-script-lib\") pod \"ovnkube-node-w5x5x\" (UID: \"44036c3e-1fdc-40d4-899c-9009cd4589e3\") " pod="openshift-ovn-kubernetes/ovnkube-node-w5x5x"
Apr 24 14:23:59.887890 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:23:59.887098    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/e87608cb-741d-47f5-bb29-fc68c0dc8bc4-hostroot\") pod \"multus-rwdxf\" (UID: \"e87608cb-741d-47f5-bb29-fc68c0dc8bc4\") " pod="openshift-multus/multus-rwdxf"
Apr 24 14:23:59.887890 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:23:59.887138    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/c9814dbf-8db1-4d4a-9d40-d7a0fdc96b26-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-j2jc7\" (UID: \"c9814dbf-8db1-4d4a-9d40-d7a0fdc96b26\") " pod="openshift-multus/multus-additional-cni-plugins-j2jc7"
Apr 24 14:23:59.887890 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:23:59.887153    2574 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-n64fl\" (UniqueName: \"kubernetes.io/projected/e87608cb-741d-47f5-bb29-fc68c0dc8bc4-kube-api-access-n64fl\") pod \"multus-rwdxf\" (UID: \"e87608cb-741d-47f5-bb29-fc68c0dc8bc4\") " pod="openshift-multus/multus-rwdxf"
Apr 24 14:23:59.887890 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:23:59.887181    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/e87608cb-741d-47f5-bb29-fc68c0dc8bc4-cni-binary-copy\") pod \"multus-rwdxf\" (UID: \"e87608cb-741d-47f5-bb29-fc68c0dc8bc4\") " pod="openshift-multus/multus-rwdxf"
Apr 24 14:23:59.887890 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:23:59.887212    2574 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/44036c3e-1fdc-40d4-899c-9009cd4589e3-host-run-ovn-kubernetes\") pod \"ovnkube-node-w5x5x\" (UID: \"44036c3e-1fdc-40d4-899c-9009cd4589e3\") " pod="openshift-ovn-kubernetes/ovnkube-node-w5x5x"
Apr 24 14:23:59.887890 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:23:59.887246    2574 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/e87608cb-741d-47f5-bb29-fc68c0dc8bc4-system-cni-dir\") pod \"multus-rwdxf\" (UID: \"e87608cb-741d-47f5-bb29-fc68c0dc8bc4\") " pod="openshift-multus/multus-rwdxf"
Apr 24 14:23:59.887890 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:23:59.887270    2574 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/e87608cb-741d-47f5-bb29-fc68c0dc8bc4-host-run-k8s-cni-cncf-io\") pod \"multus-rwdxf\" (UID: \"e87608cb-741d-47f5-bb29-fc68c0dc8bc4\") " pod="openshift-multus/multus-rwdxf"
Apr 24 14:23:59.887890 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:23:59.887294    2574 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/44036c3e-1fdc-40d4-899c-9009cd4589e3-host-slash\") pod \"ovnkube-node-w5x5x\" (UID: \"44036c3e-1fdc-40d4-899c-9009cd4589e3\") " pod="openshift-ovn-kubernetes/ovnkube-node-w5x5x"
Apr 24 14:23:59.887890 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:23:59.887303    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/44036c3e-1fdc-40d4-899c-9009cd4589e3-host-run-ovn-kubernetes\") pod \"ovnkube-node-w5x5x\" (UID: \"44036c3e-1fdc-40d4-899c-9009cd4589e3\") " pod="openshift-ovn-kubernetes/ovnkube-node-w5x5x"
Apr 24 14:23:59.887890 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:23:59.887308    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/e87608cb-741d-47f5-bb29-fc68c0dc8bc4-system-cni-dir\") pod \"multus-rwdxf\" (UID: \"e87608cb-741d-47f5-bb29-fc68c0dc8bc4\") " pod="openshift-multus/multus-rwdxf"
Apr 24 14:23:59.887890 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:23:59.887321    2574 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/c9814dbf-8db1-4d4a-9d40-d7a0fdc96b26-cni-binary-copy\") pod \"multus-additional-cni-plugins-j2jc7\" (UID: \"c9814dbf-8db1-4d4a-9d40-d7a0fdc96b26\") " pod="openshift-multus/multus-additional-cni-plugins-j2jc7"
Apr 24 14:23:59.887890 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:23:59.887354    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/e87608cb-741d-47f5-bb29-fc68c0dc8bc4-host-run-k8s-cni-cncf-io\") pod \"multus-rwdxf\" (UID: \"e87608cb-741d-47f5-bb29-fc68c0dc8bc4\") " pod="openshift-multus/multus-rwdxf"
Apr 24 14:23:59.887890 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:23:59.887364    2574 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/e87608cb-741d-47f5-bb29-fc68c0dc8bc4-host-var-lib-kubelet\") pod \"multus-rwdxf\" (UID: \"e87608cb-741d-47f5-bb29-fc68c0dc8bc4\") " pod="openshift-multus/multus-rwdxf"
Apr 24 14:23:59.887890 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:23:59.887373    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/44036c3e-1fdc-40d4-899c-9009cd4589e3-host-slash\") pod \"ovnkube-node-w5x5x\" (UID: \"44036c3e-1fdc-40d4-899c-9009cd4589e3\") " pod="openshift-ovn-kubernetes/ovnkube-node-w5x5x"
Apr 24 14:23:59.888736 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:23:59.887393    2574 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-kfrc8\" (UniqueName: \"kubernetes.io/projected/c9814dbf-8db1-4d4a-9d40-d7a0fdc96b26-kube-api-access-kfrc8\") pod \"multus-additional-cni-plugins-j2jc7\" (UID: \"c9814dbf-8db1-4d4a-9d40-d7a0fdc96b26\") " pod="openshift-multus/multus-additional-cni-plugins-j2jc7"
Apr 24 14:23:59.888736 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:23:59.887427    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/e87608cb-741d-47f5-bb29-fc68c0dc8bc4-host-var-lib-kubelet\") pod \"multus-rwdxf\" (UID: \"e87608cb-741d-47f5-bb29-fc68c0dc8bc4\") " pod="openshift-multus/multus-rwdxf"
Apr 24 14:23:59.888736 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:23:59.887456    2574 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/44036c3e-1fdc-40d4-899c-9009cd4589e3-run-ovn\") pod \"ovnkube-node-w5x5x\" (UID: \"44036c3e-1fdc-40d4-899c-9009cd4589e3\") " pod="openshift-ovn-kubernetes/ovnkube-node-w5x5x"
Apr 24 14:23:59.888736 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:23:59.887476    2574 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/0b8fb5e7-eaa1-4b51-9f84-37bb0eb1a711-hosts-file\") pod \"node-resolver-bt99l\" (UID: \"0b8fb5e7-eaa1-4b51-9f84-37bb0eb1a711\") " pod="openshift-dns/node-resolver-bt99l"
Apr 24 14:23:59.888736 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:23:59.887491    2574 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/e87608cb-741d-47f5-bb29-fc68c0dc8bc4-multus-socket-dir-parent\") pod \"multus-rwdxf\" (UID: \"e87608cb-741d-47f5-bb29-fc68c0dc8bc4\") " pod="openshift-multus/multus-rwdxf"
Apr 24 14:23:59.888736 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:23:59.887507    2574 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/e87608cb-741d-47f5-bb29-fc68c0dc8bc4-host-var-lib-cni-multus\") pod \"multus-rwdxf\" (UID: \"e87608cb-741d-47f5-bb29-fc68c0dc8bc4\") " pod="openshift-multus/multus-rwdxf"
Apr 24 14:23:59.888736 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:23:59.887510    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/44036c3e-1fdc-40d4-899c-9009cd4589e3-run-ovn\") pod \"ovnkube-node-w5x5x\" (UID: \"44036c3e-1fdc-40d4-899c-9009cd4589e3\") " pod="openshift-ovn-kubernetes/ovnkube-node-w5x5x"
Apr 24 14:23:59.888736 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:23:59.887532    2574 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/e87608cb-741d-47f5-bb29-fc68c0dc8bc4-host-run-multus-certs\") pod \"multus-rwdxf\" (UID: \"e87608cb-741d-47f5-bb29-fc68c0dc8bc4\") " pod="openshift-multus/multus-rwdxf"
Apr 24 14:23:59.888736 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:23:59.887564    2574 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/44036c3e-1fdc-40d4-899c-9009cd4589e3-ovnkube-config\") pod \"ovnkube-node-w5x5x\" (UID: \"44036c3e-1fdc-40d4-899c-9009cd4589e3\") " pod="openshift-ovn-kubernetes/ovnkube-node-w5x5x"
Apr 24 14:23:59.888736 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:23:59.887569    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/0b8fb5e7-eaa1-4b51-9f84-37bb0eb1a711-hosts-file\") pod \"node-resolver-bt99l\" (UID: \"0b8fb5e7-eaa1-4b51-9f84-37bb0eb1a711\") " pod="openshift-dns/node-resolver-bt99l"
Apr 24 14:23:59.888736 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:23:59.887577    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/e87608cb-741d-47f5-bb29-fc68c0dc8bc4-multus-socket-dir-parent\") pod \"multus-rwdxf\" (UID: \"e87608cb-741d-47f5-bb29-fc68c0dc8bc4\") " pod="openshift-multus/multus-rwdxf"
Apr 24 14:23:59.888736 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:23:59.887588    2574 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/44036c3e-1fdc-40d4-899c-9009cd4589e3-env-overrides\") pod \"ovnkube-node-w5x5x\" (UID: \"44036c3e-1fdc-40d4-899c-9009cd4589e3\") " pod="openshift-ovn-kubernetes/ovnkube-node-w5x5x"
Apr 24 14:23:59.888736 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:23:59.887612    2574 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/0b8fb5e7-eaa1-4b51-9f84-37bb0eb1a711-tmp-dir\") pod \"node-resolver-bt99l\" (UID: \"0b8fb5e7-eaa1-4b51-9f84-37bb0eb1a711\") " pod="openshift-dns/node-resolver-bt99l"
Apr 24 14:23:59.888736 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:23:59.887619    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/e87608cb-741d-47f5-bb29-fc68c0dc8bc4-host-run-multus-certs\") pod \"multus-rwdxf\" (UID: \"e87608cb-741d-47f5-bb29-fc68c0dc8bc4\") " pod="openshift-multus/multus-rwdxf"
Apr 24 14:23:59.888736 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:23:59.887633    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/e87608cb-741d-47f5-bb29-fc68c0dc8bc4-host-var-lib-cni-multus\") pod \"multus-rwdxf\" (UID: \"e87608cb-741d-47f5-bb29-fc68c0dc8bc4\") " pod="openshift-multus/multus-rwdxf"
Apr 24 14:23:59.888736 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:23:59.887636    2574 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/e87608cb-741d-47f5-bb29-fc68c0dc8bc4-multus-daemon-config\") pod \"multus-rwdxf\" (UID: \"e87608cb-741d-47f5-bb29-fc68c0dc8bc4\") " pod="openshift-multus/multus-rwdxf"
Apr 24 14:23:59.888736 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:23:59.887744    2574 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/44036c3e-1fdc-40d4-899c-9009cd4589e3-run-openvswitch\") pod \"ovnkube-node-w5x5x\" (UID: \"44036c3e-1fdc-40d4-899c-9009cd4589e3\") " pod="openshift-ovn-kubernetes/ovnkube-node-w5x5x"
Apr 24 14:23:59.888736 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:23:59.887763    2574 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/44036c3e-1fdc-40d4-899c-9009cd4589e3-node-log\") pod \"ovnkube-node-w5x5x\" (UID: \"44036c3e-1fdc-40d4-899c-9009cd4589e3\") " pod="openshift-ovn-kubernetes/ovnkube-node-w5x5x"
Apr 24 14:23:59.889546 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:23:59.887790    2574 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/e87608cb-741d-47f5-bb29-fc68c0dc8bc4-multus-conf-dir\") pod \"multus-rwdxf\" (UID: \"e87608cb-741d-47f5-bb29-fc68c0dc8bc4\") " pod="openshift-multus/multus-rwdxf"
Apr 24 14:23:59.889546 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:23:59.887806    2574 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/e87608cb-741d-47f5-bb29-fc68c0dc8bc4-etc-kubernetes\") pod \"multus-rwdxf\" (UID: \"e87608cb-741d-47f5-bb29-fc68c0dc8bc4\") " pod="openshift-multus/multus-rwdxf"
Apr 24 14:23:59.889546 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:23:59.887821    2574 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/44036c3e-1fdc-40d4-899c-9009cd4589e3-log-socket\") pod \"ovnkube-node-w5x5x\" (UID: \"44036c3e-1fdc-40d4-899c-9009cd4589e3\") " pod="openshift-ovn-kubernetes/ovnkube-node-w5x5x"
Apr 24 14:23:59.889546 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:23:59.887836    2574 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-662k2\" (UniqueName: \"kubernetes.io/projected/44036c3e-1fdc-40d4-899c-9009cd4589e3-kube-api-access-662k2\") pod \"ovnkube-node-w5x5x\" (UID: \"44036c3e-1fdc-40d4-899c-9009cd4589e3\") " pod="openshift-ovn-kubernetes/ovnkube-node-w5x5x"
Apr 24 14:23:59.889546 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:23:59.887853    2574 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/e87608cb-741d-47f5-bb29-fc68c0dc8bc4-cnibin\") pod \"multus-rwdxf\" (UID: \"e87608cb-741d-47f5-bb29-fc68c0dc8bc4\") " pod="openshift-multus/multus-rwdxf"
Apr 24 14:23:59.889546 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:23:59.887906    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/e87608cb-741d-47f5-bb29-fc68c0dc8bc4-cnibin\") pod \"multus-rwdxf\" (UID: \"e87608cb-741d-47f5-bb29-fc68c0dc8bc4\") " pod="openshift-multus/multus-rwdxf"
Apr 24 14:23:59.889546 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:23:59.887909    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/0b8fb5e7-eaa1-4b51-9f84-37bb0eb1a711-tmp-dir\") pod \"node-resolver-bt99l\" (UID: \"0b8fb5e7-eaa1-4b51-9f84-37bb0eb1a711\") " pod="openshift-dns/node-resolver-bt99l"
Apr 24 14:23:59.889546 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:23:59.887956    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/44036c3e-1fdc-40d4-899c-9009cd4589e3-env-overrides\") pod \"ovnkube-node-w5x5x\" (UID: \"44036c3e-1fdc-40d4-899c-9009cd4589e3\") " pod="openshift-ovn-kubernetes/ovnkube-node-w5x5x"
Apr 24 14:23:59.889546 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:23:59.887968    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/e87608cb-741d-47f5-bb29-fc68c0dc8bc4-multus-conf-dir\") pod \"multus-rwdxf\" (UID: \"e87608cb-741d-47f5-bb29-fc68c0dc8bc4\") " pod="openshift-multus/multus-rwdxf"
Apr 24 14:23:59.889546 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:23:59.888006    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/44036c3e-1fdc-40d4-899c-9009cd4589e3-run-openvswitch\") pod \"ovnkube-node-w5x5x\" (UID: \"44036c3e-1fdc-40d4-899c-9009cd4589e3\") " pod="openshift-ovn-kubernetes/ovnkube-node-w5x5x"
Apr 24 14:23:59.889546 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:23:59.888044    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/44036c3e-1fdc-40d4-899c-9009cd4589e3-node-log\") pod \"ovnkube-node-w5x5x\" (UID: \"44036c3e-1fdc-40d4-899c-9009cd4589e3\") " pod="openshift-ovn-kubernetes/ovnkube-node-w5x5x"
Apr 24 14:23:59.889546 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:23:59.888051    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/e87608cb-741d-47f5-bb29-fc68c0dc8bc4-multus-daemon-config\") pod \"multus-rwdxf\" (UID: \"e87608cb-741d-47f5-bb29-fc68c0dc8bc4\") " pod="openshift-multus/multus-rwdxf"
Apr 24 14:23:59.889546 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:23:59.888059    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/44036c3e-1fdc-40d4-899c-9009cd4589e3-log-socket\") pod \"ovnkube-node-w5x5x\" (UID: \"44036c3e-1fdc-40d4-899c-9009cd4589e3\") " pod="openshift-ovn-kubernetes/ovnkube-node-w5x5x"
Apr 24 14:23:59.889546 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:23:59.888112    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/e87608cb-741d-47f5-bb29-fc68c0dc8bc4-etc-kubernetes\") pod \"multus-rwdxf\" (UID: \"e87608cb-741d-47f5-bb29-fc68c0dc8bc4\") " pod="openshift-multus/multus-rwdxf"
Apr 24 14:23:59.889546 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:23:59.888159    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/c9814dbf-8db1-4d4a-9d40-d7a0fdc96b26-cni-binary-copy\") pod \"multus-additional-cni-plugins-j2jc7\" (UID: \"c9814dbf-8db1-4d4a-9d40-d7a0fdc96b26\") " pod="openshift-multus/multus-additional-cni-plugins-j2jc7"
Apr 24 14:23:59.889546 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:23:59.888348    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/44036c3e-1fdc-40d4-899c-9009cd4589e3-ovnkube-config\") pod \"ovnkube-node-w5x5x\" (UID: \"44036c3e-1fdc-40d4-899c-9009cd4589e3\") " pod="openshift-ovn-kubernetes/ovnkube-node-w5x5x"
Apr 24 14:23:59.889546 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:23:59.889199    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/44036c3e-1fdc-40d4-899c-9009cd4589e3-ovn-node-metrics-cert\") pod \"ovnkube-node-w5x5x\" (UID: \"44036c3e-1fdc-40d4-899c-9009cd4589e3\") " pod="openshift-ovn-kubernetes/ovnkube-node-w5x5x"
Apr 24 14:23:59.894243 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:23:59.894223    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-n64fl\" (UniqueName: \"kubernetes.io/projected/e87608cb-741d-47f5-bb29-fc68c0dc8bc4-kube-api-access-n64fl\") pod \"multus-rwdxf\" (UID: \"e87608cb-741d-47f5-bb29-fc68c0dc8bc4\") " pod="openshift-multus/multus-rwdxf"
Apr 24 14:23:59.894555 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:23:59.894533    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-kfrc8\" (UniqueName: \"kubernetes.io/projected/c9814dbf-8db1-4d4a-9d40-d7a0fdc96b26-kube-api-access-kfrc8\") pod \"multus-additional-cni-plugins-j2jc7\" (UID: \"c9814dbf-8db1-4d4a-9d40-d7a0fdc96b26\") " pod="openshift-multus/multus-additional-cni-plugins-j2jc7"
Apr 24 14:23:59.894665 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:23:59.894648    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-w8qz8\" (UniqueName: \"kubernetes.io/projected/0b8fb5e7-eaa1-4b51-9f84-37bb0eb1a711-kube-api-access-w8qz8\") pod \"node-resolver-bt99l\" (UID: \"0b8fb5e7-eaa1-4b51-9f84-37bb0eb1a711\") " pod="openshift-dns/node-resolver-bt99l"
Apr 24 14:23:59.894964 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:23:59.894946    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-662k2\" (UniqueName: \"kubernetes.io/projected/44036c3e-1fdc-40d4-899c-9009cd4589e3-kube-api-access-662k2\") pod \"ovnkube-node-w5x5x\" (UID: \"44036c3e-1fdc-40d4-899c-9009cd4589e3\") " pod="openshift-ovn-kubernetes/ovnkube-node-w5x5x"
Apr 24 14:23:59.972918 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:23:59.972883    2574 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-zz678"
Apr 24 14:23:59.980949 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:23:59.980923    2574 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-6mgkm"
Apr 24 14:23:59.989609 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:23:59.989583    2574 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-4kqkj"
Apr 24 14:23:59.995155 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:23:59.995134    2574 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-lgh9n"
Apr 24 14:24:00.001741 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:24:00.001718    2574 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-r2pxq"
Apr 24 14:24:00.011421 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:24:00.011399    2574 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-w5x5x"
Apr 24 14:24:00.017551 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:24:00.017526    2574 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-bt99l"
Apr 24 14:24:00.024132 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:24:00.024064    2574 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-j2jc7"
Apr 24 14:24:00.029641 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:24:00.029620    2574 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-rwdxf"
Apr 24 14:24:00.290659 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:24:00.290566    2574 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/3bc44c73-963f-4a9e-8a52-1e8b031a08dc-metrics-certs\") pod \"network-metrics-daemon-zhkmq\" (UID: \"3bc44c73-963f-4a9e-8a52-1e8b031a08dc\") " pod="openshift-multus/network-metrics-daemon-zhkmq"
Apr 24 14:24:00.290659 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:24:00.290621    2574 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-css2f\" (UniqueName: \"kubernetes.io/projected/3964207c-9c58-4994-a8a9-aa4e4feca773-kube-api-access-css2f\") pod \"network-check-target-6tsjh\" (UID: \"3964207c-9c58-4994-a8a9-aa4e4feca773\") " pod="openshift-network-diagnostics/network-check-target-6tsjh"
Apr 24 14:24:00.290877 ip-10-0-136-132 kubenswrapper[2574]: E0424 14:24:00.290741    2574 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Apr 24 14:24:00.290877 ip-10-0-136-132 kubenswrapper[2574]: E0424 14:24:00.290824    2574 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/3bc44c73-963f-4a9e-8a52-1e8b031a08dc-metrics-certs podName:3bc44c73-963f-4a9e-8a52-1e8b031a08dc nodeName:}" failed. No retries permitted until 2026-04-24 14:24:01.290801519 +0000 UTC m=+4.022903149 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/3bc44c73-963f-4a9e-8a52-1e8b031a08dc-metrics-certs") pod "network-metrics-daemon-zhkmq" (UID: "3bc44c73-963f-4a9e-8a52-1e8b031a08dc") : object "openshift-multus"/"metrics-daemon-secret" not registered
Apr 24 14:24:00.290877 ip-10-0-136-132 kubenswrapper[2574]: E0424 14:24:00.290749    2574 projected.go:289] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Apr 24 14:24:00.290877 ip-10-0-136-132 kubenswrapper[2574]: E0424 14:24:00.290855    2574 projected.go:289] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Apr 24 14:24:00.290877 ip-10-0-136-132 kubenswrapper[2574]: E0424 14:24:00.290869    2574 projected.go:194] Error preparing data for projected volume kube-api-access-css2f for pod openshift-network-diagnostics/network-check-target-6tsjh: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Apr 24 14:24:00.291098 ip-10-0-136-132 kubenswrapper[2574]: E0424 14:24:00.290925    2574 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/3964207c-9c58-4994-a8a9-aa4e4feca773-kube-api-access-css2f podName:3964207c-9c58-4994-a8a9-aa4e4feca773 nodeName:}" failed. No retries permitted until 2026-04-24 14:24:01.290911298 +0000 UTC m=+4.023012944 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-api-access-css2f" (UniqueName: "kubernetes.io/projected/3964207c-9c58-4994-a8a9-aa4e4feca773-kube-api-access-css2f") pod "network-check-target-6tsjh" (UID: "3964207c-9c58-4994-a8a9-aa4e4feca773") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Apr 24 14:24:00.398258 ip-10-0-136-132 kubenswrapper[2574]: W0424 14:24:00.398229    2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod21ff8454_68c7_440e_9994_e8405b47919a.slice/crio-9a55954de136044ee7eb234b3e042acce46fe26da233715a107fb7de436bce47 WatchSource:0}: Error finding container 9a55954de136044ee7eb234b3e042acce46fe26da233715a107fb7de436bce47: Status 404 returned error can't find the container with id 9a55954de136044ee7eb234b3e042acce46fe26da233715a107fb7de436bce47
Apr 24 14:24:00.399534 ip-10-0-136-132 kubenswrapper[2574]: W0424 14:24:00.399503    2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod95f221a8_f16d_4d5f_9638_29db9dd3a969.slice/crio-f2a3f00f644f449414c13c6e2c8d00c29a94ff519b05f3916fccaa2e8903f654 WatchSource:0}: Error finding container f2a3f00f644f449414c13c6e2c8d00c29a94ff519b05f3916fccaa2e8903f654: Status 404 returned error can't find the container with id f2a3f00f644f449414c13c6e2c8d00c29a94ff519b05f3916fccaa2e8903f654
Apr 24 14:24:00.400846 ip-10-0-136-132 kubenswrapper[2574]: W0424 14:24:00.400828    2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0b8fb5e7_eaa1_4b51_9f84_37bb0eb1a711.slice/crio-5d12e4a67f964d1bfefb91c51874887725e33bec86f5db79322c50bff67ed46a WatchSource:0}: Error finding container 5d12e4a67f964d1bfefb91c51874887725e33bec86f5db79322c50bff67ed46a: Status 404 returned error can't find the container with id 5d12e4a67f964d1bfefb91c51874887725e33bec86f5db79322c50bff67ed46a
Apr 24 14:24:00.406190 ip-10-0-136-132 kubenswrapper[2574]: W0424 14:24:00.406159    2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc9814dbf_8db1_4d4a_9d40_d7a0fdc96b26.slice/crio-35ec62c5696264b5163ffc18ed7bdf56e7245bcce42de519305c93d657c3b104 WatchSource:0}: Error finding container 35ec62c5696264b5163ffc18ed7bdf56e7245bcce42de519305c93d657c3b104: Status 404 returned error can't find the container with id 35ec62c5696264b5163ffc18ed7bdf56e7245bcce42de519305c93d657c3b104
Apr 24 14:24:00.407018 ip-10-0-136-132 kubenswrapper[2574]: W0424 14:24:00.407000    2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode87608cb_741d_47f5_bb29_fc68c0dc8bc4.slice/crio-1be94e8813ee53ba4c30cd3a520b10114a13d6867f681c91e95520f017b79cfd WatchSource:0}: Error finding container 1be94e8813ee53ba4c30cd3a520b10114a13d6867f681c91e95520f017b79cfd: Status 404 returned error can't find the container with id 1be94e8813ee53ba4c30cd3a520b10114a13d6867f681c91e95520f017b79cfd
Apr 24 14:24:00.408280 ip-10-0-136-132 kubenswrapper[2574]: W0424 14:24:00.408233    2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5e4a7a90_d6ce_491b_ad65_52f0b2a127d6.slice/crio-b0c047c34aec719590675caa20491776c6f485734d3e9e6d59c96e95375ba5e8 WatchSource:0}: Error finding container b0c047c34aec719590675caa20491776c6f485734d3e9e6d59c96e95375ba5e8: Status 404 returned error can't find the container with id b0c047c34aec719590675caa20491776c6f485734d3e9e6d59c96e95375ba5e8
Apr 24 14:24:00.408719 ip-10-0-136-132 kubenswrapper[2574]: W0424 14:24:00.408656    2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod44036c3e_1fdc_40d4_899c_9009cd4589e3.slice/crio-5edb06dddfec61acfd4f09c454caf9c850acd430a20ffb73740ef28d944ae7ed WatchSource:0}: Error finding container 5edb06dddfec61acfd4f09c454caf9c850acd430a20ffb73740ef28d944ae7ed: Status 404 returned error can't find the container with id 5edb06dddfec61acfd4f09c454caf9c850acd430a20ffb73740ef28d944ae7ed
Apr 24 14:24:00.409526 ip-10-0-136-132 kubenswrapper[2574]: W0424 14:24:00.409446    2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb1844926_4d01_4f6e_8970_a8d662acd7c6.slice/crio-4a45263b124e330c31685d082bae50fee2bd83340e60b1ceb9a661a3beb9cef8 WatchSource:0}: Error finding container 4a45263b124e330c31685d082bae50fee2bd83340e60b1ceb9a661a3beb9cef8: Status 404 returned error can't find the container with id 4a45263b124e330c31685d082bae50fee2bd83340e60b1ceb9a661a3beb9cef8
Apr 24 14:24:00.410530 ip-10-0-136-132 kubenswrapper[2574]: W0424 14:24:00.410510    2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod97869a65_fbd6_4a1b_9eac_7adc13e4c254.slice/crio-a3a025fc69f0ca46f3a3069baecfbc0937aea63857785a1352f442f42b192b0f WatchSource:0}: Error finding container a3a025fc69f0ca46f3a3069baecfbc0937aea63857785a1352f442f42b192b0f: Status 404 returned error can't find the container with id a3a025fc69f0ca46f3a3069baecfbc0937aea63857785a1352f442f42b192b0f
Apr 24 14:24:00.712870 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:24:00.712789    2574 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-04-23 14:18:58 +0000 UTC" deadline="2027-11-17 10:39:52.571722226 +0000 UTC"
Apr 24 14:24:00.712870 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:24:00.712825    2574 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="13724h15m51.858900834s"
Apr 24 14:24:00.774403 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:24:00.773712    2574 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-136-132.ec2.internal" event={"ID":"682e7062ab8cd6aae842785f377acfd4","Type":"ContainerStarted","Data":"6cca257399538216c5d43a555322c2431b5e015fe298d6108ae62a25f21b91cd"}
Apr 24 14:24:00.778409 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:24:00.778312    2574 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-w5x5x" event={"ID":"44036c3e-1fdc-40d4-899c-9009cd4589e3","Type":"ContainerStarted","Data":"5edb06dddfec61acfd4f09c454caf9c850acd430a20ffb73740ef28d944ae7ed"}
Apr 24 14:24:00.779522 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:24:00.779479    2574 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-lgh9n" event={"ID":"5e4a7a90-d6ce-491b-ad65-52f0b2a127d6","Type":"ContainerStarted","Data":"b0c047c34aec719590675caa20491776c6f485734d3e9e6d59c96e95375ba5e8"}
Apr 24 14:24:00.780881 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:24:00.780845    2574 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-bt99l" event={"ID":"0b8fb5e7-eaa1-4b51-9f84-37bb0eb1a711","Type":"ContainerStarted","Data":"5d12e4a67f964d1bfefb91c51874887725e33bec86f5db79322c50bff67ed46a"}
Apr 24 14:24:00.782385 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:24:00.782351    2574 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-4kqkj" event={"ID":"21ff8454-68c7-440e-9994-e8405b47919a","Type":"ContainerStarted","Data":"9a55954de136044ee7eb234b3e042acce46fe26da233715a107fb7de436bce47"}
Apr 24 14:24:00.783676 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:24:00.783639    2574 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-6mgkm" event={"ID":"97869a65-fbd6-4a1b-9eac-7adc13e4c254","Type":"ContainerStarted","Data":"a3a025fc69f0ca46f3a3069baecfbc0937aea63857785a1352f442f42b192b0f"}
Apr 24 14:24:00.786421 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:24:00.786382    2574 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-rwdxf" event={"ID":"e87608cb-741d-47f5-bb29-fc68c0dc8bc4","Type":"ContainerStarted","Data":"1be94e8813ee53ba4c30cd3a520b10114a13d6867f681c91e95520f017b79cfd"}
Apr 24 14:24:00.787775 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:24:00.787170    2574 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="kube-system/kube-apiserver-proxy-ip-10-0-136-132.ec2.internal" podStartSLOduration=1.7871564 podStartE2EDuration="1.7871564s" podCreationTimestamp="2026-04-24 14:23:59 +0000 UTC" firstStartedPulling="0001-01-01 00:00:00 +0000 UTC" lastFinishedPulling="0001-01-01 00:00:00 +0000 UTC" observedRunningTime="2026-04-24 14:24:00.786634308 +0000 UTC m=+3.518735973" watchObservedRunningTime="2026-04-24 14:24:00.7871564 +0000 UTC m=+3.519258052"
Apr 24 14:24:00.789048 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:24:00.788992    2574 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-zz678" event={"ID":"b1844926-4d01-4f6e-8970-a8d662acd7c6","Type":"ContainerStarted","Data":"4a45263b124e330c31685d082bae50fee2bd83340e60b1ceb9a661a3beb9cef8"}
Apr 24 14:24:00.791645 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:24:00.791352    2574 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-j2jc7" event={"ID":"c9814dbf-8db1-4d4a-9d40-d7a0fdc96b26","Type":"ContainerStarted","Data":"35ec62c5696264b5163ffc18ed7bdf56e7245bcce42de519305c93d657c3b104"}
Apr 24 14:24:00.792844 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:24:00.792707    2574 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-r2pxq" event={"ID":"95f221a8-f16d-4d5f-9638-29db9dd3a969","Type":"ContainerStarted","Data":"f2a3f00f644f449414c13c6e2c8d00c29a94ff519b05f3916fccaa2e8903f654"}
Apr 24 14:24:01.298392 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:24:01.298355    2574 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/3bc44c73-963f-4a9e-8a52-1e8b031a08dc-metrics-certs\") pod \"network-metrics-daemon-zhkmq\" (UID: \"3bc44c73-963f-4a9e-8a52-1e8b031a08dc\") " pod="openshift-multus/network-metrics-daemon-zhkmq"
Apr 24 14:24:01.298565 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:24:01.298420    2574 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-css2f\" (UniqueName: \"kubernetes.io/projected/3964207c-9c58-4994-a8a9-aa4e4feca773-kube-api-access-css2f\") pod \"network-check-target-6tsjh\" (UID: \"3964207c-9c58-4994-a8a9-aa4e4feca773\") " pod="openshift-network-diagnostics/network-check-target-6tsjh"
Apr 24 14:24:01.298565 ip-10-0-136-132 kubenswrapper[2574]: E0424 14:24:01.298558    2574 projected.go:289] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Apr 24 14:24:01.298669 ip-10-0-136-132 kubenswrapper[2574]: E0424 14:24:01.298578    2574 projected.go:289] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Apr 24 14:24:01.298669 ip-10-0-136-132 kubenswrapper[2574]: E0424 14:24:01.298591    2574 projected.go:194] Error preparing data for projected volume kube-api-access-css2f for pod openshift-network-diagnostics/network-check-target-6tsjh: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Apr 24 14:24:01.298669 ip-10-0-136-132 kubenswrapper[2574]: E0424 14:24:01.298648    2574 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/3964207c-9c58-4994-a8a9-aa4e4feca773-kube-api-access-css2f podName:3964207c-9c58-4994-a8a9-aa4e4feca773 nodeName:}" failed. No retries permitted until 2026-04-24 14:24:03.298629194 +0000 UTC m=+6.030730844 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-api-access-css2f" (UniqueName: "kubernetes.io/projected/3964207c-9c58-4994-a8a9-aa4e4feca773-kube-api-access-css2f") pod "network-check-target-6tsjh" (UID: "3964207c-9c58-4994-a8a9-aa4e4feca773") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Apr 24 14:24:01.302567 ip-10-0-136-132 kubenswrapper[2574]: E0424 14:24:01.302537    2574 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Apr 24 14:24:01.302707 ip-10-0-136-132 kubenswrapper[2574]: E0424 14:24:01.302622    2574 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/3bc44c73-963f-4a9e-8a52-1e8b031a08dc-metrics-certs podName:3bc44c73-963f-4a9e-8a52-1e8b031a08dc nodeName:}" failed. No retries permitted until 2026-04-24 14:24:03.302602562 +0000 UTC m=+6.034704199 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/3bc44c73-963f-4a9e-8a52-1e8b031a08dc-metrics-certs") pod "network-metrics-daemon-zhkmq" (UID: "3bc44c73-963f-4a9e-8a52-1e8b031a08dc") : object "openshift-multus"/"metrics-daemon-secret" not registered
Apr 24 14:24:01.769466 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:24:01.769434    2574 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-zhkmq"
Apr 24 14:24:01.769907 ip-10-0-136-132 kubenswrapper[2574]: E0424 14:24:01.769578    2574 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-zhkmq" podUID="3bc44c73-963f-4a9e-8a52-1e8b031a08dc"
Apr 24 14:24:01.769907 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:24:01.769674    2574 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-6tsjh"
Apr 24 14:24:01.769907 ip-10-0-136-132 kubenswrapper[2574]: E0424 14:24:01.769743    2574 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-6tsjh" podUID="3964207c-9c58-4994-a8a9-aa4e4feca773"
Apr 24 14:24:01.816645 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:24:01.816214    2574 generic.go:358] "Generic (PLEG): container finished" podID="b5277dfd83a996ad93ff188797af276d" containerID="936ff347211b94e2ad8862e9b2bf8aaa887eb026473e5c722245f75a38544d9c" exitCode=0
Apr 24 14:24:01.817151 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:24:01.817085    2574 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-136-132.ec2.internal" event={"ID":"b5277dfd83a996ad93ff188797af276d","Type":"ContainerDied","Data":"936ff347211b94e2ad8862e9b2bf8aaa887eb026473e5c722245f75a38544d9c"}
Apr 24 14:24:02.835926 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:24:02.835878    2574 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-136-132.ec2.internal" event={"ID":"b5277dfd83a996ad93ff188797af276d","Type":"ContainerStarted","Data":"e2864f3363b40341490e1fbc897c9b14da90ad81cae62c78995e543425d4fb4f"}
Apr 24 14:24:02.848785 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:24:02.848733    2574 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-136-132.ec2.internal" podStartSLOduration=3.84871427 podStartE2EDuration="3.84871427s" podCreationTimestamp="2026-04-24 14:23:59 +0000 UTC" firstStartedPulling="0001-01-01 00:00:00 +0000 UTC" lastFinishedPulling="0001-01-01 00:00:00 +0000 UTC" observedRunningTime="2026-04-24 14:24:02.848398913 +0000 UTC m=+5.580500563" watchObservedRunningTime="2026-04-24 14:24:02.84871427 +0000 UTC m=+5.580815922"
Apr 24 14:24:03.315064 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:24:03.314494    2574 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-css2f\" (UniqueName: \"kubernetes.io/projected/3964207c-9c58-4994-a8a9-aa4e4feca773-kube-api-access-css2f\") pod \"network-check-target-6tsjh\" (UID: \"3964207c-9c58-4994-a8a9-aa4e4feca773\") " pod="openshift-network-diagnostics/network-check-target-6tsjh"
Apr 24 14:24:03.315064 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:24:03.314561    2574 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/3bc44c73-963f-4a9e-8a52-1e8b031a08dc-metrics-certs\") pod \"network-metrics-daemon-zhkmq\" (UID: \"3bc44c73-963f-4a9e-8a52-1e8b031a08dc\") " pod="openshift-multus/network-metrics-daemon-zhkmq"
Apr 24 14:24:03.315064 ip-10-0-136-132 kubenswrapper[2574]: E0424 14:24:03.314657    2574 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Apr 24 14:24:03.315064 ip-10-0-136-132 kubenswrapper[2574]: E0424 14:24:03.314712    2574 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/3bc44c73-963f-4a9e-8a52-1e8b031a08dc-metrics-certs podName:3bc44c73-963f-4a9e-8a52-1e8b031a08dc nodeName:}" failed. No retries permitted until 2026-04-24 14:24:07.314696515 +0000 UTC m=+10.046798146 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/3bc44c73-963f-4a9e-8a52-1e8b031a08dc-metrics-certs") pod "network-metrics-daemon-zhkmq" (UID: "3bc44c73-963f-4a9e-8a52-1e8b031a08dc") : object "openshift-multus"/"metrics-daemon-secret" not registered
Apr 24 14:24:03.315064 ip-10-0-136-132 kubenswrapper[2574]: E0424 14:24:03.314658    2574 projected.go:289] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Apr 24 14:24:03.315064 ip-10-0-136-132 kubenswrapper[2574]: E0424 14:24:03.314761    2574 projected.go:289] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Apr 24 14:24:03.315064 ip-10-0-136-132 kubenswrapper[2574]: E0424 14:24:03.314774    2574 projected.go:194] Error preparing data for projected volume kube-api-access-css2f for pod openshift-network-diagnostics/network-check-target-6tsjh: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Apr 24 14:24:03.315064 ip-10-0-136-132 kubenswrapper[2574]: E0424 14:24:03.314814    2574 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/3964207c-9c58-4994-a8a9-aa4e4feca773-kube-api-access-css2f podName:3964207c-9c58-4994-a8a9-aa4e4feca773 nodeName:}" failed. No retries permitted until 2026-04-24 14:24:07.314801338 +0000 UTC m=+10.046902978 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-api-access-css2f" (UniqueName: "kubernetes.io/projected/3964207c-9c58-4994-a8a9-aa4e4feca773-kube-api-access-css2f") pod "network-check-target-6tsjh" (UID: "3964207c-9c58-4994-a8a9-aa4e4feca773") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Apr 24 14:24:03.769630 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:24:03.768906    2574 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-zhkmq"
Apr 24 14:24:03.769630 ip-10-0-136-132 kubenswrapper[2574]: E0424 14:24:03.769041    2574 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-zhkmq" podUID="3bc44c73-963f-4a9e-8a52-1e8b031a08dc"
Apr 24 14:24:03.769630 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:24:03.769430    2574 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-6tsjh"
Apr 24 14:24:03.769630 ip-10-0-136-132 kubenswrapper[2574]: E0424 14:24:03.769515    2574 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-6tsjh" podUID="3964207c-9c58-4994-a8a9-aa4e4feca773"
Apr 24 14:24:05.765984 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:24:05.765933    2574 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-zhkmq"
Apr 24 14:24:05.766524 ip-10-0-136-132 kubenswrapper[2574]: E0424 14:24:05.766086    2574 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-zhkmq" podUID="3bc44c73-963f-4a9e-8a52-1e8b031a08dc"
Apr 24 14:24:05.766524 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:24:05.766299    2574 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-6tsjh"
Apr 24 14:24:05.766524 ip-10-0-136-132 kubenswrapper[2574]: E0424 14:24:05.766423    2574 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-6tsjh" podUID="3964207c-9c58-4994-a8a9-aa4e4feca773"
Apr 24 14:24:07.347372 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:24:07.347314    2574 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/3bc44c73-963f-4a9e-8a52-1e8b031a08dc-metrics-certs\") pod \"network-metrics-daemon-zhkmq\" (UID: \"3bc44c73-963f-4a9e-8a52-1e8b031a08dc\") " pod="openshift-multus/network-metrics-daemon-zhkmq"
Apr 24 14:24:07.347801 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:24:07.347410    2574 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-css2f\" (UniqueName: \"kubernetes.io/projected/3964207c-9c58-4994-a8a9-aa4e4feca773-kube-api-access-css2f\") pod \"network-check-target-6tsjh\" (UID: \"3964207c-9c58-4994-a8a9-aa4e4feca773\") " pod="openshift-network-diagnostics/network-check-target-6tsjh"
Apr 24 14:24:07.347801 ip-10-0-136-132 kubenswrapper[2574]: E0424 14:24:07.347440    2574 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Apr 24 14:24:07.347801 ip-10-0-136-132 kubenswrapper[2574]: E0424 14:24:07.347510    2574 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/3bc44c73-963f-4a9e-8a52-1e8b031a08dc-metrics-certs podName:3bc44c73-963f-4a9e-8a52-1e8b031a08dc nodeName:}" failed. No retries permitted until 2026-04-24 14:24:15.347488266 +0000 UTC m=+18.079589903 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/3bc44c73-963f-4a9e-8a52-1e8b031a08dc-metrics-certs") pod "network-metrics-daemon-zhkmq" (UID: "3bc44c73-963f-4a9e-8a52-1e8b031a08dc") : object "openshift-multus"/"metrics-daemon-secret" not registered
Apr 24 14:24:07.347801 ip-10-0-136-132 kubenswrapper[2574]: E0424 14:24:07.347595    2574 projected.go:289] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Apr 24 14:24:07.347801 ip-10-0-136-132 kubenswrapper[2574]: E0424 14:24:07.347617    2574 projected.go:289] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Apr 24 14:24:07.347801 ip-10-0-136-132 kubenswrapper[2574]: E0424 14:24:07.347632    2574 projected.go:194] Error preparing data for projected volume kube-api-access-css2f for pod openshift-network-diagnostics/network-check-target-6tsjh: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Apr 24 14:24:07.347801 ip-10-0-136-132 kubenswrapper[2574]: E0424 14:24:07.347683    2574 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/3964207c-9c58-4994-a8a9-aa4e4feca773-kube-api-access-css2f podName:3964207c-9c58-4994-a8a9-aa4e4feca773 nodeName:}" failed. No retries permitted until 2026-04-24 14:24:15.347666327 +0000 UTC m=+18.079767958 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "kube-api-access-css2f" (UniqueName: "kubernetes.io/projected/3964207c-9c58-4994-a8a9-aa4e4feca773-kube-api-access-css2f") pod "network-check-target-6tsjh" (UID: "3964207c-9c58-4994-a8a9-aa4e4feca773") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Apr 24 14:24:07.767472 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:24:07.766853    2574 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-zhkmq"
Apr 24 14:24:07.767472 ip-10-0-136-132 kubenswrapper[2574]: E0424 14:24:07.766970    2574 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-zhkmq" podUID="3bc44c73-963f-4a9e-8a52-1e8b031a08dc"
Apr 24 14:24:07.767472 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:24:07.767282    2574 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-6tsjh"
Apr 24 14:24:07.767472 ip-10-0-136-132 kubenswrapper[2574]: E0424 14:24:07.767391    2574 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-6tsjh" podUID="3964207c-9c58-4994-a8a9-aa4e4feca773"
Apr 24 14:24:09.766205 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:24:09.766109    2574 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-6tsjh"
Apr 24 14:24:09.766205 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:24:09.766150    2574 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-zhkmq"
Apr 24 14:24:09.766675 ip-10-0-136-132 kubenswrapper[2574]: E0424 14:24:09.766241    2574 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-6tsjh" podUID="3964207c-9c58-4994-a8a9-aa4e4feca773"
Apr 24 14:24:09.766675 ip-10-0-136-132 kubenswrapper[2574]: E0424 14:24:09.766408    2574 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-zhkmq" podUID="3bc44c73-963f-4a9e-8a52-1e8b031a08dc"
Apr 24 14:24:11.765585 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:24:11.765551    2574 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-6tsjh"
Apr 24 14:24:11.765585 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:24:11.765564    2574 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-zhkmq"
Apr 24 14:24:11.766144 ip-10-0-136-132 kubenswrapper[2574]: E0424 14:24:11.765678    2574 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-6tsjh" podUID="3964207c-9c58-4994-a8a9-aa4e4feca773"
Apr 24 14:24:11.766144 ip-10-0-136-132 kubenswrapper[2574]: E0424 14:24:11.765833    2574 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-zhkmq" podUID="3bc44c73-963f-4a9e-8a52-1e8b031a08dc"
Apr 24 14:24:13.765570 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:24:13.765535    2574 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-zhkmq"
Apr 24 14:24:13.766040 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:24:13.765533    2574 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-6tsjh"
Apr 24 14:24:13.766040 ip-10-0-136-132 kubenswrapper[2574]: E0424 14:24:13.765674    2574 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-zhkmq" podUID="3bc44c73-963f-4a9e-8a52-1e8b031a08dc"
Apr 24 14:24:13.766040 ip-10-0-136-132 kubenswrapper[2574]: E0424 14:24:13.765745    2574 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-6tsjh" podUID="3964207c-9c58-4994-a8a9-aa4e4feca773"
Apr 24 14:24:14.450354 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:24:14.450306    2574 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["kube-system/global-pull-secret-syncer-nwx74"]
Apr 24 14:24:14.455916 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:24:14.455888    2574 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-nwx74"
Apr 24 14:24:14.456039 ip-10-0-136-132 kubenswrapper[2574]: E0424 14:24:14.455966    2574 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-nwx74" podUID="5e2a3789-4ab2-4df5-bbbf-e649d90f15a5"
Apr 24 14:24:14.500648 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:24:14.500607    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/5e2a3789-4ab2-4df5-bbbf-e649d90f15a5-original-pull-secret\") pod \"global-pull-secret-syncer-nwx74\" (UID: \"5e2a3789-4ab2-4df5-bbbf-e649d90f15a5\") " pod="kube-system/global-pull-secret-syncer-nwx74"
Apr 24 14:24:14.500823 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:24:14.500665    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/5e2a3789-4ab2-4df5-bbbf-e649d90f15a5-dbus\") pod \"global-pull-secret-syncer-nwx74\" (UID: \"5e2a3789-4ab2-4df5-bbbf-e649d90f15a5\") " pod="kube-system/global-pull-secret-syncer-nwx74"
Apr 24 14:24:14.500823 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:24:14.500691    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/5e2a3789-4ab2-4df5-bbbf-e649d90f15a5-kubelet-config\") pod \"global-pull-secret-syncer-nwx74\" (UID: \"5e2a3789-4ab2-4df5-bbbf-e649d90f15a5\") " pod="kube-system/global-pull-secret-syncer-nwx74"
Apr 24 14:24:14.601077 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:24:14.601048    2574 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/5e2a3789-4ab2-4df5-bbbf-e649d90f15a5-dbus\") pod \"global-pull-secret-syncer-nwx74\" (UID: \"5e2a3789-4ab2-4df5-bbbf-e649d90f15a5\") " pod="kube-system/global-pull-secret-syncer-nwx74"
Apr 24 14:24:14.601077 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:24:14.601086    2574 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/5e2a3789-4ab2-4df5-bbbf-e649d90f15a5-kubelet-config\") pod \"global-pull-secret-syncer-nwx74\" (UID: \"5e2a3789-4ab2-4df5-bbbf-e649d90f15a5\") " pod="kube-system/global-pull-secret-syncer-nwx74"
Apr 24 14:24:14.601306 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:24:14.601170    2574 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/5e2a3789-4ab2-4df5-bbbf-e649d90f15a5-original-pull-secret\") pod \"global-pull-secret-syncer-nwx74\" (UID: \"5e2a3789-4ab2-4df5-bbbf-e649d90f15a5\") " pod="kube-system/global-pull-secret-syncer-nwx74"
Apr 24 14:24:14.601306 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:24:14.601178    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/5e2a3789-4ab2-4df5-bbbf-e649d90f15a5-dbus\") pod \"global-pull-secret-syncer-nwx74\" (UID: \"5e2a3789-4ab2-4df5-bbbf-e649d90f15a5\") " pod="kube-system/global-pull-secret-syncer-nwx74"
Apr 24 14:24:14.601306 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:24:14.601271    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/5e2a3789-4ab2-4df5-bbbf-e649d90f15a5-kubelet-config\") pod \"global-pull-secret-syncer-nwx74\" (UID: \"5e2a3789-4ab2-4df5-bbbf-e649d90f15a5\") " pod="kube-system/global-pull-secret-syncer-nwx74"
Apr 24 14:24:14.601306 ip-10-0-136-132 kubenswrapper[2574]: E0424 14:24:14.601276    2574 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Apr 24 14:24:14.601502 ip-10-0-136-132 kubenswrapper[2574]: E0424 14:24:14.601367    2574 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/5e2a3789-4ab2-4df5-bbbf-e649d90f15a5-original-pull-secret podName:5e2a3789-4ab2-4df5-bbbf-e649d90f15a5 nodeName:}" failed. No retries permitted until 2026-04-24 14:24:15.101350383 +0000 UTC m=+17.833452023 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/5e2a3789-4ab2-4df5-bbbf-e649d90f15a5-original-pull-secret") pod "global-pull-secret-syncer-nwx74" (UID: "5e2a3789-4ab2-4df5-bbbf-e649d90f15a5") : object "kube-system"/"original-pull-secret" not registered
Apr 24 14:24:15.104435 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:24:15.104392    2574 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/5e2a3789-4ab2-4df5-bbbf-e649d90f15a5-original-pull-secret\") pod \"global-pull-secret-syncer-nwx74\" (UID: \"5e2a3789-4ab2-4df5-bbbf-e649d90f15a5\") " pod="kube-system/global-pull-secret-syncer-nwx74"
Apr 24 14:24:15.104890 ip-10-0-136-132 kubenswrapper[2574]: E0424 14:24:15.104541    2574 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Apr 24 14:24:15.104890 ip-10-0-136-132 kubenswrapper[2574]: E0424 14:24:15.104613    2574 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/5e2a3789-4ab2-4df5-bbbf-e649d90f15a5-original-pull-secret podName:5e2a3789-4ab2-4df5-bbbf-e649d90f15a5 nodeName:}" failed. No retries permitted until 2026-04-24 14:24:16.104596092 +0000 UTC m=+18.836697731 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/5e2a3789-4ab2-4df5-bbbf-e649d90f15a5-original-pull-secret") pod "global-pull-secret-syncer-nwx74" (UID: "5e2a3789-4ab2-4df5-bbbf-e649d90f15a5") : object "kube-system"/"original-pull-secret" not registered
Apr 24 14:24:15.407328 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:24:15.407244    2574 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-css2f\" (UniqueName: \"kubernetes.io/projected/3964207c-9c58-4994-a8a9-aa4e4feca773-kube-api-access-css2f\") pod \"network-check-target-6tsjh\" (UID: \"3964207c-9c58-4994-a8a9-aa4e4feca773\") " pod="openshift-network-diagnostics/network-check-target-6tsjh"
Apr 24 14:24:15.407503 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:24:15.407372    2574 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/3bc44c73-963f-4a9e-8a52-1e8b031a08dc-metrics-certs\") pod \"network-metrics-daemon-zhkmq\" (UID: \"3bc44c73-963f-4a9e-8a52-1e8b031a08dc\") " pod="openshift-multus/network-metrics-daemon-zhkmq"
Apr 24 14:24:15.407503 ip-10-0-136-132 kubenswrapper[2574]: E0424 14:24:15.407393    2574 projected.go:289] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Apr 24 14:24:15.407503 ip-10-0-136-132 kubenswrapper[2574]: E0424 14:24:15.407413    2574 projected.go:289] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Apr 24 14:24:15.407503 ip-10-0-136-132 kubenswrapper[2574]: E0424 14:24:15.407427    2574 projected.go:194] Error preparing data for projected volume kube-api-access-css2f for pod openshift-network-diagnostics/network-check-target-6tsjh: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Apr 24 14:24:15.407503 ip-10-0-136-132 kubenswrapper[2574]: E0424 14:24:15.407484    2574 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/3964207c-9c58-4994-a8a9-aa4e4feca773-kube-api-access-css2f podName:3964207c-9c58-4994-a8a9-aa4e4feca773 nodeName:}" failed. No retries permitted until 2026-04-24 14:24:31.407469255 +0000 UTC m=+34.139570897 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "kube-api-access-css2f" (UniqueName: "kubernetes.io/projected/3964207c-9c58-4994-a8a9-aa4e4feca773-kube-api-access-css2f") pod "network-check-target-6tsjh" (UID: "3964207c-9c58-4994-a8a9-aa4e4feca773") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Apr 24 14:24:15.407771 ip-10-0-136-132 kubenswrapper[2574]: E0424 14:24:15.407506    2574 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Apr 24 14:24:15.407771 ip-10-0-136-132 kubenswrapper[2574]: E0424 14:24:15.407577    2574 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/3bc44c73-963f-4a9e-8a52-1e8b031a08dc-metrics-certs podName:3bc44c73-963f-4a9e-8a52-1e8b031a08dc nodeName:}" failed. No retries permitted until 2026-04-24 14:24:31.407558813 +0000 UTC m=+34.139660452 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/3bc44c73-963f-4a9e-8a52-1e8b031a08dc-metrics-certs") pod "network-metrics-daemon-zhkmq" (UID: "3bc44c73-963f-4a9e-8a52-1e8b031a08dc") : object "openshift-multus"/"metrics-daemon-secret" not registered
Apr 24 14:24:15.765378 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:24:15.765220    2574 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-6tsjh"
Apr 24 14:24:15.765546 ip-10-0-136-132 kubenswrapper[2574]: E0424 14:24:15.765378    2574 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-6tsjh" podUID="3964207c-9c58-4994-a8a9-aa4e4feca773"
Apr 24 14:24:15.765546 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:24:15.765439    2574 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-zhkmq"
Apr 24 14:24:15.765647 ip-10-0-136-132 kubenswrapper[2574]: E0424 14:24:15.765564    2574 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-zhkmq" podUID="3bc44c73-963f-4a9e-8a52-1e8b031a08dc"
Apr 24 14:24:16.113704 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:24:16.113604    2574 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/5e2a3789-4ab2-4df5-bbbf-e649d90f15a5-original-pull-secret\") pod \"global-pull-secret-syncer-nwx74\" (UID: \"5e2a3789-4ab2-4df5-bbbf-e649d90f15a5\") " pod="kube-system/global-pull-secret-syncer-nwx74"
Apr 24 14:24:16.114145 ip-10-0-136-132 kubenswrapper[2574]: E0424 14:24:16.113781    2574 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Apr 24 14:24:16.114145 ip-10-0-136-132 kubenswrapper[2574]: E0424 14:24:16.113866    2574 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/5e2a3789-4ab2-4df5-bbbf-e649d90f15a5-original-pull-secret podName:5e2a3789-4ab2-4df5-bbbf-e649d90f15a5 nodeName:}" failed. No retries permitted until 2026-04-24 14:24:18.113844647 +0000 UTC m=+20.845946284 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/5e2a3789-4ab2-4df5-bbbf-e649d90f15a5-original-pull-secret") pod "global-pull-secret-syncer-nwx74" (UID: "5e2a3789-4ab2-4df5-bbbf-e649d90f15a5") : object "kube-system"/"original-pull-secret" not registered
Apr 24 14:24:16.765802 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:24:16.765732    2574 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-nwx74"
Apr 24 14:24:16.765932 ip-10-0-136-132 kubenswrapper[2574]: E0424 14:24:16.765839    2574 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-nwx74" podUID="5e2a3789-4ab2-4df5-bbbf-e649d90f15a5"
Apr 24 14:24:17.766688 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:24:17.766495    2574 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-zhkmq"
Apr 24 14:24:17.767423 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:24:17.766565    2574 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-6tsjh"
Apr 24 14:24:17.767423 ip-10-0-136-132 kubenswrapper[2574]: E0424 14:24:17.766760    2574 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-zhkmq" podUID="3bc44c73-963f-4a9e-8a52-1e8b031a08dc"
Apr 24 14:24:17.767423 ip-10-0-136-132 kubenswrapper[2574]: E0424 14:24:17.766863    2574 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-6tsjh" podUID="3964207c-9c58-4994-a8a9-aa4e4feca773"
Apr 24 14:24:17.863012 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:24:17.862974    2574 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-4kqkj" event={"ID":"21ff8454-68c7-440e-9994-e8405b47919a","Type":"ContainerStarted","Data":"770e70b94ad33a4cc7956cff1be4ed52e4aa183a12557420b96498ce0f0b9cd6"}
Apr 24 14:24:17.864362 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:24:17.864305    2574 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-6mgkm" event={"ID":"97869a65-fbd6-4a1b-9eac-7adc13e4c254","Type":"ContainerStarted","Data":"1b594c72601bf4a14fc0f44a63a480c420e3559cbba0f2ce7fd8786cbf18cb08"}
Apr 24 14:24:17.865722 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:24:17.865701    2574 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-rwdxf" event={"ID":"e87608cb-741d-47f5-bb29-fc68c0dc8bc4","Type":"ContainerStarted","Data":"9bc96db7e58e3071e3bdc4a733bba863db8651329f845c1bed7bbb56e935dde1"}
Apr 24 14:24:17.867288 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:24:17.867263    2574 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-zz678" event={"ID":"b1844926-4d01-4f6e-8970-a8d662acd7c6","Type":"ContainerStarted","Data":"1952c981c1912dace7db9e39ca30fd9028bfd5b9518bf1b1a71c729a02c62b1c"}
Apr 24 14:24:17.868821 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:24:17.868795    2574 generic.go:358] "Generic (PLEG): container finished" podID="c9814dbf-8db1-4d4a-9d40-d7a0fdc96b26" containerID="03a37ec2f0db930eedda6b553d28c3f9d628be024f1aa487c25c19411124a82d" exitCode=0
Apr 24 14:24:17.868915 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:24:17.868874    2574 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-j2jc7" event={"ID":"c9814dbf-8db1-4d4a-9d40-d7a0fdc96b26","Type":"ContainerDied","Data":"03a37ec2f0db930eedda6b553d28c3f9d628be024f1aa487c25c19411124a82d"}
Apr 24 14:24:17.871575 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:24:17.871556    2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-w5x5x_44036c3e-1fdc-40d4-899c-9009cd4589e3/ovn-acl-logging/0.log"
Apr 24 14:24:17.871885 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:24:17.871866    2574 generic.go:358] "Generic (PLEG): container finished" podID="44036c3e-1fdc-40d4-899c-9009cd4589e3" containerID="2154270a235264b607e9c06815df67b3a01c006641f78800b2ac8ac8d578e4bf" exitCode=1
Apr 24 14:24:17.871961 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:24:17.871927    2574 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-w5x5x" event={"ID":"44036c3e-1fdc-40d4-899c-9009cd4589e3","Type":"ContainerStarted","Data":"5bdee9e3890cbfc46c805fbeeef154e8c3b687f5e11c7ac600859344ecb0cdac"}
Apr 24 14:24:17.871961 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:24:17.871955    2574 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-w5x5x" event={"ID":"44036c3e-1fdc-40d4-899c-9009cd4589e3","Type":"ContainerStarted","Data":"9dae61efccfe01ed84a99e906b235090875a2e93b0a69919e7a365c74c067ae8"}
Apr 24 14:24:17.872055 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:24:17.871970    2574 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-w5x5x" event={"ID":"44036c3e-1fdc-40d4-899c-9009cd4589e3","Type":"ContainerStarted","Data":"0c6acea960973ca9a7e1d20ab9a53c883eeaadd0db9f1fac78d4dc2eef4fa87d"}
Apr 24 14:24:17.872055 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:24:17.871983    2574 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-w5x5x" event={"ID":"44036c3e-1fdc-40d4-899c-9009cd4589e3","Type":"ContainerStarted","Data":"a32b5d7b09d460957dde87b64b96aad7f82f5d644a3db5ceee3cf93220878786"}
Apr 24 14:24:17.872055 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:24:17.871995    2574 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-w5x5x" event={"ID":"44036c3e-1fdc-40d4-899c-9009cd4589e3","Type":"ContainerDied","Data":"2154270a235264b607e9c06815df67b3a01c006641f78800b2ac8ac8d578e4bf"}
Apr 24 14:24:17.872055 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:24:17.872011    2574 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-w5x5x" event={"ID":"44036c3e-1fdc-40d4-899c-9009cd4589e3","Type":"ContainerStarted","Data":"a724af6d1ee4ccd179b8c7785e0b9fa62303097854576bf8b3f9b542125d7dc2"}
Apr 24 14:24:17.873194 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:24:17.873127    2574 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-lgh9n" event={"ID":"5e4a7a90-d6ce-491b-ad65-52f0b2a127d6","Type":"ContainerStarted","Data":"6b56978329597bbb55ed95ca635eff6d04c8af3d72c9a616e8e1c8a231e3a4ba"}
Apr 24 14:24:17.874687 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:24:17.874665    2574 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-bt99l" event={"ID":"0b8fb5e7-eaa1-4b51-9f84-37bb0eb1a711","Type":"ContainerStarted","Data":"84de38caa1042e2c25386a67046bd490578fe1c3c9144f3b6e873bab4b4d63ee"}
Apr 24 14:24:17.880973 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:24:17.880912    2574 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-cluster-node-tuning-operator/tuned-4kqkj" podStartSLOduration=4.233102563 podStartE2EDuration="20.880898693s" podCreationTimestamp="2026-04-24 14:23:57 +0000 UTC" firstStartedPulling="2026-04-24 14:24:00.39999533 +0000 UTC m=+3.132096957" lastFinishedPulling="2026-04-24 14:24:17.047791446 +0000 UTC m=+19.779893087" observedRunningTime="2026-04-24 14:24:17.880746541 +0000 UTC m=+20.612848191" watchObservedRunningTime="2026-04-24 14:24:17.880898693 +0000 UTC m=+20.613000344"
Apr 24 14:24:17.893352 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:24:17.893295    2574 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-dns/node-resolver-bt99l" podStartSLOduration=4.192979041 podStartE2EDuration="20.893281125s" podCreationTimestamp="2026-04-24 14:23:57 +0000 UTC" firstStartedPulling="2026-04-24 14:24:00.403891728 +0000 UTC m=+3.135993355" lastFinishedPulling="2026-04-24 14:24:17.104193808 +0000 UTC m=+19.836295439" observedRunningTime="2026-04-24 14:24:17.892808074 +0000 UTC m=+20.624909737" watchObservedRunningTime="2026-04-24 14:24:17.893281125 +0000 UTC m=+20.625382773"
Apr 24 14:24:17.930165 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:24:17.930118    2574 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-multus/multus-rwdxf" podStartSLOduration=4.23343858 podStartE2EDuration="20.930102945s" podCreationTimestamp="2026-04-24 14:23:57 +0000 UTC" firstStartedPulling="2026-04-24 14:24:00.4092181 +0000 UTC m=+3.141319738" lastFinishedPulling="2026-04-24 14:24:17.105882475 +0000 UTC m=+19.837984103" observedRunningTime="2026-04-24 14:24:17.929493511 +0000 UTC m=+20.661595185" watchObservedRunningTime="2026-04-24 14:24:17.930102945 +0000 UTC m=+20.662204595"
Apr 24 14:24:17.944665 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:24:17.944622    2574 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="kube-system/konnectivity-agent-zz678" podStartSLOduration=4.305956069 podStartE2EDuration="20.944608049s" podCreationTimestamp="2026-04-24 14:23:57 +0000 UTC" firstStartedPulling="2026-04-24 14:24:00.411923296 +0000 UTC m=+3.144024937" lastFinishedPulling="2026-04-24 14:24:17.050575279 +0000 UTC m=+19.782676917" observedRunningTime="2026-04-24 14:24:17.944256412 +0000 UTC m=+20.676358073" watchObservedRunningTime="2026-04-24 14:24:17.944608049 +0000 UTC m=+20.676709698"
Apr 24 14:24:17.958000 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:24:17.957933    2574 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-image-registry/node-ca-lgh9n" podStartSLOduration=4.319586646 podStartE2EDuration="20.957913912s" podCreationTimestamp="2026-04-24 14:23:57 +0000 UTC" firstStartedPulling="2026-04-24 14:24:00.409645479 +0000 UTC m=+3.141747112" lastFinishedPulling="2026-04-24 14:24:17.047972751 +0000 UTC m=+19.780074378" observedRunningTime="2026-04-24 14:24:17.957498993 +0000 UTC m=+20.689600642" watchObservedRunningTime="2026-04-24 14:24:17.957913912 +0000 UTC m=+20.690015562"
Apr 24 14:24:18.129358 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:24:18.129313    2574 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/5e2a3789-4ab2-4df5-bbbf-e649d90f15a5-original-pull-secret\") pod \"global-pull-secret-syncer-nwx74\" (UID: \"5e2a3789-4ab2-4df5-bbbf-e649d90f15a5\") " pod="kube-system/global-pull-secret-syncer-nwx74"
Apr 24 14:24:18.129501 ip-10-0-136-132 kubenswrapper[2574]: E0424 14:24:18.129462    2574 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Apr 24 14:24:18.129555 ip-10-0-136-132 kubenswrapper[2574]: E0424 14:24:18.129534    2574 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/5e2a3789-4ab2-4df5-bbbf-e649d90f15a5-original-pull-secret podName:5e2a3789-4ab2-4df5-bbbf-e649d90f15a5 nodeName:}" failed. No retries permitted until 2026-04-24 14:24:22.129516795 +0000 UTC m=+24.861618422 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/5e2a3789-4ab2-4df5-bbbf-e649d90f15a5-original-pull-secret") pod "global-pull-secret-syncer-nwx74" (UID: "5e2a3789-4ab2-4df5-bbbf-e649d90f15a5") : object "kube-system"/"original-pull-secret" not registered
Apr 24 14:24:18.328731 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:24:18.328706    2574 plugin_watcher.go:194] "Adding socket path or updating timestamp to desired state cache" path="/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock"
Apr 24 14:24:18.374418 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:24:18.374385    2574 kubelet.go:2658] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="kube-system/konnectivity-agent-zz678"
Apr 24 14:24:18.375037 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:24:18.374983    2574 kubelet.go:2658] "SyncLoop (probe)" probe="startup" status="started" pod="kube-system/konnectivity-agent-zz678"
Apr 24 14:24:18.736680 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:24:18.736570    2574 reconciler.go:161] "OperationExecutor.RegisterPlugin started" plugin={"SocketPath":"/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock","Timestamp":"2026-04-24T14:24:18.328724415Z","UUID":"6c234c6d-5f8f-45d8-98cc-bcb74ffe5089","Handler":null,"Name":"","Endpoint":""}
Apr 24 14:24:18.738637 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:24:18.738613    2574 csi_plugin.go:106] kubernetes.io/csi: Trying to validate a new CSI Driver with name: ebs.csi.aws.com endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock versions: 1.0.0
Apr 24 14:24:18.738797 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:24:18.738648    2574 csi_plugin.go:119] kubernetes.io/csi: Register new plugin with name: ebs.csi.aws.com at endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock
Apr 24 14:24:18.766141 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:24:18.766112    2574 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-nwx74"
Apr 24 14:24:18.766307 ip-10-0-136-132 kubenswrapper[2574]: E0424 14:24:18.766234    2574 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-nwx74" podUID="5e2a3789-4ab2-4df5-bbbf-e649d90f15a5"
Apr 24 14:24:18.878957 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:24:18.878921    2574 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-r2pxq" event={"ID":"95f221a8-f16d-4d5f-9638-29db9dd3a969","Type":"ContainerStarted","Data":"550538a30f2f84d190b3b8357ece4d0c7dbc2af923acf2a936afd473bb771b85"}
Apr 24 14:24:18.881806 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:24:18.881769    2574 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-6mgkm" event={"ID":"97869a65-fbd6-4a1b-9eac-7adc13e4c254","Type":"ContainerStarted","Data":"841358fc74933c82f070acd850d5409d5089b9df6c2c3b29173c108c09da3e6f"}
Apr 24 14:24:18.882720 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:24:18.882701    2574 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kube-system/konnectivity-agent-zz678"
Apr 24 14:24:18.883311 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:24:18.883283    2574 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="kube-system/konnectivity-agent-zz678"
Apr 24 14:24:18.892695 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:24:18.892509    2574 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-network-operator/iptables-alerter-r2pxq" podStartSLOduration=5.202322433 podStartE2EDuration="21.89249454s" podCreationTimestamp="2026-04-24 14:23:57 +0000 UTC" firstStartedPulling="2026-04-24 14:24:00.404016659 +0000 UTC m=+3.136118291" lastFinishedPulling="2026-04-24 14:24:17.094188767 +0000 UTC m=+19.826290398" observedRunningTime="2026-04-24 14:24:18.89237469 +0000 UTC m=+21.624476339" watchObservedRunningTime="2026-04-24 14:24:18.89249454 +0000 UTC m=+21.624596200"
Apr 24 14:24:19.765701 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:24:19.765670    2574 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-6tsjh"
Apr 24 14:24:19.765846 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:24:19.765701    2574 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-zhkmq"
Apr 24 14:24:19.765846 ip-10-0-136-132 kubenswrapper[2574]: E0424 14:24:19.765808    2574 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-6tsjh" podUID="3964207c-9c58-4994-a8a9-aa4e4feca773"
Apr 24 14:24:19.765961 ip-10-0-136-132 kubenswrapper[2574]: E0424 14:24:19.765935    2574 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-zhkmq" podUID="3bc44c73-963f-4a9e-8a52-1e8b031a08dc"
Apr 24 14:24:19.886816 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:24:19.886784    2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-w5x5x_44036c3e-1fdc-40d4-899c-9009cd4589e3/ovn-acl-logging/0.log"
Apr 24 14:24:19.887521 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:24:19.887179    2574 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-w5x5x" event={"ID":"44036c3e-1fdc-40d4-899c-9009cd4589e3","Type":"ContainerStarted","Data":"9b6b73fb26c5d3c80a8fa7050bf997256a4af9a706fe12950a288a5c4dd83f49"}
Apr 24 14:24:19.889311 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:24:19.889279    2574 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-6mgkm" event={"ID":"97869a65-fbd6-4a1b-9eac-7adc13e4c254","Type":"ContainerStarted","Data":"928f1047d24417d9b526df34809e716e680fa4bb0e014a6eb19f7b24fd6b015f"}
Apr 24 14:24:19.906266 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:24:19.906214    2574 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-6mgkm" podStartSLOduration=4.166916972 podStartE2EDuration="22.90619458s" podCreationTimestamp="2026-04-24 14:23:57 +0000 UTC" firstStartedPulling="2026-04-24 14:24:00.412483643 +0000 UTC m=+3.144585270" lastFinishedPulling="2026-04-24 14:24:19.151761239 +0000 UTC m=+21.883862878" observedRunningTime="2026-04-24 14:24:19.905624876 +0000 UTC m=+22.637726525" watchObservedRunningTime="2026-04-24 14:24:19.90619458 +0000 UTC m=+22.638296230"
Apr 24 14:24:20.765449 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:24:20.765414    2574 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-nwx74"
Apr 24 14:24:20.765634 ip-10-0-136-132 kubenswrapper[2574]: E0424 14:24:20.765549    2574 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-nwx74" podUID="5e2a3789-4ab2-4df5-bbbf-e649d90f15a5"
Apr 24 14:24:21.765827 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:24:21.765791    2574 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-6tsjh"
Apr 24 14:24:21.765827 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:24:21.765823    2574 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-zhkmq"
Apr 24 14:24:21.766495 ip-10-0-136-132 kubenswrapper[2574]: E0424 14:24:21.765908    2574 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-6tsjh" podUID="3964207c-9c58-4994-a8a9-aa4e4feca773"
Apr 24 14:24:21.766495 ip-10-0-136-132 kubenswrapper[2574]: E0424 14:24:21.766368    2574 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-zhkmq" podUID="3bc44c73-963f-4a9e-8a52-1e8b031a08dc"
Apr 24 14:24:22.161149 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:24:22.160224    2574 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/5e2a3789-4ab2-4df5-bbbf-e649d90f15a5-original-pull-secret\") pod \"global-pull-secret-syncer-nwx74\" (UID: \"5e2a3789-4ab2-4df5-bbbf-e649d90f15a5\") " pod="kube-system/global-pull-secret-syncer-nwx74"
Apr 24 14:24:22.164072 ip-10-0-136-132 kubenswrapper[2574]: E0424 14:24:22.161491    2574 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Apr 24 14:24:22.164072 ip-10-0-136-132 kubenswrapper[2574]: E0424 14:24:22.161583    2574 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/5e2a3789-4ab2-4df5-bbbf-e649d90f15a5-original-pull-secret podName:5e2a3789-4ab2-4df5-bbbf-e649d90f15a5 nodeName:}" failed. No retries permitted until 2026-04-24 14:24:30.161552656 +0000 UTC m=+32.893654296 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/5e2a3789-4ab2-4df5-bbbf-e649d90f15a5-original-pull-secret") pod "global-pull-secret-syncer-nwx74" (UID: "5e2a3789-4ab2-4df5-bbbf-e649d90f15a5") : object "kube-system"/"original-pull-secret" not registered
Apr 24 14:24:22.765389 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:24:22.765357    2574 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-nwx74"
Apr 24 14:24:22.765583 ip-10-0-136-132 kubenswrapper[2574]: E0424 14:24:22.765454    2574 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-nwx74" podUID="5e2a3789-4ab2-4df5-bbbf-e649d90f15a5"
Apr 24 14:24:22.895974 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:24:22.895943    2574 generic.go:358] "Generic (PLEG): container finished" podID="c9814dbf-8db1-4d4a-9d40-d7a0fdc96b26" containerID="4f0cdd2cdb52bc0c5f494a4df04b0653c05784bd3c3282bcbe2fbdaac6a0c54d" exitCode=0
Apr 24 14:24:22.896440 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:24:22.896030    2574 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-j2jc7" event={"ID":"c9814dbf-8db1-4d4a-9d40-d7a0fdc96b26","Type":"ContainerDied","Data":"4f0cdd2cdb52bc0c5f494a4df04b0653c05784bd3c3282bcbe2fbdaac6a0c54d"}
Apr 24 14:24:22.899146 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:24:22.899129    2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-w5x5x_44036c3e-1fdc-40d4-899c-9009cd4589e3/ovn-acl-logging/0.log"
Apr 24 14:24:22.899504 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:24:22.899485    2574 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-w5x5x" event={"ID":"44036c3e-1fdc-40d4-899c-9009cd4589e3","Type":"ContainerStarted","Data":"e00bd16f5cfb15f531e95c4ee8efba7279e2260f8f74ba8c9ae4cf999038702d"}
Apr 24 14:24:22.899790 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:24:22.899774    2574 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-w5x5x"
Apr 24 14:24:22.899852 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:24:22.899797    2574 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-w5x5x"
Apr 24 14:24:22.899974 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:24:22.899955    2574 scope.go:117] "RemoveContainer" containerID="2154270a235264b607e9c06815df67b3a01c006641f78800b2ac8ac8d578e4bf"
Apr 24 14:24:22.915586 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:24:22.915480    2574 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-w5x5x"
Apr 24 14:24:23.767969 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:24:23.767805    2574 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-6tsjh"
Apr 24 14:24:23.768181 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:24:23.767809    2574 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-zhkmq"
Apr 24 14:24:23.768181 ip-10-0-136-132 kubenswrapper[2574]: E0424 14:24:23.768029    2574 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-6tsjh" podUID="3964207c-9c58-4994-a8a9-aa4e4feca773"
Apr 24 14:24:23.768181 ip-10-0-136-132 kubenswrapper[2574]: E0424 14:24:23.768107    2574 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-zhkmq" podUID="3bc44c73-963f-4a9e-8a52-1e8b031a08dc"
Apr 24 14:24:23.854355 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:24:23.854313    2574 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-zhkmq"]
Apr 24 14:24:23.861609 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:24:23.861581    2574 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-6tsjh"]
Apr 24 14:24:23.862226 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:24:23.862196    2574 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-nwx74"]
Apr 24 14:24:23.862367 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:24:23.862330    2574 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-nwx74"
Apr 24 14:24:23.862489 ip-10-0-136-132 kubenswrapper[2574]: E0424 14:24:23.862464    2574 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-nwx74" podUID="5e2a3789-4ab2-4df5-bbbf-e649d90f15a5"
Apr 24 14:24:23.905758 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:24:23.905694    2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-w5x5x_44036c3e-1fdc-40d4-899c-9009cd4589e3/ovn-acl-logging/0.log"
Apr 24 14:24:23.906103 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:24:23.906046    2574 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-6tsjh"
Apr 24 14:24:23.906144 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:24:23.906037    2574 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-w5x5x" event={"ID":"44036c3e-1fdc-40d4-899c-9009cd4589e3","Type":"ContainerStarted","Data":"a4fd8ebd284f28885afe2b3962882d1ccc9ae984b50d1f93d127e0b486ba89b0"}
Apr 24 14:24:23.906177 ip-10-0-136-132 kubenswrapper[2574]: E0424 14:24:23.906153    2574 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-6tsjh" podUID="3964207c-9c58-4994-a8a9-aa4e4feca773"
Apr 24 14:24:23.906377 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:24:23.906319    2574 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-w5x5x"
Apr 24 14:24:23.906377 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:24:23.906362    2574 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-zhkmq"
Apr 24 14:24:23.906873 ip-10-0-136-132 kubenswrapper[2574]: E0424 14:24:23.906839    2574 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-zhkmq" podUID="3bc44c73-963f-4a9e-8a52-1e8b031a08dc"
Apr 24 14:24:23.921443 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:24:23.921419    2574 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-w5x5x"
Apr 24 14:24:23.938896 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:24:23.938853    2574 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-ovn-kubernetes/ovnkube-node-w5x5x" podStartSLOduration=10.187543058 podStartE2EDuration="26.93884152s" podCreationTimestamp="2026-04-24 14:23:57 +0000 UTC" firstStartedPulling="2026-04-24 14:24:00.410990391 +0000 UTC m=+3.143092027" lastFinishedPulling="2026-04-24 14:24:17.162288859 +0000 UTC m=+19.894390489" observedRunningTime="2026-04-24 14:24:23.938238709 +0000 UTC m=+26.670340368" watchObservedRunningTime="2026-04-24 14:24:23.93884152 +0000 UTC m=+26.670943163"
Apr 24 14:24:24.909905 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:24:24.909871    2574 generic.go:358] "Generic (PLEG): container finished" podID="c9814dbf-8db1-4d4a-9d40-d7a0fdc96b26" containerID="d2c928082b615b4b68725b78f28703c4cc17a8880576fc6a670a6bd45b8f2426" exitCode=0
Apr 24 14:24:24.910473 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:24:24.909956    2574 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-j2jc7" event={"ID":"c9814dbf-8db1-4d4a-9d40-d7a0fdc96b26","Type":"ContainerDied","Data":"d2c928082b615b4b68725b78f28703c4cc17a8880576fc6a670a6bd45b8f2426"}
Apr 24 14:24:25.766005 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:24:25.765967    2574 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-zhkmq"
Apr 24 14:24:25.766005 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:24:25.765996    2574 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-6tsjh"
Apr 24 14:24:25.766203 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:24:25.765995    2574 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-nwx74"
Apr 24 14:24:25.766203 ip-10-0-136-132 kubenswrapper[2574]: E0424 14:24:25.766084    2574 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-zhkmq" podUID="3bc44c73-963f-4a9e-8a52-1e8b031a08dc"
Apr 24 14:24:25.768930 ip-10-0-136-132 kubenswrapper[2574]: E0424 14:24:25.766538    2574 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-6tsjh" podUID="3964207c-9c58-4994-a8a9-aa4e4feca773"
Apr 24 14:24:25.768930 ip-10-0-136-132 kubenswrapper[2574]: E0424 14:24:25.766712    2574 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-nwx74" podUID="5e2a3789-4ab2-4df5-bbbf-e649d90f15a5"
Apr 24 14:24:26.915350 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:24:26.915244    2574 generic.go:358] "Generic (PLEG): container finished" podID="c9814dbf-8db1-4d4a-9d40-d7a0fdc96b26" containerID="d350e828c67fda167086f3f31a6f2f20173e3ff84b938a0074b942742c73085c" exitCode=0
Apr 24 14:24:26.915350 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:24:26.915296    2574 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-j2jc7" event={"ID":"c9814dbf-8db1-4d4a-9d40-d7a0fdc96b26","Type":"ContainerDied","Data":"d350e828c67fda167086f3f31a6f2f20173e3ff84b938a0074b942742c73085c"}
Apr 24 14:24:27.769266 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:24:27.769171    2574 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-6tsjh"
Apr 24 14:24:27.769266 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:24:27.769213    2574 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-nwx74"
Apr 24 14:24:27.769560 ip-10-0-136-132 kubenswrapper[2574]: E0424 14:24:27.769420    2574 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-nwx74" podUID="5e2a3789-4ab2-4df5-bbbf-e649d90f15a5"
Apr 24 14:24:27.769560 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:24:27.769476    2574 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-zhkmq"
Apr 24 14:24:27.769750 ip-10-0-136-132 kubenswrapper[2574]: E0424 14:24:27.769563    2574 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-zhkmq" podUID="3bc44c73-963f-4a9e-8a52-1e8b031a08dc"
Apr 24 14:24:27.769750 ip-10-0-136-132 kubenswrapper[2574]: E0424 14:24:27.769312    2574 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-6tsjh" podUID="3964207c-9c58-4994-a8a9-aa4e4feca773"
Apr 24 14:24:29.765739 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:24:29.765708    2574 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-6tsjh"
Apr 24 14:24:29.765739 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:24:29.765730    2574 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-zhkmq"
Apr 24 14:24:29.766284 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:24:29.765730    2574 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-nwx74"
Apr 24 14:24:29.766284 ip-10-0-136-132 kubenswrapper[2574]: E0424 14:24:29.765805    2574 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-6tsjh" podUID="3964207c-9c58-4994-a8a9-aa4e4feca773"
Apr 24 14:24:29.766284 ip-10-0-136-132 kubenswrapper[2574]: E0424 14:24:29.765900    2574 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-zhkmq" podUID="3bc44c73-963f-4a9e-8a52-1e8b031a08dc"
Apr 24 14:24:29.766284 ip-10-0-136-132 kubenswrapper[2574]: E0424 14:24:29.765965    2574 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-nwx74" podUID="5e2a3789-4ab2-4df5-bbbf-e649d90f15a5"
Apr 24 14:24:30.165298 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:24:30.165228    2574 kubelet_node_status.go:736] "Recording event message for node" node="ip-10-0-136-132.ec2.internal" event="NodeReady"
Apr 24 14:24:30.165462 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:24:30.165361    2574 kubelet_node_status.go:550] "Fast updating node status as it just became ready"
Apr 24 14:24:30.218653 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:24:30.218614    2574 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["openshift-ingress-canary/ingress-canary-tzbfv"]
Apr 24 14:24:30.227399 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:24:30.227365    2574 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/5e2a3789-4ab2-4df5-bbbf-e649d90f15a5-original-pull-secret\") pod \"global-pull-secret-syncer-nwx74\" (UID: \"5e2a3789-4ab2-4df5-bbbf-e649d90f15a5\") " pod="kube-system/global-pull-secret-syncer-nwx74"
Apr 24 14:24:30.227540 ip-10-0-136-132 kubenswrapper[2574]: E0424 14:24:30.227475    2574 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Apr 24 14:24:30.227540 ip-10-0-136-132 kubenswrapper[2574]: E0424 14:24:30.227523    2574 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/5e2a3789-4ab2-4df5-bbbf-e649d90f15a5-original-pull-secret podName:5e2a3789-4ab2-4df5-bbbf-e649d90f15a5 nodeName:}" failed. No retries permitted until 2026-04-24 14:24:46.227511108 +0000 UTC m=+48.959612734 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/5e2a3789-4ab2-4df5-bbbf-e649d90f15a5-original-pull-secret") pod "global-pull-secret-syncer-nwx74" (UID: "5e2a3789-4ab2-4df5-bbbf-e649d90f15a5") : object "kube-system"/"original-pull-secret" not registered
Apr 24 14:24:30.238952 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:24:30.238786    2574 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["openshift-dns/dns-default-nvz9x"]
Apr 24 14:24:30.239117 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:24:30.238927    2574 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-tzbfv"
Apr 24 14:24:30.240832 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:24:30.240807    2574 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"default-dockercfg-9v25p\""
Apr 24 14:24:30.241714 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:24:30.241687    2574 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"openshift-service-ca.crt\""
Apr 24 14:24:30.241714 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:24:30.241705    2574 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"canary-serving-cert\""
Apr 24 14:24:30.242589 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:24:30.242569    2574 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"kube-root-ca.crt\""
Apr 24 14:24:30.270066 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:24:30.270039    2574 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-tzbfv"]
Apr 24 14:24:30.270066 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:24:30.270066    2574 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-nvz9x"]
Apr 24 14:24:30.270269 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:24:30.270176    2574 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-nvz9x"
Apr 24 14:24:30.272388 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:24:30.272365    2574 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"dns-default\""
Apr 24 14:24:30.272532 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:24:30.272515    2574 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-6p8rb\""
Apr 24 14:24:30.272916 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:24:30.272896    2574 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-default-metrics-tls\""
Apr 24 14:24:30.328005 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:24:30.327969    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/4ed72386-1812-4697-9811-c3534a37dc95-cert\") pod \"ingress-canary-tzbfv\" (UID: \"4ed72386-1812-4697-9811-c3534a37dc95\") " pod="openshift-ingress-canary/ingress-canary-tzbfv"
Apr 24 14:24:30.328005 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:24:30.328005    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9c9rf\" (UniqueName: \"kubernetes.io/projected/4ed72386-1812-4697-9811-c3534a37dc95-kube-api-access-9c9rf\") pod \"ingress-canary-tzbfv\" (UID: \"4ed72386-1812-4697-9811-c3534a37dc95\") " pod="openshift-ingress-canary/ingress-canary-tzbfv"
Apr 24 14:24:30.428911 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:24:30.428798    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/b4b7957c-9f4f-4c2d-ab2b-85c958db734a-metrics-tls\") pod \"dns-default-nvz9x\" (UID: \"b4b7957c-9f4f-4c2d-ab2b-85c958db734a\") " pod="openshift-dns/dns-default-nvz9x"
Apr 24 14:24:30.428911 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:24:30.428840    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/b4b7957c-9f4f-4c2d-ab2b-85c958db734a-config-volume\") pod \"dns-default-nvz9x\" (UID: \"b4b7957c-9f4f-4c2d-ab2b-85c958db734a\") " pod="openshift-dns/dns-default-nvz9x"
Apr 24 14:24:30.429114 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:24:30.428921    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/b4b7957c-9f4f-4c2d-ab2b-85c958db734a-tmp-dir\") pod \"dns-default-nvz9x\" (UID: \"b4b7957c-9f4f-4c2d-ab2b-85c958db734a\") " pod="openshift-dns/dns-default-nvz9x"
Apr 24 14:24:30.429114 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:24:30.429015    2574 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/4ed72386-1812-4697-9811-c3534a37dc95-cert\") pod \"ingress-canary-tzbfv\" (UID: \"4ed72386-1812-4697-9811-c3534a37dc95\") " pod="openshift-ingress-canary/ingress-canary-tzbfv"
Apr 24 14:24:30.429114 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:24:30.429055    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-pcksd\" (UniqueName: \"kubernetes.io/projected/b4b7957c-9f4f-4c2d-ab2b-85c958db734a-kube-api-access-pcksd\") pod \"dns-default-nvz9x\" (UID: \"b4b7957c-9f4f-4c2d-ab2b-85c958db734a\") " pod="openshift-dns/dns-default-nvz9x"
Apr 24 14:24:30.429114 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:24:30.429087    2574 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-9c9rf\" (UniqueName: \"kubernetes.io/projected/4ed72386-1812-4697-9811-c3534a37dc95-kube-api-access-9c9rf\") pod \"ingress-canary-tzbfv\" (UID: \"4ed72386-1812-4697-9811-c3534a37dc95\") " pod="openshift-ingress-canary/ingress-canary-tzbfv"
Apr 24 14:24:30.429275 ip-10-0-136-132 kubenswrapper[2574]: E0424 14:24:30.429133    2574 secret.go:189] Couldn't get secret openshift-ingress-canary/canary-serving-cert: secret "canary-serving-cert" not found
Apr 24 14:24:30.429275 ip-10-0-136-132 kubenswrapper[2574]: E0424 14:24:30.429208    2574 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/4ed72386-1812-4697-9811-c3534a37dc95-cert podName:4ed72386-1812-4697-9811-c3534a37dc95 nodeName:}" failed. No retries permitted until 2026-04-24 14:24:30.929189327 +0000 UTC m=+33.661290969 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "cert" (UniqueName: "kubernetes.io/secret/4ed72386-1812-4697-9811-c3534a37dc95-cert") pod "ingress-canary-tzbfv" (UID: "4ed72386-1812-4697-9811-c3534a37dc95") : secret "canary-serving-cert" not found
Apr 24 14:24:30.439640 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:24:30.439610    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-9c9rf\" (UniqueName: \"kubernetes.io/projected/4ed72386-1812-4697-9811-c3534a37dc95-kube-api-access-9c9rf\") pod \"ingress-canary-tzbfv\" (UID: \"4ed72386-1812-4697-9811-c3534a37dc95\") " pod="openshift-ingress-canary/ingress-canary-tzbfv"
Apr 24 14:24:30.529648 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:24:30.529608    2574 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/b4b7957c-9f4f-4c2d-ab2b-85c958db734a-tmp-dir\") pod \"dns-default-nvz9x\" (UID: \"b4b7957c-9f4f-4c2d-ab2b-85c958db734a\") " pod="openshift-dns/dns-default-nvz9x"
Apr 24 14:24:30.529827 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:24:30.529701    2574 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-pcksd\" (UniqueName: \"kubernetes.io/projected/b4b7957c-9f4f-4c2d-ab2b-85c958db734a-kube-api-access-pcksd\") pod \"dns-default-nvz9x\" (UID: \"b4b7957c-9f4f-4c2d-ab2b-85c958db734a\") " pod="openshift-dns/dns-default-nvz9x"
Apr 24 14:24:30.529827 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:24:30.529741    2574 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/b4b7957c-9f4f-4c2d-ab2b-85c958db734a-metrics-tls\") pod \"dns-default-nvz9x\" (UID: \"b4b7957c-9f4f-4c2d-ab2b-85c958db734a\") " pod="openshift-dns/dns-default-nvz9x"
Apr 24 14:24:30.529827 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:24:30.529760    2574 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/b4b7957c-9f4f-4c2d-ab2b-85c958db734a-config-volume\") pod \"dns-default-nvz9x\" (UID: \"b4b7957c-9f4f-4c2d-ab2b-85c958db734a\") " pod="openshift-dns/dns-default-nvz9x"
Apr 24 14:24:30.529997 ip-10-0-136-132 kubenswrapper[2574]: E0424 14:24:30.529976    2574 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Apr 24 14:24:30.530046 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:24:30.530000    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/b4b7957c-9f4f-4c2d-ab2b-85c958db734a-tmp-dir\") pod \"dns-default-nvz9x\" (UID: \"b4b7957c-9f4f-4c2d-ab2b-85c958db734a\") " pod="openshift-dns/dns-default-nvz9x"
Apr 24 14:24:30.530087 ip-10-0-136-132 kubenswrapper[2574]: E0424 14:24:30.530050    2574 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b4b7957c-9f4f-4c2d-ab2b-85c958db734a-metrics-tls podName:b4b7957c-9f4f-4c2d-ab2b-85c958db734a nodeName:}" failed. No retries permitted until 2026-04-24 14:24:31.03003129 +0000 UTC m=+33.762132931 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/b4b7957c-9f4f-4c2d-ab2b-85c958db734a-metrics-tls") pod "dns-default-nvz9x" (UID: "b4b7957c-9f4f-4c2d-ab2b-85c958db734a") : secret "dns-default-metrics-tls" not found
Apr 24 14:24:30.530363 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:24:30.530330    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/b4b7957c-9f4f-4c2d-ab2b-85c958db734a-config-volume\") pod \"dns-default-nvz9x\" (UID: \"b4b7957c-9f4f-4c2d-ab2b-85c958db734a\") " pod="openshift-dns/dns-default-nvz9x"
Apr 24 14:24:30.538035 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:24:30.538010    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-pcksd\" (UniqueName: \"kubernetes.io/projected/b4b7957c-9f4f-4c2d-ab2b-85c958db734a-kube-api-access-pcksd\") pod \"dns-default-nvz9x\" (UID: \"b4b7957c-9f4f-4c2d-ab2b-85c958db734a\") " pod="openshift-dns/dns-default-nvz9x"
Apr 24 14:24:30.932884 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:24:30.932849    2574 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/4ed72386-1812-4697-9811-c3534a37dc95-cert\") pod \"ingress-canary-tzbfv\" (UID: \"4ed72386-1812-4697-9811-c3534a37dc95\") " pod="openshift-ingress-canary/ingress-canary-tzbfv"
Apr 24 14:24:30.933330 ip-10-0-136-132 kubenswrapper[2574]: E0424 14:24:30.933021    2574 secret.go:189] Couldn't get secret openshift-ingress-canary/canary-serving-cert: secret "canary-serving-cert" not found
Apr 24 14:24:30.933330 ip-10-0-136-132 kubenswrapper[2574]: E0424 14:24:30.933107    2574 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/4ed72386-1812-4697-9811-c3534a37dc95-cert podName:4ed72386-1812-4697-9811-c3534a37dc95 nodeName:}" failed. No retries permitted until 2026-04-24 14:24:31.933085906 +0000 UTC m=+34.665187536 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "cert" (UniqueName: "kubernetes.io/secret/4ed72386-1812-4697-9811-c3534a37dc95-cert") pod "ingress-canary-tzbfv" (UID: "4ed72386-1812-4697-9811-c3534a37dc95") : secret "canary-serving-cert" not found
Apr 24 14:24:31.033321 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:24:31.033278    2574 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/b4b7957c-9f4f-4c2d-ab2b-85c958db734a-metrics-tls\") pod \"dns-default-nvz9x\" (UID: \"b4b7957c-9f4f-4c2d-ab2b-85c958db734a\") " pod="openshift-dns/dns-default-nvz9x"
Apr 24 14:24:31.033519 ip-10-0-136-132 kubenswrapper[2574]: E0424 14:24:31.033454    2574 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Apr 24 14:24:31.033577 ip-10-0-136-132 kubenswrapper[2574]: E0424 14:24:31.033530    2574 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b4b7957c-9f4f-4c2d-ab2b-85c958db734a-metrics-tls podName:b4b7957c-9f4f-4c2d-ab2b-85c958db734a nodeName:}" failed. No retries permitted until 2026-04-24 14:24:32.033509906 +0000 UTC m=+34.765611535 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/b4b7957c-9f4f-4c2d-ab2b-85c958db734a-metrics-tls") pod "dns-default-nvz9x" (UID: "b4b7957c-9f4f-4c2d-ab2b-85c958db734a") : secret "dns-default-metrics-tls" not found
Apr 24 14:24:31.436434 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:24:31.436394    2574 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/3bc44c73-963f-4a9e-8a52-1e8b031a08dc-metrics-certs\") pod \"network-metrics-daemon-zhkmq\" (UID: \"3bc44c73-963f-4a9e-8a52-1e8b031a08dc\") " pod="openshift-multus/network-metrics-daemon-zhkmq"
Apr 24 14:24:31.436628 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:24:31.436465    2574 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-css2f\" (UniqueName: \"kubernetes.io/projected/3964207c-9c58-4994-a8a9-aa4e4feca773-kube-api-access-css2f\") pod \"network-check-target-6tsjh\" (UID: \"3964207c-9c58-4994-a8a9-aa4e4feca773\") " pod="openshift-network-diagnostics/network-check-target-6tsjh"
Apr 24 14:24:31.436628 ip-10-0-136-132 kubenswrapper[2574]: E0424 14:24:31.436581    2574 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Apr 24 14:24:31.436628 ip-10-0-136-132 kubenswrapper[2574]: E0424 14:24:31.436602    2574 projected.go:289] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Apr 24 14:24:31.436628 ip-10-0-136-132 kubenswrapper[2574]: E0424 14:24:31.436624    2574 projected.go:289] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Apr 24 14:24:31.436832 ip-10-0-136-132 kubenswrapper[2574]: E0424 14:24:31.436638    2574 projected.go:194] Error preparing data for projected volume kube-api-access-css2f for pod openshift-network-diagnostics/network-check-target-6tsjh: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Apr 24 14:24:31.436832 ip-10-0-136-132 kubenswrapper[2574]: E0424 14:24:31.436693    2574 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/3bc44c73-963f-4a9e-8a52-1e8b031a08dc-metrics-certs podName:3bc44c73-963f-4a9e-8a52-1e8b031a08dc nodeName:}" failed. No retries permitted until 2026-04-24 14:25:03.436670535 +0000 UTC m=+66.168772162 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/3bc44c73-963f-4a9e-8a52-1e8b031a08dc-metrics-certs") pod "network-metrics-daemon-zhkmq" (UID: "3bc44c73-963f-4a9e-8a52-1e8b031a08dc") : object "openshift-multus"/"metrics-daemon-secret" not registered
Apr 24 14:24:31.436832 ip-10-0-136-132 kubenswrapper[2574]: E0424 14:24:31.436718    2574 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/3964207c-9c58-4994-a8a9-aa4e4feca773-kube-api-access-css2f podName:3964207c-9c58-4994-a8a9-aa4e4feca773 nodeName:}" failed. No retries permitted until 2026-04-24 14:25:03.436706005 +0000 UTC m=+66.168807633 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "kube-api-access-css2f" (UniqueName: "kubernetes.io/projected/3964207c-9c58-4994-a8a9-aa4e4feca773-kube-api-access-css2f") pod "network-check-target-6tsjh" (UID: "3964207c-9c58-4994-a8a9-aa4e4feca773") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Apr 24 14:24:31.765758 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:24:31.765665    2574 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-6tsjh"
Apr 24 14:24:31.765920 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:24:31.765665    2574 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-nwx74"
Apr 24 14:24:31.765982 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:24:31.765952    2574 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-zhkmq"
Apr 24 14:24:31.768074 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:24:31.767869    2574 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\""
Apr 24 14:24:31.768074 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:24:31.767900    2574 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\""
Apr 24 14:24:31.768260 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:24:31.768234    2574 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\""
Apr 24 14:24:31.768306 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:24:31.768260    2574 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-4q5p5\""
Apr 24 14:24:31.768374 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:24:31.768319    2574 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-nctch\""
Apr 24 14:24:31.768374 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:24:31.768349    2574 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\""
Apr 24 14:24:31.939642 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:24:31.939593    2574 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/4ed72386-1812-4697-9811-c3534a37dc95-cert\") pod \"ingress-canary-tzbfv\" (UID: \"4ed72386-1812-4697-9811-c3534a37dc95\") " pod="openshift-ingress-canary/ingress-canary-tzbfv"
Apr 24 14:24:31.940102 ip-10-0-136-132 kubenswrapper[2574]: E0424 14:24:31.939723    2574 secret.go:189] Couldn't get secret openshift-ingress-canary/canary-serving-cert: secret "canary-serving-cert" not found
Apr 24 14:24:31.940102 ip-10-0-136-132 kubenswrapper[2574]: E0424 14:24:31.939805    2574 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/4ed72386-1812-4697-9811-c3534a37dc95-cert podName:4ed72386-1812-4697-9811-c3534a37dc95 nodeName:}" failed. No retries permitted until 2026-04-24 14:24:33.939780844 +0000 UTC m=+36.671882484 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "cert" (UniqueName: "kubernetes.io/secret/4ed72386-1812-4697-9811-c3534a37dc95-cert") pod "ingress-canary-tzbfv" (UID: "4ed72386-1812-4697-9811-c3534a37dc95") : secret "canary-serving-cert" not found
Apr 24 14:24:32.040579 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:24:32.040489    2574 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/b4b7957c-9f4f-4c2d-ab2b-85c958db734a-metrics-tls\") pod \"dns-default-nvz9x\" (UID: \"b4b7957c-9f4f-4c2d-ab2b-85c958db734a\") " pod="openshift-dns/dns-default-nvz9x"
Apr 24 14:24:32.040756 ip-10-0-136-132 kubenswrapper[2574]: E0424 14:24:32.040638    2574 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Apr 24 14:24:32.040756 ip-10-0-136-132 kubenswrapper[2574]: E0424 14:24:32.040730    2574 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b4b7957c-9f4f-4c2d-ab2b-85c958db734a-metrics-tls podName:b4b7957c-9f4f-4c2d-ab2b-85c958db734a nodeName:}" failed. No retries permitted until 2026-04-24 14:24:34.040709849 +0000 UTC m=+36.772811482 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/b4b7957c-9f4f-4c2d-ab2b-85c958db734a-metrics-tls") pod "dns-default-nvz9x" (UID: "b4b7957c-9f4f-4c2d-ab2b-85c958db734a") : secret "dns-default-metrics-tls" not found
Apr 24 14:24:32.929594 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:24:32.929551    2574 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-j2jc7" event={"ID":"c9814dbf-8db1-4d4a-9d40-d7a0fdc96b26","Type":"ContainerStarted","Data":"c341af4b242bd356ffa54cd9ebe073beb2bc4f0b5af03fae4ed796630bbeb5e1"}
Apr 24 14:24:33.933496 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:24:33.933456    2574 generic.go:358] "Generic (PLEG): container finished" podID="c9814dbf-8db1-4d4a-9d40-d7a0fdc96b26" containerID="c341af4b242bd356ffa54cd9ebe073beb2bc4f0b5af03fae4ed796630bbeb5e1" exitCode=0
Apr 24 14:24:33.933865 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:24:33.933505    2574 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-j2jc7" event={"ID":"c9814dbf-8db1-4d4a-9d40-d7a0fdc96b26","Type":"ContainerDied","Data":"c341af4b242bd356ffa54cd9ebe073beb2bc4f0b5af03fae4ed796630bbeb5e1"}
Apr 24 14:24:33.952441 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:24:33.952282    2574 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/4ed72386-1812-4697-9811-c3534a37dc95-cert\") pod \"ingress-canary-tzbfv\" (UID: \"4ed72386-1812-4697-9811-c3534a37dc95\") " pod="openshift-ingress-canary/ingress-canary-tzbfv"
Apr 24 14:24:33.952441 ip-10-0-136-132 kubenswrapper[2574]: E0424 14:24:33.952422    2574 secret.go:189] Couldn't get secret openshift-ingress-canary/canary-serving-cert: secret "canary-serving-cert" not found
Apr 24 14:24:33.952530 ip-10-0-136-132 kubenswrapper[2574]: E0424 14:24:33.952478    2574 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/4ed72386-1812-4697-9811-c3534a37dc95-cert podName:4ed72386-1812-4697-9811-c3534a37dc95 nodeName:}" failed. No retries permitted until 2026-04-24 14:24:37.95246265 +0000 UTC m=+40.684564276 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "cert" (UniqueName: "kubernetes.io/secret/4ed72386-1812-4697-9811-c3534a37dc95-cert") pod "ingress-canary-tzbfv" (UID: "4ed72386-1812-4697-9811-c3534a37dc95") : secret "canary-serving-cert" not found
Apr 24 14:24:34.053301 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:24:34.053267    2574 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/b4b7957c-9f4f-4c2d-ab2b-85c958db734a-metrics-tls\") pod \"dns-default-nvz9x\" (UID: \"b4b7957c-9f4f-4c2d-ab2b-85c958db734a\") " pod="openshift-dns/dns-default-nvz9x"
Apr 24 14:24:34.053448 ip-10-0-136-132 kubenswrapper[2574]: E0424 14:24:34.053428    2574 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Apr 24 14:24:34.053536 ip-10-0-136-132 kubenswrapper[2574]: E0424 14:24:34.053522    2574 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b4b7957c-9f4f-4c2d-ab2b-85c958db734a-metrics-tls podName:b4b7957c-9f4f-4c2d-ab2b-85c958db734a nodeName:}" failed. No retries permitted until 2026-04-24 14:24:38.05349842 +0000 UTC m=+40.785600051 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/b4b7957c-9f4f-4c2d-ab2b-85c958db734a-metrics-tls") pod "dns-default-nvz9x" (UID: "b4b7957c-9f4f-4c2d-ab2b-85c958db734a") : secret "dns-default-metrics-tls" not found
Apr 24 14:24:34.938288 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:24:34.938250    2574 generic.go:358] "Generic (PLEG): container finished" podID="c9814dbf-8db1-4d4a-9d40-d7a0fdc96b26" containerID="c236baf78e1aaf9f1ced477007fe857663a71d57b9c0d36a0c79dcf60865c363" exitCode=0
Apr 24 14:24:34.938669 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:24:34.938315    2574 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-j2jc7" event={"ID":"c9814dbf-8db1-4d4a-9d40-d7a0fdc96b26","Type":"ContainerDied","Data":"c236baf78e1aaf9f1ced477007fe857663a71d57b9c0d36a0c79dcf60865c363"}
Apr 24 14:24:35.943112 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:24:35.942928    2574 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-j2jc7" event={"ID":"c9814dbf-8db1-4d4a-9d40-d7a0fdc96b26","Type":"ContainerStarted","Data":"225db0424172e028b7eb70c6712209f287e434867bf5487d7e073c2dad1906d4"}
Apr 24 14:24:35.964327 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:24:35.964275    2574 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-multus/multus-additional-cni-plugins-j2jc7" podStartSLOduration=6.672253752 podStartE2EDuration="38.964258796s" podCreationTimestamp="2026-04-24 14:23:57 +0000 UTC" firstStartedPulling="2026-04-24 14:24:00.408485546 +0000 UTC m=+3.140587188" lastFinishedPulling="2026-04-24 14:24:32.700490606 +0000 UTC m=+35.432592232" observedRunningTime="2026-04-24 14:24:35.963204353 +0000 UTC m=+38.695306002" watchObservedRunningTime="2026-04-24 14:24:35.964258796 +0000 UTC m=+38.696360445"
Apr 24 14:24:37.980252 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:24:37.980211    2574 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/4ed72386-1812-4697-9811-c3534a37dc95-cert\") pod \"ingress-canary-tzbfv\" (UID: \"4ed72386-1812-4697-9811-c3534a37dc95\") " pod="openshift-ingress-canary/ingress-canary-tzbfv"
Apr 24 14:24:37.980642 ip-10-0-136-132 kubenswrapper[2574]: E0424 14:24:37.980348    2574 secret.go:189] Couldn't get secret openshift-ingress-canary/canary-serving-cert: secret "canary-serving-cert" not found
Apr 24 14:24:37.980642 ip-10-0-136-132 kubenswrapper[2574]: E0424 14:24:37.980399    2574 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/4ed72386-1812-4697-9811-c3534a37dc95-cert podName:4ed72386-1812-4697-9811-c3534a37dc95 nodeName:}" failed. No retries permitted until 2026-04-24 14:24:45.980385685 +0000 UTC m=+48.712487312 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "cert" (UniqueName: "kubernetes.io/secret/4ed72386-1812-4697-9811-c3534a37dc95-cert") pod "ingress-canary-tzbfv" (UID: "4ed72386-1812-4697-9811-c3534a37dc95") : secret "canary-serving-cert" not found
Apr 24 14:24:38.081372 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:24:38.081322    2574 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/b4b7957c-9f4f-4c2d-ab2b-85c958db734a-metrics-tls\") pod \"dns-default-nvz9x\" (UID: \"b4b7957c-9f4f-4c2d-ab2b-85c958db734a\") " pod="openshift-dns/dns-default-nvz9x"
Apr 24 14:24:38.081505 ip-10-0-136-132 kubenswrapper[2574]: E0424 14:24:38.081467    2574 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Apr 24 14:24:38.081544 ip-10-0-136-132 kubenswrapper[2574]: E0424 14:24:38.081525    2574 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b4b7957c-9f4f-4c2d-ab2b-85c958db734a-metrics-tls podName:b4b7957c-9f4f-4c2d-ab2b-85c958db734a nodeName:}" failed. No retries permitted until 2026-04-24 14:24:46.081509932 +0000 UTC m=+48.813611564 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/b4b7957c-9f4f-4c2d-ab2b-85c958db734a-metrics-tls") pod "dns-default-nvz9x" (UID: "b4b7957c-9f4f-4c2d-ab2b-85c958db734a") : secret "dns-default-metrics-tls" not found
Apr 24 14:24:46.033380 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:24:46.033319    2574 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/4ed72386-1812-4697-9811-c3534a37dc95-cert\") pod \"ingress-canary-tzbfv\" (UID: \"4ed72386-1812-4697-9811-c3534a37dc95\") " pod="openshift-ingress-canary/ingress-canary-tzbfv"
Apr 24 14:24:46.033776 ip-10-0-136-132 kubenswrapper[2574]: E0424 14:24:46.033469    2574 secret.go:189] Couldn't get secret openshift-ingress-canary/canary-serving-cert: secret "canary-serving-cert" not found
Apr 24 14:24:46.033776 ip-10-0-136-132 kubenswrapper[2574]: E0424 14:24:46.033532    2574 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/4ed72386-1812-4697-9811-c3534a37dc95-cert podName:4ed72386-1812-4697-9811-c3534a37dc95 nodeName:}" failed. No retries permitted until 2026-04-24 14:25:02.033517725 +0000 UTC m=+64.765619355 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "cert" (UniqueName: "kubernetes.io/secret/4ed72386-1812-4697-9811-c3534a37dc95-cert") pod "ingress-canary-tzbfv" (UID: "4ed72386-1812-4697-9811-c3534a37dc95") : secret "canary-serving-cert" not found
Apr 24 14:24:46.134256 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:24:46.134211    2574 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/b4b7957c-9f4f-4c2d-ab2b-85c958db734a-metrics-tls\") pod \"dns-default-nvz9x\" (UID: \"b4b7957c-9f4f-4c2d-ab2b-85c958db734a\") " pod="openshift-dns/dns-default-nvz9x"
Apr 24 14:24:46.134446 ip-10-0-136-132 kubenswrapper[2574]: E0424 14:24:46.134323    2574 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Apr 24 14:24:46.134446 ip-10-0-136-132 kubenswrapper[2574]: E0424 14:24:46.134405    2574 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b4b7957c-9f4f-4c2d-ab2b-85c958db734a-metrics-tls podName:b4b7957c-9f4f-4c2d-ab2b-85c958db734a nodeName:}" failed. No retries permitted until 2026-04-24 14:25:02.13439035 +0000 UTC m=+64.866491976 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/b4b7957c-9f4f-4c2d-ab2b-85c958db734a-metrics-tls") pod "dns-default-nvz9x" (UID: "b4b7957c-9f4f-4c2d-ab2b-85c958db734a") : secret "dns-default-metrics-tls" not found
Apr 24 14:24:46.235557 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:24:46.235508    2574 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/5e2a3789-4ab2-4df5-bbbf-e649d90f15a5-original-pull-secret\") pod \"global-pull-secret-syncer-nwx74\" (UID: \"5e2a3789-4ab2-4df5-bbbf-e649d90f15a5\") " pod="kube-system/global-pull-secret-syncer-nwx74"
Apr 24 14:24:46.238830 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:24:46.238796    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/5e2a3789-4ab2-4df5-bbbf-e649d90f15a5-original-pull-secret\") pod \"global-pull-secret-syncer-nwx74\" (UID: \"5e2a3789-4ab2-4df5-bbbf-e649d90f15a5\") " pod="kube-system/global-pull-secret-syncer-nwx74"
Apr 24 14:24:46.484502 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:24:46.484463    2574 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-nwx74"
Apr 24 14:24:46.610657 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:24:46.610628    2574 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-nwx74"]
Apr 24 14:24:46.614314 ip-10-0-136-132 kubenswrapper[2574]: W0424 14:24:46.614279    2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5e2a3789_4ab2_4df5_bbbf_e649d90f15a5.slice/crio-3c9e91796b306cebba7c2345ae248a7c3b57febf34e9d731953da54f848b6d7a WatchSource:0}: Error finding container 3c9e91796b306cebba7c2345ae248a7c3b57febf34e9d731953da54f848b6d7a: Status 404 returned error can't find the container with id 3c9e91796b306cebba7c2345ae248a7c3b57febf34e9d731953da54f848b6d7a
Apr 24 14:24:46.965083 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:24:46.965050    2574 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-nwx74" event={"ID":"5e2a3789-4ab2-4df5-bbbf-e649d90f15a5","Type":"ContainerStarted","Data":"3c9e91796b306cebba7c2345ae248a7c3b57febf34e9d731953da54f848b6d7a"}
Apr 24 14:24:50.974427 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:24:50.974388    2574 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-nwx74" event={"ID":"5e2a3789-4ab2-4df5-bbbf-e649d90f15a5","Type":"ContainerStarted","Data":"ff316cb9d2f12e3ca86cab23334b9b3dd435e17e6e6341be4df26bf51ff5cc08"}
Apr 24 14:24:50.990929 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:24:50.990881    2574 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="kube-system/global-pull-secret-syncer-nwx74" podStartSLOduration=33.002689029 podStartE2EDuration="36.990867251s" podCreationTimestamp="2026-04-24 14:24:14 +0000 UTC" firstStartedPulling="2026-04-24 14:24:46.61644906 +0000 UTC m=+49.348550687" lastFinishedPulling="2026-04-24 14:24:50.604627274 +0000 UTC m=+53.336728909" observedRunningTime="2026-04-24 14:24:50.990093074 +0000 UTC m=+53.722194747" watchObservedRunningTime="2026-04-24 14:24:50.990867251 +0000 UTC m=+53.722968899"
Apr 24 14:24:55.922928 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:24:55.922902    2574 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-w5x5x"
Apr 24 14:25:02.047760 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:25:02.047717    2574 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/4ed72386-1812-4697-9811-c3534a37dc95-cert\") pod \"ingress-canary-tzbfv\" (UID: \"4ed72386-1812-4697-9811-c3534a37dc95\") " pod="openshift-ingress-canary/ingress-canary-tzbfv"
Apr 24 14:25:02.048162 ip-10-0-136-132 kubenswrapper[2574]: E0424 14:25:02.047859    2574 secret.go:189] Couldn't get secret openshift-ingress-canary/canary-serving-cert: secret "canary-serving-cert" not found
Apr 24 14:25:02.048162 ip-10-0-136-132 kubenswrapper[2574]: E0424 14:25:02.047931    2574 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/4ed72386-1812-4697-9811-c3534a37dc95-cert podName:4ed72386-1812-4697-9811-c3534a37dc95 nodeName:}" failed. No retries permitted until 2026-04-24 14:25:34.047914788 +0000 UTC m=+96.780016414 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "cert" (UniqueName: "kubernetes.io/secret/4ed72386-1812-4697-9811-c3534a37dc95-cert") pod "ingress-canary-tzbfv" (UID: "4ed72386-1812-4697-9811-c3534a37dc95") : secret "canary-serving-cert" not found
Apr 24 14:25:02.148245 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:25:02.148207    2574 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/b4b7957c-9f4f-4c2d-ab2b-85c958db734a-metrics-tls\") pod \"dns-default-nvz9x\" (UID: \"b4b7957c-9f4f-4c2d-ab2b-85c958db734a\") " pod="openshift-dns/dns-default-nvz9x"
Apr 24 14:25:02.148422 ip-10-0-136-132 kubenswrapper[2574]: E0424 14:25:02.148328    2574 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Apr 24 14:25:02.148422 ip-10-0-136-132 kubenswrapper[2574]: E0424 14:25:02.148405    2574 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b4b7957c-9f4f-4c2d-ab2b-85c958db734a-metrics-tls podName:b4b7957c-9f4f-4c2d-ab2b-85c958db734a nodeName:}" failed. No retries permitted until 2026-04-24 14:25:34.148387866 +0000 UTC m=+96.880489619 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/b4b7957c-9f4f-4c2d-ab2b-85c958db734a-metrics-tls") pod "dns-default-nvz9x" (UID: "b4b7957c-9f4f-4c2d-ab2b-85c958db734a") : secret "dns-default-metrics-tls" not found
Apr 24 14:25:03.456291 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:25:03.456244    2574 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/3bc44c73-963f-4a9e-8a52-1e8b031a08dc-metrics-certs\") pod \"network-metrics-daemon-zhkmq\" (UID: \"3bc44c73-963f-4a9e-8a52-1e8b031a08dc\") " pod="openshift-multus/network-metrics-daemon-zhkmq"
Apr 24 14:25:03.456755 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:25:03.456311    2574 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-css2f\" (UniqueName: \"kubernetes.io/projected/3964207c-9c58-4994-a8a9-aa4e4feca773-kube-api-access-css2f\") pod \"network-check-target-6tsjh\" (UID: \"3964207c-9c58-4994-a8a9-aa4e4feca773\") " pod="openshift-network-diagnostics/network-check-target-6tsjh"
Apr 24 14:25:03.458613 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:25:03.458593    2574 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\""
Apr 24 14:25:03.459053 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:25:03.459037    2574 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\""
Apr 24 14:25:03.467208 ip-10-0-136-132 kubenswrapper[2574]: E0424 14:25:03.467178    2574 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: secret "metrics-daemon-secret" not found
Apr 24 14:25:03.467274 ip-10-0-136-132 kubenswrapper[2574]: E0424 14:25:03.467264    2574 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/3bc44c73-963f-4a9e-8a52-1e8b031a08dc-metrics-certs podName:3bc44c73-963f-4a9e-8a52-1e8b031a08dc nodeName:}" failed. No retries permitted until 2026-04-24 14:26:07.467248133 +0000 UTC m=+130.199349760 (durationBeforeRetry 1m4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/3bc44c73-963f-4a9e-8a52-1e8b031a08dc-metrics-certs") pod "network-metrics-daemon-zhkmq" (UID: "3bc44c73-963f-4a9e-8a52-1e8b031a08dc") : secret "metrics-daemon-secret" not found
Apr 24 14:25:03.469097 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:25:03.469081    2574 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\""
Apr 24 14:25:03.479871 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:25:03.479852    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-css2f\" (UniqueName: \"kubernetes.io/projected/3964207c-9c58-4994-a8a9-aa4e4feca773-kube-api-access-css2f\") pod \"network-check-target-6tsjh\" (UID: \"3964207c-9c58-4994-a8a9-aa4e4feca773\") " pod="openshift-network-diagnostics/network-check-target-6tsjh"
Apr 24 14:25:03.579632 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:25:03.579603    2574 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-4q5p5\""
Apr 24 14:25:03.588046 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:25:03.588023    2574 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-6tsjh"
Apr 24 14:25:03.709769 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:25:03.709695    2574 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-6tsjh"]
Apr 24 14:25:03.712735 ip-10-0-136-132 kubenswrapper[2574]: W0424 14:25:03.712699    2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3964207c_9c58_4994_a8a9_aa4e4feca773.slice/crio-29cad1c9f65bf1bcd346853359adde98befe9fc25aaa45fba7cedab084b5b573 WatchSource:0}: Error finding container 29cad1c9f65bf1bcd346853359adde98befe9fc25aaa45fba7cedab084b5b573: Status 404 returned error can't find the container with id 29cad1c9f65bf1bcd346853359adde98befe9fc25aaa45fba7cedab084b5b573
Apr 24 14:25:03.999818 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:25:03.999743    2574 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-6tsjh" event={"ID":"3964207c-9c58-4994-a8a9-aa4e4feca773","Type":"ContainerStarted","Data":"29cad1c9f65bf1bcd346853359adde98befe9fc25aaa45fba7cedab084b5b573"}
Apr 24 14:25:07.007035 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:25:07.006995    2574 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-6tsjh" event={"ID":"3964207c-9c58-4994-a8a9-aa4e4feca773","Type":"ContainerStarted","Data":"2408d740d1683f173fd6f31e7a6e90872cae67293b1343a8e964ddc5246094f2"}
Apr 24 14:25:07.007527 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:25:07.007191    2574 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-network-diagnostics/network-check-target-6tsjh"
Apr 24 14:25:07.023040 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:25:07.022990    2574 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-network-diagnostics/network-check-target-6tsjh" podStartSLOduration=67.465781345 podStartE2EDuration="1m10.022976495s" podCreationTimestamp="2026-04-24 14:23:57 +0000 UTC" firstStartedPulling="2026-04-24 14:25:03.714603304 +0000 UTC m=+66.446704931" lastFinishedPulling="2026-04-24 14:25:06.271798443 +0000 UTC m=+69.003900081" observedRunningTime="2026-04-24 14:25:07.022155679 +0000 UTC m=+69.754257328" watchObservedRunningTime="2026-04-24 14:25:07.022976495 +0000 UTC m=+69.755078122"
Apr 24 14:25:34.064483 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:25:34.064450    2574 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/4ed72386-1812-4697-9811-c3534a37dc95-cert\") pod \"ingress-canary-tzbfv\" (UID: \"4ed72386-1812-4697-9811-c3534a37dc95\") " pod="openshift-ingress-canary/ingress-canary-tzbfv"
Apr 24 14:25:34.064908 ip-10-0-136-132 kubenswrapper[2574]: E0424 14:25:34.064594    2574 secret.go:189] Couldn't get secret openshift-ingress-canary/canary-serving-cert: secret "canary-serving-cert" not found
Apr 24 14:25:34.064908 ip-10-0-136-132 kubenswrapper[2574]: E0424 14:25:34.064661    2574 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/4ed72386-1812-4697-9811-c3534a37dc95-cert podName:4ed72386-1812-4697-9811-c3534a37dc95 nodeName:}" failed. No retries permitted until 2026-04-24 14:26:38.064647652 +0000 UTC m=+160.796749278 (durationBeforeRetry 1m4s). Error: MountVolume.SetUp failed for volume "cert" (UniqueName: "kubernetes.io/secret/4ed72386-1812-4697-9811-c3534a37dc95-cert") pod "ingress-canary-tzbfv" (UID: "4ed72386-1812-4697-9811-c3534a37dc95") : secret "canary-serving-cert" not found
Apr 24 14:25:34.165742 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:25:34.165711    2574 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/b4b7957c-9f4f-4c2d-ab2b-85c958db734a-metrics-tls\") pod \"dns-default-nvz9x\" (UID: \"b4b7957c-9f4f-4c2d-ab2b-85c958db734a\") " pod="openshift-dns/dns-default-nvz9x"
Apr 24 14:25:34.165833 ip-10-0-136-132 kubenswrapper[2574]: E0424 14:25:34.165820    2574 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Apr 24 14:25:34.165893 ip-10-0-136-132 kubenswrapper[2574]: E0424 14:25:34.165883    2574 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b4b7957c-9f4f-4c2d-ab2b-85c958db734a-metrics-tls podName:b4b7957c-9f4f-4c2d-ab2b-85c958db734a nodeName:}" failed. No retries permitted until 2026-04-24 14:26:38.165868363 +0000 UTC m=+160.897969990 (durationBeforeRetry 1m4s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/b4b7957c-9f4f-4c2d-ab2b-85c958db734a-metrics-tls") pod "dns-default-nvz9x" (UID: "b4b7957c-9f4f-4c2d-ab2b-85c958db734a") : secret "dns-default-metrics-tls" not found
Apr 24 14:25:38.012087 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:25:38.012056    2574 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-network-diagnostics/network-check-target-6tsjh"
Apr 24 14:26:07.492143 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:26:07.492099    2574 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/3bc44c73-963f-4a9e-8a52-1e8b031a08dc-metrics-certs\") pod \"network-metrics-daemon-zhkmq\" (UID: \"3bc44c73-963f-4a9e-8a52-1e8b031a08dc\") " pod="openshift-multus/network-metrics-daemon-zhkmq"
Apr 24 14:26:07.492738 ip-10-0-136-132 kubenswrapper[2574]: E0424 14:26:07.492235    2574 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: secret "metrics-daemon-secret" not found
Apr 24 14:26:07.492738 ip-10-0-136-132 kubenswrapper[2574]: E0424 14:26:07.492310    2574 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/3bc44c73-963f-4a9e-8a52-1e8b031a08dc-metrics-certs podName:3bc44c73-963f-4a9e-8a52-1e8b031a08dc nodeName:}" failed. No retries permitted until 2026-04-24 14:28:09.49229617 +0000 UTC m=+252.224397797 (durationBeforeRetry 2m2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/3bc44c73-963f-4a9e-8a52-1e8b031a08dc-metrics-certs") pod "network-metrics-daemon-zhkmq" (UID: "3bc44c73-963f-4a9e-8a52-1e8b031a08dc") : secret "metrics-daemon-secret" not found
Apr 24 14:26:18.038689 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:26:18.038654    2574 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["openshift-monitoring/cluster-monitoring-operator-75587bd455-q9jbv"]
Apr 24 14:26:18.041520 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:26:18.041494    2574 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/cluster-monitoring-operator-75587bd455-q9jbv"
Apr 24 14:26:18.044134 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:26:18.044109    2574 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kube-root-ca.crt\""
Apr 24 14:26:18.044734 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:26:18.044708    2574 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"cluster-monitoring-operator-tls\""
Apr 24 14:26:18.044884 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:26:18.044865    2574 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"openshift-service-ca.crt\""
Apr 24 14:26:18.045074 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:26:18.044815    2574 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"telemetry-config\""
Apr 24 14:26:18.045175 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:26:18.045158    2574 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"cluster-monitoring-operator-dockercfg-w8rhb\""
Apr 24 14:26:18.046198 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:26:18.046175    2574 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-6769c5d45-rmc8n"]
Apr 24 14:26:18.049291 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:26:18.049073    2574 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-6769c5d45-rmc8n"
Apr 24 14:26:18.051254 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:26:18.051236    2574 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"serving-cert\""
Apr 24 14:26:18.051376 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:26:18.051307    2574 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"kube-storage-version-migrator-operator-dockercfg-m9ls5\""
Apr 24 14:26:18.051376 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:26:18.051322    2574 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"kube-root-ca.crt\""
Apr 24 14:26:18.051482 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:26:18.051354    2574 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"openshift-service-ca.crt\""
Apr 24 14:26:18.051648 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:26:18.051634    2574 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"config\""
Apr 24 14:26:18.058466 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:26:18.058447    2574 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/cluster-monitoring-operator-75587bd455-q9jbv"]
Apr 24 14:26:18.059001 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:26:18.058984    2574 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-6769c5d45-rmc8n"]
Apr 24 14:26:18.142754 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:26:18.142724    2574 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["openshift-service-ca-operator/service-ca-operator-d6fc45fc5-nwsrw"]
Apr 24 14:26:18.145668 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:26:18.145649    2574 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-service-ca-operator/service-ca-operator-d6fc45fc5-nwsrw"
Apr 24 14:26:18.146949 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:26:18.146925    2574 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["openshift-cluster-storage-operator/volume-data-source-validator-7c6cbb6c87-j2gr5"]
Apr 24 14:26:18.147549 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:26:18.147527    2574 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-service-ca-operator\"/\"serving-cert\""
Apr 24 14:26:18.147651 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:26:18.147612    2574 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca-operator\"/\"service-ca-operator-config\""
Apr 24 14:26:18.147900 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:26:18.147883    2574 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca-operator\"/\"openshift-service-ca.crt\""
Apr 24 14:26:18.148035 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:26:18.148020    2574 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca-operator\"/\"kube-root-ca.crt\""
Apr 24 14:26:18.149577 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:26:18.149560    2574 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["openshift-console-operator/console-operator-9d4b6777b-f6ngr"]
Apr 24 14:26:18.149741 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:26:18.149713    2574 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-storage-operator/volume-data-source-validator-7c6cbb6c87-j2gr5"
Apr 24 14:26:18.150649 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:26:18.150634    2574 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-service-ca-operator\"/\"service-ca-operator-dockercfg-2qd78\""
Apr 24 14:26:18.151248 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:26:18.151233    2574 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-storage-operator\"/\"kube-root-ca.crt\""
Apr 24 14:26:18.151248 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:26:18.151241    2574 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-storage-operator\"/\"openshift-service-ca.crt\""
Apr 24 14:26:18.151432 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:26:18.151409    2574 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-storage-operator\"/\"volume-data-source-validator-dockercfg-g9fmv\""
Apr 24 14:26:18.152289 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:26:18.152275    2574 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console-operator/console-operator-9d4b6777b-f6ngr"
Apr 24 14:26:18.154537 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:26:18.154520    2574 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console-operator\"/\"openshift-service-ca.crt\""
Apr 24 14:26:18.154632 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:26:18.154579    2574 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console-operator\"/\"console-operator-config\""
Apr 24 14:26:18.154691 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:26:18.154658    2574 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console-operator\"/\"console-operator-dockercfg-pzqcv\""
Apr 24 14:26:18.154815 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:26:18.154800    2574 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console-operator\"/\"serving-cert\""
Apr 24 14:26:18.155188 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:26:18.155172    2574 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console-operator\"/\"kube-root-ca.crt\""
Apr 24 14:26:18.160792 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:26:18.160769    2574 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console-operator\"/\"trusted-ca\""
Apr 24 14:26:18.163321 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:26:18.163298    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-br28n\" (UniqueName: \"kubernetes.io/projected/5bc2f6f9-dff9-48b0-bc63-5f778a1ed590-kube-api-access-br28n\") pod \"kube-storage-version-migrator-operator-6769c5d45-rmc8n\" (UID: \"5bc2f6f9-dff9-48b0-bc63-5f778a1ed590\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-6769c5d45-rmc8n"
Apr 24 14:26:18.163445 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:26:18.163374    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/5bc2f6f9-dff9-48b0-bc63-5f778a1ed590-serving-cert\") pod \"kube-storage-version-migrator-operator-6769c5d45-rmc8n\" (UID: \"5bc2f6f9-dff9-48b0-bc63-5f778a1ed590\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-6769c5d45-rmc8n"
Apr 24 14:26:18.163509 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:26:18.163470    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/0be19048-1b01-4c64-bcca-94c9bff7768d-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-75587bd455-q9jbv\" (UID: \"0be19048-1b01-4c64-bcca-94c9bff7768d\") " pod="openshift-monitoring/cluster-monitoring-operator-75587bd455-q9jbv"
Apr 24 14:26:18.163563 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:26:18.163511    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/5bc2f6f9-dff9-48b0-bc63-5f778a1ed590-config\") pod \"kube-storage-version-migrator-operator-6769c5d45-rmc8n\" (UID: \"5bc2f6f9-dff9-48b0-bc63-5f778a1ed590\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-6769c5d45-rmc8n"
Apr 24 14:26:18.163563 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:26:18.163538    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hm8th\" (UniqueName: \"kubernetes.io/projected/0be19048-1b01-4c64-bcca-94c9bff7768d-kube-api-access-hm8th\") pod \"cluster-monitoring-operator-75587bd455-q9jbv\" (UID: \"0be19048-1b01-4c64-bcca-94c9bff7768d\") " pod="openshift-monitoring/cluster-monitoring-operator-75587bd455-q9jbv"
Apr 24 14:26:18.163661 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:26:18.163606    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"telemetry-config\" (UniqueName: \"kubernetes.io/configmap/0be19048-1b01-4c64-bcca-94c9bff7768d-telemetry-config\") pod \"cluster-monitoring-operator-75587bd455-q9jbv\" (UID: \"0be19048-1b01-4c64-bcca-94c9bff7768d\") " pod="openshift-monitoring/cluster-monitoring-operator-75587bd455-q9jbv"
Apr 24 14:26:18.164521 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:26:18.164491    2574 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-service-ca-operator/service-ca-operator-d6fc45fc5-nwsrw"]
Apr 24 14:26:18.165482 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:26:18.165466    2574 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-cluster-storage-operator/volume-data-source-validator-7c6cbb6c87-j2gr5"]
Apr 24 14:26:18.166553 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:26:18.166534    2574 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-console-operator/console-operator-9d4b6777b-f6ngr"]
Apr 24 14:26:18.264483 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:26:18.264443    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/2571829f-666a-422a-9b02-2d2bb4c89b25-trusted-ca\") pod \"console-operator-9d4b6777b-f6ngr\" (UID: \"2571829f-666a-422a-9b02-2d2bb4c89b25\") " pod="openshift-console-operator/console-operator-9d4b6777b-f6ngr"
Apr 24 14:26:18.264483 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:26:18.264491    2574 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"telemetry-config\" (UniqueName: \"kubernetes.io/configmap/0be19048-1b01-4c64-bcca-94c9bff7768d-telemetry-config\") pod \"cluster-monitoring-operator-75587bd455-q9jbv\" (UID: \"0be19048-1b01-4c64-bcca-94c9bff7768d\") " pod="openshift-monitoring/cluster-monitoring-operator-75587bd455-q9jbv"
Apr 24 14:26:18.264732 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:26:18.264545    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-t6xfm\" (UniqueName: \"kubernetes.io/projected/de9f9e0b-653f-481d-b23f-5c408b5cc75e-kube-api-access-t6xfm\") pod \"volume-data-source-validator-7c6cbb6c87-j2gr5\" (UID: \"de9f9e0b-653f-481d-b23f-5c408b5cc75e\") " pod="openshift-cluster-storage-operator/volume-data-source-validator-7c6cbb6c87-j2gr5"
Apr 24 14:26:18.264732 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:26:18.264596    2574 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-br28n\" (UniqueName: \"kubernetes.io/projected/5bc2f6f9-dff9-48b0-bc63-5f778a1ed590-kube-api-access-br28n\") pod \"kube-storage-version-migrator-operator-6769c5d45-rmc8n\" (UID: \"5bc2f6f9-dff9-48b0-bc63-5f778a1ed590\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-6769c5d45-rmc8n"
Apr 24 14:26:18.264732 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:26:18.264617    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/2571829f-666a-422a-9b02-2d2bb4c89b25-serving-cert\") pod \"console-operator-9d4b6777b-f6ngr\" (UID: \"2571829f-666a-422a-9b02-2d2bb4c89b25\") " pod="openshift-console-operator/console-operator-9d4b6777b-f6ngr"
Apr 24 14:26:18.264732 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:26:18.264642    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/ddb5d8a7-6366-44cb-a7d4-58e3627b0187-config\") pod \"service-ca-operator-d6fc45fc5-nwsrw\" (UID: \"ddb5d8a7-6366-44cb-a7d4-58e3627b0187\") " pod="openshift-service-ca-operator/service-ca-operator-d6fc45fc5-nwsrw"
Apr 24 14:26:18.264932 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:26:18.264756    2574 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/5bc2f6f9-dff9-48b0-bc63-5f778a1ed590-serving-cert\") pod \"kube-storage-version-migrator-operator-6769c5d45-rmc8n\" (UID: \"5bc2f6f9-dff9-48b0-bc63-5f778a1ed590\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-6769c5d45-rmc8n"
Apr 24 14:26:18.264932 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:26:18.264799    2574 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/0be19048-1b01-4c64-bcca-94c9bff7768d-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-75587bd455-q9jbv\" (UID: \"0be19048-1b01-4c64-bcca-94c9bff7768d\") " pod="openshift-monitoring/cluster-monitoring-operator-75587bd455-q9jbv"
Apr 24 14:26:18.264932 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:26:18.264833    2574 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/5bc2f6f9-dff9-48b0-bc63-5f778a1ed590-config\") pod \"kube-storage-version-migrator-operator-6769c5d45-rmc8n\" (UID: \"5bc2f6f9-dff9-48b0-bc63-5f778a1ed590\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-6769c5d45-rmc8n"
Apr 24 14:26:18.264932 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:26:18.264879    2574 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-hm8th\" (UniqueName: \"kubernetes.io/projected/0be19048-1b01-4c64-bcca-94c9bff7768d-kube-api-access-hm8th\") pod \"cluster-monitoring-operator-75587bd455-q9jbv\" (UID: \"0be19048-1b01-4c64-bcca-94c9bff7768d\") " pod="openshift-monitoring/cluster-monitoring-operator-75587bd455-q9jbv"
Apr 24 14:26:18.264932 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:26:18.264912    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-s9zjz\" (UniqueName: \"kubernetes.io/projected/2571829f-666a-422a-9b02-2d2bb4c89b25-kube-api-access-s9zjz\") pod \"console-operator-9d4b6777b-f6ngr\" (UID: \"2571829f-666a-422a-9b02-2d2bb4c89b25\") " pod="openshift-console-operator/console-operator-9d4b6777b-f6ngr"
Apr 24 14:26:18.265171 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:26:18.264943    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/ddb5d8a7-6366-44cb-a7d4-58e3627b0187-serving-cert\") pod \"service-ca-operator-d6fc45fc5-nwsrw\" (UID: \"ddb5d8a7-6366-44cb-a7d4-58e3627b0187\") " pod="openshift-service-ca-operator/service-ca-operator-d6fc45fc5-nwsrw"
Apr 24 14:26:18.265171 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:26:18.264971    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-q68w2\" (UniqueName: \"kubernetes.io/projected/ddb5d8a7-6366-44cb-a7d4-58e3627b0187-kube-api-access-q68w2\") pod \"service-ca-operator-d6fc45fc5-nwsrw\" (UID: \"ddb5d8a7-6366-44cb-a7d4-58e3627b0187\") " pod="openshift-service-ca-operator/service-ca-operator-d6fc45fc5-nwsrw"
Apr 24 14:26:18.265171 ip-10-0-136-132 kubenswrapper[2574]: E0424 14:26:18.265024    2574 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found
Apr 24 14:26:18.265171 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:26:18.265028    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/2571829f-666a-422a-9b02-2d2bb4c89b25-config\") pod \"console-operator-9d4b6777b-f6ngr\" (UID: \"2571829f-666a-422a-9b02-2d2bb4c89b25\") " pod="openshift-console-operator/console-operator-9d4b6777b-f6ngr"
Apr 24 14:26:18.265171 ip-10-0-136-132 kubenswrapper[2574]: E0424 14:26:18.265094    2574 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/0be19048-1b01-4c64-bcca-94c9bff7768d-cluster-monitoring-operator-tls podName:0be19048-1b01-4c64-bcca-94c9bff7768d nodeName:}" failed. No retries permitted until 2026-04-24 14:26:18.765075871 +0000 UTC m=+141.497177513 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/0be19048-1b01-4c64-bcca-94c9bff7768d-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-75587bd455-q9jbv" (UID: "0be19048-1b01-4c64-bcca-94c9bff7768d") : secret "cluster-monitoring-operator-tls" not found
Apr 24 14:26:18.265461 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:26:18.265298    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"telemetry-config\" (UniqueName: \"kubernetes.io/configmap/0be19048-1b01-4c64-bcca-94c9bff7768d-telemetry-config\") pod \"cluster-monitoring-operator-75587bd455-q9jbv\" (UID: \"0be19048-1b01-4c64-bcca-94c9bff7768d\") " pod="openshift-monitoring/cluster-monitoring-operator-75587bd455-q9jbv"
Apr 24 14:26:18.265461 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:26:18.265439    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/configmap/5bc2f6f9-dff9-48b0-bc63-5f778a1ed590-config\") pod \"kube-storage-version-migrator-operator-6769c5d45-rmc8n\" (UID: \"5bc2f6f9-dff9-48b0-bc63-5f778a1ed590\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-6769c5d45-rmc8n"
Apr 24 14:26:18.267011 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:26:18.266991    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/5bc2f6f9-dff9-48b0-bc63-5f778a1ed590-serving-cert\") pod \"kube-storage-version-migrator-operator-6769c5d45-rmc8n\" (UID: \"5bc2f6f9-dff9-48b0-bc63-5f778a1ed590\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-6769c5d45-rmc8n"
Apr 24 14:26:18.273050 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:26:18.273030    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-hm8th\" (UniqueName: \"kubernetes.io/projected/0be19048-1b01-4c64-bcca-94c9bff7768d-kube-api-access-hm8th\") pod \"cluster-monitoring-operator-75587bd455-q9jbv\" (UID: \"0be19048-1b01-4c64-bcca-94c9bff7768d\") " pod="openshift-monitoring/cluster-monitoring-operator-75587bd455-q9jbv"
Apr 24 14:26:18.273161 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:26:18.273141    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-br28n\" (UniqueName: \"kubernetes.io/projected/5bc2f6f9-dff9-48b0-bc63-5f778a1ed590-kube-api-access-br28n\") pod \"kube-storage-version-migrator-operator-6769c5d45-rmc8n\" (UID: \"5bc2f6f9-dff9-48b0-bc63-5f778a1ed590\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-6769c5d45-rmc8n"
Apr 24 14:26:18.358322 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:26:18.358219    2574 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-6769c5d45-rmc8n"
Apr 24 14:26:18.366222 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:26:18.366171    2574 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/2571829f-666a-422a-9b02-2d2bb4c89b25-trusted-ca\") pod \"console-operator-9d4b6777b-f6ngr\" (UID: \"2571829f-666a-422a-9b02-2d2bb4c89b25\") " pod="openshift-console-operator/console-operator-9d4b6777b-f6ngr"
Apr 24 14:26:18.366870 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:26:18.366845    2574 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-t6xfm\" (UniqueName: \"kubernetes.io/projected/de9f9e0b-653f-481d-b23f-5c408b5cc75e-kube-api-access-t6xfm\") pod \"volume-data-source-validator-7c6cbb6c87-j2gr5\" (UID: \"de9f9e0b-653f-481d-b23f-5c408b5cc75e\") " pod="openshift-cluster-storage-operator/volume-data-source-validator-7c6cbb6c87-j2gr5"
Apr 24 14:26:18.366941 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:26:18.366904    2574 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/2571829f-666a-422a-9b02-2d2bb4c89b25-serving-cert\") pod \"console-operator-9d4b6777b-f6ngr\" (UID: \"2571829f-666a-422a-9b02-2d2bb4c89b25\") " pod="openshift-console-operator/console-operator-9d4b6777b-f6ngr"
Apr 24 14:26:18.367001 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:26:18.366938    2574 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/ddb5d8a7-6366-44cb-a7d4-58e3627b0187-config\") pod \"service-ca-operator-d6fc45fc5-nwsrw\" (UID: \"ddb5d8a7-6366-44cb-a7d4-58e3627b0187\") " pod="openshift-service-ca-operator/service-ca-operator-d6fc45fc5-nwsrw"
Apr 24 14:26:18.367052 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:26:18.367021    2574 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-s9zjz\" (UniqueName: \"kubernetes.io/projected/2571829f-666a-422a-9b02-2d2bb4c89b25-kube-api-access-s9zjz\") pod \"console-operator-9d4b6777b-f6ngr\" (UID: \"2571829f-666a-422a-9b02-2d2bb4c89b25\") " pod="openshift-console-operator/console-operator-9d4b6777b-f6ngr"
Apr 24 14:26:18.367102 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:26:18.367048    2574 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/ddb5d8a7-6366-44cb-a7d4-58e3627b0187-serving-cert\") pod \"service-ca-operator-d6fc45fc5-nwsrw\" (UID: \"ddb5d8a7-6366-44cb-a7d4-58e3627b0187\") " pod="openshift-service-ca-operator/service-ca-operator-d6fc45fc5-nwsrw"
Apr 24 14:26:18.367102 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:26:18.367072    2574 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-q68w2\" (UniqueName: \"kubernetes.io/projected/ddb5d8a7-6366-44cb-a7d4-58e3627b0187-kube-api-access-q68w2\") pod \"service-ca-operator-d6fc45fc5-nwsrw\" (UID: \"ddb5d8a7-6366-44cb-a7d4-58e3627b0187\") " pod="openshift-service-ca-operator/service-ca-operator-d6fc45fc5-nwsrw"
Apr 24 14:26:18.367200 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:26:18.367110    2574 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/2571829f-666a-422a-9b02-2d2bb4c89b25-config\") pod \"console-operator-9d4b6777b-f6ngr\" (UID: \"2571829f-666a-422a-9b02-2d2bb4c89b25\") " pod="openshift-console-operator/console-operator-9d4b6777b-f6ngr"
Apr 24 14:26:18.368428 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:26:18.368108    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/2571829f-666a-422a-9b02-2d2bb4c89b25-trusted-ca\") pod \"console-operator-9d4b6777b-f6ngr\" (UID: \"2571829f-666a-422a-9b02-2d2bb4c89b25\") " pod="openshift-console-operator/console-operator-9d4b6777b-f6ngr"
Apr 24 14:26:18.368528 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:26:18.368458    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/configmap/ddb5d8a7-6366-44cb-a7d4-58e3627b0187-config\") pod \"service-ca-operator-d6fc45fc5-nwsrw\" (UID: \"ddb5d8a7-6366-44cb-a7d4-58e3627b0187\") " pod="openshift-service-ca-operator/service-ca-operator-d6fc45fc5-nwsrw"
Apr 24 14:26:18.372995 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:26:18.368820    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/configmap/2571829f-666a-422a-9b02-2d2bb4c89b25-config\") pod \"console-operator-9d4b6777b-f6ngr\" (UID: \"2571829f-666a-422a-9b02-2d2bb4c89b25\") " pod="openshift-console-operator/console-operator-9d4b6777b-f6ngr"
Apr 24 14:26:18.372995 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:26:18.371571    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/2571829f-666a-422a-9b02-2d2bb4c89b25-serving-cert\") pod \"console-operator-9d4b6777b-f6ngr\" (UID: \"2571829f-666a-422a-9b02-2d2bb4c89b25\") " pod="openshift-console-operator/console-operator-9d4b6777b-f6ngr"
Apr 24 14:26:18.373787 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:26:18.373752    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/ddb5d8a7-6366-44cb-a7d4-58e3627b0187-serving-cert\") pod \"service-ca-operator-d6fc45fc5-nwsrw\" (UID: \"ddb5d8a7-6366-44cb-a7d4-58e3627b0187\") " pod="openshift-service-ca-operator/service-ca-operator-d6fc45fc5-nwsrw"
Apr 24 14:26:18.375257 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:26:18.375234    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-s9zjz\" (UniqueName: \"kubernetes.io/projected/2571829f-666a-422a-9b02-2d2bb4c89b25-kube-api-access-s9zjz\") pod \"console-operator-9d4b6777b-f6ngr\" (UID: \"2571829f-666a-422a-9b02-2d2bb4c89b25\") " pod="openshift-console-operator/console-operator-9d4b6777b-f6ngr"
Apr 24 14:26:18.375515 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:26:18.375498    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-q68w2\" (UniqueName: \"kubernetes.io/projected/ddb5d8a7-6366-44cb-a7d4-58e3627b0187-kube-api-access-q68w2\") pod \"service-ca-operator-d6fc45fc5-nwsrw\" (UID: \"ddb5d8a7-6366-44cb-a7d4-58e3627b0187\") " pod="openshift-service-ca-operator/service-ca-operator-d6fc45fc5-nwsrw"
Apr 24 14:26:18.375982 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:26:18.375962    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-t6xfm\" (UniqueName: \"kubernetes.io/projected/de9f9e0b-653f-481d-b23f-5c408b5cc75e-kube-api-access-t6xfm\") pod \"volume-data-source-validator-7c6cbb6c87-j2gr5\" (UID: \"de9f9e0b-653f-481d-b23f-5c408b5cc75e\") " pod="openshift-cluster-storage-operator/volume-data-source-validator-7c6cbb6c87-j2gr5"
Apr 24 14:26:18.455858 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:26:18.455828    2574 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-service-ca-operator/service-ca-operator-d6fc45fc5-nwsrw"
Apr 24 14:26:18.462514 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:26:18.462484    2574 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-storage-operator/volume-data-source-validator-7c6cbb6c87-j2gr5"
Apr 24 14:26:18.469168 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:26:18.469125    2574 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console-operator/console-operator-9d4b6777b-f6ngr"
Apr 24 14:26:18.470916 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:26:18.470817    2574 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-6769c5d45-rmc8n"]
Apr 24 14:26:18.475634 ip-10-0-136-132 kubenswrapper[2574]: W0424 14:26:18.475593    2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5bc2f6f9_dff9_48b0_bc63_5f778a1ed590.slice/crio-62d065e8c71acbea076aa11bfc99bdf1be5f41d09aa6491ac747187a6577f505 WatchSource:0}: Error finding container 62d065e8c71acbea076aa11bfc99bdf1be5f41d09aa6491ac747187a6577f505: Status 404 returned error can't find the container with id 62d065e8c71acbea076aa11bfc99bdf1be5f41d09aa6491ac747187a6577f505
Apr 24 14:26:18.585007 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:26:18.584960    2574 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-service-ca-operator/service-ca-operator-d6fc45fc5-nwsrw"]
Apr 24 14:26:18.589000 ip-10-0-136-132 kubenswrapper[2574]: W0424 14:26:18.588969    2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podddb5d8a7_6366_44cb_a7d4_58e3627b0187.slice/crio-f4b70a3dc17626e08aba696206958d5401205dccaafb0b91e7590c63b24b4794 WatchSource:0}: Error finding container f4b70a3dc17626e08aba696206958d5401205dccaafb0b91e7590c63b24b4794: Status 404 returned error can't find the container with id f4b70a3dc17626e08aba696206958d5401205dccaafb0b91e7590c63b24b4794
Apr 24 14:26:18.600301 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:26:18.600275    2574 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-cluster-storage-operator/volume-data-source-validator-7c6cbb6c87-j2gr5"]
Apr 24 14:26:18.603474 ip-10-0-136-132 kubenswrapper[2574]: W0424 14:26:18.603447    2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podde9f9e0b_653f_481d_b23f_5c408b5cc75e.slice/crio-ea6362489b29834325657ec148611fa6fc27d530a0f5121abfa1eba35219612c WatchSource:0}: Error finding container ea6362489b29834325657ec148611fa6fc27d530a0f5121abfa1eba35219612c: Status 404 returned error can't find the container with id ea6362489b29834325657ec148611fa6fc27d530a0f5121abfa1eba35219612c
Apr 24 14:26:18.616446 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:26:18.616389    2574 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-console-operator/console-operator-9d4b6777b-f6ngr"]
Apr 24 14:26:18.619633 ip-10-0-136-132 kubenswrapper[2574]: W0424 14:26:18.619613    2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2571829f_666a_422a_9b02_2d2bb4c89b25.slice/crio-7331babea7bc9424134a65291b049c50f0f93f7f9d07bdbc22e76a27a9e6a657 WatchSource:0}: Error finding container 7331babea7bc9424134a65291b049c50f0f93f7f9d07bdbc22e76a27a9e6a657: Status 404 returned error can't find the container with id 7331babea7bc9424134a65291b049c50f0f93f7f9d07bdbc22e76a27a9e6a657
Apr 24 14:26:18.770177 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:26:18.770144    2574 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/0be19048-1b01-4c64-bcca-94c9bff7768d-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-75587bd455-q9jbv\" (UID: \"0be19048-1b01-4c64-bcca-94c9bff7768d\") " pod="openshift-monitoring/cluster-monitoring-operator-75587bd455-q9jbv"
Apr 24 14:26:18.770376 ip-10-0-136-132 kubenswrapper[2574]: E0424 14:26:18.770311    2574 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found
Apr 24 14:26:18.770425 ip-10-0-136-132 kubenswrapper[2574]: E0424 14:26:18.770399    2574 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/0be19048-1b01-4c64-bcca-94c9bff7768d-cluster-monitoring-operator-tls podName:0be19048-1b01-4c64-bcca-94c9bff7768d nodeName:}" failed. No retries permitted until 2026-04-24 14:26:19.770383578 +0000 UTC m=+142.502485212 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/0be19048-1b01-4c64-bcca-94c9bff7768d-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-75587bd455-q9jbv" (UID: "0be19048-1b01-4c64-bcca-94c9bff7768d") : secret "cluster-monitoring-operator-tls" not found
Apr 24 14:26:19.144478 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:26:19.144434    2574 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-storage-operator/volume-data-source-validator-7c6cbb6c87-j2gr5" event={"ID":"de9f9e0b-653f-481d-b23f-5c408b5cc75e","Type":"ContainerStarted","Data":"ea6362489b29834325657ec148611fa6fc27d530a0f5121abfa1eba35219612c"}
Apr 24 14:26:19.145694 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:26:19.145662    2574 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-console-operator/console-operator-9d4b6777b-f6ngr" event={"ID":"2571829f-666a-422a-9b02-2d2bb4c89b25","Type":"ContainerStarted","Data":"7331babea7bc9424134a65291b049c50f0f93f7f9d07bdbc22e76a27a9e6a657"}
Apr 24 14:26:19.147038 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:26:19.147009    2574 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca-operator/service-ca-operator-d6fc45fc5-nwsrw" event={"ID":"ddb5d8a7-6366-44cb-a7d4-58e3627b0187","Type":"ContainerStarted","Data":"f4b70a3dc17626e08aba696206958d5401205dccaafb0b91e7590c63b24b4794"}
Apr 24 14:26:19.148102 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:26:19.148063    2574 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-6769c5d45-rmc8n" event={"ID":"5bc2f6f9-dff9-48b0-bc63-5f778a1ed590","Type":"ContainerStarted","Data":"62d065e8c71acbea076aa11bfc99bdf1be5f41d09aa6491ac747187a6577f505"}
Apr 24 14:26:19.779829 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:26:19.779759    2574 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/0be19048-1b01-4c64-bcca-94c9bff7768d-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-75587bd455-q9jbv\" (UID: \"0be19048-1b01-4c64-bcca-94c9bff7768d\") " pod="openshift-monitoring/cluster-monitoring-operator-75587bd455-q9jbv"
Apr 24 14:26:19.779988 ip-10-0-136-132 kubenswrapper[2574]: E0424 14:26:19.779918    2574 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found
Apr 24 14:26:19.780028 ip-10-0-136-132 kubenswrapper[2574]: E0424 14:26:19.779995    2574 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/0be19048-1b01-4c64-bcca-94c9bff7768d-cluster-monitoring-operator-tls podName:0be19048-1b01-4c64-bcca-94c9bff7768d nodeName:}" failed. No retries permitted until 2026-04-24 14:26:21.779974661 +0000 UTC m=+144.512076297 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/0be19048-1b01-4c64-bcca-94c9bff7768d-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-75587bd455-q9jbv" (UID: "0be19048-1b01-4c64-bcca-94c9bff7768d") : secret "cluster-monitoring-operator-tls" not found
Apr 24 14:26:21.562570 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:26:21.562530    2574 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["openshift-network-diagnostics/network-check-source-8894fc9bd-kj5df"]
Apr 24 14:26:21.566003 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:26:21.565979    2574 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-source-8894fc9bd-kj5df"
Apr 24 14:26:21.567657 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:26:21.567632    2574 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"network-diagnostics-dockercfg-qmcwq\""
Apr 24 14:26:21.571165 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:26:21.571121    2574 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-source-8894fc9bd-kj5df"]
Apr 24 14:26:21.697928 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:26:21.697902    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-prftd\" (UniqueName: \"kubernetes.io/projected/c429617c-d32e-4466-b249-d0a28cfd5b9d-kube-api-access-prftd\") pod \"network-check-source-8894fc9bd-kj5df\" (UID: \"c429617c-d32e-4466-b249-d0a28cfd5b9d\") " pod="openshift-network-diagnostics/network-check-source-8894fc9bd-kj5df"
Apr 24 14:26:21.799085 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:26:21.799044    2574 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-prftd\" (UniqueName: \"kubernetes.io/projected/c429617c-d32e-4466-b249-d0a28cfd5b9d-kube-api-access-prftd\") pod \"network-check-source-8894fc9bd-kj5df\" (UID: \"c429617c-d32e-4466-b249-d0a28cfd5b9d\") " pod="openshift-network-diagnostics/network-check-source-8894fc9bd-kj5df"
Apr 24 14:26:21.799249 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:26:21.799142    2574 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/0be19048-1b01-4c64-bcca-94c9bff7768d-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-75587bd455-q9jbv\" (UID: \"0be19048-1b01-4c64-bcca-94c9bff7768d\") " pod="openshift-monitoring/cluster-monitoring-operator-75587bd455-q9jbv"
Apr 24 14:26:21.799249 ip-10-0-136-132 kubenswrapper[2574]: E0424 14:26:21.799241    2574 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found
Apr 24 14:26:21.799404 ip-10-0-136-132 kubenswrapper[2574]: E0424 14:26:21.799309    2574 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/0be19048-1b01-4c64-bcca-94c9bff7768d-cluster-monitoring-operator-tls podName:0be19048-1b01-4c64-bcca-94c9bff7768d nodeName:}" failed. No retries permitted until 2026-04-24 14:26:25.79929411 +0000 UTC m=+148.531395743 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/0be19048-1b01-4c64-bcca-94c9bff7768d-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-75587bd455-q9jbv" (UID: "0be19048-1b01-4c64-bcca-94c9bff7768d") : secret "cluster-monitoring-operator-tls" not found
Apr 24 14:26:21.807364 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:26:21.807292    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-prftd\" (UniqueName: \"kubernetes.io/projected/c429617c-d32e-4466-b249-d0a28cfd5b9d-kube-api-access-prftd\") pod \"network-check-source-8894fc9bd-kj5df\" (UID: \"c429617c-d32e-4466-b249-d0a28cfd5b9d\") " pod="openshift-network-diagnostics/network-check-source-8894fc9bd-kj5df"
Apr 24 14:26:21.877451 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:26:21.877414    2574 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-source-8894fc9bd-kj5df"
Apr 24 14:26:21.995436 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:26:21.995404    2574 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-source-8894fc9bd-kj5df"]
Apr 24 14:26:21.999044 ip-10-0-136-132 kubenswrapper[2574]: W0424 14:26:21.999018    2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc429617c_d32e_4466_b249_d0a28cfd5b9d.slice/crio-c058abaa72a2204a990c0ea5ad87e3c9b98526a2925b33ce39858d5875d1b5bc WatchSource:0}: Error finding container c058abaa72a2204a990c0ea5ad87e3c9b98526a2925b33ce39858d5875d1b5bc: Status 404 returned error can't find the container with id c058abaa72a2204a990c0ea5ad87e3c9b98526a2925b33ce39858d5875d1b5bc
Apr 24 14:26:22.156910 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:26:22.156882    2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-9d4b6777b-f6ngr_2571829f-666a-422a-9b02-2d2bb4c89b25/console-operator/0.log"
Apr 24 14:26:22.157076 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:26:22.156922    2574 generic.go:358] "Generic (PLEG): container finished" podID="2571829f-666a-422a-9b02-2d2bb4c89b25" containerID="738374c320fc522a27c64aa002715dc0f6eda5b36b913b52de3df99614697ff7" exitCode=255
Apr 24 14:26:22.157076 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:26:22.156986    2574 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-console-operator/console-operator-9d4b6777b-f6ngr" event={"ID":"2571829f-666a-422a-9b02-2d2bb4c89b25","Type":"ContainerDied","Data":"738374c320fc522a27c64aa002715dc0f6eda5b36b913b52de3df99614697ff7"}
Apr 24 14:26:22.157351 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:26:22.157307    2574 scope.go:117] "RemoveContainer" containerID="738374c320fc522a27c64aa002715dc0f6eda5b36b913b52de3df99614697ff7"
Apr 24 14:26:22.158718 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:26:22.158683    2574 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca-operator/service-ca-operator-d6fc45fc5-nwsrw" event={"ID":"ddb5d8a7-6366-44cb-a7d4-58e3627b0187","Type":"ContainerStarted","Data":"dc090e14ce6ba899508244969eca5ba01315fc2c2d0cfaa471ef040fc0db0a9d"}
Apr 24 14:26:22.160288 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:26:22.160268    2574 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-source-8894fc9bd-kj5df" event={"ID":"c429617c-d32e-4466-b249-d0a28cfd5b9d","Type":"ContainerStarted","Data":"a3e86269e244c4c32b0733da1f4bcd29344f7c64f3af14b6ed75c64bbbbc5cf5"}
Apr 24 14:26:22.160386 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:26:22.160292    2574 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-source-8894fc9bd-kj5df" event={"ID":"c429617c-d32e-4466-b249-d0a28cfd5b9d","Type":"ContainerStarted","Data":"c058abaa72a2204a990c0ea5ad87e3c9b98526a2925b33ce39858d5875d1b5bc"}
Apr 24 14:26:22.161781 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:26:22.161742    2574 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-6769c5d45-rmc8n" event={"ID":"5bc2f6f9-dff9-48b0-bc63-5f778a1ed590","Type":"ContainerStarted","Data":"b01cb06bbbf923aeac282247b0b055b3c183ba9fc8c128a0fc3f76c9397bf060"}
Apr 24 14:26:22.163254 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:26:22.163229    2574 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-storage-operator/volume-data-source-validator-7c6cbb6c87-j2gr5" event={"ID":"de9f9e0b-653f-481d-b23f-5c408b5cc75e","Type":"ContainerStarted","Data":"fca271b5c973ef4dbeaea23bc50d33ede2b90410a8b353533f133b24fcd3e20a"}
Apr 24 14:26:22.185458 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:26:22.185413    2574 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-6769c5d45-rmc8n" podStartSLOduration=0.98615087 podStartE2EDuration="4.185397149s" podCreationTimestamp="2026-04-24 14:26:18 +0000 UTC" firstStartedPulling="2026-04-24 14:26:18.477671197 +0000 UTC m=+141.209772829" lastFinishedPulling="2026-04-24 14:26:21.676917465 +0000 UTC m=+144.409019108" observedRunningTime="2026-04-24 14:26:22.184863903 +0000 UTC m=+144.916965556" watchObservedRunningTime="2026-04-24 14:26:22.185397149 +0000 UTC m=+144.917498798"
Apr 24 14:26:22.197306 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:26:22.197258    2574 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-service-ca-operator/service-ca-operator-d6fc45fc5-nwsrw" podStartSLOduration=1.106117924 podStartE2EDuration="4.197239558s" podCreationTimestamp="2026-04-24 14:26:18 +0000 UTC" firstStartedPulling="2026-04-24 14:26:18.591687241 +0000 UTC m=+141.323788871" lastFinishedPulling="2026-04-24 14:26:21.682808868 +0000 UTC m=+144.414910505" observedRunningTime="2026-04-24 14:26:22.196715318 +0000 UTC m=+144.928816985" watchObservedRunningTime="2026-04-24 14:26:22.197239558 +0000 UTC m=+144.929341208"
Apr 24 14:26:22.211631 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:26:22.210799    2574 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-cluster-storage-operator/volume-data-source-validator-7c6cbb6c87-j2gr5" podStartSLOduration=1.138860084 podStartE2EDuration="4.210780945s" podCreationTimestamp="2026-04-24 14:26:18 +0000 UTC" firstStartedPulling="2026-04-24 14:26:18.605202706 +0000 UTC m=+141.337304348" lastFinishedPulling="2026-04-24 14:26:21.677123582 +0000 UTC m=+144.409225209" observedRunningTime="2026-04-24 14:26:22.209843055 +0000 UTC m=+144.941944707" watchObservedRunningTime="2026-04-24 14:26:22.210780945 +0000 UTC m=+144.942882595"
Apr 24 14:26:22.224864 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:26:22.224815    2574 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-network-diagnostics/network-check-source-8894fc9bd-kj5df" podStartSLOduration=1.224798521 podStartE2EDuration="1.224798521s" podCreationTimestamp="2026-04-24 14:26:21 +0000 UTC" firstStartedPulling="0001-01-01 00:00:00 +0000 UTC" lastFinishedPulling="0001-01-01 00:00:00 +0000 UTC" observedRunningTime="2026-04-24 14:26:22.222946493 +0000 UTC m=+144.955048156" watchObservedRunningTime="2026-04-24 14:26:22.224798521 +0000 UTC m=+144.956900174"
Apr 24 14:26:23.004216 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:26:23.004131    2574 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["openshift-kube-storage-version-migrator/migrator-74bb7799d9-xcnvv"]
Apr 24 14:26:23.007622 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:26:23.007605    2574 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-kube-storage-version-migrator/migrator-74bb7799d9-xcnvv"
Apr 24 14:26:23.010317 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:26:23.010282    2574 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator\"/\"kube-root-ca.crt\""
Apr 24 14:26:23.010688 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:26:23.010669    2574 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator\"/\"openshift-service-ca.crt\""
Apr 24 14:26:23.010688 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:26:23.010686    2574 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-kube-storage-version-migrator\"/\"kube-storage-version-migrator-sa-dockercfg-6mcg5\""
Apr 24 14:26:23.016147 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:26:23.016120    2574 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-kube-storage-version-migrator/migrator-74bb7799d9-xcnvv"]
Apr 24 14:26:23.110730 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:26:23.110685    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wb4db\" (UniqueName: \"kubernetes.io/projected/a1182186-de0f-42da-82d2-92fa7ed16130-kube-api-access-wb4db\") pod \"migrator-74bb7799d9-xcnvv\" (UID: \"a1182186-de0f-42da-82d2-92fa7ed16130\") " pod="openshift-kube-storage-version-migrator/migrator-74bb7799d9-xcnvv"
Apr 24 14:26:23.167184 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:26:23.167161    2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-9d4b6777b-f6ngr_2571829f-666a-422a-9b02-2d2bb4c89b25/console-operator/1.log"
Apr 24 14:26:23.167592 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:26:23.167573    2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-9d4b6777b-f6ngr_2571829f-666a-422a-9b02-2d2bb4c89b25/console-operator/0.log"
Apr 24 14:26:23.167655 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:26:23.167616    2574 generic.go:358] "Generic (PLEG): container finished" podID="2571829f-666a-422a-9b02-2d2bb4c89b25" containerID="0cd856b57d33036a858491ca39d0f512a802afd840f4615ff351f846414d2cce" exitCode=255
Apr 24 14:26:23.167751 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:26:23.167719    2574 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-console-operator/console-operator-9d4b6777b-f6ngr" event={"ID":"2571829f-666a-422a-9b02-2d2bb4c89b25","Type":"ContainerDied","Data":"0cd856b57d33036a858491ca39d0f512a802afd840f4615ff351f846414d2cce"}
Apr 24 14:26:23.167887 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:26:23.167770    2574 scope.go:117] "RemoveContainer" containerID="738374c320fc522a27c64aa002715dc0f6eda5b36b913b52de3df99614697ff7"
Apr 24 14:26:23.167959 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:26:23.167927    2574 scope.go:117] "RemoveContainer" containerID="0cd856b57d33036a858491ca39d0f512a802afd840f4615ff351f846414d2cce"
Apr 24 14:26:23.168746 ip-10-0-136-132 kubenswrapper[2574]: E0424 14:26:23.168709    2574 pod_workers.go:1301] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"console-operator\" with CrashLoopBackOff: \"back-off 10s restarting failed container=console-operator pod=console-operator-9d4b6777b-f6ngr_openshift-console-operator(2571829f-666a-422a-9b02-2d2bb4c89b25)\"" pod="openshift-console-operator/console-operator-9d4b6777b-f6ngr" podUID="2571829f-666a-422a-9b02-2d2bb4c89b25"
Apr 24 14:26:23.211671 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:26:23.211633    2574 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-wb4db\" (UniqueName: \"kubernetes.io/projected/a1182186-de0f-42da-82d2-92fa7ed16130-kube-api-access-wb4db\") pod \"migrator-74bb7799d9-xcnvv\" (UID: \"a1182186-de0f-42da-82d2-92fa7ed16130\") " pod="openshift-kube-storage-version-migrator/migrator-74bb7799d9-xcnvv"
Apr 24 14:26:23.220227 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:26:23.220199    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-wb4db\" (UniqueName: \"kubernetes.io/projected/a1182186-de0f-42da-82d2-92fa7ed16130-kube-api-access-wb4db\") pod \"migrator-74bb7799d9-xcnvv\" (UID: \"a1182186-de0f-42da-82d2-92fa7ed16130\") " pod="openshift-kube-storage-version-migrator/migrator-74bb7799d9-xcnvv"
Apr 24 14:26:23.318400 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:26:23.318300    2574 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-kube-storage-version-migrator/migrator-74bb7799d9-xcnvv"
Apr 24 14:26:23.437202 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:26:23.437117    2574 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-kube-storage-version-migrator/migrator-74bb7799d9-xcnvv"]
Apr 24 14:26:23.439697 ip-10-0-136-132 kubenswrapper[2574]: W0424 14:26:23.439657    2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda1182186_de0f_42da_82d2_92fa7ed16130.slice/crio-d0badf14d28de8698a31a0de09e89d2420548cbdbf40b441262af875a982fedc WatchSource:0}: Error finding container d0badf14d28de8698a31a0de09e89d2420548cbdbf40b441262af875a982fedc: Status 404 returned error can't find the container with id d0badf14d28de8698a31a0de09e89d2420548cbdbf40b441262af875a982fedc
Apr 24 14:26:23.924328 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:26:23.924301    2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-bt99l_0b8fb5e7-eaa1-4b51-9f84-37bb0eb1a711/dns-node-resolver/0.log"
Apr 24 14:26:24.173358 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:26:24.173294    2574 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator/migrator-74bb7799d9-xcnvv" event={"ID":"a1182186-de0f-42da-82d2-92fa7ed16130","Type":"ContainerStarted","Data":"d0badf14d28de8698a31a0de09e89d2420548cbdbf40b441262af875a982fedc"}
Apr 24 14:26:24.175150 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:26:24.175075    2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-9d4b6777b-f6ngr_2571829f-666a-422a-9b02-2d2bb4c89b25/console-operator/1.log"
Apr 24 14:26:24.175510 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:26:24.175496    2574 scope.go:117] "RemoveContainer" containerID="0cd856b57d33036a858491ca39d0f512a802afd840f4615ff351f846414d2cce"
Apr 24 14:26:24.175686 ip-10-0-136-132 kubenswrapper[2574]: E0424 14:26:24.175670    2574 pod_workers.go:1301] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"console-operator\" with CrashLoopBackOff: \"back-off 10s restarting failed container=console-operator pod=console-operator-9d4b6777b-f6ngr_openshift-console-operator(2571829f-666a-422a-9b02-2d2bb4c89b25)\"" pod="openshift-console-operator/console-operator-9d4b6777b-f6ngr" podUID="2571829f-666a-422a-9b02-2d2bb4c89b25"
Apr 24 14:26:24.923117 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:26:24.923090    2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-lgh9n_5e4a7a90-d6ce-491b-ad65-52f0b2a127d6/node-ca/0.log"
Apr 24 14:26:25.123005 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:26:25.122966    2574 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["openshift-service-ca/service-ca-865cb79987-9krmv"]
Apr 24 14:26:25.126035 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:26:25.126011    2574 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-service-ca/service-ca-865cb79987-9krmv"
Apr 24 14:26:25.127851 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:26:25.127831    2574 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca\"/\"kube-root-ca.crt\""
Apr 24 14:26:25.127959 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:26:25.127868    2574 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca\"/\"signing-cabundle\""
Apr 24 14:26:25.128275 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:26:25.128250    2574 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca\"/\"openshift-service-ca.crt\""
Apr 24 14:26:25.128393 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:26:25.128299    2574 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-service-ca\"/\"service-ca-dockercfg-f77n4\""
Apr 24 14:26:25.128393 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:26:25.128317    2574 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-service-ca\"/\"signing-key\""
Apr 24 14:26:25.133218 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:26:25.133197    2574 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-service-ca/service-ca-865cb79987-9krmv"]
Apr 24 14:26:25.179828 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:26:25.179743    2574 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator/migrator-74bb7799d9-xcnvv" event={"ID":"a1182186-de0f-42da-82d2-92fa7ed16130","Type":"ContainerStarted","Data":"1c2138471a099b072ef145585a156c1adb592c8d62210a06a61b91fef8d236cd"}
Apr 24 14:26:25.179828 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:26:25.179783    2574 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator/migrator-74bb7799d9-xcnvv" event={"ID":"a1182186-de0f-42da-82d2-92fa7ed16130","Type":"ContainerStarted","Data":"40dc63e159d383c4a733b1cf4834f0f5369bd4fc45ec74b26d740ff91dc357b5"}
Apr 24 14:26:25.194096 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:26:25.194044    2574 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-kube-storage-version-migrator/migrator-74bb7799d9-xcnvv" podStartSLOduration=2.261905204 podStartE2EDuration="3.194028986s" podCreationTimestamp="2026-04-24 14:26:22 +0000 UTC" firstStartedPulling="2026-04-24 14:26:23.441585619 +0000 UTC m=+146.173687246" lastFinishedPulling="2026-04-24 14:26:24.373709385 +0000 UTC m=+147.105811028" observedRunningTime="2026-04-24 14:26:25.192985973 +0000 UTC m=+147.925087644" watchObservedRunningTime="2026-04-24 14:26:25.194028986 +0000 UTC m=+147.926130634"
Apr 24 14:26:25.225700 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:26:25.225658    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"signing-key\" (UniqueName: \"kubernetes.io/secret/2e3b1b21-f481-4e7d-a951-34d8eefe2d6e-signing-key\") pod \"service-ca-865cb79987-9krmv\" (UID: \"2e3b1b21-f481-4e7d-a951-34d8eefe2d6e\") " pod="openshift-service-ca/service-ca-865cb79987-9krmv"
Apr 24 14:26:25.225700 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:26:25.225714    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rrllj\" (UniqueName: \"kubernetes.io/projected/2e3b1b21-f481-4e7d-a951-34d8eefe2d6e-kube-api-access-rrllj\") pod \"service-ca-865cb79987-9krmv\" (UID: \"2e3b1b21-f481-4e7d-a951-34d8eefe2d6e\") " pod="openshift-service-ca/service-ca-865cb79987-9krmv"
Apr 24 14:26:25.225898 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:26:25.225774    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"signing-cabundle\" (UniqueName: \"kubernetes.io/configmap/2e3b1b21-f481-4e7d-a951-34d8eefe2d6e-signing-cabundle\") pod \"service-ca-865cb79987-9krmv\" (UID: \"2e3b1b21-f481-4e7d-a951-34d8eefe2d6e\") " pod="openshift-service-ca/service-ca-865cb79987-9krmv"
Apr 24 14:26:25.327203 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:26:25.327152    2574 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"signing-key\" (UniqueName: \"kubernetes.io/secret/2e3b1b21-f481-4e7d-a951-34d8eefe2d6e-signing-key\") pod \"service-ca-865cb79987-9krmv\" (UID: \"2e3b1b21-f481-4e7d-a951-34d8eefe2d6e\") " pod="openshift-service-ca/service-ca-865cb79987-9krmv"
Apr 24 14:26:25.327414 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:26:25.327235    2574 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-rrllj\" (UniqueName: \"kubernetes.io/projected/2e3b1b21-f481-4e7d-a951-34d8eefe2d6e-kube-api-access-rrllj\") pod \"service-ca-865cb79987-9krmv\" (UID: \"2e3b1b21-f481-4e7d-a951-34d8eefe2d6e\") " pod="openshift-service-ca/service-ca-865cb79987-9krmv"
Apr 24 14:26:25.327414 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:26:25.327287    2574 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"signing-cabundle\" (UniqueName: \"kubernetes.io/configmap/2e3b1b21-f481-4e7d-a951-34d8eefe2d6e-signing-cabundle\") pod \"service-ca-865cb79987-9krmv\" (UID: \"2e3b1b21-f481-4e7d-a951-34d8eefe2d6e\") " pod="openshift-service-ca/service-ca-865cb79987-9krmv"
Apr 24 14:26:25.327946 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:26:25.327922    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"signing-cabundle\" (UniqueName: \"kubernetes.io/configmap/2e3b1b21-f481-4e7d-a951-34d8eefe2d6e-signing-cabundle\") pod \"service-ca-865cb79987-9krmv\" (UID: \"2e3b1b21-f481-4e7d-a951-34d8eefe2d6e\") " pod="openshift-service-ca/service-ca-865cb79987-9krmv"
Apr 24 14:26:25.329615 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:26:25.329595    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"signing-key\" (UniqueName: \"kubernetes.io/secret/2e3b1b21-f481-4e7d-a951-34d8eefe2d6e-signing-key\") pod \"service-ca-865cb79987-9krmv\" (UID: \"2e3b1b21-f481-4e7d-a951-34d8eefe2d6e\") " pod="openshift-service-ca/service-ca-865cb79987-9krmv"
Apr 24 14:26:25.335185 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:26:25.335156    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-rrllj\" (UniqueName: \"kubernetes.io/projected/2e3b1b21-f481-4e7d-a951-34d8eefe2d6e-kube-api-access-rrllj\") pod \"service-ca-865cb79987-9krmv\" (UID: \"2e3b1b21-f481-4e7d-a951-34d8eefe2d6e\") " pod="openshift-service-ca/service-ca-865cb79987-9krmv"
Apr 24 14:26:25.436194 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:26:25.436106    2574 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-service-ca/service-ca-865cb79987-9krmv"
Apr 24 14:26:25.551240 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:26:25.551209    2574 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-service-ca/service-ca-865cb79987-9krmv"]
Apr 24 14:26:25.554191 ip-10-0-136-132 kubenswrapper[2574]: W0424 14:26:25.554165    2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2e3b1b21_f481_4e7d_a951_34d8eefe2d6e.slice/crio-1aea0f794c22d227cba89329c834c6f2e08c6fc87100d3b4960301392d5dff4c WatchSource:0}: Error finding container 1aea0f794c22d227cba89329c834c6f2e08c6fc87100d3b4960301392d5dff4c: Status 404 returned error can't find the container with id 1aea0f794c22d227cba89329c834c6f2e08c6fc87100d3b4960301392d5dff4c
Apr 24 14:26:25.831389 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:26:25.831278    2574 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/0be19048-1b01-4c64-bcca-94c9bff7768d-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-75587bd455-q9jbv\" (UID: \"0be19048-1b01-4c64-bcca-94c9bff7768d\") " pod="openshift-monitoring/cluster-monitoring-operator-75587bd455-q9jbv"
Apr 24 14:26:25.831521 ip-10-0-136-132 kubenswrapper[2574]: E0424 14:26:25.831411    2574 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found
Apr 24 14:26:25.831521 ip-10-0-136-132 kubenswrapper[2574]: E0424 14:26:25.831466    2574 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/0be19048-1b01-4c64-bcca-94c9bff7768d-cluster-monitoring-operator-tls podName:0be19048-1b01-4c64-bcca-94c9bff7768d nodeName:}" failed. No retries permitted until 2026-04-24 14:26:33.831451994 +0000 UTC m=+156.563553620 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/0be19048-1b01-4c64-bcca-94c9bff7768d-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-75587bd455-q9jbv" (UID: "0be19048-1b01-4c64-bcca-94c9bff7768d") : secret "cluster-monitoring-operator-tls" not found
Apr 24 14:26:26.183734 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:26:26.183694    2574 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca/service-ca-865cb79987-9krmv" event={"ID":"2e3b1b21-f481-4e7d-a951-34d8eefe2d6e","Type":"ContainerStarted","Data":"ab3d62bc89f23a54ddc5688a619f73e0b24717d9fb5ec19f5b3b7a2680222c7e"}
Apr 24 14:26:26.183734 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:26:26.183738    2574 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca/service-ca-865cb79987-9krmv" event={"ID":"2e3b1b21-f481-4e7d-a951-34d8eefe2d6e","Type":"ContainerStarted","Data":"1aea0f794c22d227cba89329c834c6f2e08c6fc87100d3b4960301392d5dff4c"}
Apr 24 14:26:26.199599 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:26:26.199549    2574 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-service-ca/service-ca-865cb79987-9krmv" podStartSLOduration=1.199534584 podStartE2EDuration="1.199534584s" podCreationTimestamp="2026-04-24 14:26:25 +0000 UTC" firstStartedPulling="0001-01-01 00:00:00 +0000 UTC" lastFinishedPulling="0001-01-01 00:00:00 +0000 UTC" observedRunningTime="2026-04-24 14:26:26.198647754 +0000 UTC m=+148.930749416" watchObservedRunningTime="2026-04-24 14:26:26.199534584 +0000 UTC m=+148.931636232"
Apr 24 14:26:28.469248 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:26:28.469205    2574 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console-operator/console-operator-9d4b6777b-f6ngr"
Apr 24 14:26:28.469631 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:26:28.469261    2574 kubelet.go:2658] "SyncLoop (probe)" probe="liveness" status="unhealthy" pod="openshift-console-operator/console-operator-9d4b6777b-f6ngr"
Apr 24 14:26:28.469666 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:26:28.469658    2574 scope.go:117] "RemoveContainer" containerID="0cd856b57d33036a858491ca39d0f512a802afd840f4615ff351f846414d2cce"
Apr 24 14:26:28.469838 ip-10-0-136-132 kubenswrapper[2574]: E0424 14:26:28.469821    2574 pod_workers.go:1301] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"console-operator\" with CrashLoopBackOff: \"back-off 10s restarting failed container=console-operator pod=console-operator-9d4b6777b-f6ngr_openshift-console-operator(2571829f-666a-422a-9b02-2d2bb4c89b25)\"" pod="openshift-console-operator/console-operator-9d4b6777b-f6ngr" podUID="2571829f-666a-422a-9b02-2d2bb4c89b25"
Apr 24 14:26:33.249792 ip-10-0-136-132 kubenswrapper[2574]: E0424 14:26:33.249746    2574 pod_workers.go:1301] "Error syncing pod, skipping" err="unmounted volumes=[cert], unattached volumes=[], failed to process volumes=[]: context deadline exceeded" pod="openshift-ingress-canary/ingress-canary-tzbfv" podUID="4ed72386-1812-4697-9811-c3534a37dc95"
Apr 24 14:26:33.279565 ip-10-0-136-132 kubenswrapper[2574]: E0424 14:26:33.279531    2574 pod_workers.go:1301] "Error syncing pod, skipping" err="unmounted volumes=[metrics-tls], unattached volumes=[], failed to process volumes=[]: context deadline exceeded" pod="openshift-dns/dns-default-nvz9x" podUID="b4b7957c-9f4f-4c2d-ab2b-85c958db734a"
Apr 24 14:26:33.886646 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:26:33.886604    2574 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/0be19048-1b01-4c64-bcca-94c9bff7768d-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-75587bd455-q9jbv\" (UID: \"0be19048-1b01-4c64-bcca-94c9bff7768d\") " pod="openshift-monitoring/cluster-monitoring-operator-75587bd455-q9jbv"
Apr 24 14:26:33.886830 ip-10-0-136-132 kubenswrapper[2574]: E0424 14:26:33.886753    2574 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found
Apr 24 14:26:33.886871 ip-10-0-136-132 kubenswrapper[2574]: E0424 14:26:33.886842    2574 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/0be19048-1b01-4c64-bcca-94c9bff7768d-cluster-monitoring-operator-tls podName:0be19048-1b01-4c64-bcca-94c9bff7768d nodeName:}" failed. No retries permitted until 2026-04-24 14:26:49.886825941 +0000 UTC m=+172.618927568 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/0be19048-1b01-4c64-bcca-94c9bff7768d-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-75587bd455-q9jbv" (UID: "0be19048-1b01-4c64-bcca-94c9bff7768d") : secret "cluster-monitoring-operator-tls" not found
Apr 24 14:26:34.206934 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:26:34.206852    2574 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-tzbfv"
Apr 24 14:26:34.790090 ip-10-0-136-132 kubenswrapper[2574]: E0424 14:26:34.790054    2574 pod_workers.go:1301] "Error syncing pod, skipping" err="unmounted volumes=[metrics-certs], unattached volumes=[], failed to process volumes=[]: context deadline exceeded" pod="openshift-multus/network-metrics-daemon-zhkmq" podUID="3bc44c73-963f-4a9e-8a52-1e8b031a08dc"
Apr 24 14:26:38.116238 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:26:38.116176    2574 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/4ed72386-1812-4697-9811-c3534a37dc95-cert\") pod \"ingress-canary-tzbfv\" (UID: \"4ed72386-1812-4697-9811-c3534a37dc95\") " pod="openshift-ingress-canary/ingress-canary-tzbfv"
Apr 24 14:26:38.118576 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:26:38.118544    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/4ed72386-1812-4697-9811-c3534a37dc95-cert\") pod \"ingress-canary-tzbfv\" (UID: \"4ed72386-1812-4697-9811-c3534a37dc95\") " pod="openshift-ingress-canary/ingress-canary-tzbfv"
Apr 24 14:26:38.217043 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:26:38.217012    2574 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/b4b7957c-9f4f-4c2d-ab2b-85c958db734a-metrics-tls\") pod \"dns-default-nvz9x\" (UID: \"b4b7957c-9f4f-4c2d-ab2b-85c958db734a\") " pod="openshift-dns/dns-default-nvz9x"
Apr 24 14:26:38.219661 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:26:38.219635    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/b4b7957c-9f4f-4c2d-ab2b-85c958db734a-metrics-tls\") pod \"dns-default-nvz9x\" (UID: \"b4b7957c-9f4f-4c2d-ab2b-85c958db734a\") " pod="openshift-dns/dns-default-nvz9x"
Apr 24 14:26:38.409766 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:26:38.409738    2574 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"default-dockercfg-9v25p\""
Apr 24 14:26:38.418399 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:26:38.418360    2574 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-tzbfv"
Apr 24 14:26:38.550166 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:26:38.549972    2574 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-tzbfv"]
Apr 24 14:26:38.552615 ip-10-0-136-132 kubenswrapper[2574]: W0424 14:26:38.552584    2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4ed72386_1812_4697_9811_c3534a37dc95.slice/crio-560a9cf5e723c7387063978034717df34eae1fb1a6977d8455570d045eb3d1a5 WatchSource:0}: Error finding container 560a9cf5e723c7387063978034717df34eae1fb1a6977d8455570d045eb3d1a5: Status 404 returned error can't find the container with id 560a9cf5e723c7387063978034717df34eae1fb1a6977d8455570d045eb3d1a5
Apr 24 14:26:39.221180 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:26:39.221144    2574 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-tzbfv" event={"ID":"4ed72386-1812-4697-9811-c3534a37dc95","Type":"ContainerStarted","Data":"560a9cf5e723c7387063978034717df34eae1fb1a6977d8455570d045eb3d1a5"}
Apr 24 14:26:39.765662 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:26:39.765634    2574 scope.go:117] "RemoveContainer" containerID="0cd856b57d33036a858491ca39d0f512a802afd840f4615ff351f846414d2cce"
Apr 24 14:26:41.227965 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:26:41.227938    2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-9d4b6777b-f6ngr_2571829f-666a-422a-9b02-2d2bb4c89b25/console-operator/1.log"
Apr 24 14:26:41.228403 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:26:41.228012    2574 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-console-operator/console-operator-9d4b6777b-f6ngr" event={"ID":"2571829f-666a-422a-9b02-2d2bb4c89b25","Type":"ContainerStarted","Data":"c78b400a91c66625bc6c27f88641ac3a24dab55e7ed862c2a1838e3cbf1a29f0"}
Apr 24 14:26:41.228403 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:26:41.228291    2574 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console-operator/console-operator-9d4b6777b-f6ngr"
Apr 24 14:26:41.229535 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:26:41.229503    2574 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-tzbfv" event={"ID":"4ed72386-1812-4697-9811-c3534a37dc95","Type":"ContainerStarted","Data":"dfc5738efe8457b136af3ccc3efee1ea1f64486288e13b7a1f6cce017ac9160e"}
Apr 24 14:26:41.233116 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:26:41.233092    2574 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console-operator/console-operator-9d4b6777b-f6ngr"
Apr 24 14:26:41.244448 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:26:41.244406    2574 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-console-operator/console-operator-9d4b6777b-f6ngr" podStartSLOduration=20.186908878 podStartE2EDuration="23.244391933s" podCreationTimestamp="2026-04-24 14:26:18 +0000 UTC" firstStartedPulling="2026-04-24 14:26:18.621241612 +0000 UTC m=+141.353343239" lastFinishedPulling="2026-04-24 14:26:21.678724668 +0000 UTC m=+144.410826294" observedRunningTime="2026-04-24 14:26:41.243804106 +0000 UTC m=+163.975905753" watchObservedRunningTime="2026-04-24 14:26:41.244391933 +0000 UTC m=+163.976493581"
Apr 24 14:26:41.256411 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:26:41.256368    2574 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-ingress-canary/ingress-canary-tzbfv" podStartSLOduration=129.567026012 podStartE2EDuration="2m11.256355388s" podCreationTimestamp="2026-04-24 14:24:30 +0000 UTC" firstStartedPulling="2026-04-24 14:26:38.554500927 +0000 UTC m=+161.286602553" lastFinishedPulling="2026-04-24 14:26:40.243830299 +0000 UTC m=+162.975931929" observedRunningTime="2026-04-24 14:26:41.255951936 +0000 UTC m=+163.988053600" watchObservedRunningTime="2026-04-24 14:26:41.256355388 +0000 UTC m=+163.988457065"
Apr 24 14:26:47.451946 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:26:47.451909    2574 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["openshift-network-console/networking-console-plugin-cb95c66f6-96ndb"]
Apr 24 14:26:47.456504 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:26:47.456479    2574 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-console/networking-console-plugin-cb95c66f6-96ndb"
Apr 24 14:26:47.458929 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:26:47.458904    2574 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-console\"/\"default-dockercfg-dmhdz\""
Apr 24 14:26:47.459032 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:26:47.458906    2574 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-console\"/\"networking-console-plugin-cert\""
Apr 24 14:26:47.459032 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:26:47.458906    2574 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-console\"/\"networking-console-plugin\""
Apr 24 14:26:47.467036 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:26:47.467012    2574 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-network-console/networking-console-plugin-cb95c66f6-96ndb"]
Apr 24 14:26:47.477511 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:26:47.476564    2574 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["openshift-insights/insights-runtime-extractor-qgxfn"]
Apr 24 14:26:47.480861 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:26:47.480839    2574 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-qgxfn"
Apr 24 14:26:47.481921 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:26:47.481901    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/6a47c891-13d9-4726-8204-e7559e6e6ff4-networking-console-plugin-cert\") pod \"networking-console-plugin-cb95c66f6-96ndb\" (UID: \"6a47c891-13d9-4726-8204-e7559e6e6ff4\") " pod="openshift-network-console/networking-console-plugin-cb95c66f6-96ndb"
Apr 24 14:26:47.482009 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:26:47.481927    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/6a47c891-13d9-4726-8204-e7559e6e6ff4-nginx-conf\") pod \"networking-console-plugin-cb95c66f6-96ndb\" (UID: \"6a47c891-13d9-4726-8204-e7559e6e6ff4\") " pod="openshift-network-console/networking-console-plugin-cb95c66f6-96ndb"
Apr 24 14:26:47.484418 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:26:47.484399    2574 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-rbac-proxy\""
Apr 24 14:26:47.484526 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:26:47.484442    2574 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-root-ca.crt\""
Apr 24 14:26:47.484584 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:26:47.484547    2574 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-sa-dockercfg-zf9np\""
Apr 24 14:26:47.485040 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:26:47.485024    2574 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"openshift-service-ca.crt\""
Apr 24 14:26:47.490797 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:26:47.490780    2574 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-tls\""
Apr 24 14:26:47.504430 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:26:47.504400    2574 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-qgxfn"]
Apr 24 14:26:47.582428 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:26:47.582396    2574 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/6a47c891-13d9-4726-8204-e7559e6e6ff4-networking-console-plugin-cert\") pod \"networking-console-plugin-cb95c66f6-96ndb\" (UID: \"6a47c891-13d9-4726-8204-e7559e6e6ff4\") " pod="openshift-network-console/networking-console-plugin-cb95c66f6-96ndb"
Apr 24 14:26:47.582581 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:26:47.582440    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/660bbad4-5502-418e-a02d-869defb74468-data-volume\") pod \"insights-runtime-extractor-qgxfn\" (UID: \"660bbad4-5502-418e-a02d-869defb74468\") " pod="openshift-insights/insights-runtime-extractor-qgxfn"
Apr 24 14:26:47.582581 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:26:47.582474    2574 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/6a47c891-13d9-4726-8204-e7559e6e6ff4-nginx-conf\") pod \"networking-console-plugin-cb95c66f6-96ndb\" (UID: \"6a47c891-13d9-4726-8204-e7559e6e6ff4\") " pod="openshift-network-console/networking-console-plugin-cb95c66f6-96ndb"
Apr 24 14:26:47.582581 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:26:47.582523    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/660bbad4-5502-418e-a02d-869defb74468-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-qgxfn\" (UID: \"660bbad4-5502-418e-a02d-869defb74468\") " pod="openshift-insights/insights-runtime-extractor-qgxfn"
Apr 24 14:26:47.582708 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:26:47.582613    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/660bbad4-5502-418e-a02d-869defb74468-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-qgxfn\" (UID: \"660bbad4-5502-418e-a02d-869defb74468\") " pod="openshift-insights/insights-runtime-extractor-qgxfn"
Apr 24 14:26:47.582708 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:26:47.582645    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/660bbad4-5502-418e-a02d-869defb74468-crio-socket\") pod \"insights-runtime-extractor-qgxfn\" (UID: \"660bbad4-5502-418e-a02d-869defb74468\") " pod="openshift-insights/insights-runtime-extractor-qgxfn"
Apr 24 14:26:47.582708 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:26:47.582681    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9bfdp\" (UniqueName: \"kubernetes.io/projected/660bbad4-5502-418e-a02d-869defb74468-kube-api-access-9bfdp\") pod \"insights-runtime-extractor-qgxfn\" (UID: \"660bbad4-5502-418e-a02d-869defb74468\") " pod="openshift-insights/insights-runtime-extractor-qgxfn"
Apr 24 14:26:47.583095 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:26:47.583077    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/6a47c891-13d9-4726-8204-e7559e6e6ff4-nginx-conf\") pod \"networking-console-plugin-cb95c66f6-96ndb\" (UID: \"6a47c891-13d9-4726-8204-e7559e6e6ff4\") " pod="openshift-network-console/networking-console-plugin-cb95c66f6-96ndb"
Apr 24 14:26:47.584921 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:26:47.584900    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/6a47c891-13d9-4726-8204-e7559e6e6ff4-networking-console-plugin-cert\") pod \"networking-console-plugin-cb95c66f6-96ndb\" (UID: \"6a47c891-13d9-4726-8204-e7559e6e6ff4\") " pod="openshift-network-console/networking-console-plugin-cb95c66f6-96ndb"
Apr 24 14:26:47.683698 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:26:47.683657    2574 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-9bfdp\" (UniqueName: \"kubernetes.io/projected/660bbad4-5502-418e-a02d-869defb74468-kube-api-access-9bfdp\") pod \"insights-runtime-extractor-qgxfn\" (UID: \"660bbad4-5502-418e-a02d-869defb74468\") " pod="openshift-insights/insights-runtime-extractor-qgxfn"
Apr 24 14:26:47.683879 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:26:47.683737    2574 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/660bbad4-5502-418e-a02d-869defb74468-data-volume\") pod \"insights-runtime-extractor-qgxfn\" (UID: \"660bbad4-5502-418e-a02d-869defb74468\") " pod="openshift-insights/insights-runtime-extractor-qgxfn"
Apr 24 14:26:47.683879 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:26:47.683780    2574 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/660bbad4-5502-418e-a02d-869defb74468-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-qgxfn\" (UID: \"660bbad4-5502-418e-a02d-869defb74468\") " pod="openshift-insights/insights-runtime-extractor-qgxfn"
Apr 24 14:26:47.683879 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:26:47.683857    2574 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/660bbad4-5502-418e-a02d-869defb74468-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-qgxfn\" (UID: \"660bbad4-5502-418e-a02d-869defb74468\") " pod="openshift-insights/insights-runtime-extractor-qgxfn"
Apr 24 14:26:47.684022 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:26:47.683883    2574 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/660bbad4-5502-418e-a02d-869defb74468-crio-socket\") pod \"insights-runtime-extractor-qgxfn\" (UID: \"660bbad4-5502-418e-a02d-869defb74468\") " pod="openshift-insights/insights-runtime-extractor-qgxfn"
Apr 24 14:26:47.684022 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:26:47.683967    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/660bbad4-5502-418e-a02d-869defb74468-crio-socket\") pod \"insights-runtime-extractor-qgxfn\" (UID: \"660bbad4-5502-418e-a02d-869defb74468\") " pod="openshift-insights/insights-runtime-extractor-qgxfn"
Apr 24 14:26:47.684190 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:26:47.684166    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/660bbad4-5502-418e-a02d-869defb74468-data-volume\") pod \"insights-runtime-extractor-qgxfn\" (UID: \"660bbad4-5502-418e-a02d-869defb74468\") " pod="openshift-insights/insights-runtime-extractor-qgxfn"
Apr 24 14:26:47.684515 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:26:47.684494    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/660bbad4-5502-418e-a02d-869defb74468-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-qgxfn\" (UID: \"660bbad4-5502-418e-a02d-869defb74468\") " pod="openshift-insights/insights-runtime-extractor-qgxfn"
Apr 24 14:26:47.686116 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:26:47.686093    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/660bbad4-5502-418e-a02d-869defb74468-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-qgxfn\" (UID: \"660bbad4-5502-418e-a02d-869defb74468\") " pod="openshift-insights/insights-runtime-extractor-qgxfn"
Apr 24 14:26:47.694584 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:26:47.694559    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-9bfdp\" (UniqueName: \"kubernetes.io/projected/660bbad4-5502-418e-a02d-869defb74468-kube-api-access-9bfdp\") pod \"insights-runtime-extractor-qgxfn\" (UID: \"660bbad4-5502-418e-a02d-869defb74468\") " pod="openshift-insights/insights-runtime-extractor-qgxfn"
Apr 24 14:26:47.765642 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:26:47.765558    2574 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-console/networking-console-plugin-cb95c66f6-96ndb"
Apr 24 14:26:47.767023 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:26:47.767001    2574 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-nvz9x"
Apr 24 14:26:47.768908 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:26:47.768885    2574 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-6p8rb\""
Apr 24 14:26:47.777428 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:26:47.777401    2574 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-nvz9x"
Apr 24 14:26:47.790138 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:26:47.790112    2574 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-qgxfn"
Apr 24 14:26:47.903571 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:26:47.903539    2574 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-network-console/networking-console-plugin-cb95c66f6-96ndb"]
Apr 24 14:26:47.912523 ip-10-0-136-132 kubenswrapper[2574]: W0424 14:26:47.912487    2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6a47c891_13d9_4726_8204_e7559e6e6ff4.slice/crio-33d2cc6cfe03ebff52ac2bba2c7f9fe1c31ed1d5f81e93d6fca205b4f377313e WatchSource:0}: Error finding container 33d2cc6cfe03ebff52ac2bba2c7f9fe1c31ed1d5f81e93d6fca205b4f377313e: Status 404 returned error can't find the container with id 33d2cc6cfe03ebff52ac2bba2c7f9fe1c31ed1d5f81e93d6fca205b4f377313e
Apr 24 14:26:47.925252 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:26:47.925226    2574 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-nvz9x"]
Apr 24 14:26:47.929081 ip-10-0-136-132 kubenswrapper[2574]: W0424 14:26:47.929049    2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb4b7957c_9f4f_4c2d_ab2b_85c958db734a.slice/crio-25263cf3254bbe8652768614f1ee776c1bf08c5b5d0098adc8dabe4a3c3aa625 WatchSource:0}: Error finding container 25263cf3254bbe8652768614f1ee776c1bf08c5b5d0098adc8dabe4a3c3aa625: Status 404 returned error can't find the container with id 25263cf3254bbe8652768614f1ee776c1bf08c5b5d0098adc8dabe4a3c3aa625
Apr 24 14:26:47.947290 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:26:47.947262    2574 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-qgxfn"]
Apr 24 14:26:47.950384 ip-10-0-136-132 kubenswrapper[2574]: W0424 14:26:47.950358    2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod660bbad4_5502_418e_a02d_869defb74468.slice/crio-d1cb254029166ecbb124a0ccd66ecee7e14e4e95052847b3f2320d8a1a918e2f WatchSource:0}: Error finding container d1cb254029166ecbb124a0ccd66ecee7e14e4e95052847b3f2320d8a1a918e2f: Status 404 returned error can't find the container with id d1cb254029166ecbb124a0ccd66ecee7e14e4e95052847b3f2320d8a1a918e2f
Apr 24 14:26:48.247802 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:26:48.247767    2574 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-network-console/networking-console-plugin-cb95c66f6-96ndb" event={"ID":"6a47c891-13d9-4726-8204-e7559e6e6ff4","Type":"ContainerStarted","Data":"33d2cc6cfe03ebff52ac2bba2c7f9fe1c31ed1d5f81e93d6fca205b4f377313e"}
Apr 24 14:26:48.248962 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:26:48.248934    2574 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-qgxfn" event={"ID":"660bbad4-5502-418e-a02d-869defb74468","Type":"ContainerStarted","Data":"5d3997193bf9b25362e256c9e370ffefd32b288d33fea727b626c5c6f4d74838"}
Apr 24 14:26:48.249087 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:26:48.248967    2574 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-qgxfn" event={"ID":"660bbad4-5502-418e-a02d-869defb74468","Type":"ContainerStarted","Data":"d1cb254029166ecbb124a0ccd66ecee7e14e4e95052847b3f2320d8a1a918e2f"}
Apr 24 14:26:48.249851 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:26:48.249830    2574 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-nvz9x" event={"ID":"b4b7957c-9f4f-4c2d-ab2b-85c958db734a","Type":"ContainerStarted","Data":"25263cf3254bbe8652768614f1ee776c1bf08c5b5d0098adc8dabe4a3c3aa625"}
Apr 24 14:26:48.766266 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:26:48.766234    2574 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-zhkmq"
Apr 24 14:26:49.255054 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:26:49.255014    2574 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-qgxfn" event={"ID":"660bbad4-5502-418e-a02d-869defb74468","Type":"ContainerStarted","Data":"f79b6895ae297ab215f6ee8dd774d06a04ebcef78befc17169b6399bc67ac6f5"}
Apr 24 14:26:49.902149 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:26:49.902112    2574 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/0be19048-1b01-4c64-bcca-94c9bff7768d-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-75587bd455-q9jbv\" (UID: \"0be19048-1b01-4c64-bcca-94c9bff7768d\") " pod="openshift-monitoring/cluster-monitoring-operator-75587bd455-q9jbv"
Apr 24 14:26:49.904744 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:26:49.904712    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/0be19048-1b01-4c64-bcca-94c9bff7768d-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-75587bd455-q9jbv\" (UID: \"0be19048-1b01-4c64-bcca-94c9bff7768d\") " pod="openshift-monitoring/cluster-monitoring-operator-75587bd455-q9jbv"
Apr 24 14:26:50.152255 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:26:50.152167    2574 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/cluster-monitoring-operator-75587bd455-q9jbv"
Apr 24 14:26:50.259038 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:26:50.258996    2574 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-network-console/networking-console-plugin-cb95c66f6-96ndb" event={"ID":"6a47c891-13d9-4726-8204-e7559e6e6ff4","Type":"ContainerStarted","Data":"f698698a32ccfd54b9f2dcbddc0bb295c746bcc02e94cd9aa1e4ad21f5d85209"}
Apr 24 14:26:50.260738 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:26:50.260707    2574 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-nvz9x" event={"ID":"b4b7957c-9f4f-4c2d-ab2b-85c958db734a","Type":"ContainerStarted","Data":"744878a25392e6bf85dbf39fd749ecde157dd238a7d41e4dc890c3f0e5951f90"}
Apr 24 14:26:50.260879 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:26:50.260741    2574 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-nvz9x" event={"ID":"b4b7957c-9f4f-4c2d-ab2b-85c958db734a","Type":"ContainerStarted","Data":"3647134670e4c08d7454aab445d1e2fee15ef0503bcb3a05ddf05edaab97acc3"}
Apr 24 14:26:50.260879 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:26:50.260830    2574 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-dns/dns-default-nvz9x"
Apr 24 14:26:50.274022 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:26:50.273976    2574 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-network-console/networking-console-plugin-cb95c66f6-96ndb" podStartSLOduration=1.8170007780000001 podStartE2EDuration="3.273960566s" podCreationTimestamp="2026-04-24 14:26:47 +0000 UTC" firstStartedPulling="2026-04-24 14:26:47.915073975 +0000 UTC m=+170.647175620" lastFinishedPulling="2026-04-24 14:26:49.372033764 +0000 UTC m=+172.104135408" observedRunningTime="2026-04-24 14:26:50.272613408 +0000 UTC m=+173.004715051" watchObservedRunningTime="2026-04-24 14:26:50.273960566 +0000 UTC m=+173.006062215"
Apr 24 14:26:50.296194 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:26:50.296144    2574 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-dns/dns-default-nvz9x" podStartSLOduration=138.851808059 podStartE2EDuration="2m20.296113883s" podCreationTimestamp="2026-04-24 14:24:30 +0000 UTC" firstStartedPulling="2026-04-24 14:26:47.931139657 +0000 UTC m=+170.663241285" lastFinishedPulling="2026-04-24 14:26:49.375445477 +0000 UTC m=+172.107547109" observedRunningTime="2026-04-24 14:26:50.290652375 +0000 UTC m=+173.022754024" watchObservedRunningTime="2026-04-24 14:26:50.296113883 +0000 UTC m=+173.028215534"
Apr 24 14:26:50.423091 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:26:50.423050    2574 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/cluster-monitoring-operator-75587bd455-q9jbv"]
Apr 24 14:26:50.425692 ip-10-0-136-132 kubenswrapper[2574]: W0424 14:26:50.425660    2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0be19048_1b01_4c64_bcca_94c9bff7768d.slice/crio-50080ad575d2b0b044d3df3650e8ad13bb57567ca2d083fa352840bcb64705d5 WatchSource:0}: Error finding container 50080ad575d2b0b044d3df3650e8ad13bb57567ca2d083fa352840bcb64705d5: Status 404 returned error can't find the container with id 50080ad575d2b0b044d3df3650e8ad13bb57567ca2d083fa352840bcb64705d5
Apr 24 14:26:51.164441 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:26:51.164404    2574 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["openshift-console/console-5cb8dd7bdc-bpg26"]
Apr 24 14:26:51.167759 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:26:51.167730    2574 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-5cb8dd7bdc-bpg26"
Apr 24 14:26:51.169963 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:26:51.169935    2574 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"console-config\""
Apr 24 14:26:51.170102 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:26:51.169937    2574 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"oauth-serving-cert\""
Apr 24 14:26:51.170206 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:26:51.170186    2574 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-oauth-config\""
Apr 24 14:26:51.170266 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:26:51.170215    2574 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"openshift-service-ca.crt\""
Apr 24 14:26:51.170616 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:26:51.170582    2574 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"service-ca\""
Apr 24 14:26:51.170616 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:26:51.170583    2574 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-serving-cert\""
Apr 24 14:26:51.170778 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:26:51.170623    2574 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"kube-root-ca.crt\""
Apr 24 14:26:51.170778 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:26:51.170630    2574 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-dockercfg-z8khd\""
Apr 24 14:26:51.175309 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:26:51.175284    2574 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-5cb8dd7bdc-bpg26"]
Apr 24 14:26:51.214400 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:26:51.214359    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/674cd4ff-c759-42ff-8b5b-f5e4eb2c45b7-oauth-serving-cert\") pod \"console-5cb8dd7bdc-bpg26\" (UID: \"674cd4ff-c759-42ff-8b5b-f5e4eb2c45b7\") " pod="openshift-console/console-5cb8dd7bdc-bpg26"
Apr 24 14:26:51.214400 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:26:51.214403    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/674cd4ff-c759-42ff-8b5b-f5e4eb2c45b7-service-ca\") pod \"console-5cb8dd7bdc-bpg26\" (UID: \"674cd4ff-c759-42ff-8b5b-f5e4eb2c45b7\") " pod="openshift-console/console-5cb8dd7bdc-bpg26"
Apr 24 14:26:51.214630 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:26:51.214437    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/674cd4ff-c759-42ff-8b5b-f5e4eb2c45b7-console-oauth-config\") pod \"console-5cb8dd7bdc-bpg26\" (UID: \"674cd4ff-c759-42ff-8b5b-f5e4eb2c45b7\") " pod="openshift-console/console-5cb8dd7bdc-bpg26"
Apr 24 14:26:51.214630 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:26:51.214518    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/674cd4ff-c759-42ff-8b5b-f5e4eb2c45b7-console-config\") pod \"console-5cb8dd7bdc-bpg26\" (UID: \"674cd4ff-c759-42ff-8b5b-f5e4eb2c45b7\") " pod="openshift-console/console-5cb8dd7bdc-bpg26"
Apr 24 14:26:51.214630 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:26:51.214560    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9xdl8\" (UniqueName: \"kubernetes.io/projected/674cd4ff-c759-42ff-8b5b-f5e4eb2c45b7-kube-api-access-9xdl8\") pod \"console-5cb8dd7bdc-bpg26\" (UID: \"674cd4ff-c759-42ff-8b5b-f5e4eb2c45b7\") " pod="openshift-console/console-5cb8dd7bdc-bpg26"
Apr 24 14:26:51.214630 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:26:51.214590    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/674cd4ff-c759-42ff-8b5b-f5e4eb2c45b7-console-serving-cert\") pod \"console-5cb8dd7bdc-bpg26\" (UID: \"674cd4ff-c759-42ff-8b5b-f5e4eb2c45b7\") " pod="openshift-console/console-5cb8dd7bdc-bpg26"
Apr 24 14:26:51.264229 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:26:51.264187    2574 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/cluster-monitoring-operator-75587bd455-q9jbv" event={"ID":"0be19048-1b01-4c64-bcca-94c9bff7768d","Type":"ContainerStarted","Data":"50080ad575d2b0b044d3df3650e8ad13bb57567ca2d083fa352840bcb64705d5"}
Apr 24 14:26:51.266027 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:26:51.265995    2574 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-qgxfn" event={"ID":"660bbad4-5502-418e-a02d-869defb74468","Type":"ContainerStarted","Data":"428a72fc62751b75d2db03491b0f96131e2037a608f017b7f3fa77430351cd85"}
Apr 24 14:26:51.285000 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:26:51.284952    2574 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-insights/insights-runtime-extractor-qgxfn" podStartSLOduration=1.949503494 podStartE2EDuration="4.284937174s" podCreationTimestamp="2026-04-24 14:26:47 +0000 UTC" firstStartedPulling="2026-04-24 14:26:48.007583139 +0000 UTC m=+170.739684766" lastFinishedPulling="2026-04-24 14:26:50.343016819 +0000 UTC m=+173.075118446" observedRunningTime="2026-04-24 14:26:51.282911147 +0000 UTC m=+174.015012820" watchObservedRunningTime="2026-04-24 14:26:51.284937174 +0000 UTC m=+174.017038822"
Apr 24 14:26:51.315933 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:26:51.315898    2574 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/674cd4ff-c759-42ff-8b5b-f5e4eb2c45b7-console-config\") pod \"console-5cb8dd7bdc-bpg26\" (UID: \"674cd4ff-c759-42ff-8b5b-f5e4eb2c45b7\") " pod="openshift-console/console-5cb8dd7bdc-bpg26"
Apr 24 14:26:51.316111 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:26:51.315979    2574 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-9xdl8\" (UniqueName: \"kubernetes.io/projected/674cd4ff-c759-42ff-8b5b-f5e4eb2c45b7-kube-api-access-9xdl8\") pod \"console-5cb8dd7bdc-bpg26\" (UID: \"674cd4ff-c759-42ff-8b5b-f5e4eb2c45b7\") " pod="openshift-console/console-5cb8dd7bdc-bpg26"
Apr 24 14:26:51.316111 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:26:51.316025    2574 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/674cd4ff-c759-42ff-8b5b-f5e4eb2c45b7-console-serving-cert\") pod \"console-5cb8dd7bdc-bpg26\" (UID: \"674cd4ff-c759-42ff-8b5b-f5e4eb2c45b7\") " pod="openshift-console/console-5cb8dd7bdc-bpg26"
Apr 24 14:26:51.316237 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:26:51.316135    2574 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/674cd4ff-c759-42ff-8b5b-f5e4eb2c45b7-oauth-serving-cert\") pod \"console-5cb8dd7bdc-bpg26\" (UID: \"674cd4ff-c759-42ff-8b5b-f5e4eb2c45b7\") " pod="openshift-console/console-5cb8dd7bdc-bpg26"
Apr 24 14:26:51.316237 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:26:51.316180    2574 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/674cd4ff-c759-42ff-8b5b-f5e4eb2c45b7-service-ca\") pod \"console-5cb8dd7bdc-bpg26\" (UID: \"674cd4ff-c759-42ff-8b5b-f5e4eb2c45b7\") " pod="openshift-console/console-5cb8dd7bdc-bpg26"
Apr 24 14:26:51.316420 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:26:51.316272    2574 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/674cd4ff-c759-42ff-8b5b-f5e4eb2c45b7-console-oauth-config\") pod \"console-5cb8dd7bdc-bpg26\" (UID: \"674cd4ff-c759-42ff-8b5b-f5e4eb2c45b7\") " pod="openshift-console/console-5cb8dd7bdc-bpg26"
Apr 24 14:26:51.316940 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:26:51.316912    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/674cd4ff-c759-42ff-8b5b-f5e4eb2c45b7-service-ca\") pod \"console-5cb8dd7bdc-bpg26\" (UID: \"674cd4ff-c759-42ff-8b5b-f5e4eb2c45b7\") " pod="openshift-console/console-5cb8dd7bdc-bpg26"
Apr 24 14:26:51.317041 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:26:51.316957    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/674cd4ff-c759-42ff-8b5b-f5e4eb2c45b7-oauth-serving-cert\") pod \"console-5cb8dd7bdc-bpg26\" (UID: \"674cd4ff-c759-42ff-8b5b-f5e4eb2c45b7\") " pod="openshift-console/console-5cb8dd7bdc-bpg26"
Apr 24 14:26:51.317096 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:26:51.317082    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/674cd4ff-c759-42ff-8b5b-f5e4eb2c45b7-console-config\") pod \"console-5cb8dd7bdc-bpg26\" (UID: \"674cd4ff-c759-42ff-8b5b-f5e4eb2c45b7\") " pod="openshift-console/console-5cb8dd7bdc-bpg26"
Apr 24 14:26:51.319329 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:26:51.319300    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/674cd4ff-c759-42ff-8b5b-f5e4eb2c45b7-console-serving-cert\") pod \"console-5cb8dd7bdc-bpg26\" (UID: \"674cd4ff-c759-42ff-8b5b-f5e4eb2c45b7\") " pod="openshift-console/console-5cb8dd7bdc-bpg26"
Apr 24 14:26:51.319662 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:26:51.319640    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/674cd4ff-c759-42ff-8b5b-f5e4eb2c45b7-console-oauth-config\") pod \"console-5cb8dd7bdc-bpg26\" (UID: \"674cd4ff-c759-42ff-8b5b-f5e4eb2c45b7\") " pod="openshift-console/console-5cb8dd7bdc-bpg26"
Apr 24 14:26:51.323710 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:26:51.323690    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-9xdl8\" (UniqueName: \"kubernetes.io/projected/674cd4ff-c759-42ff-8b5b-f5e4eb2c45b7-kube-api-access-9xdl8\") pod \"console-5cb8dd7bdc-bpg26\" (UID: \"674cd4ff-c759-42ff-8b5b-f5e4eb2c45b7\") " pod="openshift-console/console-5cb8dd7bdc-bpg26"
Apr 24 14:26:51.479376 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:26:51.479168    2574 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-5cb8dd7bdc-bpg26"
Apr 24 14:26:51.962139 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:26:51.962100    2574 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-5cb8dd7bdc-bpg26"]
Apr 24 14:26:51.965098 ip-10-0-136-132 kubenswrapper[2574]: W0424 14:26:51.965069    2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod674cd4ff_c759_42ff_8b5b_f5e4eb2c45b7.slice/crio-f86e2e36b9561df68cacac956d831aa8ac0e5c1a6e094eafb36b3f4a10f43398 WatchSource:0}: Error finding container f86e2e36b9561df68cacac956d831aa8ac0e5c1a6e094eafb36b3f4a10f43398: Status 404 returned error can't find the container with id f86e2e36b9561df68cacac956d831aa8ac0e5c1a6e094eafb36b3f4a10f43398
Apr 24 14:26:52.269918 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:26:52.269883    2574 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-5cb8dd7bdc-bpg26" event={"ID":"674cd4ff-c759-42ff-8b5b-f5e4eb2c45b7","Type":"ContainerStarted","Data":"f86e2e36b9561df68cacac956d831aa8ac0e5c1a6e094eafb36b3f4a10f43398"}
Apr 24 14:26:52.271161 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:26:52.271133    2574 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/cluster-monitoring-operator-75587bd455-q9jbv" event={"ID":"0be19048-1b01-4c64-bcca-94c9bff7768d","Type":"ContainerStarted","Data":"b3e7dbde3ecd4396a9b3e107c0a0ee6fc5067fa44d7235418fa664142a0425b9"}
Apr 24 14:26:52.287323 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:26:52.287259    2574 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-monitoring/cluster-monitoring-operator-75587bd455-q9jbv" podStartSLOduration=32.825301887 podStartE2EDuration="34.287243615s" podCreationTimestamp="2026-04-24 14:26:18 +0000 UTC" firstStartedPulling="2026-04-24 14:26:50.427565878 +0000 UTC m=+173.159667509" lastFinishedPulling="2026-04-24 14:26:51.889507597 +0000 UTC m=+174.621609237" observedRunningTime="2026-04-24 14:26:52.286833157 +0000 UTC m=+175.018934831" watchObservedRunningTime="2026-04-24 14:26:52.287243615 +0000 UTC m=+175.019345264"
Apr 24 14:26:55.280582 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:26:55.280545    2574 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-5cb8dd7bdc-bpg26" event={"ID":"674cd4ff-c759-42ff-8b5b-f5e4eb2c45b7","Type":"ContainerStarted","Data":"41cc9379018e1cc23612d4a01c3b92890a1d15091f1dfc4630f8051edf8f19ec"}
Apr 24 14:26:55.298979 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:26:55.298918    2574 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-console/console-5cb8dd7bdc-bpg26" podStartSLOduration=1.53789898 podStartE2EDuration="4.298902233s" podCreationTimestamp="2026-04-24 14:26:51 +0000 UTC" firstStartedPulling="2026-04-24 14:26:51.967630699 +0000 UTC m=+174.699732326" lastFinishedPulling="2026-04-24 14:26:54.728633948 +0000 UTC m=+177.460735579" observedRunningTime="2026-04-24 14:26:55.298654185 +0000 UTC m=+178.030755835" watchObservedRunningTime="2026-04-24 14:26:55.298902233 +0000 UTC m=+178.031003883"
Apr 24 14:27:00.268680 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:27:00.268648    2574 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-dns/dns-default-nvz9x"
Apr 24 14:27:01.479961 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:27:01.479917    2574 kubelet.go:2658] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-5cb8dd7bdc-bpg26"
Apr 24 14:27:01.480383 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:27:01.479993    2574 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-5cb8dd7bdc-bpg26"
Apr 24 14:27:01.484588 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:27:01.484567    2574 kubelet.go:2658] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-5cb8dd7bdc-bpg26"
Apr 24 14:27:02.302824 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:27:02.302793    2574 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-5cb8dd7bdc-bpg26"
Apr 24 14:27:06.789899 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:27:06.789866    2574 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["openshift-monitoring/openshift-state-metrics-9d44df66c-hkfgq"]
Apr 24 14:27:06.794346 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:27:06.794313    2574 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/openshift-state-metrics-9d44df66c-hkfgq"
Apr 24 14:27:06.796620 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:27:06.796600    2574 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"metrics-client-ca\""
Apr 24 14:27:06.796753 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:27:06.796600    2574 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"openshift-state-metrics-dockercfg-rvpnj\""
Apr 24 14:27:06.796900 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:27:06.796598    2574 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"openshift-state-metrics-tls\""
Apr 24 14:27:06.796900 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:27:06.796600    2574 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"openshift-state-metrics-kube-rbac-proxy-config\""
Apr 24 14:27:06.803788 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:27:06.803739    2574 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/openshift-state-metrics-9d44df66c-hkfgq"]
Apr 24 14:27:06.812671 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:27:06.812644    2574 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["openshift-monitoring/node-exporter-tprms"]
Apr 24 14:27:06.816301 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:27:06.816282    2574 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-tprms"
Apr 24 14:27:06.819509 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:27:06.819486    2574 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["openshift-monitoring/kube-state-metrics-69db897b98-spwlq"]
Apr 24 14:27:06.819653 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:27:06.819630    2574 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-kube-rbac-proxy-config\""
Apr 24 14:27:06.819735 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:27:06.819716    2574 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-dockercfg-lpfj8\""
Apr 24 14:27:06.819799 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:27:06.819538    2574 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-tls\""
Apr 24 14:27:06.819962 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:27:06.819938    2574 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"node-exporter-accelerators-collector-config\""
Apr 24 14:27:06.823452 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:27:06.823434    2574 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/kube-state-metrics-69db897b98-spwlq"
Apr 24 14:27:06.826677 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:27:06.826648    2574 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-custom-resource-state-configmap\""
Apr 24 14:27:06.826677 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:27:06.826669    2574 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-tls\""
Apr 24 14:27:06.826825 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:27:06.826698    2574 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-dockercfg-c25jz\""
Apr 24 14:27:06.826825 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:27:06.826652    2574 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-kube-rbac-proxy-config\""
Apr 24 14:27:06.839170 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:27:06.839144    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qzpht\" (UniqueName: \"kubernetes.io/projected/80e82582-a13a-4c7b-a6ea-3074aff3a5f5-kube-api-access-qzpht\") pod \"openshift-state-metrics-9d44df66c-hkfgq\" (UID: \"80e82582-a13a-4c7b-a6ea-3074aff3a5f5\") " pod="openshift-monitoring/openshift-state-metrics-9d44df66c-hkfgq"
Apr 24 14:27:06.839306 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:27:06.839177    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/80e82582-a13a-4c7b-a6ea-3074aff3a5f5-openshift-state-metrics-tls\") pod \"openshift-state-metrics-9d44df66c-hkfgq\" (UID: \"80e82582-a13a-4c7b-a6ea-3074aff3a5f5\") " pod="openshift-monitoring/openshift-state-metrics-9d44df66c-hkfgq"
Apr 24 14:27:06.839306 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:27:06.839296    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/80e82582-a13a-4c7b-a6ea-3074aff3a5f5-openshift-state-metrics-kube-rbac-proxy-config\") pod \"openshift-state-metrics-9d44df66c-hkfgq\" (UID: \"80e82582-a13a-4c7b-a6ea-3074aff3a5f5\") " pod="openshift-monitoring/openshift-state-metrics-9d44df66c-hkfgq"
Apr 24 14:27:06.839427 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:27:06.839322    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/80e82582-a13a-4c7b-a6ea-3074aff3a5f5-metrics-client-ca\") pod \"openshift-state-metrics-9d44df66c-hkfgq\" (UID: \"80e82582-a13a-4c7b-a6ea-3074aff3a5f5\") " pod="openshift-monitoring/openshift-state-metrics-9d44df66c-hkfgq"
Apr 24 14:27:06.844328 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:27:06.844296    2574 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/kube-state-metrics-69db897b98-spwlq"]
Apr 24 14:27:06.939766 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:27:06.939727    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/0694c1e6-ce19-4731-a344-f8558961d7d4-node-exporter-accelerators-collector-config\") pod \"node-exporter-tprms\" (UID: \"0694c1e6-ce19-4731-a344-f8558961d7d4\") " pod="openshift-monitoring/node-exporter-tprms"
Apr 24 14:27:06.939933 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:27:06.939773    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/e22659f8-0faa-4d68-99a6-d7d61e263c6a-metrics-client-ca\") pod \"kube-state-metrics-69db897b98-spwlq\" (UID: \"e22659f8-0faa-4d68-99a6-d7d61e263c6a\") " pod="openshift-monitoring/kube-state-metrics-69db897b98-spwlq"
Apr 24 14:27:06.939933 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:27:06.939806    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hp8tc\" (UniqueName: \"kubernetes.io/projected/0694c1e6-ce19-4731-a344-f8558961d7d4-kube-api-access-hp8tc\") pod \"node-exporter-tprms\" (UID: \"0694c1e6-ce19-4731-a344-f8558961d7d4\") " pod="openshift-monitoring/node-exporter-tprms"
Apr 24 14:27:06.939933 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:27:06.939838    2574 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-qzpht\" (UniqueName: \"kubernetes.io/projected/80e82582-a13a-4c7b-a6ea-3074aff3a5f5-kube-api-access-qzpht\") pod \"openshift-state-metrics-9d44df66c-hkfgq\" (UID: \"80e82582-a13a-4c7b-a6ea-3074aff3a5f5\") " pod="openshift-monitoring/openshift-state-metrics-9d44df66c-hkfgq"
Apr 24 14:27:06.939933 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:27:06.939864    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/0694c1e6-ce19-4731-a344-f8558961d7d4-node-exporter-tls\") pod \"node-exporter-tprms\" (UID: \"0694c1e6-ce19-4731-a344-f8558961d7d4\") " pod="openshift-monitoring/node-exporter-tprms"
Apr 24 14:27:06.939933 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:27:06.939888    2574 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"openshift-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/80e82582-a13a-4c7b-a6ea-3074aff3a5f5-openshift-state-metrics-tls\") pod \"openshift-state-metrics-9d44df66c-hkfgq\" (UID: \"80e82582-a13a-4c7b-a6ea-3074aff3a5f5\") " pod="openshift-monitoring/openshift-state-metrics-9d44df66c-hkfgq"
Apr 24 14:27:06.940206 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:27:06.939940    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/e22659f8-0faa-4d68-99a6-d7d61e263c6a-kube-state-metrics-kube-rbac-proxy-config\") pod \"kube-state-metrics-69db897b98-spwlq\" (UID: \"e22659f8-0faa-4d68-99a6-d7d61e263c6a\") " pod="openshift-monitoring/kube-state-metrics-69db897b98-spwlq"
Apr 24 14:27:06.940206 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:27:06.939973    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/0694c1e6-ce19-4731-a344-f8558961d7d4-node-exporter-textfile\") pod \"node-exporter-tprms\" (UID: \"0694c1e6-ce19-4731-a344-f8558961d7d4\") " pod="openshift-monitoring/node-exporter-tprms"
Apr 24 14:27:06.940206 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:27:06.940022    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/0694c1e6-ce19-4731-a344-f8558961d7d4-root\") pod \"node-exporter-tprms\" (UID: \"0694c1e6-ce19-4731-a344-f8558961d7d4\") " pod="openshift-monitoring/node-exporter-tprms"
Apr 24 14:27:06.940206 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:27:06.940051    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/0694c1e6-ce19-4731-a344-f8558961d7d4-sys\") pod \"node-exporter-tprms\" (UID: \"0694c1e6-ce19-4731-a344-f8558961d7d4\") " pod="openshift-monitoring/node-exporter-tprms"
Apr 24 14:27:06.940206 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:27:06.940098    2574 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"openshift-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/80e82582-a13a-4c7b-a6ea-3074aff3a5f5-openshift-state-metrics-kube-rbac-proxy-config\") pod \"openshift-state-metrics-9d44df66c-hkfgq\" (UID: \"80e82582-a13a-4c7b-a6ea-3074aff3a5f5\") " pod="openshift-monitoring/openshift-state-metrics-9d44df66c-hkfgq"
Apr 24 14:27:06.940206 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:27:06.940128    2574 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/80e82582-a13a-4c7b-a6ea-3074aff3a5f5-metrics-client-ca\") pod \"openshift-state-metrics-9d44df66c-hkfgq\" (UID: \"80e82582-a13a-4c7b-a6ea-3074aff3a5f5\") " pod="openshift-monitoring/openshift-state-metrics-9d44df66c-hkfgq"
Apr 24 14:27:06.940206 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:27:06.940159    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/0694c1e6-ce19-4731-a344-f8558961d7d4-node-exporter-wtmp\") pod \"node-exporter-tprms\" (UID: \"0694c1e6-ce19-4731-a344-f8558961d7d4\") " pod="openshift-monitoring/node-exporter-tprms"
Apr 24 14:27:06.940206 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:27:06.940188    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/0694c1e6-ce19-4731-a344-f8558961d7d4-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-tprms\" (UID: \"0694c1e6-ce19-4731-a344-f8558961d7d4\") " pod="openshift-monitoring/node-exporter-tprms"
Apr 24 14:27:06.940582 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:27:06.940213    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"volume-directive-shadow\" (UniqueName: \"kubernetes.io/empty-dir/e22659f8-0faa-4d68-99a6-d7d61e263c6a-volume-directive-shadow\") pod \"kube-state-metrics-69db897b98-spwlq\" (UID: \"e22659f8-0faa-4d68-99a6-d7d61e263c6a\") " pod="openshift-monitoring/kube-state-metrics-69db897b98-spwlq"
Apr 24 14:27:06.940582 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:27:06.940316    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/e22659f8-0faa-4d68-99a6-d7d61e263c6a-kube-state-metrics-tls\") pod \"kube-state-metrics-69db897b98-spwlq\" (UID: \"e22659f8-0faa-4d68-99a6-d7d61e263c6a\") " pod="openshift-monitoring/kube-state-metrics-69db897b98-spwlq"
Apr 24 14:27:06.940582 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:27:06.940398    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/0694c1e6-ce19-4731-a344-f8558961d7d4-metrics-client-ca\") pod \"node-exporter-tprms\" (UID: \"0694c1e6-ce19-4731-a344-f8558961d7d4\") " pod="openshift-monitoring/node-exporter-tprms"
Apr 24 14:27:06.940582 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:27:06.940440    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-state-metrics-custom-resource-state-configmap\" (UniqueName: \"kubernetes.io/configmap/e22659f8-0faa-4d68-99a6-d7d61e263c6a-kube-state-metrics-custom-resource-state-configmap\") pod \"kube-state-metrics-69db897b98-spwlq\" (UID: \"e22659f8-0faa-4d68-99a6-d7d61e263c6a\") " pod="openshift-monitoring/kube-state-metrics-69db897b98-spwlq"
Apr 24 14:27:06.940582 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:27:06.940466    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-r26wc\" (UniqueName: \"kubernetes.io/projected/e22659f8-0faa-4d68-99a6-d7d61e263c6a-kube-api-access-r26wc\") pod \"kube-state-metrics-69db897b98-spwlq\" (UID: \"e22659f8-0faa-4d68-99a6-d7d61e263c6a\") " pod="openshift-monitoring/kube-state-metrics-69db897b98-spwlq"
Apr 24 14:27:06.940830 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:27:06.940777    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/80e82582-a13a-4c7b-a6ea-3074aff3a5f5-metrics-client-ca\") pod \"openshift-state-metrics-9d44df66c-hkfgq\" (UID: \"80e82582-a13a-4c7b-a6ea-3074aff3a5f5\") " pod="openshift-monitoring/openshift-state-metrics-9d44df66c-hkfgq"
Apr 24 14:27:06.942445 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:27:06.942418    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/80e82582-a13a-4c7b-a6ea-3074aff3a5f5-openshift-state-metrics-tls\") pod \"openshift-state-metrics-9d44df66c-hkfgq\" (UID: \"80e82582-a13a-4c7b-a6ea-3074aff3a5f5\") " pod="openshift-monitoring/openshift-state-metrics-9d44df66c-hkfgq"
Apr 24 14:27:06.942554 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:27:06.942478    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/80e82582-a13a-4c7b-a6ea-3074aff3a5f5-openshift-state-metrics-kube-rbac-proxy-config\") pod \"openshift-state-metrics-9d44df66c-hkfgq\" (UID: \"80e82582-a13a-4c7b-a6ea-3074aff3a5f5\") " pod="openshift-monitoring/openshift-state-metrics-9d44df66c-hkfgq"
Apr 24 14:27:06.952052 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:27:06.952022    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-qzpht\" (UniqueName: \"kubernetes.io/projected/80e82582-a13a-4c7b-a6ea-3074aff3a5f5-kube-api-access-qzpht\") pod \"openshift-state-metrics-9d44df66c-hkfgq\" (UID: \"80e82582-a13a-4c7b-a6ea-3074aff3a5f5\") " pod="openshift-monitoring/openshift-state-metrics-9d44df66c-hkfgq"
Apr 24 14:27:07.041534 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:27:07.041432    2574 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/e22659f8-0faa-4d68-99a6-d7d61e263c6a-kube-state-metrics-kube-rbac-proxy-config\") pod \"kube-state-metrics-69db897b98-spwlq\" (UID: \"e22659f8-0faa-4d68-99a6-d7d61e263c6a\") " pod="openshift-monitoring/kube-state-metrics-69db897b98-spwlq"
Apr 24 14:27:07.041534 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:27:07.041477    2574 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/0694c1e6-ce19-4731-a344-f8558961d7d4-node-exporter-textfile\") pod \"node-exporter-tprms\" (UID: \"0694c1e6-ce19-4731-a344-f8558961d7d4\") " pod="openshift-monitoring/node-exporter-tprms"
Apr 24 14:27:07.041534 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:27:07.041506    2574 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/0694c1e6-ce19-4731-a344-f8558961d7d4-root\") pod \"node-exporter-tprms\" (UID: \"0694c1e6-ce19-4731-a344-f8558961d7d4\") " pod="openshift-monitoring/node-exporter-tprms"
Apr 24 14:27:07.041820 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:27:07.041561    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"root\" (UniqueName: \"kubernetes.io/host-path/0694c1e6-ce19-4731-a344-f8558961d7d4-root\") pod \"node-exporter-tprms\" (UID: \"0694c1e6-ce19-4731-a344-f8558961d7d4\") " pod="openshift-monitoring/node-exporter-tprms"
Apr 24 14:27:07.041820 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:27:07.041623    2574 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/0694c1e6-ce19-4731-a344-f8558961d7d4-sys\") pod \"node-exporter-tprms\" (UID: \"0694c1e6-ce19-4731-a344-f8558961d7d4\") " pod="openshift-monitoring/node-exporter-tprms"
Apr 24 14:27:07.041820 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:27:07.041679    2574 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/0694c1e6-ce19-4731-a344-f8558961d7d4-node-exporter-wtmp\") pod \"node-exporter-tprms\" (UID: \"0694c1e6-ce19-4731-a344-f8558961d7d4\") " pod="openshift-monitoring/node-exporter-tprms"
Apr 24 14:27:07.041820 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:27:07.041717    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/0694c1e6-ce19-4731-a344-f8558961d7d4-sys\") pod \"node-exporter-tprms\" (UID: \"0694c1e6-ce19-4731-a344-f8558961d7d4\") " pod="openshift-monitoring/node-exporter-tprms"
Apr 24 14:27:07.041820 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:27:07.041748    2574 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/0694c1e6-ce19-4731-a344-f8558961d7d4-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-tprms\" (UID: \"0694c1e6-ce19-4731-a344-f8558961d7d4\") " pod="openshift-monitoring/node-exporter-tprms"
Apr 24 14:27:07.041820 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:27:07.041773    2574 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"volume-directive-shadow\" (UniqueName: \"kubernetes.io/empty-dir/e22659f8-0faa-4d68-99a6-d7d61e263c6a-volume-directive-shadow\") pod \"kube-state-metrics-69db897b98-spwlq\" (UID: \"e22659f8-0faa-4d68-99a6-d7d61e263c6a\") " pod="openshift-monitoring/kube-state-metrics-69db897b98-spwlq"
Apr 24 14:27:07.041820 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:27:07.041783    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/0694c1e6-ce19-4731-a344-f8558961d7d4-node-exporter-wtmp\") pod \"node-exporter-tprms\" (UID: \"0694c1e6-ce19-4731-a344-f8558961d7d4\") " pod="openshift-monitoring/node-exporter-tprms"
Apr 24 14:27:07.042121 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:27:07.041822    2574 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/e22659f8-0faa-4d68-99a6-d7d61e263c6a-kube-state-metrics-tls\") pod \"kube-state-metrics-69db897b98-spwlq\" (UID: \"e22659f8-0faa-4d68-99a6-d7d61e263c6a\") " pod="openshift-monitoring/kube-state-metrics-69db897b98-spwlq"
Apr 24 14:27:07.042121 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:27:07.041856    2574 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/0694c1e6-ce19-4731-a344-f8558961d7d4-metrics-client-ca\") pod \"node-exporter-tprms\" (UID: \"0694c1e6-ce19-4731-a344-f8558961d7d4\") " pod="openshift-monitoring/node-exporter-tprms"
Apr 24 14:27:07.042121 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:27:07.041888    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/0694c1e6-ce19-4731-a344-f8558961d7d4-node-exporter-textfile\") pod \"node-exporter-tprms\" (UID: \"0694c1e6-ce19-4731-a344-f8558961d7d4\") " pod="openshift-monitoring/node-exporter-tprms"
Apr 24 14:27:07.042121 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:27:07.041891    2574 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-state-metrics-custom-resource-state-configmap\" (UniqueName: \"kubernetes.io/configmap/e22659f8-0faa-4d68-99a6-d7d61e263c6a-kube-state-metrics-custom-resource-state-configmap\") pod \"kube-state-metrics-69db897b98-spwlq\" (UID: \"e22659f8-0faa-4d68-99a6-d7d61e263c6a\") " pod="openshift-monitoring/kube-state-metrics-69db897b98-spwlq"
Apr 24 14:27:07.042121 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:27:07.041958    2574 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-r26wc\" (UniqueName: \"kubernetes.io/projected/e22659f8-0faa-4d68-99a6-d7d61e263c6a-kube-api-access-r26wc\") pod \"kube-state-metrics-69db897b98-spwlq\" (UID: \"e22659f8-0faa-4d68-99a6-d7d61e263c6a\") " pod="openshift-monitoring/kube-state-metrics-69db897b98-spwlq"
Apr 24 14:27:07.042121 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:27:07.041991    2574 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/0694c1e6-ce19-4731-a344-f8558961d7d4-node-exporter-accelerators-collector-config\") pod \"node-exporter-tprms\" (UID: \"0694c1e6-ce19-4731-a344-f8558961d7d4\") " pod="openshift-monitoring/node-exporter-tprms"
Apr 24 14:27:07.042121 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:27:07.042018    2574 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/e22659f8-0faa-4d68-99a6-d7d61e263c6a-metrics-client-ca\") pod \"kube-state-metrics-69db897b98-spwlq\" (UID: \"e22659f8-0faa-4d68-99a6-d7d61e263c6a\") " pod="openshift-monitoring/kube-state-metrics-69db897b98-spwlq"
Apr 24 14:27:07.042121 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:27:07.042050    2574 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-hp8tc\" (UniqueName: \"kubernetes.io/projected/0694c1e6-ce19-4731-a344-f8558961d7d4-kube-api-access-hp8tc\") pod \"node-exporter-tprms\" (UID: \"0694c1e6-ce19-4731-a344-f8558961d7d4\") " pod="openshift-monitoring/node-exporter-tprms"
Apr 24 14:27:07.042121 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:27:07.042080    2574 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/0694c1e6-ce19-4731-a344-f8558961d7d4-node-exporter-tls\") pod \"node-exporter-tprms\" (UID: \"0694c1e6-ce19-4731-a344-f8558961d7d4\") " pod="openshift-monitoring/node-exporter-tprms"
Apr 24 14:27:07.042121 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:27:07.042109    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"volume-directive-shadow\" (UniqueName: \"kubernetes.io/empty-dir/e22659f8-0faa-4d68-99a6-d7d61e263c6a-volume-directive-shadow\") pod \"kube-state-metrics-69db897b98-spwlq\" (UID: \"e22659f8-0faa-4d68-99a6-d7d61e263c6a\") " pod="openshift-monitoring/kube-state-metrics-69db897b98-spwlq"
Apr 24 14:27:07.042674 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:27:07.042647    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/0694c1e6-ce19-4731-a344-f8558961d7d4-node-exporter-accelerators-collector-config\") pod \"node-exporter-tprms\" (UID: \"0694c1e6-ce19-4731-a344-f8558961d7d4\") " pod="openshift-monitoring/node-exporter-tprms"
Apr 24 14:27:07.042674 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:27:07.042665    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-state-metrics-custom-resource-state-configmap\" (UniqueName: \"kubernetes.io/configmap/e22659f8-0faa-4d68-99a6-d7d61e263c6a-kube-state-metrics-custom-resource-state-configmap\") pod \"kube-state-metrics-69db897b98-spwlq\" (UID: \"e22659f8-0faa-4d68-99a6-d7d61e263c6a\") " pod="openshift-monitoring/kube-state-metrics-69db897b98-spwlq"
Apr 24 14:27:07.042820 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:27:07.042794    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/e22659f8-0faa-4d68-99a6-d7d61e263c6a-metrics-client-ca\") pod \"kube-state-metrics-69db897b98-spwlq\" (UID: \"e22659f8-0faa-4d68-99a6-d7d61e263c6a\") " pod="openshift-monitoring/kube-state-metrics-69db897b98-spwlq"
Apr 24 14:27:07.043258 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:27:07.043238    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/0694c1e6-ce19-4731-a344-f8558961d7d4-metrics-client-ca\") pod \"node-exporter-tprms\" (UID: \"0694c1e6-ce19-4731-a344-f8558961d7d4\") " pod="openshift-monitoring/node-exporter-tprms"
Apr 24 14:27:07.044455 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:27:07.044429    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/0694c1e6-ce19-4731-a344-f8558961d7d4-node-exporter-tls\") pod \"node-exporter-tprms\" (UID: \"0694c1e6-ce19-4731-a344-f8558961d7d4\") " pod="openshift-monitoring/node-exporter-tprms"
Apr 24 14:27:07.044531 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:27:07.044455    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/0694c1e6-ce19-4731-a344-f8558961d7d4-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-tprms\" (UID: \"0694c1e6-ce19-4731-a344-f8558961d7d4\") " pod="openshift-monitoring/node-exporter-tprms"
Apr 24 14:27:07.044604 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:27:07.044587    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/e22659f8-0faa-4d68-99a6-d7d61e263c6a-kube-state-metrics-tls\") pod \"kube-state-metrics-69db897b98-spwlq\" (UID: \"e22659f8-0faa-4d68-99a6-d7d61e263c6a\") " pod="openshift-monitoring/kube-state-metrics-69db897b98-spwlq"
Apr 24 14:27:07.044653 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:27:07.044634    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/e22659f8-0faa-4d68-99a6-d7d61e263c6a-kube-state-metrics-kube-rbac-proxy-config\") pod \"kube-state-metrics-69db897b98-spwlq\" (UID: \"e22659f8-0faa-4d68-99a6-d7d61e263c6a\") " pod="openshift-monitoring/kube-state-metrics-69db897b98-spwlq"
Apr 24 14:27:07.054531 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:27:07.054511    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-hp8tc\" (UniqueName: \"kubernetes.io/projected/0694c1e6-ce19-4731-a344-f8558961d7d4-kube-api-access-hp8tc\") pod \"node-exporter-tprms\" (UID: \"0694c1e6-ce19-4731-a344-f8558961d7d4\") " pod="openshift-monitoring/node-exporter-tprms"
Apr 24 14:27:07.054726 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:27:07.054710    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-r26wc\" (UniqueName: \"kubernetes.io/projected/e22659f8-0faa-4d68-99a6-d7d61e263c6a-kube-api-access-r26wc\") pod \"kube-state-metrics-69db897b98-spwlq\" (UID: \"e22659f8-0faa-4d68-99a6-d7d61e263c6a\") " pod="openshift-monitoring/kube-state-metrics-69db897b98-spwlq"
Apr 24 14:27:07.104832 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:27:07.104800    2574 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/openshift-state-metrics-9d44df66c-hkfgq"
Apr 24 14:27:07.128547 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:27:07.128444    2574 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-tprms"
Apr 24 14:27:07.135677 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:27:07.135543    2574 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/kube-state-metrics-69db897b98-spwlq"
Apr 24 14:27:07.146185 ip-10-0-136-132 kubenswrapper[2574]: W0424 14:27:07.146124    2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0694c1e6_ce19_4731_a344_f8558961d7d4.slice/crio-223d310d7fc2073238df5a782be7ac1c6bfb07ad077dcea50e2a2b7d6d98512d WatchSource:0}: Error finding container 223d310d7fc2073238df5a782be7ac1c6bfb07ad077dcea50e2a2b7d6d98512d: Status 404 returned error can't find the container with id 223d310d7fc2073238df5a782be7ac1c6bfb07ad077dcea50e2a2b7d6d98512d
Apr 24 14:27:07.245000 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:27:07.244857    2574 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/openshift-state-metrics-9d44df66c-hkfgq"]
Apr 24 14:27:07.248613 ip-10-0-136-132 kubenswrapper[2574]: W0424 14:27:07.248562    2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod80e82582_a13a_4c7b_a6ea_3074aff3a5f5.slice/crio-800434588f87d06ca8783c4e083ae5caa738767ff773da2709c4839f1a0850ea WatchSource:0}: Error finding container 800434588f87d06ca8783c4e083ae5caa738767ff773da2709c4839f1a0850ea: Status 404 returned error can't find the container with id 800434588f87d06ca8783c4e083ae5caa738767ff773da2709c4839f1a0850ea
Apr 24 14:27:07.278919 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:27:07.278815    2574 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/kube-state-metrics-69db897b98-spwlq"]
Apr 24 14:27:07.283392 ip-10-0-136-132 kubenswrapper[2574]: W0424 14:27:07.283364    2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode22659f8_0faa_4d68_99a6_d7d61e263c6a.slice/crio-deeb14cf773087f8e40b11dbd8e22aacb006e125ab492b0bdd6ee4c62748af0f WatchSource:0}: Error finding container deeb14cf773087f8e40b11dbd8e22aacb006e125ab492b0bdd6ee4c62748af0f: Status 404 returned error can't find the container with id deeb14cf773087f8e40b11dbd8e22aacb006e125ab492b0bdd6ee4c62748af0f
Apr 24 14:27:07.314044 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:27:07.314006    2574 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/openshift-state-metrics-9d44df66c-hkfgq" event={"ID":"80e82582-a13a-4c7b-a6ea-3074aff3a5f5","Type":"ContainerStarted","Data":"800434588f87d06ca8783c4e083ae5caa738767ff773da2709c4839f1a0850ea"}
Apr 24 14:27:07.315378 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:27:07.315123    2574 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-tprms" event={"ID":"0694c1e6-ce19-4731-a344-f8558961d7d4","Type":"ContainerStarted","Data":"223d310d7fc2073238df5a782be7ac1c6bfb07ad077dcea50e2a2b7d6d98512d"}
Apr 24 14:27:07.317539 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:27:07.317516    2574 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/kube-state-metrics-69db897b98-spwlq" event={"ID":"e22659f8-0faa-4d68-99a6-d7d61e263c6a","Type":"ContainerStarted","Data":"deeb14cf773087f8e40b11dbd8e22aacb006e125ab492b0bdd6ee4c62748af0f"}
Apr 24 14:27:07.902376 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:27:07.901100    2574 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["openshift-monitoring/alertmanager-main-0"]
Apr 24 14:27:07.907151 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:27:07.907129    2574 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0"
Apr 24 14:27:07.915857 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:27:07.911912    2574 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy-metric\""
Apr 24 14:27:07.915857 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:27:07.912310    2574 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"alertmanager-trusted-ca-bundle\""
Apr 24 14:27:07.915857 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:27:07.912532    2574 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-cluster-tls-config\""
Apr 24 14:27:07.915857 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:27:07.912719    2574 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-dockercfg-s522n\""
Apr 24 14:27:07.915857 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:27:07.913020    2574 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-tls-assets-0\""
Apr 24 14:27:07.915857 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:27:07.915714    2574 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-generated\""
Apr 24 14:27:07.915857 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:27:07.915758    2574 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy\""
Apr 24 14:27:07.917089 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:27:07.916467    2574 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-tls\""
Apr 24 14:27:07.917089 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:27:07.916699    2574 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-web-config\""
Apr 24 14:27:07.917089 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:27:07.916900    2574 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy-web\""
Apr 24 14:27:07.925653 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:27:07.925259    2574 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/alertmanager-main-0"]
Apr 24 14:27:07.950971 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:27:07.950363    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/51022bd6-a15d-4d4a-aa38-101a7f8e558f-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"51022bd6-a15d-4d4a-aa38-101a7f8e558f\") " pod="openshift-monitoring/alertmanager-main-0"
Apr 24 14:27:07.950971 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:27:07.950416    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/51022bd6-a15d-4d4a-aa38-101a7f8e558f-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"51022bd6-a15d-4d4a-aa38-101a7f8e558f\") " pod="openshift-monitoring/alertmanager-main-0"
Apr 24 14:27:07.950971 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:27:07.950485    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/51022bd6-a15d-4d4a-aa38-101a7f8e558f-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"51022bd6-a15d-4d4a-aa38-101a7f8e558f\") " pod="openshift-monitoring/alertmanager-main-0"
Apr 24 14:27:07.950971 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:27:07.950520    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/51022bd6-a15d-4d4a-aa38-101a7f8e558f-config-volume\") pod \"alertmanager-main-0\" (UID: \"51022bd6-a15d-4d4a-aa38-101a7f8e558f\") " pod="openshift-monitoring/alertmanager-main-0"
Apr 24 14:27:07.950971 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:27:07.950544    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/51022bd6-a15d-4d4a-aa38-101a7f8e558f-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"51022bd6-a15d-4d4a-aa38-101a7f8e558f\") " pod="openshift-monitoring/alertmanager-main-0"
Apr 24 14:27:07.950971 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:27:07.950572    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/51022bd6-a15d-4d4a-aa38-101a7f8e558f-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"51022bd6-a15d-4d4a-aa38-101a7f8e558f\") " pod="openshift-monitoring/alertmanager-main-0"
Apr 24 14:27:07.950971 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:27:07.950597    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/51022bd6-a15d-4d4a-aa38-101a7f8e558f-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"51022bd6-a15d-4d4a-aa38-101a7f8e558f\") " pod="openshift-monitoring/alertmanager-main-0"
Apr 24 14:27:07.950971 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:27:07.950622    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/51022bd6-a15d-4d4a-aa38-101a7f8e558f-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"51022bd6-a15d-4d4a-aa38-101a7f8e558f\") " pod="openshift-monitoring/alertmanager-main-0"
Apr 24 14:27:07.950971 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:27:07.950650    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-j4ghb\" (UniqueName: \"kubernetes.io/projected/51022bd6-a15d-4d4a-aa38-101a7f8e558f-kube-api-access-j4ghb\") pod \"alertmanager-main-0\" (UID: \"51022bd6-a15d-4d4a-aa38-101a7f8e558f\") " pod="openshift-monitoring/alertmanager-main-0"
Apr 24 14:27:07.950971 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:27:07.950686    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/51022bd6-a15d-4d4a-aa38-101a7f8e558f-config-out\") pod \"alertmanager-main-0\" (UID: \"51022bd6-a15d-4d4a-aa38-101a7f8e558f\") " pod="openshift-monitoring/alertmanager-main-0"
Apr 24 14:27:07.950971 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:27:07.950723    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/51022bd6-a15d-4d4a-aa38-101a7f8e558f-tls-assets\") pod \"alertmanager-main-0\" (UID: \"51022bd6-a15d-4d4a-aa38-101a7f8e558f\") " pod="openshift-monitoring/alertmanager-main-0"
Apr 24 14:27:07.950971 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:27:07.950759    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/51022bd6-a15d-4d4a-aa38-101a7f8e558f-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"51022bd6-a15d-4d4a-aa38-101a7f8e558f\") " pod="openshift-monitoring/alertmanager-main-0"
Apr 24 14:27:07.950971 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:27:07.950791    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/51022bd6-a15d-4d4a-aa38-101a7f8e558f-web-config\") pod \"alertmanager-main-0\" (UID: \"51022bd6-a15d-4d4a-aa38-101a7f8e558f\") " pod="openshift-monitoring/alertmanager-main-0"
Apr 24 14:27:08.052120 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:27:08.052082    2574 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/51022bd6-a15d-4d4a-aa38-101a7f8e558f-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"51022bd6-a15d-4d4a-aa38-101a7f8e558f\") " pod="openshift-monitoring/alertmanager-main-0"
Apr 24 14:27:08.052303 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:27:08.052145    2574 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/51022bd6-a15d-4d4a-aa38-101a7f8e558f-config-volume\") pod \"alertmanager-main-0\" (UID: \"51022bd6-a15d-4d4a-aa38-101a7f8e558f\") " pod="openshift-monitoring/alertmanager-main-0"
Apr 24 14:27:08.052303 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:27:08.052175    2574 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/51022bd6-a15d-4d4a-aa38-101a7f8e558f-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"51022bd6-a15d-4d4a-aa38-101a7f8e558f\") " pod="openshift-monitoring/alertmanager-main-0"
Apr 24 14:27:08.052303 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:27:08.052206    2574 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/51022bd6-a15d-4d4a-aa38-101a7f8e558f-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"51022bd6-a15d-4d4a-aa38-101a7f8e558f\") " pod="openshift-monitoring/alertmanager-main-0"
Apr 24 14:27:08.052303 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:27:08.052234    2574 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/51022bd6-a15d-4d4a-aa38-101a7f8e558f-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"51022bd6-a15d-4d4a-aa38-101a7f8e558f\") " pod="openshift-monitoring/alertmanager-main-0"
Apr 24 14:27:08.052303 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:27:08.052260    2574 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/51022bd6-a15d-4d4a-aa38-101a7f8e558f-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"51022bd6-a15d-4d4a-aa38-101a7f8e558f\") " pod="openshift-monitoring/alertmanager-main-0"
Apr 24 14:27:08.052303 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:27:08.052297    2574 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-j4ghb\" (UniqueName: \"kubernetes.io/projected/51022bd6-a15d-4d4a-aa38-101a7f8e558f-kube-api-access-j4ghb\") pod \"alertmanager-main-0\" (UID: \"51022bd6-a15d-4d4a-aa38-101a7f8e558f\") " pod="openshift-monitoring/alertmanager-main-0"
Apr 24 14:27:08.052619 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:27:08.052358    2574 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/51022bd6-a15d-4d4a-aa38-101a7f8e558f-config-out\") pod \"alertmanager-main-0\" (UID: \"51022bd6-a15d-4d4a-aa38-101a7f8e558f\") " pod="openshift-monitoring/alertmanager-main-0"
Apr 24 14:27:08.052619 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:27:08.052402    2574 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/51022bd6-a15d-4d4a-aa38-101a7f8e558f-tls-assets\") pod \"alertmanager-main-0\" (UID: \"51022bd6-a15d-4d4a-aa38-101a7f8e558f\") " pod="openshift-monitoring/alertmanager-main-0"
Apr 24 14:27:08.052619 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:27:08.052443    2574 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/51022bd6-a15d-4d4a-aa38-101a7f8e558f-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"51022bd6-a15d-4d4a-aa38-101a7f8e558f\") " pod="openshift-monitoring/alertmanager-main-0"
Apr 24 14:27:08.052619 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:27:08.052487    2574 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/51022bd6-a15d-4d4a-aa38-101a7f8e558f-web-config\") pod \"alertmanager-main-0\" (UID: \"51022bd6-a15d-4d4a-aa38-101a7f8e558f\") " pod="openshift-monitoring/alertmanager-main-0"
Apr 24 14:27:08.052619 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:27:08.052522    2574 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/51022bd6-a15d-4d4a-aa38-101a7f8e558f-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"51022bd6-a15d-4d4a-aa38-101a7f8e558f\") " pod="openshift-monitoring/alertmanager-main-0"
Apr 24 14:27:08.052619 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:27:08.052551    2574 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/51022bd6-a15d-4d4a-aa38-101a7f8e558f-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"51022bd6-a15d-4d4a-aa38-101a7f8e558f\") " pod="openshift-monitoring/alertmanager-main-0"
Apr 24 14:27:08.052899 ip-10-0-136-132 kubenswrapper[2574]: E0424 14:27:08.052876    2574 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/51022bd6-a15d-4d4a-aa38-101a7f8e558f-alertmanager-trusted-ca-bundle podName:51022bd6-a15d-4d4a-aa38-101a7f8e558f nodeName:}" failed. No retries permitted until 2026-04-24 14:27:08.552853827 +0000 UTC m=+191.284955460 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "alertmanager-trusted-ca-bundle" (UniqueName: "kubernetes.io/configmap/51022bd6-a15d-4d4a-aa38-101a7f8e558f-alertmanager-trusted-ca-bundle") pod "alertmanager-main-0" (UID: "51022bd6-a15d-4d4a-aa38-101a7f8e558f") : configmap references non-existent config key: ca-bundle.crt
Apr 24 14:27:08.053837 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:27:08.053811    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/51022bd6-a15d-4d4a-aa38-101a7f8e558f-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"51022bd6-a15d-4d4a-aa38-101a7f8e558f\") " pod="openshift-monitoring/alertmanager-main-0"
Apr 24 14:27:08.054124 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:27:08.054102    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/51022bd6-a15d-4d4a-aa38-101a7f8e558f-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"51022bd6-a15d-4d4a-aa38-101a7f8e558f\") " pod="openshift-monitoring/alertmanager-main-0"
Apr 24 14:27:08.057693 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:27:08.057669    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/51022bd6-a15d-4d4a-aa38-101a7f8e558f-config-out\") pod \"alertmanager-main-0\" (UID: \"51022bd6-a15d-4d4a-aa38-101a7f8e558f\") " pod="openshift-monitoring/alertmanager-main-0"
Apr 24 14:27:08.062854 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:27:08.062810    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/51022bd6-a15d-4d4a-aa38-101a7f8e558f-config-volume\") pod \"alertmanager-main-0\" (UID: \"51022bd6-a15d-4d4a-aa38-101a7f8e558f\") " pod="openshift-monitoring/alertmanager-main-0"
Apr 24 14:27:08.063592 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:27:08.063469    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/51022bd6-a15d-4d4a-aa38-101a7f8e558f-web-config\") pod \"alertmanager-main-0\" (UID: \"51022bd6-a15d-4d4a-aa38-101a7f8e558f\") " pod="openshift-monitoring/alertmanager-main-0"
Apr 24 14:27:08.063918 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:27:08.063864    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/51022bd6-a15d-4d4a-aa38-101a7f8e558f-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"51022bd6-a15d-4d4a-aa38-101a7f8e558f\") " pod="openshift-monitoring/alertmanager-main-0"
Apr 24 14:27:08.063918 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:27:08.063867    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/51022bd6-a15d-4d4a-aa38-101a7f8e558f-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"51022bd6-a15d-4d4a-aa38-101a7f8e558f\") " pod="openshift-monitoring/alertmanager-main-0"
Apr 24 14:27:08.064054 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:27:08.064000    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/51022bd6-a15d-4d4a-aa38-101a7f8e558f-tls-assets\") pod \"alertmanager-main-0\" (UID: \"51022bd6-a15d-4d4a-aa38-101a7f8e558f\") " pod="openshift-monitoring/alertmanager-main-0"
Apr 24 14:27:08.064464 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:27:08.064310    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/51022bd6-a15d-4d4a-aa38-101a7f8e558f-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"51022bd6-a15d-4d4a-aa38-101a7f8e558f\") " pod="openshift-monitoring/alertmanager-main-0"
Apr 24 14:27:08.064941 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:27:08.064899    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/51022bd6-a15d-4d4a-aa38-101a7f8e558f-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"51022bd6-a15d-4d4a-aa38-101a7f8e558f\") " pod="openshift-monitoring/alertmanager-main-0"
Apr 24 14:27:08.065300 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:27:08.065275    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-j4ghb\" (UniqueName: \"kubernetes.io/projected/51022bd6-a15d-4d4a-aa38-101a7f8e558f-kube-api-access-j4ghb\") pod \"alertmanager-main-0\" (UID: \"51022bd6-a15d-4d4a-aa38-101a7f8e558f\") " pod="openshift-monitoring/alertmanager-main-0"
Apr 24 14:27:08.065555 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:27:08.065532    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/51022bd6-a15d-4d4a-aa38-101a7f8e558f-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"51022bd6-a15d-4d4a-aa38-101a7f8e558f\") " pod="openshift-monitoring/alertmanager-main-0"
Apr 24 14:27:08.322708 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:27:08.322659    2574 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-tprms" event={"ID":"0694c1e6-ce19-4731-a344-f8558961d7d4","Type":"ContainerStarted","Data":"42fe8d2ee36b0da64bb79803e01ffbdcf8a0802f2439787f18e883791e515def"}
Apr 24 14:27:08.324637 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:27:08.324605    2574 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/openshift-state-metrics-9d44df66c-hkfgq" event={"ID":"80e82582-a13a-4c7b-a6ea-3074aff3a5f5","Type":"ContainerStarted","Data":"fc7ff8de1747797d9b89c272960614b0176bef5929132d1f12a8795182196dd1"}
Apr 24 14:27:08.324757 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:27:08.324641    2574 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/openshift-state-metrics-9d44df66c-hkfgq" event={"ID":"80e82582-a13a-4c7b-a6ea-3074aff3a5f5","Type":"ContainerStarted","Data":"16368199941af428490eb6ba5b9ced2fba0a8d585fe9cc4d55d803b3ba04208a"}
Apr 24 14:27:08.557234 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:27:08.557194    2574 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/51022bd6-a15d-4d4a-aa38-101a7f8e558f-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"51022bd6-a15d-4d4a-aa38-101a7f8e558f\") " pod="openshift-monitoring/alertmanager-main-0"
Apr 24 14:27:08.558090 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:27:08.558060    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/51022bd6-a15d-4d4a-aa38-101a7f8e558f-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"51022bd6-a15d-4d4a-aa38-101a7f8e558f\") " pod="openshift-monitoring/alertmanager-main-0"
Apr 24 14:27:08.828745 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:27:08.828716    2574 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0"
Apr 24 14:27:09.039469 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:27:09.039386    2574 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/alertmanager-main-0"]
Apr 24 14:27:09.329596 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:27:09.329503    2574 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/openshift-state-metrics-9d44df66c-hkfgq" event={"ID":"80e82582-a13a-4c7b-a6ea-3074aff3a5f5","Type":"ContainerStarted","Data":"622460d9f7cb834db66e66623ce198b2ba1bd35a43fe3292e6b8f33f5d32a36f"}
Apr 24 14:27:09.330733 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:27:09.330699    2574 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"51022bd6-a15d-4d4a-aa38-101a7f8e558f","Type":"ContainerStarted","Data":"1e58f34642f758d3c20c3d22471d41da26761aefe241e11b1a47ec69436f77e8"}
Apr 24 14:27:09.332008 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:27:09.331977    2574 generic.go:358] "Generic (PLEG): container finished" podID="0694c1e6-ce19-4731-a344-f8558961d7d4" containerID="42fe8d2ee36b0da64bb79803e01ffbdcf8a0802f2439787f18e883791e515def" exitCode=0
Apr 24 14:27:09.332143 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:27:09.332061    2574 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-tprms" event={"ID":"0694c1e6-ce19-4731-a344-f8558961d7d4","Type":"ContainerDied","Data":"42fe8d2ee36b0da64bb79803e01ffbdcf8a0802f2439787f18e883791e515def"}
Apr 24 14:27:09.334091 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:27:09.334070    2574 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/kube-state-metrics-69db897b98-spwlq" event={"ID":"e22659f8-0faa-4d68-99a6-d7d61e263c6a","Type":"ContainerStarted","Data":"8763b2d1cfa7a783c5f354cfcce6a90ab476f461aeff3506af4a42b1a44035be"}
Apr 24 14:27:09.334200 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:27:09.334094    2574 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/kube-state-metrics-69db897b98-spwlq" event={"ID":"e22659f8-0faa-4d68-99a6-d7d61e263c6a","Type":"ContainerStarted","Data":"2a777eeae2bb628bf271efe712bb95e4ecd1aacdb5906e7fa3ca466d4e8da427"}
Apr 24 14:27:09.334200 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:27:09.334105    2574 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/kube-state-metrics-69db897b98-spwlq" event={"ID":"e22659f8-0faa-4d68-99a6-d7d61e263c6a","Type":"ContainerStarted","Data":"8790bc4f4c9f80cf9ebb13d388ae28e33309deca1545d05c38e1892357ee96a0"}
Apr 24 14:27:09.345758 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:27:09.345719    2574 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-monitoring/openshift-state-metrics-9d44df66c-hkfgq" podStartSLOduration=1.9565841640000001 podStartE2EDuration="3.345705133s" podCreationTimestamp="2026-04-24 14:27:06 +0000 UTC" firstStartedPulling="2026-04-24 14:27:07.395490322 +0000 UTC m=+190.127591949" lastFinishedPulling="2026-04-24 14:27:08.784611291 +0000 UTC m=+191.516712918" observedRunningTime="2026-04-24 14:27:09.344517831 +0000 UTC m=+192.076619480" watchObservedRunningTime="2026-04-24 14:27:09.345705133 +0000 UTC m=+192.077806818"
Apr 24 14:27:09.384805 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:27:09.384751    2574 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-monitoring/kube-state-metrics-69db897b98-spwlq" podStartSLOduration=1.882454705 podStartE2EDuration="3.384734486s" podCreationTimestamp="2026-04-24 14:27:06 +0000 UTC" firstStartedPulling="2026-04-24 14:27:07.285424646 +0000 UTC m=+190.017526275" lastFinishedPulling="2026-04-24 14:27:08.787704414 +0000 UTC m=+191.519806056" observedRunningTime="2026-04-24 14:27:09.383180586 +0000 UTC m=+192.115282231" watchObservedRunningTime="2026-04-24 14:27:09.384734486 +0000 UTC m=+192.116836179"
Apr 24 14:27:10.338434 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:27:10.338330    2574 generic.go:358] "Generic (PLEG): container finished" podID="51022bd6-a15d-4d4a-aa38-101a7f8e558f" containerID="8fe4f9066e0005297e706ae20951c1c94f64e2332bdc9a59180109691f528222" exitCode=0
Apr 24 14:27:10.338842 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:27:10.338434    2574 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"51022bd6-a15d-4d4a-aa38-101a7f8e558f","Type":"ContainerDied","Data":"8fe4f9066e0005297e706ae20951c1c94f64e2332bdc9a59180109691f528222"}
Apr 24 14:27:10.340466 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:27:10.340442    2574 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-tprms" event={"ID":"0694c1e6-ce19-4731-a344-f8558961d7d4","Type":"ContainerStarted","Data":"361969d0eed9ec153e634ac3991e4593a26899e72e4588f72a94fce6fb6633bd"}
Apr 24 14:27:10.340564 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:27:10.340475    2574 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-tprms" event={"ID":"0694c1e6-ce19-4731-a344-f8558961d7d4","Type":"ContainerStarted","Data":"7c0599e545260c2f638dc6d4362a630550593497f92927ff79e09992c169ec5c"}
Apr 24 14:27:10.377853 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:27:10.377806    2574 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-monitoring/node-exporter-tprms" podStartSLOduration=3.46044811 podStartE2EDuration="4.377791266s" podCreationTimestamp="2026-04-24 14:27:06 +0000 UTC" firstStartedPulling="2026-04-24 14:27:07.149478474 +0000 UTC m=+189.881580111" lastFinishedPulling="2026-04-24 14:27:08.066821631 +0000 UTC m=+190.798923267" observedRunningTime="2026-04-24 14:27:10.376608313 +0000 UTC m=+193.108709962" watchObservedRunningTime="2026-04-24 14:27:10.377791266 +0000 UTC m=+193.109892917"
Apr 24 14:27:10.974059 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:27:10.974024    2574 kubelet.go:2553] "SyncLoop DELETE" source="api" pods=["openshift-console/console-5cb8dd7bdc-bpg26"]
Apr 24 14:27:11.211115 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:27:11.211083    2574 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["openshift-monitoring/metrics-server-6cf575bddc-lztrh"]
Apr 24 14:27:11.215102 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:27:11.215075    2574 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/metrics-server-6cf575bddc-lztrh"
Apr 24 14:27:11.217025 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:27:11.217004    2574 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kubelet-serving-ca-bundle\""
Apr 24 14:27:11.217390 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:27:11.217369    2574 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-server-tls\""
Apr 24 14:27:11.217390 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:27:11.217397    2574 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-server-dockercfg-nb4dk\""
Apr 24 14:27:11.217552 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:27:11.217469    2574 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"metrics-server-audit-profiles\""
Apr 24 14:27:11.217552 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:27:11.217516    2574 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-server-9dlodqfb0f8l\""
Apr 24 14:27:11.217648 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:27:11.217521    2574 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-server-client-certs\""
Apr 24 14:27:11.224702 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:27:11.224640    2574 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/metrics-server-6cf575bddc-lztrh"]
Apr 24 14:27:11.284396 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:27:11.284330    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-metrics-server-client-certs\" (UniqueName: \"kubernetes.io/secret/9dda3a92-755c-4dee-a886-0f8519fe84af-secret-metrics-server-client-certs\") pod \"metrics-server-6cf575bddc-lztrh\" (UID: \"9dda3a92-755c-4dee-a886-0f8519fe84af\") " pod="openshift-monitoring/metrics-server-6cf575bddc-lztrh"
Apr 24 14:27:11.284572 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:27:11.284426    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"client-ca-bundle\" (UniqueName: \"kubernetes.io/secret/9dda3a92-755c-4dee-a886-0f8519fe84af-client-ca-bundle\") pod \"metrics-server-6cf575bddc-lztrh\" (UID: \"9dda3a92-755c-4dee-a886-0f8519fe84af\") " pod="openshift-monitoring/metrics-server-6cf575bddc-lztrh"
Apr 24 14:27:11.284572 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:27:11.284477    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"audit-log\" (UniqueName: \"kubernetes.io/empty-dir/9dda3a92-755c-4dee-a886-0f8519fe84af-audit-log\") pod \"metrics-server-6cf575bddc-lztrh\" (UID: \"9dda3a92-755c-4dee-a886-0f8519fe84af\") " pod="openshift-monitoring/metrics-server-6cf575bddc-lztrh"
Apr 24 14:27:11.284572 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:27:11.284542    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-server-audit-profiles\" (UniqueName: \"kubernetes.io/configmap/9dda3a92-755c-4dee-a886-0f8519fe84af-metrics-server-audit-profiles\") pod \"metrics-server-6cf575bddc-lztrh\" (UID: \"9dda3a92-755c-4dee-a886-0f8519fe84af\") " pod="openshift-monitoring/metrics-server-6cf575bddc-lztrh"
Apr 24 14:27:11.284719 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:27:11.284577    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-metrics-server-tls\" (UniqueName: \"kubernetes.io/secret/9dda3a92-755c-4dee-a886-0f8519fe84af-secret-metrics-server-tls\") pod \"metrics-server-6cf575bddc-lztrh\" (UID: \"9dda3a92-755c-4dee-a886-0f8519fe84af\") " pod="openshift-monitoring/metrics-server-6cf575bddc-lztrh"
Apr 24 14:27:11.284719 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:27:11.284619    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mmgnw\" (UniqueName: \"kubernetes.io/projected/9dda3a92-755c-4dee-a886-0f8519fe84af-kube-api-access-mmgnw\") pod \"metrics-server-6cf575bddc-lztrh\" (UID: \"9dda3a92-755c-4dee-a886-0f8519fe84af\") " pod="openshift-monitoring/metrics-server-6cf575bddc-lztrh"
Apr 24 14:27:11.284719 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:27:11.284688    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/9dda3a92-755c-4dee-a886-0f8519fe84af-configmap-kubelet-serving-ca-bundle\") pod \"metrics-server-6cf575bddc-lztrh\" (UID: \"9dda3a92-755c-4dee-a886-0f8519fe84af\") " pod="openshift-monitoring/metrics-server-6cf575bddc-lztrh"
Apr 24 14:27:11.385133 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:27:11.385095    2574 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"client-ca-bundle\" (UniqueName: \"kubernetes.io/secret/9dda3a92-755c-4dee-a886-0f8519fe84af-client-ca-bundle\") pod \"metrics-server-6cf575bddc-lztrh\" (UID: \"9dda3a92-755c-4dee-a886-0f8519fe84af\") " pod="openshift-monitoring/metrics-server-6cf575bddc-lztrh"
Apr 24 14:27:11.385625 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:27:11.385138    2574 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"audit-log\" (UniqueName: \"kubernetes.io/empty-dir/9dda3a92-755c-4dee-a886-0f8519fe84af-audit-log\") pod \"metrics-server-6cf575bddc-lztrh\" (UID: \"9dda3a92-755c-4dee-a886-0f8519fe84af\") " pod="openshift-monitoring/metrics-server-6cf575bddc-lztrh"
Apr 24 14:27:11.385625 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:27:11.385187    2574 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"metrics-server-audit-profiles\" (UniqueName: \"kubernetes.io/configmap/9dda3a92-755c-4dee-a886-0f8519fe84af-metrics-server-audit-profiles\") pod \"metrics-server-6cf575bddc-lztrh\" (UID: \"9dda3a92-755c-4dee-a886-0f8519fe84af\") " pod="openshift-monitoring/metrics-server-6cf575bddc-lztrh"
Apr 24 14:27:11.385625 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:27:11.385216    2574 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"secret-metrics-server-tls\" (UniqueName: \"kubernetes.io/secret/9dda3a92-755c-4dee-a886-0f8519fe84af-secret-metrics-server-tls\") pod \"metrics-server-6cf575bddc-lztrh\" (UID: \"9dda3a92-755c-4dee-a886-0f8519fe84af\") " pod="openshift-monitoring/metrics-server-6cf575bddc-lztrh"
Apr 24 14:27:11.385625 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:27:11.385254    2574 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-mmgnw\" (UniqueName: \"kubernetes.io/projected/9dda3a92-755c-4dee-a886-0f8519fe84af-kube-api-access-mmgnw\") pod \"metrics-server-6cf575bddc-lztrh\" (UID: \"9dda3a92-755c-4dee-a886-0f8519fe84af\") " pod="openshift-monitoring/metrics-server-6cf575bddc-lztrh"
Apr 24 14:27:11.385625 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:27:11.385293    2574 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/9dda3a92-755c-4dee-a886-0f8519fe84af-configmap-kubelet-serving-ca-bundle\") pod \"metrics-server-6cf575bddc-lztrh\" (UID: \"9dda3a92-755c-4dee-a886-0f8519fe84af\") " pod="openshift-monitoring/metrics-server-6cf575bddc-lztrh"
Apr 24 14:27:11.385625 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:27:11.385358    2574 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"secret-metrics-server-client-certs\" (UniqueName: \"kubernetes.io/secret/9dda3a92-755c-4dee-a886-0f8519fe84af-secret-metrics-server-client-certs\") pod \"metrics-server-6cf575bddc-lztrh\" (UID: \"9dda3a92-755c-4dee-a886-0f8519fe84af\") " pod="openshift-monitoring/metrics-server-6cf575bddc-lztrh"
Apr 24 14:27:11.385946 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:27:11.385695    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"audit-log\" (UniqueName: \"kubernetes.io/empty-dir/9dda3a92-755c-4dee-a886-0f8519fe84af-audit-log\") pod \"metrics-server-6cf575bddc-lztrh\" (UID: \"9dda3a92-755c-4dee-a886-0f8519fe84af\") " pod="openshift-monitoring/metrics-server-6cf575bddc-lztrh"
Apr 24 14:27:11.386708 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:27:11.386674    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/9dda3a92-755c-4dee-a886-0f8519fe84af-configmap-kubelet-serving-ca-bundle\") pod \"metrics-server-6cf575bddc-lztrh\" (UID: \"9dda3a92-755c-4dee-a886-0f8519fe84af\") " pod="openshift-monitoring/metrics-server-6cf575bddc-lztrh"
Apr 24 14:27:11.386708 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:27:11.386685    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-server-audit-profiles\" (UniqueName: \"kubernetes.io/configmap/9dda3a92-755c-4dee-a886-0f8519fe84af-metrics-server-audit-profiles\") pod \"metrics-server-6cf575bddc-lztrh\" (UID: \"9dda3a92-755c-4dee-a886-0f8519fe84af\") " pod="openshift-monitoring/metrics-server-6cf575bddc-lztrh"
Apr 24 14:27:11.388286 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:27:11.388204    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"client-ca-bundle\" (UniqueName: \"kubernetes.io/secret/9dda3a92-755c-4dee-a886-0f8519fe84af-client-ca-bundle\") pod \"metrics-server-6cf575bddc-lztrh\" (UID: \"9dda3a92-755c-4dee-a886-0f8519fe84af\") " pod="openshift-monitoring/metrics-server-6cf575bddc-lztrh"
Apr 24 14:27:11.388420 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:27:11.388285    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-metrics-server-tls\" (UniqueName: \"kubernetes.io/secret/9dda3a92-755c-4dee-a886-0f8519fe84af-secret-metrics-server-tls\") pod \"metrics-server-6cf575bddc-lztrh\" (UID: \"9dda3a92-755c-4dee-a886-0f8519fe84af\") " pod="openshift-monitoring/metrics-server-6cf575bddc-lztrh"
Apr 24 14:27:11.388497 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:27:11.388479    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-metrics-server-client-certs\" (UniqueName: \"kubernetes.io/secret/9dda3a92-755c-4dee-a886-0f8519fe84af-secret-metrics-server-client-certs\") pod \"metrics-server-6cf575bddc-lztrh\" (UID: \"9dda3a92-755c-4dee-a886-0f8519fe84af\") " pod="openshift-monitoring/metrics-server-6cf575bddc-lztrh"
Apr 24 14:27:11.392497 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:27:11.392479    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-mmgnw\" (UniqueName: \"kubernetes.io/projected/9dda3a92-755c-4dee-a886-0f8519fe84af-kube-api-access-mmgnw\") pod \"metrics-server-6cf575bddc-lztrh\" (UID: \"9dda3a92-755c-4dee-a886-0f8519fe84af\") " pod="openshift-monitoring/metrics-server-6cf575bddc-lztrh"
Apr 24 14:27:11.525962 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:27:11.525937    2574 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/metrics-server-6cf575bddc-lztrh"
Apr 24 14:27:11.602950 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:27:11.602922    2574 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["openshift-monitoring/monitoring-plugin-7dccd58f55-4v2k7"]
Apr 24 14:27:11.606055 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:27:11.606033    2574 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/monitoring-plugin-7dccd58f55-4v2k7"
Apr 24 14:27:11.607988 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:27:11.607753    2574 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"monitoring-plugin-cert\""
Apr 24 14:27:11.607988 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:27:11.607809    2574 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"default-dockercfg-lbxmg\""
Apr 24 14:27:11.617964 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:27:11.617913    2574 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/monitoring-plugin-7dccd58f55-4v2k7"]
Apr 24 14:27:11.669665 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:27:11.669630    2574 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/metrics-server-6cf575bddc-lztrh"]
Apr 24 14:27:11.672699 ip-10-0-136-132 kubenswrapper[2574]: W0424 14:27:11.672676    2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9dda3a92_755c_4dee_a886_0f8519fe84af.slice/crio-955789af47c723ed1ce44c8086ddac0c60f05a6b1d92b37e0fc9029360947a91 WatchSource:0}: Error finding container 955789af47c723ed1ce44c8086ddac0c60f05a6b1d92b37e0fc9029360947a91: Status 404 returned error can't find the container with id 955789af47c723ed1ce44c8086ddac0c60f05a6b1d92b37e0fc9029360947a91
Apr 24 14:27:11.687757 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:27:11.687701    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"monitoring-plugin-cert\" (UniqueName: \"kubernetes.io/secret/9192b2ee-7b51-4184-9f8f-fd91ff3f3535-monitoring-plugin-cert\") pod \"monitoring-plugin-7dccd58f55-4v2k7\" (UID: \"9192b2ee-7b51-4184-9f8f-fd91ff3f3535\") " pod="openshift-monitoring/monitoring-plugin-7dccd58f55-4v2k7"
Apr 24 14:27:11.788476 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:27:11.788440    2574 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"monitoring-plugin-cert\" (UniqueName: \"kubernetes.io/secret/9192b2ee-7b51-4184-9f8f-fd91ff3f3535-monitoring-plugin-cert\") pod \"monitoring-plugin-7dccd58f55-4v2k7\" (UID: \"9192b2ee-7b51-4184-9f8f-fd91ff3f3535\") " pod="openshift-monitoring/monitoring-plugin-7dccd58f55-4v2k7"
Apr 24 14:27:11.788614 ip-10-0-136-132 kubenswrapper[2574]: E0424 14:27:11.788560    2574 secret.go:189] Couldn't get secret openshift-monitoring/monitoring-plugin-cert: secret "monitoring-plugin-cert" not found
Apr 24 14:27:11.788658 ip-10-0-136-132 kubenswrapper[2574]: E0424 14:27:11.788614    2574 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/9192b2ee-7b51-4184-9f8f-fd91ff3f3535-monitoring-plugin-cert podName:9192b2ee-7b51-4184-9f8f-fd91ff3f3535 nodeName:}" failed. No retries permitted until 2026-04-24 14:27:12.288601338 +0000 UTC m=+195.020702965 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "monitoring-plugin-cert" (UniqueName: "kubernetes.io/secret/9192b2ee-7b51-4184-9f8f-fd91ff3f3535-monitoring-plugin-cert") pod "monitoring-plugin-7dccd58f55-4v2k7" (UID: "9192b2ee-7b51-4184-9f8f-fd91ff3f3535") : secret "monitoring-plugin-cert" not found
Apr 24 14:27:12.292930 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:27:12.292894    2574 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"monitoring-plugin-cert\" (UniqueName: \"kubernetes.io/secret/9192b2ee-7b51-4184-9f8f-fd91ff3f3535-monitoring-plugin-cert\") pod \"monitoring-plugin-7dccd58f55-4v2k7\" (UID: \"9192b2ee-7b51-4184-9f8f-fd91ff3f3535\") " pod="openshift-monitoring/monitoring-plugin-7dccd58f55-4v2k7"
Apr 24 14:27:12.295302 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:27:12.295269    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"monitoring-plugin-cert\" (UniqueName: \"kubernetes.io/secret/9192b2ee-7b51-4184-9f8f-fd91ff3f3535-monitoring-plugin-cert\") pod \"monitoring-plugin-7dccd58f55-4v2k7\" (UID: \"9192b2ee-7b51-4184-9f8f-fd91ff3f3535\") " pod="openshift-monitoring/monitoring-plugin-7dccd58f55-4v2k7"
Apr 24 14:27:12.348673 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:27:12.348635    2574 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/metrics-server-6cf575bddc-lztrh" event={"ID":"9dda3a92-755c-4dee-a886-0f8519fe84af","Type":"ContainerStarted","Data":"955789af47c723ed1ce44c8086ddac0c60f05a6b1d92b37e0fc9029360947a91"}
Apr 24 14:27:12.351916 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:27:12.351883    2574 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"51022bd6-a15d-4d4a-aa38-101a7f8e558f","Type":"ContainerStarted","Data":"35124e0de577b2501f6d488232739d48cc68c8c6c875bd51fbb4b0fb91ee3f1d"}
Apr 24 14:27:12.351916 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:27:12.351918    2574 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"51022bd6-a15d-4d4a-aa38-101a7f8e558f","Type":"ContainerStarted","Data":"3bf6f339c7c6ac08a658fca8e90e38f6ba581f94fa0c8ae0605d67d3e4526c8a"}
Apr 24 14:27:12.352167 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:27:12.351929    2574 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"51022bd6-a15d-4d4a-aa38-101a7f8e558f","Type":"ContainerStarted","Data":"06f94868f48a92b9df01560a6c073819c61bad0ef1c0dedd4829e114a24cc608"}
Apr 24 14:27:12.352167 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:27:12.351940    2574 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"51022bd6-a15d-4d4a-aa38-101a7f8e558f","Type":"ContainerStarted","Data":"21b3505b1b6ec7a3789d223b2220b6156449fc3cf864b9ab03b9969f3300e18e"}
Apr 24 14:27:12.352167 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:27:12.351965    2574 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"51022bd6-a15d-4d4a-aa38-101a7f8e558f","Type":"ContainerStarted","Data":"beb9b4bd217697bce4ded5f782d413e8b12bad131f8fb43a3be3c421c1d4ca77"}
Apr 24 14:27:12.520506 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:27:12.520468    2574 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/monitoring-plugin-7dccd58f55-4v2k7"
Apr 24 14:27:12.779183 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:27:12.779152    2574 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["openshift-console/console-65485c4fbf-pmgnc"]
Apr 24 14:27:12.788076 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:27:12.788041    2574 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-65485c4fbf-pmgnc"
Apr 24 14:27:12.789419 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:27:12.789373    2574 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-65485c4fbf-pmgnc"]
Apr 24 14:27:12.794764 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:27:12.794740    2574 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"trusted-ca-bundle\""
Apr 24 14:27:12.898485 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:27:12.898447    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/a9fe99c0-ecfb-4b06-b60a-2dbe93db2d3a-console-serving-cert\") pod \"console-65485c4fbf-pmgnc\" (UID: \"a9fe99c0-ecfb-4b06-b60a-2dbe93db2d3a\") " pod="openshift-console/console-65485c4fbf-pmgnc"
Apr 24 14:27:12.898485 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:27:12.898487    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-t7b7r\" (UniqueName: \"kubernetes.io/projected/a9fe99c0-ecfb-4b06-b60a-2dbe93db2d3a-kube-api-access-t7b7r\") pod \"console-65485c4fbf-pmgnc\" (UID: \"a9fe99c0-ecfb-4b06-b60a-2dbe93db2d3a\") " pod="openshift-console/console-65485c4fbf-pmgnc"
Apr 24 14:27:12.898703 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:27:12.898516    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/a9fe99c0-ecfb-4b06-b60a-2dbe93db2d3a-trusted-ca-bundle\") pod \"console-65485c4fbf-pmgnc\" (UID: \"a9fe99c0-ecfb-4b06-b60a-2dbe93db2d3a\") " pod="openshift-console/console-65485c4fbf-pmgnc"
Apr 24 14:27:12.898703 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:27:12.898632    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/a9fe99c0-ecfb-4b06-b60a-2dbe93db2d3a-console-oauth-config\") pod \"console-65485c4fbf-pmgnc\" (UID: \"a9fe99c0-ecfb-4b06-b60a-2dbe93db2d3a\") " pod="openshift-console/console-65485c4fbf-pmgnc"
Apr 24 14:27:12.898703 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:27:12.898676    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/a9fe99c0-ecfb-4b06-b60a-2dbe93db2d3a-oauth-serving-cert\") pod \"console-65485c4fbf-pmgnc\" (UID: \"a9fe99c0-ecfb-4b06-b60a-2dbe93db2d3a\") " pod="openshift-console/console-65485c4fbf-pmgnc"
Apr 24 14:27:12.898825 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:27:12.898711    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/a9fe99c0-ecfb-4b06-b60a-2dbe93db2d3a-service-ca\") pod \"console-65485c4fbf-pmgnc\" (UID: \"a9fe99c0-ecfb-4b06-b60a-2dbe93db2d3a\") " pod="openshift-console/console-65485c4fbf-pmgnc"
Apr 24 14:27:12.898825 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:27:12.898809    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/a9fe99c0-ecfb-4b06-b60a-2dbe93db2d3a-console-config\") pod \"console-65485c4fbf-pmgnc\" (UID: \"a9fe99c0-ecfb-4b06-b60a-2dbe93db2d3a\") " pod="openshift-console/console-65485c4fbf-pmgnc"
Apr 24 14:27:13.000012 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:27:12.999974    2574 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/a9fe99c0-ecfb-4b06-b60a-2dbe93db2d3a-console-serving-cert\") pod \"console-65485c4fbf-pmgnc\" (UID: \"a9fe99c0-ecfb-4b06-b60a-2dbe93db2d3a\") " pod="openshift-console/console-65485c4fbf-pmgnc"
Apr 24 14:27:13.000012 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:27:13.000020    2574 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-t7b7r\" (UniqueName: \"kubernetes.io/projected/a9fe99c0-ecfb-4b06-b60a-2dbe93db2d3a-kube-api-access-t7b7r\") pod \"console-65485c4fbf-pmgnc\" (UID: \"a9fe99c0-ecfb-4b06-b60a-2dbe93db2d3a\") " pod="openshift-console/console-65485c4fbf-pmgnc"
Apr 24 14:27:13.000286 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:27:13.000048    2574 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/a9fe99c0-ecfb-4b06-b60a-2dbe93db2d3a-trusted-ca-bundle\") pod \"console-65485c4fbf-pmgnc\" (UID: \"a9fe99c0-ecfb-4b06-b60a-2dbe93db2d3a\") " pod="openshift-console/console-65485c4fbf-pmgnc"
Apr 24 14:27:13.000286 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:27:13.000099    2574 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/a9fe99c0-ecfb-4b06-b60a-2dbe93db2d3a-console-oauth-config\") pod \"console-65485c4fbf-pmgnc\" (UID: \"a9fe99c0-ecfb-4b06-b60a-2dbe93db2d3a\") " pod="openshift-console/console-65485c4fbf-pmgnc"
Apr 24 14:27:13.000286 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:27:13.000129    2574 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/a9fe99c0-ecfb-4b06-b60a-2dbe93db2d3a-oauth-serving-cert\") pod \"console-65485c4fbf-pmgnc\" (UID: \"a9fe99c0-ecfb-4b06-b60a-2dbe93db2d3a\") " pod="openshift-console/console-65485c4fbf-pmgnc"
Apr 24 14:27:13.000286 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:27:13.000161    2574 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/a9fe99c0-ecfb-4b06-b60a-2dbe93db2d3a-service-ca\") pod \"console-65485c4fbf-pmgnc\" (UID: \"a9fe99c0-ecfb-4b06-b60a-2dbe93db2d3a\") " pod="openshift-console/console-65485c4fbf-pmgnc"
Apr 24 14:27:13.000286 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:27:13.000203    2574 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/a9fe99c0-ecfb-4b06-b60a-2dbe93db2d3a-console-config\") pod \"console-65485c4fbf-pmgnc\" (UID: \"a9fe99c0-ecfb-4b06-b60a-2dbe93db2d3a\") " pod="openshift-console/console-65485c4fbf-pmgnc"
Apr 24 14:27:13.001086 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:27:13.000975    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/a9fe99c0-ecfb-4b06-b60a-2dbe93db2d3a-console-config\") pod \"console-65485c4fbf-pmgnc\" (UID: \"a9fe99c0-ecfb-4b06-b60a-2dbe93db2d3a\") " pod="openshift-console/console-65485c4fbf-pmgnc"
Apr 24 14:27:13.001200 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:27:13.001142    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/a9fe99c0-ecfb-4b06-b60a-2dbe93db2d3a-oauth-serving-cert\") pod \"console-65485c4fbf-pmgnc\" (UID: \"a9fe99c0-ecfb-4b06-b60a-2dbe93db2d3a\") " pod="openshift-console/console-65485c4fbf-pmgnc"
Apr 24 14:27:13.001263 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:27:13.001219    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/a9fe99c0-ecfb-4b06-b60a-2dbe93db2d3a-service-ca\") pod \"console-65485c4fbf-pmgnc\" (UID: \"a9fe99c0-ecfb-4b06-b60a-2dbe93db2d3a\") " pod="openshift-console/console-65485c4fbf-pmgnc"
Apr 24 14:27:13.001263 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:27:13.001234    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/a9fe99c0-ecfb-4b06-b60a-2dbe93db2d3a-trusted-ca-bundle\") pod \"console-65485c4fbf-pmgnc\" (UID: \"a9fe99c0-ecfb-4b06-b60a-2dbe93db2d3a\") " pod="openshift-console/console-65485c4fbf-pmgnc"
Apr 24 14:27:13.003134 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:27:13.003113    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/a9fe99c0-ecfb-4b06-b60a-2dbe93db2d3a-console-oauth-config\") pod \"console-65485c4fbf-pmgnc\" (UID: \"a9fe99c0-ecfb-4b06-b60a-2dbe93db2d3a\") " pod="openshift-console/console-65485c4fbf-pmgnc"
Apr 24 14:27:13.003295 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:27:13.003275    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/a9fe99c0-ecfb-4b06-b60a-2dbe93db2d3a-console-serving-cert\") pod \"console-65485c4fbf-pmgnc\" (UID: \"a9fe99c0-ecfb-4b06-b60a-2dbe93db2d3a\") " pod="openshift-console/console-65485c4fbf-pmgnc"
Apr 24 14:27:13.013491 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:27:13.013466    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-t7b7r\" (UniqueName: \"kubernetes.io/projected/a9fe99c0-ecfb-4b06-b60a-2dbe93db2d3a-kube-api-access-t7b7r\") pod \"console-65485c4fbf-pmgnc\" (UID: \"a9fe99c0-ecfb-4b06-b60a-2dbe93db2d3a\") " pod="openshift-console/console-65485c4fbf-pmgnc"
Apr 24 14:27:13.100726 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:27:13.100598    2574 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-65485c4fbf-pmgnc"
Apr 24 14:27:13.185008 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:27:13.184968    2574 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/monitoring-plugin-7dccd58f55-4v2k7"]
Apr 24 14:27:13.187088 ip-10-0-136-132 kubenswrapper[2574]: W0424 14:27:13.187058    2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9192b2ee_7b51_4184_9f8f_fd91ff3f3535.slice/crio-1ac2f1bc4dc20fa0a7e38a016d2cf65487d1c57c78f260c205a4578d0757d49b WatchSource:0}: Error finding container 1ac2f1bc4dc20fa0a7e38a016d2cf65487d1c57c78f260c205a4578d0757d49b: Status 404 returned error can't find the container with id 1ac2f1bc4dc20fa0a7e38a016d2cf65487d1c57c78f260c205a4578d0757d49b
Apr 24 14:27:13.248186 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:27:13.248161    2574 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-65485c4fbf-pmgnc"]
Apr 24 14:27:13.251702 ip-10-0-136-132 kubenswrapper[2574]: W0424 14:27:13.251676    2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda9fe99c0_ecfb_4b06_b60a_2dbe93db2d3a.slice/crio-994bdd2916f2518b776bbc09038a4375554d0f8eb443034b11bd7e76c6f0b98c WatchSource:0}: Error finding container 994bdd2916f2518b776bbc09038a4375554d0f8eb443034b11bd7e76c6f0b98c: Status 404 returned error can't find the container with id 994bdd2916f2518b776bbc09038a4375554d0f8eb443034b11bd7e76c6f0b98c
Apr 24 14:27:13.356516 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:27:13.356482    2574 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/metrics-server-6cf575bddc-lztrh" event={"ID":"9dda3a92-755c-4dee-a886-0f8519fe84af","Type":"ContainerStarted","Data":"24ddc8c70a1fb315afc920f12d48d97d30963c7f8ca7e1c9103f6affab4bbd4c"}
Apr 24 14:27:13.359552 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:27:13.359510    2574 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"51022bd6-a15d-4d4a-aa38-101a7f8e558f","Type":"ContainerStarted","Data":"6c3e0b63b1587f9c88dbbbd70db4a0b25a84a46407ecac2d637225abe1e15130"}
Apr 24 14:27:13.360948 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:27:13.360923    2574 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-65485c4fbf-pmgnc" event={"ID":"a9fe99c0-ecfb-4b06-b60a-2dbe93db2d3a","Type":"ContainerStarted","Data":"4354ff06b964bd71c3cd36606b7a571c06e6c287f1ea646eb2e0e1c203f52863"}
Apr 24 14:27:13.361067 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:27:13.360953    2574 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-65485c4fbf-pmgnc" event={"ID":"a9fe99c0-ecfb-4b06-b60a-2dbe93db2d3a","Type":"ContainerStarted","Data":"994bdd2916f2518b776bbc09038a4375554d0f8eb443034b11bd7e76c6f0b98c"}
Apr 24 14:27:13.363351 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:27:13.363315    2574 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/monitoring-plugin-7dccd58f55-4v2k7" event={"ID":"9192b2ee-7b51-4184-9f8f-fd91ff3f3535","Type":"ContainerStarted","Data":"1ac2f1bc4dc20fa0a7e38a016d2cf65487d1c57c78f260c205a4578d0757d49b"}
Apr 24 14:27:13.374900 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:27:13.374850    2574 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-monitoring/metrics-server-6cf575bddc-lztrh" podStartSLOduration=0.999908761 podStartE2EDuration="2.374837025s" podCreationTimestamp="2026-04-24 14:27:11 +0000 UTC" firstStartedPulling="2026-04-24 14:27:11.67455963 +0000 UTC m=+194.406661261" lastFinishedPulling="2026-04-24 14:27:13.04948789 +0000 UTC m=+195.781589525" observedRunningTime="2026-04-24 14:27:13.373303175 +0000 UTC m=+196.105404886" watchObservedRunningTime="2026-04-24 14:27:13.374837025 +0000 UTC m=+196.106938744"
Apr 24 14:27:13.396976 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:27:13.396931    2574 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-console/console-65485c4fbf-pmgnc" podStartSLOduration=1.396916934 podStartE2EDuration="1.396916934s" podCreationTimestamp="2026-04-24 14:27:12 +0000 UTC" firstStartedPulling="0001-01-01 00:00:00 +0000 UTC" lastFinishedPulling="0001-01-01 00:00:00 +0000 UTC" observedRunningTime="2026-04-24 14:27:13.395085649 +0000 UTC m=+196.127187298" watchObservedRunningTime="2026-04-24 14:27:13.396916934 +0000 UTC m=+196.129018583"
Apr 24 14:27:13.424166 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:27:13.424101    2574 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-monitoring/alertmanager-main-0" podStartSLOduration=2.422003523 podStartE2EDuration="6.424082176s" podCreationTimestamp="2026-04-24 14:27:07 +0000 UTC" firstStartedPulling="2026-04-24 14:27:09.045298853 +0000 UTC m=+191.777400496" lastFinishedPulling="2026-04-24 14:27:13.047377507 +0000 UTC m=+195.779479149" observedRunningTime="2026-04-24 14:27:13.423092989 +0000 UTC m=+196.155194639" watchObservedRunningTime="2026-04-24 14:27:13.424082176 +0000 UTC m=+196.156183825"
Apr 24 14:27:15.370493 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:27:15.370459    2574 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/monitoring-plugin-7dccd58f55-4v2k7" event={"ID":"9192b2ee-7b51-4184-9f8f-fd91ff3f3535","Type":"ContainerStarted","Data":"4a96db2c371468056586ab046e5d9969622452f929bec5bb627394e8a84da564"}
Apr 24 14:27:15.370878 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:27:15.370661    2574 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/monitoring-plugin-7dccd58f55-4v2k7"
Apr 24 14:27:15.375614 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:27:15.375591    2574 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/monitoring-plugin-7dccd58f55-4v2k7"
Apr 24 14:27:15.384057 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:27:15.384009    2574 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-monitoring/monitoring-plugin-7dccd58f55-4v2k7" podStartSLOduration=3.107854745 podStartE2EDuration="4.383997513s" podCreationTimestamp="2026-04-24 14:27:11 +0000 UTC" firstStartedPulling="2026-04-24 14:27:13.189402192 +0000 UTC m=+195.921503819" lastFinishedPulling="2026-04-24 14:27:14.465544949 +0000 UTC m=+197.197646587" observedRunningTime="2026-04-24 14:27:15.383027689 +0000 UTC m=+198.115129351" watchObservedRunningTime="2026-04-24 14:27:15.383997513 +0000 UTC m=+198.116099176"
Apr 24 14:27:23.101540 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:27:23.101504    2574 kubelet.go:2658] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-65485c4fbf-pmgnc"
Apr 24 14:27:23.101933 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:27:23.101555    2574 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-65485c4fbf-pmgnc"
Apr 24 14:27:23.106280 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:27:23.106258    2574 kubelet.go:2658] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-65485c4fbf-pmgnc"
Apr 24 14:27:23.398293 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:27:23.398267    2574 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-65485c4fbf-pmgnc"
Apr 24 14:27:31.526768 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:27:31.526733    2574 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/metrics-server-6cf575bddc-lztrh"
Apr 24 14:27:31.526768 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:27:31.526777    2574 kubelet.go:2658] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-monitoring/metrics-server-6cf575bddc-lztrh"
Apr 24 14:27:35.996968 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:27:35.996897    2574 kuberuntime_container.go:864] "Killing container with a grace period" pod="openshift-console/console-5cb8dd7bdc-bpg26" podUID="674cd4ff-c759-42ff-8b5b-f5e4eb2c45b7" containerName="console" containerID="cri-o://41cc9379018e1cc23612d4a01c3b92890a1d15091f1dfc4630f8051edf8f19ec" gracePeriod=15
Apr 24 14:27:36.253167 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:27:36.253104    2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console_console-5cb8dd7bdc-bpg26_674cd4ff-c759-42ff-8b5b-f5e4eb2c45b7/console/0.log"
Apr 24 14:27:36.253279 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:27:36.253175    2574 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-5cb8dd7bdc-bpg26"
Apr 24 14:27:36.309913 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:27:36.309875    2574 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/674cd4ff-c759-42ff-8b5b-f5e4eb2c45b7-service-ca\") pod \"674cd4ff-c759-42ff-8b5b-f5e4eb2c45b7\" (UID: \"674cd4ff-c759-42ff-8b5b-f5e4eb2c45b7\") "
Apr 24 14:27:36.310091 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:27:36.309926    2574 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/674cd4ff-c759-42ff-8b5b-f5e4eb2c45b7-oauth-serving-cert\") pod \"674cd4ff-c759-42ff-8b5b-f5e4eb2c45b7\" (UID: \"674cd4ff-c759-42ff-8b5b-f5e4eb2c45b7\") "
Apr 24 14:27:36.310091 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:27:36.309983    2574 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/674cd4ff-c759-42ff-8b5b-f5e4eb2c45b7-console-config\") pod \"674cd4ff-c759-42ff-8b5b-f5e4eb2c45b7\" (UID: \"674cd4ff-c759-42ff-8b5b-f5e4eb2c45b7\") "
Apr 24 14:27:36.310091 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:27:36.310007    2574 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"kube-api-access-9xdl8\" (UniqueName: \"kubernetes.io/projected/674cd4ff-c759-42ff-8b5b-f5e4eb2c45b7-kube-api-access-9xdl8\") pod \"674cd4ff-c759-42ff-8b5b-f5e4eb2c45b7\" (UID: \"674cd4ff-c759-42ff-8b5b-f5e4eb2c45b7\") "
Apr 24 14:27:36.310091 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:27:36.310049    2574 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/674cd4ff-c759-42ff-8b5b-f5e4eb2c45b7-console-serving-cert\") pod \"674cd4ff-c759-42ff-8b5b-f5e4eb2c45b7\" (UID: \"674cd4ff-c759-42ff-8b5b-f5e4eb2c45b7\") "
Apr 24 14:27:36.310091 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:27:36.310075    2574 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/674cd4ff-c759-42ff-8b5b-f5e4eb2c45b7-console-oauth-config\") pod \"674cd4ff-c759-42ff-8b5b-f5e4eb2c45b7\" (UID: \"674cd4ff-c759-42ff-8b5b-f5e4eb2c45b7\") "
Apr 24 14:27:36.310449 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:27:36.310419    2574 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/674cd4ff-c759-42ff-8b5b-f5e4eb2c45b7-service-ca" (OuterVolumeSpecName: "service-ca") pod "674cd4ff-c759-42ff-8b5b-f5e4eb2c45b7" (UID: "674cd4ff-c759-42ff-8b5b-f5e4eb2c45b7"). InnerVolumeSpecName "service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Apr 24 14:27:36.310579 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:27:36.310514    2574 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/674cd4ff-c759-42ff-8b5b-f5e4eb2c45b7-oauth-serving-cert" (OuterVolumeSpecName: "oauth-serving-cert") pod "674cd4ff-c759-42ff-8b5b-f5e4eb2c45b7" (UID: "674cd4ff-c759-42ff-8b5b-f5e4eb2c45b7"). InnerVolumeSpecName "oauth-serving-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Apr 24 14:27:36.310579 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:27:36.310539    2574 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/674cd4ff-c759-42ff-8b5b-f5e4eb2c45b7-console-config" (OuterVolumeSpecName: "console-config") pod "674cd4ff-c759-42ff-8b5b-f5e4eb2c45b7" (UID: "674cd4ff-c759-42ff-8b5b-f5e4eb2c45b7"). InnerVolumeSpecName "console-config". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Apr 24 14:27:36.312458 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:27:36.312425    2574 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/674cd4ff-c759-42ff-8b5b-f5e4eb2c45b7-console-oauth-config" (OuterVolumeSpecName: "console-oauth-config") pod "674cd4ff-c759-42ff-8b5b-f5e4eb2c45b7" (UID: "674cd4ff-c759-42ff-8b5b-f5e4eb2c45b7"). InnerVolumeSpecName "console-oauth-config". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Apr 24 14:27:36.312537 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:27:36.312459    2574 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/674cd4ff-c759-42ff-8b5b-f5e4eb2c45b7-console-serving-cert" (OuterVolumeSpecName: "console-serving-cert") pod "674cd4ff-c759-42ff-8b5b-f5e4eb2c45b7" (UID: "674cd4ff-c759-42ff-8b5b-f5e4eb2c45b7"). InnerVolumeSpecName "console-serving-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Apr 24 14:27:36.312537 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:27:36.312510    2574 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/674cd4ff-c759-42ff-8b5b-f5e4eb2c45b7-kube-api-access-9xdl8" (OuterVolumeSpecName: "kube-api-access-9xdl8") pod "674cd4ff-c759-42ff-8b5b-f5e4eb2c45b7" (UID: "674cd4ff-c759-42ff-8b5b-f5e4eb2c45b7"). InnerVolumeSpecName "kube-api-access-9xdl8". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Apr 24 14:27:36.411653 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:27:36.411617    2574 reconciler_common.go:299] "Volume detached for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/674cd4ff-c759-42ff-8b5b-f5e4eb2c45b7-oauth-serving-cert\") on node \"ip-10-0-136-132.ec2.internal\" DevicePath \"\""
Apr 24 14:27:36.411653 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:27:36.411646    2574 reconciler_common.go:299] "Volume detached for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/674cd4ff-c759-42ff-8b5b-f5e4eb2c45b7-console-config\") on node \"ip-10-0-136-132.ec2.internal\" DevicePath \"\""
Apr 24 14:27:36.411653 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:27:36.411656    2574 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-9xdl8\" (UniqueName: \"kubernetes.io/projected/674cd4ff-c759-42ff-8b5b-f5e4eb2c45b7-kube-api-access-9xdl8\") on node \"ip-10-0-136-132.ec2.internal\" DevicePath \"\""
Apr 24 14:27:36.411883 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:27:36.411668    2574 reconciler_common.go:299] "Volume detached for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/674cd4ff-c759-42ff-8b5b-f5e4eb2c45b7-console-serving-cert\") on node \"ip-10-0-136-132.ec2.internal\" DevicePath \"\""
Apr 24 14:27:36.411883 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:27:36.411677    2574 reconciler_common.go:299] "Volume detached for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/674cd4ff-c759-42ff-8b5b-f5e4eb2c45b7-console-oauth-config\") on node \"ip-10-0-136-132.ec2.internal\" DevicePath \"\""
Apr 24 14:27:36.411883 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:27:36.411688    2574 reconciler_common.go:299] "Volume detached for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/674cd4ff-c759-42ff-8b5b-f5e4eb2c45b7-service-ca\") on node \"ip-10-0-136-132.ec2.internal\" DevicePath \"\""
Apr 24 14:27:36.431903 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:27:36.431875    2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console_console-5cb8dd7bdc-bpg26_674cd4ff-c759-42ff-8b5b-f5e4eb2c45b7/console/0.log"
Apr 24 14:27:36.432028 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:27:36.431911    2574 generic.go:358] "Generic (PLEG): container finished" podID="674cd4ff-c759-42ff-8b5b-f5e4eb2c45b7" containerID="41cc9379018e1cc23612d4a01c3b92890a1d15091f1dfc4630f8051edf8f19ec" exitCode=2
Apr 24 14:27:36.432028 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:27:36.431979    2574 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-5cb8dd7bdc-bpg26"
Apr 24 14:27:36.432028 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:27:36.431986    2574 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-5cb8dd7bdc-bpg26" event={"ID":"674cd4ff-c759-42ff-8b5b-f5e4eb2c45b7","Type":"ContainerDied","Data":"41cc9379018e1cc23612d4a01c3b92890a1d15091f1dfc4630f8051edf8f19ec"}
Apr 24 14:27:36.432028 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:27:36.432012    2574 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-5cb8dd7bdc-bpg26" event={"ID":"674cd4ff-c759-42ff-8b5b-f5e4eb2c45b7","Type":"ContainerDied","Data":"f86e2e36b9561df68cacac956d831aa8ac0e5c1a6e094eafb36b3f4a10f43398"}
Apr 24 14:27:36.432028 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:27:36.432026    2574 scope.go:117] "RemoveContainer" containerID="41cc9379018e1cc23612d4a01c3b92890a1d15091f1dfc4630f8051edf8f19ec"
Apr 24 14:27:36.440183 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:27:36.440162    2574 scope.go:117] "RemoveContainer" containerID="41cc9379018e1cc23612d4a01c3b92890a1d15091f1dfc4630f8051edf8f19ec"
Apr 24 14:27:36.440484 ip-10-0-136-132 kubenswrapper[2574]: E0424 14:27:36.440463    2574 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"41cc9379018e1cc23612d4a01c3b92890a1d15091f1dfc4630f8051edf8f19ec\": container with ID starting with 41cc9379018e1cc23612d4a01c3b92890a1d15091f1dfc4630f8051edf8f19ec not found: ID does not exist" containerID="41cc9379018e1cc23612d4a01c3b92890a1d15091f1dfc4630f8051edf8f19ec"
Apr 24 14:27:36.440543 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:27:36.440493    2574 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"41cc9379018e1cc23612d4a01c3b92890a1d15091f1dfc4630f8051edf8f19ec"} err="failed to get container status \"41cc9379018e1cc23612d4a01c3b92890a1d15091f1dfc4630f8051edf8f19ec\": rpc error: code = NotFound desc = could not find container \"41cc9379018e1cc23612d4a01c3b92890a1d15091f1dfc4630f8051edf8f19ec\": container with ID starting with 41cc9379018e1cc23612d4a01c3b92890a1d15091f1dfc4630f8051edf8f19ec not found: ID does not exist"
Apr 24 14:27:36.451908 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:27:36.451879    2574 kubelet.go:2553] "SyncLoop DELETE" source="api" pods=["openshift-console/console-5cb8dd7bdc-bpg26"]
Apr 24 14:27:36.458898 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:27:36.458878    2574 kubelet.go:2547] "SyncLoop REMOVE" source="api" pods=["openshift-console/console-5cb8dd7bdc-bpg26"]
Apr 24 14:27:37.769544 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:27:37.769511    2574 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="674cd4ff-c759-42ff-8b5b-f5e4eb2c45b7" path="/var/lib/kubelet/pods/674cd4ff-c759-42ff-8b5b-f5e4eb2c45b7/volumes"
Apr 24 14:27:42.451634 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:27:42.451596    2574 generic.go:358] "Generic (PLEG): container finished" podID="ddb5d8a7-6366-44cb-a7d4-58e3627b0187" containerID="dc090e14ce6ba899508244969eca5ba01315fc2c2d0cfaa471ef040fc0db0a9d" exitCode=0
Apr 24 14:27:42.452048 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:27:42.451670    2574 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca-operator/service-ca-operator-d6fc45fc5-nwsrw" event={"ID":"ddb5d8a7-6366-44cb-a7d4-58e3627b0187","Type":"ContainerDied","Data":"dc090e14ce6ba899508244969eca5ba01315fc2c2d0cfaa471ef040fc0db0a9d"}
Apr 24 14:27:42.452048 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:27:42.452023    2574 scope.go:117] "RemoveContainer" containerID="dc090e14ce6ba899508244969eca5ba01315fc2c2d0cfaa471ef040fc0db0a9d"
Apr 24 14:27:43.456220 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:27:43.456184    2574 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca-operator/service-ca-operator-d6fc45fc5-nwsrw" event={"ID":"ddb5d8a7-6366-44cb-a7d4-58e3627b0187","Type":"ContainerStarted","Data":"57a94de22f433324d2633f284df3ad0563c0212b78cea89fb24ed28e403ae60f"}
Apr 24 14:27:47.469754 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:27:47.469715    2574 generic.go:358] "Generic (PLEG): container finished" podID="5bc2f6f9-dff9-48b0-bc63-5f778a1ed590" containerID="b01cb06bbbf923aeac282247b0b055b3c183ba9fc8c128a0fc3f76c9397bf060" exitCode=0
Apr 24 14:27:47.470147 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:27:47.469789    2574 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-6769c5d45-rmc8n" event={"ID":"5bc2f6f9-dff9-48b0-bc63-5f778a1ed590","Type":"ContainerDied","Data":"b01cb06bbbf923aeac282247b0b055b3c183ba9fc8c128a0fc3f76c9397bf060"}
Apr 24 14:27:47.470147 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:27:47.470137    2574 scope.go:117] "RemoveContainer" containerID="b01cb06bbbf923aeac282247b0b055b3c183ba9fc8c128a0fc3f76c9397bf060"
Apr 24 14:27:48.474115 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:27:48.474074    2574 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-6769c5d45-rmc8n" event={"ID":"5bc2f6f9-dff9-48b0-bc63-5f778a1ed590","Type":"ContainerStarted","Data":"89c08158e5b537a1fd242dfa50dd25be669b4ebd57e1b2b0d5018036acbaaf6a"}
Apr 24 14:27:51.531785 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:27:51.531758    2574 kubelet.go:2658] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-monitoring/metrics-server-6cf575bddc-lztrh"
Apr 24 14:27:51.535709 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:27:51.535689    2574 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/metrics-server-6cf575bddc-lztrh"
Apr 24 14:28:09.502299 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:28:09.502260    2574 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/3bc44c73-963f-4a9e-8a52-1e8b031a08dc-metrics-certs\") pod \"network-metrics-daemon-zhkmq\" (UID: \"3bc44c73-963f-4a9e-8a52-1e8b031a08dc\") " pod="openshift-multus/network-metrics-daemon-zhkmq"
Apr 24 14:28:09.504633 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:28:09.504607    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/3bc44c73-963f-4a9e-8a52-1e8b031a08dc-metrics-certs\") pod \"network-metrics-daemon-zhkmq\" (UID: \"3bc44c73-963f-4a9e-8a52-1e8b031a08dc\") " pod="openshift-multus/network-metrics-daemon-zhkmq"
Apr 24 14:28:09.769247 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:28:09.769161    2574 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-nctch\""
Apr 24 14:28:09.777456 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:28:09.777436    2574 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-zhkmq"
Apr 24 14:28:09.895348 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:28:09.895306    2574 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-zhkmq"]
Apr 24 14:28:09.897912 ip-10-0-136-132 kubenswrapper[2574]: W0424 14:28:09.897882    2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3bc44c73_963f_4a9e_8a52_1e8b031a08dc.slice/crio-36ccddd88911bb784a8d25ac6a7d2df120acb3877587a7430289b990ca9d0105 WatchSource:0}: Error finding container 36ccddd88911bb784a8d25ac6a7d2df120acb3877587a7430289b990ca9d0105: Status 404 returned error can't find the container with id 36ccddd88911bb784a8d25ac6a7d2df120acb3877587a7430289b990ca9d0105
Apr 24 14:28:10.541359 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:28:10.541265    2574 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-zhkmq" event={"ID":"3bc44c73-963f-4a9e-8a52-1e8b031a08dc","Type":"ContainerStarted","Data":"36ccddd88911bb784a8d25ac6a7d2df120acb3877587a7430289b990ca9d0105"}
Apr 24 14:28:11.546078 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:28:11.546046    2574 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-zhkmq" event={"ID":"3bc44c73-963f-4a9e-8a52-1e8b031a08dc","Type":"ContainerStarted","Data":"aa8a243b0c2cc30bb8565f4cd0d4fcaf6a5ff8a0a7fdbeaa05737f99f97c8b2a"}
Apr 24 14:28:11.546078 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:28:11.546080    2574 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-zhkmq" event={"ID":"3bc44c73-963f-4a9e-8a52-1e8b031a08dc","Type":"ContainerStarted","Data":"e310f4cdc3d4562ea079f9cc93c4bebb7004d7c8935697da453f960d2030293b"}
Apr 24 14:28:11.561063 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:28:11.561016    2574 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-multus/network-metrics-daemon-zhkmq" podStartSLOduration=253.693238161 podStartE2EDuration="4m14.561002222s" podCreationTimestamp="2026-04-24 14:23:57 +0000 UTC" firstStartedPulling="2026-04-24 14:28:09.899787044 +0000 UTC m=+252.631888674" lastFinishedPulling="2026-04-24 14:28:10.767551108 +0000 UTC m=+253.499652735" observedRunningTime="2026-04-24 14:28:11.559679087 +0000 UTC m=+254.291780758" watchObservedRunningTime="2026-04-24 14:28:11.561002222 +0000 UTC m=+254.293103870"
Apr 24 14:28:31.112549 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:28:31.112509    2574 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["openshift-monitoring/telemeter-client-5cf48867f6-bv6xf"]
Apr 24 14:28:31.113054 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:28:31.112825    2574 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="674cd4ff-c759-42ff-8b5b-f5e4eb2c45b7" containerName="console"
Apr 24 14:28:31.113054 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:28:31.112836    2574 state_mem.go:107] "Deleted CPUSet assignment" podUID="674cd4ff-c759-42ff-8b5b-f5e4eb2c45b7" containerName="console"
Apr 24 14:28:31.113054 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:28:31.112898    2574 memory_manager.go:356] "RemoveStaleState removing state" podUID="674cd4ff-c759-42ff-8b5b-f5e4eb2c45b7" containerName="console"
Apr 24 14:28:31.117987 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:28:31.117970    2574 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/telemeter-client-5cf48867f6-bv6xf"
Apr 24 14:28:31.120307 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:28:31.120283    2574 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"telemeter-client-serving-certs-ca-bundle\""
Apr 24 14:28:31.120468 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:28:31.120446    2574 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"telemeter-client-kube-rbac-proxy-config\""
Apr 24 14:28:31.120540 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:28:31.120468    2574 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"federate-client-certs\""
Apr 24 14:28:31.120540 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:28:31.120480    2574 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"telemeter-client\""
Apr 24 14:28:31.120629 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:28:31.120484    2574 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"telemeter-client-dockercfg-bdtvd\""
Apr 24 14:28:31.120947 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:28:31.120930    2574 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"telemeter-client-tls\""
Apr 24 14:28:31.127090 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:28:31.127072    2574 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"telemeter-trusted-ca-bundle-8i12ta5c71j38\""
Apr 24 14:28:31.133081 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:28:31.133045    2574 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/telemeter-client-5cf48867f6-bv6xf"]
Apr 24 14:28:31.174890 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:28:31.174852    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"telemeter-client-tls\" (UniqueName: \"kubernetes.io/secret/c978078e-58d2-4fb8-bb17-0913013b8d7a-telemeter-client-tls\") pod \"telemeter-client-5cf48867f6-bv6xf\" (UID: \"c978078e-58d2-4fb8-bb17-0913013b8d7a\") " pod="openshift-monitoring/telemeter-client-5cf48867f6-bv6xf"
Apr 24 14:28:31.174890 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:28:31.174889    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"federate-client-tls\" (UniqueName: \"kubernetes.io/secret/c978078e-58d2-4fb8-bb17-0913013b8d7a-federate-client-tls\") pod \"telemeter-client-5cf48867f6-bv6xf\" (UID: \"c978078e-58d2-4fb8-bb17-0913013b8d7a\") " pod="openshift-monitoring/telemeter-client-5cf48867f6-bv6xf"
Apr 24 14:28:31.175147 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:28:31.174921    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/c978078e-58d2-4fb8-bb17-0913013b8d7a-metrics-client-ca\") pod \"telemeter-client-5cf48867f6-bv6xf\" (UID: \"c978078e-58d2-4fb8-bb17-0913013b8d7a\") " pod="openshift-monitoring/telemeter-client-5cf48867f6-bv6xf"
Apr 24 14:28:31.175147 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:28:31.175016    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hsckd\" (UniqueName: \"kubernetes.io/projected/c978078e-58d2-4fb8-bb17-0913013b8d7a-kube-api-access-hsckd\") pod \"telemeter-client-5cf48867f6-bv6xf\" (UID: \"c978078e-58d2-4fb8-bb17-0913013b8d7a\") " pod="openshift-monitoring/telemeter-client-5cf48867f6-bv6xf"
Apr 24 14:28:31.175147 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:28:31.175070    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/c978078e-58d2-4fb8-bb17-0913013b8d7a-serving-certs-ca-bundle\") pod \"telemeter-client-5cf48867f6-bv6xf\" (UID: \"c978078e-58d2-4fb8-bb17-0913013b8d7a\") " pod="openshift-monitoring/telemeter-client-5cf48867f6-bv6xf"
Apr 24 14:28:31.175147 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:28:31.175125    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-telemeter-client\" (UniqueName: \"kubernetes.io/secret/c978078e-58d2-4fb8-bb17-0913013b8d7a-secret-telemeter-client\") pod \"telemeter-client-5cf48867f6-bv6xf\" (UID: \"c978078e-58d2-4fb8-bb17-0913013b8d7a\") " pod="openshift-monitoring/telemeter-client-5cf48867f6-bv6xf"
Apr 24 14:28:31.175319 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:28:31.175158    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-telemeter-client-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/c978078e-58d2-4fb8-bb17-0913013b8d7a-secret-telemeter-client-kube-rbac-proxy-config\") pod \"telemeter-client-5cf48867f6-bv6xf\" (UID: \"c978078e-58d2-4fb8-bb17-0913013b8d7a\") " pod="openshift-monitoring/telemeter-client-5cf48867f6-bv6xf"
Apr 24 14:28:31.175319 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:28:31.175186    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"telemeter-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/c978078e-58d2-4fb8-bb17-0913013b8d7a-telemeter-trusted-ca-bundle\") pod \"telemeter-client-5cf48867f6-bv6xf\" (UID: \"c978078e-58d2-4fb8-bb17-0913013b8d7a\") " pod="openshift-monitoring/telemeter-client-5cf48867f6-bv6xf"
Apr 24 14:28:31.276294 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:28:31.276248    2574 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"secret-telemeter-client\" (UniqueName: \"kubernetes.io/secret/c978078e-58d2-4fb8-bb17-0913013b8d7a-secret-telemeter-client\") pod \"telemeter-client-5cf48867f6-bv6xf\" (UID: \"c978078e-58d2-4fb8-bb17-0913013b8d7a\") " pod="openshift-monitoring/telemeter-client-5cf48867f6-bv6xf"
Apr 24 14:28:31.276294 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:28:31.276296    2574 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"secret-telemeter-client-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/c978078e-58d2-4fb8-bb17-0913013b8d7a-secret-telemeter-client-kube-rbac-proxy-config\") pod \"telemeter-client-5cf48867f6-bv6xf\" (UID: \"c978078e-58d2-4fb8-bb17-0913013b8d7a\") " pod="openshift-monitoring/telemeter-client-5cf48867f6-bv6xf"
Apr 24 14:28:31.276594 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:28:31.276419    2574 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"telemeter-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/c978078e-58d2-4fb8-bb17-0913013b8d7a-telemeter-trusted-ca-bundle\") pod \"telemeter-client-5cf48867f6-bv6xf\" (UID: \"c978078e-58d2-4fb8-bb17-0913013b8d7a\") " pod="openshift-monitoring/telemeter-client-5cf48867f6-bv6xf"
Apr 24 14:28:31.276594 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:28:31.276458    2574 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"telemeter-client-tls\" (UniqueName: \"kubernetes.io/secret/c978078e-58d2-4fb8-bb17-0913013b8d7a-telemeter-client-tls\") pod \"telemeter-client-5cf48867f6-bv6xf\" (UID: \"c978078e-58d2-4fb8-bb17-0913013b8d7a\") " pod="openshift-monitoring/telemeter-client-5cf48867f6-bv6xf"
Apr 24 14:28:31.276594 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:28:31.276485    2574 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"federate-client-tls\" (UniqueName: \"kubernetes.io/secret/c978078e-58d2-4fb8-bb17-0913013b8d7a-federate-client-tls\") pod \"telemeter-client-5cf48867f6-bv6xf\" (UID: \"c978078e-58d2-4fb8-bb17-0913013b8d7a\") " pod="openshift-monitoring/telemeter-client-5cf48867f6-bv6xf"
Apr 24 14:28:31.276594 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:28:31.276549    2574 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/c978078e-58d2-4fb8-bb17-0913013b8d7a-metrics-client-ca\") pod \"telemeter-client-5cf48867f6-bv6xf\" (UID: \"c978078e-58d2-4fb8-bb17-0913013b8d7a\") " pod="openshift-monitoring/telemeter-client-5cf48867f6-bv6xf"
Apr 24 14:28:31.276810 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:28:31.276618    2574 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-hsckd\" (UniqueName: \"kubernetes.io/projected/c978078e-58d2-4fb8-bb17-0913013b8d7a-kube-api-access-hsckd\") pod \"telemeter-client-5cf48867f6-bv6xf\" (UID: \"c978078e-58d2-4fb8-bb17-0913013b8d7a\") " pod="openshift-monitoring/telemeter-client-5cf48867f6-bv6xf"
Apr 24 14:28:31.276810 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:28:31.276659    2574 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/c978078e-58d2-4fb8-bb17-0913013b8d7a-serving-certs-ca-bundle\") pod \"telemeter-client-5cf48867f6-bv6xf\" (UID: \"c978078e-58d2-4fb8-bb17-0913013b8d7a\") " pod="openshift-monitoring/telemeter-client-5cf48867f6-bv6xf"
Apr 24 14:28:31.277406 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:28:31.277377    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"telemeter-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/c978078e-58d2-4fb8-bb17-0913013b8d7a-telemeter-trusted-ca-bundle\") pod \"telemeter-client-5cf48867f6-bv6xf\" (UID: \"c978078e-58d2-4fb8-bb17-0913013b8d7a\") " pod="openshift-monitoring/telemeter-client-5cf48867f6-bv6xf"
Apr 24 14:28:31.277528 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:28:31.277428    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/c978078e-58d2-4fb8-bb17-0913013b8d7a-metrics-client-ca\") pod \"telemeter-client-5cf48867f6-bv6xf\" (UID: \"c978078e-58d2-4fb8-bb17-0913013b8d7a\") " pod="openshift-monitoring/telemeter-client-5cf48867f6-bv6xf"
Apr 24 14:28:31.277528 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:28:31.277495    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/c978078e-58d2-4fb8-bb17-0913013b8d7a-serving-certs-ca-bundle\") pod \"telemeter-client-5cf48867f6-bv6xf\" (UID: \"c978078e-58d2-4fb8-bb17-0913013b8d7a\") " pod="openshift-monitoring/telemeter-client-5cf48867f6-bv6xf"
Apr 24 14:28:31.278986 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:28:31.278962    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"federate-client-tls\" (UniqueName: \"kubernetes.io/secret/c978078e-58d2-4fb8-bb17-0913013b8d7a-federate-client-tls\") pod \"telemeter-client-5cf48867f6-bv6xf\" (UID: \"c978078e-58d2-4fb8-bb17-0913013b8d7a\") " pod="openshift-monitoring/telemeter-client-5cf48867f6-bv6xf"
Apr 24 14:28:31.279132 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:28:31.279111    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-telemeter-client\" (UniqueName: \"kubernetes.io/secret/c978078e-58d2-4fb8-bb17-0913013b8d7a-secret-telemeter-client\") pod \"telemeter-client-5cf48867f6-bv6xf\" (UID: \"c978078e-58d2-4fb8-bb17-0913013b8d7a\") " pod="openshift-monitoring/telemeter-client-5cf48867f6-bv6xf"
Apr 24 14:28:31.279178 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:28:31.279140    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-telemeter-client-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/c978078e-58d2-4fb8-bb17-0913013b8d7a-secret-telemeter-client-kube-rbac-proxy-config\") pod \"telemeter-client-5cf48867f6-bv6xf\" (UID: \"c978078e-58d2-4fb8-bb17-0913013b8d7a\") " pod="openshift-monitoring/telemeter-client-5cf48867f6-bv6xf"
Apr 24 14:28:31.279269 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:28:31.279254    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"telemeter-client-tls\" (UniqueName: \"kubernetes.io/secret/c978078e-58d2-4fb8-bb17-0913013b8d7a-telemeter-client-tls\") pod \"telemeter-client-5cf48867f6-bv6xf\" (UID: \"c978078e-58d2-4fb8-bb17-0913013b8d7a\") " pod="openshift-monitoring/telemeter-client-5cf48867f6-bv6xf"
Apr 24 14:28:31.287969 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:28:31.287944    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-hsckd\" (UniqueName: \"kubernetes.io/projected/c978078e-58d2-4fb8-bb17-0913013b8d7a-kube-api-access-hsckd\") pod \"telemeter-client-5cf48867f6-bv6xf\" (UID: \"c978078e-58d2-4fb8-bb17-0913013b8d7a\") " pod="openshift-monitoring/telemeter-client-5cf48867f6-bv6xf"
Apr 24 14:28:31.428076 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:28:31.428030    2574 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/telemeter-client-5cf48867f6-bv6xf"
Apr 24 14:28:31.556252 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:28:31.556214    2574 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/telemeter-client-5cf48867f6-bv6xf"]
Apr 24 14:28:31.561638 ip-10-0-136-132 kubenswrapper[2574]: W0424 14:28:31.561608    2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc978078e_58d2_4fb8_bb17_0913013b8d7a.slice/crio-068cb2c2c851536e8c5fcc14044d010e8484068d877a2b8382db5f2613485cc0 WatchSource:0}: Error finding container 068cb2c2c851536e8c5fcc14044d010e8484068d877a2b8382db5f2613485cc0: Status 404 returned error can't find the container with id 068cb2c2c851536e8c5fcc14044d010e8484068d877a2b8382db5f2613485cc0
Apr 24 14:28:31.607174 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:28:31.607136    2574 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/telemeter-client-5cf48867f6-bv6xf" event={"ID":"c978078e-58d2-4fb8-bb17-0913013b8d7a","Type":"ContainerStarted","Data":"068cb2c2c851536e8c5fcc14044d010e8484068d877a2b8382db5f2613485cc0"}
Apr 24 14:28:33.615284 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:28:33.615258    2574 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/telemeter-client-5cf48867f6-bv6xf" event={"ID":"c978078e-58d2-4fb8-bb17-0913013b8d7a","Type":"ContainerStarted","Data":"41bc218929c02adb9497ca7e92e0149b0f5f7d620d6999cecd13a8c982db5b95"}
Apr 24 14:28:33.615588 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:28:33.615290    2574 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/telemeter-client-5cf48867f6-bv6xf" event={"ID":"c978078e-58d2-4fb8-bb17-0913013b8d7a","Type":"ContainerStarted","Data":"31ab0aae1343f80eee18c87ce9545a32a82337b5a21fa8f2cd7d2d45c735ecc4"}
Apr 24 14:28:34.620453 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:28:34.620417    2574 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/telemeter-client-5cf48867f6-bv6xf" event={"ID":"c978078e-58d2-4fb8-bb17-0913013b8d7a","Type":"ContainerStarted","Data":"24e381e3c808f1708ad88deb4ab5416486ecf5baf587e40b6f2e96165b22d696"}
Apr 24 14:28:34.641044 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:28:34.640979    2574 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-monitoring/telemeter-client-5cf48867f6-bv6xf" podStartSLOduration=1.769758274 podStartE2EDuration="3.64095856s" podCreationTimestamp="2026-04-24 14:28:31 +0000 UTC" firstStartedPulling="2026-04-24 14:28:31.563361689 +0000 UTC m=+274.295463315" lastFinishedPulling="2026-04-24 14:28:33.434561969 +0000 UTC m=+276.166663601" observedRunningTime="2026-04-24 14:28:34.638936305 +0000 UTC m=+277.371037955" watchObservedRunningTime="2026-04-24 14:28:34.64095856 +0000 UTC m=+277.373060209"
Apr 24 14:28:35.324362 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:28:35.324298    2574 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["openshift-console/console-6dc7fb9bf6-mvxwf"]
Apr 24 14:28:35.327868 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:28:35.327838    2574 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-6dc7fb9bf6-mvxwf"
Apr 24 14:28:35.337818 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:28:35.337792    2574 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-6dc7fb9bf6-mvxwf"]
Apr 24 14:28:35.412098 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:28:35.412066    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/e8f2c70f-87b2-483d-90ed-297890d884d7-console-serving-cert\") pod \"console-6dc7fb9bf6-mvxwf\" (UID: \"e8f2c70f-87b2-483d-90ed-297890d884d7\") " pod="openshift-console/console-6dc7fb9bf6-mvxwf"
Apr 24 14:28:35.412098 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:28:35.412108    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/e8f2c70f-87b2-483d-90ed-297890d884d7-trusted-ca-bundle\") pod \"console-6dc7fb9bf6-mvxwf\" (UID: \"e8f2c70f-87b2-483d-90ed-297890d884d7\") " pod="openshift-console/console-6dc7fb9bf6-mvxwf"
Apr 24 14:28:35.412385 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:28:35.412132    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/e8f2c70f-87b2-483d-90ed-297890d884d7-service-ca\") pod \"console-6dc7fb9bf6-mvxwf\" (UID: \"e8f2c70f-87b2-483d-90ed-297890d884d7\") " pod="openshift-console/console-6dc7fb9bf6-mvxwf"
Apr 24 14:28:35.412385 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:28:35.412232    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/e8f2c70f-87b2-483d-90ed-297890d884d7-console-oauth-config\") pod \"console-6dc7fb9bf6-mvxwf\" (UID: \"e8f2c70f-87b2-483d-90ed-297890d884d7\") " pod="openshift-console/console-6dc7fb9bf6-mvxwf"
Apr 24 14:28:35.412385 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:28:35.412304    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9m8c6\" (UniqueName: \"kubernetes.io/projected/e8f2c70f-87b2-483d-90ed-297890d884d7-kube-api-access-9m8c6\") pod \"console-6dc7fb9bf6-mvxwf\" (UID: \"e8f2c70f-87b2-483d-90ed-297890d884d7\") " pod="openshift-console/console-6dc7fb9bf6-mvxwf"
Apr 24 14:28:35.412385 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:28:35.412371    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/e8f2c70f-87b2-483d-90ed-297890d884d7-console-config\") pod \"console-6dc7fb9bf6-mvxwf\" (UID: \"e8f2c70f-87b2-483d-90ed-297890d884d7\") " pod="openshift-console/console-6dc7fb9bf6-mvxwf"
Apr 24 14:28:35.412555 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:28:35.412410    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/e8f2c70f-87b2-483d-90ed-297890d884d7-oauth-serving-cert\") pod \"console-6dc7fb9bf6-mvxwf\" (UID: \"e8f2c70f-87b2-483d-90ed-297890d884d7\") " pod="openshift-console/console-6dc7fb9bf6-mvxwf"
Apr 24 14:28:35.513402 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:28:35.513328    2574 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/e8f2c70f-87b2-483d-90ed-297890d884d7-console-serving-cert\") pod \"console-6dc7fb9bf6-mvxwf\" (UID: \"e8f2c70f-87b2-483d-90ed-297890d884d7\") " pod="openshift-console/console-6dc7fb9bf6-mvxwf"
Apr 24 14:28:35.513402 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:28:35.513410    2574 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/e8f2c70f-87b2-483d-90ed-297890d884d7-trusted-ca-bundle\") pod \"console-6dc7fb9bf6-mvxwf\" (UID: \"e8f2c70f-87b2-483d-90ed-297890d884d7\") " pod="openshift-console/console-6dc7fb9bf6-mvxwf"
Apr 24 14:28:35.513651 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:28:35.513435    2574 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/e8f2c70f-87b2-483d-90ed-297890d884d7-service-ca\") pod \"console-6dc7fb9bf6-mvxwf\" (UID: \"e8f2c70f-87b2-483d-90ed-297890d884d7\") " pod="openshift-console/console-6dc7fb9bf6-mvxwf"
Apr 24 14:28:35.513651 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:28:35.513464    2574 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/e8f2c70f-87b2-483d-90ed-297890d884d7-console-oauth-config\") pod \"console-6dc7fb9bf6-mvxwf\" (UID: \"e8f2c70f-87b2-483d-90ed-297890d884d7\") " pod="openshift-console/console-6dc7fb9bf6-mvxwf"
Apr 24 14:28:35.513651 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:28:35.513511    2574 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-9m8c6\" (UniqueName: \"kubernetes.io/projected/e8f2c70f-87b2-483d-90ed-297890d884d7-kube-api-access-9m8c6\") pod \"console-6dc7fb9bf6-mvxwf\" (UID: \"e8f2c70f-87b2-483d-90ed-297890d884d7\") " pod="openshift-console/console-6dc7fb9bf6-mvxwf"
Apr 24 14:28:35.513651 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:28:35.513543    2574 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/e8f2c70f-87b2-483d-90ed-297890d884d7-console-config\") pod \"console-6dc7fb9bf6-mvxwf\" (UID: \"e8f2c70f-87b2-483d-90ed-297890d884d7\") " pod="openshift-console/console-6dc7fb9bf6-mvxwf"
Apr 24 14:28:35.513651 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:28:35.513579    2574 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/e8f2c70f-87b2-483d-90ed-297890d884d7-oauth-serving-cert\") pod \"console-6dc7fb9bf6-mvxwf\" (UID: \"e8f2c70f-87b2-483d-90ed-297890d884d7\") " pod="openshift-console/console-6dc7fb9bf6-mvxwf"
Apr 24 14:28:35.514298 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:28:35.514268    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/e8f2c70f-87b2-483d-90ed-297890d884d7-service-ca\") pod \"console-6dc7fb9bf6-mvxwf\" (UID: \"e8f2c70f-87b2-483d-90ed-297890d884d7\") " pod="openshift-console/console-6dc7fb9bf6-mvxwf"
Apr 24 14:28:35.514508 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:28:35.514309    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/e8f2c70f-87b2-483d-90ed-297890d884d7-oauth-serving-cert\") pod \"console-6dc7fb9bf6-mvxwf\" (UID: \"e8f2c70f-87b2-483d-90ed-297890d884d7\") " pod="openshift-console/console-6dc7fb9bf6-mvxwf"
Apr 24 14:28:35.514508 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:28:35.514406    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/e8f2c70f-87b2-483d-90ed-297890d884d7-trusted-ca-bundle\") pod \"console-6dc7fb9bf6-mvxwf\" (UID: \"e8f2c70f-87b2-483d-90ed-297890d884d7\") " pod="openshift-console/console-6dc7fb9bf6-mvxwf"
Apr 24 14:28:35.514508 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:28:35.514426    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/e8f2c70f-87b2-483d-90ed-297890d884d7-console-config\") pod \"console-6dc7fb9bf6-mvxwf\" (UID: \"e8f2c70f-87b2-483d-90ed-297890d884d7\") " pod="openshift-console/console-6dc7fb9bf6-mvxwf"
Apr 24 14:28:35.515898 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:28:35.515873    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/e8f2c70f-87b2-483d-90ed-297890d884d7-console-oauth-config\") pod \"console-6dc7fb9bf6-mvxwf\" (UID: \"e8f2c70f-87b2-483d-90ed-297890d884d7\") " pod="openshift-console/console-6dc7fb9bf6-mvxwf"
Apr 24 14:28:35.515995 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:28:35.515969    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/e8f2c70f-87b2-483d-90ed-297890d884d7-console-serving-cert\") pod \"console-6dc7fb9bf6-mvxwf\" (UID: \"e8f2c70f-87b2-483d-90ed-297890d884d7\") " pod="openshift-console/console-6dc7fb9bf6-mvxwf"
Apr 24 14:28:35.521691 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:28:35.521665    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-9m8c6\" (UniqueName: \"kubernetes.io/projected/e8f2c70f-87b2-483d-90ed-297890d884d7-kube-api-access-9m8c6\") pod \"console-6dc7fb9bf6-mvxwf\" (UID: \"e8f2c70f-87b2-483d-90ed-297890d884d7\") " pod="openshift-console/console-6dc7fb9bf6-mvxwf"
Apr 24 14:28:35.639316 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:28:35.639268    2574 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-6dc7fb9bf6-mvxwf"
Apr 24 14:28:35.760804 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:28:35.760777    2574 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-6dc7fb9bf6-mvxwf"]
Apr 24 14:28:35.763309 ip-10-0-136-132 kubenswrapper[2574]: W0424 14:28:35.763278    2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode8f2c70f_87b2_483d_90ed_297890d884d7.slice/crio-0a9ad52d5da4f91bb227f05c5233b3bb6c2e17e7682f2892e5755baee4d9574e WatchSource:0}: Error finding container 0a9ad52d5da4f91bb227f05c5233b3bb6c2e17e7682f2892e5755baee4d9574e: Status 404 returned error can't find the container with id 0a9ad52d5da4f91bb227f05c5233b3bb6c2e17e7682f2892e5755baee4d9574e
Apr 24 14:28:36.628197 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:28:36.628163    2574 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-6dc7fb9bf6-mvxwf" event={"ID":"e8f2c70f-87b2-483d-90ed-297890d884d7","Type":"ContainerStarted","Data":"d09d1ce03223b2da179f41da58e9c2eda681980c1f227a78413fc869a2ce286f"}
Apr 24 14:28:36.628197 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:28:36.628200    2574 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-6dc7fb9bf6-mvxwf" event={"ID":"e8f2c70f-87b2-483d-90ed-297890d884d7","Type":"ContainerStarted","Data":"0a9ad52d5da4f91bb227f05c5233b3bb6c2e17e7682f2892e5755baee4d9574e"}
Apr 24 14:28:36.645591 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:28:36.645544    2574 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-console/console-6dc7fb9bf6-mvxwf" podStartSLOduration=1.645530125 podStartE2EDuration="1.645530125s" podCreationTimestamp="2026-04-24 14:28:35 +0000 UTC" firstStartedPulling="0001-01-01 00:00:00 +0000 UTC" lastFinishedPulling="0001-01-01 00:00:00 +0000 UTC" observedRunningTime="2026-04-24 14:28:36.643753571 +0000 UTC m=+279.375855221" watchObservedRunningTime="2026-04-24 14:28:36.645530125 +0000 UTC m=+279.377631774"
Apr 24 14:28:45.640296 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:28:45.640239    2574 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-6dc7fb9bf6-mvxwf"
Apr 24 14:28:45.640832 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:28:45.640366    2574 kubelet.go:2658] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-6dc7fb9bf6-mvxwf"
Apr 24 14:28:45.645291 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:28:45.645268    2574 kubelet.go:2658] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-6dc7fb9bf6-mvxwf"
Apr 24 14:28:45.659450 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:28:45.659417    2574 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-6dc7fb9bf6-mvxwf"
Apr 24 14:28:45.711744 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:28:45.711702    2574 kubelet.go:2553] "SyncLoop DELETE" source="api" pods=["openshift-console/console-65485c4fbf-pmgnc"]
Apr 24 14:28:45.828444 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:28:45.828404    2574 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["openshift-console/console-554bd76846-mq6sk"]
Apr 24 14:28:45.831040 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:28:45.831014    2574 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-554bd76846-mq6sk"
Apr 24 14:28:45.843264 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:28:45.843235    2574 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-554bd76846-mq6sk"]
Apr 24 14:28:45.904694 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:28:45.904605    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-x5dcc\" (UniqueName: \"kubernetes.io/projected/26d01817-940e-4b88-8608-22f16a507506-kube-api-access-x5dcc\") pod \"console-554bd76846-mq6sk\" (UID: \"26d01817-940e-4b88-8608-22f16a507506\") " pod="openshift-console/console-554bd76846-mq6sk"
Apr 24 14:28:45.904694 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:28:45.904677    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/26d01817-940e-4b88-8608-22f16a507506-console-oauth-config\") pod \"console-554bd76846-mq6sk\" (UID: \"26d01817-940e-4b88-8608-22f16a507506\") " pod="openshift-console/console-554bd76846-mq6sk"
Apr 24 14:28:45.904880 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:28:45.904715    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/26d01817-940e-4b88-8608-22f16a507506-service-ca\") pod \"console-554bd76846-mq6sk\" (UID: \"26d01817-940e-4b88-8608-22f16a507506\") " pod="openshift-console/console-554bd76846-mq6sk"
Apr 24 14:28:45.904880 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:28:45.904734    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/26d01817-940e-4b88-8608-22f16a507506-console-config\") pod \"console-554bd76846-mq6sk\" (UID: \"26d01817-940e-4b88-8608-22f16a507506\") " pod="openshift-console/console-554bd76846-mq6sk"
Apr 24 14:28:45.904880 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:28:45.904756    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/26d01817-940e-4b88-8608-22f16a507506-console-serving-cert\") pod \"console-554bd76846-mq6sk\" (UID: \"26d01817-940e-4b88-8608-22f16a507506\") " pod="openshift-console/console-554bd76846-mq6sk"
Apr 24 14:28:45.904880 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:28:45.904814    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/26d01817-940e-4b88-8608-22f16a507506-trusted-ca-bundle\") pod \"console-554bd76846-mq6sk\" (UID: \"26d01817-940e-4b88-8608-22f16a507506\") " pod="openshift-console/console-554bd76846-mq6sk"
Apr 24 14:28:45.904880 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:28:45.904839    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/26d01817-940e-4b88-8608-22f16a507506-oauth-serving-cert\") pod \"console-554bd76846-mq6sk\" (UID: \"26d01817-940e-4b88-8608-22f16a507506\") " pod="openshift-console/console-554bd76846-mq6sk"
Apr 24 14:28:46.005953 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:28:46.005913    2574 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-x5dcc\" (UniqueName: \"kubernetes.io/projected/26d01817-940e-4b88-8608-22f16a507506-kube-api-access-x5dcc\") pod \"console-554bd76846-mq6sk\" (UID: \"26d01817-940e-4b88-8608-22f16a507506\") " pod="openshift-console/console-554bd76846-mq6sk"
Apr 24 14:28:46.006076 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:28:46.005983    2574 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/26d01817-940e-4b88-8608-22f16a507506-console-oauth-config\") pod \"console-554bd76846-mq6sk\" (UID: \"26d01817-940e-4b88-8608-22f16a507506\") " pod="openshift-console/console-554bd76846-mq6sk"
Apr 24 14:28:46.006076 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:28:46.006036    2574 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/26d01817-940e-4b88-8608-22f16a507506-service-ca\") pod \"console-554bd76846-mq6sk\" (UID: \"26d01817-940e-4b88-8608-22f16a507506\") " pod="openshift-console/console-554bd76846-mq6sk"
Apr 24 14:28:46.006076 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:28:46.006058    2574 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/26d01817-940e-4b88-8608-22f16a507506-console-config\") pod \"console-554bd76846-mq6sk\" (UID: \"26d01817-940e-4b88-8608-22f16a507506\") " pod="openshift-console/console-554bd76846-mq6sk"
Apr 24 14:28:46.006222 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:28:46.006081    2574 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/26d01817-940e-4b88-8608-22f16a507506-console-serving-cert\") pod \"console-554bd76846-mq6sk\" (UID: \"26d01817-940e-4b88-8608-22f16a507506\") " pod="openshift-console/console-554bd76846-mq6sk"
Apr 24 14:28:46.006222 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:28:46.006105    2574 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/26d01817-940e-4b88-8608-22f16a507506-trusted-ca-bundle\") pod \"console-554bd76846-mq6sk\" (UID: \"26d01817-940e-4b88-8608-22f16a507506\") " pod="openshift-console/console-554bd76846-mq6sk"
Apr 24 14:28:46.006222 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:28:46.006123    2574 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/26d01817-940e-4b88-8608-22f16a507506-oauth-serving-cert\") pod \"console-554bd76846-mq6sk\" (UID: \"26d01817-940e-4b88-8608-22f16a507506\") " pod="openshift-console/console-554bd76846-mq6sk"
Apr 24 14:28:46.006927 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:28:46.006896    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/26d01817-940e-4b88-8608-22f16a507506-service-ca\") pod \"console-554bd76846-mq6sk\" (UID: \"26d01817-940e-4b88-8608-22f16a507506\") " pod="openshift-console/console-554bd76846-mq6sk"
Apr 24 14:28:46.007042 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:28:46.006896    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/26d01817-940e-4b88-8608-22f16a507506-console-config\") pod \"console-554bd76846-mq6sk\" (UID: \"26d01817-940e-4b88-8608-22f16a507506\") " pod="openshift-console/console-554bd76846-mq6sk"
Apr 24 14:28:46.007042 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:28:46.006967    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/26d01817-940e-4b88-8608-22f16a507506-oauth-serving-cert\") pod \"console-554bd76846-mq6sk\" (UID: \"26d01817-940e-4b88-8608-22f16a507506\") " pod="openshift-console/console-554bd76846-mq6sk"
Apr 24 14:28:46.007168 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:28:46.007074    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/26d01817-940e-4b88-8608-22f16a507506-trusted-ca-bundle\") pod \"console-554bd76846-mq6sk\" (UID: \"26d01817-940e-4b88-8608-22f16a507506\") " pod="openshift-console/console-554bd76846-mq6sk"
Apr 24 14:28:46.008478 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:28:46.008459    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/26d01817-940e-4b88-8608-22f16a507506-console-oauth-config\") pod \"console-554bd76846-mq6sk\" (UID: \"26d01817-940e-4b88-8608-22f16a507506\") " pod="openshift-console/console-554bd76846-mq6sk"
Apr 24 14:28:46.008685 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:28:46.008667    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/26d01817-940e-4b88-8608-22f16a507506-console-serving-cert\") pod \"console-554bd76846-mq6sk\" (UID: \"26d01817-940e-4b88-8608-22f16a507506\") " pod="openshift-console/console-554bd76846-mq6sk"
Apr 24 14:28:46.013819 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:28:46.013797    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-x5dcc\" (UniqueName: \"kubernetes.io/projected/26d01817-940e-4b88-8608-22f16a507506-kube-api-access-x5dcc\") pod \"console-554bd76846-mq6sk\" (UID: \"26d01817-940e-4b88-8608-22f16a507506\") " pod="openshift-console/console-554bd76846-mq6sk"
Apr 24 14:28:46.143425 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:28:46.143371    2574 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-554bd76846-mq6sk"
Apr 24 14:28:46.264297 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:28:46.264271    2574 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-554bd76846-mq6sk"]
Apr 24 14:28:46.266823 ip-10-0-136-132 kubenswrapper[2574]: W0424 14:28:46.266781    2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod26d01817_940e_4b88_8608_22f16a507506.slice/crio-84be2b2daef1e7faafd00c96f4ec36c875e98d193c85e7be05f52b1fa032eaaf WatchSource:0}: Error finding container 84be2b2daef1e7faafd00c96f4ec36c875e98d193c85e7be05f52b1fa032eaaf: Status 404 returned error can't find the container with id 84be2b2daef1e7faafd00c96f4ec36c875e98d193c85e7be05f52b1fa032eaaf
Apr 24 14:28:46.660423 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:28:46.660382    2574 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-554bd76846-mq6sk" event={"ID":"26d01817-940e-4b88-8608-22f16a507506","Type":"ContainerStarted","Data":"25768a7c7bebb37f5c591a510780d4644355e8f443cf6539994832d3a92aefd8"}
Apr 24 14:28:46.660803 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:28:46.660430    2574 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-554bd76846-mq6sk" event={"ID":"26d01817-940e-4b88-8608-22f16a507506","Type":"ContainerStarted","Data":"84be2b2daef1e7faafd00c96f4ec36c875e98d193c85e7be05f52b1fa032eaaf"}
Apr 24 14:28:46.676740 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:28:46.676695    2574 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-console/console-554bd76846-mq6sk" podStartSLOduration=1.676680202 podStartE2EDuration="1.676680202s" podCreationTimestamp="2026-04-24 14:28:45 +0000 UTC" firstStartedPulling="0001-01-01 00:00:00 +0000 UTC" lastFinishedPulling="0001-01-01 00:00:00 +0000 UTC" observedRunningTime="2026-04-24 14:28:46.675708779 +0000 UTC m=+289.407810429" watchObservedRunningTime="2026-04-24 14:28:46.676680202 +0000 UTC m=+289.408782047"
Apr 24 14:28:56.144287 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:28:56.144252    2574 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-554bd76846-mq6sk"
Apr 24 14:28:56.144287 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:28:56.144296    2574 kubelet.go:2658] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-554bd76846-mq6sk"
Apr 24 14:28:56.149083 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:28:56.149054    2574 kubelet.go:2658] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-554bd76846-mq6sk"
Apr 24 14:28:56.698864 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:28:56.698833    2574 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-554bd76846-mq6sk"
Apr 24 14:28:56.743704 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:28:56.743673    2574 kubelet.go:2553] "SyncLoop DELETE" source="api" pods=["openshift-console/console-6dc7fb9bf6-mvxwf"]
Apr 24 14:28:57.690850 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:28:57.690823    2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-9d4b6777b-f6ngr_2571829f-666a-422a-9b02-2d2bb4c89b25/console-operator/1.log"
Apr 24 14:28:57.690850 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:28:57.690848    2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-9d4b6777b-f6ngr_2571829f-666a-422a-9b02-2d2bb4c89b25/console-operator/1.log"
Apr 24 14:28:57.696522 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:28:57.696501    2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-w5x5x_44036c3e-1fdc-40d4-899c-9009cd4589e3/ovn-acl-logging/0.log"
Apr 24 14:28:57.696657 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:28:57.696502    2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-w5x5x_44036c3e-1fdc-40d4-899c-9009cd4589e3/ovn-acl-logging/0.log"
Apr 24 14:28:57.702272 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:28:57.702251    2574 kubelet.go:1628] "Image garbage collection succeeded"
Apr 24 14:29:10.732263 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:29:10.732201    2574 kuberuntime_container.go:864] "Killing container with a grace period" pod="openshift-console/console-65485c4fbf-pmgnc" podUID="a9fe99c0-ecfb-4b06-b60a-2dbe93db2d3a" containerName="console" containerID="cri-o://4354ff06b964bd71c3cd36606b7a571c06e6c287f1ea646eb2e0e1c203f52863" gracePeriod=15
Apr 24 14:29:10.970856 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:29:10.970825    2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console_console-65485c4fbf-pmgnc_a9fe99c0-ecfb-4b06-b60a-2dbe93db2d3a/console/0.log"
Apr 24 14:29:10.970977 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:29:10.970885    2574 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-65485c4fbf-pmgnc"
Apr 24 14:29:11.140180 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:29:11.140147    2574 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/a9fe99c0-ecfb-4b06-b60a-2dbe93db2d3a-console-serving-cert\") pod \"a9fe99c0-ecfb-4b06-b60a-2dbe93db2d3a\" (UID: \"a9fe99c0-ecfb-4b06-b60a-2dbe93db2d3a\") "
Apr 24 14:29:11.140390 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:29:11.140200    2574 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/a9fe99c0-ecfb-4b06-b60a-2dbe93db2d3a-trusted-ca-bundle\") pod \"a9fe99c0-ecfb-4b06-b60a-2dbe93db2d3a\" (UID: \"a9fe99c0-ecfb-4b06-b60a-2dbe93db2d3a\") "
Apr 24 14:29:11.140390 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:29:11.140247    2574 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/a9fe99c0-ecfb-4b06-b60a-2dbe93db2d3a-console-oauth-config\") pod \"a9fe99c0-ecfb-4b06-b60a-2dbe93db2d3a\" (UID: \"a9fe99c0-ecfb-4b06-b60a-2dbe93db2d3a\") "
Apr 24 14:29:11.140390 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:29:11.140291    2574 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/a9fe99c0-ecfb-4b06-b60a-2dbe93db2d3a-service-ca\") pod \"a9fe99c0-ecfb-4b06-b60a-2dbe93db2d3a\" (UID: \"a9fe99c0-ecfb-4b06-b60a-2dbe93db2d3a\") "
Apr 24 14:29:11.140390 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:29:11.140380    2574 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"kube-api-access-t7b7r\" (UniqueName: \"kubernetes.io/projected/a9fe99c0-ecfb-4b06-b60a-2dbe93db2d3a-kube-api-access-t7b7r\") pod \"a9fe99c0-ecfb-4b06-b60a-2dbe93db2d3a\" (UID: \"a9fe99c0-ecfb-4b06-b60a-2dbe93db2d3a\") "
Apr 24 14:29:11.140594 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:29:11.140409    2574 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/a9fe99c0-ecfb-4b06-b60a-2dbe93db2d3a-oauth-serving-cert\") pod \"a9fe99c0-ecfb-4b06-b60a-2dbe93db2d3a\" (UID: \"a9fe99c0-ecfb-4b06-b60a-2dbe93db2d3a\") "
Apr 24 14:29:11.140594 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:29:11.140436    2574 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/a9fe99c0-ecfb-4b06-b60a-2dbe93db2d3a-console-config\") pod \"a9fe99c0-ecfb-4b06-b60a-2dbe93db2d3a\" (UID: \"a9fe99c0-ecfb-4b06-b60a-2dbe93db2d3a\") "
Apr 24 14:29:11.140973 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:29:11.140739    2574 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/a9fe99c0-ecfb-4b06-b60a-2dbe93db2d3a-service-ca" (OuterVolumeSpecName: "service-ca") pod "a9fe99c0-ecfb-4b06-b60a-2dbe93db2d3a" (UID: "a9fe99c0-ecfb-4b06-b60a-2dbe93db2d3a"). InnerVolumeSpecName "service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Apr 24 14:29:11.140973 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:29:11.140745    2574 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/a9fe99c0-ecfb-4b06-b60a-2dbe93db2d3a-trusted-ca-bundle" (OuterVolumeSpecName: "trusted-ca-bundle") pod "a9fe99c0-ecfb-4b06-b60a-2dbe93db2d3a" (UID: "a9fe99c0-ecfb-4b06-b60a-2dbe93db2d3a"). InnerVolumeSpecName "trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Apr 24 14:29:11.140973 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:29:11.140937    2574 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/a9fe99c0-ecfb-4b06-b60a-2dbe93db2d3a-oauth-serving-cert" (OuterVolumeSpecName: "oauth-serving-cert") pod "a9fe99c0-ecfb-4b06-b60a-2dbe93db2d3a" (UID: "a9fe99c0-ecfb-4b06-b60a-2dbe93db2d3a"). InnerVolumeSpecName "oauth-serving-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Apr 24 14:29:11.140973 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:29:11.140972    2574 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/a9fe99c0-ecfb-4b06-b60a-2dbe93db2d3a-console-config" (OuterVolumeSpecName: "console-config") pod "a9fe99c0-ecfb-4b06-b60a-2dbe93db2d3a" (UID: "a9fe99c0-ecfb-4b06-b60a-2dbe93db2d3a"). InnerVolumeSpecName "console-config". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Apr 24 14:29:11.142427 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:29:11.142404    2574 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/a9fe99c0-ecfb-4b06-b60a-2dbe93db2d3a-console-serving-cert" (OuterVolumeSpecName: "console-serving-cert") pod "a9fe99c0-ecfb-4b06-b60a-2dbe93db2d3a" (UID: "a9fe99c0-ecfb-4b06-b60a-2dbe93db2d3a"). InnerVolumeSpecName "console-serving-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Apr 24 14:29:11.142427 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:29:11.142416    2574 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a9fe99c0-ecfb-4b06-b60a-2dbe93db2d3a-kube-api-access-t7b7r" (OuterVolumeSpecName: "kube-api-access-t7b7r") pod "a9fe99c0-ecfb-4b06-b60a-2dbe93db2d3a" (UID: "a9fe99c0-ecfb-4b06-b60a-2dbe93db2d3a"). InnerVolumeSpecName "kube-api-access-t7b7r". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Apr 24 14:29:11.142538 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:29:11.142517    2574 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/a9fe99c0-ecfb-4b06-b60a-2dbe93db2d3a-console-oauth-config" (OuterVolumeSpecName: "console-oauth-config") pod "a9fe99c0-ecfb-4b06-b60a-2dbe93db2d3a" (UID: "a9fe99c0-ecfb-4b06-b60a-2dbe93db2d3a"). InnerVolumeSpecName "console-oauth-config". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Apr 24 14:29:11.241882 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:29:11.241854    2574 reconciler_common.go:299] "Volume detached for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/a9fe99c0-ecfb-4b06-b60a-2dbe93db2d3a-console-serving-cert\") on node \"ip-10-0-136-132.ec2.internal\" DevicePath \"\""
Apr 24 14:29:11.241882 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:29:11.241879    2574 reconciler_common.go:299] "Volume detached for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/a9fe99c0-ecfb-4b06-b60a-2dbe93db2d3a-trusted-ca-bundle\") on node \"ip-10-0-136-132.ec2.internal\" DevicePath \"\""
Apr 24 14:29:11.242027 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:29:11.241891    2574 reconciler_common.go:299] "Volume detached for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/a9fe99c0-ecfb-4b06-b60a-2dbe93db2d3a-console-oauth-config\") on node \"ip-10-0-136-132.ec2.internal\" DevicePath \"\""
Apr 24 14:29:11.242027 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:29:11.241900    2574 reconciler_common.go:299] "Volume detached for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/a9fe99c0-ecfb-4b06-b60a-2dbe93db2d3a-service-ca\") on node \"ip-10-0-136-132.ec2.internal\" DevicePath \"\""
Apr 24 14:29:11.242027 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:29:11.241909    2574 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-t7b7r\" (UniqueName: \"kubernetes.io/projected/a9fe99c0-ecfb-4b06-b60a-2dbe93db2d3a-kube-api-access-t7b7r\") on node \"ip-10-0-136-132.ec2.internal\" DevicePath \"\""
Apr 24 14:29:11.242027 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:29:11.241917    2574 reconciler_common.go:299] "Volume detached for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/a9fe99c0-ecfb-4b06-b60a-2dbe93db2d3a-oauth-serving-cert\") on node \"ip-10-0-136-132.ec2.internal\" DevicePath \"\""
Apr 24 14:29:11.242027 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:29:11.241927    2574 reconciler_common.go:299] "Volume detached for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/a9fe99c0-ecfb-4b06-b60a-2dbe93db2d3a-console-config\") on node \"ip-10-0-136-132.ec2.internal\" DevicePath \"\""
Apr 24 14:29:11.746005 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:29:11.745977    2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console_console-65485c4fbf-pmgnc_a9fe99c0-ecfb-4b06-b60a-2dbe93db2d3a/console/0.log"
Apr 24 14:29:11.746436 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:29:11.746016    2574 generic.go:358] "Generic (PLEG): container finished" podID="a9fe99c0-ecfb-4b06-b60a-2dbe93db2d3a" containerID="4354ff06b964bd71c3cd36606b7a571c06e6c287f1ea646eb2e0e1c203f52863" exitCode=2
Apr 24 14:29:11.746436 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:29:11.746059    2574 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-65485c4fbf-pmgnc" event={"ID":"a9fe99c0-ecfb-4b06-b60a-2dbe93db2d3a","Type":"ContainerDied","Data":"4354ff06b964bd71c3cd36606b7a571c06e6c287f1ea646eb2e0e1c203f52863"}
Apr 24 14:29:11.746436 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:29:11.746081    2574 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-65485c4fbf-pmgnc" event={"ID":"a9fe99c0-ecfb-4b06-b60a-2dbe93db2d3a","Type":"ContainerDied","Data":"994bdd2916f2518b776bbc09038a4375554d0f8eb443034b11bd7e76c6f0b98c"}
Apr 24 14:29:11.746436 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:29:11.746085    2574 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-65485c4fbf-pmgnc"
Apr 24 14:29:11.746436 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:29:11.746095    2574 scope.go:117] "RemoveContainer" containerID="4354ff06b964bd71c3cd36606b7a571c06e6c287f1ea646eb2e0e1c203f52863"
Apr 24 14:29:11.755103 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:29:11.755082    2574 scope.go:117] "RemoveContainer" containerID="4354ff06b964bd71c3cd36606b7a571c06e6c287f1ea646eb2e0e1c203f52863"
Apr 24 14:29:11.755374 ip-10-0-136-132 kubenswrapper[2574]: E0424 14:29:11.755353    2574 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"4354ff06b964bd71c3cd36606b7a571c06e6c287f1ea646eb2e0e1c203f52863\": container with ID starting with 4354ff06b964bd71c3cd36606b7a571c06e6c287f1ea646eb2e0e1c203f52863 not found: ID does not exist" containerID="4354ff06b964bd71c3cd36606b7a571c06e6c287f1ea646eb2e0e1c203f52863"
Apr 24 14:29:11.755444 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:29:11.755388    2574 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"4354ff06b964bd71c3cd36606b7a571c06e6c287f1ea646eb2e0e1c203f52863"} err="failed to get container status \"4354ff06b964bd71c3cd36606b7a571c06e6c287f1ea646eb2e0e1c203f52863\": rpc error: code = NotFound desc = could not find container \"4354ff06b964bd71c3cd36606b7a571c06e6c287f1ea646eb2e0e1c203f52863\": container with ID starting with 4354ff06b964bd71c3cd36606b7a571c06e6c287f1ea646eb2e0e1c203f52863 not found: ID does not exist"
Apr 24 14:29:11.769457 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:29:11.769436    2574 kubelet.go:2553] "SyncLoop DELETE" source="api" pods=["openshift-console/console-65485c4fbf-pmgnc"]
Apr 24 14:29:11.774289 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:29:11.774269    2574 kubelet.go:2547] "SyncLoop REMOVE" source="api" pods=["openshift-console/console-65485c4fbf-pmgnc"]
Apr 24 14:29:13.769990 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:29:13.769947    2574 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="a9fe99c0-ecfb-4b06-b60a-2dbe93db2d3a" path="/var/lib/kubelet/pods/a9fe99c0-ecfb-4b06-b60a-2dbe93db2d3a/volumes"
Apr 24 14:29:21.767643 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:29:21.767603    2574 kuberuntime_container.go:864] "Killing container with a grace period" pod="openshift-console/console-6dc7fb9bf6-mvxwf" podUID="e8f2c70f-87b2-483d-90ed-297890d884d7" containerName="console" containerID="cri-o://d09d1ce03223b2da179f41da58e9c2eda681980c1f227a78413fc869a2ce286f" gracePeriod=15
Apr 24 14:29:22.007474 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:29:22.007450    2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console_console-6dc7fb9bf6-mvxwf_e8f2c70f-87b2-483d-90ed-297890d884d7/console/0.log"
Apr 24 14:29:22.007606 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:29:22.007511    2574 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-6dc7fb9bf6-mvxwf"
Apr 24 14:29:22.020618 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:29:22.020533    2574 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/e8f2c70f-87b2-483d-90ed-297890d884d7-service-ca\") pod \"e8f2c70f-87b2-483d-90ed-297890d884d7\" (UID: \"e8f2c70f-87b2-483d-90ed-297890d884d7\") "
Apr 24 14:29:22.020618 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:29:22.020589    2574 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/e8f2c70f-87b2-483d-90ed-297890d884d7-console-oauth-config\") pod \"e8f2c70f-87b2-483d-90ed-297890d884d7\" (UID: \"e8f2c70f-87b2-483d-90ed-297890d884d7\") "
Apr 24 14:29:22.020836 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:29:22.020633    2574 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/e8f2c70f-87b2-483d-90ed-297890d884d7-console-serving-cert\") pod \"e8f2c70f-87b2-483d-90ed-297890d884d7\" (UID: \"e8f2c70f-87b2-483d-90ed-297890d884d7\") "
Apr 24 14:29:22.020836 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:29:22.020657    2574 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/e8f2c70f-87b2-483d-90ed-297890d884d7-oauth-serving-cert\") pod \"e8f2c70f-87b2-483d-90ed-297890d884d7\" (UID: \"e8f2c70f-87b2-483d-90ed-297890d884d7\") "
Apr 24 14:29:22.020836 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:29:22.020677    2574 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/e8f2c70f-87b2-483d-90ed-297890d884d7-console-config\") pod \"e8f2c70f-87b2-483d-90ed-297890d884d7\" (UID: \"e8f2c70f-87b2-483d-90ed-297890d884d7\") "
Apr 24 14:29:22.020836 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:29:22.020716    2574 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/e8f2c70f-87b2-483d-90ed-297890d884d7-trusted-ca-bundle\") pod \"e8f2c70f-87b2-483d-90ed-297890d884d7\" (UID: \"e8f2c70f-87b2-483d-90ed-297890d884d7\") "
Apr 24 14:29:22.020836 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:29:22.020744    2574 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"kube-api-access-9m8c6\" (UniqueName: \"kubernetes.io/projected/e8f2c70f-87b2-483d-90ed-297890d884d7-kube-api-access-9m8c6\") pod \"e8f2c70f-87b2-483d-90ed-297890d884d7\" (UID: \"e8f2c70f-87b2-483d-90ed-297890d884d7\") "
Apr 24 14:29:22.021074 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:29:22.021028    2574 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/e8f2c70f-87b2-483d-90ed-297890d884d7-service-ca" (OuterVolumeSpecName: "service-ca") pod "e8f2c70f-87b2-483d-90ed-297890d884d7" (UID: "e8f2c70f-87b2-483d-90ed-297890d884d7"). InnerVolumeSpecName "service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Apr 24 14:29:22.021124 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:29:22.021093    2574 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/e8f2c70f-87b2-483d-90ed-297890d884d7-console-config" (OuterVolumeSpecName: "console-config") pod "e8f2c70f-87b2-483d-90ed-297890d884d7" (UID: "e8f2c70f-87b2-483d-90ed-297890d884d7"). InnerVolumeSpecName "console-config". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Apr 24 14:29:22.021167 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:29:22.021117    2574 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/e8f2c70f-87b2-483d-90ed-297890d884d7-oauth-serving-cert" (OuterVolumeSpecName: "oauth-serving-cert") pod "e8f2c70f-87b2-483d-90ed-297890d884d7" (UID: "e8f2c70f-87b2-483d-90ed-297890d884d7"). InnerVolumeSpecName "oauth-serving-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Apr 24 14:29:22.021212 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:29:22.021191    2574 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/e8f2c70f-87b2-483d-90ed-297890d884d7-trusted-ca-bundle" (OuterVolumeSpecName: "trusted-ca-bundle") pod "e8f2c70f-87b2-483d-90ed-297890d884d7" (UID: "e8f2c70f-87b2-483d-90ed-297890d884d7"). InnerVolumeSpecName "trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Apr 24 14:29:22.022837 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:29:22.022812    2574 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/e8f2c70f-87b2-483d-90ed-297890d884d7-console-serving-cert" (OuterVolumeSpecName: "console-serving-cert") pod "e8f2c70f-87b2-483d-90ed-297890d884d7" (UID: "e8f2c70f-87b2-483d-90ed-297890d884d7"). InnerVolumeSpecName "console-serving-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Apr 24 14:29:22.022988 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:29:22.022967    2574 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e8f2c70f-87b2-483d-90ed-297890d884d7-kube-api-access-9m8c6" (OuterVolumeSpecName: "kube-api-access-9m8c6") pod "e8f2c70f-87b2-483d-90ed-297890d884d7" (UID: "e8f2c70f-87b2-483d-90ed-297890d884d7"). InnerVolumeSpecName "kube-api-access-9m8c6". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Apr 24 14:29:22.023031 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:29:22.022991    2574 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/e8f2c70f-87b2-483d-90ed-297890d884d7-console-oauth-config" (OuterVolumeSpecName: "console-oauth-config") pod "e8f2c70f-87b2-483d-90ed-297890d884d7" (UID: "e8f2c70f-87b2-483d-90ed-297890d884d7"). InnerVolumeSpecName "console-oauth-config". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Apr 24 14:29:22.122168 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:29:22.122126    2574 reconciler_common.go:299] "Volume detached for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/e8f2c70f-87b2-483d-90ed-297890d884d7-oauth-serving-cert\") on node \"ip-10-0-136-132.ec2.internal\" DevicePath \"\""
Apr 24 14:29:22.122168 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:29:22.122160    2574 reconciler_common.go:299] "Volume detached for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/e8f2c70f-87b2-483d-90ed-297890d884d7-console-config\") on node \"ip-10-0-136-132.ec2.internal\" DevicePath \"\""
Apr 24 14:29:22.122168 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:29:22.122173    2574 reconciler_common.go:299] "Volume detached for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/e8f2c70f-87b2-483d-90ed-297890d884d7-trusted-ca-bundle\") on node \"ip-10-0-136-132.ec2.internal\" DevicePath \"\""
Apr 24 14:29:22.122457 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:29:22.122185    2574 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-9m8c6\" (UniqueName: \"kubernetes.io/projected/e8f2c70f-87b2-483d-90ed-297890d884d7-kube-api-access-9m8c6\") on node \"ip-10-0-136-132.ec2.internal\" DevicePath \"\""
Apr 24 14:29:22.122457 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:29:22.122206    2574 reconciler_common.go:299] "Volume detached for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/e8f2c70f-87b2-483d-90ed-297890d884d7-service-ca\") on node \"ip-10-0-136-132.ec2.internal\" DevicePath \"\""
Apr 24 14:29:22.122457 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:29:22.122219    2574 reconciler_common.go:299] "Volume detached for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/e8f2c70f-87b2-483d-90ed-297890d884d7-console-oauth-config\") on node \"ip-10-0-136-132.ec2.internal\" DevicePath \"\""
Apr 24 14:29:22.122457 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:29:22.122235    2574 reconciler_common.go:299] "Volume detached for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/e8f2c70f-87b2-483d-90ed-297890d884d7-console-serving-cert\") on node \"ip-10-0-136-132.ec2.internal\" DevicePath \"\""
Apr 24 14:29:22.784243 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:29:22.784217    2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console_console-6dc7fb9bf6-mvxwf_e8f2c70f-87b2-483d-90ed-297890d884d7/console/0.log"
Apr 24 14:29:22.784676 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:29:22.784255    2574 generic.go:358] "Generic (PLEG): container finished" podID="e8f2c70f-87b2-483d-90ed-297890d884d7" containerID="d09d1ce03223b2da179f41da58e9c2eda681980c1f227a78413fc869a2ce286f" exitCode=2
Apr 24 14:29:22.784676 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:29:22.784290    2574 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-6dc7fb9bf6-mvxwf" event={"ID":"e8f2c70f-87b2-483d-90ed-297890d884d7","Type":"ContainerDied","Data":"d09d1ce03223b2da179f41da58e9c2eda681980c1f227a78413fc869a2ce286f"}
Apr 24 14:29:22.784676 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:29:22.784322    2574 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-6dc7fb9bf6-mvxwf"
Apr 24 14:29:22.784676 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:29:22.784357    2574 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-6dc7fb9bf6-mvxwf" event={"ID":"e8f2c70f-87b2-483d-90ed-297890d884d7","Type":"ContainerDied","Data":"0a9ad52d5da4f91bb227f05c5233b3bb6c2e17e7682f2892e5755baee4d9574e"}
Apr 24 14:29:22.784676 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:29:22.784375    2574 scope.go:117] "RemoveContainer" containerID="d09d1ce03223b2da179f41da58e9c2eda681980c1f227a78413fc869a2ce286f"
Apr 24 14:29:22.798180 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:29:22.798159    2574 scope.go:117] "RemoveContainer" containerID="d09d1ce03223b2da179f41da58e9c2eda681980c1f227a78413fc869a2ce286f"
Apr 24 14:29:22.798525 ip-10-0-136-132 kubenswrapper[2574]: E0424 14:29:22.798506    2574 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"d09d1ce03223b2da179f41da58e9c2eda681980c1f227a78413fc869a2ce286f\": container with ID starting with d09d1ce03223b2da179f41da58e9c2eda681980c1f227a78413fc869a2ce286f not found: ID does not exist" containerID="d09d1ce03223b2da179f41da58e9c2eda681980c1f227a78413fc869a2ce286f"
Apr 24 14:29:22.798616 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:29:22.798536    2574 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d09d1ce03223b2da179f41da58e9c2eda681980c1f227a78413fc869a2ce286f"} err="failed to get container status \"d09d1ce03223b2da179f41da58e9c2eda681980c1f227a78413fc869a2ce286f\": rpc error: code = NotFound desc = could not find container \"d09d1ce03223b2da179f41da58e9c2eda681980c1f227a78413fc869a2ce286f\": container with ID starting with d09d1ce03223b2da179f41da58e9c2eda681980c1f227a78413fc869a2ce286f not found: ID does not exist"
Apr 24 14:29:22.807324 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:29:22.807299    2574 kubelet.go:2553] "SyncLoop DELETE" source="api" pods=["openshift-console/console-6dc7fb9bf6-mvxwf"]
Apr 24 14:29:22.811324 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:29:22.811300    2574 kubelet.go:2547] "SyncLoop REMOVE" source="api" pods=["openshift-console/console-6dc7fb9bf6-mvxwf"]
Apr 24 14:29:23.769944 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:29:23.769905    2574 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="e8f2c70f-87b2-483d-90ed-297890d884d7" path="/var/lib/kubelet/pods/e8f2c70f-87b2-483d-90ed-297890d884d7/volumes"
Apr 24 14:29:47.840091 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:29:47.840013    2574 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["openshift-marketplace/59039e319e11338a40c6b6f1054d265f40bb50ceac6068d5c59955d29c4nlx4"]
Apr 24 14:29:47.840554 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:29:47.840380    2574 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e8f2c70f-87b2-483d-90ed-297890d884d7" containerName="console"
Apr 24 14:29:47.840554 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:29:47.840393    2574 state_mem.go:107] "Deleted CPUSet assignment" podUID="e8f2c70f-87b2-483d-90ed-297890d884d7" containerName="console"
Apr 24 14:29:47.840554 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:29:47.840416    2574 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a9fe99c0-ecfb-4b06-b60a-2dbe93db2d3a" containerName="console"
Apr 24 14:29:47.840554 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:29:47.840421    2574 state_mem.go:107] "Deleted CPUSet assignment" podUID="a9fe99c0-ecfb-4b06-b60a-2dbe93db2d3a" containerName="console"
Apr 24 14:29:47.840554 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:29:47.840471    2574 memory_manager.go:356] "RemoveStaleState removing state" podUID="a9fe99c0-ecfb-4b06-b60a-2dbe93db2d3a" containerName="console"
Apr 24 14:29:47.840554 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:29:47.840480    2574 memory_manager.go:356] "RemoveStaleState removing state" podUID="e8f2c70f-87b2-483d-90ed-297890d884d7" containerName="console"
Apr 24 14:29:47.843581 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:29:47.843563    2574 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/59039e319e11338a40c6b6f1054d265f40bb50ceac6068d5c59955d29c4nlx4"
Apr 24 14:29:47.845563 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:29:47.845538    2574 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"openshift-service-ca.crt\""
Apr 24 14:29:47.845673 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:29:47.845538    2574 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"kube-root-ca.crt\""
Apr 24 14:29:47.846004 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:29:47.845990    2574 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-marketplace\"/\"default-dockercfg-dnr8q\""
Apr 24 14:29:47.851014 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:29:47.850992    2574 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/59039e319e11338a40c6b6f1054d265f40bb50ceac6068d5c59955d29c4nlx4"]
Apr 24 14:29:47.949081 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:29:47.949043    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/21bae6ac-1707-4e61-a4b1-055bd7deeb4a-util\") pod \"59039e319e11338a40c6b6f1054d265f40bb50ceac6068d5c59955d29c4nlx4\" (UID: \"21bae6ac-1707-4e61-a4b1-055bd7deeb4a\") " pod="openshift-marketplace/59039e319e11338a40c6b6f1054d265f40bb50ceac6068d5c59955d29c4nlx4"
Apr 24 14:29:47.949255 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:29:47.949149    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/21bae6ac-1707-4e61-a4b1-055bd7deeb4a-bundle\") pod \"59039e319e11338a40c6b6f1054d265f40bb50ceac6068d5c59955d29c4nlx4\" (UID: \"21bae6ac-1707-4e61-a4b1-055bd7deeb4a\") " pod="openshift-marketplace/59039e319e11338a40c6b6f1054d265f40bb50ceac6068d5c59955d29c4nlx4"
Apr 24 14:29:47.949255 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:29:47.949197    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rmt9s\" (UniqueName: \"kubernetes.io/projected/21bae6ac-1707-4e61-a4b1-055bd7deeb4a-kube-api-access-rmt9s\") pod \"59039e319e11338a40c6b6f1054d265f40bb50ceac6068d5c59955d29c4nlx4\" (UID: \"21bae6ac-1707-4e61-a4b1-055bd7deeb4a\") " pod="openshift-marketplace/59039e319e11338a40c6b6f1054d265f40bb50ceac6068d5c59955d29c4nlx4"
Apr 24 14:29:48.049856 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:29:48.049811    2574 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/21bae6ac-1707-4e61-a4b1-055bd7deeb4a-util\") pod \"59039e319e11338a40c6b6f1054d265f40bb50ceac6068d5c59955d29c4nlx4\" (UID: \"21bae6ac-1707-4e61-a4b1-055bd7deeb4a\") " pod="openshift-marketplace/59039e319e11338a40c6b6f1054d265f40bb50ceac6068d5c59955d29c4nlx4"
Apr 24 14:29:48.050044 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:29:48.049904    2574 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/21bae6ac-1707-4e61-a4b1-055bd7deeb4a-bundle\") pod \"59039e319e11338a40c6b6f1054d265f40bb50ceac6068d5c59955d29c4nlx4\" (UID: \"21bae6ac-1707-4e61-a4b1-055bd7deeb4a\") " pod="openshift-marketplace/59039e319e11338a40c6b6f1054d265f40bb50ceac6068d5c59955d29c4nlx4"
Apr 24 14:29:48.050044 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:29:48.049967    2574 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-rmt9s\" (UniqueName: \"kubernetes.io/projected/21bae6ac-1707-4e61-a4b1-055bd7deeb4a-kube-api-access-rmt9s\") pod \"59039e319e11338a40c6b6f1054d265f40bb50ceac6068d5c59955d29c4nlx4\" (UID: \"21bae6ac-1707-4e61-a4b1-055bd7deeb4a\") " pod="openshift-marketplace/59039e319e11338a40c6b6f1054d265f40bb50ceac6068d5c59955d29c4nlx4"
Apr 24 14:29:48.050273 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:29:48.050252    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/21bae6ac-1707-4e61-a4b1-055bd7deeb4a-util\") pod \"59039e319e11338a40c6b6f1054d265f40bb50ceac6068d5c59955d29c4nlx4\" (UID: \"21bae6ac-1707-4e61-a4b1-055bd7deeb4a\") " pod="openshift-marketplace/59039e319e11338a40c6b6f1054d265f40bb50ceac6068d5c59955d29c4nlx4"
Apr 24 14:29:48.050378 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:29:48.050305    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/21bae6ac-1707-4e61-a4b1-055bd7deeb4a-bundle\") pod \"59039e319e11338a40c6b6f1054d265f40bb50ceac6068d5c59955d29c4nlx4\" (UID: \"21bae6ac-1707-4e61-a4b1-055bd7deeb4a\") " pod="openshift-marketplace/59039e319e11338a40c6b6f1054d265f40bb50ceac6068d5c59955d29c4nlx4"
Apr 24 14:29:48.058199 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:29:48.058181    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-rmt9s\" (UniqueName: \"kubernetes.io/projected/21bae6ac-1707-4e61-a4b1-055bd7deeb4a-kube-api-access-rmt9s\") pod \"59039e319e11338a40c6b6f1054d265f40bb50ceac6068d5c59955d29c4nlx4\" (UID: \"21bae6ac-1707-4e61-a4b1-055bd7deeb4a\") " pod="openshift-marketplace/59039e319e11338a40c6b6f1054d265f40bb50ceac6068d5c59955d29c4nlx4"
Apr 24 14:29:48.153285 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:29:48.153240    2574 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/59039e319e11338a40c6b6f1054d265f40bb50ceac6068d5c59955d29c4nlx4"
Apr 24 14:29:48.272043 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:29:48.271990    2574 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/59039e319e11338a40c6b6f1054d265f40bb50ceac6068d5c59955d29c4nlx4"]
Apr 24 14:29:48.274676 ip-10-0-136-132 kubenswrapper[2574]: W0424 14:29:48.274643    2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod21bae6ac_1707_4e61_a4b1_055bd7deeb4a.slice/crio-09c1f341eacf7ac1b5d77596971f33c20a8f363e6e0487a89dbe5914cc9b2a0e WatchSource:0}: Error finding container 09c1f341eacf7ac1b5d77596971f33c20a8f363e6e0487a89dbe5914cc9b2a0e: Status 404 returned error can't find the container with id 09c1f341eacf7ac1b5d77596971f33c20a8f363e6e0487a89dbe5914cc9b2a0e
Apr 24 14:29:48.276473 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:29:48.276452    2574 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Apr 24 14:29:48.876064 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:29:48.876026    2574 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/59039e319e11338a40c6b6f1054d265f40bb50ceac6068d5c59955d29c4nlx4" event={"ID":"21bae6ac-1707-4e61-a4b1-055bd7deeb4a","Type":"ContainerStarted","Data":"09c1f341eacf7ac1b5d77596971f33c20a8f363e6e0487a89dbe5914cc9b2a0e"}
Apr 24 14:29:53.894451 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:29:53.894415    2574 generic.go:358] "Generic (PLEG): container finished" podID="21bae6ac-1707-4e61-a4b1-055bd7deeb4a" containerID="9540d0a4833eea258e1c4c7d5b9eb5e421ad4506863747ed0585e1cdc45f58e0" exitCode=0
Apr 24 14:29:53.894927 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:29:53.894479    2574 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/59039e319e11338a40c6b6f1054d265f40bb50ceac6068d5c59955d29c4nlx4" event={"ID":"21bae6ac-1707-4e61-a4b1-055bd7deeb4a","Type":"ContainerDied","Data":"9540d0a4833eea258e1c4c7d5b9eb5e421ad4506863747ed0585e1cdc45f58e0"}
Apr 24 14:29:56.909174 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:29:56.909139    2574 generic.go:358] "Generic (PLEG): container finished" podID="21bae6ac-1707-4e61-a4b1-055bd7deeb4a" containerID="3bd0b035febf48337fc5dd3a5c1b5a159b7948c534a920d533b31db9d50b5d1e" exitCode=0
Apr 24 14:29:56.909578 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:29:56.909234    2574 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/59039e319e11338a40c6b6f1054d265f40bb50ceac6068d5c59955d29c4nlx4" event={"ID":"21bae6ac-1707-4e61-a4b1-055bd7deeb4a","Type":"ContainerDied","Data":"3bd0b035febf48337fc5dd3a5c1b5a159b7948c534a920d533b31db9d50b5d1e"}
Apr 24 14:30:03.935816 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:30:03.935781    2574 generic.go:358] "Generic (PLEG): container finished" podID="21bae6ac-1707-4e61-a4b1-055bd7deeb4a" containerID="162dc2272706a547ad108be1ed5b1084c28438b78be95dd357066c56bd3c9066" exitCode=0
Apr 24 14:30:03.936254 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:30:03.935874    2574 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/59039e319e11338a40c6b6f1054d265f40bb50ceac6068d5c59955d29c4nlx4" event={"ID":"21bae6ac-1707-4e61-a4b1-055bd7deeb4a","Type":"ContainerDied","Data":"162dc2272706a547ad108be1ed5b1084c28438b78be95dd357066c56bd3c9066"}
Apr 24 14:30:05.060065 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:30:05.060036    2574 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/59039e319e11338a40c6b6f1054d265f40bb50ceac6068d5c59955d29c4nlx4"
Apr 24 14:30:05.101838 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:30:05.101801    2574 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/21bae6ac-1707-4e61-a4b1-055bd7deeb4a-util\") pod \"21bae6ac-1707-4e61-a4b1-055bd7deeb4a\" (UID: \"21bae6ac-1707-4e61-a4b1-055bd7deeb4a\") "
Apr 24 14:30:05.101997 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:30:05.101850    2574 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/21bae6ac-1707-4e61-a4b1-055bd7deeb4a-bundle\") pod \"21bae6ac-1707-4e61-a4b1-055bd7deeb4a\" (UID: \"21bae6ac-1707-4e61-a4b1-055bd7deeb4a\") "
Apr 24 14:30:05.101997 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:30:05.101884    2574 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"kube-api-access-rmt9s\" (UniqueName: \"kubernetes.io/projected/21bae6ac-1707-4e61-a4b1-055bd7deeb4a-kube-api-access-rmt9s\") pod \"21bae6ac-1707-4e61-a4b1-055bd7deeb4a\" (UID: \"21bae6ac-1707-4e61-a4b1-055bd7deeb4a\") "
Apr 24 14:30:05.102471 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:30:05.102439    2574 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/21bae6ac-1707-4e61-a4b1-055bd7deeb4a-bundle" (OuterVolumeSpecName: "bundle") pod "21bae6ac-1707-4e61-a4b1-055bd7deeb4a" (UID: "21bae6ac-1707-4e61-a4b1-055bd7deeb4a"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Apr 24 14:30:05.104081 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:30:05.104053    2574 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/21bae6ac-1707-4e61-a4b1-055bd7deeb4a-kube-api-access-rmt9s" (OuterVolumeSpecName: "kube-api-access-rmt9s") pod "21bae6ac-1707-4e61-a4b1-055bd7deeb4a" (UID: "21bae6ac-1707-4e61-a4b1-055bd7deeb4a"). InnerVolumeSpecName "kube-api-access-rmt9s". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Apr 24 14:30:05.106282 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:30:05.106259    2574 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/21bae6ac-1707-4e61-a4b1-055bd7deeb4a-util" (OuterVolumeSpecName: "util") pod "21bae6ac-1707-4e61-a4b1-055bd7deeb4a" (UID: "21bae6ac-1707-4e61-a4b1-055bd7deeb4a"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Apr 24 14:30:05.203467 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:30:05.203383    2574 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/21bae6ac-1707-4e61-a4b1-055bd7deeb4a-util\") on node \"ip-10-0-136-132.ec2.internal\" DevicePath \"\""
Apr 24 14:30:05.203467 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:30:05.203411    2574 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/21bae6ac-1707-4e61-a4b1-055bd7deeb4a-bundle\") on node \"ip-10-0-136-132.ec2.internal\" DevicePath \"\""
Apr 24 14:30:05.203467 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:30:05.203422    2574 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-rmt9s\" (UniqueName: \"kubernetes.io/projected/21bae6ac-1707-4e61-a4b1-055bd7deeb4a-kube-api-access-rmt9s\") on node \"ip-10-0-136-132.ec2.internal\" DevicePath \"\""
Apr 24 14:30:05.943675 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:30:05.943644    2574 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/59039e319e11338a40c6b6f1054d265f40bb50ceac6068d5c59955d29c4nlx4"
Apr 24 14:30:05.943861 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:30:05.943644    2574 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/59039e319e11338a40c6b6f1054d265f40bb50ceac6068d5c59955d29c4nlx4" event={"ID":"21bae6ac-1707-4e61-a4b1-055bd7deeb4a","Type":"ContainerDied","Data":"09c1f341eacf7ac1b5d77596971f33c20a8f363e6e0487a89dbe5914cc9b2a0e"}
Apr 24 14:30:05.943861 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:30:05.943749    2574 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="09c1f341eacf7ac1b5d77596971f33c20a8f363e6e0487a89dbe5914cc9b2a0e"
Apr 24 14:30:09.453674 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:30:09.453641    2574 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["openshift-keda/custom-metrics-autoscaler-operator-bbf89fd5d-wv24f"]
Apr 24 14:30:09.454143 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:30:09.453990    2574 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="21bae6ac-1707-4e61-a4b1-055bd7deeb4a" containerName="extract"
Apr 24 14:30:09.454143 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:30:09.454001    2574 state_mem.go:107] "Deleted CPUSet assignment" podUID="21bae6ac-1707-4e61-a4b1-055bd7deeb4a" containerName="extract"
Apr 24 14:30:09.454143 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:30:09.454020    2574 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="21bae6ac-1707-4e61-a4b1-055bd7deeb4a" containerName="util"
Apr 24 14:30:09.454143 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:30:09.454026    2574 state_mem.go:107] "Deleted CPUSet assignment" podUID="21bae6ac-1707-4e61-a4b1-055bd7deeb4a" containerName="util"
Apr 24 14:30:09.454143 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:30:09.454033    2574 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="21bae6ac-1707-4e61-a4b1-055bd7deeb4a" containerName="pull"
Apr 24 14:30:09.454143 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:30:09.454038    2574 state_mem.go:107] "Deleted CPUSet assignment" podUID="21bae6ac-1707-4e61-a4b1-055bd7deeb4a" containerName="pull"
Apr 24 14:30:09.454143 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:30:09.454091    2574 memory_manager.go:356] "RemoveStaleState removing state" podUID="21bae6ac-1707-4e61-a4b1-055bd7deeb4a" containerName="extract"
Apr 24 14:30:09.458287 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:30:09.458269    2574 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-keda/custom-metrics-autoscaler-operator-bbf89fd5d-wv24f"
Apr 24 14:30:09.460353 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:30:09.460317    2574 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-keda\"/\"openshift-service-ca.crt\""
Apr 24 14:30:09.460466 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:30:09.460397    2574 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-keda\"/\"custom-metrics-autoscaler-operator-dockercfg-4hg6t\""
Apr 24 14:30:09.460466 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:30:09.460402    2574 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-keda\"/\"kedaorg-certs\""
Apr 24 14:30:09.460466 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:30:09.460403    2574 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-keda\"/\"kube-root-ca.crt\""
Apr 24 14:30:09.465991 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:30:09.465663    2574 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-keda/custom-metrics-autoscaler-operator-bbf89fd5d-wv24f"]
Apr 24 14:30:09.540887 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:30:09.540855    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-77mt2\" (UniqueName: \"kubernetes.io/projected/67e5f1fa-18df-4567-a0f4-ad621bf5da56-kube-api-access-77mt2\") pod \"custom-metrics-autoscaler-operator-bbf89fd5d-wv24f\" (UID: \"67e5f1fa-18df-4567-a0f4-ad621bf5da56\") " pod="openshift-keda/custom-metrics-autoscaler-operator-bbf89fd5d-wv24f"
Apr 24 14:30:09.541075 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:30:09.540909    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"certificates\" (UniqueName: \"kubernetes.io/secret/67e5f1fa-18df-4567-a0f4-ad621bf5da56-certificates\") pod \"custom-metrics-autoscaler-operator-bbf89fd5d-wv24f\" (UID: \"67e5f1fa-18df-4567-a0f4-ad621bf5da56\") " pod="openshift-keda/custom-metrics-autoscaler-operator-bbf89fd5d-wv24f"
Apr 24 14:30:09.641413 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:30:09.641368    2574 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-77mt2\" (UniqueName: \"kubernetes.io/projected/67e5f1fa-18df-4567-a0f4-ad621bf5da56-kube-api-access-77mt2\") pod \"custom-metrics-autoscaler-operator-bbf89fd5d-wv24f\" (UID: \"67e5f1fa-18df-4567-a0f4-ad621bf5da56\") " pod="openshift-keda/custom-metrics-autoscaler-operator-bbf89fd5d-wv24f"
Apr 24 14:30:09.641593 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:30:09.641437    2574 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"certificates\" (UniqueName: \"kubernetes.io/secret/67e5f1fa-18df-4567-a0f4-ad621bf5da56-certificates\") pod \"custom-metrics-autoscaler-operator-bbf89fd5d-wv24f\" (UID: \"67e5f1fa-18df-4567-a0f4-ad621bf5da56\") " pod="openshift-keda/custom-metrics-autoscaler-operator-bbf89fd5d-wv24f"
Apr 24 14:30:09.643717 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:30:09.643692    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"certificates\" (UniqueName: \"kubernetes.io/secret/67e5f1fa-18df-4567-a0f4-ad621bf5da56-certificates\") pod \"custom-metrics-autoscaler-operator-bbf89fd5d-wv24f\" (UID: \"67e5f1fa-18df-4567-a0f4-ad621bf5da56\") " pod="openshift-keda/custom-metrics-autoscaler-operator-bbf89fd5d-wv24f"
Apr 24 14:30:09.649128 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:30:09.649101    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-77mt2\" (UniqueName: \"kubernetes.io/projected/67e5f1fa-18df-4567-a0f4-ad621bf5da56-kube-api-access-77mt2\") pod \"custom-metrics-autoscaler-operator-bbf89fd5d-wv24f\" (UID: \"67e5f1fa-18df-4567-a0f4-ad621bf5da56\") " pod="openshift-keda/custom-metrics-autoscaler-operator-bbf89fd5d-wv24f"
Apr 24 14:30:09.769657 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:30:09.769578    2574 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-keda/custom-metrics-autoscaler-operator-bbf89fd5d-wv24f"
Apr 24 14:30:09.940266 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:30:09.940232    2574 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-keda/custom-metrics-autoscaler-operator-bbf89fd5d-wv24f"]
Apr 24 14:30:09.941436 ip-10-0-136-132 kubenswrapper[2574]: W0424 14:30:09.941409    2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod67e5f1fa_18df_4567_a0f4_ad621bf5da56.slice/crio-beb1d17df848d2a8cabf24b9165ef54f6d9a88faab05da450b11bd088a80aa90 WatchSource:0}: Error finding container beb1d17df848d2a8cabf24b9165ef54f6d9a88faab05da450b11bd088a80aa90: Status 404 returned error can't find the container with id beb1d17df848d2a8cabf24b9165ef54f6d9a88faab05da450b11bd088a80aa90
Apr 24 14:30:09.956678 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:30:09.956647    2574 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-keda/custom-metrics-autoscaler-operator-bbf89fd5d-wv24f" event={"ID":"67e5f1fa-18df-4567-a0f4-ad621bf5da56","Type":"ContainerStarted","Data":"beb1d17df848d2a8cabf24b9165ef54f6d9a88faab05da450b11bd088a80aa90"}
Apr 24 14:30:13.971576 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:30:13.971501    2574 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-keda/custom-metrics-autoscaler-operator-bbf89fd5d-wv24f" event={"ID":"67e5f1fa-18df-4567-a0f4-ad621bf5da56","Type":"ContainerStarted","Data":"9019472914faf1d339cfc15268e9b1fff0ba329900b00edf034734d6506def24"}
Apr 24 14:30:13.972299 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:30:13.972271    2574 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-keda/custom-metrics-autoscaler-operator-bbf89fd5d-wv24f"
Apr 24 14:30:13.995823 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:30:13.995772    2574 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-keda/custom-metrics-autoscaler-operator-bbf89fd5d-wv24f" podStartSLOduration=1.628714164 podStartE2EDuration="4.995754285s" podCreationTimestamp="2026-04-24 14:30:09 +0000 UTC" firstStartedPulling="2026-04-24 14:30:09.943275263 +0000 UTC m=+372.675376905" lastFinishedPulling="2026-04-24 14:30:13.310315384 +0000 UTC m=+376.042417026" observedRunningTime="2026-04-24 14:30:13.994702811 +0000 UTC m=+376.726804470" watchObservedRunningTime="2026-04-24 14:30:13.995754285 +0000 UTC m=+376.727855936"
Apr 24 14:30:14.005766 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:30:14.005731    2574 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["openshift-keda/keda-operator-ffbb595cb-dqb48"]
Apr 24 14:30:14.009206 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:30:14.009184    2574 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-keda/keda-operator-ffbb595cb-dqb48"
Apr 24 14:30:14.014244 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:30:14.014222    2574 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-keda\"/\"keda-operator-certs\""
Apr 24 14:30:14.014376 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:30:14.014227    2574 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-keda\"/\"keda-operator-dockercfg-9dtt9\""
Apr 24 14:30:14.014562 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:30:14.014543    2574 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-keda\"/\"keda-ocp-cabundle\""
Apr 24 14:30:14.022789 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:30:14.022762    2574 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-keda/keda-operator-ffbb595cb-dqb48"]
Apr 24 14:30:14.084887 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:30:14.082156    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-f4vh8\" (UniqueName: \"kubernetes.io/projected/90ae1e0d-fc9a-43e8-98a4-f025c0624b83-kube-api-access-f4vh8\") pod \"keda-operator-ffbb595cb-dqb48\" (UID: \"90ae1e0d-fc9a-43e8-98a4-f025c0624b83\") " pod="openshift-keda/keda-operator-ffbb595cb-dqb48"
Apr 24 14:30:14.084887 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:30:14.082230    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cabundle0\" (UniqueName: \"kubernetes.io/configmap/90ae1e0d-fc9a-43e8-98a4-f025c0624b83-cabundle0\") pod \"keda-operator-ffbb595cb-dqb48\" (UID: \"90ae1e0d-fc9a-43e8-98a4-f025c0624b83\") " pod="openshift-keda/keda-operator-ffbb595cb-dqb48"
Apr 24 14:30:14.084887 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:30:14.082409    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"certificates\" (UniqueName: \"kubernetes.io/projected/90ae1e0d-fc9a-43e8-98a4-f025c0624b83-certificates\") pod \"keda-operator-ffbb595cb-dqb48\" (UID: \"90ae1e0d-fc9a-43e8-98a4-f025c0624b83\") " pod="openshift-keda/keda-operator-ffbb595cb-dqb48"
Apr 24 14:30:14.183602 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:30:14.183569    2574 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"certificates\" (UniqueName: \"kubernetes.io/projected/90ae1e0d-fc9a-43e8-98a4-f025c0624b83-certificates\") pod \"keda-operator-ffbb595cb-dqb48\" (UID: \"90ae1e0d-fc9a-43e8-98a4-f025c0624b83\") " pod="openshift-keda/keda-operator-ffbb595cb-dqb48"
Apr 24 14:30:14.183602 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:30:14.183609    2574 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-f4vh8\" (UniqueName: \"kubernetes.io/projected/90ae1e0d-fc9a-43e8-98a4-f025c0624b83-kube-api-access-f4vh8\") pod \"keda-operator-ffbb595cb-dqb48\" (UID: \"90ae1e0d-fc9a-43e8-98a4-f025c0624b83\") " pod="openshift-keda/keda-operator-ffbb595cb-dqb48"
Apr 24 14:30:14.183852 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:30:14.183629    2574 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"cabundle0\" (UniqueName: \"kubernetes.io/configmap/90ae1e0d-fc9a-43e8-98a4-f025c0624b83-cabundle0\") pod \"keda-operator-ffbb595cb-dqb48\" (UID: \"90ae1e0d-fc9a-43e8-98a4-f025c0624b83\") " pod="openshift-keda/keda-operator-ffbb595cb-dqb48"
Apr 24 14:30:14.183852 ip-10-0-136-132 kubenswrapper[2574]: E0424 14:30:14.183732    2574 secret.go:281] references non-existent secret key: ca.crt
Apr 24 14:30:14.183852 ip-10-0-136-132 kubenswrapper[2574]: E0424 14:30:14.183753    2574 projected.go:277] Couldn't get secret payload openshift-keda/kedaorg-certs: references non-existent secret key: ca.crt
Apr 24 14:30:14.183852 ip-10-0-136-132 kubenswrapper[2574]: E0424 14:30:14.183768    2574 projected.go:194] Error preparing data for projected volume certificates for pod openshift-keda/keda-operator-ffbb595cb-dqb48: references non-existent secret key: ca.crt
Apr 24 14:30:14.183852 ip-10-0-136-132 kubenswrapper[2574]: E0424 14:30:14.183824    2574 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/90ae1e0d-fc9a-43e8-98a4-f025c0624b83-certificates podName:90ae1e0d-fc9a-43e8-98a4-f025c0624b83 nodeName:}" failed. No retries permitted until 2026-04-24 14:30:14.683808899 +0000 UTC m=+377.415910527 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "certificates" (UniqueName: "kubernetes.io/projected/90ae1e0d-fc9a-43e8-98a4-f025c0624b83-certificates") pod "keda-operator-ffbb595cb-dqb48" (UID: "90ae1e0d-fc9a-43e8-98a4-f025c0624b83") : references non-existent secret key: ca.crt
Apr 24 14:30:14.184253 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:30:14.184234    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cabundle0\" (UniqueName: \"kubernetes.io/configmap/90ae1e0d-fc9a-43e8-98a4-f025c0624b83-cabundle0\") pod \"keda-operator-ffbb595cb-dqb48\" (UID: \"90ae1e0d-fc9a-43e8-98a4-f025c0624b83\") " pod="openshift-keda/keda-operator-ffbb595cb-dqb48"
Apr 24 14:30:14.192707 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:30:14.192678    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-f4vh8\" (UniqueName: \"kubernetes.io/projected/90ae1e0d-fc9a-43e8-98a4-f025c0624b83-kube-api-access-f4vh8\") pod \"keda-operator-ffbb595cb-dqb48\" (UID: \"90ae1e0d-fc9a-43e8-98a4-f025c0624b83\") " pod="openshift-keda/keda-operator-ffbb595cb-dqb48"
Apr 24 14:30:14.688152 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:30:14.688080    2574 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"certificates\" (UniqueName: \"kubernetes.io/projected/90ae1e0d-fc9a-43e8-98a4-f025c0624b83-certificates\") pod \"keda-operator-ffbb595cb-dqb48\" (UID: \"90ae1e0d-fc9a-43e8-98a4-f025c0624b83\") " pod="openshift-keda/keda-operator-ffbb595cb-dqb48"
Apr 24 14:30:14.688355 ip-10-0-136-132 kubenswrapper[2574]: E0424 14:30:14.688255    2574 secret.go:281] references non-existent secret key: ca.crt
Apr 24 14:30:14.688355 ip-10-0-136-132 kubenswrapper[2574]: E0424 14:30:14.688295    2574 projected.go:277] Couldn't get secret payload openshift-keda/kedaorg-certs: references non-existent secret key: ca.crt
Apr 24 14:30:14.688355 ip-10-0-136-132 kubenswrapper[2574]: E0424 14:30:14.688308    2574 projected.go:194] Error preparing data for projected volume certificates for pod openshift-keda/keda-operator-ffbb595cb-dqb48: references non-existent secret key: ca.crt
Apr 24 14:30:14.688509 ip-10-0-136-132 kubenswrapper[2574]: E0424 14:30:14.688391    2574 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/90ae1e0d-fc9a-43e8-98a4-f025c0624b83-certificates podName:90ae1e0d-fc9a-43e8-98a4-f025c0624b83 nodeName:}" failed. No retries permitted until 2026-04-24 14:30:15.688372176 +0000 UTC m=+378.420473817 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "certificates" (UniqueName: "kubernetes.io/projected/90ae1e0d-fc9a-43e8-98a4-f025c0624b83-certificates") pod "keda-operator-ffbb595cb-dqb48" (UID: "90ae1e0d-fc9a-43e8-98a4-f025c0624b83") : references non-existent secret key: ca.crt
Apr 24 14:30:14.803513 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:30:14.803476    2574 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["openshift-keda/keda-admission-cf49989db-sdmk4"]
Apr 24 14:30:14.807095 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:30:14.807071    2574 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-keda/keda-admission-cf49989db-sdmk4"
Apr 24 14:30:14.813432 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:30:14.813405    2574 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-keda\"/\"keda-admission-webhooks-certs\""
Apr 24 14:30:14.838520 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:30:14.838488    2574 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-keda/keda-admission-cf49989db-sdmk4"]
Apr 24 14:30:14.890111 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:30:14.890079    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9zfxd\" (UniqueName: \"kubernetes.io/projected/25929d25-c161-4b13-8668-acff54c00b01-kube-api-access-9zfxd\") pod \"keda-admission-cf49989db-sdmk4\" (UID: \"25929d25-c161-4b13-8668-acff54c00b01\") " pod="openshift-keda/keda-admission-cf49989db-sdmk4"
Apr 24 14:30:14.890111 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:30:14.890114    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"certificates\" (UniqueName: \"kubernetes.io/projected/25929d25-c161-4b13-8668-acff54c00b01-certificates\") pod \"keda-admission-cf49989db-sdmk4\" (UID: \"25929d25-c161-4b13-8668-acff54c00b01\") " pod="openshift-keda/keda-admission-cf49989db-sdmk4"
Apr 24 14:30:14.991681 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:30:14.991586    2574 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-9zfxd\" (UniqueName: \"kubernetes.io/projected/25929d25-c161-4b13-8668-acff54c00b01-kube-api-access-9zfxd\") pod \"keda-admission-cf49989db-sdmk4\" (UID: \"25929d25-c161-4b13-8668-acff54c00b01\") " pod="openshift-keda/keda-admission-cf49989db-sdmk4"
Apr 24 14:30:14.992206 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:30:14.992176    2574 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"certificates\" (UniqueName: \"kubernetes.io/projected/25929d25-c161-4b13-8668-acff54c00b01-certificates\") pod \"keda-admission-cf49989db-sdmk4\" (UID: \"25929d25-c161-4b13-8668-acff54c00b01\") " pod="openshift-keda/keda-admission-cf49989db-sdmk4"
Apr 24 14:30:14.995734 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:30:14.995703    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"certificates\" (UniqueName: \"kubernetes.io/projected/25929d25-c161-4b13-8668-acff54c00b01-certificates\") pod \"keda-admission-cf49989db-sdmk4\" (UID: \"25929d25-c161-4b13-8668-acff54c00b01\") " pod="openshift-keda/keda-admission-cf49989db-sdmk4"
Apr 24 14:30:15.008084 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:30:15.008056    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-9zfxd\" (UniqueName: \"kubernetes.io/projected/25929d25-c161-4b13-8668-acff54c00b01-kube-api-access-9zfxd\") pod \"keda-admission-cf49989db-sdmk4\" (UID: \"25929d25-c161-4b13-8668-acff54c00b01\") " pod="openshift-keda/keda-admission-cf49989db-sdmk4"
Apr 24 14:30:15.117657 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:30:15.117622    2574 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-keda/keda-admission-cf49989db-sdmk4"
Apr 24 14:30:15.264320 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:30:15.264283    2574 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-keda/keda-admission-cf49989db-sdmk4"]
Apr 24 14:30:15.265731 ip-10-0-136-132 kubenswrapper[2574]: W0424 14:30:15.265706    2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod25929d25_c161_4b13_8668_acff54c00b01.slice/crio-6f36f5556b52cec473836cfdafa70a35bd4cf6f2a6569ce686971c80500c9eca WatchSource:0}: Error finding container 6f36f5556b52cec473836cfdafa70a35bd4cf6f2a6569ce686971c80500c9eca: Status 404 returned error can't find the container with id 6f36f5556b52cec473836cfdafa70a35bd4cf6f2a6569ce686971c80500c9eca
Apr 24 14:30:15.700615 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:30:15.700581    2574 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"certificates\" (UniqueName: \"kubernetes.io/projected/90ae1e0d-fc9a-43e8-98a4-f025c0624b83-certificates\") pod \"keda-operator-ffbb595cb-dqb48\" (UID: \"90ae1e0d-fc9a-43e8-98a4-f025c0624b83\") " pod="openshift-keda/keda-operator-ffbb595cb-dqb48"
Apr 24 14:30:15.700773 ip-10-0-136-132 kubenswrapper[2574]: E0424 14:30:15.700692    2574 secret.go:281] references non-existent secret key: ca.crt
Apr 24 14:30:15.700773 ip-10-0-136-132 kubenswrapper[2574]: E0424 14:30:15.700704    2574 projected.go:277] Couldn't get secret payload openshift-keda/kedaorg-certs: references non-existent secret key: ca.crt
Apr 24 14:30:15.700773 ip-10-0-136-132 kubenswrapper[2574]: E0424 14:30:15.700712    2574 projected.go:194] Error preparing data for projected volume certificates for pod openshift-keda/keda-operator-ffbb595cb-dqb48: references non-existent secret key: ca.crt
Apr 24 14:30:15.700897 ip-10-0-136-132 kubenswrapper[2574]: E0424 14:30:15.700780    2574 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/90ae1e0d-fc9a-43e8-98a4-f025c0624b83-certificates podName:90ae1e0d-fc9a-43e8-98a4-f025c0624b83 nodeName:}" failed. No retries permitted until 2026-04-24 14:30:17.700752939 +0000 UTC m=+380.432854566 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "certificates" (UniqueName: "kubernetes.io/projected/90ae1e0d-fc9a-43e8-98a4-f025c0624b83-certificates") pod "keda-operator-ffbb595cb-dqb48" (UID: "90ae1e0d-fc9a-43e8-98a4-f025c0624b83") : references non-existent secret key: ca.crt
Apr 24 14:30:15.979005 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:30:15.978905    2574 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-keda/keda-admission-cf49989db-sdmk4" event={"ID":"25929d25-c161-4b13-8668-acff54c00b01","Type":"ContainerStarted","Data":"6f36f5556b52cec473836cfdafa70a35bd4cf6f2a6569ce686971c80500c9eca"}
Apr 24 14:30:16.984093 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:30:16.984005    2574 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-keda/keda-admission-cf49989db-sdmk4" event={"ID":"25929d25-c161-4b13-8668-acff54c00b01","Type":"ContainerStarted","Data":"c403e4743901af948b5a2a10aeec7c7de43c04b3d65ff5d5511fb5eb4d39e352"}
Apr 24 14:30:16.984499 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:30:16.984145    2574 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-keda/keda-admission-cf49989db-sdmk4"
Apr 24 14:30:17.000568 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:30:17.000524    2574 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-keda/keda-admission-cf49989db-sdmk4" podStartSLOduration=1.646990865 podStartE2EDuration="3.000508707s" podCreationTimestamp="2026-04-24 14:30:14 +0000 UTC" firstStartedPulling="2026-04-24 14:30:15.266980824 +0000 UTC m=+377.999082450" lastFinishedPulling="2026-04-24 14:30:16.62049865 +0000 UTC m=+379.352600292" observedRunningTime="2026-04-24 14:30:16.998830158 +0000 UTC m=+379.730931808" watchObservedRunningTime="2026-04-24 14:30:17.000508707 +0000 UTC m=+379.732610356"
Apr 24 14:30:17.719601 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:30:17.719559    2574 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"certificates\" (UniqueName: \"kubernetes.io/projected/90ae1e0d-fc9a-43e8-98a4-f025c0624b83-certificates\") pod \"keda-operator-ffbb595cb-dqb48\" (UID: \"90ae1e0d-fc9a-43e8-98a4-f025c0624b83\") " pod="openshift-keda/keda-operator-ffbb595cb-dqb48"
Apr 24 14:30:17.719790 ip-10-0-136-132 kubenswrapper[2574]: E0424 14:30:17.719684    2574 secret.go:281] references non-existent secret key: ca.crt
Apr 24 14:30:17.719790 ip-10-0-136-132 kubenswrapper[2574]: E0424 14:30:17.719698    2574 projected.go:277] Couldn't get secret payload openshift-keda/kedaorg-certs: references non-existent secret key: ca.crt
Apr 24 14:30:17.719790 ip-10-0-136-132 kubenswrapper[2574]: E0424 14:30:17.719707    2574 projected.go:194] Error preparing data for projected volume certificates for pod openshift-keda/keda-operator-ffbb595cb-dqb48: references non-existent secret key: ca.crt
Apr 24 14:30:17.719790 ip-10-0-136-132 kubenswrapper[2574]: E0424 14:30:17.719753    2574 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/90ae1e0d-fc9a-43e8-98a4-f025c0624b83-certificates podName:90ae1e0d-fc9a-43e8-98a4-f025c0624b83 nodeName:}" failed. No retries permitted until 2026-04-24 14:30:21.719739325 +0000 UTC m=+384.451840952 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "certificates" (UniqueName: "kubernetes.io/projected/90ae1e0d-fc9a-43e8-98a4-f025c0624b83-certificates") pod "keda-operator-ffbb595cb-dqb48" (UID: "90ae1e0d-fc9a-43e8-98a4-f025c0624b83") : references non-existent secret key: ca.crt
Apr 24 14:30:21.754785 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:30:21.754747    2574 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"certificates\" (UniqueName: \"kubernetes.io/projected/90ae1e0d-fc9a-43e8-98a4-f025c0624b83-certificates\") pod \"keda-operator-ffbb595cb-dqb48\" (UID: \"90ae1e0d-fc9a-43e8-98a4-f025c0624b83\") " pod="openshift-keda/keda-operator-ffbb595cb-dqb48"
Apr 24 14:30:21.757220 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:30:21.757198    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"certificates\" (UniqueName: \"kubernetes.io/projected/90ae1e0d-fc9a-43e8-98a4-f025c0624b83-certificates\") pod \"keda-operator-ffbb595cb-dqb48\" (UID: \"90ae1e0d-fc9a-43e8-98a4-f025c0624b83\") " pod="openshift-keda/keda-operator-ffbb595cb-dqb48"
Apr 24 14:30:21.819931 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:30:21.819880    2574 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-keda/keda-operator-ffbb595cb-dqb48"
Apr 24 14:30:21.943357 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:30:21.943319    2574 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-keda/keda-operator-ffbb595cb-dqb48"]
Apr 24 14:30:21.945462 ip-10-0-136-132 kubenswrapper[2574]: W0424 14:30:21.945431    2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod90ae1e0d_fc9a_43e8_98a4_f025c0624b83.slice/crio-fc93eec0ac2dcc688055482a882578e55c43ebcdc126153ec2b5ebb78ca93666 WatchSource:0}: Error finding container fc93eec0ac2dcc688055482a882578e55c43ebcdc126153ec2b5ebb78ca93666: Status 404 returned error can't find the container with id fc93eec0ac2dcc688055482a882578e55c43ebcdc126153ec2b5ebb78ca93666
Apr 24 14:30:22.007290 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:30:22.007203    2574 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-keda/keda-operator-ffbb595cb-dqb48" event={"ID":"90ae1e0d-fc9a-43e8-98a4-f025c0624b83","Type":"ContainerStarted","Data":"fc93eec0ac2dcc688055482a882578e55c43ebcdc126153ec2b5ebb78ca93666"}
Apr 24 14:30:25.019899 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:30:25.019859    2574 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-keda/keda-operator-ffbb595cb-dqb48" event={"ID":"90ae1e0d-fc9a-43e8-98a4-f025c0624b83","Type":"ContainerStarted","Data":"8c228b5ce9be055715d894354104569c387a81c5df570fa9f4af98e5e76c8615"}
Apr 24 14:30:25.020275 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:30:25.019919    2574 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-keda/keda-operator-ffbb595cb-dqb48"
Apr 24 14:30:25.035676 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:30:25.035625    2574 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-keda/keda-operator-ffbb595cb-dqb48" podStartSLOduration=9.154307815 podStartE2EDuration="12.0356104s" podCreationTimestamp="2026-04-24 14:30:13 +0000 UTC" firstStartedPulling="2026-04-24 14:30:21.94672571 +0000 UTC m=+384.678827337" lastFinishedPulling="2026-04-24 14:30:24.828028295 +0000 UTC m=+387.560129922" observedRunningTime="2026-04-24 14:30:25.033692757 +0000 UTC m=+387.765794404" watchObservedRunningTime="2026-04-24 14:30:25.0356104 +0000 UTC m=+387.767712049"
Apr 24 14:30:35.981973 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:30:35.981938    2574 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-keda/custom-metrics-autoscaler-operator-bbf89fd5d-wv24f"
Apr 24 14:30:37.990517 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:30:37.990486    2574 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-keda/keda-admission-cf49989db-sdmk4"
Apr 24 14:30:46.026605 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:30:46.026577    2574 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-keda/keda-operator-ffbb595cb-dqb48"
Apr 24 14:31:07.730180 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:31:07.730138    2574 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["openshift-marketplace/2bb52b92bd31ddf2ebbc335370ac517be50e61a93c2fe375393413c19djgzqr"]
Apr 24 14:31:07.741600 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:31:07.741569    2574 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/2bb52b92bd31ddf2ebbc335370ac517be50e61a93c2fe375393413c19djgzqr"]
Apr 24 14:31:07.741752 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:31:07.741680    2574 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/2bb52b92bd31ddf2ebbc335370ac517be50e61a93c2fe375393413c19djgzqr"
Apr 24 14:31:07.743864 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:31:07.743844    2574 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-marketplace\"/\"default-dockercfg-dnr8q\""
Apr 24 14:31:07.744018 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:31:07.743895    2574 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"kube-root-ca.crt\""
Apr 24 14:31:07.745352 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:31:07.745316    2574 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"openshift-service-ca.crt\""
Apr 24 14:31:07.753502 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:31:07.753479    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/e7478a44-2348-49ba-9d91-3adf7a776adf-bundle\") pod \"2bb52b92bd31ddf2ebbc335370ac517be50e61a93c2fe375393413c19djgzqr\" (UID: \"e7478a44-2348-49ba-9d91-3adf7a776adf\") " pod="openshift-marketplace/2bb52b92bd31ddf2ebbc335370ac517be50e61a93c2fe375393413c19djgzqr"
Apr 24 14:31:07.753625 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:31:07.753508    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/e7478a44-2348-49ba-9d91-3adf7a776adf-util\") pod \"2bb52b92bd31ddf2ebbc335370ac517be50e61a93c2fe375393413c19djgzqr\" (UID: \"e7478a44-2348-49ba-9d91-3adf7a776adf\") " pod="openshift-marketplace/2bb52b92bd31ddf2ebbc335370ac517be50e61a93c2fe375393413c19djgzqr"
Apr 24 14:31:07.753625 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:31:07.753547    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-p9h47\" (UniqueName: \"kubernetes.io/projected/e7478a44-2348-49ba-9d91-3adf7a776adf-kube-api-access-p9h47\") pod \"2bb52b92bd31ddf2ebbc335370ac517be50e61a93c2fe375393413c19djgzqr\" (UID: \"e7478a44-2348-49ba-9d91-3adf7a776adf\") " pod="openshift-marketplace/2bb52b92bd31ddf2ebbc335370ac517be50e61a93c2fe375393413c19djgzqr"
Apr 24 14:31:07.854647 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:31:07.854609    2574 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-p9h47\" (UniqueName: \"kubernetes.io/projected/e7478a44-2348-49ba-9d91-3adf7a776adf-kube-api-access-p9h47\") pod \"2bb52b92bd31ddf2ebbc335370ac517be50e61a93c2fe375393413c19djgzqr\" (UID: \"e7478a44-2348-49ba-9d91-3adf7a776adf\") " pod="openshift-marketplace/2bb52b92bd31ddf2ebbc335370ac517be50e61a93c2fe375393413c19djgzqr"
Apr 24 14:31:07.854844 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:31:07.854693    2574 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/e7478a44-2348-49ba-9d91-3adf7a776adf-bundle\") pod \"2bb52b92bd31ddf2ebbc335370ac517be50e61a93c2fe375393413c19djgzqr\" (UID: \"e7478a44-2348-49ba-9d91-3adf7a776adf\") " pod="openshift-marketplace/2bb52b92bd31ddf2ebbc335370ac517be50e61a93c2fe375393413c19djgzqr"
Apr 24 14:31:07.854844 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:31:07.854714    2574 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/e7478a44-2348-49ba-9d91-3adf7a776adf-util\") pod \"2bb52b92bd31ddf2ebbc335370ac517be50e61a93c2fe375393413c19djgzqr\" (UID: \"e7478a44-2348-49ba-9d91-3adf7a776adf\") " pod="openshift-marketplace/2bb52b92bd31ddf2ebbc335370ac517be50e61a93c2fe375393413c19djgzqr"
Apr 24 14:31:07.855046 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:31:07.855028    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/e7478a44-2348-49ba-9d91-3adf7a776adf-util\") pod \"2bb52b92bd31ddf2ebbc335370ac517be50e61a93c2fe375393413c19djgzqr\" (UID: \"e7478a44-2348-49ba-9d91-3adf7a776adf\") " pod="openshift-marketplace/2bb52b92bd31ddf2ebbc335370ac517be50e61a93c2fe375393413c19djgzqr"
Apr 24 14:31:07.855136 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:31:07.855114    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/e7478a44-2348-49ba-9d91-3adf7a776adf-bundle\") pod \"2bb52b92bd31ddf2ebbc335370ac517be50e61a93c2fe375393413c19djgzqr\" (UID: \"e7478a44-2348-49ba-9d91-3adf7a776adf\") " pod="openshift-marketplace/2bb52b92bd31ddf2ebbc335370ac517be50e61a93c2fe375393413c19djgzqr"
Apr 24 14:31:07.862639 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:31:07.862608    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-p9h47\" (UniqueName: \"kubernetes.io/projected/e7478a44-2348-49ba-9d91-3adf7a776adf-kube-api-access-p9h47\") pod \"2bb52b92bd31ddf2ebbc335370ac517be50e61a93c2fe375393413c19djgzqr\" (UID: \"e7478a44-2348-49ba-9d91-3adf7a776adf\") " pod="openshift-marketplace/2bb52b92bd31ddf2ebbc335370ac517be50e61a93c2fe375393413c19djgzqr"
Apr 24 14:31:08.051842 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:31:08.051754    2574 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/2bb52b92bd31ddf2ebbc335370ac517be50e61a93c2fe375393413c19djgzqr"
Apr 24 14:31:08.173453 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:31:08.173426    2574 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/2bb52b92bd31ddf2ebbc335370ac517be50e61a93c2fe375393413c19djgzqr"]
Apr 24 14:31:08.176224 ip-10-0-136-132 kubenswrapper[2574]: W0424 14:31:08.176196    2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode7478a44_2348_49ba_9d91_3adf7a776adf.slice/crio-7df4989d8a5d67596abcacfc7bca8f398ad142e29e98453e04c0a3da91e553ea WatchSource:0}: Error finding container 7df4989d8a5d67596abcacfc7bca8f398ad142e29e98453e04c0a3da91e553ea: Status 404 returned error can't find the container with id 7df4989d8a5d67596abcacfc7bca8f398ad142e29e98453e04c0a3da91e553ea
Apr 24 14:31:09.163410 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:31:09.163376    2574 generic.go:358] "Generic (PLEG): container finished" podID="e7478a44-2348-49ba-9d91-3adf7a776adf" containerID="c4bda31c04030f22be2d755f4f0e45ded8b55f45cdf502e13f9b0a75c28251fc" exitCode=0
Apr 24 14:31:09.163773 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:31:09.163424    2574 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/2bb52b92bd31ddf2ebbc335370ac517be50e61a93c2fe375393413c19djgzqr" event={"ID":"e7478a44-2348-49ba-9d91-3adf7a776adf","Type":"ContainerDied","Data":"c4bda31c04030f22be2d755f4f0e45ded8b55f45cdf502e13f9b0a75c28251fc"}
Apr 24 14:31:09.163773 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:31:09.163456    2574 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/2bb52b92bd31ddf2ebbc335370ac517be50e61a93c2fe375393413c19djgzqr" event={"ID":"e7478a44-2348-49ba-9d91-3adf7a776adf","Type":"ContainerStarted","Data":"7df4989d8a5d67596abcacfc7bca8f398ad142e29e98453e04c0a3da91e553ea"}
Apr 24 14:31:13.179465 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:31:13.179429    2574 generic.go:358] "Generic (PLEG): container finished" podID="e7478a44-2348-49ba-9d91-3adf7a776adf" containerID="1e40511ebd5f46c142e8898677775cc776688e7b4c8d58ca849a98b1146e3d88" exitCode=0
Apr 24 14:31:13.179860 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:31:13.179496    2574 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/2bb52b92bd31ddf2ebbc335370ac517be50e61a93c2fe375393413c19djgzqr" event={"ID":"e7478a44-2348-49ba-9d91-3adf7a776adf","Type":"ContainerDied","Data":"1e40511ebd5f46c142e8898677775cc776688e7b4c8d58ca849a98b1146e3d88"}
Apr 24 14:31:14.184815 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:31:14.184778    2574 generic.go:358] "Generic (PLEG): container finished" podID="e7478a44-2348-49ba-9d91-3adf7a776adf" containerID="6040e5b22ab4823529bf19d209541f3894fb4dfb7f4c3f0ba1fd8759bb504cb7" exitCode=0
Apr 24 14:31:14.185211 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:31:14.184829    2574 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/2bb52b92bd31ddf2ebbc335370ac517be50e61a93c2fe375393413c19djgzqr" event={"ID":"e7478a44-2348-49ba-9d91-3adf7a776adf","Type":"ContainerDied","Data":"6040e5b22ab4823529bf19d209541f3894fb4dfb7f4c3f0ba1fd8759bb504cb7"}
Apr 24 14:31:15.311963 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:31:15.311938    2574 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/2bb52b92bd31ddf2ebbc335370ac517be50e61a93c2fe375393413c19djgzqr"
Apr 24 14:31:15.412606 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:31:15.412577    2574 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/e7478a44-2348-49ba-9d91-3adf7a776adf-bundle\") pod \"e7478a44-2348-49ba-9d91-3adf7a776adf\" (UID: \"e7478a44-2348-49ba-9d91-3adf7a776adf\") "
Apr 24 14:31:15.412782 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:31:15.412657    2574 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"kube-api-access-p9h47\" (UniqueName: \"kubernetes.io/projected/e7478a44-2348-49ba-9d91-3adf7a776adf-kube-api-access-p9h47\") pod \"e7478a44-2348-49ba-9d91-3adf7a776adf\" (UID: \"e7478a44-2348-49ba-9d91-3adf7a776adf\") "
Apr 24 14:31:15.412782 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:31:15.412682    2574 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/e7478a44-2348-49ba-9d91-3adf7a776adf-util\") pod \"e7478a44-2348-49ba-9d91-3adf7a776adf\" (UID: \"e7478a44-2348-49ba-9d91-3adf7a776adf\") "
Apr 24 14:31:15.413568 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:31:15.413516    2574 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e7478a44-2348-49ba-9d91-3adf7a776adf-bundle" (OuterVolumeSpecName: "bundle") pod "e7478a44-2348-49ba-9d91-3adf7a776adf" (UID: "e7478a44-2348-49ba-9d91-3adf7a776adf"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Apr 24 14:31:15.414723 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:31:15.414700    2574 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e7478a44-2348-49ba-9d91-3adf7a776adf-kube-api-access-p9h47" (OuterVolumeSpecName: "kube-api-access-p9h47") pod "e7478a44-2348-49ba-9d91-3adf7a776adf" (UID: "e7478a44-2348-49ba-9d91-3adf7a776adf"). InnerVolumeSpecName "kube-api-access-p9h47". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Apr 24 14:31:15.417178 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:31:15.417158    2574 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e7478a44-2348-49ba-9d91-3adf7a776adf-util" (OuterVolumeSpecName: "util") pod "e7478a44-2348-49ba-9d91-3adf7a776adf" (UID: "e7478a44-2348-49ba-9d91-3adf7a776adf"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Apr 24 14:31:15.513872 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:31:15.513764    2574 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-p9h47\" (UniqueName: \"kubernetes.io/projected/e7478a44-2348-49ba-9d91-3adf7a776adf-kube-api-access-p9h47\") on node \"ip-10-0-136-132.ec2.internal\" DevicePath \"\""
Apr 24 14:31:15.513872 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:31:15.513813    2574 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/e7478a44-2348-49ba-9d91-3adf7a776adf-util\") on node \"ip-10-0-136-132.ec2.internal\" DevicePath \"\""
Apr 24 14:31:15.513872 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:31:15.513825    2574 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/e7478a44-2348-49ba-9d91-3adf7a776adf-bundle\") on node \"ip-10-0-136-132.ec2.internal\" DevicePath \"\""
Apr 24 14:31:16.192494 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:31:16.192461    2574 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/2bb52b92bd31ddf2ebbc335370ac517be50e61a93c2fe375393413c19djgzqr" event={"ID":"e7478a44-2348-49ba-9d91-3adf7a776adf","Type":"ContainerDied","Data":"7df4989d8a5d67596abcacfc7bca8f398ad142e29e98453e04c0a3da91e553ea"}
Apr 24 14:31:16.192494 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:31:16.192480    2574 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/2bb52b92bd31ddf2ebbc335370ac517be50e61a93c2fe375393413c19djgzqr"
Apr 24 14:31:16.192494 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:31:16.192497    2574 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="7df4989d8a5d67596abcacfc7bca8f398ad142e29e98453e04c0a3da91e553ea"
Apr 24 14:31:20.596021 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:31:20.595985    2574 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["cert-manager-operator/cert-manager-operator-controller-manager-54b9655956-c4qkg"]
Apr 24 14:31:20.596569 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:31:20.596552    2574 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e7478a44-2348-49ba-9d91-3adf7a776adf" containerName="extract"
Apr 24 14:31:20.596617 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:31:20.596573    2574 state_mem.go:107] "Deleted CPUSet assignment" podUID="e7478a44-2348-49ba-9d91-3adf7a776adf" containerName="extract"
Apr 24 14:31:20.596617 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:31:20.596595    2574 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e7478a44-2348-49ba-9d91-3adf7a776adf" containerName="pull"
Apr 24 14:31:20.596617 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:31:20.596604    2574 state_mem.go:107] "Deleted CPUSet assignment" podUID="e7478a44-2348-49ba-9d91-3adf7a776adf" containerName="pull"
Apr 24 14:31:20.596704 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:31:20.596619    2574 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e7478a44-2348-49ba-9d91-3adf7a776adf" containerName="util"
Apr 24 14:31:20.596704 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:31:20.596628    2574 state_mem.go:107] "Deleted CPUSet assignment" podUID="e7478a44-2348-49ba-9d91-3adf7a776adf" containerName="util"
Apr 24 14:31:20.596765 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:31:20.596704    2574 memory_manager.go:356] "RemoveStaleState removing state" podUID="e7478a44-2348-49ba-9d91-3adf7a776adf" containerName="extract"
Apr 24 14:31:20.601839 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:31:20.601823    2574 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager-operator/cert-manager-operator-controller-manager-54b9655956-c4qkg"
Apr 24 14:31:20.603842 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:31:20.603815    2574 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"cert-manager-operator\"/\"cert-manager-operator-controller-manager-dockercfg-xp7sr\""
Apr 24 14:31:20.603842 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:31:20.603833    2574 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager-operator\"/\"openshift-service-ca.crt\""
Apr 24 14:31:20.603999 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:31:20.603824    2574 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager-operator\"/\"kube-root-ca.crt\""
Apr 24 14:31:20.611267 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:31:20.611246    2574 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["cert-manager-operator/cert-manager-operator-controller-manager-54b9655956-c4qkg"]
Apr 24 14:31:20.658293 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:31:20.658257    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/e59056bd-cd9c-4382-a71f-b7e5b04c650a-tmp\") pod \"cert-manager-operator-controller-manager-54b9655956-c4qkg\" (UID: \"e59056bd-cd9c-4382-a71f-b7e5b04c650a\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-54b9655956-c4qkg"
Apr 24 14:31:20.658480 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:31:20.658312    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9pwf7\" (UniqueName: \"kubernetes.io/projected/e59056bd-cd9c-4382-a71f-b7e5b04c650a-kube-api-access-9pwf7\") pod \"cert-manager-operator-controller-manager-54b9655956-c4qkg\" (UID: \"e59056bd-cd9c-4382-a71f-b7e5b04c650a\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-54b9655956-c4qkg"
Apr 24 14:31:20.759557 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:31:20.759521    2574 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/e59056bd-cd9c-4382-a71f-b7e5b04c650a-tmp\") pod \"cert-manager-operator-controller-manager-54b9655956-c4qkg\" (UID: \"e59056bd-cd9c-4382-a71f-b7e5b04c650a\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-54b9655956-c4qkg"
Apr 24 14:31:20.759721 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:31:20.759569    2574 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-9pwf7\" (UniqueName: \"kubernetes.io/projected/e59056bd-cd9c-4382-a71f-b7e5b04c650a-kube-api-access-9pwf7\") pod \"cert-manager-operator-controller-manager-54b9655956-c4qkg\" (UID: \"e59056bd-cd9c-4382-a71f-b7e5b04c650a\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-54b9655956-c4qkg"
Apr 24 14:31:20.759980 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:31:20.759955    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/e59056bd-cd9c-4382-a71f-b7e5b04c650a-tmp\") pod \"cert-manager-operator-controller-manager-54b9655956-c4qkg\" (UID: \"e59056bd-cd9c-4382-a71f-b7e5b04c650a\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-54b9655956-c4qkg"
Apr 24 14:31:20.767817 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:31:20.767787    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-9pwf7\" (UniqueName: \"kubernetes.io/projected/e59056bd-cd9c-4382-a71f-b7e5b04c650a-kube-api-access-9pwf7\") pod \"cert-manager-operator-controller-manager-54b9655956-c4qkg\" (UID: \"e59056bd-cd9c-4382-a71f-b7e5b04c650a\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-54b9655956-c4qkg"
Apr 24 14:31:20.911785 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:31:20.911750    2574 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager-operator/cert-manager-operator-controller-manager-54b9655956-c4qkg"
Apr 24 14:31:21.068399 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:31:21.068372    2574 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["cert-manager-operator/cert-manager-operator-controller-manager-54b9655956-c4qkg"]
Apr 24 14:31:21.071408 ip-10-0-136-132 kubenswrapper[2574]: W0424 14:31:21.071379    2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode59056bd_cd9c_4382_a71f_b7e5b04c650a.slice/crio-5beb0bdb7104d50aac5ab57191e23880d2dae3c2f9ef750179182e35830adc5d WatchSource:0}: Error finding container 5beb0bdb7104d50aac5ab57191e23880d2dae3c2f9ef750179182e35830adc5d: Status 404 returned error can't find the container with id 5beb0bdb7104d50aac5ab57191e23880d2dae3c2f9ef750179182e35830adc5d
Apr 24 14:31:21.211032 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:31:21.210949    2574 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="cert-manager-operator/cert-manager-operator-controller-manager-54b9655956-c4qkg" event={"ID":"e59056bd-cd9c-4382-a71f-b7e5b04c650a","Type":"ContainerStarted","Data":"5beb0bdb7104d50aac5ab57191e23880d2dae3c2f9ef750179182e35830adc5d"}
Apr 24 14:31:24.226850 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:31:24.226803    2574 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="cert-manager-operator/cert-manager-operator-controller-manager-54b9655956-c4qkg" event={"ID":"e59056bd-cd9c-4382-a71f-b7e5b04c650a","Type":"ContainerStarted","Data":"8e2d93f3f591af67136b789c204e48557003a653fb80c270daa45ef51a749e8a"}
Apr 24 14:31:24.251086 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:31:24.251023    2574 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="cert-manager-operator/cert-manager-operator-controller-manager-54b9655956-c4qkg" podStartSLOduration=1.5913821559999999 podStartE2EDuration="4.251002752s" podCreationTimestamp="2026-04-24 14:31:20 +0000 UTC" firstStartedPulling="2026-04-24 14:31:21.073739605 +0000 UTC m=+443.805841237" lastFinishedPulling="2026-04-24 14:31:23.733360203 +0000 UTC m=+446.465461833" observedRunningTime="2026-04-24 14:31:24.247946776 +0000 UTC m=+446.980048426" watchObservedRunningTime="2026-04-24 14:31:24.251002752 +0000 UTC m=+446.983104402"
Apr 24 14:31:29.853551 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:31:29.853515    2574 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f9c4h7"]
Apr 24 14:31:29.857289 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:31:29.857273    2574 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f9c4h7"
Apr 24 14:31:29.860172 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:31:29.859798    2574 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"kube-root-ca.crt\""
Apr 24 14:31:29.860172 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:31:29.859850    2574 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"openshift-service-ca.crt\""
Apr 24 14:31:29.863315 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:31:29.863295    2574 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-marketplace\"/\"default-dockercfg-dnr8q\""
Apr 24 14:31:29.866907 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:31:29.866879    2574 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f9c4h7"]
Apr 24 14:31:29.936011 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:31:29.935975    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-t6cd8\" (UniqueName: \"kubernetes.io/projected/e6c487a4-5df5-409b-9cf0-ece6c7a1baa3-kube-api-access-t6cd8\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f9c4h7\" (UID: \"e6c487a4-5df5-409b-9cf0-ece6c7a1baa3\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f9c4h7"
Apr 24 14:31:29.936179 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:31:29.936085    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/e6c487a4-5df5-409b-9cf0-ece6c7a1baa3-util\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f9c4h7\" (UID: \"e6c487a4-5df5-409b-9cf0-ece6c7a1baa3\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f9c4h7"
Apr 24 14:31:29.936179 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:31:29.936122    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/e6c487a4-5df5-409b-9cf0-ece6c7a1baa3-bundle\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f9c4h7\" (UID: \"e6c487a4-5df5-409b-9cf0-ece6c7a1baa3\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f9c4h7"
Apr 24 14:31:30.036565 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:31:30.036527    2574 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-t6cd8\" (UniqueName: \"kubernetes.io/projected/e6c487a4-5df5-409b-9cf0-ece6c7a1baa3-kube-api-access-t6cd8\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f9c4h7\" (UID: \"e6c487a4-5df5-409b-9cf0-ece6c7a1baa3\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f9c4h7"
Apr 24 14:31:30.036715 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:31:30.036620    2574 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/e6c487a4-5df5-409b-9cf0-ece6c7a1baa3-util\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f9c4h7\" (UID: \"e6c487a4-5df5-409b-9cf0-ece6c7a1baa3\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f9c4h7"
Apr 24 14:31:30.036715 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:31:30.036650    2574 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/e6c487a4-5df5-409b-9cf0-ece6c7a1baa3-bundle\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f9c4h7\" (UID: \"e6c487a4-5df5-409b-9cf0-ece6c7a1baa3\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f9c4h7"
Apr 24 14:31:30.037016 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:31:30.037000    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/e6c487a4-5df5-409b-9cf0-ece6c7a1baa3-bundle\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f9c4h7\" (UID: \"e6c487a4-5df5-409b-9cf0-ece6c7a1baa3\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f9c4h7"
Apr 24 14:31:30.037076 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:31:30.037057    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/e6c487a4-5df5-409b-9cf0-ece6c7a1baa3-util\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f9c4h7\" (UID: \"e6c487a4-5df5-409b-9cf0-ece6c7a1baa3\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f9c4h7"
Apr 24 14:31:30.044322 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:31:30.044298    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-t6cd8\" (UniqueName: \"kubernetes.io/projected/e6c487a4-5df5-409b-9cf0-ece6c7a1baa3-kube-api-access-t6cd8\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f9c4h7\" (UID: \"e6c487a4-5df5-409b-9cf0-ece6c7a1baa3\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f9c4h7"
Apr 24 14:31:30.170904 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:31:30.170866    2574 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f9c4h7"
Apr 24 14:31:30.293410 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:31:30.293286    2574 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f9c4h7"]
Apr 24 14:31:30.295708 ip-10-0-136-132 kubenswrapper[2574]: W0424 14:31:30.295664    2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode6c487a4_5df5_409b_9cf0_ece6c7a1baa3.slice/crio-8070dcdeab41d45c54e68f11ef855f265df6ed559f8c7b2bd6195656b78850c6 WatchSource:0}: Error finding container 8070dcdeab41d45c54e68f11ef855f265df6ed559f8c7b2bd6195656b78850c6: Status 404 returned error can't find the container with id 8070dcdeab41d45c54e68f11ef855f265df6ed559f8c7b2bd6195656b78850c6
Apr 24 14:31:31.251912 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:31:31.251877    2574 generic.go:358] "Generic (PLEG): container finished" podID="e6c487a4-5df5-409b-9cf0-ece6c7a1baa3" containerID="e886b8293592fa6ee96d44d50ec57a69d7dc9574dac5f6ec32e8f12252e7811f" exitCode=0
Apr 24 14:31:31.252249 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:31:31.251923    2574 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f9c4h7" event={"ID":"e6c487a4-5df5-409b-9cf0-ece6c7a1baa3","Type":"ContainerDied","Data":"e886b8293592fa6ee96d44d50ec57a69d7dc9574dac5f6ec32e8f12252e7811f"}
Apr 24 14:31:31.252249 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:31:31.251944    2574 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f9c4h7" event={"ID":"e6c487a4-5df5-409b-9cf0-ece6c7a1baa3","Type":"ContainerStarted","Data":"8070dcdeab41d45c54e68f11ef855f265df6ed559f8c7b2bd6195656b78850c6"}
Apr 24 14:31:31.984706 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:31:31.984661    2574 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["cert-manager/cert-manager-webhook-587ccfb98-g9srj"]
Apr 24 14:31:31.988677 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:31:31.988652    2574 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-webhook-587ccfb98-g9srj"
Apr 24 14:31:31.990505 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:31:31.990479    2574 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager\"/\"openshift-service-ca.crt\""
Apr 24 14:31:31.990628 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:31:31.990522    2574 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager\"/\"kube-root-ca.crt\""
Apr 24 14:31:31.990628 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:31:31.990554    2574 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"cert-manager\"/\"cert-manager-webhook-dockercfg-mqbmh\""
Apr 24 14:31:31.994730 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:31:31.994705    2574 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-webhook-587ccfb98-g9srj"]
Apr 24 14:31:32.053917 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:31:32.053875    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-x2tdq\" (UniqueName: \"kubernetes.io/projected/a2e4d9d8-792e-4729-bd4c-6e3a94a3507f-kube-api-access-x2tdq\") pod \"cert-manager-webhook-587ccfb98-g9srj\" (UID: \"a2e4d9d8-792e-4729-bd4c-6e3a94a3507f\") " pod="cert-manager/cert-manager-webhook-587ccfb98-g9srj"
Apr 24 14:31:32.054102 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:31:32.053977    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/a2e4d9d8-792e-4729-bd4c-6e3a94a3507f-bound-sa-token\") pod \"cert-manager-webhook-587ccfb98-g9srj\" (UID: \"a2e4d9d8-792e-4729-bd4c-6e3a94a3507f\") " pod="cert-manager/cert-manager-webhook-587ccfb98-g9srj"
Apr 24 14:31:32.154825 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:31:32.154783    2574 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/a2e4d9d8-792e-4729-bd4c-6e3a94a3507f-bound-sa-token\") pod \"cert-manager-webhook-587ccfb98-g9srj\" (UID: \"a2e4d9d8-792e-4729-bd4c-6e3a94a3507f\") " pod="cert-manager/cert-manager-webhook-587ccfb98-g9srj"
Apr 24 14:31:32.155028 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:31:32.154900    2574 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-x2tdq\" (UniqueName: \"kubernetes.io/projected/a2e4d9d8-792e-4729-bd4c-6e3a94a3507f-kube-api-access-x2tdq\") pod \"cert-manager-webhook-587ccfb98-g9srj\" (UID: \"a2e4d9d8-792e-4729-bd4c-6e3a94a3507f\") " pod="cert-manager/cert-manager-webhook-587ccfb98-g9srj"
Apr 24 14:31:32.162992 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:31:32.162965    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-x2tdq\" (UniqueName: \"kubernetes.io/projected/a2e4d9d8-792e-4729-bd4c-6e3a94a3507f-kube-api-access-x2tdq\") pod \"cert-manager-webhook-587ccfb98-g9srj\" (UID: \"a2e4d9d8-792e-4729-bd4c-6e3a94a3507f\") " pod="cert-manager/cert-manager-webhook-587ccfb98-g9srj"
Apr 24 14:31:32.162992 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:31:32.162986    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/a2e4d9d8-792e-4729-bd4c-6e3a94a3507f-bound-sa-token\") pod \"cert-manager-webhook-587ccfb98-g9srj\" (UID: \"a2e4d9d8-792e-4729-bd4c-6e3a94a3507f\") " pod="cert-manager/cert-manager-webhook-587ccfb98-g9srj"
Apr 24 14:31:32.308712 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:31:32.308630    2574 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-webhook-587ccfb98-g9srj"
Apr 24 14:31:32.466819 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:31:32.466772    2574 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-webhook-587ccfb98-g9srj"]
Apr 24 14:31:32.469474 ip-10-0-136-132 kubenswrapper[2574]: W0424 14:31:32.469433    2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-poda2e4d9d8_792e_4729_bd4c_6e3a94a3507f.slice/crio-9a251e8077535156acf4ee7de76331fc9c57c27e14b3fdfb660dce7d97156566 WatchSource:0}: Error finding container 9a251e8077535156acf4ee7de76331fc9c57c27e14b3fdfb660dce7d97156566: Status 404 returned error can't find the container with id 9a251e8077535156acf4ee7de76331fc9c57c27e14b3fdfb660dce7d97156566
Apr 24 14:31:33.260991 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:31:33.260954    2574 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-webhook-587ccfb98-g9srj" event={"ID":"a2e4d9d8-792e-4729-bd4c-6e3a94a3507f","Type":"ContainerStarted","Data":"9a251e8077535156acf4ee7de76331fc9c57c27e14b3fdfb660dce7d97156566"}
Apr 24 14:31:34.266349 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:31:34.266305    2574 generic.go:358] "Generic (PLEG): container finished" podID="e6c487a4-5df5-409b-9cf0-ece6c7a1baa3" containerID="bbd25465c9b3515328ae4bd364a0cf146b70738a26deb1b97d5c0030f0e96cf7" exitCode=0
Apr 24 14:31:34.266765 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:31:34.266422    2574 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f9c4h7" event={"ID":"e6c487a4-5df5-409b-9cf0-ece6c7a1baa3","Type":"ContainerDied","Data":"bbd25465c9b3515328ae4bd364a0cf146b70738a26deb1b97d5c0030f0e96cf7"}
Apr 24 14:31:35.272706 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:31:35.272669    2574 generic.go:358] "Generic (PLEG): container finished" podID="e6c487a4-5df5-409b-9cf0-ece6c7a1baa3" containerID="fabc067ad1c67b8f1072a95c8aff119ac9764ca208a704cf1cec6c06c384c8e7" exitCode=0
Apr 24 14:31:35.273113 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:31:35.272754    2574 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f9c4h7" event={"ID":"e6c487a4-5df5-409b-9cf0-ece6c7a1baa3","Type":"ContainerDied","Data":"fabc067ad1c67b8f1072a95c8aff119ac9764ca208a704cf1cec6c06c384c8e7"}
Apr 24 14:31:35.274094 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:31:35.274052    2574 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-webhook-587ccfb98-g9srj" event={"ID":"a2e4d9d8-792e-4729-bd4c-6e3a94a3507f","Type":"ContainerStarted","Data":"ff62020b7eafb4e9b334592f3b9d592f78741022553f05749b77f4db6d25d793"}
Apr 24 14:31:35.274200 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:31:35.274164    2574 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="cert-manager/cert-manager-webhook-587ccfb98-g9srj"
Apr 24 14:31:35.304002 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:31:35.303868    2574 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="cert-manager/cert-manager-webhook-587ccfb98-g9srj" podStartSLOduration=1.663171956 podStartE2EDuration="4.303852588s" podCreationTimestamp="2026-04-24 14:31:31 +0000 UTC" firstStartedPulling="2026-04-24 14:31:32.471762877 +0000 UTC m=+455.203864504" lastFinishedPulling="2026-04-24 14:31:35.112443491 +0000 UTC m=+457.844545136" observedRunningTime="2026-04-24 14:31:35.303103868 +0000 UTC m=+458.035205519" watchObservedRunningTime="2026-04-24 14:31:35.303852588 +0000 UTC m=+458.035954237"
Apr 24 14:31:36.399967 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:31:36.399944    2574 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f9c4h7"
Apr 24 14:31:36.492738 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:31:36.492701    2574 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"kube-api-access-t6cd8\" (UniqueName: \"kubernetes.io/projected/e6c487a4-5df5-409b-9cf0-ece6c7a1baa3-kube-api-access-t6cd8\") pod \"e6c487a4-5df5-409b-9cf0-ece6c7a1baa3\" (UID: \"e6c487a4-5df5-409b-9cf0-ece6c7a1baa3\") "
Apr 24 14:31:36.492893 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:31:36.492750    2574 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/e6c487a4-5df5-409b-9cf0-ece6c7a1baa3-util\") pod \"e6c487a4-5df5-409b-9cf0-ece6c7a1baa3\" (UID: \"e6c487a4-5df5-409b-9cf0-ece6c7a1baa3\") "
Apr 24 14:31:36.492893 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:31:36.492783    2574 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/e6c487a4-5df5-409b-9cf0-ece6c7a1baa3-bundle\") pod \"e6c487a4-5df5-409b-9cf0-ece6c7a1baa3\" (UID: \"e6c487a4-5df5-409b-9cf0-ece6c7a1baa3\") "
Apr 24 14:31:36.493214 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:31:36.493187    2574 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e6c487a4-5df5-409b-9cf0-ece6c7a1baa3-bundle" (OuterVolumeSpecName: "bundle") pod "e6c487a4-5df5-409b-9cf0-ece6c7a1baa3" (UID: "e6c487a4-5df5-409b-9cf0-ece6c7a1baa3"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Apr 24 14:31:36.494846 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:31:36.494816    2574 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e6c487a4-5df5-409b-9cf0-ece6c7a1baa3-kube-api-access-t6cd8" (OuterVolumeSpecName: "kube-api-access-t6cd8") pod "e6c487a4-5df5-409b-9cf0-ece6c7a1baa3" (UID: "e6c487a4-5df5-409b-9cf0-ece6c7a1baa3"). InnerVolumeSpecName "kube-api-access-t6cd8". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Apr 24 14:31:36.496521 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:31:36.496495    2574 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e6c487a4-5df5-409b-9cf0-ece6c7a1baa3-util" (OuterVolumeSpecName: "util") pod "e6c487a4-5df5-409b-9cf0-ece6c7a1baa3" (UID: "e6c487a4-5df5-409b-9cf0-ece6c7a1baa3"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Apr 24 14:31:36.594152 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:31:36.594062    2574 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/e6c487a4-5df5-409b-9cf0-ece6c7a1baa3-util\") on node \"ip-10-0-136-132.ec2.internal\" DevicePath \"\""
Apr 24 14:31:36.594152 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:31:36.594096    2574 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/e6c487a4-5df5-409b-9cf0-ece6c7a1baa3-bundle\") on node \"ip-10-0-136-132.ec2.internal\" DevicePath \"\""
Apr 24 14:31:36.594152 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:31:36.594106    2574 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-t6cd8\" (UniqueName: \"kubernetes.io/projected/e6c487a4-5df5-409b-9cf0-ece6c7a1baa3-kube-api-access-t6cd8\") on node \"ip-10-0-136-132.ec2.internal\" DevicePath \"\""
Apr 24 14:31:37.283418 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:31:37.283387    2574 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f9c4h7"
Apr 24 14:31:37.283560 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:31:37.283374    2574 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f9c4h7" event={"ID":"e6c487a4-5df5-409b-9cf0-ece6c7a1baa3","Type":"ContainerDied","Data":"8070dcdeab41d45c54e68f11ef855f265df6ed559f8c7b2bd6195656b78850c6"}
Apr 24 14:31:37.283560 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:31:37.283499    2574 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="8070dcdeab41d45c54e68f11ef855f265df6ed559f8c7b2bd6195656b78850c6"
Apr 24 14:31:41.280908 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:31:41.280875    2574 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="cert-manager/cert-manager-webhook-587ccfb98-g9srj"
Apr 24 14:31:56.490067 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:31:56.490031    2574 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["openshift-lws-operator/lws-controller-manager-6d967bb649-nc7mb"]
Apr 24 14:31:56.490510 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:31:56.490424    2574 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e6c487a4-5df5-409b-9cf0-ece6c7a1baa3" containerName="extract"
Apr 24 14:31:56.490510 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:31:56.490441    2574 state_mem.go:107] "Deleted CPUSet assignment" podUID="e6c487a4-5df5-409b-9cf0-ece6c7a1baa3" containerName="extract"
Apr 24 14:31:56.490510 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:31:56.490468    2574 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e6c487a4-5df5-409b-9cf0-ece6c7a1baa3" containerName="pull"
Apr 24 14:31:56.490510 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:31:56.490477    2574 state_mem.go:107] "Deleted CPUSet assignment" podUID="e6c487a4-5df5-409b-9cf0-ece6c7a1baa3" containerName="pull"
Apr 24 14:31:56.490510 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:31:56.490489    2574 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e6c487a4-5df5-409b-9cf0-ece6c7a1baa3" containerName="util"
Apr 24 14:31:56.490510 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:31:56.490499    2574 state_mem.go:107] "Deleted CPUSet assignment" podUID="e6c487a4-5df5-409b-9cf0-ece6c7a1baa3" containerName="util"
Apr 24 14:31:56.490766 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:31:56.490590    2574 memory_manager.go:356] "RemoveStaleState removing state" podUID="e6c487a4-5df5-409b-9cf0-ece6c7a1baa3" containerName="extract"
Apr 24 14:31:56.498779 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:31:56.498761    2574 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/lws-controller-manager-6d967bb649-nc7mb"
Apr 24 14:31:56.500676 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:31:56.500654    2574 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"lws-manager-config\""
Apr 24 14:31:56.500788 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:31:56.500709    2574 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"kube-root-ca.crt\""
Apr 24 14:31:56.501212 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:31:56.501188    2574 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"lws-controller-manager-dockercfg-dkwp9\""
Apr 24 14:31:56.501212 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:31:56.501188    2574 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"webhook-server-cert\""
Apr 24 14:31:56.501465 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:31:56.501240    2574 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"openshift-service-ca.crt\""
Apr 24 14:31:56.501465 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:31:56.501409    2574 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"metrics-server-cert\""
Apr 24 14:31:56.504624 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:31:56.504601    2574 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/lws-controller-manager-6d967bb649-nc7mb"]
Apr 24 14:31:56.568271 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:31:56.568238    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-r4h99\" (UniqueName: \"kubernetes.io/projected/73112bb2-b649-468e-97cd-22458526fe47-kube-api-access-r4h99\") pod \"lws-controller-manager-6d967bb649-nc7mb\" (UID: \"73112bb2-b649-468e-97cd-22458526fe47\") " pod="openshift-lws-operator/lws-controller-manager-6d967bb649-nc7mb"
Apr 24 14:31:56.568465 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:31:56.568278    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/73112bb2-b649-468e-97cd-22458526fe47-metrics-cert\") pod \"lws-controller-manager-6d967bb649-nc7mb\" (UID: \"73112bb2-b649-468e-97cd-22458526fe47\") " pod="openshift-lws-operator/lws-controller-manager-6d967bb649-nc7mb"
Apr 24 14:31:56.568465 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:31:56.568326    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/73112bb2-b649-468e-97cd-22458526fe47-cert\") pod \"lws-controller-manager-6d967bb649-nc7mb\" (UID: \"73112bb2-b649-468e-97cd-22458526fe47\") " pod="openshift-lws-operator/lws-controller-manager-6d967bb649-nc7mb"
Apr 24 14:31:56.568465 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:31:56.568369    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/73112bb2-b649-468e-97cd-22458526fe47-manager-config\") pod \"lws-controller-manager-6d967bb649-nc7mb\" (UID: \"73112bb2-b649-468e-97cd-22458526fe47\") " pod="openshift-lws-operator/lws-controller-manager-6d967bb649-nc7mb"
Apr 24 14:31:56.669397 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:31:56.669358    2574 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-r4h99\" (UniqueName: \"kubernetes.io/projected/73112bb2-b649-468e-97cd-22458526fe47-kube-api-access-r4h99\") pod \"lws-controller-manager-6d967bb649-nc7mb\" (UID: \"73112bb2-b649-468e-97cd-22458526fe47\") " pod="openshift-lws-operator/lws-controller-manager-6d967bb649-nc7mb"
Apr 24 14:31:56.669397 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:31:56.669405    2574 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/73112bb2-b649-468e-97cd-22458526fe47-metrics-cert\") pod \"lws-controller-manager-6d967bb649-nc7mb\" (UID: \"73112bb2-b649-468e-97cd-22458526fe47\") " pod="openshift-lws-operator/lws-controller-manager-6d967bb649-nc7mb"
Apr 24 14:31:56.669666 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:31:56.669447    2574 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/73112bb2-b649-468e-97cd-22458526fe47-cert\") pod \"lws-controller-manager-6d967bb649-nc7mb\" (UID: \"73112bb2-b649-468e-97cd-22458526fe47\") " pod="openshift-lws-operator/lws-controller-manager-6d967bb649-nc7mb"
Apr 24 14:31:56.669666 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:31:56.669468    2574 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/73112bb2-b649-468e-97cd-22458526fe47-manager-config\") pod \"lws-controller-manager-6d967bb649-nc7mb\" (UID: \"73112bb2-b649-468e-97cd-22458526fe47\") " pod="openshift-lws-operator/lws-controller-manager-6d967bb649-nc7mb"
Apr 24 14:31:56.670047 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:31:56.670024    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/73112bb2-b649-468e-97cd-22458526fe47-manager-config\") pod \"lws-controller-manager-6d967bb649-nc7mb\" (UID: \"73112bb2-b649-468e-97cd-22458526fe47\") " pod="openshift-lws-operator/lws-controller-manager-6d967bb649-nc7mb"
Apr 24 14:31:56.671899 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:31:56.671869    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/73112bb2-b649-468e-97cd-22458526fe47-cert\") pod \"lws-controller-manager-6d967bb649-nc7mb\" (UID: \"73112bb2-b649-468e-97cd-22458526fe47\") " pod="openshift-lws-operator/lws-controller-manager-6d967bb649-nc7mb"
Apr 24 14:31:56.672083 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:31:56.672061    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/73112bb2-b649-468e-97cd-22458526fe47-metrics-cert\") pod \"lws-controller-manager-6d967bb649-nc7mb\" (UID: \"73112bb2-b649-468e-97cd-22458526fe47\") " pod="openshift-lws-operator/lws-controller-manager-6d967bb649-nc7mb"
Apr 24 14:31:56.677104 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:31:56.677082    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-r4h99\" (UniqueName: \"kubernetes.io/projected/73112bb2-b649-468e-97cd-22458526fe47-kube-api-access-r4h99\") pod \"lws-controller-manager-6d967bb649-nc7mb\" (UID: \"73112bb2-b649-468e-97cd-22458526fe47\") " pod="openshift-lws-operator/lws-controller-manager-6d967bb649-nc7mb"
Apr 24 14:31:56.809014 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:31:56.808914    2574 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/lws-controller-manager-6d967bb649-nc7mb"
Apr 24 14:31:56.946696 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:31:56.946510    2574 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/lws-controller-manager-6d967bb649-nc7mb"]
Apr 24 14:31:56.949241 ip-10-0-136-132 kubenswrapper[2574]: W0424 14:31:56.949210    2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod73112bb2_b649_468e_97cd_22458526fe47.slice/crio-7ef2e4ca441be637a210bb3ce66dfb0e4ed19a9dcbd3dce41672b8cbb0081659 WatchSource:0}: Error finding container 7ef2e4ca441be637a210bb3ce66dfb0e4ed19a9dcbd3dce41672b8cbb0081659: Status 404 returned error can't find the container with id 7ef2e4ca441be637a210bb3ce66dfb0e4ed19a9dcbd3dce41672b8cbb0081659
Apr 24 14:31:57.363311 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:31:57.363271    2574 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/lws-controller-manager-6d967bb649-nc7mb" event={"ID":"73112bb2-b649-468e-97cd-22458526fe47","Type":"ContainerStarted","Data":"7ef2e4ca441be637a210bb3ce66dfb0e4ed19a9dcbd3dce41672b8cbb0081659"}
Apr 24 14:31:58.261446 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:31:58.261415    2574 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["openshift-marketplace/4c892a6d2d8a57acbc49427d3a7d24e253bab25be2c607ef405d2c4835d9wg5"]
Apr 24 14:31:58.265165 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:31:58.265142    2574 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/4c892a6d2d8a57acbc49427d3a7d24e253bab25be2c607ef405d2c4835d9wg5"
Apr 24 14:31:58.266984 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:31:58.266963    2574 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"openshift-service-ca.crt\""
Apr 24 14:31:58.267427 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:31:58.267381    2574 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-marketplace\"/\"default-dockercfg-dnr8q\""
Apr 24 14:31:58.267427 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:31:58.267413    2574 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"kube-root-ca.crt\""
Apr 24 14:31:58.272162 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:31:58.272130    2574 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/4c892a6d2d8a57acbc49427d3a7d24e253bab25be2c607ef405d2c4835d9wg5"]
Apr 24 14:31:58.386623 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:31:58.386589    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-lpdwk\" (UniqueName: \"kubernetes.io/projected/57a4ce8e-7804-45de-b443-fb6c191e609b-kube-api-access-lpdwk\") pod \"4c892a6d2d8a57acbc49427d3a7d24e253bab25be2c607ef405d2c4835d9wg5\" (UID: \"57a4ce8e-7804-45de-b443-fb6c191e609b\") " pod="openshift-marketplace/4c892a6d2d8a57acbc49427d3a7d24e253bab25be2c607ef405d2c4835d9wg5"
Apr 24 14:31:58.386805 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:31:58.386648    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/57a4ce8e-7804-45de-b443-fb6c191e609b-bundle\") pod \"4c892a6d2d8a57acbc49427d3a7d24e253bab25be2c607ef405d2c4835d9wg5\" (UID: \"57a4ce8e-7804-45de-b443-fb6c191e609b\") " pod="openshift-marketplace/4c892a6d2d8a57acbc49427d3a7d24e253bab25be2c607ef405d2c4835d9wg5"
Apr 24 14:31:58.386862 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:31:58.386800    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/57a4ce8e-7804-45de-b443-fb6c191e609b-util\") pod \"4c892a6d2d8a57acbc49427d3a7d24e253bab25be2c607ef405d2c4835d9wg5\" (UID: \"57a4ce8e-7804-45de-b443-fb6c191e609b\") " pod="openshift-marketplace/4c892a6d2d8a57acbc49427d3a7d24e253bab25be2c607ef405d2c4835d9wg5"
Apr 24 14:31:58.487676 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:31:58.487639    2574 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/57a4ce8e-7804-45de-b443-fb6c191e609b-bundle\") pod \"4c892a6d2d8a57acbc49427d3a7d24e253bab25be2c607ef405d2c4835d9wg5\" (UID: \"57a4ce8e-7804-45de-b443-fb6c191e609b\") " pod="openshift-marketplace/4c892a6d2d8a57acbc49427d3a7d24e253bab25be2c607ef405d2c4835d9wg5"
Apr 24 14:31:58.487865 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:31:58.487767    2574 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/57a4ce8e-7804-45de-b443-fb6c191e609b-util\") pod \"4c892a6d2d8a57acbc49427d3a7d24e253bab25be2c607ef405d2c4835d9wg5\" (UID: \"57a4ce8e-7804-45de-b443-fb6c191e609b\") " pod="openshift-marketplace/4c892a6d2d8a57acbc49427d3a7d24e253bab25be2c607ef405d2c4835d9wg5"
Apr 24 14:31:58.487865 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:31:58.487844    2574 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-lpdwk\" (UniqueName: \"kubernetes.io/projected/57a4ce8e-7804-45de-b443-fb6c191e609b-kube-api-access-lpdwk\") pod \"4c892a6d2d8a57acbc49427d3a7d24e253bab25be2c607ef405d2c4835d9wg5\" (UID: \"57a4ce8e-7804-45de-b443-fb6c191e609b\") " pod="openshift-marketplace/4c892a6d2d8a57acbc49427d3a7d24e253bab25be2c607ef405d2c4835d9wg5"
Apr 24 14:31:58.488130 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:31:58.488101    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/57a4ce8e-7804-45de-b443-fb6c191e609b-bundle\") pod \"4c892a6d2d8a57acbc49427d3a7d24e253bab25be2c607ef405d2c4835d9wg5\" (UID: \"57a4ce8e-7804-45de-b443-fb6c191e609b\") " pod="openshift-marketplace/4c892a6d2d8a57acbc49427d3a7d24e253bab25be2c607ef405d2c4835d9wg5"
Apr 24 14:31:58.488211 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:31:58.488138    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/57a4ce8e-7804-45de-b443-fb6c191e609b-util\") pod \"4c892a6d2d8a57acbc49427d3a7d24e253bab25be2c607ef405d2c4835d9wg5\" (UID: \"57a4ce8e-7804-45de-b443-fb6c191e609b\") " pod="openshift-marketplace/4c892a6d2d8a57acbc49427d3a7d24e253bab25be2c607ef405d2c4835d9wg5"
Apr 24 14:31:58.496248 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:31:58.496214    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-lpdwk\" (UniqueName: \"kubernetes.io/projected/57a4ce8e-7804-45de-b443-fb6c191e609b-kube-api-access-lpdwk\") pod \"4c892a6d2d8a57acbc49427d3a7d24e253bab25be2c607ef405d2c4835d9wg5\" (UID: \"57a4ce8e-7804-45de-b443-fb6c191e609b\") " pod="openshift-marketplace/4c892a6d2d8a57acbc49427d3a7d24e253bab25be2c607ef405d2c4835d9wg5"
Apr 24 14:31:58.575864 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:31:58.575766    2574 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/4c892a6d2d8a57acbc49427d3a7d24e253bab25be2c607ef405d2c4835d9wg5"
Apr 24 14:31:58.711802 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:31:58.711773    2574 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/4c892a6d2d8a57acbc49427d3a7d24e253bab25be2c607ef405d2c4835d9wg5"]
Apr 24 14:31:58.962148 ip-10-0-136-132 kubenswrapper[2574]: W0424 14:31:58.962117    2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod57a4ce8e_7804_45de_b443_fb6c191e609b.slice/crio-ed10878f4ce63d5098e865ef8013c7205b7652479c4a94a870f5063dccb61d0a WatchSource:0}: Error finding container ed10878f4ce63d5098e865ef8013c7205b7652479c4a94a870f5063dccb61d0a: Status 404 returned error can't find the container with id ed10878f4ce63d5098e865ef8013c7205b7652479c4a94a870f5063dccb61d0a
Apr 24 14:31:59.073541 ip-10-0-136-132 kubenswrapper[2574]: E0424 14:31:59.073508    2574 cadvisor_stats_provider.go:525] "Partial failure issuing cadvisor.ContainerInfoV2" err="partial failures: [\"/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod57a4ce8e_7804_45de_b443_fb6c191e609b.slice/crio-a9f0eedd96d09367a69cfca9f42bb53c72feedfde35f12e6ca1a871f8c06fb98.scope\": RecentStats: unable to find data in memory cache]"
Apr 24 14:31:59.374720 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:31:59.374685    2574 generic.go:358] "Generic (PLEG): container finished" podID="57a4ce8e-7804-45de-b443-fb6c191e609b" containerID="a9f0eedd96d09367a69cfca9f42bb53c72feedfde35f12e6ca1a871f8c06fb98" exitCode=0
Apr 24 14:31:59.375166 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:31:59.374779    2574 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/4c892a6d2d8a57acbc49427d3a7d24e253bab25be2c607ef405d2c4835d9wg5" event={"ID":"57a4ce8e-7804-45de-b443-fb6c191e609b","Type":"ContainerDied","Data":"a9f0eedd96d09367a69cfca9f42bb53c72feedfde35f12e6ca1a871f8c06fb98"}
Apr 24 14:31:59.375166 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:31:59.374815    2574 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/4c892a6d2d8a57acbc49427d3a7d24e253bab25be2c607ef405d2c4835d9wg5" event={"ID":"57a4ce8e-7804-45de-b443-fb6c191e609b","Type":"ContainerStarted","Data":"ed10878f4ce63d5098e865ef8013c7205b7652479c4a94a870f5063dccb61d0a"}
Apr 24 14:31:59.376181 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:31:59.376145    2574 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/lws-controller-manager-6d967bb649-nc7mb" event={"ID":"73112bb2-b649-468e-97cd-22458526fe47","Type":"ContainerStarted","Data":"a05e48f25f4f02666a847b18b4bb6c4b9fff5be107595133bbe3a046cb4fc86b"}
Apr 24 14:31:59.376305 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:31:59.376285    2574 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-lws-operator/lws-controller-manager-6d967bb649-nc7mb"
Apr 24 14:31:59.404153 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:31:59.403671    2574 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-lws-operator/lws-controller-manager-6d967bb649-nc7mb" podStartSLOduration=1.330339305 podStartE2EDuration="3.403617798s" podCreationTimestamp="2026-04-24 14:31:56 +0000 UTC" firstStartedPulling="2026-04-24 14:31:56.951280981 +0000 UTC m=+479.683382610" lastFinishedPulling="2026-04-24 14:31:59.024559476 +0000 UTC m=+481.756661103" observedRunningTime="2026-04-24 14:31:59.403578807 +0000 UTC m=+482.135680459" watchObservedRunningTime="2026-04-24 14:31:59.403617798 +0000 UTC m=+482.135719450"
Apr 24 14:32:01.384680 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:32:01.384646    2574 generic.go:358] "Generic (PLEG): container finished" podID="57a4ce8e-7804-45de-b443-fb6c191e609b" containerID="146c958a53714af5bf634572c6c92dde776d2b2bb15490ff9e2f6b88995005c8" exitCode=0
Apr 24 14:32:01.385048 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:32:01.384738    2574 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/4c892a6d2d8a57acbc49427d3a7d24e253bab25be2c607ef405d2c4835d9wg5" event={"ID":"57a4ce8e-7804-45de-b443-fb6c191e609b","Type":"ContainerDied","Data":"146c958a53714af5bf634572c6c92dde776d2b2bb15490ff9e2f6b88995005c8"}
Apr 24 14:32:02.390665 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:32:02.390634    2574 generic.go:358] "Generic (PLEG): container finished" podID="57a4ce8e-7804-45de-b443-fb6c191e609b" containerID="f2faad01d21a21eb0bd1126e15e840a61a0f0fbfa77690b8cf2a9870077b307a" exitCode=0
Apr 24 14:32:02.391027 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:32:02.390714    2574 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/4c892a6d2d8a57acbc49427d3a7d24e253bab25be2c607ef405d2c4835d9wg5" event={"ID":"57a4ce8e-7804-45de-b443-fb6c191e609b","Type":"ContainerDied","Data":"f2faad01d21a21eb0bd1126e15e840a61a0f0fbfa77690b8cf2a9870077b307a"}
Apr 24 14:32:03.525088 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:32:03.525064    2574 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/4c892a6d2d8a57acbc49427d3a7d24e253bab25be2c607ef405d2c4835d9wg5"
Apr 24 14:32:03.635169 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:32:03.635122    2574 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/57a4ce8e-7804-45de-b443-fb6c191e609b-util\") pod \"57a4ce8e-7804-45de-b443-fb6c191e609b\" (UID: \"57a4ce8e-7804-45de-b443-fb6c191e609b\") "
Apr 24 14:32:03.635326 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:32:03.635204    2574 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"kube-api-access-lpdwk\" (UniqueName: \"kubernetes.io/projected/57a4ce8e-7804-45de-b443-fb6c191e609b-kube-api-access-lpdwk\") pod \"57a4ce8e-7804-45de-b443-fb6c191e609b\" (UID: \"57a4ce8e-7804-45de-b443-fb6c191e609b\") "
Apr 24 14:32:03.635326 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:32:03.635230    2574 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/57a4ce8e-7804-45de-b443-fb6c191e609b-bundle\") pod \"57a4ce8e-7804-45de-b443-fb6c191e609b\" (UID: \"57a4ce8e-7804-45de-b443-fb6c191e609b\") "
Apr 24 14:32:03.636079 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:32:03.636051    2574 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/57a4ce8e-7804-45de-b443-fb6c191e609b-bundle" (OuterVolumeSpecName: "bundle") pod "57a4ce8e-7804-45de-b443-fb6c191e609b" (UID: "57a4ce8e-7804-45de-b443-fb6c191e609b"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Apr 24 14:32:03.637363 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:32:03.637321    2574 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/57a4ce8e-7804-45de-b443-fb6c191e609b-kube-api-access-lpdwk" (OuterVolumeSpecName: "kube-api-access-lpdwk") pod "57a4ce8e-7804-45de-b443-fb6c191e609b" (UID: "57a4ce8e-7804-45de-b443-fb6c191e609b"). InnerVolumeSpecName "kube-api-access-lpdwk". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Apr 24 14:32:03.640454 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:32:03.640428    2574 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/57a4ce8e-7804-45de-b443-fb6c191e609b-util" (OuterVolumeSpecName: "util") pod "57a4ce8e-7804-45de-b443-fb6c191e609b" (UID: "57a4ce8e-7804-45de-b443-fb6c191e609b"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Apr 24 14:32:03.736237 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:32:03.736146    2574 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/57a4ce8e-7804-45de-b443-fb6c191e609b-util\") on node \"ip-10-0-136-132.ec2.internal\" DevicePath \"\""
Apr 24 14:32:03.736237 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:32:03.736178    2574 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-lpdwk\" (UniqueName: \"kubernetes.io/projected/57a4ce8e-7804-45de-b443-fb6c191e609b-kube-api-access-lpdwk\") on node \"ip-10-0-136-132.ec2.internal\" DevicePath \"\""
Apr 24 14:32:03.736237 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:32:03.736192    2574 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/57a4ce8e-7804-45de-b443-fb6c191e609b-bundle\") on node \"ip-10-0-136-132.ec2.internal\" DevicePath \"\""
Apr 24 14:32:04.399203 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:32:04.399162    2574 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/4c892a6d2d8a57acbc49427d3a7d24e253bab25be2c607ef405d2c4835d9wg5" event={"ID":"57a4ce8e-7804-45de-b443-fb6c191e609b","Type":"ContainerDied","Data":"ed10878f4ce63d5098e865ef8013c7205b7652479c4a94a870f5063dccb61d0a"}
Apr 24 14:32:04.399203 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:32:04.399208    2574 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="ed10878f4ce63d5098e865ef8013c7205b7652479c4a94a870f5063dccb61d0a"
Apr 24 14:32:04.399439 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:32:04.399176    2574 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/4c892a6d2d8a57acbc49427d3a7d24e253bab25be2c607ef405d2c4835d9wg5"
Apr 24 14:32:10.381440 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:32:10.381411    2574 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-lws-operator/lws-controller-manager-6d967bb649-nc7mb"
Apr 24 14:32:13.416597 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:32:13.416565    2574 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["openshift-marketplace/7797e60c2c2aaccf623f93d365c9b5c5cd3662e5f903c80e749ff805eb6hwmz"]
Apr 24 14:32:13.417247 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:32:13.416942    2574 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="57a4ce8e-7804-45de-b443-fb6c191e609b" containerName="util"
Apr 24 14:32:13.417247 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:32:13.416956    2574 state_mem.go:107] "Deleted CPUSet assignment" podUID="57a4ce8e-7804-45de-b443-fb6c191e609b" containerName="util"
Apr 24 14:32:13.417247 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:32:13.416975    2574 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="57a4ce8e-7804-45de-b443-fb6c191e609b" containerName="pull"
Apr 24 14:32:13.417247 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:32:13.416981    2574 state_mem.go:107] "Deleted CPUSet assignment" podUID="57a4ce8e-7804-45de-b443-fb6c191e609b" containerName="pull"
Apr 24 14:32:13.417247 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:32:13.416988    2574 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="57a4ce8e-7804-45de-b443-fb6c191e609b" containerName="extract"
Apr 24 14:32:13.417247 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:32:13.416994    2574 state_mem.go:107] "Deleted CPUSet assignment" podUID="57a4ce8e-7804-45de-b443-fb6c191e609b" containerName="extract"
Apr 24 14:32:13.417247 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:32:13.417068    2574 memory_manager.go:356] "RemoveStaleState removing state" podUID="57a4ce8e-7804-45de-b443-fb6c191e609b" containerName="extract"
Apr 24 14:32:13.419945 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:32:13.419928    2574 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/7797e60c2c2aaccf623f93d365c9b5c5cd3662e5f903c80e749ff805eb6hwmz"
Apr 24 14:32:13.421975 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:32:13.421956    2574 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"kube-root-ca.crt\""
Apr 24 14:32:13.422105 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:32:13.421976    2574 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"openshift-service-ca.crt\""
Apr 24 14:32:13.422105 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:32:13.422002    2574 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-marketplace\"/\"default-dockercfg-dnr8q\""
Apr 24 14:32:13.429324 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:32:13.429281    2574 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/7797e60c2c2aaccf623f93d365c9b5c5cd3662e5f903c80e749ff805eb6hwmz"]
Apr 24 14:32:13.517838 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:32:13.517800    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/41500644-259f-46dc-9f95-b4613614e674-util\") pod \"7797e60c2c2aaccf623f93d365c9b5c5cd3662e5f903c80e749ff805eb6hwmz\" (UID: \"41500644-259f-46dc-9f95-b4613614e674\") " pod="openshift-marketplace/7797e60c2c2aaccf623f93d365c9b5c5cd3662e5f903c80e749ff805eb6hwmz"
Apr 24 14:32:13.518007 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:32:13.517880    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/41500644-259f-46dc-9f95-b4613614e674-bundle\") pod \"7797e60c2c2aaccf623f93d365c9b5c5cd3662e5f903c80e749ff805eb6hwmz\" (UID: \"41500644-259f-46dc-9f95-b4613614e674\") " pod="openshift-marketplace/7797e60c2c2aaccf623f93d365c9b5c5cd3662e5f903c80e749ff805eb6hwmz"
Apr 24 14:32:13.518007 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:32:13.517906    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2vrvn\" (UniqueName: \"kubernetes.io/projected/41500644-259f-46dc-9f95-b4613614e674-kube-api-access-2vrvn\") pod \"7797e60c2c2aaccf623f93d365c9b5c5cd3662e5f903c80e749ff805eb6hwmz\" (UID: \"41500644-259f-46dc-9f95-b4613614e674\") " pod="openshift-marketplace/7797e60c2c2aaccf623f93d365c9b5c5cd3662e5f903c80e749ff805eb6hwmz"
Apr 24 14:32:13.619411 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:32:13.619356    2574 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/41500644-259f-46dc-9f95-b4613614e674-util\") pod \"7797e60c2c2aaccf623f93d365c9b5c5cd3662e5f903c80e749ff805eb6hwmz\" (UID: \"41500644-259f-46dc-9f95-b4613614e674\") " pod="openshift-marketplace/7797e60c2c2aaccf623f93d365c9b5c5cd3662e5f903c80e749ff805eb6hwmz"
Apr 24 14:32:13.619580 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:32:13.619480    2574 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/41500644-259f-46dc-9f95-b4613614e674-bundle\") pod \"7797e60c2c2aaccf623f93d365c9b5c5cd3662e5f903c80e749ff805eb6hwmz\" (UID: \"41500644-259f-46dc-9f95-b4613614e674\") " pod="openshift-marketplace/7797e60c2c2aaccf623f93d365c9b5c5cd3662e5f903c80e749ff805eb6hwmz"
Apr 24 14:32:13.619580 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:32:13.619508    2574 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-2vrvn\" (UniqueName: \"kubernetes.io/projected/41500644-259f-46dc-9f95-b4613614e674-kube-api-access-2vrvn\") pod \"7797e60c2c2aaccf623f93d365c9b5c5cd3662e5f903c80e749ff805eb6hwmz\" (UID: \"41500644-259f-46dc-9f95-b4613614e674\") " pod="openshift-marketplace/7797e60c2c2aaccf623f93d365c9b5c5cd3662e5f903c80e749ff805eb6hwmz"
Apr 24 14:32:13.619684 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:32:13.619662    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/41500644-259f-46dc-9f95-b4613614e674-util\") pod \"7797e60c2c2aaccf623f93d365c9b5c5cd3662e5f903c80e749ff805eb6hwmz\" (UID: \"41500644-259f-46dc-9f95-b4613614e674\") " pod="openshift-marketplace/7797e60c2c2aaccf623f93d365c9b5c5cd3662e5f903c80e749ff805eb6hwmz"
Apr 24 14:32:13.619876 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:32:13.619850    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/41500644-259f-46dc-9f95-b4613614e674-bundle\") pod \"7797e60c2c2aaccf623f93d365c9b5c5cd3662e5f903c80e749ff805eb6hwmz\" (UID: \"41500644-259f-46dc-9f95-b4613614e674\") " pod="openshift-marketplace/7797e60c2c2aaccf623f93d365c9b5c5cd3662e5f903c80e749ff805eb6hwmz"
Apr 24 14:32:13.639917 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:32:13.639890    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-2vrvn\" (UniqueName: \"kubernetes.io/projected/41500644-259f-46dc-9f95-b4613614e674-kube-api-access-2vrvn\") pod \"7797e60c2c2aaccf623f93d365c9b5c5cd3662e5f903c80e749ff805eb6hwmz\" (UID: \"41500644-259f-46dc-9f95-b4613614e674\") " pod="openshift-marketplace/7797e60c2c2aaccf623f93d365c9b5c5cd3662e5f903c80e749ff805eb6hwmz"
Apr 24 14:32:13.730830 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:32:13.730742    2574 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/7797e60c2c2aaccf623f93d365c9b5c5cd3662e5f903c80e749ff805eb6hwmz"
Apr 24 14:32:13.873642 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:32:13.873615    2574 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/7797e60c2c2aaccf623f93d365c9b5c5cd3662e5f903c80e749ff805eb6hwmz"]
Apr 24 14:32:13.876124 ip-10-0-136-132 kubenswrapper[2574]: W0424 14:32:13.876094    2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod41500644_259f_46dc_9f95_b4613614e674.slice/crio-b3285bcf63d517ed87fbdecd777c6f5a593147c4c522c48e51c590ad242468fb WatchSource:0}: Error finding container b3285bcf63d517ed87fbdecd777c6f5a593147c4c522c48e51c590ad242468fb: Status 404 returned error can't find the container with id b3285bcf63d517ed87fbdecd777c6f5a593147c4c522c48e51c590ad242468fb
Apr 24 14:32:14.438744 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:32:14.438708    2574 generic.go:358] "Generic (PLEG): container finished" podID="41500644-259f-46dc-9f95-b4613614e674" containerID="d6b772992f89cf4f7649ab9c8657f917f95af1f6f956d78ae84ee9fe649ab723" exitCode=0
Apr 24 14:32:14.439115 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:32:14.438795    2574 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/7797e60c2c2aaccf623f93d365c9b5c5cd3662e5f903c80e749ff805eb6hwmz" event={"ID":"41500644-259f-46dc-9f95-b4613614e674","Type":"ContainerDied","Data":"d6b772992f89cf4f7649ab9c8657f917f95af1f6f956d78ae84ee9fe649ab723"}
Apr 24 14:32:14.439115 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:32:14.438827    2574 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/7797e60c2c2aaccf623f93d365c9b5c5cd3662e5f903c80e749ff805eb6hwmz" event={"ID":"41500644-259f-46dc-9f95-b4613614e674","Type":"ContainerStarted","Data":"b3285bcf63d517ed87fbdecd777c6f5a593147c4c522c48e51c590ad242468fb"}
Apr 24 14:32:15.443699 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:32:15.443667    2574 generic.go:358] "Generic (PLEG): container finished" podID="41500644-259f-46dc-9f95-b4613614e674" containerID="ae5c76778ab28ea640d06abb350701d5b24f89ce3095c19a67c690995c032ee8" exitCode=0
Apr 24 14:32:15.444148 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:32:15.443747    2574 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/7797e60c2c2aaccf623f93d365c9b5c5cd3662e5f903c80e749ff805eb6hwmz" event={"ID":"41500644-259f-46dc-9f95-b4613614e674","Type":"ContainerDied","Data":"ae5c76778ab28ea640d06abb350701d5b24f89ce3095c19a67c690995c032ee8"}
Apr 24 14:32:16.449824 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:32:16.449791    2574 generic.go:358] "Generic (PLEG): container finished" podID="41500644-259f-46dc-9f95-b4613614e674" containerID="c9d069ec1ee1ca81ca4e21ffee65b1758d538a93f1f77db8d533c31ca7fef8ad" exitCode=0
Apr 24 14:32:16.449824 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:32:16.449829    2574 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/7797e60c2c2aaccf623f93d365c9b5c5cd3662e5f903c80e749ff805eb6hwmz" event={"ID":"41500644-259f-46dc-9f95-b4613614e674","Type":"ContainerDied","Data":"c9d069ec1ee1ca81ca4e21ffee65b1758d538a93f1f77db8d533c31ca7fef8ad"}
Apr 24 14:32:17.585028 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:32:17.585005    2574 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/7797e60c2c2aaccf623f93d365c9b5c5cd3662e5f903c80e749ff805eb6hwmz"
Apr 24 14:32:17.756603 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:32:17.756518    2574 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/41500644-259f-46dc-9f95-b4613614e674-util\") pod \"41500644-259f-46dc-9f95-b4613614e674\" (UID: \"41500644-259f-46dc-9f95-b4613614e674\") "
Apr 24 14:32:17.756603 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:32:17.756573    2574 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"kube-api-access-2vrvn\" (UniqueName: \"kubernetes.io/projected/41500644-259f-46dc-9f95-b4613614e674-kube-api-access-2vrvn\") pod \"41500644-259f-46dc-9f95-b4613614e674\" (UID: \"41500644-259f-46dc-9f95-b4613614e674\") "
Apr 24 14:32:17.756824 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:32:17.756634    2574 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/41500644-259f-46dc-9f95-b4613614e674-bundle\") pod \"41500644-259f-46dc-9f95-b4613614e674\" (UID: \"41500644-259f-46dc-9f95-b4613614e674\") "
Apr 24 14:32:17.757581 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:32:17.757552    2574 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/41500644-259f-46dc-9f95-b4613614e674-bundle" (OuterVolumeSpecName: "bundle") pod "41500644-259f-46dc-9f95-b4613614e674" (UID: "41500644-259f-46dc-9f95-b4613614e674"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Apr 24 14:32:17.758677 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:32:17.758652    2574 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/41500644-259f-46dc-9f95-b4613614e674-kube-api-access-2vrvn" (OuterVolumeSpecName: "kube-api-access-2vrvn") pod "41500644-259f-46dc-9f95-b4613614e674" (UID: "41500644-259f-46dc-9f95-b4613614e674"). InnerVolumeSpecName "kube-api-access-2vrvn". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Apr 24 14:32:17.763848 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:32:17.763817    2574 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/41500644-259f-46dc-9f95-b4613614e674-util" (OuterVolumeSpecName: "util") pod "41500644-259f-46dc-9f95-b4613614e674" (UID: "41500644-259f-46dc-9f95-b4613614e674"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Apr 24 14:32:17.857920 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:32:17.857878    2574 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/41500644-259f-46dc-9f95-b4613614e674-util\") on node \"ip-10-0-136-132.ec2.internal\" DevicePath \"\""
Apr 24 14:32:17.857920 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:32:17.857910    2574 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-2vrvn\" (UniqueName: \"kubernetes.io/projected/41500644-259f-46dc-9f95-b4613614e674-kube-api-access-2vrvn\") on node \"ip-10-0-136-132.ec2.internal\" DevicePath \"\""
Apr 24 14:32:17.857920 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:32:17.857921    2574 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/41500644-259f-46dc-9f95-b4613614e674-bundle\") on node \"ip-10-0-136-132.ec2.internal\" DevicePath \"\""
Apr 24 14:32:18.459442 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:32:18.459404    2574 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/7797e60c2c2aaccf623f93d365c9b5c5cd3662e5f903c80e749ff805eb6hwmz" event={"ID":"41500644-259f-46dc-9f95-b4613614e674","Type":"ContainerDied","Data":"b3285bcf63d517ed87fbdecd777c6f5a593147c4c522c48e51c590ad242468fb"}
Apr 24 14:32:18.459442 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:32:18.459445    2574 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="b3285bcf63d517ed87fbdecd777c6f5a593147c4c522c48e51c590ad242468fb"
Apr 24 14:32:18.459691 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:32:18.459422    2574 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/7797e60c2c2aaccf623f93d365c9b5c5cd3662e5f903c80e749ff805eb6hwmz"
Apr 24 14:33:16.872779 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:33:16.872744    2574 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["openshift-marketplace/ef1189e9861fa30b9414ceb420c2d78e85403a7e10097f37afdacfec88tgbwr"]
Apr 24 14:33:16.873215 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:33:16.873099    2574 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="41500644-259f-46dc-9f95-b4613614e674" containerName="extract"
Apr 24 14:33:16.873215 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:33:16.873109    2574 state_mem.go:107] "Deleted CPUSet assignment" podUID="41500644-259f-46dc-9f95-b4613614e674" containerName="extract"
Apr 24 14:33:16.873215 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:33:16.873124    2574 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="41500644-259f-46dc-9f95-b4613614e674" containerName="util"
Apr 24 14:33:16.873215 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:33:16.873130    2574 state_mem.go:107] "Deleted CPUSet assignment" podUID="41500644-259f-46dc-9f95-b4613614e674" containerName="util"
Apr 24 14:33:16.873215 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:33:16.873140    2574 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="41500644-259f-46dc-9f95-b4613614e674" containerName="pull"
Apr 24 14:33:16.873215 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:33:16.873146    2574 state_mem.go:107] "Deleted CPUSet assignment" podUID="41500644-259f-46dc-9f95-b4613614e674" containerName="pull"
Apr 24 14:33:16.873215 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:33:16.873202    2574 memory_manager.go:356] "RemoveStaleState removing state" podUID="41500644-259f-46dc-9f95-b4613614e674" containerName="extract"
Apr 24 14:33:16.875239 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:33:16.875219    2574 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/ef1189e9861fa30b9414ceb420c2d78e85403a7e10097f37afdacfec88tgbwr"
Apr 24 14:33:16.877213 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:33:16.877193    2574 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"openshift-service-ca.crt\""
Apr 24 14:33:16.877306 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:33:16.877238    2574 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-marketplace\"/\"default-dockercfg-dnr8q\""
Apr 24 14:33:16.877386 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:33:16.877316    2574 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"kube-root-ca.crt\""
Apr 24 14:33:16.883088 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:33:16.883064    2574 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/ef1189e9861fa30b9414ceb420c2d78e85403a7e10097f37afdacfec88tgbwr"]
Apr 24 14:33:16.965664 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:33:16.965630    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/c3f2eaf3-730c-4f91-8700-498b592681f3-bundle\") pod \"ef1189e9861fa30b9414ceb420c2d78e85403a7e10097f37afdacfec88tgbwr\" (UID: \"c3f2eaf3-730c-4f91-8700-498b592681f3\") " pod="openshift-marketplace/ef1189e9861fa30b9414ceb420c2d78e85403a7e10097f37afdacfec88tgbwr"
Apr 24 14:33:16.965827 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:33:16.965686    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/c3f2eaf3-730c-4f91-8700-498b592681f3-util\") pod \"ef1189e9861fa30b9414ceb420c2d78e85403a7e10097f37afdacfec88tgbwr\" (UID: \"c3f2eaf3-730c-4f91-8700-498b592681f3\") " pod="openshift-marketplace/ef1189e9861fa30b9414ceb420c2d78e85403a7e10097f37afdacfec88tgbwr"
Apr 24 14:33:16.965827 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:33:16.965709    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-f42vn\" (UniqueName: \"kubernetes.io/projected/c3f2eaf3-730c-4f91-8700-498b592681f3-kube-api-access-f42vn\") pod \"ef1189e9861fa30b9414ceb420c2d78e85403a7e10097f37afdacfec88tgbwr\" (UID: \"c3f2eaf3-730c-4f91-8700-498b592681f3\") " pod="openshift-marketplace/ef1189e9861fa30b9414ceb420c2d78e85403a7e10097f37afdacfec88tgbwr"
Apr 24 14:33:16.980854 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:33:16.980825    2574 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["openshift-marketplace/d54990151e6a3040f48962708f776cca7120e97625eb3a76d6fde2767bjplv4"]
Apr 24 14:33:16.983275 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:33:16.983260    2574 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/d54990151e6a3040f48962708f776cca7120e97625eb3a76d6fde2767bjplv4"
Apr 24 14:33:16.993114 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:33:16.993089    2574 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/d54990151e6a3040f48962708f776cca7120e97625eb3a76d6fde2767bjplv4"]
Apr 24 14:33:17.067082 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:33:17.067046    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/fbd7920a-c23d-4bf8-b4af-25408c94e843-bundle\") pod \"d54990151e6a3040f48962708f776cca7120e97625eb3a76d6fde2767bjplv4\" (UID: \"fbd7920a-c23d-4bf8-b4af-25408c94e843\") " pod="openshift-marketplace/d54990151e6a3040f48962708f776cca7120e97625eb3a76d6fde2767bjplv4"
Apr 24 14:33:17.067261 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:33:17.067104    2574 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/c3f2eaf3-730c-4f91-8700-498b592681f3-bundle\") pod \"ef1189e9861fa30b9414ceb420c2d78e85403a7e10097f37afdacfec88tgbwr\" (UID: \"c3f2eaf3-730c-4f91-8700-498b592681f3\") " pod="openshift-marketplace/ef1189e9861fa30b9414ceb420c2d78e85403a7e10097f37afdacfec88tgbwr"
Apr 24 14:33:17.067261 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:33:17.067226    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/fbd7920a-c23d-4bf8-b4af-25408c94e843-util\") pod \"d54990151e6a3040f48962708f776cca7120e97625eb3a76d6fde2767bjplv4\" (UID: \"fbd7920a-c23d-4bf8-b4af-25408c94e843\") " pod="openshift-marketplace/d54990151e6a3040f48962708f776cca7120e97625eb3a76d6fde2767bjplv4"
Apr 24 14:33:17.067261 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:33:17.067254    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vj7cf\" (UniqueName: \"kubernetes.io/projected/fbd7920a-c23d-4bf8-b4af-25408c94e843-kube-api-access-vj7cf\") pod \"d54990151e6a3040f48962708f776cca7120e97625eb3a76d6fde2767bjplv4\" (UID: \"fbd7920a-c23d-4bf8-b4af-25408c94e843\") " pod="openshift-marketplace/d54990151e6a3040f48962708f776cca7120e97625eb3a76d6fde2767bjplv4"
Apr 24 14:33:17.067413 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:33:17.067289    2574 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/c3f2eaf3-730c-4f91-8700-498b592681f3-util\") pod \"ef1189e9861fa30b9414ceb420c2d78e85403a7e10097f37afdacfec88tgbwr\" (UID: \"c3f2eaf3-730c-4f91-8700-498b592681f3\") " pod="openshift-marketplace/ef1189e9861fa30b9414ceb420c2d78e85403a7e10097f37afdacfec88tgbwr"
Apr 24 14:33:17.067413 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:33:17.067318    2574 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-f42vn\" (UniqueName: \"kubernetes.io/projected/c3f2eaf3-730c-4f91-8700-498b592681f3-kube-api-access-f42vn\") pod \"ef1189e9861fa30b9414ceb420c2d78e85403a7e10097f37afdacfec88tgbwr\" (UID: \"c3f2eaf3-730c-4f91-8700-498b592681f3\") " pod="openshift-marketplace/ef1189e9861fa30b9414ceb420c2d78e85403a7e10097f37afdacfec88tgbwr"
Apr 24 14:33:17.067496 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:33:17.067481    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/c3f2eaf3-730c-4f91-8700-498b592681f3-bundle\") pod \"ef1189e9861fa30b9414ceb420c2d78e85403a7e10097f37afdacfec88tgbwr\" (UID: \"c3f2eaf3-730c-4f91-8700-498b592681f3\") " pod="openshift-marketplace/ef1189e9861fa30b9414ceb420c2d78e85403a7e10097f37afdacfec88tgbwr"
Apr 24 14:33:17.067634 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:33:17.067617    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/c3f2eaf3-730c-4f91-8700-498b592681f3-util\") pod \"ef1189e9861fa30b9414ceb420c2d78e85403a7e10097f37afdacfec88tgbwr\" (UID: \"c3f2eaf3-730c-4f91-8700-498b592681f3\") " pod="openshift-marketplace/ef1189e9861fa30b9414ceb420c2d78e85403a7e10097f37afdacfec88tgbwr"
Apr 24 14:33:17.080255 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:33:17.080220    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-f42vn\" (UniqueName: \"kubernetes.io/projected/c3f2eaf3-730c-4f91-8700-498b592681f3-kube-api-access-f42vn\") pod \"ef1189e9861fa30b9414ceb420c2d78e85403a7e10097f37afdacfec88tgbwr\" (UID: \"c3f2eaf3-730c-4f91-8700-498b592681f3\") " pod="openshift-marketplace/ef1189e9861fa30b9414ceb420c2d78e85403a7e10097f37afdacfec88tgbwr"
Apr 24 14:33:17.088076 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:33:17.088050    2574 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["openshift-marketplace/5b107722e7a95ad7441cca2226d3d656a8bf76dc65e15066bc01f8e503khl8s"]
Apr 24 14:33:17.091282 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:33:17.091267    2574 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/5b107722e7a95ad7441cca2226d3d656a8bf76dc65e15066bc01f8e503khl8s"
Apr 24 14:33:17.099879 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:33:17.099854    2574 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/5b107722e7a95ad7441cca2226d3d656a8bf76dc65e15066bc01f8e503khl8s"]
Apr 24 14:33:17.168733 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:33:17.168637    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hhtwd\" (UniqueName: \"kubernetes.io/projected/6ab86bf6-5147-4edc-b660-9f144520f444-kube-api-access-hhtwd\") pod \"5b107722e7a95ad7441cca2226d3d656a8bf76dc65e15066bc01f8e503khl8s\" (UID: \"6ab86bf6-5147-4edc-b660-9f144520f444\") " pod="openshift-marketplace/5b107722e7a95ad7441cca2226d3d656a8bf76dc65e15066bc01f8e503khl8s"
Apr 24 14:33:17.168733 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:33:17.168685    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/6ab86bf6-5147-4edc-b660-9f144520f444-util\") pod \"5b107722e7a95ad7441cca2226d3d656a8bf76dc65e15066bc01f8e503khl8s\" (UID: \"6ab86bf6-5147-4edc-b660-9f144520f444\") " pod="openshift-marketplace/5b107722e7a95ad7441cca2226d3d656a8bf76dc65e15066bc01f8e503khl8s"
Apr 24 14:33:17.168925 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:33:17.168760    2574 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/fbd7920a-c23d-4bf8-b4af-25408c94e843-util\") pod \"d54990151e6a3040f48962708f776cca7120e97625eb3a76d6fde2767bjplv4\" (UID: \"fbd7920a-c23d-4bf8-b4af-25408c94e843\") " pod="openshift-marketplace/d54990151e6a3040f48962708f776cca7120e97625eb3a76d6fde2767bjplv4"
Apr 24 14:33:17.168925 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:33:17.168825    2574 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-vj7cf\" (UniqueName: \"kubernetes.io/projected/fbd7920a-c23d-4bf8-b4af-25408c94e843-kube-api-access-vj7cf\") pod \"d54990151e6a3040f48962708f776cca7120e97625eb3a76d6fde2767bjplv4\" (UID: \"fbd7920a-c23d-4bf8-b4af-25408c94e843\") " pod="openshift-marketplace/d54990151e6a3040f48962708f776cca7120e97625eb3a76d6fde2767bjplv4"
Apr 24 14:33:17.168925 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:33:17.168872    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/6ab86bf6-5147-4edc-b660-9f144520f444-bundle\") pod \"5b107722e7a95ad7441cca2226d3d656a8bf76dc65e15066bc01f8e503khl8s\" (UID: \"6ab86bf6-5147-4edc-b660-9f144520f444\") " pod="openshift-marketplace/5b107722e7a95ad7441cca2226d3d656a8bf76dc65e15066bc01f8e503khl8s"
Apr 24 14:33:17.169041 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:33:17.168933    2574 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/fbd7920a-c23d-4bf8-b4af-25408c94e843-bundle\") pod \"d54990151e6a3040f48962708f776cca7120e97625eb3a76d6fde2767bjplv4\" (UID: \"fbd7920a-c23d-4bf8-b4af-25408c94e843\") " pod="openshift-marketplace/d54990151e6a3040f48962708f776cca7120e97625eb3a76d6fde2767bjplv4"
Apr 24 14:33:17.169086 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:33:17.169068    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/fbd7920a-c23d-4bf8-b4af-25408c94e843-util\") pod \"d54990151e6a3040f48962708f776cca7120e97625eb3a76d6fde2767bjplv4\" (UID: \"fbd7920a-c23d-4bf8-b4af-25408c94e843\") " pod="openshift-marketplace/d54990151e6a3040f48962708f776cca7120e97625eb3a76d6fde2767bjplv4"
Apr 24 14:33:17.169199 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:33:17.169185    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/fbd7920a-c23d-4bf8-b4af-25408c94e843-bundle\") pod \"d54990151e6a3040f48962708f776cca7120e97625eb3a76d6fde2767bjplv4\" (UID: \"fbd7920a-c23d-4bf8-b4af-25408c94e843\") " pod="openshift-marketplace/d54990151e6a3040f48962708f776cca7120e97625eb3a76d6fde2767bjplv4"
Apr 24 14:33:17.179197 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:33:17.179170    2574 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["openshift-marketplace/309c32b5cc4cbd8148882f0ed2adbe6c47ee2761cc0c22c627755a3c30lcv9z"]
Apr 24 14:33:17.181786 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:33:17.181767    2574 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/309c32b5cc4cbd8148882f0ed2adbe6c47ee2761cc0c22c627755a3c30lcv9z"
Apr 24 14:33:17.184136 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:33:17.184113    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-vj7cf\" (UniqueName: \"kubernetes.io/projected/fbd7920a-c23d-4bf8-b4af-25408c94e843-kube-api-access-vj7cf\") pod \"d54990151e6a3040f48962708f776cca7120e97625eb3a76d6fde2767bjplv4\" (UID: \"fbd7920a-c23d-4bf8-b4af-25408c94e843\") " pod="openshift-marketplace/d54990151e6a3040f48962708f776cca7120e97625eb3a76d6fde2767bjplv4"
Apr 24 14:33:17.184831 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:33:17.184815    2574 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/ef1189e9861fa30b9414ceb420c2d78e85403a7e10097f37afdacfec88tgbwr"
Apr 24 14:33:17.190412 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:33:17.190389    2574 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/309c32b5cc4cbd8148882f0ed2adbe6c47ee2761cc0c22c627755a3c30lcv9z"]
Apr 24 14:33:17.270115 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:33:17.270076    2574 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-hhtwd\" (UniqueName: \"kubernetes.io/projected/6ab86bf6-5147-4edc-b660-9f144520f444-kube-api-access-hhtwd\") pod \"5b107722e7a95ad7441cca2226d3d656a8bf76dc65e15066bc01f8e503khl8s\" (UID: \"6ab86bf6-5147-4edc-b660-9f144520f444\") " pod="openshift-marketplace/5b107722e7a95ad7441cca2226d3d656a8bf76dc65e15066bc01f8e503khl8s"
Apr 24 14:33:17.270295 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:33:17.270126    2574 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/6ab86bf6-5147-4edc-b660-9f144520f444-util\") pod \"5b107722e7a95ad7441cca2226d3d656a8bf76dc65e15066bc01f8e503khl8s\" (UID: \"6ab86bf6-5147-4edc-b660-9f144520f444\") " pod="openshift-marketplace/5b107722e7a95ad7441cca2226d3d656a8bf76dc65e15066bc01f8e503khl8s"
Apr 24 14:33:17.270295 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:33:17.270152    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/e02d2148-c267-43de-bd3d-9339754a382f-bundle\") pod \"309c32b5cc4cbd8148882f0ed2adbe6c47ee2761cc0c22c627755a3c30lcv9z\" (UID: \"e02d2148-c267-43de-bd3d-9339754a382f\") " pod="openshift-marketplace/309c32b5cc4cbd8148882f0ed2adbe6c47ee2761cc0c22c627755a3c30lcv9z"
Apr 24 14:33:17.270295 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:33:17.270190    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6gzbr\" (UniqueName: \"kubernetes.io/projected/e02d2148-c267-43de-bd3d-9339754a382f-kube-api-access-6gzbr\") pod \"309c32b5cc4cbd8148882f0ed2adbe6c47ee2761cc0c22c627755a3c30lcv9z\" (UID: \"e02d2148-c267-43de-bd3d-9339754a382f\") " pod="openshift-marketplace/309c32b5cc4cbd8148882f0ed2adbe6c47ee2761cc0c22c627755a3c30lcv9z"
Apr 24 14:33:17.270295 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:33:17.270217    2574 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/6ab86bf6-5147-4edc-b660-9f144520f444-bundle\") pod \"5b107722e7a95ad7441cca2226d3d656a8bf76dc65e15066bc01f8e503khl8s\" (UID: \"6ab86bf6-5147-4edc-b660-9f144520f444\") " pod="openshift-marketplace/5b107722e7a95ad7441cca2226d3d656a8bf76dc65e15066bc01f8e503khl8s"
Apr 24 14:33:17.270295 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:33:17.270265    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/e02d2148-c267-43de-bd3d-9339754a382f-util\") pod \"309c32b5cc4cbd8148882f0ed2adbe6c47ee2761cc0c22c627755a3c30lcv9z\" (UID: \"e02d2148-c267-43de-bd3d-9339754a382f\") " pod="openshift-marketplace/309c32b5cc4cbd8148882f0ed2adbe6c47ee2761cc0c22c627755a3c30lcv9z"
Apr 24 14:33:17.270604 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:33:17.270530    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/6ab86bf6-5147-4edc-b660-9f144520f444-bundle\") pod \"5b107722e7a95ad7441cca2226d3d656a8bf76dc65e15066bc01f8e503khl8s\" (UID: \"6ab86bf6-5147-4edc-b660-9f144520f444\") " pod="openshift-marketplace/5b107722e7a95ad7441cca2226d3d656a8bf76dc65e15066bc01f8e503khl8s"
Apr 24 14:33:17.270643 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:33:17.270627    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/6ab86bf6-5147-4edc-b660-9f144520f444-util\") pod \"5b107722e7a95ad7441cca2226d3d656a8bf76dc65e15066bc01f8e503khl8s\" (UID: \"6ab86bf6-5147-4edc-b660-9f144520f444\") " pod="openshift-marketplace/5b107722e7a95ad7441cca2226d3d656a8bf76dc65e15066bc01f8e503khl8s"
Apr 24 14:33:17.278417 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:33:17.278392    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-hhtwd\" (UniqueName: \"kubernetes.io/projected/6ab86bf6-5147-4edc-b660-9f144520f444-kube-api-access-hhtwd\") pod \"5b107722e7a95ad7441cca2226d3d656a8bf76dc65e15066bc01f8e503khl8s\" (UID: \"6ab86bf6-5147-4edc-b660-9f144520f444\") " pod="openshift-marketplace/5b107722e7a95ad7441cca2226d3d656a8bf76dc65e15066bc01f8e503khl8s"
Apr 24 14:33:17.293254 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:33:17.293222    2574 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/d54990151e6a3040f48962708f776cca7120e97625eb3a76d6fde2767bjplv4"
Apr 24 14:33:17.324968 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:33:17.324939    2574 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/ef1189e9861fa30b9414ceb420c2d78e85403a7e10097f37afdacfec88tgbwr"]
Apr 24 14:33:17.326311 ip-10-0-136-132 kubenswrapper[2574]: W0424 14:33:17.326284    2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc3f2eaf3_730c_4f91_8700_498b592681f3.slice/crio-710b958ecf50fb6fdf65910d6cdc5779e7a3c56d44ef3e6751f5899f74de5d92 WatchSource:0}: Error finding container 710b958ecf50fb6fdf65910d6cdc5779e7a3c56d44ef3e6751f5899f74de5d92: Status 404 returned error can't find the container with id 710b958ecf50fb6fdf65910d6cdc5779e7a3c56d44ef3e6751f5899f74de5d92
Apr 24 14:33:17.371550 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:33:17.371517    2574 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/e02d2148-c267-43de-bd3d-9339754a382f-bundle\") pod \"309c32b5cc4cbd8148882f0ed2adbe6c47ee2761cc0c22c627755a3c30lcv9z\" (UID: \"e02d2148-c267-43de-bd3d-9339754a382f\") " pod="openshift-marketplace/309c32b5cc4cbd8148882f0ed2adbe6c47ee2761cc0c22c627755a3c30lcv9z"
Apr 24 14:33:17.371690 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:33:17.371600    2574 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-6gzbr\" (UniqueName: \"kubernetes.io/projected/e02d2148-c267-43de-bd3d-9339754a382f-kube-api-access-6gzbr\") pod \"309c32b5cc4cbd8148882f0ed2adbe6c47ee2761cc0c22c627755a3c30lcv9z\" (UID: \"e02d2148-c267-43de-bd3d-9339754a382f\") " pod="openshift-marketplace/309c32b5cc4cbd8148882f0ed2adbe6c47ee2761cc0c22c627755a3c30lcv9z"
Apr 24 14:33:17.371690 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:33:17.371653    2574 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/e02d2148-c267-43de-bd3d-9339754a382f-util\") pod \"309c32b5cc4cbd8148882f0ed2adbe6c47ee2761cc0c22c627755a3c30lcv9z\" (UID: \"e02d2148-c267-43de-bd3d-9339754a382f\") " pod="openshift-marketplace/309c32b5cc4cbd8148882f0ed2adbe6c47ee2761cc0c22c627755a3c30lcv9z"
Apr 24 14:33:17.371949 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:33:17.371931    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/e02d2148-c267-43de-bd3d-9339754a382f-bundle\") pod \"309c32b5cc4cbd8148882f0ed2adbe6c47ee2761cc0c22c627755a3c30lcv9z\" (UID: \"e02d2148-c267-43de-bd3d-9339754a382f\") " pod="openshift-marketplace/309c32b5cc4cbd8148882f0ed2adbe6c47ee2761cc0c22c627755a3c30lcv9z"
Apr 24 14:33:17.372035 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:33:17.372006    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/e02d2148-c267-43de-bd3d-9339754a382f-util\") pod \"309c32b5cc4cbd8148882f0ed2adbe6c47ee2761cc0c22c627755a3c30lcv9z\" (UID: \"e02d2148-c267-43de-bd3d-9339754a382f\") " pod="openshift-marketplace/309c32b5cc4cbd8148882f0ed2adbe6c47ee2761cc0c22c627755a3c30lcv9z"
Apr 24 14:33:17.380754 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:33:17.380727    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-6gzbr\" (UniqueName: \"kubernetes.io/projected/e02d2148-c267-43de-bd3d-9339754a382f-kube-api-access-6gzbr\") pod \"309c32b5cc4cbd8148882f0ed2adbe6c47ee2761cc0c22c627755a3c30lcv9z\" (UID: \"e02d2148-c267-43de-bd3d-9339754a382f\") " pod="openshift-marketplace/309c32b5cc4cbd8148882f0ed2adbe6c47ee2761cc0c22c627755a3c30lcv9z"
Apr 24 14:33:17.402381 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:33:17.402356    2574 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/5b107722e7a95ad7441cca2226d3d656a8bf76dc65e15066bc01f8e503khl8s"
Apr 24 14:33:17.427852 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:33:17.427823    2574 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/d54990151e6a3040f48962708f776cca7120e97625eb3a76d6fde2767bjplv4"]
Apr 24 14:33:17.430288 ip-10-0-136-132 kubenswrapper[2574]: W0424 14:33:17.430258    2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podfbd7920a_c23d_4bf8_b4af_25408c94e843.slice/crio-a842e8ba8344b626659bc33e5033955570a3103c81be563a67eacda0e6ea21a5 WatchSource:0}: Error finding container a842e8ba8344b626659bc33e5033955570a3103c81be563a67eacda0e6ea21a5: Status 404 returned error can't find the container with id a842e8ba8344b626659bc33e5033955570a3103c81be563a67eacda0e6ea21a5
Apr 24 14:33:17.525992 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:33:17.525969    2574 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/309c32b5cc4cbd8148882f0ed2adbe6c47ee2761cc0c22c627755a3c30lcv9z"
Apr 24 14:33:17.536894 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:33:17.536868    2574 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/5b107722e7a95ad7441cca2226d3d656a8bf76dc65e15066bc01f8e503khl8s"]
Apr 24 14:33:17.538610 ip-10-0-136-132 kubenswrapper[2574]: W0424 14:33:17.538586    2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6ab86bf6_5147_4edc_b660_9f144520f444.slice/crio-8186121b75ff20c81fee8e4db12a3c41c55dd7f213a09385838fccd66521d934 WatchSource:0}: Error finding container 8186121b75ff20c81fee8e4db12a3c41c55dd7f213a09385838fccd66521d934: Status 404 returned error can't find the container with id 8186121b75ff20c81fee8e4db12a3c41c55dd7f213a09385838fccd66521d934
Apr 24 14:33:17.658132 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:33:17.658111    2574 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/309c32b5cc4cbd8148882f0ed2adbe6c47ee2761cc0c22c627755a3c30lcv9z"]
Apr 24 14:33:17.683234 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:33:17.683156    2574 generic.go:358] "Generic (PLEG): container finished" podID="fbd7920a-c23d-4bf8-b4af-25408c94e843" containerID="e4666d54aaa39ac16775fc1f0fee093f51ee0b1e6dba27e76757ff2ac5059654" exitCode=0
Apr 24 14:33:17.683415 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:33:17.683240    2574 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/d54990151e6a3040f48962708f776cca7120e97625eb3a76d6fde2767bjplv4" event={"ID":"fbd7920a-c23d-4bf8-b4af-25408c94e843","Type":"ContainerDied","Data":"e4666d54aaa39ac16775fc1f0fee093f51ee0b1e6dba27e76757ff2ac5059654"}
Apr 24 14:33:17.683415 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:33:17.683281    2574 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/d54990151e6a3040f48962708f776cca7120e97625eb3a76d6fde2767bjplv4" event={"ID":"fbd7920a-c23d-4bf8-b4af-25408c94e843","Type":"ContainerStarted","Data":"a842e8ba8344b626659bc33e5033955570a3103c81be563a67eacda0e6ea21a5"}
Apr 24 14:33:17.684682 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:33:17.684655    2574 generic.go:358] "Generic (PLEG): container finished" podID="6ab86bf6-5147-4edc-b660-9f144520f444" containerID="421370db7f7b347cff9dbca4602500d18d8968536917e43668a6e83cb9cba99a" exitCode=0
Apr 24 14:33:17.684779 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:33:17.684683    2574 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/5b107722e7a95ad7441cca2226d3d656a8bf76dc65e15066bc01f8e503khl8s" event={"ID":"6ab86bf6-5147-4edc-b660-9f144520f444","Type":"ContainerDied","Data":"421370db7f7b347cff9dbca4602500d18d8968536917e43668a6e83cb9cba99a"}
Apr 24 14:33:17.684779 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:33:17.684727    2574 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/5b107722e7a95ad7441cca2226d3d656a8bf76dc65e15066bc01f8e503khl8s" event={"ID":"6ab86bf6-5147-4edc-b660-9f144520f444","Type":"ContainerStarted","Data":"8186121b75ff20c81fee8e4db12a3c41c55dd7f213a09385838fccd66521d934"}
Apr 24 14:33:17.686126 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:33:17.686108    2574 generic.go:358] "Generic (PLEG): container finished" podID="c3f2eaf3-730c-4f91-8700-498b592681f3" containerID="27a6f5b50b6f6bc6b7984c4ef65f9c6edf924d96f962b69aeb97be80efe3070b" exitCode=0
Apr 24 14:33:17.686201 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:33:17.686133    2574 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/ef1189e9861fa30b9414ceb420c2d78e85403a7e10097f37afdacfec88tgbwr" event={"ID":"c3f2eaf3-730c-4f91-8700-498b592681f3","Type":"ContainerDied","Data":"27a6f5b50b6f6bc6b7984c4ef65f9c6edf924d96f962b69aeb97be80efe3070b"}
Apr 24 14:33:17.686253 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:33:17.686168    2574 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/ef1189e9861fa30b9414ceb420c2d78e85403a7e10097f37afdacfec88tgbwr" event={"ID":"c3f2eaf3-730c-4f91-8700-498b592681f3","Type":"ContainerStarted","Data":"710b958ecf50fb6fdf65910d6cdc5779e7a3c56d44ef3e6751f5899f74de5d92"}
Apr 24 14:33:17.689325 ip-10-0-136-132 kubenswrapper[2574]: W0424 14:33:17.689295    2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode02d2148_c267_43de_bd3d_9339754a382f.slice/crio-cd17c0e799216f8056cc921350aea8ef44a5f312b6e6fa6594c4718c290d2059 WatchSource:0}: Error finding container cd17c0e799216f8056cc921350aea8ef44a5f312b6e6fa6594c4718c290d2059: Status 404 returned error can't find the container with id cd17c0e799216f8056cc921350aea8ef44a5f312b6e6fa6594c4718c290d2059
Apr 24 14:33:18.692161 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:33:18.692131    2574 generic.go:358] "Generic (PLEG): container finished" podID="6ab86bf6-5147-4edc-b660-9f144520f444" containerID="63e90d0ab90a229777cfa7f31ec064b6f135deb1c74632dc73484ee63802044b" exitCode=0
Apr 24 14:33:18.692570 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:33:18.692213    2574 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/5b107722e7a95ad7441cca2226d3d656a8bf76dc65e15066bc01f8e503khl8s" event={"ID":"6ab86bf6-5147-4edc-b660-9f144520f444","Type":"ContainerDied","Data":"63e90d0ab90a229777cfa7f31ec064b6f135deb1c74632dc73484ee63802044b"}
Apr 24 14:33:18.694081 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:33:18.694064    2574 generic.go:358] "Generic (PLEG): container finished" podID="c3f2eaf3-730c-4f91-8700-498b592681f3" containerID="052349de9d764abe91d12935cee86e2ce021c916d8ca2417685dbc96040cad9b" exitCode=0
Apr 24 14:33:18.694161 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:33:18.694117    2574 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/ef1189e9861fa30b9414ceb420c2d78e85403a7e10097f37afdacfec88tgbwr" event={"ID":"c3f2eaf3-730c-4f91-8700-498b592681f3","Type":"ContainerDied","Data":"052349de9d764abe91d12935cee86e2ce021c916d8ca2417685dbc96040cad9b"}
Apr 24 14:33:18.695753 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:33:18.695669    2574 generic.go:358] "Generic (PLEG): container finished" podID="e02d2148-c267-43de-bd3d-9339754a382f" containerID="917a8ef211bda30a53373bf5ece7d299b781d8dc4e01d1b317f215cb3ca50caa" exitCode=0
Apr 24 14:33:18.695842 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:33:18.695809    2574 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/309c32b5cc4cbd8148882f0ed2adbe6c47ee2761cc0c22c627755a3c30lcv9z" event={"ID":"e02d2148-c267-43de-bd3d-9339754a382f","Type":"ContainerDied","Data":"917a8ef211bda30a53373bf5ece7d299b781d8dc4e01d1b317f215cb3ca50caa"}
Apr 24 14:33:18.695924 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:33:18.695847    2574 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/309c32b5cc4cbd8148882f0ed2adbe6c47ee2761cc0c22c627755a3c30lcv9z" event={"ID":"e02d2148-c267-43de-bd3d-9339754a382f","Type":"ContainerStarted","Data":"cd17c0e799216f8056cc921350aea8ef44a5f312b6e6fa6594c4718c290d2059"}
Apr 24 14:33:18.698436 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:33:18.698312    2574 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/d54990151e6a3040f48962708f776cca7120e97625eb3a76d6fde2767bjplv4" event={"ID":"fbd7920a-c23d-4bf8-b4af-25408c94e843","Type":"ContainerStarted","Data":"d83a873a4591dde103697368124ae00ffbd13a3131e3813422e05f4691af7e75"}
Apr 24 14:33:19.704110 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:33:19.704016    2574 generic.go:358] "Generic (PLEG): container finished" podID="6ab86bf6-5147-4edc-b660-9f144520f444" containerID="753b187e56e5c666ead3d1050829ccd926360b1ac9ac9624c3ec6eba5f1e0fe8" exitCode=0
Apr 24 14:33:19.704547 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:33:19.704104    2574 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/5b107722e7a95ad7441cca2226d3d656a8bf76dc65e15066bc01f8e503khl8s" event={"ID":"6ab86bf6-5147-4edc-b660-9f144520f444","Type":"ContainerDied","Data":"753b187e56e5c666ead3d1050829ccd926360b1ac9ac9624c3ec6eba5f1e0fe8"}
Apr 24 14:33:19.705861 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:33:19.705838    2574 generic.go:358] "Generic (PLEG): container finished" podID="c3f2eaf3-730c-4f91-8700-498b592681f3" containerID="0cec87e905a28b49c845ac540dcfead46d1d88cf8c4ecdcee79e7f4c8b04b4f0" exitCode=0
Apr 24 14:33:19.705970 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:33:19.705891    2574 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/ef1189e9861fa30b9414ceb420c2d78e85403a7e10097f37afdacfec88tgbwr" event={"ID":"c3f2eaf3-730c-4f91-8700-498b592681f3","Type":"ContainerDied","Data":"0cec87e905a28b49c845ac540dcfead46d1d88cf8c4ecdcee79e7f4c8b04b4f0"}
Apr 24 14:33:19.707378 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:33:19.707354    2574 generic.go:358] "Generic (PLEG): container finished" podID="e02d2148-c267-43de-bd3d-9339754a382f" containerID="5e2f2870a2a575178676843c64fe75cb17e1f927fd0b8d67e64f9b463174c4a5" exitCode=0
Apr 24 14:33:19.707466 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:33:19.707428    2574 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/309c32b5cc4cbd8148882f0ed2adbe6c47ee2761cc0c22c627755a3c30lcv9z" event={"ID":"e02d2148-c267-43de-bd3d-9339754a382f","Type":"ContainerDied","Data":"5e2f2870a2a575178676843c64fe75cb17e1f927fd0b8d67e64f9b463174c4a5"}
Apr 24 14:33:19.709279 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:33:19.709253    2574 generic.go:358] "Generic (PLEG): container finished" podID="fbd7920a-c23d-4bf8-b4af-25408c94e843" containerID="d83a873a4591dde103697368124ae00ffbd13a3131e3813422e05f4691af7e75" exitCode=0
Apr 24 14:33:19.709279 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:33:19.709279    2574 generic.go:358] "Generic (PLEG): container finished" podID="fbd7920a-c23d-4bf8-b4af-25408c94e843" containerID="2b535cffa994617e6fe98585dd82075f5eb44f46c2aa3084ea14db974c6627d5" exitCode=0
Apr 24 14:33:19.709443 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:33:19.709310    2574 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/d54990151e6a3040f48962708f776cca7120e97625eb3a76d6fde2767bjplv4" event={"ID":"fbd7920a-c23d-4bf8-b4af-25408c94e843","Type":"ContainerDied","Data":"d83a873a4591dde103697368124ae00ffbd13a3131e3813422e05f4691af7e75"}
Apr 24 14:33:19.709443 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:33:19.709360    2574 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/d54990151e6a3040f48962708f776cca7120e97625eb3a76d6fde2767bjplv4" event={"ID":"fbd7920a-c23d-4bf8-b4af-25408c94e843","Type":"ContainerDied","Data":"2b535cffa994617e6fe98585dd82075f5eb44f46c2aa3084ea14db974c6627d5"}
Apr 24 14:33:20.714908 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:33:20.714873    2574 generic.go:358] "Generic (PLEG): container finished" podID="e02d2148-c267-43de-bd3d-9339754a382f" containerID="d73a828017f3526b9ba849ab9e5d8a6b45913f9cb13b26349278c13ecad9f9e5" exitCode=0
Apr 24 14:33:20.715307 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:33:20.714959    2574 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/309c32b5cc4cbd8148882f0ed2adbe6c47ee2761cc0c22c627755a3c30lcv9z" event={"ID":"e02d2148-c267-43de-bd3d-9339754a382f","Type":"ContainerDied","Data":"d73a828017f3526b9ba849ab9e5d8a6b45913f9cb13b26349278c13ecad9f9e5"}
Apr 24 14:33:20.851329 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:33:20.851296    2574 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/5b107722e7a95ad7441cca2226d3d656a8bf76dc65e15066bc01f8e503khl8s"
Apr 24 14:33:20.895182 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:33:20.895159    2574 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/d54990151e6a3040f48962708f776cca7120e97625eb3a76d6fde2767bjplv4"
Apr 24 14:33:20.898734 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:33:20.898715    2574 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/ef1189e9861fa30b9414ceb420c2d78e85403a7e10097f37afdacfec88tgbwr"
Apr 24 14:33:20.904031 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:33:20.904010    2574 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/6ab86bf6-5147-4edc-b660-9f144520f444-bundle\") pod \"6ab86bf6-5147-4edc-b660-9f144520f444\" (UID: \"6ab86bf6-5147-4edc-b660-9f144520f444\") "
Apr 24 14:33:20.904212 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:33:20.904041    2574 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"kube-api-access-hhtwd\" (UniqueName: \"kubernetes.io/projected/6ab86bf6-5147-4edc-b660-9f144520f444-kube-api-access-hhtwd\") pod \"6ab86bf6-5147-4edc-b660-9f144520f444\" (UID: \"6ab86bf6-5147-4edc-b660-9f144520f444\") "
Apr 24 14:33:20.904212 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:33:20.904057    2574 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/6ab86bf6-5147-4edc-b660-9f144520f444-util\") pod \"6ab86bf6-5147-4edc-b660-9f144520f444\" (UID: \"6ab86bf6-5147-4edc-b660-9f144520f444\") "
Apr 24 14:33:20.904758 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:33:20.904718    2574 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/6ab86bf6-5147-4edc-b660-9f144520f444-bundle" (OuterVolumeSpecName: "bundle") pod "6ab86bf6-5147-4edc-b660-9f144520f444" (UID: "6ab86bf6-5147-4edc-b660-9f144520f444"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Apr 24 14:33:20.906190 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:33:20.906168    2574 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/6ab86bf6-5147-4edc-b660-9f144520f444-kube-api-access-hhtwd" (OuterVolumeSpecName: "kube-api-access-hhtwd") pod "6ab86bf6-5147-4edc-b660-9f144520f444" (UID: "6ab86bf6-5147-4edc-b660-9f144520f444"). InnerVolumeSpecName "kube-api-access-hhtwd". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Apr 24 14:33:20.909667 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:33:20.909647    2574 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/6ab86bf6-5147-4edc-b660-9f144520f444-util" (OuterVolumeSpecName: "util") pod "6ab86bf6-5147-4edc-b660-9f144520f444" (UID: "6ab86bf6-5147-4edc-b660-9f144520f444"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Apr 24 14:33:21.004635 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:33:21.004542    2574 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"kube-api-access-f42vn\" (UniqueName: \"kubernetes.io/projected/c3f2eaf3-730c-4f91-8700-498b592681f3-kube-api-access-f42vn\") pod \"c3f2eaf3-730c-4f91-8700-498b592681f3\" (UID: \"c3f2eaf3-730c-4f91-8700-498b592681f3\") "
Apr 24 14:33:21.004802 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:33:21.004643    2574 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/fbd7920a-c23d-4bf8-b4af-25408c94e843-bundle\") pod \"fbd7920a-c23d-4bf8-b4af-25408c94e843\" (UID: \"fbd7920a-c23d-4bf8-b4af-25408c94e843\") "
Apr 24 14:33:21.004802 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:33:21.004675    2574 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"kube-api-access-vj7cf\" (UniqueName: \"kubernetes.io/projected/fbd7920a-c23d-4bf8-b4af-25408c94e843-kube-api-access-vj7cf\") pod \"fbd7920a-c23d-4bf8-b4af-25408c94e843\" (UID: \"fbd7920a-c23d-4bf8-b4af-25408c94e843\") "
Apr 24 14:33:21.004802 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:33:21.004716    2574 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/c3f2eaf3-730c-4f91-8700-498b592681f3-bundle\") pod \"c3f2eaf3-730c-4f91-8700-498b592681f3\" (UID: \"c3f2eaf3-730c-4f91-8700-498b592681f3\") "
Apr 24 14:33:21.004802 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:33:21.004737    2574 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/c3f2eaf3-730c-4f91-8700-498b592681f3-util\") pod \"c3f2eaf3-730c-4f91-8700-498b592681f3\" (UID: \"c3f2eaf3-730c-4f91-8700-498b592681f3\") "
Apr 24 14:33:21.004802 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:33:21.004784    2574 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/fbd7920a-c23d-4bf8-b4af-25408c94e843-util\") pod \"fbd7920a-c23d-4bf8-b4af-25408c94e843\" (UID: \"fbd7920a-c23d-4bf8-b4af-25408c94e843\") "
Apr 24 14:33:21.005064 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:33:21.005048    2574 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/6ab86bf6-5147-4edc-b660-9f144520f444-bundle\") on node \"ip-10-0-136-132.ec2.internal\" DevicePath \"\""
Apr 24 14:33:21.005121 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:33:21.005072    2574 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-hhtwd\" (UniqueName: \"kubernetes.io/projected/6ab86bf6-5147-4edc-b660-9f144520f444-kube-api-access-hhtwd\") on node \"ip-10-0-136-132.ec2.internal\" DevicePath \"\""
Apr 24 14:33:21.005121 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:33:21.005087    2574 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/6ab86bf6-5147-4edc-b660-9f144520f444-util\") on node \"ip-10-0-136-132.ec2.internal\" DevicePath \"\""
Apr 24 14:33:21.005247 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:33:21.005219    2574 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/fbd7920a-c23d-4bf8-b4af-25408c94e843-bundle" (OuterVolumeSpecName: "bundle") pod "fbd7920a-c23d-4bf8-b4af-25408c94e843" (UID: "fbd7920a-c23d-4bf8-b4af-25408c94e843"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Apr 24 14:33:21.005360 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:33:21.005319    2574 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c3f2eaf3-730c-4f91-8700-498b592681f3-bundle" (OuterVolumeSpecName: "bundle") pod "c3f2eaf3-730c-4f91-8700-498b592681f3" (UID: "c3f2eaf3-730c-4f91-8700-498b592681f3"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Apr 24 14:33:21.006859 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:33:21.006835    2574 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/c3f2eaf3-730c-4f91-8700-498b592681f3-kube-api-access-f42vn" (OuterVolumeSpecName: "kube-api-access-f42vn") pod "c3f2eaf3-730c-4f91-8700-498b592681f3" (UID: "c3f2eaf3-730c-4f91-8700-498b592681f3"). InnerVolumeSpecName "kube-api-access-f42vn". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Apr 24 14:33:21.006961 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:33:21.006889    2574 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/fbd7920a-c23d-4bf8-b4af-25408c94e843-kube-api-access-vj7cf" (OuterVolumeSpecName: "kube-api-access-vj7cf") pod "fbd7920a-c23d-4bf8-b4af-25408c94e843" (UID: "fbd7920a-c23d-4bf8-b4af-25408c94e843"). InnerVolumeSpecName "kube-api-access-vj7cf". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Apr 24 14:33:21.013532 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:33:21.013505    2574 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c3f2eaf3-730c-4f91-8700-498b592681f3-util" (OuterVolumeSpecName: "util") pod "c3f2eaf3-730c-4f91-8700-498b592681f3" (UID: "c3f2eaf3-730c-4f91-8700-498b592681f3"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Apr 24 14:33:21.013532 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:33:21.013518    2574 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/fbd7920a-c23d-4bf8-b4af-25408c94e843-util" (OuterVolumeSpecName: "util") pod "fbd7920a-c23d-4bf8-b4af-25408c94e843" (UID: "fbd7920a-c23d-4bf8-b4af-25408c94e843"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Apr 24 14:33:21.105908 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:33:21.105871    2574 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/fbd7920a-c23d-4bf8-b4af-25408c94e843-bundle\") on node \"ip-10-0-136-132.ec2.internal\" DevicePath \"\""
Apr 24 14:33:21.105908 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:33:21.105903    2574 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-vj7cf\" (UniqueName: \"kubernetes.io/projected/fbd7920a-c23d-4bf8-b4af-25408c94e843-kube-api-access-vj7cf\") on node \"ip-10-0-136-132.ec2.internal\" DevicePath \"\""
Apr 24 14:33:21.105908 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:33:21.105914    2574 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/c3f2eaf3-730c-4f91-8700-498b592681f3-bundle\") on node \"ip-10-0-136-132.ec2.internal\" DevicePath \"\""
Apr 24 14:33:21.106134 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:33:21.105922    2574 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/c3f2eaf3-730c-4f91-8700-498b592681f3-util\") on node \"ip-10-0-136-132.ec2.internal\" DevicePath \"\""
Apr 24 14:33:21.106134 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:33:21.105931    2574 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/fbd7920a-c23d-4bf8-b4af-25408c94e843-util\") on node \"ip-10-0-136-132.ec2.internal\" DevicePath \"\""
Apr 24 14:33:21.106134 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:33:21.105939    2574 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-f42vn\" (UniqueName: \"kubernetes.io/projected/c3f2eaf3-730c-4f91-8700-498b592681f3-kube-api-access-f42vn\") on node \"ip-10-0-136-132.ec2.internal\" DevicePath \"\""
Apr 24 14:33:21.719959 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:33:21.719925    2574 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/d54990151e6a3040f48962708f776cca7120e97625eb3a76d6fde2767bjplv4"
Apr 24 14:33:21.719959 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:33:21.719952    2574 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/d54990151e6a3040f48962708f776cca7120e97625eb3a76d6fde2767bjplv4" event={"ID":"fbd7920a-c23d-4bf8-b4af-25408c94e843","Type":"ContainerDied","Data":"a842e8ba8344b626659bc33e5033955570a3103c81be563a67eacda0e6ea21a5"}
Apr 24 14:33:21.720452 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:33:21.719983    2574 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="a842e8ba8344b626659bc33e5033955570a3103c81be563a67eacda0e6ea21a5"
Apr 24 14:33:21.721616 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:33:21.721587    2574 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/5b107722e7a95ad7441cca2226d3d656a8bf76dc65e15066bc01f8e503khl8s" event={"ID":"6ab86bf6-5147-4edc-b660-9f144520f444","Type":"ContainerDied","Data":"8186121b75ff20c81fee8e4db12a3c41c55dd7f213a09385838fccd66521d934"}
Apr 24 14:33:21.721616 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:33:21.721611    2574 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/5b107722e7a95ad7441cca2226d3d656a8bf76dc65e15066bc01f8e503khl8s"
Apr 24 14:33:21.721794 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:33:21.721620    2574 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="8186121b75ff20c81fee8e4db12a3c41c55dd7f213a09385838fccd66521d934"
Apr 24 14:33:21.723476 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:33:21.723449    2574 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/ef1189e9861fa30b9414ceb420c2d78e85403a7e10097f37afdacfec88tgbwr" event={"ID":"c3f2eaf3-730c-4f91-8700-498b592681f3","Type":"ContainerDied","Data":"710b958ecf50fb6fdf65910d6cdc5779e7a3c56d44ef3e6751f5899f74de5d92"}
Apr 24 14:33:21.723476 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:33:21.723474    2574 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/ef1189e9861fa30b9414ceb420c2d78e85403a7e10097f37afdacfec88tgbwr"
Apr 24 14:33:21.723619 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:33:21.723481    2574 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="710b958ecf50fb6fdf65910d6cdc5779e7a3c56d44ef3e6751f5899f74de5d92"
Apr 24 14:33:21.849069 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:33:21.849040    2574 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/309c32b5cc4cbd8148882f0ed2adbe6c47ee2761cc0c22c627755a3c30lcv9z"
Apr 24 14:33:21.911725 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:33:21.911693    2574 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/e02d2148-c267-43de-bd3d-9339754a382f-bundle\") pod \"e02d2148-c267-43de-bd3d-9339754a382f\" (UID: \"e02d2148-c267-43de-bd3d-9339754a382f\") "
Apr 24 14:33:21.911883 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:33:21.911741    2574 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"kube-api-access-6gzbr\" (UniqueName: \"kubernetes.io/projected/e02d2148-c267-43de-bd3d-9339754a382f-kube-api-access-6gzbr\") pod \"e02d2148-c267-43de-bd3d-9339754a382f\" (UID: \"e02d2148-c267-43de-bd3d-9339754a382f\") "
Apr 24 14:33:21.911883 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:33:21.911800    2574 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/e02d2148-c267-43de-bd3d-9339754a382f-util\") pod \"e02d2148-c267-43de-bd3d-9339754a382f\" (UID: \"e02d2148-c267-43de-bd3d-9339754a382f\") "
Apr 24 14:33:21.912245 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:33:21.912220    2574 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e02d2148-c267-43de-bd3d-9339754a382f-bundle" (OuterVolumeSpecName: "bundle") pod "e02d2148-c267-43de-bd3d-9339754a382f" (UID: "e02d2148-c267-43de-bd3d-9339754a382f"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Apr 24 14:33:21.913843 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:33:21.913817    2574 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e02d2148-c267-43de-bd3d-9339754a382f-kube-api-access-6gzbr" (OuterVolumeSpecName: "kube-api-access-6gzbr") pod "e02d2148-c267-43de-bd3d-9339754a382f" (UID: "e02d2148-c267-43de-bd3d-9339754a382f"). InnerVolumeSpecName "kube-api-access-6gzbr". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Apr 24 14:33:21.916854 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:33:21.916820    2574 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e02d2148-c267-43de-bd3d-9339754a382f-util" (OuterVolumeSpecName: "util") pod "e02d2148-c267-43de-bd3d-9339754a382f" (UID: "e02d2148-c267-43de-bd3d-9339754a382f"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Apr 24 14:33:22.013290 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:33:22.013196    2574 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/e02d2148-c267-43de-bd3d-9339754a382f-bundle\") on node \"ip-10-0-136-132.ec2.internal\" DevicePath \"\""
Apr 24 14:33:22.013290 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:33:22.013231    2574 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-6gzbr\" (UniqueName: \"kubernetes.io/projected/e02d2148-c267-43de-bd3d-9339754a382f-kube-api-access-6gzbr\") on node \"ip-10-0-136-132.ec2.internal\" DevicePath \"\""
Apr 24 14:33:22.013290 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:33:22.013241    2574 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/e02d2148-c267-43de-bd3d-9339754a382f-util\") on node \"ip-10-0-136-132.ec2.internal\" DevicePath \"\""
Apr 24 14:33:22.729109 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:33:22.729078    2574 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/309c32b5cc4cbd8148882f0ed2adbe6c47ee2761cc0c22c627755a3c30lcv9z"
Apr 24 14:33:22.729514 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:33:22.729081    2574 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/309c32b5cc4cbd8148882f0ed2adbe6c47ee2761cc0c22c627755a3c30lcv9z" event={"ID":"e02d2148-c267-43de-bd3d-9339754a382f","Type":"ContainerDied","Data":"cd17c0e799216f8056cc921350aea8ef44a5f312b6e6fa6594c4718c290d2059"}
Apr 24 14:33:22.729514 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:33:22.729188    2574 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="cd17c0e799216f8056cc921350aea8ef44a5f312b6e6fa6594c4718c290d2059"
Apr 24 14:33:47.326469 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:33:47.326429    2574 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["kuadrant-system/dns-operator-controller-manager-844548ff4c-qzx86"]
Apr 24 14:33:47.327030 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:33:47.327009    2574 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="fbd7920a-c23d-4bf8-b4af-25408c94e843" containerName="util"
Apr 24 14:33:47.327105 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:33:47.327034    2574 state_mem.go:107] "Deleted CPUSet assignment" podUID="fbd7920a-c23d-4bf8-b4af-25408c94e843" containerName="util"
Apr 24 14:33:47.327105 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:33:47.327048    2574 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="6ab86bf6-5147-4edc-b660-9f144520f444" containerName="extract"
Apr 24 14:33:47.327105 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:33:47.327056    2574 state_mem.go:107] "Deleted CPUSet assignment" podUID="6ab86bf6-5147-4edc-b660-9f144520f444" containerName="extract"
Apr 24 14:33:47.327105 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:33:47.327078    2574 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c3f2eaf3-730c-4f91-8700-498b592681f3" containerName="extract"
Apr 24 14:33:47.327105 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:33:47.327088    2574 state_mem.go:107] "Deleted CPUSet assignment" podUID="c3f2eaf3-730c-4f91-8700-498b592681f3" containerName="extract"
Apr 24 14:33:47.327105 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:33:47.327100    2574 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c3f2eaf3-730c-4f91-8700-498b592681f3" containerName="util"
Apr 24 14:33:47.327452 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:33:47.327108    2574 state_mem.go:107] "Deleted CPUSet assignment" podUID="c3f2eaf3-730c-4f91-8700-498b592681f3" containerName="util"
Apr 24 14:33:47.327452 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:33:47.327118    2574 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="fbd7920a-c23d-4bf8-b4af-25408c94e843" containerName="pull"
Apr 24 14:33:47.327452 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:33:47.327125    2574 state_mem.go:107] "Deleted CPUSet assignment" podUID="fbd7920a-c23d-4bf8-b4af-25408c94e843" containerName="pull"
Apr 24 14:33:47.327452 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:33:47.327149    2574 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e02d2148-c267-43de-bd3d-9339754a382f" containerName="util"
Apr 24 14:33:47.327452 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:33:47.327156    2574 state_mem.go:107] "Deleted CPUSet assignment" podUID="e02d2148-c267-43de-bd3d-9339754a382f" containerName="util"
Apr 24 14:33:47.327452 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:33:47.327169    2574 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="6ab86bf6-5147-4edc-b660-9f144520f444" containerName="util"
Apr 24 14:33:47.327452 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:33:47.327177    2574 state_mem.go:107] "Deleted CPUSet assignment" podUID="6ab86bf6-5147-4edc-b660-9f144520f444" containerName="util"
Apr 24 14:33:47.327452 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:33:47.327187    2574 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c3f2eaf3-730c-4f91-8700-498b592681f3" containerName="pull"
Apr 24 14:33:47.327452 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:33:47.327195    2574 state_mem.go:107] "Deleted CPUSet assignment" podUID="c3f2eaf3-730c-4f91-8700-498b592681f3" containerName="pull"
Apr 24 14:33:47.327452 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:33:47.327204    2574 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="6ab86bf6-5147-4edc-b660-9f144520f444" containerName="pull"
Apr 24 14:33:47.327452 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:33:47.327212    2574 state_mem.go:107] "Deleted CPUSet assignment" podUID="6ab86bf6-5147-4edc-b660-9f144520f444" containerName="pull"
Apr 24 14:33:47.327452 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:33:47.327223    2574 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="fbd7920a-c23d-4bf8-b4af-25408c94e843" containerName="extract"
Apr 24 14:33:47.327452 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:33:47.327230    2574 state_mem.go:107] "Deleted CPUSet assignment" podUID="fbd7920a-c23d-4bf8-b4af-25408c94e843" containerName="extract"
Apr 24 14:33:47.327452 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:33:47.327245    2574 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e02d2148-c267-43de-bd3d-9339754a382f" containerName="extract"
Apr 24 14:33:47.327452 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:33:47.327254    2574 state_mem.go:107] "Deleted CPUSet assignment" podUID="e02d2148-c267-43de-bd3d-9339754a382f" containerName="extract"
Apr 24 14:33:47.327452 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:33:47.327263    2574 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e02d2148-c267-43de-bd3d-9339754a382f" containerName="pull"
Apr 24 14:33:47.327452 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:33:47.327272    2574 state_mem.go:107] "Deleted CPUSet assignment" podUID="e02d2148-c267-43de-bd3d-9339754a382f" containerName="pull"
Apr 24 14:33:47.327452 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:33:47.327383    2574 memory_manager.go:356] "RemoveStaleState removing state" podUID="e02d2148-c267-43de-bd3d-9339754a382f" containerName="extract"
Apr 24 14:33:47.327452 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:33:47.327398    2574 memory_manager.go:356] "RemoveStaleState removing state" podUID="fbd7920a-c23d-4bf8-b4af-25408c94e843" containerName="extract"
Apr 24 14:33:47.327452 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:33:47.327409    2574 memory_manager.go:356] "RemoveStaleState removing state" podUID="6ab86bf6-5147-4edc-b660-9f144520f444" containerName="extract"
Apr 24 14:33:47.327452 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:33:47.327422    2574 memory_manager.go:356] "RemoveStaleState removing state" podUID="c3f2eaf3-730c-4f91-8700-498b592681f3" containerName="extract"
Apr 24 14:33:47.329561 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:33:47.329542    2574 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/dns-operator-controller-manager-844548ff4c-qzx86"
Apr 24 14:33:47.331610 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:33:47.331586    2574 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"openshift-service-ca.crt\""
Apr 24 14:33:47.331729 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:33:47.331607    2574 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"dns-operator-controller-env\""
Apr 24 14:33:47.331729 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:33:47.331698    2574 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"kube-root-ca.crt\""
Apr 24 14:33:47.332063 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:33:47.332048    2574 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"dns-operator-controller-manager-dockercfg-8b9qc\""
Apr 24 14:33:47.340185 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:33:47.340162    2574 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/dns-operator-controller-manager-844548ff4c-qzx86"]
Apr 24 14:33:47.437656 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:33:47.437621    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-97wlt\" (UniqueName: \"kubernetes.io/projected/323de2eb-8fc1-4879-92cb-5b7daa1c7a99-kube-api-access-97wlt\") pod \"dns-operator-controller-manager-844548ff4c-qzx86\" (UID: \"323de2eb-8fc1-4879-92cb-5b7daa1c7a99\") " pod="kuadrant-system/dns-operator-controller-manager-844548ff4c-qzx86"
Apr 24 14:33:47.538692 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:33:47.538637    2574 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-97wlt\" (UniqueName: \"kubernetes.io/projected/323de2eb-8fc1-4879-92cb-5b7daa1c7a99-kube-api-access-97wlt\") pod \"dns-operator-controller-manager-844548ff4c-qzx86\" (UID: \"323de2eb-8fc1-4879-92cb-5b7daa1c7a99\") " pod="kuadrant-system/dns-operator-controller-manager-844548ff4c-qzx86"
Apr 24 14:33:47.547773 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:33:47.547738    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-97wlt\" (UniqueName: \"kubernetes.io/projected/323de2eb-8fc1-4879-92cb-5b7daa1c7a99-kube-api-access-97wlt\") pod \"dns-operator-controller-manager-844548ff4c-qzx86\" (UID: \"323de2eb-8fc1-4879-92cb-5b7daa1c7a99\") " pod="kuadrant-system/dns-operator-controller-manager-844548ff4c-qzx86"
Apr 24 14:33:47.641347 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:33:47.641304    2574 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/dns-operator-controller-manager-844548ff4c-qzx86"
Apr 24 14:33:47.772179 ip-10-0-136-132 kubenswrapper[2574]: W0424 14:33:47.772141    2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod323de2eb_8fc1_4879_92cb_5b7daa1c7a99.slice/crio-ea99a78313e05647b46c9fea079fad9675bf3c63481c393f1da2ec9a94a3cc04 WatchSource:0}: Error finding container ea99a78313e05647b46c9fea079fad9675bf3c63481c393f1da2ec9a94a3cc04: Status 404 returned error can't find the container with id ea99a78313e05647b46c9fea079fad9675bf3c63481c393f1da2ec9a94a3cc04
Apr 24 14:33:47.772179 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:33:47.772153    2574 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/dns-operator-controller-manager-844548ff4c-qzx86"]
Apr 24 14:33:47.827648 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:33:47.827615    2574 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/dns-operator-controller-manager-844548ff4c-qzx86" event={"ID":"323de2eb-8fc1-4879-92cb-5b7daa1c7a99","Type":"ContainerStarted","Data":"ea99a78313e05647b46c9fea079fad9675bf3c63481c393f1da2ec9a94a3cc04"}
Apr 24 14:33:49.933304 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:33:49.933264    2574 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-operator-7587b89b76-649lf"]
Apr 24 14:33:49.937445 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:33:49.937416    2574 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-operator-7587b89b76-649lf"
Apr 24 14:33:49.939588 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:33:49.939558    2574 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"authorino-operator-dockercfg-vj564\""
Apr 24 14:33:49.947805 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:33:49.947781    2574 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-operator-7587b89b76-649lf"]
Apr 24 14:33:50.061703 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:33:50.061651    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-lf9hw\" (UniqueName: \"kubernetes.io/projected/a5174eb2-b50f-486d-a789-823a9f147bde-kube-api-access-lf9hw\") pod \"authorino-operator-7587b89b76-649lf\" (UID: \"a5174eb2-b50f-486d-a789-823a9f147bde\") " pod="kuadrant-system/authorino-operator-7587b89b76-649lf"
Apr 24 14:33:50.163090 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:33:50.162928    2574 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-lf9hw\" (UniqueName: \"kubernetes.io/projected/a5174eb2-b50f-486d-a789-823a9f147bde-kube-api-access-lf9hw\") pod \"authorino-operator-7587b89b76-649lf\" (UID: \"a5174eb2-b50f-486d-a789-823a9f147bde\") " pod="kuadrant-system/authorino-operator-7587b89b76-649lf"
Apr 24 14:33:50.172106 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:33:50.172047    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-lf9hw\" (UniqueName: \"kubernetes.io/projected/a5174eb2-b50f-486d-a789-823a9f147bde-kube-api-access-lf9hw\") pod \"authorino-operator-7587b89b76-649lf\" (UID: \"a5174eb2-b50f-486d-a789-823a9f147bde\") " pod="kuadrant-system/authorino-operator-7587b89b76-649lf"
Apr 24 14:33:50.253783 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:33:50.253682    2574 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-operator-7587b89b76-649lf"
Apr 24 14:33:50.410402 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:33:50.410374    2574 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-operator-7587b89b76-649lf"]
Apr 24 14:33:50.411608 ip-10-0-136-132 kubenswrapper[2574]: W0424 14:33:50.411581    2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda5174eb2_b50f_486d_a789_823a9f147bde.slice/crio-bf7765421822cf0c7c4c197cdbd2649ee7b9aa8f349e081f34cc710ef2617679 WatchSource:0}: Error finding container bf7765421822cf0c7c4c197cdbd2649ee7b9aa8f349e081f34cc710ef2617679: Status 404 returned error can't find the container with id bf7765421822cf0c7c4c197cdbd2649ee7b9aa8f349e081f34cc710ef2617679
Apr 24 14:33:50.841383 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:33:50.841312    2574 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-operator-7587b89b76-649lf" event={"ID":"a5174eb2-b50f-486d-a789-823a9f147bde","Type":"ContainerStarted","Data":"bf7765421822cf0c7c4c197cdbd2649ee7b9aa8f349e081f34cc710ef2617679"}
Apr 24 14:33:51.847987 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:33:51.847889    2574 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/dns-operator-controller-manager-844548ff4c-qzx86" event={"ID":"323de2eb-8fc1-4879-92cb-5b7daa1c7a99","Type":"ContainerStarted","Data":"5514c9eefda2ee68274e898d0b6cdf9bf4fcb00fb82b5f155b91e68a594ba76d"}
Apr 24 14:33:51.848449 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:33:51.848041    2574 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/dns-operator-controller-manager-844548ff4c-qzx86"
Apr 24 14:33:51.865425 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:33:51.865364    2574 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="kuadrant-system/dns-operator-controller-manager-844548ff4c-qzx86" podStartSLOduration=1.049429723 podStartE2EDuration="4.865329318s" podCreationTimestamp="2026-04-24 14:33:47 +0000 UTC" firstStartedPulling="2026-04-24 14:33:47.774186405 +0000 UTC m=+590.506288036" lastFinishedPulling="2026-04-24 14:33:51.590086 +0000 UTC m=+594.322187631" observedRunningTime="2026-04-24 14:33:51.862777894 +0000 UTC m=+594.594879544" watchObservedRunningTime="2026-04-24 14:33:51.865329318 +0000 UTC m=+594.597430968"
Apr 24 14:33:52.157525 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:33:52.157485    2574 kubelet.go:2553] "SyncLoop DELETE" source="api" pods=["openshift-console/console-554bd76846-mq6sk"]
Apr 24 14:33:52.853255 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:33:52.853215    2574 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-operator-7587b89b76-649lf" event={"ID":"a5174eb2-b50f-486d-a789-823a9f147bde","Type":"ContainerStarted","Data":"6e51de60505960959b6b43549a4c41f06f6c6e18c69310cc024d54b50fdbe384"}
Apr 24 14:33:52.853682 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:33:52.853279    2574 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/authorino-operator-7587b89b76-649lf"
Apr 24 14:33:52.870698 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:33:52.870644    2574 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="kuadrant-system/authorino-operator-7587b89b76-649lf" podStartSLOduration=2.011856973 podStartE2EDuration="3.870626616s" podCreationTimestamp="2026-04-24 14:33:49 +0000 UTC" firstStartedPulling="2026-04-24 14:33:50.414153624 +0000 UTC m=+593.146255252" lastFinishedPulling="2026-04-24 14:33:52.272923264 +0000 UTC m=+595.005024895" observedRunningTime="2026-04-24 14:33:52.867828629 +0000 UTC m=+595.599930277" watchObservedRunningTime="2026-04-24 14:33:52.870626616 +0000 UTC m=+595.602728265"
Apr 24 14:33:57.727499 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:33:57.727469    2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-9d4b6777b-f6ngr_2571829f-666a-422a-9b02-2d2bb4c89b25/console-operator/1.log"
Apr 24 14:33:57.728119 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:33:57.728098    2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-9d4b6777b-f6ngr_2571829f-666a-422a-9b02-2d2bb4c89b25/console-operator/1.log"
Apr 24 14:33:57.732109 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:33:57.732087    2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-w5x5x_44036c3e-1fdc-40d4-899c-9009cd4589e3/ovn-acl-logging/0.log"
Apr 24 14:33:57.732554 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:33:57.732525    2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-w5x5x_44036c3e-1fdc-40d4-899c-9009cd4589e3/ovn-acl-logging/0.log"
Apr 24 14:34:02.856460 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:34:02.856423    2574 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/dns-operator-controller-manager-844548ff4c-qzx86"
Apr 24 14:34:03.859482 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:34:03.859447    2574 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/authorino-operator-7587b89b76-649lf"
Apr 24 14:34:17.184899 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:34:17.184837    2574 kuberuntime_container.go:864] "Killing container with a grace period" pod="openshift-console/console-554bd76846-mq6sk" podUID="26d01817-940e-4b88-8608-22f16a507506" containerName="console" containerID="cri-o://25768a7c7bebb37f5c591a510780d4644355e8f443cf6539994832d3a92aefd8" gracePeriod=15
Apr 24 14:34:17.432585 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:34:17.432558    2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console_console-554bd76846-mq6sk_26d01817-940e-4b88-8608-22f16a507506/console/0.log"
Apr 24 14:34:17.432719 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:34:17.432620    2574 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-554bd76846-mq6sk"
Apr 24 14:34:17.514034 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:34:17.513950    2574 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/26d01817-940e-4b88-8608-22f16a507506-trusted-ca-bundle\") pod \"26d01817-940e-4b88-8608-22f16a507506\" (UID: \"26d01817-940e-4b88-8608-22f16a507506\") "
Apr 24 14:34:17.514034 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:34:17.514012    2574 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"kube-api-access-x5dcc\" (UniqueName: \"kubernetes.io/projected/26d01817-940e-4b88-8608-22f16a507506-kube-api-access-x5dcc\") pod \"26d01817-940e-4b88-8608-22f16a507506\" (UID: \"26d01817-940e-4b88-8608-22f16a507506\") "
Apr 24 14:34:17.514034 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:34:17.514029    2574 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/26d01817-940e-4b88-8608-22f16a507506-console-oauth-config\") pod \"26d01817-940e-4b88-8608-22f16a507506\" (UID: \"26d01817-940e-4b88-8608-22f16a507506\") "
Apr 24 14:34:17.514289 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:34:17.514047    2574 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/26d01817-940e-4b88-8608-22f16a507506-service-ca\") pod \"26d01817-940e-4b88-8608-22f16a507506\" (UID: \"26d01817-940e-4b88-8608-22f16a507506\") "
Apr 24 14:34:17.514289 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:34:17.514097    2574 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/26d01817-940e-4b88-8608-22f16a507506-console-config\") pod \"26d01817-940e-4b88-8608-22f16a507506\" (UID: \"26d01817-940e-4b88-8608-22f16a507506\") "
Apr 24 14:34:17.514289 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:34:17.514134    2574 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/26d01817-940e-4b88-8608-22f16a507506-console-serving-cert\") pod \"26d01817-940e-4b88-8608-22f16a507506\" (UID: \"26d01817-940e-4b88-8608-22f16a507506\") "
Apr 24 14:34:17.514289 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:34:17.514252    2574 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/26d01817-940e-4b88-8608-22f16a507506-oauth-serving-cert\") pod \"26d01817-940e-4b88-8608-22f16a507506\" (UID: \"26d01817-940e-4b88-8608-22f16a507506\") "
Apr 24 14:34:17.514631 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:34:17.514587    2574 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/26d01817-940e-4b88-8608-22f16a507506-trusted-ca-bundle" (OuterVolumeSpecName: "trusted-ca-bundle") pod "26d01817-940e-4b88-8608-22f16a507506" (UID: "26d01817-940e-4b88-8608-22f16a507506"). InnerVolumeSpecName "trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Apr 24 14:34:17.514775 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:34:17.514647    2574 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/26d01817-940e-4b88-8608-22f16a507506-service-ca" (OuterVolumeSpecName: "service-ca") pod "26d01817-940e-4b88-8608-22f16a507506" (UID: "26d01817-940e-4b88-8608-22f16a507506"). InnerVolumeSpecName "service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Apr 24 14:34:17.514775 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:34:17.514669    2574 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/26d01817-940e-4b88-8608-22f16a507506-oauth-serving-cert" (OuterVolumeSpecName: "oauth-serving-cert") pod "26d01817-940e-4b88-8608-22f16a507506" (UID: "26d01817-940e-4b88-8608-22f16a507506"). InnerVolumeSpecName "oauth-serving-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Apr 24 14:34:17.514891 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:34:17.514826    2574 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/26d01817-940e-4b88-8608-22f16a507506-console-config" (OuterVolumeSpecName: "console-config") pod "26d01817-940e-4b88-8608-22f16a507506" (UID: "26d01817-940e-4b88-8608-22f16a507506"). InnerVolumeSpecName "console-config". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Apr 24 14:34:17.514891 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:34:17.514834    2574 reconciler_common.go:299] "Volume detached for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/26d01817-940e-4b88-8608-22f16a507506-oauth-serving-cert\") on node \"ip-10-0-136-132.ec2.internal\" DevicePath \"\""
Apr 24 14:34:17.514891 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:34:17.514869    2574 reconciler_common.go:299] "Volume detached for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/26d01817-940e-4b88-8608-22f16a507506-trusted-ca-bundle\") on node \"ip-10-0-136-132.ec2.internal\" DevicePath \"\""
Apr 24 14:34:17.514891 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:34:17.514886    2574 reconciler_common.go:299] "Volume detached for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/26d01817-940e-4b88-8608-22f16a507506-service-ca\") on node \"ip-10-0-136-132.ec2.internal\" DevicePath \"\""
Apr 24 14:34:17.516424 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:34:17.516401    2574 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/26d01817-940e-4b88-8608-22f16a507506-console-serving-cert" (OuterVolumeSpecName: "console-serving-cert") pod "26d01817-940e-4b88-8608-22f16a507506" (UID: "26d01817-940e-4b88-8608-22f16a507506"). InnerVolumeSpecName "console-serving-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Apr 24 14:34:17.516529 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:34:17.516436    2574 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/26d01817-940e-4b88-8608-22f16a507506-kube-api-access-x5dcc" (OuterVolumeSpecName: "kube-api-access-x5dcc") pod "26d01817-940e-4b88-8608-22f16a507506" (UID: "26d01817-940e-4b88-8608-22f16a507506"). InnerVolumeSpecName "kube-api-access-x5dcc". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Apr 24 14:34:17.516529 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:34:17.516442    2574 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/26d01817-940e-4b88-8608-22f16a507506-console-oauth-config" (OuterVolumeSpecName: "console-oauth-config") pod "26d01817-940e-4b88-8608-22f16a507506" (UID: "26d01817-940e-4b88-8608-22f16a507506"). InnerVolumeSpecName "console-oauth-config". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Apr 24 14:34:17.615532 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:34:17.615495    2574 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-x5dcc\" (UniqueName: \"kubernetes.io/projected/26d01817-940e-4b88-8608-22f16a507506-kube-api-access-x5dcc\") on node \"ip-10-0-136-132.ec2.internal\" DevicePath \"\""
Apr 24 14:34:17.615532 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:34:17.615528    2574 reconciler_common.go:299] "Volume detached for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/26d01817-940e-4b88-8608-22f16a507506-console-oauth-config\") on node \"ip-10-0-136-132.ec2.internal\" DevicePath \"\""
Apr 24 14:34:17.615532 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:34:17.615538    2574 reconciler_common.go:299] "Volume detached for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/26d01817-940e-4b88-8608-22f16a507506-console-config\") on node \"ip-10-0-136-132.ec2.internal\" DevicePath \"\""
Apr 24 14:34:17.615758 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:34:17.615547    2574 reconciler_common.go:299] "Volume detached for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/26d01817-940e-4b88-8608-22f16a507506-console-serving-cert\") on node \"ip-10-0-136-132.ec2.internal\" DevicePath \"\""
Apr 24 14:34:17.953499 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:34:17.953470    2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console_console-554bd76846-mq6sk_26d01817-940e-4b88-8608-22f16a507506/console/0.log"
Apr 24 14:34:17.953697 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:34:17.953515    2574 generic.go:358] "Generic (PLEG): container finished" podID="26d01817-940e-4b88-8608-22f16a507506" containerID="25768a7c7bebb37f5c591a510780d4644355e8f443cf6539994832d3a92aefd8" exitCode=2
Apr 24 14:34:17.953697 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:34:17.953594    2574 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-554bd76846-mq6sk"
Apr 24 14:34:17.953697 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:34:17.953614    2574 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-554bd76846-mq6sk" event={"ID":"26d01817-940e-4b88-8608-22f16a507506","Type":"ContainerDied","Data":"25768a7c7bebb37f5c591a510780d4644355e8f443cf6539994832d3a92aefd8"}
Apr 24 14:34:17.953697 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:34:17.953661    2574 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-554bd76846-mq6sk" event={"ID":"26d01817-940e-4b88-8608-22f16a507506","Type":"ContainerDied","Data":"84be2b2daef1e7faafd00c96f4ec36c875e98d193c85e7be05f52b1fa032eaaf"}
Apr 24 14:34:17.953697 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:34:17.953681    2574 scope.go:117] "RemoveContainer" containerID="25768a7c7bebb37f5c591a510780d4644355e8f443cf6539994832d3a92aefd8"
Apr 24 14:34:17.962285 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:34:17.962266    2574 scope.go:117] "RemoveContainer" containerID="25768a7c7bebb37f5c591a510780d4644355e8f443cf6539994832d3a92aefd8"
Apr 24 14:34:17.962557 ip-10-0-136-132 kubenswrapper[2574]: E0424 14:34:17.962537    2574 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"25768a7c7bebb37f5c591a510780d4644355e8f443cf6539994832d3a92aefd8\": container with ID starting with 25768a7c7bebb37f5c591a510780d4644355e8f443cf6539994832d3a92aefd8 not found: ID does not exist" containerID="25768a7c7bebb37f5c591a510780d4644355e8f443cf6539994832d3a92aefd8"
Apr 24 14:34:17.962631 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:34:17.962564    2574 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"25768a7c7bebb37f5c591a510780d4644355e8f443cf6539994832d3a92aefd8"} err="failed to get container status \"25768a7c7bebb37f5c591a510780d4644355e8f443cf6539994832d3a92aefd8\": rpc error: code = NotFound desc = could not find container \"25768a7c7bebb37f5c591a510780d4644355e8f443cf6539994832d3a92aefd8\": container with ID starting with 25768a7c7bebb37f5c591a510780d4644355e8f443cf6539994832d3a92aefd8 not found: ID does not exist"
Apr 24 14:34:17.969070 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:34:17.969049    2574 kubelet.go:2553] "SyncLoop DELETE" source="api" pods=["openshift-console/console-554bd76846-mq6sk"]
Apr 24 14:34:17.974858 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:34:17.974835    2574 kubelet.go:2547] "SyncLoop REMOVE" source="api" pods=["openshift-console/console-554bd76846-mq6sk"]
Apr 24 14:34:19.771229 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:34:19.771198    2574 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="26d01817-940e-4b88-8608-22f16a507506" path="/var/lib/kubelet/pods/26d01817-940e-4b88-8608-22f16a507506/volumes"
Apr 24 14:34:38.241475 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:34:38.241441    2574 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-674b59b84c-85wgl"]
Apr 24 14:34:38.241956 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:34:38.241939    2574 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="26d01817-940e-4b88-8608-22f16a507506" containerName="console"
Apr 24 14:34:38.241999 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:34:38.241958    2574 state_mem.go:107] "Deleted CPUSet assignment" podUID="26d01817-940e-4b88-8608-22f16a507506" containerName="console"
Apr 24 14:34:38.242040 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:34:38.242031    2574 memory_manager.go:356] "RemoveStaleState removing state" podUID="26d01817-940e-4b88-8608-22f16a507506" containerName="console"
Apr 24 14:34:38.249206 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:34:38.249179    2574 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-674b59b84c-85wgl"
Apr 24 14:34:38.251159 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:34:38.251136    2574 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"authorino-authorino-dockercfg-vdx7l\""
Apr 24 14:34:38.251376 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:34:38.251355    2574 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-674b59b84c-85wgl"]
Apr 24 14:34:38.333110 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:34:38.333079    2574 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-79cbc94b89-dtgl2"]
Apr 24 14:34:38.336696 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:34:38.336681    2574 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-79cbc94b89-dtgl2"
Apr 24 14:34:38.346884 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:34:38.346857    2574 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-79cbc94b89-dtgl2"]
Apr 24 14:34:38.408098 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:34:38.408074    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-cfrlv\" (UniqueName: \"kubernetes.io/projected/6f6362ca-5f95-4ffa-887f-79882d2a5a0c-kube-api-access-cfrlv\") pod \"authorino-674b59b84c-85wgl\" (UID: \"6f6362ca-5f95-4ffa-887f-79882d2a5a0c\") " pod="kuadrant-system/authorino-674b59b84c-85wgl"
Apr 24 14:34:38.508983 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:34:38.508911    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-fjgzq\" (UniqueName: \"kubernetes.io/projected/9a8d92a9-b51b-41dd-a3f3-07290bff973b-kube-api-access-fjgzq\") pod \"authorino-79cbc94b89-dtgl2\" (UID: \"9a8d92a9-b51b-41dd-a3f3-07290bff973b\") " pod="kuadrant-system/authorino-79cbc94b89-dtgl2"
Apr 24 14:34:38.508983 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:34:38.508965    2574 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-cfrlv\" (UniqueName: \"kubernetes.io/projected/6f6362ca-5f95-4ffa-887f-79882d2a5a0c-kube-api-access-cfrlv\") pod \"authorino-674b59b84c-85wgl\" (UID: \"6f6362ca-5f95-4ffa-887f-79882d2a5a0c\") " pod="kuadrant-system/authorino-674b59b84c-85wgl"
Apr 24 14:34:38.518732 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:34:38.518709    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-cfrlv\" (UniqueName: \"kubernetes.io/projected/6f6362ca-5f95-4ffa-887f-79882d2a5a0c-kube-api-access-cfrlv\") pod \"authorino-674b59b84c-85wgl\" (UID: \"6f6362ca-5f95-4ffa-887f-79882d2a5a0c\") " pod="kuadrant-system/authorino-674b59b84c-85wgl"
Apr 24 14:34:38.559471 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:34:38.559444    2574 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-674b59b84c-85wgl"
Apr 24 14:34:38.609596 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:34:38.609562    2574 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-fjgzq\" (UniqueName: \"kubernetes.io/projected/9a8d92a9-b51b-41dd-a3f3-07290bff973b-kube-api-access-fjgzq\") pod \"authorino-79cbc94b89-dtgl2\" (UID: \"9a8d92a9-b51b-41dd-a3f3-07290bff973b\") " pod="kuadrant-system/authorino-79cbc94b89-dtgl2"
Apr 24 14:34:38.616877 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:34:38.616851    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-fjgzq\" (UniqueName: \"kubernetes.io/projected/9a8d92a9-b51b-41dd-a3f3-07290bff973b-kube-api-access-fjgzq\") pod \"authorino-79cbc94b89-dtgl2\" (UID: \"9a8d92a9-b51b-41dd-a3f3-07290bff973b\") " pod="kuadrant-system/authorino-79cbc94b89-dtgl2"
Apr 24 14:34:38.646658 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:34:38.646619    2574 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-79cbc94b89-dtgl2"
Apr 24 14:34:38.681745 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:34:38.681718    2574 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-674b59b84c-85wgl"]
Apr 24 14:34:38.683270 ip-10-0-136-132 kubenswrapper[2574]: W0424 14:34:38.683238    2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod6f6362ca_5f95_4ffa_887f_79882d2a5a0c.slice/crio-b7e301c01738434eb05b471fc01551795e59324ea55a74523375c85e416733b6 WatchSource:0}: Error finding container b7e301c01738434eb05b471fc01551795e59324ea55a74523375c85e416733b6: Status 404 returned error can't find the container with id b7e301c01738434eb05b471fc01551795e59324ea55a74523375c85e416733b6
Apr 24 14:34:38.770107 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:34:38.770088    2574 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-79cbc94b89-dtgl2"]
Apr 24 14:34:38.771695 ip-10-0-136-132 kubenswrapper[2574]: W0424 14:34:38.771671    2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod9a8d92a9_b51b_41dd_a3f3_07290bff973b.slice/crio-a9aa0cdfee5b565c32bff9e9b3483c69dda1de51702b3d54b8a731b746356906 WatchSource:0}: Error finding container a9aa0cdfee5b565c32bff9e9b3483c69dda1de51702b3d54b8a731b746356906: Status 404 returned error can't find the container with id a9aa0cdfee5b565c32bff9e9b3483c69dda1de51702b3d54b8a731b746356906
Apr 24 14:34:39.041945 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:34:39.041849    2574 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-79cbc94b89-dtgl2" event={"ID":"9a8d92a9-b51b-41dd-a3f3-07290bff973b","Type":"ContainerStarted","Data":"a9aa0cdfee5b565c32bff9e9b3483c69dda1de51702b3d54b8a731b746356906"}
Apr 24 14:34:39.042836 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:34:39.042812    2574 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-674b59b84c-85wgl" event={"ID":"6f6362ca-5f95-4ffa-887f-79882d2a5a0c","Type":"ContainerStarted","Data":"b7e301c01738434eb05b471fc01551795e59324ea55a74523375c85e416733b6"}
Apr 24 14:34:43.065625 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:34:43.065586    2574 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-674b59b84c-85wgl" event={"ID":"6f6362ca-5f95-4ffa-887f-79882d2a5a0c","Type":"ContainerStarted","Data":"65098aebb89b6d1a796324d0a84a742fb575d6464731ad29b88f36f4e7992568"}
Apr 24 14:34:43.066901 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:34:43.066877    2574 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-79cbc94b89-dtgl2" event={"ID":"9a8d92a9-b51b-41dd-a3f3-07290bff973b","Type":"ContainerStarted","Data":"1166ef57886c453c670a285b5974d2d15a44bf3f0749c35731735eb4cc102033"}
Apr 24 14:34:43.079824 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:34:43.079777    2574 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="kuadrant-system/authorino-674b59b84c-85wgl" podStartSLOduration=1.154880569 podStartE2EDuration="5.079763154s" podCreationTimestamp="2026-04-24 14:34:38 +0000 UTC" firstStartedPulling="2026-04-24 14:34:38.684546578 +0000 UTC m=+641.416648205" lastFinishedPulling="2026-04-24 14:34:42.609429162 +0000 UTC m=+645.341530790" observedRunningTime="2026-04-24 14:34:43.077838771 +0000 UTC m=+645.809940420" watchObservedRunningTime="2026-04-24 14:34:43.079763154 +0000 UTC m=+645.811864869"
Apr 24 14:34:43.092258 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:34:43.092199    2574 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="kuadrant-system/authorino-79cbc94b89-dtgl2" podStartSLOduration=1.243095036 podStartE2EDuration="5.092183044s" podCreationTimestamp="2026-04-24 14:34:38 +0000 UTC" firstStartedPulling="2026-04-24 14:34:38.773049746 +0000 UTC m=+641.505151374" lastFinishedPulling="2026-04-24 14:34:42.622137739 +0000 UTC m=+645.354239382" observedRunningTime="2026-04-24 14:34:43.089547281 +0000 UTC m=+645.821648925" watchObservedRunningTime="2026-04-24 14:34:43.092183044 +0000 UTC m=+645.824284693"
Apr 24 14:34:43.116089 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:34:43.116054    2574 kubelet.go:2553] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-674b59b84c-85wgl"]
Apr 24 14:34:45.074124 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:34:45.074085    2574 kuberuntime_container.go:864] "Killing container with a grace period" pod="kuadrant-system/authorino-674b59b84c-85wgl" podUID="6f6362ca-5f95-4ffa-887f-79882d2a5a0c" containerName="authorino" containerID="cri-o://65098aebb89b6d1a796324d0a84a742fb575d6464731ad29b88f36f4e7992568" gracePeriod=30
Apr 24 14:34:45.319447 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:34:45.319422    2574 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-674b59b84c-85wgl"
Apr 24 14:34:45.365812 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:34:45.365714    2574 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"kube-api-access-cfrlv\" (UniqueName: \"kubernetes.io/projected/6f6362ca-5f95-4ffa-887f-79882d2a5a0c-kube-api-access-cfrlv\") pod \"6f6362ca-5f95-4ffa-887f-79882d2a5a0c\" (UID: \"6f6362ca-5f95-4ffa-887f-79882d2a5a0c\") "
Apr 24 14:34:45.367855 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:34:45.367822    2574 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/6f6362ca-5f95-4ffa-887f-79882d2a5a0c-kube-api-access-cfrlv" (OuterVolumeSpecName: "kube-api-access-cfrlv") pod "6f6362ca-5f95-4ffa-887f-79882d2a5a0c" (UID: "6f6362ca-5f95-4ffa-887f-79882d2a5a0c"). InnerVolumeSpecName "kube-api-access-cfrlv". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Apr 24 14:34:45.467036 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:34:45.466990    2574 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-cfrlv\" (UniqueName: \"kubernetes.io/projected/6f6362ca-5f95-4ffa-887f-79882d2a5a0c-kube-api-access-cfrlv\") on node \"ip-10-0-136-132.ec2.internal\" DevicePath \"\""
Apr 24 14:34:46.079588 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:34:46.079497    2574 generic.go:358] "Generic (PLEG): container finished" podID="6f6362ca-5f95-4ffa-887f-79882d2a5a0c" containerID="65098aebb89b6d1a796324d0a84a742fb575d6464731ad29b88f36f4e7992568" exitCode=0
Apr 24 14:34:46.079588 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:34:46.079547    2574 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-674b59b84c-85wgl"
Apr 24 14:34:46.080046 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:34:46.079580    2574 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-674b59b84c-85wgl" event={"ID":"6f6362ca-5f95-4ffa-887f-79882d2a5a0c","Type":"ContainerDied","Data":"65098aebb89b6d1a796324d0a84a742fb575d6464731ad29b88f36f4e7992568"}
Apr 24 14:34:46.080046 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:34:46.079618    2574 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-674b59b84c-85wgl" event={"ID":"6f6362ca-5f95-4ffa-887f-79882d2a5a0c","Type":"ContainerDied","Data":"b7e301c01738434eb05b471fc01551795e59324ea55a74523375c85e416733b6"}
Apr 24 14:34:46.080046 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:34:46.079633    2574 scope.go:117] "RemoveContainer" containerID="65098aebb89b6d1a796324d0a84a742fb575d6464731ad29b88f36f4e7992568"
Apr 24 14:34:46.088716 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:34:46.088678    2574 scope.go:117] "RemoveContainer" containerID="65098aebb89b6d1a796324d0a84a742fb575d6464731ad29b88f36f4e7992568"
Apr 24 14:34:46.088981 ip-10-0-136-132 kubenswrapper[2574]: E0424 14:34:46.088963    2574 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"65098aebb89b6d1a796324d0a84a742fb575d6464731ad29b88f36f4e7992568\": container with ID starting with 65098aebb89b6d1a796324d0a84a742fb575d6464731ad29b88f36f4e7992568 not found: ID does not exist" containerID="65098aebb89b6d1a796324d0a84a742fb575d6464731ad29b88f36f4e7992568"
Apr 24 14:34:46.089028 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:34:46.088990    2574 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"65098aebb89b6d1a796324d0a84a742fb575d6464731ad29b88f36f4e7992568"} err="failed to get container status \"65098aebb89b6d1a796324d0a84a742fb575d6464731ad29b88f36f4e7992568\": rpc error: code = NotFound desc = could not find container \"65098aebb89b6d1a796324d0a84a742fb575d6464731ad29b88f36f4e7992568\": container with ID starting with 65098aebb89b6d1a796324d0a84a742fb575d6464731ad29b88f36f4e7992568 not found: ID does not exist"
Apr 24 14:34:46.094326 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:34:46.094303    2574 kubelet.go:2553] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-674b59b84c-85wgl"]
Apr 24 14:34:46.096623 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:34:46.096603    2574 kubelet.go:2547] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/authorino-674b59b84c-85wgl"]
Apr 24 14:34:47.771171 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:34:47.771128    2574 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="6f6362ca-5f95-4ffa-887f-79882d2a5a0c" path="/var/lib/kubelet/pods/6f6362ca-5f95-4ffa-887f-79882d2a5a0c/volumes"
Apr 24 14:35:02.477035 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:35:02.476997    2574 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-68bd676465-8cn2c"]
Apr 24 14:35:02.477483 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:35:02.477409    2574 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="6f6362ca-5f95-4ffa-887f-79882d2a5a0c" containerName="authorino"
Apr 24 14:35:02.477483 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:35:02.477425    2574 state_mem.go:107] "Deleted CPUSet assignment" podUID="6f6362ca-5f95-4ffa-887f-79882d2a5a0c" containerName="authorino"
Apr 24 14:35:02.477604 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:35:02.477520    2574 memory_manager.go:356] "RemoveStaleState removing state" podUID="6f6362ca-5f95-4ffa-887f-79882d2a5a0c" containerName="authorino"
Apr 24 14:35:02.485415 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:35:02.485390    2574 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-68bd676465-8cn2c"
Apr 24 14:35:02.487391 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:35:02.487360    2574 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-68bd676465-8cn2c"]
Apr 24 14:35:02.487563 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:35:02.487538    2574 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"authorino-server-cert\""
Apr 24 14:35:02.611584 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:35:02.611549    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/98a9fc6f-299c-4729-8d6b-24d9464f30b7-tls-cert\") pod \"authorino-68bd676465-8cn2c\" (UID: \"98a9fc6f-299c-4729-8d6b-24d9464f30b7\") " pod="kuadrant-system/authorino-68bd676465-8cn2c"
Apr 24 14:35:02.611736 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:35:02.611626    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-lhkhk\" (UniqueName: \"kubernetes.io/projected/98a9fc6f-299c-4729-8d6b-24d9464f30b7-kube-api-access-lhkhk\") pod \"authorino-68bd676465-8cn2c\" (UID: \"98a9fc6f-299c-4729-8d6b-24d9464f30b7\") " pod="kuadrant-system/authorino-68bd676465-8cn2c"
Apr 24 14:35:02.712023 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:35:02.711992    2574 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/98a9fc6f-299c-4729-8d6b-24d9464f30b7-tls-cert\") pod \"authorino-68bd676465-8cn2c\" (UID: \"98a9fc6f-299c-4729-8d6b-24d9464f30b7\") " pod="kuadrant-system/authorino-68bd676465-8cn2c"
Apr 24 14:35:02.712217 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:35:02.712055    2574 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-lhkhk\" (UniqueName: \"kubernetes.io/projected/98a9fc6f-299c-4729-8d6b-24d9464f30b7-kube-api-access-lhkhk\") pod \"authorino-68bd676465-8cn2c\" (UID: \"98a9fc6f-299c-4729-8d6b-24d9464f30b7\") " pod="kuadrant-system/authorino-68bd676465-8cn2c"
Apr 24 14:35:02.714539 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:35:02.714517    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/98a9fc6f-299c-4729-8d6b-24d9464f30b7-tls-cert\") pod \"authorino-68bd676465-8cn2c\" (UID: \"98a9fc6f-299c-4729-8d6b-24d9464f30b7\") " pod="kuadrant-system/authorino-68bd676465-8cn2c"
Apr 24 14:35:02.719071 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:35:02.719049    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-lhkhk\" (UniqueName: \"kubernetes.io/projected/98a9fc6f-299c-4729-8d6b-24d9464f30b7-kube-api-access-lhkhk\") pod \"authorino-68bd676465-8cn2c\" (UID: \"98a9fc6f-299c-4729-8d6b-24d9464f30b7\") " pod="kuadrant-system/authorino-68bd676465-8cn2c"
Apr 24 14:35:02.795176 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:35:02.795098    2574 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-68bd676465-8cn2c"
Apr 24 14:35:02.915208 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:35:02.915169    2574 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-68bd676465-8cn2c"]
Apr 24 14:35:02.917063 ip-10-0-136-132 kubenswrapper[2574]: W0424 14:35:02.917037    2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod98a9fc6f_299c_4729_8d6b_24d9464f30b7.slice/crio-fe3dbe0a703417a041198b57f57e6817f0b54ce8076143e44df4240ed884f46e WatchSource:0}: Error finding container fe3dbe0a703417a041198b57f57e6817f0b54ce8076143e44df4240ed884f46e: Status 404 returned error can't find the container with id fe3dbe0a703417a041198b57f57e6817f0b54ce8076143e44df4240ed884f46e
Apr 24 14:35:02.918562 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:35:02.918544    2574 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Apr 24 14:35:03.153689 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:35:03.153657    2574 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-68bd676465-8cn2c" event={"ID":"98a9fc6f-299c-4729-8d6b-24d9464f30b7","Type":"ContainerStarted","Data":"fe3dbe0a703417a041198b57f57e6817f0b54ce8076143e44df4240ed884f46e"}
Apr 24 14:35:04.158723 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:35:04.158691    2574 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-68bd676465-8cn2c" event={"ID":"98a9fc6f-299c-4729-8d6b-24d9464f30b7","Type":"ContainerStarted","Data":"f12586df64499929eb13e541ba3859fe1146630ec0c9ee5782273c9db0e5aa44"}
Apr 24 14:35:04.172884 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:35:04.172834    2574 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="kuadrant-system/authorino-68bd676465-8cn2c" podStartSLOduration=1.521919044 podStartE2EDuration="2.172820068s" podCreationTimestamp="2026-04-24 14:35:02 +0000 UTC" firstStartedPulling="2026-04-24 14:35:02.918717356 +0000 UTC m=+665.650818984" lastFinishedPulling="2026-04-24 14:35:03.569618382 +0000 UTC m=+666.301720008" observedRunningTime="2026-04-24 14:35:04.171981965 +0000 UTC m=+666.904083615" watchObservedRunningTime="2026-04-24 14:35:04.172820068 +0000 UTC m=+666.904921716"
Apr 24 14:35:04.200896 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:35:04.200864    2574 kubelet.go:2553] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-79cbc94b89-dtgl2"]
Apr 24 14:35:04.201142 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:35:04.201111    2574 kuberuntime_container.go:864] "Killing container with a grace period" pod="kuadrant-system/authorino-79cbc94b89-dtgl2" podUID="9a8d92a9-b51b-41dd-a3f3-07290bff973b" containerName="authorino" containerID="cri-o://1166ef57886c453c670a285b5974d2d15a44bf3f0749c35731735eb4cc102033" gracePeriod=30
Apr 24 14:35:04.442402 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:35:04.442378    2574 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-79cbc94b89-dtgl2"
Apr 24 14:35:04.530432 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:35:04.529000    2574 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"kube-api-access-fjgzq\" (UniqueName: \"kubernetes.io/projected/9a8d92a9-b51b-41dd-a3f3-07290bff973b-kube-api-access-fjgzq\") pod \"9a8d92a9-b51b-41dd-a3f3-07290bff973b\" (UID: \"9a8d92a9-b51b-41dd-a3f3-07290bff973b\") "
Apr 24 14:35:04.536395 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:35:04.536359    2574 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/9a8d92a9-b51b-41dd-a3f3-07290bff973b-kube-api-access-fjgzq" (OuterVolumeSpecName: "kube-api-access-fjgzq") pod "9a8d92a9-b51b-41dd-a3f3-07290bff973b" (UID: "9a8d92a9-b51b-41dd-a3f3-07290bff973b"). InnerVolumeSpecName "kube-api-access-fjgzq". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Apr 24 14:35:04.630797 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:35:04.630766    2574 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-fjgzq\" (UniqueName: \"kubernetes.io/projected/9a8d92a9-b51b-41dd-a3f3-07290bff973b-kube-api-access-fjgzq\") on node \"ip-10-0-136-132.ec2.internal\" DevicePath \"\""
Apr 24 14:35:05.164707 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:35:05.164670    2574 generic.go:358] "Generic (PLEG): container finished" podID="9a8d92a9-b51b-41dd-a3f3-07290bff973b" containerID="1166ef57886c453c670a285b5974d2d15a44bf3f0749c35731735eb4cc102033" exitCode=0
Apr 24 14:35:05.165187 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:35:05.164736    2574 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-79cbc94b89-dtgl2"
Apr 24 14:35:05.165187 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:35:05.164758    2574 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-79cbc94b89-dtgl2" event={"ID":"9a8d92a9-b51b-41dd-a3f3-07290bff973b","Type":"ContainerDied","Data":"1166ef57886c453c670a285b5974d2d15a44bf3f0749c35731735eb4cc102033"}
Apr 24 14:35:05.165187 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:35:05.164790    2574 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-79cbc94b89-dtgl2" event={"ID":"9a8d92a9-b51b-41dd-a3f3-07290bff973b","Type":"ContainerDied","Data":"a9aa0cdfee5b565c32bff9e9b3483c69dda1de51702b3d54b8a731b746356906"}
Apr 24 14:35:05.165187 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:35:05.164804    2574 scope.go:117] "RemoveContainer" containerID="1166ef57886c453c670a285b5974d2d15a44bf3f0749c35731735eb4cc102033"
Apr 24 14:35:05.173858 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:35:05.173841    2574 scope.go:117] "RemoveContainer" containerID="1166ef57886c453c670a285b5974d2d15a44bf3f0749c35731735eb4cc102033"
Apr 24 14:35:05.174103 ip-10-0-136-132 kubenswrapper[2574]: E0424 14:35:05.174082    2574 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"1166ef57886c453c670a285b5974d2d15a44bf3f0749c35731735eb4cc102033\": container with ID starting with 1166ef57886c453c670a285b5974d2d15a44bf3f0749c35731735eb4cc102033 not found: ID does not exist" containerID="1166ef57886c453c670a285b5974d2d15a44bf3f0749c35731735eb4cc102033"
Apr 24 14:35:05.174153 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:35:05.174112    2574 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"1166ef57886c453c670a285b5974d2d15a44bf3f0749c35731735eb4cc102033"} err="failed to get container status \"1166ef57886c453c670a285b5974d2d15a44bf3f0749c35731735eb4cc102033\": rpc error: code = NotFound desc = could not find container \"1166ef57886c453c670a285b5974d2d15a44bf3f0749c35731735eb4cc102033\": container with ID starting with 1166ef57886c453c670a285b5974d2d15a44bf3f0749c35731735eb4cc102033 not found: ID does not exist"
Apr 24 14:35:05.184639 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:35:05.184615    2574 kubelet.go:2553] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-79cbc94b89-dtgl2"]
Apr 24 14:35:05.188175 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:35:05.188153    2574 kubelet.go:2547] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/authorino-79cbc94b89-dtgl2"]
Apr 24 14:35:05.771489 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:35:05.771458    2574 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="9a8d92a9-b51b-41dd-a3f3-07290bff973b" path="/var/lib/kubelet/pods/9a8d92a9-b51b-41dd-a3f3-07290bff973b/volumes"
Apr 24 14:37:30.980599 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:37:30.980565    2574 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["kserve-ci-e2e-test/scheduler-configmap-ref-test-kserve-646894c5cf-pg8tf"]
Apr 24 14:37:30.980992 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:37:30.980940    2574 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="9a8d92a9-b51b-41dd-a3f3-07290bff973b" containerName="authorino"
Apr 24 14:37:30.980992 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:37:30.980951    2574 state_mem.go:107] "Deleted CPUSet assignment" podUID="9a8d92a9-b51b-41dd-a3f3-07290bff973b" containerName="authorino"
Apr 24 14:37:30.981063 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:37:30.981024    2574 memory_manager.go:356] "RemoveStaleState removing state" podUID="9a8d92a9-b51b-41dd-a3f3-07290bff973b" containerName="authorino"
Apr 24 14:37:30.984113 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:37:30.984093    2574 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/scheduler-configmap-ref-test-kserve-646894c5cf-pg8tf"
Apr 24 14:37:30.985978 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:37:30.985954    2574 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kserve-ci-e2e-test\"/\"openshift-service-ca.crt\""
Apr 24 14:37:30.986397 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:37:30.986378    2574 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kserve-ci-e2e-test\"/\"kube-root-ca.crt\""
Apr 24 14:37:30.986481 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:37:30.986399    2574 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"kserve-ci-e2e-test\"/\"scheduler-configmap-ref-test-kserve-self-signed-certs\""
Apr 24 14:37:30.986481 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:37:30.986469    2574 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"kserve-ci-e2e-test\"/\"default-dockercfg-92j8s\""
Apr 24 14:37:30.994243 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:37:30.994223    2574 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/scheduler-configmap-ref-test-kserve-646894c5cf-pg8tf"]
Apr 24 14:37:31.144399 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:37:31.144367    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/d1161f45-531c-4555-892f-10181d233174-tls-certs\") pod \"scheduler-configmap-ref-test-kserve-646894c5cf-pg8tf\" (UID: \"d1161f45-531c-4555-892f-10181d233174\") " pod="kserve-ci-e2e-test/scheduler-configmap-ref-test-kserve-646894c5cf-pg8tf"
Apr 24 14:37:31.144551 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:37:31.144429    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/d1161f45-531c-4555-892f-10181d233174-kserve-provision-location\") pod \"scheduler-configmap-ref-test-kserve-646894c5cf-pg8tf\" (UID: \"d1161f45-531c-4555-892f-10181d233174\") " pod="kserve-ci-e2e-test/scheduler-configmap-ref-test-kserve-646894c5cf-pg8tf"
Apr 24 14:37:31.144551 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:37:31.144455    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/d1161f45-531c-4555-892f-10181d233174-home\") pod \"scheduler-configmap-ref-test-kserve-646894c5cf-pg8tf\" (UID: \"d1161f45-531c-4555-892f-10181d233174\") " pod="kserve-ci-e2e-test/scheduler-configmap-ref-test-kserve-646894c5cf-pg8tf"
Apr 24 14:37:31.144551 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:37:31.144490    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/d1161f45-531c-4555-892f-10181d233174-model-cache\") pod \"scheduler-configmap-ref-test-kserve-646894c5cf-pg8tf\" (UID: \"d1161f45-531c-4555-892f-10181d233174\") " pod="kserve-ci-e2e-test/scheduler-configmap-ref-test-kserve-646894c5cf-pg8tf"
Apr 24 14:37:31.144665 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:37:31.144554    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/d1161f45-531c-4555-892f-10181d233174-dshm\") pod \"scheduler-configmap-ref-test-kserve-646894c5cf-pg8tf\" (UID: \"d1161f45-531c-4555-892f-10181d233174\") " pod="kserve-ci-e2e-test/scheduler-configmap-ref-test-kserve-646894c5cf-pg8tf"
Apr 24 14:37:31.144665 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:37:31.144574    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5xqhj\" (UniqueName: \"kubernetes.io/projected/d1161f45-531c-4555-892f-10181d233174-kube-api-access-5xqhj\") pod \"scheduler-configmap-ref-test-kserve-646894c5cf-pg8tf\" (UID: \"d1161f45-531c-4555-892f-10181d233174\") " pod="kserve-ci-e2e-test/scheduler-configmap-ref-test-kserve-646894c5cf-pg8tf"
Apr 24 14:37:31.245416 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:37:31.245318    2574 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/d1161f45-531c-4555-892f-10181d233174-tls-certs\") pod \"scheduler-configmap-ref-test-kserve-646894c5cf-pg8tf\" (UID: \"d1161f45-531c-4555-892f-10181d233174\") " pod="kserve-ci-e2e-test/scheduler-configmap-ref-test-kserve-646894c5cf-pg8tf"
Apr 24 14:37:31.245532 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:37:31.245424    2574 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/d1161f45-531c-4555-892f-10181d233174-kserve-provision-location\") pod \"scheduler-configmap-ref-test-kserve-646894c5cf-pg8tf\" (UID: \"d1161f45-531c-4555-892f-10181d233174\") " pod="kserve-ci-e2e-test/scheduler-configmap-ref-test-kserve-646894c5cf-pg8tf"
Apr 24 14:37:31.245532 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:37:31.245452    2574 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/d1161f45-531c-4555-892f-10181d233174-home\") pod \"scheduler-configmap-ref-test-kserve-646894c5cf-pg8tf\" (UID: \"d1161f45-531c-4555-892f-10181d233174\") " pod="kserve-ci-e2e-test/scheduler-configmap-ref-test-kserve-646894c5cf-pg8tf"
Apr 24 14:37:31.245532 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:37:31.245486    2574 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/d1161f45-531c-4555-892f-10181d233174-model-cache\") pod \"scheduler-configmap-ref-test-kserve-646894c5cf-pg8tf\" (UID: \"d1161f45-531c-4555-892f-10181d233174\") " pod="kserve-ci-e2e-test/scheduler-configmap-ref-test-kserve-646894c5cf-pg8tf"
Apr 24 14:37:31.245532 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:37:31.245516    2574 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/d1161f45-531c-4555-892f-10181d233174-dshm\") pod \"scheduler-configmap-ref-test-kserve-646894c5cf-pg8tf\" (UID: \"d1161f45-531c-4555-892f-10181d233174\") " pod="kserve-ci-e2e-test/scheduler-configmap-ref-test-kserve-646894c5cf-pg8tf"
Apr 24 14:37:31.245683 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:37:31.245541    2574 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-5xqhj\" (UniqueName: \"kubernetes.io/projected/d1161f45-531c-4555-892f-10181d233174-kube-api-access-5xqhj\") pod \"scheduler-configmap-ref-test-kserve-646894c5cf-pg8tf\" (UID: \"d1161f45-531c-4555-892f-10181d233174\") " pod="kserve-ci-e2e-test/scheduler-configmap-ref-test-kserve-646894c5cf-pg8tf"
Apr 24 14:37:31.246249 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:37:31.246213    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/d1161f45-531c-4555-892f-10181d233174-home\") pod \"scheduler-configmap-ref-test-kserve-646894c5cf-pg8tf\" (UID: \"d1161f45-531c-4555-892f-10181d233174\") " pod="kserve-ci-e2e-test/scheduler-configmap-ref-test-kserve-646894c5cf-pg8tf"
Apr 24 14:37:31.246466 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:37:31.246249    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/d1161f45-531c-4555-892f-10181d233174-kserve-provision-location\") pod \"scheduler-configmap-ref-test-kserve-646894c5cf-pg8tf\" (UID: \"d1161f45-531c-4555-892f-10181d233174\") " pod="kserve-ci-e2e-test/scheduler-configmap-ref-test-kserve-646894c5cf-pg8tf"
Apr 24 14:37:31.246604 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:37:31.246356    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/d1161f45-531c-4555-892f-10181d233174-model-cache\") pod \"scheduler-configmap-ref-test-kserve-646894c5cf-pg8tf\" (UID: \"d1161f45-531c-4555-892f-10181d233174\") " pod="kserve-ci-e2e-test/scheduler-configmap-ref-test-kserve-646894c5cf-pg8tf"
Apr 24 14:37:31.248670 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:37:31.248642    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/d1161f45-531c-4555-892f-10181d233174-dshm\") pod \"scheduler-configmap-ref-test-kserve-646894c5cf-pg8tf\" (UID: \"d1161f45-531c-4555-892f-10181d233174\") " pod="kserve-ci-e2e-test/scheduler-configmap-ref-test-kserve-646894c5cf-pg8tf"
Apr 24 14:37:31.249031 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:37:31.249005    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/d1161f45-531c-4555-892f-10181d233174-tls-certs\") pod \"scheduler-configmap-ref-test-kserve-646894c5cf-pg8tf\" (UID: \"d1161f45-531c-4555-892f-10181d233174\") " pod="kserve-ci-e2e-test/scheduler-configmap-ref-test-kserve-646894c5cf-pg8tf"
Apr 24 14:37:31.254461 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:37:31.254440    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-5xqhj\" (UniqueName: \"kubernetes.io/projected/d1161f45-531c-4555-892f-10181d233174-kube-api-access-5xqhj\") pod \"scheduler-configmap-ref-test-kserve-646894c5cf-pg8tf\" (UID: \"d1161f45-531c-4555-892f-10181d233174\") " pod="kserve-ci-e2e-test/scheduler-configmap-ref-test-kserve-646894c5cf-pg8tf"
Apr 24 14:37:31.295675 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:37:31.295648    2574 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/scheduler-configmap-ref-test-kserve-646894c5cf-pg8tf"
Apr 24 14:37:31.417009 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:37:31.416983    2574 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/scheduler-configmap-ref-test-kserve-646894c5cf-pg8tf"]
Apr 24 14:37:31.418977 ip-10-0-136-132 kubenswrapper[2574]: W0424 14:37:31.418950    2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd1161f45_531c_4555_892f_10181d233174.slice/crio-34ebad0d9d3a472a0e6c1bb7f247fbb8b47b5433944f082971044aa03a8e61c9 WatchSource:0}: Error finding container 34ebad0d9d3a472a0e6c1bb7f247fbb8b47b5433944f082971044aa03a8e61c9: Status 404 returned error can't find the container with id 34ebad0d9d3a472a0e6c1bb7f247fbb8b47b5433944f082971044aa03a8e61c9
Apr 24 14:37:31.717082 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:37:31.717049    2574 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/scheduler-configmap-ref-test-kserve-646894c5cf-pg8tf" event={"ID":"d1161f45-531c-4555-892f-10181d233174","Type":"ContainerStarted","Data":"34ebad0d9d3a472a0e6c1bb7f247fbb8b47b5433944f082971044aa03a8e61c9"}
Apr 24 14:37:35.736579 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:37:35.736536    2574 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/scheduler-configmap-ref-test-kserve-646894c5cf-pg8tf" event={"ID":"d1161f45-531c-4555-892f-10181d233174","Type":"ContainerStarted","Data":"bc11826c1d048dd74b7f93d985ce9a9c9e68d906e48d653c7646dbd16fee3457"}
Apr 24 14:38:57.762321 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:38:57.762232    2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-9d4b6777b-f6ngr_2571829f-666a-422a-9b02-2d2bb4c89b25/console-operator/1.log"
Apr 24 14:38:57.762933 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:38:57.762903    2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-9d4b6777b-f6ngr_2571829f-666a-422a-9b02-2d2bb4c89b25/console-operator/1.log"
Apr 24 14:38:57.767863 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:38:57.767842    2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-w5x5x_44036c3e-1fdc-40d4-899c-9009cd4589e3/ovn-acl-logging/0.log"
Apr 24 14:38:57.768523 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:38:57.768500    2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-w5x5x_44036c3e-1fdc-40d4-899c-9009cd4589e3/ovn-acl-logging/0.log"
Apr 24 14:39:11.086194 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:39:11.086163    2574 generic.go:358] "Generic (PLEG): container finished" podID="d1161f45-531c-4555-892f-10181d233174" containerID="bc11826c1d048dd74b7f93d985ce9a9c9e68d906e48d653c7646dbd16fee3457" exitCode=0
Apr 24 14:39:11.086738 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:39:11.086203    2574 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/scheduler-configmap-ref-test-kserve-646894c5cf-pg8tf" event={"ID":"d1161f45-531c-4555-892f-10181d233174","Type":"ContainerDied","Data":"bc11826c1d048dd74b7f93d985ce9a9c9e68d906e48d653c7646dbd16fee3457"}
Apr 24 14:39:13.096031 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:39:13.095998    2574 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/scheduler-configmap-ref-test-kserve-646894c5cf-pg8tf" event={"ID":"d1161f45-531c-4555-892f-10181d233174","Type":"ContainerStarted","Data":"a8c623c3267c61ef7ff872b4959a48d260864ebda62f5654e6f7bef4f86c0cc8"}
Apr 24 14:39:13.113187 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:39:13.113132    2574 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="kserve-ci-e2e-test/scheduler-configmap-ref-test-kserve-646894c5cf-pg8tf" podStartSLOduration=2.177413488 podStartE2EDuration="1m43.113116454s" podCreationTimestamp="2026-04-24 14:37:30 +0000 UTC" firstStartedPulling="2026-04-24 14:37:31.420755864 +0000 UTC m=+814.152857491" lastFinishedPulling="2026-04-24 14:39:12.356458826 +0000 UTC m=+915.088560457" observedRunningTime="2026-04-24 14:39:13.112307869 +0000 UTC m=+915.844409520" watchObservedRunningTime="2026-04-24 14:39:13.113116454 +0000 UTC m=+915.845218104"
Apr 24 14:39:21.295767 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:39:21.295726    2574 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve-ci-e2e-test/scheduler-configmap-ref-test-kserve-646894c5cf-pg8tf"
Apr 24 14:39:21.295767 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:39:21.295766    2574 kubelet.go:2658] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="kserve-ci-e2e-test/scheduler-configmap-ref-test-kserve-646894c5cf-pg8tf"
Apr 24 14:39:21.321552 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:39:21.308359    2574 kubelet.go:2658] "SyncLoop (probe)" probe="startup" status="started" pod="kserve-ci-e2e-test/scheduler-configmap-ref-test-kserve-646894c5cf-pg8tf"
Apr 24 14:39:22.146511 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:39:22.146486    2574 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="kserve-ci-e2e-test/scheduler-configmap-ref-test-kserve-646894c5cf-pg8tf"
Apr 24 14:39:22.989154 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:39:22.989123    2574 kubelet.go:2553] "SyncLoop DELETE" source="api" pods=["kserve-ci-e2e-test/scheduler-configmap-ref-test-kserve-646894c5cf-pg8tf"]
Apr 24 14:39:23.268231 ip-10-0-136-132 kubenswrapper[2574]: E0424 14:39:23.268148    2574 secret.go:189] Couldn't get secret kserve-ci-e2e-test/scheduler-configmap-ref-test-kserve-self-signed-certs: secret "scheduler-configmap-ref-test-kserve-self-signed-certs" not found
Apr 24 14:39:23.268397 ip-10-0-136-132 kubenswrapper[2574]: E0424 14:39:23.268237    2574 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d1161f45-531c-4555-892f-10181d233174-tls-certs podName:d1161f45-531c-4555-892f-10181d233174 nodeName:}" failed. No retries permitted until 2026-04-24 14:39:23.768219667 +0000 UTC m=+926.500321293 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "tls-certs" (UniqueName: "kubernetes.io/secret/d1161f45-531c-4555-892f-10181d233174-tls-certs") pod "scheduler-configmap-ref-test-kserve-646894c5cf-pg8tf" (UID: "d1161f45-531c-4555-892f-10181d233174") : secret "scheduler-configmap-ref-test-kserve-self-signed-certs" not found
Apr 24 14:39:23.772057 ip-10-0-136-132 kubenswrapper[2574]: E0424 14:39:23.772024    2574 secret.go:189] Couldn't get secret kserve-ci-e2e-test/scheduler-configmap-ref-test-kserve-self-signed-certs: secret "scheduler-configmap-ref-test-kserve-self-signed-certs" not found
Apr 24 14:39:23.772234 ip-10-0-136-132 kubenswrapper[2574]: E0424 14:39:23.772098    2574 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d1161f45-531c-4555-892f-10181d233174-tls-certs podName:d1161f45-531c-4555-892f-10181d233174 nodeName:}" failed. No retries permitted until 2026-04-24 14:39:24.772080299 +0000 UTC m=+927.504181927 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "tls-certs" (UniqueName: "kubernetes.io/secret/d1161f45-531c-4555-892f-10181d233174-tls-certs") pod "scheduler-configmap-ref-test-kserve-646894c5cf-pg8tf" (UID: "d1161f45-531c-4555-892f-10181d233174") : secret "scheduler-configmap-ref-test-kserve-self-signed-certs" not found
Apr 24 14:39:24.142704 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:39:24.142645    2574 kuberuntime_container.go:864] "Killing container with a grace period" pod="kserve-ci-e2e-test/scheduler-configmap-ref-test-kserve-646894c5cf-pg8tf" podUID="d1161f45-531c-4555-892f-10181d233174" containerName="main" containerID="cri-o://a8c623c3267c61ef7ff872b4959a48d260864ebda62f5654e6f7bef4f86c0cc8" gracePeriod=30
Apr 24 14:39:24.390266 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:39:24.390244    2574 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/scheduler-configmap-ref-test-kserve-646894c5cf-pg8tf"
Apr 24 14:39:24.478671 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:39:24.478596    2574 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"kube-api-access-5xqhj\" (UniqueName: \"kubernetes.io/projected/d1161f45-531c-4555-892f-10181d233174-kube-api-access-5xqhj\") pod \"d1161f45-531c-4555-892f-10181d233174\" (UID: \"d1161f45-531c-4555-892f-10181d233174\") "
Apr 24 14:39:24.478671 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:39:24.478643    2574 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/d1161f45-531c-4555-892f-10181d233174-home\") pod \"d1161f45-531c-4555-892f-10181d233174\" (UID: \"d1161f45-531c-4555-892f-10181d233174\") "
Apr 24 14:39:24.478837 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:39:24.478793    2574 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/d1161f45-531c-4555-892f-10181d233174-dshm\") pod \"d1161f45-531c-4555-892f-10181d233174\" (UID: \"d1161f45-531c-4555-892f-10181d233174\") "
Apr 24 14:39:24.478882 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:39:24.478847    2574 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d1161f45-531c-4555-892f-10181d233174-home" (OuterVolumeSpecName: "home") pod "d1161f45-531c-4555-892f-10181d233174" (UID: "d1161f45-531c-4555-892f-10181d233174"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Apr 24 14:39:24.478882 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:39:24.478868    2574 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/d1161f45-531c-4555-892f-10181d233174-kserve-provision-location\") pod \"d1161f45-531c-4555-892f-10181d233174\" (UID: \"d1161f45-531c-4555-892f-10181d233174\") "
Apr 24 14:39:24.478960 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:39:24.478900    2574 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/d1161f45-531c-4555-892f-10181d233174-tls-certs\") pod \"d1161f45-531c-4555-892f-10181d233174\" (UID: \"d1161f45-531c-4555-892f-10181d233174\") "
Apr 24 14:39:24.478960 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:39:24.478953    2574 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/d1161f45-531c-4555-892f-10181d233174-model-cache\") pod \"d1161f45-531c-4555-892f-10181d233174\" (UID: \"d1161f45-531c-4555-892f-10181d233174\") "
Apr 24 14:39:24.479318 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:39:24.479241    2574 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/d1161f45-531c-4555-892f-10181d233174-home\") on node \"ip-10-0-136-132.ec2.internal\" DevicePath \"\""
Apr 24 14:39:24.479318 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:39:24.479281    2574 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d1161f45-531c-4555-892f-10181d233174-model-cache" (OuterVolumeSpecName: "model-cache") pod "d1161f45-531c-4555-892f-10181d233174" (UID: "d1161f45-531c-4555-892f-10181d233174"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Apr 24 14:39:24.480813 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:39:24.480791    2574 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d1161f45-531c-4555-892f-10181d233174-kube-api-access-5xqhj" (OuterVolumeSpecName: "kube-api-access-5xqhj") pod "d1161f45-531c-4555-892f-10181d233174" (UID: "d1161f45-531c-4555-892f-10181d233174"). InnerVolumeSpecName "kube-api-access-5xqhj". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Apr 24 14:39:24.480918 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:39:24.480807    2574 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d1161f45-531c-4555-892f-10181d233174-dshm" (OuterVolumeSpecName: "dshm") pod "d1161f45-531c-4555-892f-10181d233174" (UID: "d1161f45-531c-4555-892f-10181d233174"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Apr 24 14:39:24.481281 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:39:24.481265    2574 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/d1161f45-531c-4555-892f-10181d233174-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "d1161f45-531c-4555-892f-10181d233174" (UID: "d1161f45-531c-4555-892f-10181d233174"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Apr 24 14:39:24.532403 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:39:24.532376    2574 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d1161f45-531c-4555-892f-10181d233174-kserve-provision-location" (OuterVolumeSpecName: "kserve-provision-location") pod "d1161f45-531c-4555-892f-10181d233174" (UID: "d1161f45-531c-4555-892f-10181d233174"). InnerVolumeSpecName "kserve-provision-location". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Apr 24 14:39:24.580260 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:39:24.580233    2574 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/d1161f45-531c-4555-892f-10181d233174-model-cache\") on node \"ip-10-0-136-132.ec2.internal\" DevicePath \"\""
Apr 24 14:39:24.580260 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:39:24.580259    2574 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-5xqhj\" (UniqueName: \"kubernetes.io/projected/d1161f45-531c-4555-892f-10181d233174-kube-api-access-5xqhj\") on node \"ip-10-0-136-132.ec2.internal\" DevicePath \"\""
Apr 24 14:39:24.580437 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:39:24.580272    2574 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/d1161f45-531c-4555-892f-10181d233174-dshm\") on node \"ip-10-0-136-132.ec2.internal\" DevicePath \"\""
Apr 24 14:39:24.580437 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:39:24.580285    2574 reconciler_common.go:299] "Volume detached for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/d1161f45-531c-4555-892f-10181d233174-kserve-provision-location\") on node \"ip-10-0-136-132.ec2.internal\" DevicePath \"\""
Apr 24 14:39:24.580437 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:39:24.580300    2574 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/d1161f45-531c-4555-892f-10181d233174-tls-certs\") on node \"ip-10-0-136-132.ec2.internal\" DevicePath \"\""
Apr 24 14:39:25.147127 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:39:25.147092    2574 generic.go:358] "Generic (PLEG): container finished" podID="d1161f45-531c-4555-892f-10181d233174" containerID="a8c623c3267c61ef7ff872b4959a48d260864ebda62f5654e6f7bef4f86c0cc8" exitCode=0
Apr 24 14:39:25.147617 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:39:25.147166    2574 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/scheduler-configmap-ref-test-kserve-646894c5cf-pg8tf" event={"ID":"d1161f45-531c-4555-892f-10181d233174","Type":"ContainerDied","Data":"a8c623c3267c61ef7ff872b4959a48d260864ebda62f5654e6f7bef4f86c0cc8"}
Apr 24 14:39:25.147617 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:39:25.147192    2574 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/scheduler-configmap-ref-test-kserve-646894c5cf-pg8tf"
Apr 24 14:39:25.147617 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:39:25.147205    2574 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/scheduler-configmap-ref-test-kserve-646894c5cf-pg8tf" event={"ID":"d1161f45-531c-4555-892f-10181d233174","Type":"ContainerDied","Data":"34ebad0d9d3a472a0e6c1bb7f247fbb8b47b5433944f082971044aa03a8e61c9"}
Apr 24 14:39:25.147617 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:39:25.147221    2574 scope.go:117] "RemoveContainer" containerID="a8c623c3267c61ef7ff872b4959a48d260864ebda62f5654e6f7bef4f86c0cc8"
Apr 24 14:39:25.156522 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:39:25.156496    2574 scope.go:117] "RemoveContainer" containerID="bc11826c1d048dd74b7f93d985ce9a9c9e68d906e48d653c7646dbd16fee3457"
Apr 24 14:39:25.166420 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:39:25.166400    2574 scope.go:117] "RemoveContainer" containerID="a8c623c3267c61ef7ff872b4959a48d260864ebda62f5654e6f7bef4f86c0cc8"
Apr 24 14:39:25.166663 ip-10-0-136-132 kubenswrapper[2574]: E0424 14:39:25.166642    2574 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"a8c623c3267c61ef7ff872b4959a48d260864ebda62f5654e6f7bef4f86c0cc8\": container with ID starting with a8c623c3267c61ef7ff872b4959a48d260864ebda62f5654e6f7bef4f86c0cc8 not found: ID does not exist" containerID="a8c623c3267c61ef7ff872b4959a48d260864ebda62f5654e6f7bef4f86c0cc8"
Apr 24 14:39:25.166740 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:39:25.166670    2574 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"a8c623c3267c61ef7ff872b4959a48d260864ebda62f5654e6f7bef4f86c0cc8"} err="failed to get container status \"a8c623c3267c61ef7ff872b4959a48d260864ebda62f5654e6f7bef4f86c0cc8\": rpc error: code = NotFound desc = could not find container \"a8c623c3267c61ef7ff872b4959a48d260864ebda62f5654e6f7bef4f86c0cc8\": container with ID starting with a8c623c3267c61ef7ff872b4959a48d260864ebda62f5654e6f7bef4f86c0cc8 not found: ID does not exist"
Apr 24 14:39:25.166740 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:39:25.166688    2574 scope.go:117] "RemoveContainer" containerID="bc11826c1d048dd74b7f93d985ce9a9c9e68d906e48d653c7646dbd16fee3457"
Apr 24 14:39:25.166967 ip-10-0-136-132 kubenswrapper[2574]: E0424 14:39:25.166948    2574 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"bc11826c1d048dd74b7f93d985ce9a9c9e68d906e48d653c7646dbd16fee3457\": container with ID starting with bc11826c1d048dd74b7f93d985ce9a9c9e68d906e48d653c7646dbd16fee3457 not found: ID does not exist" containerID="bc11826c1d048dd74b7f93d985ce9a9c9e68d906e48d653c7646dbd16fee3457"
Apr 24 14:39:25.167026 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:39:25.166973    2574 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"bc11826c1d048dd74b7f93d985ce9a9c9e68d906e48d653c7646dbd16fee3457"} err="failed to get container status \"bc11826c1d048dd74b7f93d985ce9a9c9e68d906e48d653c7646dbd16fee3457\": rpc error: code = NotFound desc = could not find container \"bc11826c1d048dd74b7f93d985ce9a9c9e68d906e48d653c7646dbd16fee3457\": container with ID starting with bc11826c1d048dd74b7f93d985ce9a9c9e68d906e48d653c7646dbd16fee3457 not found: ID does not exist"
Apr 24 14:39:25.169277 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:39:25.169254    2574 kubelet.go:2553] "SyncLoop DELETE" source="api" pods=["kserve-ci-e2e-test/scheduler-configmap-ref-test-kserve-646894c5cf-pg8tf"]
Apr 24 14:39:25.171478 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:39:25.171454    2574 kubelet.go:2547] "SyncLoop REMOVE" source="api" pods=["kserve-ci-e2e-test/scheduler-configmap-ref-test-kserve-646894c5cf-pg8tf"]
Apr 24 14:39:25.770442 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:39:25.770404    2574 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="d1161f45-531c-4555-892f-10181d233174" path="/var/lib/kubelet/pods/d1161f45-531c-4555-892f-10181d233174/volumes"
Apr 24 14:39:28.598454 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:39:28.598423    2574 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["kserve-ci-e2e-test/scheduler-ha-replicas-test-kserve-9d798b947-brwzq"]
Apr 24 14:39:28.598815 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:39:28.598802    2574 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d1161f45-531c-4555-892f-10181d233174" containerName="main"
Apr 24 14:39:28.598859 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:39:28.598817    2574 state_mem.go:107] "Deleted CPUSet assignment" podUID="d1161f45-531c-4555-892f-10181d233174" containerName="main"
Apr 24 14:39:28.598859 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:39:28.598834    2574 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d1161f45-531c-4555-892f-10181d233174" containerName="storage-initializer"
Apr 24 14:39:28.598859 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:39:28.598839    2574 state_mem.go:107] "Deleted CPUSet assignment" podUID="d1161f45-531c-4555-892f-10181d233174" containerName="storage-initializer"
Apr 24 14:39:28.598948 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:39:28.598892    2574 memory_manager.go:356] "RemoveStaleState removing state" podUID="d1161f45-531c-4555-892f-10181d233174" containerName="main"
Apr 24 14:39:28.604238 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:39:28.604217    2574 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/scheduler-ha-replicas-test-kserve-9d798b947-brwzq"
Apr 24 14:39:28.606464 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:39:28.606432    2574 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kserve-ci-e2e-test\"/\"kube-root-ca.crt\""
Apr 24 14:39:28.607039 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:39:28.606767    2574 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"kserve-ci-e2e-test\"/\"scheduler-ha-replicas-test-kserve-self-signed-certs\""
Apr 24 14:39:28.607039 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:39:28.606794    2574 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kserve-ci-e2e-test\"/\"openshift-service-ca.crt\""
Apr 24 14:39:28.607039 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:39:28.606775    2574 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"kserve-ci-e2e-test\"/\"default-dockercfg-92j8s\""
Apr 24 14:39:28.608898 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:39:28.608877    2574 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/scheduler-ha-replicas-test-kserve-9d798b947-brwzq"]
Apr 24 14:39:28.716496 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:39:28.716468    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/09a3eabb-dbcc-44c9-9a75-a348e3806278-dshm\") pod \"scheduler-ha-replicas-test-kserve-9d798b947-brwzq\" (UID: \"09a3eabb-dbcc-44c9-9a75-a348e3806278\") " pod="kserve-ci-e2e-test/scheduler-ha-replicas-test-kserve-9d798b947-brwzq"
Apr 24 14:39:28.716674 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:39:28.716517    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/09a3eabb-dbcc-44c9-9a75-a348e3806278-home\") pod \"scheduler-ha-replicas-test-kserve-9d798b947-brwzq\" (UID: \"09a3eabb-dbcc-44c9-9a75-a348e3806278\") " pod="kserve-ci-e2e-test/scheduler-ha-replicas-test-kserve-9d798b947-brwzq"
Apr 24 14:39:28.716674 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:39:28.716575    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/09a3eabb-dbcc-44c9-9a75-a348e3806278-model-cache\") pod \"scheduler-ha-replicas-test-kserve-9d798b947-brwzq\" (UID: \"09a3eabb-dbcc-44c9-9a75-a348e3806278\") " pod="kserve-ci-e2e-test/scheduler-ha-replicas-test-kserve-9d798b947-brwzq"
Apr 24 14:39:28.716674 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:39:28.716653    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/09a3eabb-dbcc-44c9-9a75-a348e3806278-kserve-provision-location\") pod \"scheduler-ha-replicas-test-kserve-9d798b947-brwzq\" (UID: \"09a3eabb-dbcc-44c9-9a75-a348e3806278\") " pod="kserve-ci-e2e-test/scheduler-ha-replicas-test-kserve-9d798b947-brwzq"
Apr 24 14:39:28.716843 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:39:28.716708    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-kq48z\" (UniqueName: \"kubernetes.io/projected/09a3eabb-dbcc-44c9-9a75-a348e3806278-kube-api-access-kq48z\") pod \"scheduler-ha-replicas-test-kserve-9d798b947-brwzq\" (UID: \"09a3eabb-dbcc-44c9-9a75-a348e3806278\") " pod="kserve-ci-e2e-test/scheduler-ha-replicas-test-kserve-9d798b947-brwzq"
Apr 24 14:39:28.716843 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:39:28.716760    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/09a3eabb-dbcc-44c9-9a75-a348e3806278-tls-certs\") pod \"scheduler-ha-replicas-test-kserve-9d798b947-brwzq\" (UID: \"09a3eabb-dbcc-44c9-9a75-a348e3806278\") " pod="kserve-ci-e2e-test/scheduler-ha-replicas-test-kserve-9d798b947-brwzq"
Apr 24 14:39:28.817223 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:39:28.817192    2574 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/09a3eabb-dbcc-44c9-9a75-a348e3806278-dshm\") pod \"scheduler-ha-replicas-test-kserve-9d798b947-brwzq\" (UID: \"09a3eabb-dbcc-44c9-9a75-a348e3806278\") " pod="kserve-ci-e2e-test/scheduler-ha-replicas-test-kserve-9d798b947-brwzq"
Apr 24 14:39:28.817369 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:39:28.817257    2574 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/09a3eabb-dbcc-44c9-9a75-a348e3806278-home\") pod \"scheduler-ha-replicas-test-kserve-9d798b947-brwzq\" (UID: \"09a3eabb-dbcc-44c9-9a75-a348e3806278\") " pod="kserve-ci-e2e-test/scheduler-ha-replicas-test-kserve-9d798b947-brwzq"
Apr 24 14:39:28.817369 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:39:28.817290    2574 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/09a3eabb-dbcc-44c9-9a75-a348e3806278-model-cache\") pod \"scheduler-ha-replicas-test-kserve-9d798b947-brwzq\" (UID: \"09a3eabb-dbcc-44c9-9a75-a348e3806278\") " pod="kserve-ci-e2e-test/scheduler-ha-replicas-test-kserve-9d798b947-brwzq"
Apr 24 14:39:28.817369 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:39:28.817355    2574 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/09a3eabb-dbcc-44c9-9a75-a348e3806278-kserve-provision-location\") pod \"scheduler-ha-replicas-test-kserve-9d798b947-brwzq\" (UID: \"09a3eabb-dbcc-44c9-9a75-a348e3806278\") " pod="kserve-ci-e2e-test/scheduler-ha-replicas-test-kserve-9d798b947-brwzq"
Apr 24 14:39:28.817526 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:39:28.817396    2574 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-kq48z\" (UniqueName: \"kubernetes.io/projected/09a3eabb-dbcc-44c9-9a75-a348e3806278-kube-api-access-kq48z\") pod \"scheduler-ha-replicas-test-kserve-9d798b947-brwzq\" (UID: \"09a3eabb-dbcc-44c9-9a75-a348e3806278\") " pod="kserve-ci-e2e-test/scheduler-ha-replicas-test-kserve-9d798b947-brwzq"
Apr 24 14:39:28.817526 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:39:28.817433    2574 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/09a3eabb-dbcc-44c9-9a75-a348e3806278-tls-certs\") pod \"scheduler-ha-replicas-test-kserve-9d798b947-brwzq\" (UID: \"09a3eabb-dbcc-44c9-9a75-a348e3806278\") " pod="kserve-ci-e2e-test/scheduler-ha-replicas-test-kserve-9d798b947-brwzq"
Apr 24 14:39:28.817779 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:39:28.817751    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/09a3eabb-dbcc-44c9-9a75-a348e3806278-home\") pod \"scheduler-ha-replicas-test-kserve-9d798b947-brwzq\" (UID: \"09a3eabb-dbcc-44c9-9a75-a348e3806278\") " pod="kserve-ci-e2e-test/scheduler-ha-replicas-test-kserve-9d798b947-brwzq"
Apr 24 14:39:28.817876 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:39:28.817804    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/09a3eabb-dbcc-44c9-9a75-a348e3806278-model-cache\") pod \"scheduler-ha-replicas-test-kserve-9d798b947-brwzq\" (UID: \"09a3eabb-dbcc-44c9-9a75-a348e3806278\") " pod="kserve-ci-e2e-test/scheduler-ha-replicas-test-kserve-9d798b947-brwzq"
Apr 24 14:39:28.817993 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:39:28.817975    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/09a3eabb-dbcc-44c9-9a75-a348e3806278-kserve-provision-location\") pod \"scheduler-ha-replicas-test-kserve-9d798b947-brwzq\" (UID: \"09a3eabb-dbcc-44c9-9a75-a348e3806278\") " pod="kserve-ci-e2e-test/scheduler-ha-replicas-test-kserve-9d798b947-brwzq"
Apr 24 14:39:28.819854 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:39:28.819832    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/09a3eabb-dbcc-44c9-9a75-a348e3806278-dshm\") pod \"scheduler-ha-replicas-test-kserve-9d798b947-brwzq\" (UID: \"09a3eabb-dbcc-44c9-9a75-a348e3806278\") " pod="kserve-ci-e2e-test/scheduler-ha-replicas-test-kserve-9d798b947-brwzq"
Apr 24 14:39:28.820294 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:39:28.820270    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/09a3eabb-dbcc-44c9-9a75-a348e3806278-tls-certs\") pod \"scheduler-ha-replicas-test-kserve-9d798b947-brwzq\" (UID: \"09a3eabb-dbcc-44c9-9a75-a348e3806278\") " pod="kserve-ci-e2e-test/scheduler-ha-replicas-test-kserve-9d798b947-brwzq"
Apr 24 14:39:28.828375 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:39:28.828352    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-kq48z\" (UniqueName: \"kubernetes.io/projected/09a3eabb-dbcc-44c9-9a75-a348e3806278-kube-api-access-kq48z\") pod \"scheduler-ha-replicas-test-kserve-9d798b947-brwzq\" (UID: \"09a3eabb-dbcc-44c9-9a75-a348e3806278\") " pod="kserve-ci-e2e-test/scheduler-ha-replicas-test-kserve-9d798b947-brwzq"
Apr 24 14:39:28.850634 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:39:28.850583    2574 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["kserve-ci-e2e-test/scheduler-ha-replicas-test-kserve-router-scheduler-fc6d477mptc7"]
Apr 24 14:39:28.854611 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:39:28.854598    2574 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/scheduler-ha-replicas-test-kserve-router-scheduler-fc6d477mptc7"
Apr 24 14:39:28.856455 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:39:28.856437    2574 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"kserve-ci-e2e-test\"/\"scheduler-ha-replicas-test-epp-sa-dockercfg-x4cvm\""
Apr 24 14:39:28.863635 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:39:28.863612    2574 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/scheduler-ha-replicas-test-kserve-router-scheduler-fc6d477mptc7"]
Apr 24 14:39:28.914917 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:39:28.914891    2574 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/scheduler-ha-replicas-test-kserve-9d798b947-brwzq"
Apr 24 14:39:29.018632 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:39:29.018599    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/0a4aeb8e-a962-4aeb-9918-c75cdb61f704-kserve-provision-location\") pod \"scheduler-ha-replicas-test-kserve-router-scheduler-fc6d477mptc7\" (UID: \"0a4aeb8e-a962-4aeb-9918-c75cdb61f704\") " pod="kserve-ci-e2e-test/scheduler-ha-replicas-test-kserve-router-scheduler-fc6d477mptc7"
Apr 24 14:39:29.018792 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:39:29.018643    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tokenizer-tmp\" (UniqueName: \"kubernetes.io/empty-dir/0a4aeb8e-a962-4aeb-9918-c75cdb61f704-tokenizer-tmp\") pod \"scheduler-ha-replicas-test-kserve-router-scheduler-fc6d477mptc7\" (UID: \"0a4aeb8e-a962-4aeb-9918-c75cdb61f704\") " pod="kserve-ci-e2e-test/scheduler-ha-replicas-test-kserve-router-scheduler-fc6d477mptc7"
Apr 24 14:39:29.018792 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:39:29.018703    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8v2qc\" (UniqueName: \"kubernetes.io/projected/0a4aeb8e-a962-4aeb-9918-c75cdb61f704-kube-api-access-8v2qc\") pod \"scheduler-ha-replicas-test-kserve-router-scheduler-fc6d477mptc7\" (UID: \"0a4aeb8e-a962-4aeb-9918-c75cdb61f704\") " pod="kserve-ci-e2e-test/scheduler-ha-replicas-test-kserve-router-scheduler-fc6d477mptc7"
Apr 24 14:39:29.018792 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:39:29.018781    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/0a4aeb8e-a962-4aeb-9918-c75cdb61f704-tls-certs\") pod \"scheduler-ha-replicas-test-kserve-router-scheduler-fc6d477mptc7\" (UID: \"0a4aeb8e-a962-4aeb-9918-c75cdb61f704\") " pod="kserve-ci-e2e-test/scheduler-ha-replicas-test-kserve-router-scheduler-fc6d477mptc7"
Apr 24 14:39:29.018974 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:39:29.018829    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tokenizer-uds\" (UniqueName: \"kubernetes.io/empty-dir/0a4aeb8e-a962-4aeb-9918-c75cdb61f704-tokenizer-uds\") pod \"scheduler-ha-replicas-test-kserve-router-scheduler-fc6d477mptc7\" (UID: \"0a4aeb8e-a962-4aeb-9918-c75cdb61f704\") " pod="kserve-ci-e2e-test/scheduler-ha-replicas-test-kserve-router-scheduler-fc6d477mptc7"
Apr 24 14:39:29.018974 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:39:29.018848    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tokenizer-cache\" (UniqueName: \"kubernetes.io/empty-dir/0a4aeb8e-a962-4aeb-9918-c75cdb61f704-tokenizer-cache\") pod \"scheduler-ha-replicas-test-kserve-router-scheduler-fc6d477mptc7\" (UID: \"0a4aeb8e-a962-4aeb-9918-c75cdb61f704\") " pod="kserve-ci-e2e-test/scheduler-ha-replicas-test-kserve-router-scheduler-fc6d477mptc7"
Apr 24 14:39:29.039095 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:39:29.039069    2574 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/scheduler-ha-replicas-test-kserve-9d798b947-brwzq"]
Apr 24 14:39:29.040554 ip-10-0-136-132 kubenswrapper[2574]: W0424 14:39:29.040527    2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod09a3eabb_dbcc_44c9_9a75_a348e3806278.slice/crio-81c02675c385f0441c3c116071f73a87b9e6649ba399e33871f2a4fe41ae1813 WatchSource:0}: Error finding container 81c02675c385f0441c3c116071f73a87b9e6649ba399e33871f2a4fe41ae1813: Status 404 returned error can't find the container with id 81c02675c385f0441c3c116071f73a87b9e6649ba399e33871f2a4fe41ae1813
Apr 24 14:39:29.119236 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:39:29.119208    2574 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"tokenizer-uds\" (UniqueName: \"kubernetes.io/empty-dir/0a4aeb8e-a962-4aeb-9918-c75cdb61f704-tokenizer-uds\") pod \"scheduler-ha-replicas-test-kserve-router-scheduler-fc6d477mptc7\" (UID: \"0a4aeb8e-a962-4aeb-9918-c75cdb61f704\") " pod="kserve-ci-e2e-test/scheduler-ha-replicas-test-kserve-router-scheduler-fc6d477mptc7"
Apr 24 14:39:29.119431 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:39:29.119251    2574 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"tokenizer-cache\" (UniqueName: \"kubernetes.io/empty-dir/0a4aeb8e-a962-4aeb-9918-c75cdb61f704-tokenizer-cache\") pod \"scheduler-ha-replicas-test-kserve-router-scheduler-fc6d477mptc7\" (UID: \"0a4aeb8e-a962-4aeb-9918-c75cdb61f704\") " pod="kserve-ci-e2e-test/scheduler-ha-replicas-test-kserve-router-scheduler-fc6d477mptc7"
Apr 24 14:39:29.119431 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:39:29.119319    2574 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/0a4aeb8e-a962-4aeb-9918-c75cdb61f704-kserve-provision-location\") pod \"scheduler-ha-replicas-test-kserve-router-scheduler-fc6d477mptc7\" (UID: \"0a4aeb8e-a962-4aeb-9918-c75cdb61f704\") " pod="kserve-ci-e2e-test/scheduler-ha-replicas-test-kserve-router-scheduler-fc6d477mptc7"
Apr 24 14:39:29.119561 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:39:29.119452    2574 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"tokenizer-tmp\" (UniqueName: \"kubernetes.io/empty-dir/0a4aeb8e-a962-4aeb-9918-c75cdb61f704-tokenizer-tmp\") pod \"scheduler-ha-replicas-test-kserve-router-scheduler-fc6d477mptc7\" (UID: \"0a4aeb8e-a962-4aeb-9918-c75cdb61f704\") " pod="kserve-ci-e2e-test/scheduler-ha-replicas-test-kserve-router-scheduler-fc6d477mptc7"
Apr 24 14:39:29.119561 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:39:29.119517    2574 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-8v2qc\" (UniqueName: \"kubernetes.io/projected/0a4aeb8e-a962-4aeb-9918-c75cdb61f704-kube-api-access-8v2qc\") pod \"scheduler-ha-replicas-test-kserve-router-scheduler-fc6d477mptc7\" (UID: \"0a4aeb8e-a962-4aeb-9918-c75cdb61f704\") " pod="kserve-ci-e2e-test/scheduler-ha-replicas-test-kserve-router-scheduler-fc6d477mptc7"
Apr 24 14:39:29.119664 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:39:29.119591    2574 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/0a4aeb8e-a962-4aeb-9918-c75cdb61f704-tls-certs\") pod \"scheduler-ha-replicas-test-kserve-router-scheduler-fc6d477mptc7\" (UID: \"0a4aeb8e-a962-4aeb-9918-c75cdb61f704\") " pod="kserve-ci-e2e-test/scheduler-ha-replicas-test-kserve-router-scheduler-fc6d477mptc7"
Apr 24 14:39:29.119664 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:39:29.119609    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tokenizer-uds\" (UniqueName: \"kubernetes.io/empty-dir/0a4aeb8e-a962-4aeb-9918-c75cdb61f704-tokenizer-uds\") pod \"scheduler-ha-replicas-test-kserve-router-scheduler-fc6d477mptc7\" (UID: \"0a4aeb8e-a962-4aeb-9918-c75cdb61f704\") " pod="kserve-ci-e2e-test/scheduler-ha-replicas-test-kserve-router-scheduler-fc6d477mptc7"
Apr 24 14:39:29.119664 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:39:29.119628    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tokenizer-cache\" (UniqueName: \"kubernetes.io/empty-dir/0a4aeb8e-a962-4aeb-9918-c75cdb61f704-tokenizer-cache\") pod \"scheduler-ha-replicas-test-kserve-router-scheduler-fc6d477mptc7\" (UID: \"0a4aeb8e-a962-4aeb-9918-c75cdb61f704\") " pod="kserve-ci-e2e-test/scheduler-ha-replicas-test-kserve-router-scheduler-fc6d477mptc7"
Apr 24 14:39:29.119811 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:39:29.119775    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/0a4aeb8e-a962-4aeb-9918-c75cdb61f704-kserve-provision-location\") pod \"scheduler-ha-replicas-test-kserve-router-scheduler-fc6d477mptc7\" (UID: \"0a4aeb8e-a962-4aeb-9918-c75cdb61f704\") " pod="kserve-ci-e2e-test/scheduler-ha-replicas-test-kserve-router-scheduler-fc6d477mptc7"
Apr 24 14:39:29.119811 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:39:29.119794    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tokenizer-tmp\" (UniqueName: \"kubernetes.io/empty-dir/0a4aeb8e-a962-4aeb-9918-c75cdb61f704-tokenizer-tmp\") pod \"scheduler-ha-replicas-test-kserve-router-scheduler-fc6d477mptc7\" (UID: \"0a4aeb8e-a962-4aeb-9918-c75cdb61f704\") " pod="kserve-ci-e2e-test/scheduler-ha-replicas-test-kserve-router-scheduler-fc6d477mptc7"
Apr 24 14:39:29.121800 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:39:29.121779    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/0a4aeb8e-a962-4aeb-9918-c75cdb61f704-tls-certs\") pod \"scheduler-ha-replicas-test-kserve-router-scheduler-fc6d477mptc7\" (UID: \"0a4aeb8e-a962-4aeb-9918-c75cdb61f704\") " pod="kserve-ci-e2e-test/scheduler-ha-replicas-test-kserve-router-scheduler-fc6d477mptc7"
Apr 24 14:39:29.126433 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:39:29.126414    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-8v2qc\" (UniqueName: \"kubernetes.io/projected/0a4aeb8e-a962-4aeb-9918-c75cdb61f704-kube-api-access-8v2qc\") pod \"scheduler-ha-replicas-test-kserve-router-scheduler-fc6d477mptc7\" (UID: \"0a4aeb8e-a962-4aeb-9918-c75cdb61f704\") " pod="kserve-ci-e2e-test/scheduler-ha-replicas-test-kserve-router-scheduler-fc6d477mptc7"
Apr 24 14:39:29.164845 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:39:29.164811    2574 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/scheduler-ha-replicas-test-kserve-router-scheduler-fc6d477mptc7"
Apr 24 14:39:29.166180 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:39:29.166156    2574 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/scheduler-ha-replicas-test-kserve-9d798b947-brwzq" event={"ID":"09a3eabb-dbcc-44c9-9a75-a348e3806278","Type":"ContainerStarted","Data":"c0a9029d086de8822fe52cd26b57bedd817a0003d9d64c54c93251de82c983af"}
Apr 24 14:39:29.166270 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:39:29.166192    2574 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/scheduler-ha-replicas-test-kserve-9d798b947-brwzq" event={"ID":"09a3eabb-dbcc-44c9-9a75-a348e3806278","Type":"ContainerStarted","Data":"81c02675c385f0441c3c116071f73a87b9e6649ba399e33871f2a4fe41ae1813"}
Apr 24 14:39:29.292941 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:39:29.292913    2574 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/scheduler-ha-replicas-test-kserve-router-scheduler-fc6d477mptc7"]
Apr 24 14:39:29.294906 ip-10-0-136-132 kubenswrapper[2574]: W0424 14:39:29.294870    2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0a4aeb8e_a962_4aeb_9918_c75cdb61f704.slice/crio-a630df757ba8f7951c249107045d02a967dd9ed4b0933faacc679ce945a7b3d7 WatchSource:0}: Error finding container a630df757ba8f7951c249107045d02a967dd9ed4b0933faacc679ce945a7b3d7: Status 404 returned error can't find the container with id a630df757ba8f7951c249107045d02a967dd9ed4b0933faacc679ce945a7b3d7
Apr 24 14:39:30.171326 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:39:30.171291    2574 generic.go:358] "Generic (PLEG): container finished" podID="0a4aeb8e-a962-4aeb-9918-c75cdb61f704" containerID="5da23df157b5cb65a4ca2ce297645d68c549b471d891989e7ce14a3cfafd7e1b" exitCode=0
Apr 24 14:39:30.171816 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:39:30.171367    2574 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/scheduler-ha-replicas-test-kserve-router-scheduler-fc6d477mptc7" event={"ID":"0a4aeb8e-a962-4aeb-9918-c75cdb61f704","Type":"ContainerDied","Data":"5da23df157b5cb65a4ca2ce297645d68c549b471d891989e7ce14a3cfafd7e1b"}
Apr 24 14:39:30.171816 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:39:30.171406    2574 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/scheduler-ha-replicas-test-kserve-router-scheduler-fc6d477mptc7" event={"ID":"0a4aeb8e-a962-4aeb-9918-c75cdb61f704","Type":"ContainerStarted","Data":"a630df757ba8f7951c249107045d02a967dd9ed4b0933faacc679ce945a7b3d7"}
Apr 24 14:39:32.182288 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:39:32.182249    2574 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/scheduler-ha-replicas-test-kserve-router-scheduler-fc6d477mptc7" event={"ID":"0a4aeb8e-a962-4aeb-9918-c75cdb61f704","Type":"ContainerStarted","Data":"8e56c5a8e091156281b108bb2d06f84391ca732a5a987232ab699174561bd848"}
Apr 24 14:40:00.311111 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:40:00.311071    2574 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/scheduler-ha-replicas-test-kserve-router-scheduler-fc6d477mptc7" event={"ID":"0a4aeb8e-a962-4aeb-9918-c75cdb61f704","Type":"ContainerStarted","Data":"2a6de48338806c31f29f10aebdcc4545051f6e8677089d4a00eab3acbf351c1c"}
Apr 24 14:40:00.311534 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:40:00.311148    2574 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve-ci-e2e-test/scheduler-ha-replicas-test-kserve-router-scheduler-fc6d477mptc7"
Apr 24 14:40:00.329554 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:40:00.329503    2574 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="kserve-ci-e2e-test/scheduler-ha-replicas-test-kserve-router-scheduler-fc6d477mptc7" podStartSLOduration=2.672459081 podStartE2EDuration="32.329488507s" podCreationTimestamp="2026-04-24 14:39:28 +0000 UTC" firstStartedPulling="2026-04-24 14:39:30.172542697 +0000 UTC m=+932.904644324" lastFinishedPulling="2026-04-24 14:39:59.829572123 +0000 UTC m=+962.561673750" observedRunningTime="2026-04-24 14:40:00.328629032 +0000 UTC m=+963.060730682" watchObservedRunningTime="2026-04-24 14:40:00.329488507 +0000 UTC m=+963.061590157"
Apr 24 14:40:01.317797 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:40:01.317746    2574 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/scheduler-ha-replicas-test-kserve-router-scheduler-fc6d477mptc7" podUID="0a4aeb8e-a962-4aeb-9918-c75cdb61f704" containerName="main" probeResult="failure" output="service unhealthy (responded with \"NOT_SERVING\")"
Apr 24 14:40:09.165388 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:40:09.165354    2574 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve-ci-e2e-test/scheduler-ha-replicas-test-kserve-router-scheduler-fc6d477mptc7"
Apr 24 14:40:09.165985 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:40:09.165485    2574 kubelet.go:2658] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="kserve-ci-e2e-test/scheduler-ha-replicas-test-kserve-router-scheduler-fc6d477mptc7"
Apr 24 14:40:09.166923 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:40:09.166895    2574 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/scheduler-ha-replicas-test-kserve-router-scheduler-fc6d477mptc7" podUID="0a4aeb8e-a962-4aeb-9918-c75cdb61f704" containerName="main" probeResult="failure" output="service unhealthy (responded with \"NOT_SERVING\")"
Apr 24 14:40:09.167066 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:40:09.167049    2574 kubelet.go:2658] "SyncLoop (probe)" probe="startup" status="started" pod="kserve-ci-e2e-test/scheduler-ha-replicas-test-kserve-router-scheduler-fc6d477mptc7"
Apr 24 14:40:09.350596 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:40:09.350565    2574 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/scheduler-ha-replicas-test-kserve-router-scheduler-fc6d477mptc7" podUID="0a4aeb8e-a962-4aeb-9918-c75cdb61f704" containerName="main" probeResult="failure" output="service unhealthy (responded with \"NOT_SERVING\")"
Apr 24 14:40:09.350802 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:40:09.350783    2574 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="kserve-ci-e2e-test/scheduler-ha-replicas-test-kserve-router-scheduler-fc6d477mptc7"
Apr 24 14:40:10.354032 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:40:10.353956    2574 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/scheduler-ha-replicas-test-kserve-router-scheduler-fc6d477mptc7" podUID="0a4aeb8e-a962-4aeb-9918-c75cdb61f704" containerName="main" probeResult="failure" output="service unhealthy (responded with \"NOT_SERVING\")"
Apr 24 14:40:11.358003 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:40:11.357968    2574 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/scheduler-ha-replicas-test-kserve-router-scheduler-fc6d477mptc7" podUID="0a4aeb8e-a962-4aeb-9918-c75cdb61f704" containerName="main" probeResult="failure" output="service unhealthy (responded with \"NOT_SERVING\")"
Apr 24 14:40:21.358314 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:40:21.358259    2574 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/scheduler-ha-replicas-test-kserve-router-scheduler-fc6d477mptc7" podUID="0a4aeb8e-a962-4aeb-9918-c75cdb61f704" containerName="main" probeResult="failure" output="service unhealthy (responded with \"NOT_SERVING\")"
Apr 24 14:40:24.409025 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:40:24.408994    2574 generic.go:358] "Generic (PLEG): container finished" podID="09a3eabb-dbcc-44c9-9a75-a348e3806278" containerID="c0a9029d086de8822fe52cd26b57bedd817a0003d9d64c54c93251de82c983af" exitCode=0
Apr 24 14:40:24.409403 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:40:24.409066    2574 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/scheduler-ha-replicas-test-kserve-9d798b947-brwzq" event={"ID":"09a3eabb-dbcc-44c9-9a75-a348e3806278","Type":"ContainerDied","Data":"c0a9029d086de8822fe52cd26b57bedd817a0003d9d64c54c93251de82c983af"}
Apr 24 14:40:24.410112 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:40:24.410095    2574 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Apr 24 14:40:25.414839 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:40:25.414806    2574 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/scheduler-ha-replicas-test-kserve-9d798b947-brwzq" event={"ID":"09a3eabb-dbcc-44c9-9a75-a348e3806278","Type":"ContainerStarted","Data":"361b5ee7750302f4280d636fa5d7a6a287edf5bff67a3c6010ac5fbd643ca4d4"}
Apr 24 14:40:25.433365 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:40:25.433309    2574 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="kserve-ci-e2e-test/scheduler-ha-replicas-test-kserve-9d798b947-brwzq" podStartSLOduration=57.433295806 podStartE2EDuration="57.433295806s" podCreationTimestamp="2026-04-24 14:39:28 +0000 UTC" firstStartedPulling="0001-01-01 00:00:00 +0000 UTC" lastFinishedPulling="0001-01-01 00:00:00 +0000 UTC" observedRunningTime="2026-04-24 14:40:25.432129594 +0000 UTC m=+988.164231245" watchObservedRunningTime="2026-04-24 14:40:25.433295806 +0000 UTC m=+988.165397454"
Apr 24 14:40:28.915674 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:40:28.915636    2574 kubelet.go:2658] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="kserve-ci-e2e-test/scheduler-ha-replicas-test-kserve-9d798b947-brwzq"
Apr 24 14:40:28.915674 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:40:28.915676    2574 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve-ci-e2e-test/scheduler-ha-replicas-test-kserve-9d798b947-brwzq"
Apr 24 14:40:28.928528 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:40:28.928505    2574 kubelet.go:2658] "SyncLoop (probe)" probe="startup" status="started" pod="kserve-ci-e2e-test/scheduler-ha-replicas-test-kserve-9d798b947-brwzq"
Apr 24 14:40:29.441890 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:40:29.441866    2574 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="kserve-ci-e2e-test/scheduler-ha-replicas-test-kserve-9d798b947-brwzq"
Apr 24 14:40:31.358749 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:40:31.358711    2574 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/scheduler-ha-replicas-test-kserve-router-scheduler-fc6d477mptc7" podUID="0a4aeb8e-a962-4aeb-9918-c75cdb61f704" containerName="main" probeResult="failure" output="service unhealthy (responded with \"NOT_SERVING\")"
Apr 24 14:40:41.358252 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:40:41.358213    2574 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/scheduler-ha-replicas-test-kserve-router-scheduler-fc6d477mptc7" podUID="0a4aeb8e-a962-4aeb-9918-c75cdb61f704" containerName="main" probeResult="failure" output="service unhealthy (responded with \"NOT_SERVING\")"
Apr 24 14:40:49.897486 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:40:49.897455    2574 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["kserve-ci-e2e-test/precise-prefix-cache-test-kserve-75469f89c9-dfbfc"]
Apr 24 14:40:49.905969 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:40:49.905912    2574 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/precise-prefix-cache-test-kserve-75469f89c9-dfbfc"
Apr 24 14:40:49.908407 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:40:49.908384    2574 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"kserve-ci-e2e-test\"/\"precise-prefix-cache-test-kserve-self-signed-certs\""
Apr 24 14:40:49.910188 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:40:49.910165    2574 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/precise-prefix-cache-test-kserve-75469f89c9-dfbfc"]
Apr 24 14:40:50.038193 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:40:50.038163    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-pxkpr\" (UniqueName: \"kubernetes.io/projected/af01670c-533e-4c4f-935d-4a9f8a34a81d-kube-api-access-pxkpr\") pod \"precise-prefix-cache-test-kserve-75469f89c9-dfbfc\" (UID: \"af01670c-533e-4c4f-935d-4a9f8a34a81d\") " pod="kserve-ci-e2e-test/precise-prefix-cache-test-kserve-75469f89c9-dfbfc"
Apr 24 14:40:50.038325 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:40:50.038200    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/af01670c-533e-4c4f-935d-4a9f8a34a81d-kserve-provision-location\") pod \"precise-prefix-cache-test-kserve-75469f89c9-dfbfc\" (UID: \"af01670c-533e-4c4f-935d-4a9f8a34a81d\") " pod="kserve-ci-e2e-test/precise-prefix-cache-test-kserve-75469f89c9-dfbfc"
Apr 24 14:40:50.038325 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:40:50.038261    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/af01670c-533e-4c4f-935d-4a9f8a34a81d-home\") pod \"precise-prefix-cache-test-kserve-75469f89c9-dfbfc\" (UID: \"af01670c-533e-4c4f-935d-4a9f8a34a81d\") " pod="kserve-ci-e2e-test/precise-prefix-cache-test-kserve-75469f89c9-dfbfc"
Apr 24 14:40:50.038325 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:40:50.038311    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/af01670c-533e-4c4f-935d-4a9f8a34a81d-dshm\") pod \"precise-prefix-cache-test-kserve-75469f89c9-dfbfc\" (UID: \"af01670c-533e-4c4f-935d-4a9f8a34a81d\") " pod="kserve-ci-e2e-test/precise-prefix-cache-test-kserve-75469f89c9-dfbfc"
Apr 24 14:40:50.038478 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:40:50.038388    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/af01670c-533e-4c4f-935d-4a9f8a34a81d-model-cache\") pod \"precise-prefix-cache-test-kserve-75469f89c9-dfbfc\" (UID: \"af01670c-533e-4c4f-935d-4a9f8a34a81d\") " pod="kserve-ci-e2e-test/precise-prefix-cache-test-kserve-75469f89c9-dfbfc"
Apr 24 14:40:50.038478 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:40:50.038423    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/af01670c-533e-4c4f-935d-4a9f8a34a81d-tls-certs\") pod \"precise-prefix-cache-test-kserve-75469f89c9-dfbfc\" (UID: \"af01670c-533e-4c4f-935d-4a9f8a34a81d\") " pod="kserve-ci-e2e-test/precise-prefix-cache-test-kserve-75469f89c9-dfbfc"
Apr 24 14:40:50.139598 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:40:50.139571    2574 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/af01670c-533e-4c4f-935d-4a9f8a34a81d-dshm\") pod \"precise-prefix-cache-test-kserve-75469f89c9-dfbfc\" (UID: \"af01670c-533e-4c4f-935d-4a9f8a34a81d\") " pod="kserve-ci-e2e-test/precise-prefix-cache-test-kserve-75469f89c9-dfbfc"
Apr 24 14:40:50.139700 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:40:50.139617    2574 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/af01670c-533e-4c4f-935d-4a9f8a34a81d-model-cache\") pod \"precise-prefix-cache-test-kserve-75469f89c9-dfbfc\" (UID: \"af01670c-533e-4c4f-935d-4a9f8a34a81d\") " pod="kserve-ci-e2e-test/precise-prefix-cache-test-kserve-75469f89c9-dfbfc"
Apr 24 14:40:50.139700 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:40:50.139650    2574 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/af01670c-533e-4c4f-935d-4a9f8a34a81d-tls-certs\") pod \"precise-prefix-cache-test-kserve-75469f89c9-dfbfc\" (UID: \"af01670c-533e-4c4f-935d-4a9f8a34a81d\") " pod="kserve-ci-e2e-test/precise-prefix-cache-test-kserve-75469f89c9-dfbfc"
Apr 24 14:40:50.139700 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:40:50.139690    2574 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-pxkpr\" (UniqueName: \"kubernetes.io/projected/af01670c-533e-4c4f-935d-4a9f8a34a81d-kube-api-access-pxkpr\") pod \"precise-prefix-cache-test-kserve-75469f89c9-dfbfc\" (UID: \"af01670c-533e-4c4f-935d-4a9f8a34a81d\") " pod="kserve-ci-e2e-test/precise-prefix-cache-test-kserve-75469f89c9-dfbfc"
Apr 24 14:40:50.139810 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:40:50.139725    2574 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/af01670c-533e-4c4f-935d-4a9f8a34a81d-kserve-provision-location\") pod \"precise-prefix-cache-test-kserve-75469f89c9-dfbfc\" (UID: \"af01670c-533e-4c4f-935d-4a9f8a34a81d\") " pod="kserve-ci-e2e-test/precise-prefix-cache-test-kserve-75469f89c9-dfbfc"
Apr 24 14:40:50.139810 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:40:50.139774    2574 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/af01670c-533e-4c4f-935d-4a9f8a34a81d-home\") pod \"precise-prefix-cache-test-kserve-75469f89c9-dfbfc\" (UID: \"af01670c-533e-4c4f-935d-4a9f8a34a81d\") " pod="kserve-ci-e2e-test/precise-prefix-cache-test-kserve-75469f89c9-dfbfc"
Apr 24 14:40:50.140092 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:40:50.140062    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/af01670c-533e-4c4f-935d-4a9f8a34a81d-model-cache\") pod \"precise-prefix-cache-test-kserve-75469f89c9-dfbfc\" (UID: \"af01670c-533e-4c4f-935d-4a9f8a34a81d\") " pod="kserve-ci-e2e-test/precise-prefix-cache-test-kserve-75469f89c9-dfbfc"
Apr 24 14:40:50.140228 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:40:50.140108    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/af01670c-533e-4c4f-935d-4a9f8a34a81d-kserve-provision-location\") pod \"precise-prefix-cache-test-kserve-75469f89c9-dfbfc\" (UID: \"af01670c-533e-4c4f-935d-4a9f8a34a81d\") " pod="kserve-ci-e2e-test/precise-prefix-cache-test-kserve-75469f89c9-dfbfc"
Apr 24 14:40:50.140228 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:40:50.140124    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/af01670c-533e-4c4f-935d-4a9f8a34a81d-home\") pod \"precise-prefix-cache-test-kserve-75469f89c9-dfbfc\" (UID: \"af01670c-533e-4c4f-935d-4a9f8a34a81d\") " pod="kserve-ci-e2e-test/precise-prefix-cache-test-kserve-75469f89c9-dfbfc"
Apr 24 14:40:50.141936 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:40:50.141909    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/af01670c-533e-4c4f-935d-4a9f8a34a81d-dshm\") pod \"precise-prefix-cache-test-kserve-75469f89c9-dfbfc\" (UID: \"af01670c-533e-4c4f-935d-4a9f8a34a81d\") " pod="kserve-ci-e2e-test/precise-prefix-cache-test-kserve-75469f89c9-dfbfc"
Apr 24 14:40:50.142107 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:40:50.142091    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/af01670c-533e-4c4f-935d-4a9f8a34a81d-tls-certs\") pod \"precise-prefix-cache-test-kserve-75469f89c9-dfbfc\" (UID: \"af01670c-533e-4c4f-935d-4a9f8a34a81d\") " pod="kserve-ci-e2e-test/precise-prefix-cache-test-kserve-75469f89c9-dfbfc"
Apr 24 14:40:50.147238 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:40:50.147213    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-pxkpr\" (UniqueName: \"kubernetes.io/projected/af01670c-533e-4c4f-935d-4a9f8a34a81d-kube-api-access-pxkpr\") pod \"precise-prefix-cache-test-kserve-75469f89c9-dfbfc\" (UID: \"af01670c-533e-4c4f-935d-4a9f8a34a81d\") " pod="kserve-ci-e2e-test/precise-prefix-cache-test-kserve-75469f89c9-dfbfc"
Apr 24 14:40:50.205569 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:40:50.205502    2574 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["kserve-ci-e2e-test/precise-prefix-cache-test-kserve-router-scheduler-667fbc78lr57q"]
Apr 24 14:40:50.208459 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:40:50.208439    2574 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/precise-prefix-cache-test-kserve-router-scheduler-667fbc78lr57q"
Apr 24 14:40:50.210313 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:40:50.210296    2574 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"kserve-ci-e2e-test\"/\"precise-prefix-cache-test-epp-sa-dockercfg-mwfgj\""
Apr 24 14:40:50.217612 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:40:50.217590    2574 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/precise-prefix-cache-test-kserve-75469f89c9-dfbfc"
Apr 24 14:40:50.218284 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:40:50.218263    2574 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/precise-prefix-cache-test-kserve-router-scheduler-667fbc78lr57q"]
Apr 24 14:40:50.341405 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:40:50.341377    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tokenizer-tmp\" (UniqueName: \"kubernetes.io/empty-dir/803af3d5-3ffe-40cc-94fe-56ed1cd0b30d-tokenizer-tmp\") pod \"precise-prefix-cache-test-kserve-router-scheduler-667fbc78lr57q\" (UID: \"803af3d5-3ffe-40cc-94fe-56ed1cd0b30d\") " pod="kserve-ci-e2e-test/precise-prefix-cache-test-kserve-router-scheduler-667fbc78lr57q"
Apr 24 14:40:50.341515 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:40:50.341426    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/803af3d5-3ffe-40cc-94fe-56ed1cd0b30d-tls-certs\") pod \"precise-prefix-cache-test-kserve-router-scheduler-667fbc78lr57q\" (UID: \"803af3d5-3ffe-40cc-94fe-56ed1cd0b30d\") " pod="kserve-ci-e2e-test/precise-prefix-cache-test-kserve-router-scheduler-667fbc78lr57q"
Apr 24 14:40:50.341567 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:40:50.341516    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/803af3d5-3ffe-40cc-94fe-56ed1cd0b30d-kserve-provision-location\") pod \"precise-prefix-cache-test-kserve-router-scheduler-667fbc78lr57q\" (UID: \"803af3d5-3ffe-40cc-94fe-56ed1cd0b30d\") " pod="kserve-ci-e2e-test/precise-prefix-cache-test-kserve-router-scheduler-667fbc78lr57q"
Apr 24 14:40:50.341602 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:40:50.341588    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tokenizer-cache\" (UniqueName: \"kubernetes.io/empty-dir/803af3d5-3ffe-40cc-94fe-56ed1cd0b30d-tokenizer-cache\") pod \"precise-prefix-cache-test-kserve-router-scheduler-667fbc78lr57q\" (UID: \"803af3d5-3ffe-40cc-94fe-56ed1cd0b30d\") " pod="kserve-ci-e2e-test/precise-prefix-cache-test-kserve-router-scheduler-667fbc78lr57q"
Apr 24 14:40:50.341636 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:40:50.341612    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tokenizer-uds\" (UniqueName: \"kubernetes.io/empty-dir/803af3d5-3ffe-40cc-94fe-56ed1cd0b30d-tokenizer-uds\") pod \"precise-prefix-cache-test-kserve-router-scheduler-667fbc78lr57q\" (UID: \"803af3d5-3ffe-40cc-94fe-56ed1cd0b30d\") " pod="kserve-ci-e2e-test/precise-prefix-cache-test-kserve-router-scheduler-667fbc78lr57q"
Apr 24 14:40:50.341674 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:40:50.341639    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-78vp2\" (UniqueName: \"kubernetes.io/projected/803af3d5-3ffe-40cc-94fe-56ed1cd0b30d-kube-api-access-78vp2\") pod \"precise-prefix-cache-test-kserve-router-scheduler-667fbc78lr57q\" (UID: \"803af3d5-3ffe-40cc-94fe-56ed1cd0b30d\") " pod="kserve-ci-e2e-test/precise-prefix-cache-test-kserve-router-scheduler-667fbc78lr57q"
Apr 24 14:40:50.442091 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:40:50.442062    2574 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/803af3d5-3ffe-40cc-94fe-56ed1cd0b30d-tls-certs\") pod \"precise-prefix-cache-test-kserve-router-scheduler-667fbc78lr57q\" (UID: \"803af3d5-3ffe-40cc-94fe-56ed1cd0b30d\") " pod="kserve-ci-e2e-test/precise-prefix-cache-test-kserve-router-scheduler-667fbc78lr57q"
Apr 24 14:40:50.442208 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:40:50.442108    2574 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/803af3d5-3ffe-40cc-94fe-56ed1cd0b30d-kserve-provision-location\") pod \"precise-prefix-cache-test-kserve-router-scheduler-667fbc78lr57q\" (UID: \"803af3d5-3ffe-40cc-94fe-56ed1cd0b30d\") " pod="kserve-ci-e2e-test/precise-prefix-cache-test-kserve-router-scheduler-667fbc78lr57q"
Apr 24 14:40:50.442208 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:40:50.442146    2574 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"tokenizer-cache\" (UniqueName: \"kubernetes.io/empty-dir/803af3d5-3ffe-40cc-94fe-56ed1cd0b30d-tokenizer-cache\") pod \"precise-prefix-cache-test-kserve-router-scheduler-667fbc78lr57q\" (UID: \"803af3d5-3ffe-40cc-94fe-56ed1cd0b30d\") " pod="kserve-ci-e2e-test/precise-prefix-cache-test-kserve-router-scheduler-667fbc78lr57q"
Apr 24 14:40:50.442208 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:40:50.442165    2574 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"tokenizer-uds\" (UniqueName: \"kubernetes.io/empty-dir/803af3d5-3ffe-40cc-94fe-56ed1cd0b30d-tokenizer-uds\") pod \"precise-prefix-cache-test-kserve-router-scheduler-667fbc78lr57q\" (UID: \"803af3d5-3ffe-40cc-94fe-56ed1cd0b30d\") " pod="kserve-ci-e2e-test/precise-prefix-cache-test-kserve-router-scheduler-667fbc78lr57q"
Apr 24 14:40:50.442208 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:40:50.442187    2574 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-78vp2\" (UniqueName: \"kubernetes.io/projected/803af3d5-3ffe-40cc-94fe-56ed1cd0b30d-kube-api-access-78vp2\") pod \"precise-prefix-cache-test-kserve-router-scheduler-667fbc78lr57q\" (UID: \"803af3d5-3ffe-40cc-94fe-56ed1cd0b30d\") " pod="kserve-ci-e2e-test/precise-prefix-cache-test-kserve-router-scheduler-667fbc78lr57q"
Apr 24 14:40:50.442453 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:40:50.442221    2574 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"tokenizer-tmp\" (UniqueName: \"kubernetes.io/empty-dir/803af3d5-3ffe-40cc-94fe-56ed1cd0b30d-tokenizer-tmp\") pod \"precise-prefix-cache-test-kserve-router-scheduler-667fbc78lr57q\" (UID: \"803af3d5-3ffe-40cc-94fe-56ed1cd0b30d\") " pod="kserve-ci-e2e-test/precise-prefix-cache-test-kserve-router-scheduler-667fbc78lr57q"
Apr 24 14:40:50.442634 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:40:50.442599    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/803af3d5-3ffe-40cc-94fe-56ed1cd0b30d-kserve-provision-location\") pod \"precise-prefix-cache-test-kserve-router-scheduler-667fbc78lr57q\" (UID: \"803af3d5-3ffe-40cc-94fe-56ed1cd0b30d\") " pod="kserve-ci-e2e-test/precise-prefix-cache-test-kserve-router-scheduler-667fbc78lr57q"
Apr 24 14:40:50.442693 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:40:50.442620    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tokenizer-cache\" (UniqueName: \"kubernetes.io/empty-dir/803af3d5-3ffe-40cc-94fe-56ed1cd0b30d-tokenizer-cache\") pod \"precise-prefix-cache-test-kserve-router-scheduler-667fbc78lr57q\" (UID: \"803af3d5-3ffe-40cc-94fe-56ed1cd0b30d\") " pod="kserve-ci-e2e-test/precise-prefix-cache-test-kserve-router-scheduler-667fbc78lr57q"
Apr 24 14:40:50.442693 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:40:50.442674    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tokenizer-tmp\" (UniqueName: \"kubernetes.io/empty-dir/803af3d5-3ffe-40cc-94fe-56ed1cd0b30d-tokenizer-tmp\") pod \"precise-prefix-cache-test-kserve-router-scheduler-667fbc78lr57q\" (UID: \"803af3d5-3ffe-40cc-94fe-56ed1cd0b30d\") " pod="kserve-ci-e2e-test/precise-prefix-cache-test-kserve-router-scheduler-667fbc78lr57q"
Apr 24 14:40:50.442767 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:40:50.442693    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tokenizer-uds\" (UniqueName: \"kubernetes.io/empty-dir/803af3d5-3ffe-40cc-94fe-56ed1cd0b30d-tokenizer-uds\") pod \"precise-prefix-cache-test-kserve-router-scheduler-667fbc78lr57q\" (UID: \"803af3d5-3ffe-40cc-94fe-56ed1cd0b30d\") " pod="kserve-ci-e2e-test/precise-prefix-cache-test-kserve-router-scheduler-667fbc78lr57q"
Apr 24 14:40:50.444520 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:40:50.444499    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/803af3d5-3ffe-40cc-94fe-56ed1cd0b30d-tls-certs\") pod \"precise-prefix-cache-test-kserve-router-scheduler-667fbc78lr57q\" (UID: \"803af3d5-3ffe-40cc-94fe-56ed1cd0b30d\") " pod="kserve-ci-e2e-test/precise-prefix-cache-test-kserve-router-scheduler-667fbc78lr57q"
Apr 24 14:40:50.449058 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:40:50.449036    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-78vp2\" (UniqueName: \"kubernetes.io/projected/803af3d5-3ffe-40cc-94fe-56ed1cd0b30d-kube-api-access-78vp2\") pod \"precise-prefix-cache-test-kserve-router-scheduler-667fbc78lr57q\" (UID: \"803af3d5-3ffe-40cc-94fe-56ed1cd0b30d\") " pod="kserve-ci-e2e-test/precise-prefix-cache-test-kserve-router-scheduler-667fbc78lr57q"
Apr 24 14:40:50.518579 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:40:50.518521    2574 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/precise-prefix-cache-test-kserve-router-scheduler-667fbc78lr57q"
Apr 24 14:40:50.547707 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:40:50.547680    2574 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/precise-prefix-cache-test-kserve-75469f89c9-dfbfc"]
Apr 24 14:40:50.548123 ip-10-0-136-132 kubenswrapper[2574]: W0424 14:40:50.548100    2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podaf01670c_533e_4c4f_935d_4a9f8a34a81d.slice/crio-8eef4dae57a3a9e9492e2578672b0373aa7732bba8993c40910c368a17d65eff WatchSource:0}: Error finding container 8eef4dae57a3a9e9492e2578672b0373aa7732bba8993c40910c368a17d65eff: Status 404 returned error can't find the container with id 8eef4dae57a3a9e9492e2578672b0373aa7732bba8993c40910c368a17d65eff
Apr 24 14:40:50.655857 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:40:50.655823    2574 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/precise-prefix-cache-test-kserve-router-scheduler-667fbc78lr57q"]
Apr 24 14:40:50.657105 ip-10-0-136-132 kubenswrapper[2574]: W0424 14:40:50.657074    2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod803af3d5_3ffe_40cc_94fe_56ed1cd0b30d.slice/crio-deeebf6f716cc2603547c785d9be6632427c35109ea2e6a37cbf01d25b2db121 WatchSource:0}: Error finding container deeebf6f716cc2603547c785d9be6632427c35109ea2e6a37cbf01d25b2db121: Status 404 returned error can't find the container with id deeebf6f716cc2603547c785d9be6632427c35109ea2e6a37cbf01d25b2db121
Apr 24 14:40:51.358540 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:40:51.358504    2574 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/scheduler-ha-replicas-test-kserve-router-scheduler-fc6d477mptc7" podUID="0a4aeb8e-a962-4aeb-9918-c75cdb61f704" containerName="main" probeResult="failure" output="service unhealthy (responded with \"NOT_SERVING\")"
Apr 24 14:40:51.518964 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:40:51.518930    2574 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/precise-prefix-cache-test-kserve-75469f89c9-dfbfc" event={"ID":"af01670c-533e-4c4f-935d-4a9f8a34a81d","Type":"ContainerStarted","Data":"54d8616ab70d8dc419c3359cd83eb8053047bb492cd5bae8e6d7d4fcba5d1394"}
Apr 24 14:40:51.519095 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:40:51.518976    2574 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/precise-prefix-cache-test-kserve-75469f89c9-dfbfc" event={"ID":"af01670c-533e-4c4f-935d-4a9f8a34a81d","Type":"ContainerStarted","Data":"8eef4dae57a3a9e9492e2578672b0373aa7732bba8993c40910c368a17d65eff"}
Apr 24 14:40:51.520394 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:40:51.520363    2574 generic.go:358] "Generic (PLEG): container finished" podID="803af3d5-3ffe-40cc-94fe-56ed1cd0b30d" containerID="2a425b3e51a584bc36971dfab9bdb878c40a8ebdb62965099c7ad649866ef1d7" exitCode=0
Apr 24 14:40:51.520523 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:40:51.520443    2574 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/precise-prefix-cache-test-kserve-router-scheduler-667fbc78lr57q" event={"ID":"803af3d5-3ffe-40cc-94fe-56ed1cd0b30d","Type":"ContainerDied","Data":"2a425b3e51a584bc36971dfab9bdb878c40a8ebdb62965099c7ad649866ef1d7"}
Apr 24 14:40:51.520523 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:40:51.520480    2574 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/precise-prefix-cache-test-kserve-router-scheduler-667fbc78lr57q" event={"ID":"803af3d5-3ffe-40cc-94fe-56ed1cd0b30d","Type":"ContainerStarted","Data":"deeebf6f716cc2603547c785d9be6632427c35109ea2e6a37cbf01d25b2db121"}
Apr 24 14:40:51.815062 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:40:51.815015    2574 kubelet.go:2553] "SyncLoop DELETE" source="api" pods=["kserve-ci-e2e-test/scheduler-ha-replicas-test-kserve-9d798b947-brwzq"]
Apr 24 14:40:51.815618 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:40:51.815576    2574 kuberuntime_container.go:864] "Killing container with a grace period" pod="kserve-ci-e2e-test/scheduler-ha-replicas-test-kserve-9d798b947-brwzq" podUID="09a3eabb-dbcc-44c9-9a75-a348e3806278" containerName="main" containerID="cri-o://361b5ee7750302f4280d636fa5d7a6a287edf5bff67a3c6010ac5fbd643ca4d4" gracePeriod=30
Apr 24 14:40:51.823977 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:40:51.823950    2574 kubelet.go:2553] "SyncLoop DELETE" source="api" pods=["kserve-ci-e2e-test/scheduler-ha-replicas-test-kserve-router-scheduler-fc6d477mptc7"]
Apr 24 14:40:51.824616 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:40:51.824584    2574 kuberuntime_container.go:864] "Killing container with a grace period" pod="kserve-ci-e2e-test/scheduler-ha-replicas-test-kserve-router-scheduler-fc6d477mptc7" podUID="0a4aeb8e-a962-4aeb-9918-c75cdb61f704" containerName="main" containerID="cri-o://8e56c5a8e091156281b108bb2d06f84391ca732a5a987232ab699174561bd848" gracePeriod=30
Apr 24 14:40:51.824826 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:40:51.824633    2574 kuberuntime_container.go:864] "Killing container with a grace period" pod="kserve-ci-e2e-test/scheduler-ha-replicas-test-kserve-router-scheduler-fc6d477mptc7" podUID="0a4aeb8e-a962-4aeb-9918-c75cdb61f704" containerName="tokenizer" containerID="cri-o://2a6de48338806c31f29f10aebdcc4545051f6e8677089d4a00eab3acbf351c1c" gracePeriod=30
Apr 24 14:40:51.826375 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:40:51.826280    2574 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/scheduler-ha-replicas-test-kserve-router-scheduler-fc6d477mptc7" podUID="0a4aeb8e-a962-4aeb-9918-c75cdb61f704" containerName="main" probeResult="failure" output="service unhealthy (responded with \"NOT_SERVING\")"
Apr 24 14:40:52.104185 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:40:52.104157    2574 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/scheduler-ha-replicas-test-kserve-9d798b947-brwzq"
Apr 24 14:40:52.263096 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:40:52.263006    2574 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/09a3eabb-dbcc-44c9-9a75-a348e3806278-kserve-provision-location\") pod \"09a3eabb-dbcc-44c9-9a75-a348e3806278\" (UID: \"09a3eabb-dbcc-44c9-9a75-a348e3806278\") "
Apr 24 14:40:52.263285 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:40:52.263099    2574 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"kube-api-access-kq48z\" (UniqueName: \"kubernetes.io/projected/09a3eabb-dbcc-44c9-9a75-a348e3806278-kube-api-access-kq48z\") pod \"09a3eabb-dbcc-44c9-9a75-a348e3806278\" (UID: \"09a3eabb-dbcc-44c9-9a75-a348e3806278\") "
Apr 24 14:40:52.263285 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:40:52.263146    2574 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/09a3eabb-dbcc-44c9-9a75-a348e3806278-home\") pod \"09a3eabb-dbcc-44c9-9a75-a348e3806278\" (UID: \"09a3eabb-dbcc-44c9-9a75-a348e3806278\") "
Apr 24 14:40:52.263285 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:40:52.263213    2574 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/09a3eabb-dbcc-44c9-9a75-a348e3806278-dshm\") pod \"09a3eabb-dbcc-44c9-9a75-a348e3806278\" (UID: \"09a3eabb-dbcc-44c9-9a75-a348e3806278\") "
Apr 24 14:40:52.263285 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:40:52.263257    2574 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/09a3eabb-dbcc-44c9-9a75-a348e3806278-model-cache\") pod \"09a3eabb-dbcc-44c9-9a75-a348e3806278\" (UID: \"09a3eabb-dbcc-44c9-9a75-a348e3806278\") "
Apr 24 14:40:52.263285 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:40:52.263284    2574 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/09a3eabb-dbcc-44c9-9a75-a348e3806278-tls-certs\") pod \"09a3eabb-dbcc-44c9-9a75-a348e3806278\" (UID: \"09a3eabb-dbcc-44c9-9a75-a348e3806278\") "
Apr 24 14:40:52.263556 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:40:52.263376    2574 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/09a3eabb-dbcc-44c9-9a75-a348e3806278-home" (OuterVolumeSpecName: "home") pod "09a3eabb-dbcc-44c9-9a75-a348e3806278" (UID: "09a3eabb-dbcc-44c9-9a75-a348e3806278"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Apr 24 14:40:52.263691 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:40:52.263666    2574 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/09a3eabb-dbcc-44c9-9a75-a348e3806278-home\") on node \"ip-10-0-136-132.ec2.internal\" DevicePath \"\""
Apr 24 14:40:52.263863 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:40:52.263804    2574 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/09a3eabb-dbcc-44c9-9a75-a348e3806278-model-cache" (OuterVolumeSpecName: "model-cache") pod "09a3eabb-dbcc-44c9-9a75-a348e3806278" (UID: "09a3eabb-dbcc-44c9-9a75-a348e3806278"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Apr 24 14:40:52.265696 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:40:52.265659    2574 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/09a3eabb-dbcc-44c9-9a75-a348e3806278-kube-api-access-kq48z" (OuterVolumeSpecName: "kube-api-access-kq48z") pod "09a3eabb-dbcc-44c9-9a75-a348e3806278" (UID: "09a3eabb-dbcc-44c9-9a75-a348e3806278"). InnerVolumeSpecName "kube-api-access-kq48z". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Apr 24 14:40:52.266747 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:40:52.266726    2574 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/09a3eabb-dbcc-44c9-9a75-a348e3806278-dshm" (OuterVolumeSpecName: "dshm") pod "09a3eabb-dbcc-44c9-9a75-a348e3806278" (UID: "09a3eabb-dbcc-44c9-9a75-a348e3806278"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Apr 24 14:40:52.266998 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:40:52.266980    2574 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/09a3eabb-dbcc-44c9-9a75-a348e3806278-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "09a3eabb-dbcc-44c9-9a75-a348e3806278" (UID: "09a3eabb-dbcc-44c9-9a75-a348e3806278"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Apr 24 14:40:52.351699 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:40:52.351651    2574 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/09a3eabb-dbcc-44c9-9a75-a348e3806278-kserve-provision-location" (OuterVolumeSpecName: "kserve-provision-location") pod "09a3eabb-dbcc-44c9-9a75-a348e3806278" (UID: "09a3eabb-dbcc-44c9-9a75-a348e3806278"). InnerVolumeSpecName "kserve-provision-location". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Apr 24 14:40:52.364300 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:40:52.364267    2574 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/09a3eabb-dbcc-44c9-9a75-a348e3806278-dshm\") on node \"ip-10-0-136-132.ec2.internal\" DevicePath \"\""
Apr 24 14:40:52.364300 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:40:52.364299    2574 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/09a3eabb-dbcc-44c9-9a75-a348e3806278-model-cache\") on node \"ip-10-0-136-132.ec2.internal\" DevicePath \"\""
Apr 24 14:40:52.364779 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:40:52.364311    2574 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/09a3eabb-dbcc-44c9-9a75-a348e3806278-tls-certs\") on node \"ip-10-0-136-132.ec2.internal\" DevicePath \"\""
Apr 24 14:40:52.364779 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:40:52.364325    2574 reconciler_common.go:299] "Volume detached for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/09a3eabb-dbcc-44c9-9a75-a348e3806278-kserve-provision-location\") on node \"ip-10-0-136-132.ec2.internal\" DevicePath \"\""
Apr 24 14:40:52.364779 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:40:52.364366    2574 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-kq48z\" (UniqueName: \"kubernetes.io/projected/09a3eabb-dbcc-44c9-9a75-a348e3806278-kube-api-access-kq48z\") on node \"ip-10-0-136-132.ec2.internal\" DevicePath \"\""
Apr 24 14:40:52.529158 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:40:52.529074    2574 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/precise-prefix-cache-test-kserve-router-scheduler-667fbc78lr57q" event={"ID":"803af3d5-3ffe-40cc-94fe-56ed1cd0b30d","Type":"ContainerStarted","Data":"72290df1390369ba24415bdb8a026783a801ff6d03d775307f0bcf1362d66d64"}
Apr 24 14:40:52.529158 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:40:52.529117    2574 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/precise-prefix-cache-test-kserve-router-scheduler-667fbc78lr57q" event={"ID":"803af3d5-3ffe-40cc-94fe-56ed1cd0b30d","Type":"ContainerStarted","Data":"7839abf3e398cd5a4ec2d31b8f4ca1f668ce14c0f36cf90975027e29903d507b"}
Apr 24 14:40:52.529418 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:40:52.529219    2574 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve-ci-e2e-test/precise-prefix-cache-test-kserve-router-scheduler-667fbc78lr57q"
Apr 24 14:40:52.531256 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:40:52.531225    2574 generic.go:358] "Generic (PLEG): container finished" podID="0a4aeb8e-a962-4aeb-9918-c75cdb61f704" containerID="8e56c5a8e091156281b108bb2d06f84391ca732a5a987232ab699174561bd848" exitCode=0
Apr 24 14:40:52.531384 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:40:52.531289    2574 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/scheduler-ha-replicas-test-kserve-router-scheduler-fc6d477mptc7" event={"ID":"0a4aeb8e-a962-4aeb-9918-c75cdb61f704","Type":"ContainerDied","Data":"8e56c5a8e091156281b108bb2d06f84391ca732a5a987232ab699174561bd848"}
Apr 24 14:40:52.532777 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:40:52.532749    2574 generic.go:358] "Generic (PLEG): container finished" podID="09a3eabb-dbcc-44c9-9a75-a348e3806278" containerID="361b5ee7750302f4280d636fa5d7a6a287edf5bff67a3c6010ac5fbd643ca4d4" exitCode=0
Apr 24 14:40:52.532893 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:40:52.532784    2574 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/scheduler-ha-replicas-test-kserve-9d798b947-brwzq" event={"ID":"09a3eabb-dbcc-44c9-9a75-a348e3806278","Type":"ContainerDied","Data":"361b5ee7750302f4280d636fa5d7a6a287edf5bff67a3c6010ac5fbd643ca4d4"}
Apr 24 14:40:52.532893 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:40:52.532814    2574 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/scheduler-ha-replicas-test-kserve-9d798b947-brwzq" event={"ID":"09a3eabb-dbcc-44c9-9a75-a348e3806278","Type":"ContainerDied","Data":"81c02675c385f0441c3c116071f73a87b9e6649ba399e33871f2a4fe41ae1813"}
Apr 24 14:40:52.532893 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:40:52.532837    2574 scope.go:117] "RemoveContainer" containerID="361b5ee7750302f4280d636fa5d7a6a287edf5bff67a3c6010ac5fbd643ca4d4"
Apr 24 14:40:52.532893 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:40:52.532846    2574 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/scheduler-ha-replicas-test-kserve-9d798b947-brwzq"
Apr 24 14:40:52.542983 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:40:52.542965    2574 scope.go:117] "RemoveContainer" containerID="c0a9029d086de8822fe52cd26b57bedd817a0003d9d64c54c93251de82c983af"
Apr 24 14:40:52.552285 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:40:52.552235    2574 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="kserve-ci-e2e-test/precise-prefix-cache-test-kserve-router-scheduler-667fbc78lr57q" podStartSLOduration=2.552218623 podStartE2EDuration="2.552218623s" podCreationTimestamp="2026-04-24 14:40:50 +0000 UTC" firstStartedPulling="0001-01-01 00:00:00 +0000 UTC" lastFinishedPulling="0001-01-01 00:00:00 +0000 UTC" observedRunningTime="2026-04-24 14:40:52.548693028 +0000 UTC m=+1015.280794683" watchObservedRunningTime="2026-04-24 14:40:52.552218623 +0000 UTC m=+1015.284320273"
Apr 24 14:40:52.554972 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:40:52.554947    2574 scope.go:117] "RemoveContainer" containerID="361b5ee7750302f4280d636fa5d7a6a287edf5bff67a3c6010ac5fbd643ca4d4"
Apr 24 14:40:52.555257 ip-10-0-136-132 kubenswrapper[2574]: E0424 14:40:52.555234    2574 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"361b5ee7750302f4280d636fa5d7a6a287edf5bff67a3c6010ac5fbd643ca4d4\": container with ID starting with 361b5ee7750302f4280d636fa5d7a6a287edf5bff67a3c6010ac5fbd643ca4d4 not found: ID does not exist" containerID="361b5ee7750302f4280d636fa5d7a6a287edf5bff67a3c6010ac5fbd643ca4d4"
Apr 24 14:40:52.555374 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:40:52.555261    2574 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"361b5ee7750302f4280d636fa5d7a6a287edf5bff67a3c6010ac5fbd643ca4d4"} err="failed to get container status \"361b5ee7750302f4280d636fa5d7a6a287edf5bff67a3c6010ac5fbd643ca4d4\": rpc error: code = NotFound desc = could not find container \"361b5ee7750302f4280d636fa5d7a6a287edf5bff67a3c6010ac5fbd643ca4d4\": container with ID starting with 361b5ee7750302f4280d636fa5d7a6a287edf5bff67a3c6010ac5fbd643ca4d4 not found: ID does not exist"
Apr 24 14:40:52.555374 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:40:52.555278    2574 scope.go:117] "RemoveContainer" containerID="c0a9029d086de8822fe52cd26b57bedd817a0003d9d64c54c93251de82c983af"
Apr 24 14:40:52.555552 ip-10-0-136-132 kubenswrapper[2574]: E0424 14:40:52.555527    2574 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"c0a9029d086de8822fe52cd26b57bedd817a0003d9d64c54c93251de82c983af\": container with ID starting with c0a9029d086de8822fe52cd26b57bedd817a0003d9d64c54c93251de82c983af not found: ID does not exist" containerID="c0a9029d086de8822fe52cd26b57bedd817a0003d9d64c54c93251de82c983af"
Apr 24 14:40:52.555619 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:40:52.555562    2574 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"c0a9029d086de8822fe52cd26b57bedd817a0003d9d64c54c93251de82c983af"} err="failed to get container status \"c0a9029d086de8822fe52cd26b57bedd817a0003d9d64c54c93251de82c983af\": rpc error: code = NotFound desc = could not find container \"c0a9029d086de8822fe52cd26b57bedd817a0003d9d64c54c93251de82c983af\": container with ID starting with c0a9029d086de8822fe52cd26b57bedd817a0003d9d64c54c93251de82c983af not found: ID does not exist"
Apr 24 14:40:52.562727 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:40:52.562680    2574 kubelet.go:2553] "SyncLoop DELETE" source="api" pods=["kserve-ci-e2e-test/scheduler-ha-replicas-test-kserve-9d798b947-brwzq"]
Apr 24 14:40:52.566713 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:40:52.566687    2574 kubelet.go:2547] "SyncLoop REMOVE" source="api" pods=["kserve-ci-e2e-test/scheduler-ha-replicas-test-kserve-9d798b947-brwzq"]
Apr 24 14:40:53.185892 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:40:53.185865    2574 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/scheduler-ha-replicas-test-kserve-router-scheduler-fc6d477mptc7"
Apr 24 14:40:53.272027 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:40:53.271994    2574 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/0a4aeb8e-a962-4aeb-9918-c75cdb61f704-kserve-provision-location\") pod \"0a4aeb8e-a962-4aeb-9918-c75cdb61f704\" (UID: \"0a4aeb8e-a962-4aeb-9918-c75cdb61f704\") "
Apr 24 14:40:53.272247 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:40:53.272056    2574 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"tokenizer-uds\" (UniqueName: \"kubernetes.io/empty-dir/0a4aeb8e-a962-4aeb-9918-c75cdb61f704-tokenizer-uds\") pod \"0a4aeb8e-a962-4aeb-9918-c75cdb61f704\" (UID: \"0a4aeb8e-a962-4aeb-9918-c75cdb61f704\") "
Apr 24 14:40:53.272247 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:40:53.272115    2574 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/0a4aeb8e-a962-4aeb-9918-c75cdb61f704-tls-certs\") pod \"0a4aeb8e-a962-4aeb-9918-c75cdb61f704\" (UID: \"0a4aeb8e-a962-4aeb-9918-c75cdb61f704\") "
Apr 24 14:40:53.272247 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:40:53.272133    2574 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"tokenizer-tmp\" (UniqueName: \"kubernetes.io/empty-dir/0a4aeb8e-a962-4aeb-9918-c75cdb61f704-tokenizer-tmp\") pod \"0a4aeb8e-a962-4aeb-9918-c75cdb61f704\" (UID: \"0a4aeb8e-a962-4aeb-9918-c75cdb61f704\") "
Apr 24 14:40:53.272247 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:40:53.272163    2574 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"kube-api-access-8v2qc\" (UniqueName: \"kubernetes.io/projected/0a4aeb8e-a962-4aeb-9918-c75cdb61f704-kube-api-access-8v2qc\") pod \"0a4aeb8e-a962-4aeb-9918-c75cdb61f704\" (UID: \"0a4aeb8e-a962-4aeb-9918-c75cdb61f704\") "
Apr 24 14:40:53.272247 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:40:53.272198    2574 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"tokenizer-cache\" (UniqueName: \"kubernetes.io/empty-dir/0a4aeb8e-a962-4aeb-9918-c75cdb61f704-tokenizer-cache\") pod \"0a4aeb8e-a962-4aeb-9918-c75cdb61f704\" (UID: \"0a4aeb8e-a962-4aeb-9918-c75cdb61f704\") "
Apr 24 14:40:53.272572 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:40:53.272440    2574 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/0a4aeb8e-a962-4aeb-9918-c75cdb61f704-tokenizer-uds" (OuterVolumeSpecName: "tokenizer-uds") pod "0a4aeb8e-a962-4aeb-9918-c75cdb61f704" (UID: "0a4aeb8e-a962-4aeb-9918-c75cdb61f704"). InnerVolumeSpecName "tokenizer-uds". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Apr 24 14:40:53.272632 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:40:53.272562    2574 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/0a4aeb8e-a962-4aeb-9918-c75cdb61f704-tokenizer-tmp" (OuterVolumeSpecName: "tokenizer-tmp") pod "0a4aeb8e-a962-4aeb-9918-c75cdb61f704" (UID: "0a4aeb8e-a962-4aeb-9918-c75cdb61f704"). InnerVolumeSpecName "tokenizer-tmp". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Apr 24 14:40:53.272750 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:40:53.272731    2574 reconciler_common.go:299] "Volume detached for volume \"tokenizer-uds\" (UniqueName: \"kubernetes.io/empty-dir/0a4aeb8e-a962-4aeb-9918-c75cdb61f704-tokenizer-uds\") on node \"ip-10-0-136-132.ec2.internal\" DevicePath \"\""
Apr 24 14:40:53.272818 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:40:53.272755    2574 reconciler_common.go:299] "Volume detached for volume \"tokenizer-tmp\" (UniqueName: \"kubernetes.io/empty-dir/0a4aeb8e-a962-4aeb-9918-c75cdb61f704-tokenizer-tmp\") on node \"ip-10-0-136-132.ec2.internal\" DevicePath \"\""
Apr 24 14:40:53.272986 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:40:53.272913    2574 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/0a4aeb8e-a962-4aeb-9918-c75cdb61f704-tokenizer-cache" (OuterVolumeSpecName: "tokenizer-cache") pod "0a4aeb8e-a962-4aeb-9918-c75cdb61f704" (UID: "0a4aeb8e-a962-4aeb-9918-c75cdb61f704"). InnerVolumeSpecName "tokenizer-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Apr 24 14:40:53.272986 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:40:53.272953    2574 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/0a4aeb8e-a962-4aeb-9918-c75cdb61f704-kserve-provision-location" (OuterVolumeSpecName: "kserve-provision-location") pod "0a4aeb8e-a962-4aeb-9918-c75cdb61f704" (UID: "0a4aeb8e-a962-4aeb-9918-c75cdb61f704"). InnerVolumeSpecName "kserve-provision-location". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Apr 24 14:40:53.275137 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:40:53.275114    2574 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/0a4aeb8e-a962-4aeb-9918-c75cdb61f704-kube-api-access-8v2qc" (OuterVolumeSpecName: "kube-api-access-8v2qc") pod "0a4aeb8e-a962-4aeb-9918-c75cdb61f704" (UID: "0a4aeb8e-a962-4aeb-9918-c75cdb61f704"). InnerVolumeSpecName "kube-api-access-8v2qc". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Apr 24 14:40:53.275379 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:40:53.275302    2574 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/0a4aeb8e-a962-4aeb-9918-c75cdb61f704-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "0a4aeb8e-a962-4aeb-9918-c75cdb61f704" (UID: "0a4aeb8e-a962-4aeb-9918-c75cdb61f704"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Apr 24 14:40:53.374126 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:40:53.374084    2574 reconciler_common.go:299] "Volume detached for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/0a4aeb8e-a962-4aeb-9918-c75cdb61f704-kserve-provision-location\") on node \"ip-10-0-136-132.ec2.internal\" DevicePath \"\""
Apr 24 14:40:53.374126 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:40:53.374113    2574 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/0a4aeb8e-a962-4aeb-9918-c75cdb61f704-tls-certs\") on node \"ip-10-0-136-132.ec2.internal\" DevicePath \"\""
Apr 24 14:40:53.374126 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:40:53.374124    2574 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-8v2qc\" (UniqueName: \"kubernetes.io/projected/0a4aeb8e-a962-4aeb-9918-c75cdb61f704-kube-api-access-8v2qc\") on node \"ip-10-0-136-132.ec2.internal\" DevicePath \"\""
Apr 24 14:40:53.374126 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:40:53.374133    2574 reconciler_common.go:299] "Volume detached for volume \"tokenizer-cache\" (UniqueName: \"kubernetes.io/empty-dir/0a4aeb8e-a962-4aeb-9918-c75cdb61f704-tokenizer-cache\") on node \"ip-10-0-136-132.ec2.internal\" DevicePath \"\""
Apr 24 14:40:53.539823 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:40:53.539739    2574 generic.go:358] "Generic (PLEG): container finished" podID="0a4aeb8e-a962-4aeb-9918-c75cdb61f704" containerID="2a6de48338806c31f29f10aebdcc4545051f6e8677089d4a00eab3acbf351c1c" exitCode=0
Apr 24 14:40:53.539823 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:40:53.539811    2574 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/scheduler-ha-replicas-test-kserve-router-scheduler-fc6d477mptc7"
Apr 24 14:40:53.540050 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:40:53.539821    2574 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/scheduler-ha-replicas-test-kserve-router-scheduler-fc6d477mptc7" event={"ID":"0a4aeb8e-a962-4aeb-9918-c75cdb61f704","Type":"ContainerDied","Data":"2a6de48338806c31f29f10aebdcc4545051f6e8677089d4a00eab3acbf351c1c"}
Apr 24 14:40:53.540050 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:40:53.539857    2574 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/scheduler-ha-replicas-test-kserve-router-scheduler-fc6d477mptc7" event={"ID":"0a4aeb8e-a962-4aeb-9918-c75cdb61f704","Type":"ContainerDied","Data":"a630df757ba8f7951c249107045d02a967dd9ed4b0933faacc679ce945a7b3d7"}
Apr 24 14:40:53.540050 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:40:53.539877    2574 scope.go:117] "RemoveContainer" containerID="2a6de48338806c31f29f10aebdcc4545051f6e8677089d4a00eab3acbf351c1c"
Apr 24 14:40:53.550003 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:40:53.549986    2574 scope.go:117] "RemoveContainer" containerID="8e56c5a8e091156281b108bb2d06f84391ca732a5a987232ab699174561bd848"
Apr 24 14:40:53.559932 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:40:53.559912    2574 scope.go:117] "RemoveContainer" containerID="5da23df157b5cb65a4ca2ce297645d68c549b471d891989e7ce14a3cfafd7e1b"
Apr 24 14:40:53.562040 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:40:53.562016    2574 kubelet.go:2553] "SyncLoop DELETE" source="api" pods=["kserve-ci-e2e-test/scheduler-ha-replicas-test-kserve-router-scheduler-fc6d477mptc7"]
Apr 24 14:40:53.569661 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:40:53.569643    2574 scope.go:117] "RemoveContainer" containerID="2a6de48338806c31f29f10aebdcc4545051f6e8677089d4a00eab3acbf351c1c"
Apr 24 14:40:53.569909 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:40:53.569888    2574 kubelet.go:2547] "SyncLoop REMOVE" source="api" pods=["kserve-ci-e2e-test/scheduler-ha-replicas-test-kserve-router-scheduler-fc6d477mptc7"]
Apr 24 14:40:53.569978 ip-10-0-136-132 kubenswrapper[2574]: E0424 14:40:53.569939    2574 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"2a6de48338806c31f29f10aebdcc4545051f6e8677089d4a00eab3acbf351c1c\": container with ID starting with 2a6de48338806c31f29f10aebdcc4545051f6e8677089d4a00eab3acbf351c1c not found: ID does not exist" containerID="2a6de48338806c31f29f10aebdcc4545051f6e8677089d4a00eab3acbf351c1c"
Apr 24 14:40:53.569978 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:40:53.569960    2574 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"2a6de48338806c31f29f10aebdcc4545051f6e8677089d4a00eab3acbf351c1c"} err="failed to get container status \"2a6de48338806c31f29f10aebdcc4545051f6e8677089d4a00eab3acbf351c1c\": rpc error: code = NotFound desc = could not find container \"2a6de48338806c31f29f10aebdcc4545051f6e8677089d4a00eab3acbf351c1c\": container with ID starting with 2a6de48338806c31f29f10aebdcc4545051f6e8677089d4a00eab3acbf351c1c not found: ID does not exist"
Apr 24 14:40:53.569978 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:40:53.569976    2574 scope.go:117] "RemoveContainer" containerID="8e56c5a8e091156281b108bb2d06f84391ca732a5a987232ab699174561bd848"
Apr 24 14:40:53.570243 ip-10-0-136-132 kubenswrapper[2574]: E0424 14:40:53.570227    2574 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"8e56c5a8e091156281b108bb2d06f84391ca732a5a987232ab699174561bd848\": container with ID starting with 8e56c5a8e091156281b108bb2d06f84391ca732a5a987232ab699174561bd848 not found: ID does not exist" containerID="8e56c5a8e091156281b108bb2d06f84391ca732a5a987232ab699174561bd848"
Apr 24 14:40:53.570282 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:40:53.570246    2574 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"8e56c5a8e091156281b108bb2d06f84391ca732a5a987232ab699174561bd848"} err="failed to get container status \"8e56c5a8e091156281b108bb2d06f84391ca732a5a987232ab699174561bd848\": rpc error: code = NotFound desc = could not find container \"8e56c5a8e091156281b108bb2d06f84391ca732a5a987232ab699174561bd848\": container with ID starting with 8e56c5a8e091156281b108bb2d06f84391ca732a5a987232ab699174561bd848 not found: ID does not exist"
Apr 24 14:40:53.570282 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:40:53.570258    2574 scope.go:117] "RemoveContainer" containerID="5da23df157b5cb65a4ca2ce297645d68c549b471d891989e7ce14a3cfafd7e1b"
Apr 24 14:40:53.570518 ip-10-0-136-132 kubenswrapper[2574]: E0424 14:40:53.570500    2574 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"5da23df157b5cb65a4ca2ce297645d68c549b471d891989e7ce14a3cfafd7e1b\": container with ID starting with 5da23df157b5cb65a4ca2ce297645d68c549b471d891989e7ce14a3cfafd7e1b not found: ID does not exist" containerID="5da23df157b5cb65a4ca2ce297645d68c549b471d891989e7ce14a3cfafd7e1b"
Apr 24 14:40:53.570579 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:40:53.570531    2574 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"5da23df157b5cb65a4ca2ce297645d68c549b471d891989e7ce14a3cfafd7e1b"} err="failed to get container status \"5da23df157b5cb65a4ca2ce297645d68c549b471d891989e7ce14a3cfafd7e1b\": rpc error: code = NotFound desc = could not find container \"5da23df157b5cb65a4ca2ce297645d68c549b471d891989e7ce14a3cfafd7e1b\": container with ID starting with 5da23df157b5cb65a4ca2ce297645d68c549b471d891989e7ce14a3cfafd7e1b not found: ID does not exist"
Apr 24 14:40:53.773074 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:40:53.773032    2574 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="09a3eabb-dbcc-44c9-9a75-a348e3806278" path="/var/lib/kubelet/pods/09a3eabb-dbcc-44c9-9a75-a348e3806278/volumes"
Apr 24 14:40:53.773579 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:40:53.773555    2574 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="0a4aeb8e-a962-4aeb-9918-c75cdb61f704" path="/var/lib/kubelet/pods/0a4aeb8e-a962-4aeb-9918-c75cdb61f704/volumes"
Apr 24 14:41:00.518953 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:41:00.518923    2574 kubelet.go:2658] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="kserve-ci-e2e-test/precise-prefix-cache-test-kserve-router-scheduler-667fbc78lr57q"
Apr 24 14:41:00.518953 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:41:00.518958    2574 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve-ci-e2e-test/precise-prefix-cache-test-kserve-router-scheduler-667fbc78lr57q"
Apr 24 14:41:00.520204 ip-10-0-136-132 kubenswrapper[2574]: W0424 14:41:00.520172    2574 logging.go:55] [core] [Channel #62 SubChannel #63]grpc: addrConn.createTransport failed to connect to {Addr: "10.133.0.52:9003", ServerName: "10.133.0.52:9003", }. Err: connection error: desc = "transport: Error while dialing: dial tcp 10.133.0.52:9003: connect: connection refused"
Apr 24 14:41:00.521528 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:41:00.521510    2574 kubelet.go:2658] "SyncLoop (probe)" probe="startup" status="started" pod="kserve-ci-e2e-test/precise-prefix-cache-test-kserve-router-scheduler-667fbc78lr57q"
Apr 24 14:41:00.571534 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:41:00.571505    2574 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="kserve-ci-e2e-test/precise-prefix-cache-test-kserve-router-scheduler-667fbc78lr57q"
Apr 24 14:41:01.519787 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:41:01.519739    2574 prober.go:120] "Probe failed" probeType="Liveness" pod="kserve-ci-e2e-test/precise-prefix-cache-test-kserve-router-scheduler-667fbc78lr57q" podUID="803af3d5-3ffe-40cc-94fe-56ed1cd0b30d" containerName="main" probeResult="failure" output="timeout: failed to connect service \"10.133.0.52:9003\" within 1s: context deadline exceeded"
Apr 24 14:41:10.519108 ip-10-0-136-132 kubenswrapper[2574]: W0424 14:41:10.519077    2574 logging.go:55] [core] [Channel #64 SubChannel #65]grpc: addrConn.createTransport failed to connect to {Addr: "10.133.0.52:9003", ServerName: "10.133.0.52:9003", }. Err: connection error: desc = "transport: Error while dialing: dial tcp 10.133.0.52:9003: connect: connection refused"
Apr 24 14:41:11.519948 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:41:11.519897    2574 prober.go:120] "Probe failed" probeType="Liveness" pod="kserve-ci-e2e-test/precise-prefix-cache-test-kserve-router-scheduler-667fbc78lr57q" podUID="803af3d5-3ffe-40cc-94fe-56ed1cd0b30d" containerName="main" probeResult="failure" output="timeout: failed to connect service \"10.133.0.52:9003\" within 1s: context deadline exceeded"
Apr 24 14:41:11.520430 ip-10-0-136-132 kubenswrapper[2574]: W0424 14:41:11.520256    2574 logging.go:55] [core] [Channel #64 SubChannel #65]grpc: addrConn.createTransport failed to connect to {Addr: "10.133.0.52:9003", ServerName: "10.133.0.52:9003", }. Err: connection error: desc = "error reading server preface: read tcp 10.133.0.2:57322->10.133.0.52:9003: use of closed network connection"
Apr 24 14:41:20.654241 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:41:20.654205    2574 generic.go:358] "Generic (PLEG): container finished" podID="af01670c-533e-4c4f-935d-4a9f8a34a81d" containerID="54d8616ab70d8dc419c3359cd83eb8053047bb492cd5bae8e6d7d4fcba5d1394" exitCode=0
Apr 24 14:41:20.654669 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:41:20.654280    2574 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/precise-prefix-cache-test-kserve-75469f89c9-dfbfc" event={"ID":"af01670c-533e-4c4f-935d-4a9f8a34a81d","Type":"ContainerDied","Data":"54d8616ab70d8dc419c3359cd83eb8053047bb492cd5bae8e6d7d4fcba5d1394"}
Apr 24 14:41:21.575786 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:41:21.575758    2574 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="kserve-ci-e2e-test/precise-prefix-cache-test-kserve-router-scheduler-667fbc78lr57q"
Apr 24 14:41:21.660030 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:41:21.659998    2574 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/precise-prefix-cache-test-kserve-75469f89c9-dfbfc" event={"ID":"af01670c-533e-4c4f-935d-4a9f8a34a81d","Type":"ContainerStarted","Data":"bd879f828787b25e5ce9be8b00df2c1b45a0625f968e7c446230ad470c22f92a"}
Apr 24 14:41:21.678270 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:41:21.678225    2574 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="kserve-ci-e2e-test/precise-prefix-cache-test-kserve-75469f89c9-dfbfc" podStartSLOduration=32.678211313 podStartE2EDuration="32.678211313s" podCreationTimestamp="2026-04-24 14:40:49 +0000 UTC" firstStartedPulling="0001-01-01 00:00:00 +0000 UTC" lastFinishedPulling="0001-01-01 00:00:00 +0000 UTC" observedRunningTime="2026-04-24 14:41:21.675679074 +0000 UTC m=+1044.407780727" watchObservedRunningTime="2026-04-24 14:41:21.678211313 +0000 UTC m=+1044.410312961"
Apr 24 14:41:30.218516 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:41:30.218478    2574 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve-ci-e2e-test/precise-prefix-cache-test-kserve-75469f89c9-dfbfc"
Apr 24 14:41:30.218516 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:41:30.218520    2574 kubelet.go:2658] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="kserve-ci-e2e-test/precise-prefix-cache-test-kserve-75469f89c9-dfbfc"
Apr 24 14:41:30.231086 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:41:30.231059    2574 kubelet.go:2658] "SyncLoop (probe)" probe="startup" status="started" pod="kserve-ci-e2e-test/precise-prefix-cache-test-kserve-75469f89c9-dfbfc"
Apr 24 14:41:30.708069 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:41:30.708041    2574 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="kserve-ci-e2e-test/precise-prefix-cache-test-kserve-75469f89c9-dfbfc"
Apr 24 14:41:31.789412 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:41:31.789382    2574 kubelet.go:2553] "SyncLoop DELETE" source="api" pods=["kserve-ci-e2e-test/precise-prefix-cache-test-kserve-router-scheduler-667fbc78lr57q"]
Apr 24 14:41:31.789820 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:41:31.789672    2574 kuberuntime_container.go:864] "Killing container with a grace period" pod="kserve-ci-e2e-test/precise-prefix-cache-test-kserve-router-scheduler-667fbc78lr57q" podUID="803af3d5-3ffe-40cc-94fe-56ed1cd0b30d" containerName="main" containerID="cri-o://7839abf3e398cd5a4ec2d31b8f4ca1f668ce14c0f36cf90975027e29903d507b" gracePeriod=30
Apr 24 14:41:31.789820 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:41:31.789701    2574 kuberuntime_container.go:864] "Killing container with a grace period" pod="kserve-ci-e2e-test/precise-prefix-cache-test-kserve-router-scheduler-667fbc78lr57q" podUID="803af3d5-3ffe-40cc-94fe-56ed1cd0b30d" containerName="tokenizer" containerID="cri-o://72290df1390369ba24415bdb8a026783a801ff6d03d775307f0bcf1362d66d64" gracePeriod=30
Apr 24 14:41:31.800680 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:41:31.800642    2574 kubelet.go:2553] "SyncLoop DELETE" source="api" pods=["kserve-ci-e2e-test/precise-prefix-cache-test-kserve-75469f89c9-dfbfc"]
Apr 24 14:41:31.832047 ip-10-0-136-132 kubenswrapper[2574]: E0424 14:41:31.828556    2574 secret.go:189] Couldn't get secret kserve-ci-e2e-test/precise-prefix-cache-test-kserve-self-signed-certs: secret "precise-prefix-cache-test-kserve-self-signed-certs" not found
Apr 24 14:41:31.832047 ip-10-0-136-132 kubenswrapper[2574]: E0424 14:41:31.828638    2574 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/af01670c-533e-4c4f-935d-4a9f8a34a81d-tls-certs podName:af01670c-533e-4c4f-935d-4a9f8a34a81d nodeName:}" failed. No retries permitted until 2026-04-24 14:41:32.328618333 +0000 UTC m=+1055.060719962 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "tls-certs" (UniqueName: "kubernetes.io/secret/af01670c-533e-4c4f-935d-4a9f8a34a81d-tls-certs") pod "precise-prefix-cache-test-kserve-75469f89c9-dfbfc" (UID: "af01670c-533e-4c4f-935d-4a9f8a34a81d") : secret "precise-prefix-cache-test-kserve-self-signed-certs" not found
Apr 24 14:41:32.333241 ip-10-0-136-132 kubenswrapper[2574]: E0424 14:41:32.333205    2574 secret.go:189] Couldn't get secret kserve-ci-e2e-test/precise-prefix-cache-test-kserve-self-signed-certs: secret "precise-prefix-cache-test-kserve-self-signed-certs" not found
Apr 24 14:41:32.333482 ip-10-0-136-132 kubenswrapper[2574]: E0424 14:41:32.333284    2574 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/af01670c-533e-4c4f-935d-4a9f8a34a81d-tls-certs podName:af01670c-533e-4c4f-935d-4a9f8a34a81d nodeName:}" failed. No retries permitted until 2026-04-24 14:41:33.333265981 +0000 UTC m=+1056.065367608 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "tls-certs" (UniqueName: "kubernetes.io/secret/af01670c-533e-4c4f-935d-4a9f8a34a81d-tls-certs") pod "precise-prefix-cache-test-kserve-75469f89c9-dfbfc" (UID: "af01670c-533e-4c4f-935d-4a9f8a34a81d") : secret "precise-prefix-cache-test-kserve-self-signed-certs" not found
Apr 24 14:41:32.707246 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:41:32.707212    2574 generic.go:358] "Generic (PLEG): container finished" podID="803af3d5-3ffe-40cc-94fe-56ed1cd0b30d" containerID="7839abf3e398cd5a4ec2d31b8f4ca1f668ce14c0f36cf90975027e29903d507b" exitCode=0
Apr 24 14:41:32.707469 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:41:32.707289    2574 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/precise-prefix-cache-test-kserve-router-scheduler-667fbc78lr57q" event={"ID":"803af3d5-3ffe-40cc-94fe-56ed1cd0b30d","Type":"ContainerDied","Data":"7839abf3e398cd5a4ec2d31b8f4ca1f668ce14c0f36cf90975027e29903d507b"}
Apr 24 14:41:32.707649 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:41:32.707627    2574 kuberuntime_container.go:864] "Killing container with a grace period" pod="kserve-ci-e2e-test/precise-prefix-cache-test-kserve-75469f89c9-dfbfc" podUID="af01670c-533e-4c4f-935d-4a9f8a34a81d" containerName="main" containerID="cri-o://bd879f828787b25e5ce9be8b00df2c1b45a0625f968e7c446230ad470c22f92a" gracePeriod=30
Apr 24 14:41:32.962866 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:41:32.962808    2574 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/precise-prefix-cache-test-kserve-75469f89c9-dfbfc"
Apr 24 14:41:33.040285 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:41:33.040259    2574 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"kube-api-access-pxkpr\" (UniqueName: \"kubernetes.io/projected/af01670c-533e-4c4f-935d-4a9f8a34a81d-kube-api-access-pxkpr\") pod \"af01670c-533e-4c4f-935d-4a9f8a34a81d\" (UID: \"af01670c-533e-4c4f-935d-4a9f8a34a81d\") "
Apr 24 14:41:33.040439 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:41:33.040327    2574 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/af01670c-533e-4c4f-935d-4a9f8a34a81d-kserve-provision-location\") pod \"af01670c-533e-4c4f-935d-4a9f8a34a81d\" (UID: \"af01670c-533e-4c4f-935d-4a9f8a34a81d\") "
Apr 24 14:41:33.040524 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:41:33.040502    2574 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/af01670c-533e-4c4f-935d-4a9f8a34a81d-home\") pod \"af01670c-533e-4c4f-935d-4a9f8a34a81d\" (UID: \"af01670c-533e-4c4f-935d-4a9f8a34a81d\") "
Apr 24 14:41:33.040570 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:41:33.040554    2574 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/af01670c-533e-4c4f-935d-4a9f8a34a81d-dshm\") pod \"af01670c-533e-4c4f-935d-4a9f8a34a81d\" (UID: \"af01670c-533e-4c4f-935d-4a9f8a34a81d\") "
Apr 24 14:41:33.040628 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:41:33.040612    2574 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/af01670c-533e-4c4f-935d-4a9f8a34a81d-model-cache\") pod \"af01670c-533e-4c4f-935d-4a9f8a34a81d\" (UID: \"af01670c-533e-4c4f-935d-4a9f8a34a81d\") "
Apr 24 14:41:33.040753 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:41:33.040729    2574 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/af01670c-533e-4c4f-935d-4a9f8a34a81d-home" (OuterVolumeSpecName: "home") pod "af01670c-533e-4c4f-935d-4a9f8a34a81d" (UID: "af01670c-533e-4c4f-935d-4a9f8a34a81d"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Apr 24 14:41:33.040966 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:41:33.040902    2574 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/af01670c-533e-4c4f-935d-4a9f8a34a81d-model-cache" (OuterVolumeSpecName: "model-cache") pod "af01670c-533e-4c4f-935d-4a9f8a34a81d" (UID: "af01670c-533e-4c4f-935d-4a9f8a34a81d"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Apr 24 14:41:33.041146 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:41:33.040972    2574 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/af01670c-533e-4c4f-935d-4a9f8a34a81d-tls-certs\") pod \"af01670c-533e-4c4f-935d-4a9f8a34a81d\" (UID: \"af01670c-533e-4c4f-935d-4a9f8a34a81d\") "
Apr 24 14:41:33.041717 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:41:33.041699    2574 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/af01670c-533e-4c4f-935d-4a9f8a34a81d-model-cache\") on node \"ip-10-0-136-132.ec2.internal\" DevicePath \"\""
Apr 24 14:41:33.041851 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:41:33.041837    2574 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/af01670c-533e-4c4f-935d-4a9f8a34a81d-home\") on node \"ip-10-0-136-132.ec2.internal\" DevicePath \"\""
Apr 24 14:41:33.042790 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:41:33.042766    2574 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/af01670c-533e-4c4f-935d-4a9f8a34a81d-kube-api-access-pxkpr" (OuterVolumeSpecName: "kube-api-access-pxkpr") pod "af01670c-533e-4c4f-935d-4a9f8a34a81d" (UID: "af01670c-533e-4c4f-935d-4a9f8a34a81d"). InnerVolumeSpecName "kube-api-access-pxkpr". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Apr 24 14:41:33.042946 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:41:33.042924    2574 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/af01670c-533e-4c4f-935d-4a9f8a34a81d-dshm" (OuterVolumeSpecName: "dshm") pod "af01670c-533e-4c4f-935d-4a9f8a34a81d" (UID: "af01670c-533e-4c4f-935d-4a9f8a34a81d"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Apr 24 14:41:33.043239 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:41:33.043217    2574 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/af01670c-533e-4c4f-935d-4a9f8a34a81d-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "af01670c-533e-4c4f-935d-4a9f8a34a81d" (UID: "af01670c-533e-4c4f-935d-4a9f8a34a81d"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Apr 24 14:41:33.098743 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:41:33.098706    2574 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/af01670c-533e-4c4f-935d-4a9f8a34a81d-kserve-provision-location" (OuterVolumeSpecName: "kserve-provision-location") pod "af01670c-533e-4c4f-935d-4a9f8a34a81d" (UID: "af01670c-533e-4c4f-935d-4a9f8a34a81d"). InnerVolumeSpecName "kserve-provision-location". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Apr 24 14:41:33.142379 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:41:33.142345    2574 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/af01670c-533e-4c4f-935d-4a9f8a34a81d-tls-certs\") on node \"ip-10-0-136-132.ec2.internal\" DevicePath \"\""
Apr 24 14:41:33.142379 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:41:33.142378    2574 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-pxkpr\" (UniqueName: \"kubernetes.io/projected/af01670c-533e-4c4f-935d-4a9f8a34a81d-kube-api-access-pxkpr\") on node \"ip-10-0-136-132.ec2.internal\" DevicePath \"\""
Apr 24 14:41:33.142571 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:41:33.142390    2574 reconciler_common.go:299] "Volume detached for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/af01670c-533e-4c4f-935d-4a9f8a34a81d-kserve-provision-location\") on node \"ip-10-0-136-132.ec2.internal\" DevicePath \"\""
Apr 24 14:41:33.142571 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:41:33.142399    2574 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/af01670c-533e-4c4f-935d-4a9f8a34a81d-dshm\") on node \"ip-10-0-136-132.ec2.internal\" DevicePath \"\""
Apr 24 14:41:33.223620 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:41:33.223598    2574 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/precise-prefix-cache-test-kserve-router-scheduler-667fbc78lr57q"
Apr 24 14:41:33.343684 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:41:33.343655    2574 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"tokenizer-cache\" (UniqueName: \"kubernetes.io/empty-dir/803af3d5-3ffe-40cc-94fe-56ed1cd0b30d-tokenizer-cache\") pod \"803af3d5-3ffe-40cc-94fe-56ed1cd0b30d\" (UID: \"803af3d5-3ffe-40cc-94fe-56ed1cd0b30d\") "
Apr 24 14:41:33.343874 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:41:33.343702    2574 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/803af3d5-3ffe-40cc-94fe-56ed1cd0b30d-kserve-provision-location\") pod \"803af3d5-3ffe-40cc-94fe-56ed1cd0b30d\" (UID: \"803af3d5-3ffe-40cc-94fe-56ed1cd0b30d\") "
Apr 24 14:41:33.343874 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:41:33.343789    2574 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"tokenizer-tmp\" (UniqueName: \"kubernetes.io/empty-dir/803af3d5-3ffe-40cc-94fe-56ed1cd0b30d-tokenizer-tmp\") pod \"803af3d5-3ffe-40cc-94fe-56ed1cd0b30d\" (UID: \"803af3d5-3ffe-40cc-94fe-56ed1cd0b30d\") "
Apr 24 14:41:33.343874 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:41:33.343812    2574 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"tokenizer-uds\" (UniqueName: \"kubernetes.io/empty-dir/803af3d5-3ffe-40cc-94fe-56ed1cd0b30d-tokenizer-uds\") pod \"803af3d5-3ffe-40cc-94fe-56ed1cd0b30d\" (UID: \"803af3d5-3ffe-40cc-94fe-56ed1cd0b30d\") "
Apr 24 14:41:33.343874 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:41:33.343845    2574 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/803af3d5-3ffe-40cc-94fe-56ed1cd0b30d-tls-certs\") pod \"803af3d5-3ffe-40cc-94fe-56ed1cd0b30d\" (UID: \"803af3d5-3ffe-40cc-94fe-56ed1cd0b30d\") "
Apr 24 14:41:33.343874 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:41:33.343871    2574 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"kube-api-access-78vp2\" (UniqueName: \"kubernetes.io/projected/803af3d5-3ffe-40cc-94fe-56ed1cd0b30d-kube-api-access-78vp2\") pod \"803af3d5-3ffe-40cc-94fe-56ed1cd0b30d\" (UID: \"803af3d5-3ffe-40cc-94fe-56ed1cd0b30d\") "
Apr 24 14:41:33.344148 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:41:33.344116    2574 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/803af3d5-3ffe-40cc-94fe-56ed1cd0b30d-tokenizer-tmp" (OuterVolumeSpecName: "tokenizer-tmp") pod "803af3d5-3ffe-40cc-94fe-56ed1cd0b30d" (UID: "803af3d5-3ffe-40cc-94fe-56ed1cd0b30d"). InnerVolumeSpecName "tokenizer-tmp". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Apr 24 14:41:33.344214 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:41:33.344138    2574 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/803af3d5-3ffe-40cc-94fe-56ed1cd0b30d-tokenizer-uds" (OuterVolumeSpecName: "tokenizer-uds") pod "803af3d5-3ffe-40cc-94fe-56ed1cd0b30d" (UID: "803af3d5-3ffe-40cc-94fe-56ed1cd0b30d"). InnerVolumeSpecName "tokenizer-uds". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Apr 24 14:41:33.344214 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:41:33.344049    2574 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/803af3d5-3ffe-40cc-94fe-56ed1cd0b30d-tokenizer-cache" (OuterVolumeSpecName: "tokenizer-cache") pod "803af3d5-3ffe-40cc-94fe-56ed1cd0b30d" (UID: "803af3d5-3ffe-40cc-94fe-56ed1cd0b30d"). InnerVolumeSpecName "tokenizer-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Apr 24 14:41:33.344603 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:41:33.344579    2574 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/803af3d5-3ffe-40cc-94fe-56ed1cd0b30d-kserve-provision-location" (OuterVolumeSpecName: "kserve-provision-location") pod "803af3d5-3ffe-40cc-94fe-56ed1cd0b30d" (UID: "803af3d5-3ffe-40cc-94fe-56ed1cd0b30d"). InnerVolumeSpecName "kserve-provision-location". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Apr 24 14:41:33.345962 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:41:33.345945    2574 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/803af3d5-3ffe-40cc-94fe-56ed1cd0b30d-kube-api-access-78vp2" (OuterVolumeSpecName: "kube-api-access-78vp2") pod "803af3d5-3ffe-40cc-94fe-56ed1cd0b30d" (UID: "803af3d5-3ffe-40cc-94fe-56ed1cd0b30d"). InnerVolumeSpecName "kube-api-access-78vp2". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Apr 24 14:41:33.346026 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:41:33.345963    2574 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/803af3d5-3ffe-40cc-94fe-56ed1cd0b30d-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "803af3d5-3ffe-40cc-94fe-56ed1cd0b30d" (UID: "803af3d5-3ffe-40cc-94fe-56ed1cd0b30d"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Apr 24 14:41:33.445426 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:41:33.445399    2574 reconciler_common.go:299] "Volume detached for volume \"tokenizer-tmp\" (UniqueName: \"kubernetes.io/empty-dir/803af3d5-3ffe-40cc-94fe-56ed1cd0b30d-tokenizer-tmp\") on node \"ip-10-0-136-132.ec2.internal\" DevicePath \"\""
Apr 24 14:41:33.445426 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:41:33.445422    2574 reconciler_common.go:299] "Volume detached for volume \"tokenizer-uds\" (UniqueName: \"kubernetes.io/empty-dir/803af3d5-3ffe-40cc-94fe-56ed1cd0b30d-tokenizer-uds\") on node \"ip-10-0-136-132.ec2.internal\" DevicePath \"\""
Apr 24 14:41:33.445582 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:41:33.445432    2574 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/803af3d5-3ffe-40cc-94fe-56ed1cd0b30d-tls-certs\") on node \"ip-10-0-136-132.ec2.internal\" DevicePath \"\""
Apr 24 14:41:33.445582 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:41:33.445442    2574 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-78vp2\" (UniqueName: \"kubernetes.io/projected/803af3d5-3ffe-40cc-94fe-56ed1cd0b30d-kube-api-access-78vp2\") on node \"ip-10-0-136-132.ec2.internal\" DevicePath \"\""
Apr 24 14:41:33.445582 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:41:33.445452    2574 reconciler_common.go:299] "Volume detached for volume \"tokenizer-cache\" (UniqueName: \"kubernetes.io/empty-dir/803af3d5-3ffe-40cc-94fe-56ed1cd0b30d-tokenizer-cache\") on node \"ip-10-0-136-132.ec2.internal\" DevicePath \"\""
Apr 24 14:41:33.445582 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:41:33.445463    2574 reconciler_common.go:299] "Volume detached for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/803af3d5-3ffe-40cc-94fe-56ed1cd0b30d-kserve-provision-location\") on node \"ip-10-0-136-132.ec2.internal\" DevicePath \"\""
Apr 24 14:41:33.714298 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:41:33.714261    2574 generic.go:358] "Generic (PLEG): container finished" podID="af01670c-533e-4c4f-935d-4a9f8a34a81d" containerID="bd879f828787b25e5ce9be8b00df2c1b45a0625f968e7c446230ad470c22f92a" exitCode=0
Apr 24 14:41:33.714496 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:41:33.714364    2574 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/precise-prefix-cache-test-kserve-75469f89c9-dfbfc"
Apr 24 14:41:33.714496 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:41:33.714358    2574 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/precise-prefix-cache-test-kserve-75469f89c9-dfbfc" event={"ID":"af01670c-533e-4c4f-935d-4a9f8a34a81d","Type":"ContainerDied","Data":"bd879f828787b25e5ce9be8b00df2c1b45a0625f968e7c446230ad470c22f92a"}
Apr 24 14:41:33.714496 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:41:33.714488    2574 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/precise-prefix-cache-test-kserve-75469f89c9-dfbfc" event={"ID":"af01670c-533e-4c4f-935d-4a9f8a34a81d","Type":"ContainerDied","Data":"8eef4dae57a3a9e9492e2578672b0373aa7732bba8993c40910c368a17d65eff"}
Apr 24 14:41:33.714678 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:41:33.714514    2574 scope.go:117] "RemoveContainer" containerID="bd879f828787b25e5ce9be8b00df2c1b45a0625f968e7c446230ad470c22f92a"
Apr 24 14:41:33.716024 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:41:33.715999    2574 generic.go:358] "Generic (PLEG): container finished" podID="803af3d5-3ffe-40cc-94fe-56ed1cd0b30d" containerID="72290df1390369ba24415bdb8a026783a801ff6d03d775307f0bcf1362d66d64" exitCode=0
Apr 24 14:41:33.716139 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:41:33.716062    2574 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/precise-prefix-cache-test-kserve-router-scheduler-667fbc78lr57q" event={"ID":"803af3d5-3ffe-40cc-94fe-56ed1cd0b30d","Type":"ContainerDied","Data":"72290df1390369ba24415bdb8a026783a801ff6d03d775307f0bcf1362d66d64"}
Apr 24 14:41:33.716139 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:41:33.716085    2574 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/precise-prefix-cache-test-kserve-router-scheduler-667fbc78lr57q" event={"ID":"803af3d5-3ffe-40cc-94fe-56ed1cd0b30d","Type":"ContainerDied","Data":"deeebf6f716cc2603547c785d9be6632427c35109ea2e6a37cbf01d25b2db121"}
Apr 24 14:41:33.716139 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:41:33.716088    2574 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/precise-prefix-cache-test-kserve-router-scheduler-667fbc78lr57q"
Apr 24 14:41:33.725261 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:41:33.725235    2574 scope.go:117] "RemoveContainer" containerID="54d8616ab70d8dc419c3359cd83eb8053047bb492cd5bae8e6d7d4fcba5d1394"
Apr 24 14:41:33.741255 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:41:33.741228    2574 kubelet.go:2553] "SyncLoop DELETE" source="api" pods=["kserve-ci-e2e-test/precise-prefix-cache-test-kserve-75469f89c9-dfbfc"]
Apr 24 14:41:33.745005 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:41:33.744982    2574 kubelet.go:2547] "SyncLoop REMOVE" source="api" pods=["kserve-ci-e2e-test/precise-prefix-cache-test-kserve-75469f89c9-dfbfc"]
Apr 24 14:41:33.753928 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:41:33.753906    2574 kubelet.go:2553] "SyncLoop DELETE" source="api" pods=["kserve-ci-e2e-test/precise-prefix-cache-test-kserve-router-scheduler-667fbc78lr57q"]
Apr 24 14:41:33.759386 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:41:33.759365    2574 kubelet.go:2547] "SyncLoop REMOVE" source="api" pods=["kserve-ci-e2e-test/precise-prefix-cache-test-kserve-router-scheduler-667fbc78lr57q"]
Apr 24 14:41:33.770998 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:41:33.770976    2574 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="803af3d5-3ffe-40cc-94fe-56ed1cd0b30d" path="/var/lib/kubelet/pods/803af3d5-3ffe-40cc-94fe-56ed1cd0b30d/volumes"
Apr 24 14:41:33.771458 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:41:33.771444    2574 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="af01670c-533e-4c4f-935d-4a9f8a34a81d" path="/var/lib/kubelet/pods/af01670c-533e-4c4f-935d-4a9f8a34a81d/volumes"
Apr 24 14:41:33.792723 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:41:33.792706    2574 scope.go:117] "RemoveContainer" containerID="bd879f828787b25e5ce9be8b00df2c1b45a0625f968e7c446230ad470c22f92a"
Apr 24 14:41:33.793005 ip-10-0-136-132 kubenswrapper[2574]: E0424 14:41:33.792982    2574 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"bd879f828787b25e5ce9be8b00df2c1b45a0625f968e7c446230ad470c22f92a\": container with ID starting with bd879f828787b25e5ce9be8b00df2c1b45a0625f968e7c446230ad470c22f92a not found: ID does not exist" containerID="bd879f828787b25e5ce9be8b00df2c1b45a0625f968e7c446230ad470c22f92a"
Apr 24 14:41:33.793100 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:41:33.793012    2574 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"bd879f828787b25e5ce9be8b00df2c1b45a0625f968e7c446230ad470c22f92a"} err="failed to get container status \"bd879f828787b25e5ce9be8b00df2c1b45a0625f968e7c446230ad470c22f92a\": rpc error: code = NotFound desc = could not find container \"bd879f828787b25e5ce9be8b00df2c1b45a0625f968e7c446230ad470c22f92a\": container with ID starting with bd879f828787b25e5ce9be8b00df2c1b45a0625f968e7c446230ad470c22f92a not found: ID does not exist"
Apr 24 14:41:33.793100 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:41:33.793032    2574 scope.go:117] "RemoveContainer" containerID="54d8616ab70d8dc419c3359cd83eb8053047bb492cd5bae8e6d7d4fcba5d1394"
Apr 24 14:41:33.793319 ip-10-0-136-132 kubenswrapper[2574]: E0424 14:41:33.793304    2574 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"54d8616ab70d8dc419c3359cd83eb8053047bb492cd5bae8e6d7d4fcba5d1394\": container with ID starting with 54d8616ab70d8dc419c3359cd83eb8053047bb492cd5bae8e6d7d4fcba5d1394 not found: ID does not exist" containerID="54d8616ab70d8dc419c3359cd83eb8053047bb492cd5bae8e6d7d4fcba5d1394"
Apr 24 14:41:33.793396 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:41:33.793322    2574 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"54d8616ab70d8dc419c3359cd83eb8053047bb492cd5bae8e6d7d4fcba5d1394"} err="failed to get container status \"54d8616ab70d8dc419c3359cd83eb8053047bb492cd5bae8e6d7d4fcba5d1394\": rpc error: code = NotFound desc = could not find container \"54d8616ab70d8dc419c3359cd83eb8053047bb492cd5bae8e6d7d4fcba5d1394\": container with ID starting with 54d8616ab70d8dc419c3359cd83eb8053047bb492cd5bae8e6d7d4fcba5d1394 not found: ID does not exist"
Apr 24 14:41:33.793396 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:41:33.793353    2574 scope.go:117] "RemoveContainer" containerID="72290df1390369ba24415bdb8a026783a801ff6d03d775307f0bcf1362d66d64"
Apr 24 14:41:33.801248 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:41:33.801226    2574 scope.go:117] "RemoveContainer" containerID="7839abf3e398cd5a4ec2d31b8f4ca1f668ce14c0f36cf90975027e29903d507b"
Apr 24 14:41:33.808570 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:41:33.808555    2574 scope.go:117] "RemoveContainer" containerID="2a425b3e51a584bc36971dfab9bdb878c40a8ebdb62965099c7ad649866ef1d7"
Apr 24 14:41:33.815745 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:41:33.815730    2574 scope.go:117] "RemoveContainer" containerID="72290df1390369ba24415bdb8a026783a801ff6d03d775307f0bcf1362d66d64"
Apr 24 14:41:33.815983 ip-10-0-136-132 kubenswrapper[2574]: E0424 14:41:33.815964    2574 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"72290df1390369ba24415bdb8a026783a801ff6d03d775307f0bcf1362d66d64\": container with ID starting with 72290df1390369ba24415bdb8a026783a801ff6d03d775307f0bcf1362d66d64 not found: ID does not exist" containerID="72290df1390369ba24415bdb8a026783a801ff6d03d775307f0bcf1362d66d64"
Apr 24 14:41:33.816042 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:41:33.815988    2574 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"72290df1390369ba24415bdb8a026783a801ff6d03d775307f0bcf1362d66d64"} err="failed to get container status \"72290df1390369ba24415bdb8a026783a801ff6d03d775307f0bcf1362d66d64\": rpc error: code = NotFound desc = could not find container \"72290df1390369ba24415bdb8a026783a801ff6d03d775307f0bcf1362d66d64\": container with ID starting with 72290df1390369ba24415bdb8a026783a801ff6d03d775307f0bcf1362d66d64 not found: ID does not exist"
Apr 24 14:41:33.816042 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:41:33.816011    2574 scope.go:117] "RemoveContainer" containerID="7839abf3e398cd5a4ec2d31b8f4ca1f668ce14c0f36cf90975027e29903d507b"
Apr 24 14:41:33.816262 ip-10-0-136-132 kubenswrapper[2574]: E0424 14:41:33.816244    2574 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"7839abf3e398cd5a4ec2d31b8f4ca1f668ce14c0f36cf90975027e29903d507b\": container with ID starting with 7839abf3e398cd5a4ec2d31b8f4ca1f668ce14c0f36cf90975027e29903d507b not found: ID does not exist" containerID="7839abf3e398cd5a4ec2d31b8f4ca1f668ce14c0f36cf90975027e29903d507b"
Apr 24 14:41:33.816299 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:41:33.816274    2574 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"7839abf3e398cd5a4ec2d31b8f4ca1f668ce14c0f36cf90975027e29903d507b"} err="failed to get container status \"7839abf3e398cd5a4ec2d31b8f4ca1f668ce14c0f36cf90975027e29903d507b\": rpc error: code = NotFound desc = could not find container \"7839abf3e398cd5a4ec2d31b8f4ca1f668ce14c0f36cf90975027e29903d507b\": container with ID starting with 7839abf3e398cd5a4ec2d31b8f4ca1f668ce14c0f36cf90975027e29903d507b not found: ID does not exist"
Apr 24 14:41:33.816299 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:41:33.816296    2574 scope.go:117] "RemoveContainer" containerID="2a425b3e51a584bc36971dfab9bdb878c40a8ebdb62965099c7ad649866ef1d7"
Apr 24 14:41:33.819241 ip-10-0-136-132 kubenswrapper[2574]: E0424 14:41:33.819212    2574 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"2a425b3e51a584bc36971dfab9bdb878c40a8ebdb62965099c7ad649866ef1d7\": container with ID starting with 2a425b3e51a584bc36971dfab9bdb878c40a8ebdb62965099c7ad649866ef1d7 not found: ID does not exist" containerID="2a425b3e51a584bc36971dfab9bdb878c40a8ebdb62965099c7ad649866ef1d7"
Apr 24 14:41:33.819440 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:41:33.819243    2574 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"2a425b3e51a584bc36971dfab9bdb878c40a8ebdb62965099c7ad649866ef1d7"} err="failed to get container status \"2a425b3e51a584bc36971dfab9bdb878c40a8ebdb62965099c7ad649866ef1d7\": rpc error: code = NotFound desc = could not find container \"2a425b3e51a584bc36971dfab9bdb878c40a8ebdb62965099c7ad649866ef1d7\": container with ID starting with 2a425b3e51a584bc36971dfab9bdb878c40a8ebdb62965099c7ad649866ef1d7 not found: ID does not exist"
Apr 24 14:41:37.292950 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:41:37.292916    2574 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["kserve-ci-e2e-test/stop-feature-test-kserve-88fb48646-cq9c6"]
Apr 24 14:41:37.293543 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:41:37.293354    2574 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="0a4aeb8e-a962-4aeb-9918-c75cdb61f704" containerName="storage-initializer"
Apr 24 14:41:37.293543 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:41:37.293367    2574 state_mem.go:107] "Deleted CPUSet assignment" podUID="0a4aeb8e-a962-4aeb-9918-c75cdb61f704" containerName="storage-initializer"
Apr 24 14:41:37.293543 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:41:37.293375    2574 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="0a4aeb8e-a962-4aeb-9918-c75cdb61f704" containerName="tokenizer"
Apr 24 14:41:37.293543 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:41:37.293380    2574 state_mem.go:107] "Deleted CPUSet assignment" podUID="0a4aeb8e-a962-4aeb-9918-c75cdb61f704" containerName="tokenizer"
Apr 24 14:41:37.293543 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:41:37.293394    2574 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="803af3d5-3ffe-40cc-94fe-56ed1cd0b30d" containerName="storage-initializer"
Apr 24 14:41:37.293543 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:41:37.293399    2574 state_mem.go:107] "Deleted CPUSet assignment" podUID="803af3d5-3ffe-40cc-94fe-56ed1cd0b30d" containerName="storage-initializer"
Apr 24 14:41:37.293543 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:41:37.293405    2574 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="803af3d5-3ffe-40cc-94fe-56ed1cd0b30d" containerName="main"
Apr 24 14:41:37.293543 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:41:37.293410    2574 state_mem.go:107] "Deleted CPUSet assignment" podUID="803af3d5-3ffe-40cc-94fe-56ed1cd0b30d" containerName="main"
Apr 24 14:41:37.293543 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:41:37.293420    2574 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="af01670c-533e-4c4f-935d-4a9f8a34a81d" containerName="storage-initializer"
Apr 24 14:41:37.293543 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:41:37.293425    2574 state_mem.go:107] "Deleted CPUSet assignment" podUID="af01670c-533e-4c4f-935d-4a9f8a34a81d" containerName="storage-initializer"
Apr 24 14:41:37.293543 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:41:37.293435    2574 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="0a4aeb8e-a962-4aeb-9918-c75cdb61f704" containerName="main"
Apr 24 14:41:37.293543 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:41:37.293440    2574 state_mem.go:107] "Deleted CPUSet assignment" podUID="0a4aeb8e-a962-4aeb-9918-c75cdb61f704" containerName="main"
Apr 24 14:41:37.293543 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:41:37.293445    2574 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="803af3d5-3ffe-40cc-94fe-56ed1cd0b30d" containerName="tokenizer"
Apr 24 14:41:37.293543 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:41:37.293449    2574 state_mem.go:107] "Deleted CPUSet assignment" podUID="803af3d5-3ffe-40cc-94fe-56ed1cd0b30d" containerName="tokenizer"
Apr 24 14:41:37.293543 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:41:37.293454    2574 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="af01670c-533e-4c4f-935d-4a9f8a34a81d" containerName="main"
Apr 24 14:41:37.293543 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:41:37.293459    2574 state_mem.go:107] "Deleted CPUSet assignment" podUID="af01670c-533e-4c4f-935d-4a9f8a34a81d" containerName="main"
Apr 24 14:41:37.293543 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:41:37.293475    2574 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="09a3eabb-dbcc-44c9-9a75-a348e3806278" containerName="storage-initializer"
Apr 24 14:41:37.293543 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:41:37.293480    2574 state_mem.go:107] "Deleted CPUSet assignment" podUID="09a3eabb-dbcc-44c9-9a75-a348e3806278" containerName="storage-initializer"
Apr 24 14:41:37.293543 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:41:37.293485    2574 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="09a3eabb-dbcc-44c9-9a75-a348e3806278" containerName="main"
Apr 24 14:41:37.293543 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:41:37.293490    2574 state_mem.go:107] "Deleted CPUSet assignment" podUID="09a3eabb-dbcc-44c9-9a75-a348e3806278" containerName="main"
Apr 24 14:41:37.293543 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:41:37.293545    2574 memory_manager.go:356] "RemoveStaleState removing state" podUID="803af3d5-3ffe-40cc-94fe-56ed1cd0b30d" containerName="main"
Apr 24 14:41:37.293543 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:41:37.293553    2574 memory_manager.go:356] "RemoveStaleState removing state" podUID="0a4aeb8e-a962-4aeb-9918-c75cdb61f704" containerName="main"
Apr 24 14:41:37.294691 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:41:37.293563    2574 memory_manager.go:356] "RemoveStaleState removing state" podUID="af01670c-533e-4c4f-935d-4a9f8a34a81d" containerName="main"
Apr 24 14:41:37.294691 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:41:37.293573    2574 memory_manager.go:356] "RemoveStaleState removing state" podUID="0a4aeb8e-a962-4aeb-9918-c75cdb61f704" containerName="tokenizer"
Apr 24 14:41:37.294691 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:41:37.293579    2574 memory_manager.go:356] "RemoveStaleState removing state" podUID="803af3d5-3ffe-40cc-94fe-56ed1cd0b30d" containerName="tokenizer"
Apr 24 14:41:37.294691 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:41:37.293585    2574 memory_manager.go:356] "RemoveStaleState removing state" podUID="09a3eabb-dbcc-44c9-9a75-a348e3806278" containerName="main"
Apr 24 14:41:37.299111 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:41:37.299090    2574 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/stop-feature-test-kserve-88fb48646-cq9c6"
Apr 24 14:41:37.301811 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:41:37.301748    2574 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"kserve-ci-e2e-test\"/\"default-dockercfg-92j8s\""
Apr 24 14:41:37.301811 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:41:37.301784    2574 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"kserve-ci-e2e-test\"/\"stop-feature-test-kserve-self-signed-certs\""
Apr 24 14:41:37.302690 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:41:37.302376    2574 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kserve-ci-e2e-test\"/\"openshift-service-ca.crt\""
Apr 24 14:41:37.302690 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:41:37.302666    2574 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kserve-ci-e2e-test\"/\"kube-root-ca.crt\""
Apr 24 14:41:37.306249 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:41:37.306175    2574 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/stop-feature-test-kserve-88fb48646-cq9c6"]
Apr 24 14:41:37.380883 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:41:37.380842    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/b9251f52-2109-4405-8651-0a0f450355d5-kserve-provision-location\") pod \"stop-feature-test-kserve-88fb48646-cq9c6\" (UID: \"b9251f52-2109-4405-8651-0a0f450355d5\") " pod="kserve-ci-e2e-test/stop-feature-test-kserve-88fb48646-cq9c6"
Apr 24 14:41:37.381065 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:41:37.380895    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/b9251f52-2109-4405-8651-0a0f450355d5-home\") pod \"stop-feature-test-kserve-88fb48646-cq9c6\" (UID: \"b9251f52-2109-4405-8651-0a0f450355d5\") " pod="kserve-ci-e2e-test/stop-feature-test-kserve-88fb48646-cq9c6"
Apr 24 14:41:37.381065 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:41:37.380923    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/b9251f52-2109-4405-8651-0a0f450355d5-model-cache\") pod \"stop-feature-test-kserve-88fb48646-cq9c6\" (UID: \"b9251f52-2109-4405-8651-0a0f450355d5\") " pod="kserve-ci-e2e-test/stop-feature-test-kserve-88fb48646-cq9c6"
Apr 24 14:41:37.381065 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:41:37.380948    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/b9251f52-2109-4405-8651-0a0f450355d5-dshm\") pod \"stop-feature-test-kserve-88fb48646-cq9c6\" (UID: \"b9251f52-2109-4405-8651-0a0f450355d5\") " pod="kserve-ci-e2e-test/stop-feature-test-kserve-88fb48646-cq9c6"
Apr 24 14:41:37.381065 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:41:37.380971    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/b9251f52-2109-4405-8651-0a0f450355d5-tls-certs\") pod \"stop-feature-test-kserve-88fb48646-cq9c6\" (UID: \"b9251f52-2109-4405-8651-0a0f450355d5\") " pod="kserve-ci-e2e-test/stop-feature-test-kserve-88fb48646-cq9c6"
Apr 24 14:41:37.381213 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:41:37.381062    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7kdqx\" (UniqueName: \"kubernetes.io/projected/b9251f52-2109-4405-8651-0a0f450355d5-kube-api-access-7kdqx\") pod \"stop-feature-test-kserve-88fb48646-cq9c6\" (UID: \"b9251f52-2109-4405-8651-0a0f450355d5\") " pod="kserve-ci-e2e-test/stop-feature-test-kserve-88fb48646-cq9c6"
Apr 24 14:41:37.482329 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:41:37.482295    2574 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/b9251f52-2109-4405-8651-0a0f450355d5-kserve-provision-location\") pod \"stop-feature-test-kserve-88fb48646-cq9c6\" (UID: \"b9251f52-2109-4405-8651-0a0f450355d5\") " pod="kserve-ci-e2e-test/stop-feature-test-kserve-88fb48646-cq9c6"
Apr 24 14:41:37.482542 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:41:37.482361    2574 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/b9251f52-2109-4405-8651-0a0f450355d5-home\") pod \"stop-feature-test-kserve-88fb48646-cq9c6\" (UID: \"b9251f52-2109-4405-8651-0a0f450355d5\") " pod="kserve-ci-e2e-test/stop-feature-test-kserve-88fb48646-cq9c6"
Apr 24 14:41:37.482542 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:41:37.482386    2574 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/b9251f52-2109-4405-8651-0a0f450355d5-model-cache\") pod \"stop-feature-test-kserve-88fb48646-cq9c6\" (UID: \"b9251f52-2109-4405-8651-0a0f450355d5\") " pod="kserve-ci-e2e-test/stop-feature-test-kserve-88fb48646-cq9c6"
Apr 24 14:41:37.482542 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:41:37.482408    2574 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/b9251f52-2109-4405-8651-0a0f450355d5-dshm\") pod \"stop-feature-test-kserve-88fb48646-cq9c6\" (UID: \"b9251f52-2109-4405-8651-0a0f450355d5\") " pod="kserve-ci-e2e-test/stop-feature-test-kserve-88fb48646-cq9c6"
Apr 24 14:41:37.482542 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:41:37.482423    2574 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/b9251f52-2109-4405-8651-0a0f450355d5-tls-certs\") pod \"stop-feature-test-kserve-88fb48646-cq9c6\" (UID: \"b9251f52-2109-4405-8651-0a0f450355d5\") " pod="kserve-ci-e2e-test/stop-feature-test-kserve-88fb48646-cq9c6"
Apr 24 14:41:37.482542 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:41:37.482459    2574 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-7kdqx\" (UniqueName: \"kubernetes.io/projected/b9251f52-2109-4405-8651-0a0f450355d5-kube-api-access-7kdqx\") pod \"stop-feature-test-kserve-88fb48646-cq9c6\" (UID: \"b9251f52-2109-4405-8651-0a0f450355d5\") " pod="kserve-ci-e2e-test/stop-feature-test-kserve-88fb48646-cq9c6"
Apr 24 14:41:37.482806 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:41:37.482719    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/b9251f52-2109-4405-8651-0a0f450355d5-kserve-provision-location\") pod \"stop-feature-test-kserve-88fb48646-cq9c6\" (UID: \"b9251f52-2109-4405-8651-0a0f450355d5\") " pod="kserve-ci-e2e-test/stop-feature-test-kserve-88fb48646-cq9c6"
Apr 24 14:41:37.482806 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:41:37.482784    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/b9251f52-2109-4405-8651-0a0f450355d5-model-cache\") pod \"stop-feature-test-kserve-88fb48646-cq9c6\" (UID: \"b9251f52-2109-4405-8651-0a0f450355d5\") " pod="kserve-ci-e2e-test/stop-feature-test-kserve-88fb48646-cq9c6"
Apr 24 14:41:37.482905 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:41:37.482834    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/b9251f52-2109-4405-8651-0a0f450355d5-home\") pod \"stop-feature-test-kserve-88fb48646-cq9c6\" (UID: \"b9251f52-2109-4405-8651-0a0f450355d5\") " pod="kserve-ci-e2e-test/stop-feature-test-kserve-88fb48646-cq9c6"
Apr 24 14:41:37.484757 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:41:37.484738    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/b9251f52-2109-4405-8651-0a0f450355d5-dshm\") pod \"stop-feature-test-kserve-88fb48646-cq9c6\" (UID: \"b9251f52-2109-4405-8651-0a0f450355d5\") " pod="kserve-ci-e2e-test/stop-feature-test-kserve-88fb48646-cq9c6"
Apr 24 14:41:37.484928 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:41:37.484912    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/b9251f52-2109-4405-8651-0a0f450355d5-tls-certs\") pod \"stop-feature-test-kserve-88fb48646-cq9c6\" (UID: \"b9251f52-2109-4405-8651-0a0f450355d5\") " pod="kserve-ci-e2e-test/stop-feature-test-kserve-88fb48646-cq9c6"
Apr 24 14:41:37.489436 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:41:37.489415    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-7kdqx\" (UniqueName: \"kubernetes.io/projected/b9251f52-2109-4405-8651-0a0f450355d5-kube-api-access-7kdqx\") pod \"stop-feature-test-kserve-88fb48646-cq9c6\" (UID: \"b9251f52-2109-4405-8651-0a0f450355d5\") " pod="kserve-ci-e2e-test/stop-feature-test-kserve-88fb48646-cq9c6"
Apr 24 14:41:37.525190 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:41:37.525159    2574 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["kserve-ci-e2e-test/stop-feature-test-kserve-router-scheduler-7cb4fb9c5d-vb62h"]
Apr 24 14:41:37.529316 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:41:37.529298    2574 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/stop-feature-test-kserve-router-scheduler-7cb4fb9c5d-vb62h"
Apr 24 14:41:37.531303 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:41:37.531277    2574 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"kserve-ci-e2e-test\"/\"stop-feature-test-epp-sa-dockercfg-t2hx4\""
Apr 24 14:41:37.538573 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:41:37.538551    2574 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/stop-feature-test-kserve-router-scheduler-7cb4fb9c5d-vb62h"]
Apr 24 14:41:37.611552 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:41:37.611474    2574 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/stop-feature-test-kserve-88fb48646-cq9c6"
Apr 24 14:41:37.684520 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:41:37.684489    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tokenizer-cache\" (UniqueName: \"kubernetes.io/empty-dir/e56da355-84e2-4f4e-b186-d92c4584f202-tokenizer-cache\") pod \"stop-feature-test-kserve-router-scheduler-7cb4fb9c5d-vb62h\" (UID: \"e56da355-84e2-4f4e-b186-d92c4584f202\") " pod="kserve-ci-e2e-test/stop-feature-test-kserve-router-scheduler-7cb4fb9c5d-vb62h"
Apr 24 14:41:37.684673 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:41:37.684547    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-tbl8r\" (UniqueName: \"kubernetes.io/projected/e56da355-84e2-4f4e-b186-d92c4584f202-kube-api-access-tbl8r\") pod \"stop-feature-test-kserve-router-scheduler-7cb4fb9c5d-vb62h\" (UID: \"e56da355-84e2-4f4e-b186-d92c4584f202\") " pod="kserve-ci-e2e-test/stop-feature-test-kserve-router-scheduler-7cb4fb9c5d-vb62h"
Apr 24 14:41:37.684673 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:41:37.684636    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/e56da355-84e2-4f4e-b186-d92c4584f202-tls-certs\") pod \"stop-feature-test-kserve-router-scheduler-7cb4fb9c5d-vb62h\" (UID: \"e56da355-84e2-4f4e-b186-d92c4584f202\") " pod="kserve-ci-e2e-test/stop-feature-test-kserve-router-scheduler-7cb4fb9c5d-vb62h"
Apr 24 14:41:37.684749 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:41:37.684711    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tokenizer-tmp\" (UniqueName: \"kubernetes.io/empty-dir/e56da355-84e2-4f4e-b186-d92c4584f202-tokenizer-tmp\") pod \"stop-feature-test-kserve-router-scheduler-7cb4fb9c5d-vb62h\" (UID: \"e56da355-84e2-4f4e-b186-d92c4584f202\") " pod="kserve-ci-e2e-test/stop-feature-test-kserve-router-scheduler-7cb4fb9c5d-vb62h"
Apr 24 14:41:37.684781 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:41:37.684750    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/e56da355-84e2-4f4e-b186-d92c4584f202-kserve-provision-location\") pod \"stop-feature-test-kserve-router-scheduler-7cb4fb9c5d-vb62h\" (UID: \"e56da355-84e2-4f4e-b186-d92c4584f202\") " pod="kserve-ci-e2e-test/stop-feature-test-kserve-router-scheduler-7cb4fb9c5d-vb62h"
Apr 24 14:41:37.684818 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:41:37.684798    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tokenizer-uds\" (UniqueName: \"kubernetes.io/empty-dir/e56da355-84e2-4f4e-b186-d92c4584f202-tokenizer-uds\") pod \"stop-feature-test-kserve-router-scheduler-7cb4fb9c5d-vb62h\" (UID: \"e56da355-84e2-4f4e-b186-d92c4584f202\") " pod="kserve-ci-e2e-test/stop-feature-test-kserve-router-scheduler-7cb4fb9c5d-vb62h"
Apr 24 14:41:37.734104 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:41:37.734075    2574 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/stop-feature-test-kserve-88fb48646-cq9c6"]
Apr 24 14:41:37.735484 ip-10-0-136-132 kubenswrapper[2574]: W0424 14:41:37.735456    2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb9251f52_2109_4405_8651_0a0f450355d5.slice/crio-ec35a851c26f1a5127008873945de8189db3e6cc37849fa5525348840e1de676 WatchSource:0}: Error finding container ec35a851c26f1a5127008873945de8189db3e6cc37849fa5525348840e1de676: Status 404 returned error can't find the container with id ec35a851c26f1a5127008873945de8189db3e6cc37849fa5525348840e1de676
Apr 24 14:41:37.786136 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:41:37.786112    2574 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"tokenizer-tmp\" (UniqueName: \"kubernetes.io/empty-dir/e56da355-84e2-4f4e-b186-d92c4584f202-tokenizer-tmp\") pod \"stop-feature-test-kserve-router-scheduler-7cb4fb9c5d-vb62h\" (UID: \"e56da355-84e2-4f4e-b186-d92c4584f202\") " pod="kserve-ci-e2e-test/stop-feature-test-kserve-router-scheduler-7cb4fb9c5d-vb62h"
Apr 24 14:41:37.786235 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:41:37.786147    2574 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/e56da355-84e2-4f4e-b186-d92c4584f202-kserve-provision-location\") pod \"stop-feature-test-kserve-router-scheduler-7cb4fb9c5d-vb62h\" (UID: \"e56da355-84e2-4f4e-b186-d92c4584f202\") " pod="kserve-ci-e2e-test/stop-feature-test-kserve-router-scheduler-7cb4fb9c5d-vb62h"
Apr 24 14:41:37.786235 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:41:37.786176    2574 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"tokenizer-uds\" (UniqueName: \"kubernetes.io/empty-dir/e56da355-84e2-4f4e-b186-d92c4584f202-tokenizer-uds\") pod \"stop-feature-test-kserve-router-scheduler-7cb4fb9c5d-vb62h\" (UID: \"e56da355-84e2-4f4e-b186-d92c4584f202\") " pod="kserve-ci-e2e-test/stop-feature-test-kserve-router-scheduler-7cb4fb9c5d-vb62h"
Apr 24 14:41:37.786366 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:41:37.786242    2574 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"tokenizer-cache\" (UniqueName: \"kubernetes.io/empty-dir/e56da355-84e2-4f4e-b186-d92c4584f202-tokenizer-cache\") pod \"stop-feature-test-kserve-router-scheduler-7cb4fb9c5d-vb62h\" (UID: \"e56da355-84e2-4f4e-b186-d92c4584f202\") " pod="kserve-ci-e2e-test/stop-feature-test-kserve-router-scheduler-7cb4fb9c5d-vb62h"
Apr 24 14:41:37.786366 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:41:37.786308    2574 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-tbl8r\" (UniqueName: \"kubernetes.io/projected/e56da355-84e2-4f4e-b186-d92c4584f202-kube-api-access-tbl8r\") pod \"stop-feature-test-kserve-router-scheduler-7cb4fb9c5d-vb62h\" (UID: \"e56da355-84e2-4f4e-b186-d92c4584f202\") " pod="kserve-ci-e2e-test/stop-feature-test-kserve-router-scheduler-7cb4fb9c5d-vb62h"
Apr 24 14:41:37.786482 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:41:37.786451    2574 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/e56da355-84e2-4f4e-b186-d92c4584f202-tls-certs\") pod \"stop-feature-test-kserve-router-scheduler-7cb4fb9c5d-vb62h\" (UID: \"e56da355-84e2-4f4e-b186-d92c4584f202\") " pod="kserve-ci-e2e-test/stop-feature-test-kserve-router-scheduler-7cb4fb9c5d-vb62h"
Apr 24 14:41:37.786539 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:41:37.786516    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tokenizer-tmp\" (UniqueName: \"kubernetes.io/empty-dir/e56da355-84e2-4f4e-b186-d92c4584f202-tokenizer-tmp\") pod \"stop-feature-test-kserve-router-scheduler-7cb4fb9c5d-vb62h\" (UID: \"e56da355-84e2-4f4e-b186-d92c4584f202\") " pod="kserve-ci-e2e-test/stop-feature-test-kserve-router-scheduler-7cb4fb9c5d-vb62h"
Apr 24 14:41:37.786610 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:41:37.786555    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/e56da355-84e2-4f4e-b186-d92c4584f202-kserve-provision-location\") pod \"stop-feature-test-kserve-router-scheduler-7cb4fb9c5d-vb62h\" (UID: \"e56da355-84e2-4f4e-b186-d92c4584f202\") " pod="kserve-ci-e2e-test/stop-feature-test-kserve-router-scheduler-7cb4fb9c5d-vb62h"
Apr 24 14:41:37.786677 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:41:37.786616    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tokenizer-uds\" (UniqueName: \"kubernetes.io/empty-dir/e56da355-84e2-4f4e-b186-d92c4584f202-tokenizer-uds\") pod \"stop-feature-test-kserve-router-scheduler-7cb4fb9c5d-vb62h\" (UID: \"e56da355-84e2-4f4e-b186-d92c4584f202\") " pod="kserve-ci-e2e-test/stop-feature-test-kserve-router-scheduler-7cb4fb9c5d-vb62h"
Apr 24 14:41:37.786677 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:41:37.786624    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tokenizer-cache\" (UniqueName: \"kubernetes.io/empty-dir/e56da355-84e2-4f4e-b186-d92c4584f202-tokenizer-cache\") pod \"stop-feature-test-kserve-router-scheduler-7cb4fb9c5d-vb62h\" (UID: \"e56da355-84e2-4f4e-b186-d92c4584f202\") " pod="kserve-ci-e2e-test/stop-feature-test-kserve-router-scheduler-7cb4fb9c5d-vb62h"
Apr 24 14:41:37.788722 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:41:37.788693    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/e56da355-84e2-4f4e-b186-d92c4584f202-tls-certs\") pod \"stop-feature-test-kserve-router-scheduler-7cb4fb9c5d-vb62h\" (UID: \"e56da355-84e2-4f4e-b186-d92c4584f202\") " pod="kserve-ci-e2e-test/stop-feature-test-kserve-router-scheduler-7cb4fb9c5d-vb62h"
Apr 24 14:41:37.793456 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:41:37.793436    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-tbl8r\" (UniqueName: \"kubernetes.io/projected/e56da355-84e2-4f4e-b186-d92c4584f202-kube-api-access-tbl8r\") pod \"stop-feature-test-kserve-router-scheduler-7cb4fb9c5d-vb62h\" (UID: \"e56da355-84e2-4f4e-b186-d92c4584f202\") " pod="kserve-ci-e2e-test/stop-feature-test-kserve-router-scheduler-7cb4fb9c5d-vb62h"
Apr 24 14:41:37.840365 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:41:37.840265    2574 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/stop-feature-test-kserve-router-scheduler-7cb4fb9c5d-vb62h"
Apr 24 14:41:37.981818 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:41:37.981791    2574 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/stop-feature-test-kserve-router-scheduler-7cb4fb9c5d-vb62h"]
Apr 24 14:41:37.983518 ip-10-0-136-132 kubenswrapper[2574]: W0424 14:41:37.983487    2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode56da355_84e2_4f4e_b186_d92c4584f202.slice/crio-54a340a63d015da8bcd6572ecc47fb45c816cac7ffc3f107c3d67d7984717d79 WatchSource:0}: Error finding container 54a340a63d015da8bcd6572ecc47fb45c816cac7ffc3f107c3d67d7984717d79: Status 404 returned error can't find the container with id 54a340a63d015da8bcd6572ecc47fb45c816cac7ffc3f107c3d67d7984717d79
Apr 24 14:41:38.741619 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:41:38.741578    2574 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/stop-feature-test-kserve-88fb48646-cq9c6" event={"ID":"b9251f52-2109-4405-8651-0a0f450355d5","Type":"ContainerStarted","Data":"e22f33838bdab274b88160de954942ddcbbb8e2d8d0e9589d1eeb3ccacaab0f7"}
Apr 24 14:41:38.741619 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:41:38.741624    2574 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/stop-feature-test-kserve-88fb48646-cq9c6" event={"ID":"b9251f52-2109-4405-8651-0a0f450355d5","Type":"ContainerStarted","Data":"ec35a851c26f1a5127008873945de8189db3e6cc37849fa5525348840e1de676"}
Apr 24 14:41:38.743681 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:41:38.743649    2574 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/stop-feature-test-kserve-router-scheduler-7cb4fb9c5d-vb62h" event={"ID":"e56da355-84e2-4f4e-b186-d92c4584f202","Type":"ContainerStarted","Data":"e96bbb3ee1cee17cdb19ffc1297f10a9e9b934a0df99f9c0a32cdd66bc6e8a2f"}
Apr 24 14:41:38.743843 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:41:38.743685    2574 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/stop-feature-test-kserve-router-scheduler-7cb4fb9c5d-vb62h" event={"ID":"e56da355-84e2-4f4e-b186-d92c4584f202","Type":"ContainerStarted","Data":"54a340a63d015da8bcd6572ecc47fb45c816cac7ffc3f107c3d67d7984717d79"}
Apr 24 14:41:39.749230 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:41:39.749193    2574 generic.go:358] "Generic (PLEG): container finished" podID="e56da355-84e2-4f4e-b186-d92c4584f202" containerID="e96bbb3ee1cee17cdb19ffc1297f10a9e9b934a0df99f9c0a32cdd66bc6e8a2f" exitCode=0
Apr 24 14:41:39.749789 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:41:39.749259    2574 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/stop-feature-test-kserve-router-scheduler-7cb4fb9c5d-vb62h" event={"ID":"e56da355-84e2-4f4e-b186-d92c4584f202","Type":"ContainerDied","Data":"e96bbb3ee1cee17cdb19ffc1297f10a9e9b934a0df99f9c0a32cdd66bc6e8a2f"}
Apr 24 14:41:40.757829 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:41:40.757741    2574 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/stop-feature-test-kserve-router-scheduler-7cb4fb9c5d-vb62h" event={"ID":"e56da355-84e2-4f4e-b186-d92c4584f202","Type":"ContainerStarted","Data":"edfbf99073d0f175fba20e38bb3811622d96fb3a848cb33b6ad992b5064c1ae6"}
Apr 24 14:41:40.758225 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:41:40.757825    2574 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/stop-feature-test-kserve-router-scheduler-7cb4fb9c5d-vb62h" event={"ID":"e56da355-84e2-4f4e-b186-d92c4584f202","Type":"ContainerStarted","Data":"8e36313c2ca96dbbb25ec4d91c0c77e451d27abdaee08bcdb81490dd1da5eb58"}
Apr 24 14:41:40.758225 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:41:40.758118    2574 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve-ci-e2e-test/stop-feature-test-kserve-router-scheduler-7cb4fb9c5d-vb62h"
Apr 24 14:41:40.782637 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:41:40.782575    2574 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="kserve-ci-e2e-test/stop-feature-test-kserve-router-scheduler-7cb4fb9c5d-vb62h" podStartSLOduration=3.782554898 podStartE2EDuration="3.782554898s" podCreationTimestamp="2026-04-24 14:41:37 +0000 UTC" firstStartedPulling="0001-01-01 00:00:00 +0000 UTC" lastFinishedPulling="0001-01-01 00:00:00 +0000 UTC" observedRunningTime="2026-04-24 14:41:40.778114528 +0000 UTC m=+1063.510216179" watchObservedRunningTime="2026-04-24 14:41:40.782554898 +0000 UTC m=+1063.514656550"
Apr 24 14:41:42.767534 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:41:42.767492    2574 generic.go:358] "Generic (PLEG): container finished" podID="b9251f52-2109-4405-8651-0a0f450355d5" containerID="e22f33838bdab274b88160de954942ddcbbb8e2d8d0e9589d1eeb3ccacaab0f7" exitCode=0
Apr 24 14:41:42.767909 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:41:42.767567    2574 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/stop-feature-test-kserve-88fb48646-cq9c6" event={"ID":"b9251f52-2109-4405-8651-0a0f450355d5","Type":"ContainerDied","Data":"e22f33838bdab274b88160de954942ddcbbb8e2d8d0e9589d1eeb3ccacaab0f7"}
Apr 24 14:41:47.840828 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:41:47.840793    2574 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve-ci-e2e-test/stop-feature-test-kserve-router-scheduler-7cb4fb9c5d-vb62h"
Apr 24 14:41:47.841292 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:41:47.840840    2574 kubelet.go:2658] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="kserve-ci-e2e-test/stop-feature-test-kserve-router-scheduler-7cb4fb9c5d-vb62h"
Apr 24 14:41:47.842297 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:41:47.842266    2574 prober.go:120] "Probe failed" probeType="Startup" pod="kserve-ci-e2e-test/stop-feature-test-kserve-router-scheduler-7cb4fb9c5d-vb62h" podUID="e56da355-84e2-4f4e-b186-d92c4584f202" containerName="tokenizer" probeResult="failure" output="Get \"http://10.133.0.54:8082/healthz\": dial tcp 10.133.0.54:8082: connect: connection refused"
Apr 24 14:41:57.842426 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:41:57.842367    2574 kubelet.go:2658] "SyncLoop (probe)" probe="startup" status="started" pod="kserve-ci-e2e-test/stop-feature-test-kserve-router-scheduler-7cb4fb9c5d-vb62h"
Apr 24 14:41:57.843684 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:41:57.843663    2574 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="kserve-ci-e2e-test/stop-feature-test-kserve-router-scheduler-7cb4fb9c5d-vb62h"
Apr 24 14:42:18.844127 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:42:18.844096    2574 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="kserve-ci-e2e-test/stop-feature-test-kserve-router-scheduler-7cb4fb9c5d-vb62h"
Apr 24 14:42:24.958440 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:42:24.958394    2574 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/stop-feature-test-kserve-88fb48646-cq9c6" event={"ID":"b9251f52-2109-4405-8651-0a0f450355d5","Type":"ContainerStarted","Data":"1185fbcac517f3f1a066a2111f2ed31485629deab077361f48e4c461e9799f09"}
Apr 24 14:42:24.978868 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:42:24.978812    2574 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="kserve-ci-e2e-test/stop-feature-test-kserve-88fb48646-cq9c6" podStartSLOduration=6.475541846 podStartE2EDuration="47.978771592s" podCreationTimestamp="2026-04-24 14:41:37 +0000 UTC" firstStartedPulling="2026-04-24 14:41:42.768763048 +0000 UTC m=+1065.500864675" lastFinishedPulling="2026-04-24 14:42:24.271992791 +0000 UTC m=+1107.004094421" observedRunningTime="2026-04-24 14:42:24.978278206 +0000 UTC m=+1107.710379869" watchObservedRunningTime="2026-04-24 14:42:24.978771592 +0000 UTC m=+1107.710873246"
Apr 24 14:42:27.611608 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:42:27.611565    2574 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve-ci-e2e-test/stop-feature-test-kserve-88fb48646-cq9c6"
Apr 24 14:42:27.611608 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:42:27.611613    2574 kubelet.go:2658] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="kserve-ci-e2e-test/stop-feature-test-kserve-88fb48646-cq9c6"
Apr 24 14:42:27.613194 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:42:27.613156    2574 prober.go:120] "Probe failed" probeType="Startup" pod="kserve-ci-e2e-test/stop-feature-test-kserve-88fb48646-cq9c6" podUID="b9251f52-2109-4405-8651-0a0f450355d5" containerName="main" probeResult="failure" output="Get \"https://10.133.0.53:8000/health\": dial tcp 10.133.0.53:8000: connect: connection refused"
Apr 24 14:42:37.612582 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:42:37.612542    2574 prober.go:120] "Probe failed" probeType="Startup" pod="kserve-ci-e2e-test/stop-feature-test-kserve-88fb48646-cq9c6" podUID="b9251f52-2109-4405-8651-0a0f450355d5" containerName="main" probeResult="failure" output="Get \"https://10.133.0.53:8000/health\": dial tcp 10.133.0.53:8000: connect: connection refused"
Apr 24 14:42:47.612305 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:42:47.612239    2574 prober.go:120] "Probe failed" probeType="Startup" pod="kserve-ci-e2e-test/stop-feature-test-kserve-88fb48646-cq9c6" podUID="b9251f52-2109-4405-8651-0a0f450355d5" containerName="main" probeResult="failure" output="Get \"https://10.133.0.53:8000/health\": dial tcp 10.133.0.53:8000: connect: connection refused"
Apr 24 14:42:57.612414 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:42:57.612371    2574 prober.go:120] "Probe failed" probeType="Startup" pod="kserve-ci-e2e-test/stop-feature-test-kserve-88fb48646-cq9c6" podUID="b9251f52-2109-4405-8651-0a0f450355d5" containerName="main" probeResult="failure" output="Get \"https://10.133.0.53:8000/health\": dial tcp 10.133.0.53:8000: connect: connection refused"
Apr 24 14:43:07.612591 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:43:07.612552    2574 prober.go:120] "Probe failed" probeType="Startup" pod="kserve-ci-e2e-test/stop-feature-test-kserve-88fb48646-cq9c6" podUID="b9251f52-2109-4405-8651-0a0f450355d5" containerName="main" probeResult="failure" output="Get \"https://10.133.0.53:8000/health\": dial tcp 10.133.0.53:8000: connect: connection refused"
Apr 24 14:43:17.612692 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:43:17.612612    2574 prober.go:120] "Probe failed" probeType="Startup" pod="kserve-ci-e2e-test/stop-feature-test-kserve-88fb48646-cq9c6" podUID="b9251f52-2109-4405-8651-0a0f450355d5" containerName="main" probeResult="failure" output="Get \"https://10.133.0.53:8000/health\": dial tcp 10.133.0.53:8000: connect: connection refused"
Apr 24 14:43:27.611949 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:43:27.611905    2574 prober.go:120] "Probe failed" probeType="Startup" pod="kserve-ci-e2e-test/stop-feature-test-kserve-88fb48646-cq9c6" podUID="b9251f52-2109-4405-8651-0a0f450355d5" containerName="main" probeResult="failure" output="Get \"https://10.133.0.53:8000/health\": dial tcp 10.133.0.53:8000: connect: connection refused"
Apr 24 14:43:37.612554 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:43:37.612510    2574 prober.go:120] "Probe failed" probeType="Startup" pod="kserve-ci-e2e-test/stop-feature-test-kserve-88fb48646-cq9c6" podUID="b9251f52-2109-4405-8651-0a0f450355d5" containerName="main" probeResult="failure" output="Get \"https://10.133.0.53:8000/health\": dial tcp 10.133.0.53:8000: connect: connection refused"
Apr 24 14:43:47.612067 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:43:47.612025    2574 prober.go:120] "Probe failed" probeType="Startup" pod="kserve-ci-e2e-test/stop-feature-test-kserve-88fb48646-cq9c6" podUID="b9251f52-2109-4405-8651-0a0f450355d5" containerName="main" probeResult="failure" output="Get \"https://10.133.0.53:8000/health\": dial tcp 10.133.0.53:8000: connect: connection refused"
Apr 24 14:43:57.621787 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:43:57.621742    2574 kubelet.go:2658] "SyncLoop (probe)" probe="startup" status="started" pod="kserve-ci-e2e-test/stop-feature-test-kserve-88fb48646-cq9c6"
Apr 24 14:43:57.629572 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:43:57.629548    2574 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="kserve-ci-e2e-test/stop-feature-test-kserve-88fb48646-cq9c6"
Apr 24 14:43:57.799442 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:43:57.799415    2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-9d4b6777b-f6ngr_2571829f-666a-422a-9b02-2d2bb4c89b25/console-operator/1.log"
Apr 24 14:43:57.802350 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:43:57.802318    2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-9d4b6777b-f6ngr_2571829f-666a-422a-9b02-2d2bb4c89b25/console-operator/1.log"
Apr 24 14:43:57.804370 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:43:57.804348    2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-w5x5x_44036c3e-1fdc-40d4-899c-9009cd4589e3/ovn-acl-logging/0.log"
Apr 24 14:43:57.807414 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:43:57.807392    2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-w5x5x_44036c3e-1fdc-40d4-899c-9009cd4589e3/ovn-acl-logging/0.log"
Apr 24 14:43:59.060557 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:43:59.060525    2574 kubelet.go:2553] "SyncLoop DELETE" source="api" pods=["kserve-ci-e2e-test/stop-feature-test-kserve-88fb48646-cq9c6"]
Apr 24 14:43:59.149318 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:43:59.149246    2574 kubelet.go:2553] "SyncLoop DELETE" source="api" pods=["kserve-ci-e2e-test/stop-feature-test-kserve-router-scheduler-7cb4fb9c5d-vb62h"]
Apr 24 14:43:59.149709 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:43:59.149661    2574 kuberuntime_container.go:864] "Killing container with a grace period" pod="kserve-ci-e2e-test/stop-feature-test-kserve-router-scheduler-7cb4fb9c5d-vb62h" podUID="e56da355-84e2-4f4e-b186-d92c4584f202" containerName="main" containerID="cri-o://8e36313c2ca96dbbb25ec4d91c0c77e451d27abdaee08bcdb81490dd1da5eb58" gracePeriod=30
Apr 24 14:43:59.149871 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:43:59.149702    2574 kuberuntime_container.go:864] "Killing container with a grace period" pod="kserve-ci-e2e-test/stop-feature-test-kserve-router-scheduler-7cb4fb9c5d-vb62h" podUID="e56da355-84e2-4f4e-b186-d92c4584f202" containerName="tokenizer" containerID="cri-o://edfbf99073d0f175fba20e38bb3811622d96fb3a848cb33b6ad992b5064c1ae6" gracePeriod=30
Apr 24 14:43:59.359854 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:43:59.359761    2574 generic.go:358] "Generic (PLEG): container finished" podID="e56da355-84e2-4f4e-b186-d92c4584f202" containerID="8e36313c2ca96dbbb25ec4d91c0c77e451d27abdaee08bcdb81490dd1da5eb58" exitCode=0
Apr 24 14:43:59.360451 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:43:59.360362    2574 kuberuntime_container.go:864] "Killing container with a grace period" pod="kserve-ci-e2e-test/stop-feature-test-kserve-88fb48646-cq9c6" podUID="b9251f52-2109-4405-8651-0a0f450355d5" containerName="main" containerID="cri-o://1185fbcac517f3f1a066a2111f2ed31485629deab077361f48e4c461e9799f09" gracePeriod=30
Apr 24 14:43:59.360451 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:43:59.360423    2574 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/stop-feature-test-kserve-router-scheduler-7cb4fb9c5d-vb62h" event={"ID":"e56da355-84e2-4f4e-b186-d92c4584f202","Type":"ContainerDied","Data":"8e36313c2ca96dbbb25ec4d91c0c77e451d27abdaee08bcdb81490dd1da5eb58"}
Apr 24 14:44:00.281369 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:44:00.281327    2574 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/stop-feature-test-kserve-router-scheduler-7cb4fb9c5d-vb62h"
Apr 24 14:44:00.365172 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:44:00.365083    2574 generic.go:358] "Generic (PLEG): container finished" podID="e56da355-84e2-4f4e-b186-d92c4584f202" containerID="edfbf99073d0f175fba20e38bb3811622d96fb3a848cb33b6ad992b5064c1ae6" exitCode=0
Apr 24 14:44:00.365329 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:44:00.365173    2574 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/stop-feature-test-kserve-router-scheduler-7cb4fb9c5d-vb62h"
Apr 24 14:44:00.365329 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:44:00.365168    2574 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/stop-feature-test-kserve-router-scheduler-7cb4fb9c5d-vb62h" event={"ID":"e56da355-84e2-4f4e-b186-d92c4584f202","Type":"ContainerDied","Data":"edfbf99073d0f175fba20e38bb3811622d96fb3a848cb33b6ad992b5064c1ae6"}
Apr 24 14:44:00.365329 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:44:00.365291    2574 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/stop-feature-test-kserve-router-scheduler-7cb4fb9c5d-vb62h" event={"ID":"e56da355-84e2-4f4e-b186-d92c4584f202","Type":"ContainerDied","Data":"54a340a63d015da8bcd6572ecc47fb45c816cac7ffc3f107c3d67d7984717d79"}
Apr 24 14:44:00.365329 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:44:00.365310    2574 scope.go:117] "RemoveContainer" containerID="edfbf99073d0f175fba20e38bb3811622d96fb3a848cb33b6ad992b5064c1ae6"
Apr 24 14:44:00.374011 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:44:00.373993    2574 scope.go:117] "RemoveContainer" containerID="8e36313c2ca96dbbb25ec4d91c0c77e451d27abdaee08bcdb81490dd1da5eb58"
Apr 24 14:44:00.381574 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:44:00.381558    2574 scope.go:117] "RemoveContainer" containerID="e96bbb3ee1cee17cdb19ffc1297f10a9e9b934a0df99f9c0a32cdd66bc6e8a2f"
Apr 24 14:44:00.389083 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:44:00.389068    2574 scope.go:117] "RemoveContainer" containerID="edfbf99073d0f175fba20e38bb3811622d96fb3a848cb33b6ad992b5064c1ae6"
Apr 24 14:44:00.389350 ip-10-0-136-132 kubenswrapper[2574]: E0424 14:44:00.389318    2574 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"edfbf99073d0f175fba20e38bb3811622d96fb3a848cb33b6ad992b5064c1ae6\": container with ID starting with edfbf99073d0f175fba20e38bb3811622d96fb3a848cb33b6ad992b5064c1ae6 not found: ID does not exist" containerID="edfbf99073d0f175fba20e38bb3811622d96fb3a848cb33b6ad992b5064c1ae6"
Apr 24 14:44:00.389451 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:44:00.389429    2574 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"edfbf99073d0f175fba20e38bb3811622d96fb3a848cb33b6ad992b5064c1ae6"} err="failed to get container status \"edfbf99073d0f175fba20e38bb3811622d96fb3a848cb33b6ad992b5064c1ae6\": rpc error: code = NotFound desc = could not find container \"edfbf99073d0f175fba20e38bb3811622d96fb3a848cb33b6ad992b5064c1ae6\": container with ID starting with edfbf99073d0f175fba20e38bb3811622d96fb3a848cb33b6ad992b5064c1ae6 not found: ID does not exist"
Apr 24 14:44:00.389499 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:44:00.389453    2574 scope.go:117] "RemoveContainer" containerID="8e36313c2ca96dbbb25ec4d91c0c77e451d27abdaee08bcdb81490dd1da5eb58"
Apr 24 14:44:00.389700 ip-10-0-136-132 kubenswrapper[2574]: E0424 14:44:00.389687    2574 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"8e36313c2ca96dbbb25ec4d91c0c77e451d27abdaee08bcdb81490dd1da5eb58\": container with ID starting with 8e36313c2ca96dbbb25ec4d91c0c77e451d27abdaee08bcdb81490dd1da5eb58 not found: ID does not exist" containerID="8e36313c2ca96dbbb25ec4d91c0c77e451d27abdaee08bcdb81490dd1da5eb58"
Apr 24 14:44:00.389751 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:44:00.389702    2574 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"8e36313c2ca96dbbb25ec4d91c0c77e451d27abdaee08bcdb81490dd1da5eb58"} err="failed to get container status \"8e36313c2ca96dbbb25ec4d91c0c77e451d27abdaee08bcdb81490dd1da5eb58\": rpc error: code = NotFound desc = could not find container \"8e36313c2ca96dbbb25ec4d91c0c77e451d27abdaee08bcdb81490dd1da5eb58\": container with ID starting with 8e36313c2ca96dbbb25ec4d91c0c77e451d27abdaee08bcdb81490dd1da5eb58 not found: ID does not exist"
Apr 24 14:44:00.389751 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:44:00.389715    2574 scope.go:117] "RemoveContainer" containerID="e96bbb3ee1cee17cdb19ffc1297f10a9e9b934a0df99f9c0a32cdd66bc6e8a2f"
Apr 24 14:44:00.389960 ip-10-0-136-132 kubenswrapper[2574]: E0424 14:44:00.389944    2574 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"e96bbb3ee1cee17cdb19ffc1297f10a9e9b934a0df99f9c0a32cdd66bc6e8a2f\": container with ID starting with e96bbb3ee1cee17cdb19ffc1297f10a9e9b934a0df99f9c0a32cdd66bc6e8a2f not found: ID does not exist" containerID="e96bbb3ee1cee17cdb19ffc1297f10a9e9b934a0df99f9c0a32cdd66bc6e8a2f"
Apr 24 14:44:00.390003 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:44:00.389965    2574 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e96bbb3ee1cee17cdb19ffc1297f10a9e9b934a0df99f9c0a32cdd66bc6e8a2f"} err="failed to get container status \"e96bbb3ee1cee17cdb19ffc1297f10a9e9b934a0df99f9c0a32cdd66bc6e8a2f\": rpc error: code = NotFound desc = could not find container \"e96bbb3ee1cee17cdb19ffc1297f10a9e9b934a0df99f9c0a32cdd66bc6e8a2f\": container with ID starting with e96bbb3ee1cee17cdb19ffc1297f10a9e9b934a0df99f9c0a32cdd66bc6e8a2f not found: ID does not exist"
Apr 24 14:44:00.397252 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:44:00.397232    2574 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/e56da355-84e2-4f4e-b186-d92c4584f202-kserve-provision-location\") pod \"e56da355-84e2-4f4e-b186-d92c4584f202\" (UID: \"e56da355-84e2-4f4e-b186-d92c4584f202\") "
Apr 24 14:44:00.397320 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:44:00.397271    2574 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"kube-api-access-tbl8r\" (UniqueName: \"kubernetes.io/projected/e56da355-84e2-4f4e-b186-d92c4584f202-kube-api-access-tbl8r\") pod \"e56da355-84e2-4f4e-b186-d92c4584f202\" (UID: \"e56da355-84e2-4f4e-b186-d92c4584f202\") "
Apr 24 14:44:00.397381 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:44:00.397318    2574 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/e56da355-84e2-4f4e-b186-d92c4584f202-tls-certs\") pod \"e56da355-84e2-4f4e-b186-d92c4584f202\" (UID: \"e56da355-84e2-4f4e-b186-d92c4584f202\") "
Apr 24 14:44:00.397381 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:44:00.397370    2574 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"tokenizer-tmp\" (UniqueName: \"kubernetes.io/empty-dir/e56da355-84e2-4f4e-b186-d92c4584f202-tokenizer-tmp\") pod \"e56da355-84e2-4f4e-b186-d92c4584f202\" (UID: \"e56da355-84e2-4f4e-b186-d92c4584f202\") "
Apr 24 14:44:00.397460 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:44:00.397403    2574 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"tokenizer-cache\" (UniqueName: \"kubernetes.io/empty-dir/e56da355-84e2-4f4e-b186-d92c4584f202-tokenizer-cache\") pod \"e56da355-84e2-4f4e-b186-d92c4584f202\" (UID: \"e56da355-84e2-4f4e-b186-d92c4584f202\") "
Apr 24 14:44:00.397497 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:44:00.397474    2574 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"tokenizer-uds\" (UniqueName: \"kubernetes.io/empty-dir/e56da355-84e2-4f4e-b186-d92c4584f202-tokenizer-uds\") pod \"e56da355-84e2-4f4e-b186-d92c4584f202\" (UID: \"e56da355-84e2-4f4e-b186-d92c4584f202\") "
Apr 24 14:44:00.397742 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:44:00.397716    2574 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e56da355-84e2-4f4e-b186-d92c4584f202-tokenizer-cache" (OuterVolumeSpecName: "tokenizer-cache") pod "e56da355-84e2-4f4e-b186-d92c4584f202" (UID: "e56da355-84e2-4f4e-b186-d92c4584f202"). InnerVolumeSpecName "tokenizer-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Apr 24 14:44:00.397742 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:44:00.397729    2574 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e56da355-84e2-4f4e-b186-d92c4584f202-tokenizer-tmp" (OuterVolumeSpecName: "tokenizer-tmp") pod "e56da355-84e2-4f4e-b186-d92c4584f202" (UID: "e56da355-84e2-4f4e-b186-d92c4584f202"). InnerVolumeSpecName "tokenizer-tmp". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Apr 24 14:44:00.397902 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:44:00.397811    2574 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e56da355-84e2-4f4e-b186-d92c4584f202-tokenizer-uds" (OuterVolumeSpecName: "tokenizer-uds") pod "e56da355-84e2-4f4e-b186-d92c4584f202" (UID: "e56da355-84e2-4f4e-b186-d92c4584f202"). InnerVolumeSpecName "tokenizer-uds". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Apr 24 14:44:00.398159 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:44:00.398137    2574 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e56da355-84e2-4f4e-b186-d92c4584f202-kserve-provision-location" (OuterVolumeSpecName: "kserve-provision-location") pod "e56da355-84e2-4f4e-b186-d92c4584f202" (UID: "e56da355-84e2-4f4e-b186-d92c4584f202"). InnerVolumeSpecName "kserve-provision-location". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Apr 24 14:44:00.399326 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:44:00.399303    2574 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e56da355-84e2-4f4e-b186-d92c4584f202-kube-api-access-tbl8r" (OuterVolumeSpecName: "kube-api-access-tbl8r") pod "e56da355-84e2-4f4e-b186-d92c4584f202" (UID: "e56da355-84e2-4f4e-b186-d92c4584f202"). InnerVolumeSpecName "kube-api-access-tbl8r". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Apr 24 14:44:00.399394 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:44:00.399357    2574 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/e56da355-84e2-4f4e-b186-d92c4584f202-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "e56da355-84e2-4f4e-b186-d92c4584f202" (UID: "e56da355-84e2-4f4e-b186-d92c4584f202"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Apr 24 14:44:00.498230 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:44:00.498198    2574 reconciler_common.go:299] "Volume detached for volume \"tokenizer-uds\" (UniqueName: \"kubernetes.io/empty-dir/e56da355-84e2-4f4e-b186-d92c4584f202-tokenizer-uds\") on node \"ip-10-0-136-132.ec2.internal\" DevicePath \"\""
Apr 24 14:44:00.498230 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:44:00.498225    2574 reconciler_common.go:299] "Volume detached for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/e56da355-84e2-4f4e-b186-d92c4584f202-kserve-provision-location\") on node \"ip-10-0-136-132.ec2.internal\" DevicePath \"\""
Apr 24 14:44:00.498230 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:44:00.498236    2574 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-tbl8r\" (UniqueName: \"kubernetes.io/projected/e56da355-84e2-4f4e-b186-d92c4584f202-kube-api-access-tbl8r\") on node \"ip-10-0-136-132.ec2.internal\" DevicePath \"\""
Apr 24 14:44:00.498489 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:44:00.498247    2574 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/e56da355-84e2-4f4e-b186-d92c4584f202-tls-certs\") on node \"ip-10-0-136-132.ec2.internal\" DevicePath \"\""
Apr 24 14:44:00.498489 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:44:00.498256    2574 reconciler_common.go:299] "Volume detached for volume \"tokenizer-tmp\" (UniqueName: \"kubernetes.io/empty-dir/e56da355-84e2-4f4e-b186-d92c4584f202-tokenizer-tmp\") on node \"ip-10-0-136-132.ec2.internal\" DevicePath \"\""
Apr 24 14:44:00.498489 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:44:00.498264    2574 reconciler_common.go:299] "Volume detached for volume \"tokenizer-cache\" (UniqueName: \"kubernetes.io/empty-dir/e56da355-84e2-4f4e-b186-d92c4584f202-tokenizer-cache\") on node \"ip-10-0-136-132.ec2.internal\" DevicePath \"\""
Apr 24 14:44:00.687508 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:44:00.687476    2574 kubelet.go:2553] "SyncLoop DELETE" source="api" pods=["kserve-ci-e2e-test/stop-feature-test-kserve-router-scheduler-7cb4fb9c5d-vb62h"]
Apr 24 14:44:00.693225 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:44:00.693200    2574 kubelet.go:2547] "SyncLoop REMOVE" source="api" pods=["kserve-ci-e2e-test/stop-feature-test-kserve-router-scheduler-7cb4fb9c5d-vb62h"]
Apr 24 14:44:01.770152 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:44:01.770115    2574 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="e56da355-84e2-4f4e-b186-d92c4584f202" path="/var/lib/kubelet/pods/e56da355-84e2-4f4e-b186-d92c4584f202/volumes"
Apr 24 14:44:17.302315 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:44:17.302281    2574 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["kserve-ci-e2e-test/stop-feature-test-kserve-88fb48646-7mxhg"]
Apr 24 14:44:17.302781 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:44:17.302691    2574 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e56da355-84e2-4f4e-b186-d92c4584f202" containerName="tokenizer"
Apr 24 14:44:17.302781 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:44:17.302703    2574 state_mem.go:107] "Deleted CPUSet assignment" podUID="e56da355-84e2-4f4e-b186-d92c4584f202" containerName="tokenizer"
Apr 24 14:44:17.302781 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:44:17.302720    2574 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e56da355-84e2-4f4e-b186-d92c4584f202" containerName="storage-initializer"
Apr 24 14:44:17.302781 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:44:17.302726    2574 state_mem.go:107] "Deleted CPUSet assignment" podUID="e56da355-84e2-4f4e-b186-d92c4584f202" containerName="storage-initializer"
Apr 24 14:44:17.302781 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:44:17.302742    2574 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e56da355-84e2-4f4e-b186-d92c4584f202" containerName="main"
Apr 24 14:44:17.302781 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:44:17.302747    2574 state_mem.go:107] "Deleted CPUSet assignment" podUID="e56da355-84e2-4f4e-b186-d92c4584f202" containerName="main"
Apr 24 14:44:17.302982 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:44:17.302801    2574 memory_manager.go:356] "RemoveStaleState removing state" podUID="e56da355-84e2-4f4e-b186-d92c4584f202" containerName="tokenizer"
Apr 24 14:44:17.302982 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:44:17.302815    2574 memory_manager.go:356] "RemoveStaleState removing state" podUID="e56da355-84e2-4f4e-b186-d92c4584f202" containerName="main"
Apr 24 14:44:17.304912 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:44:17.304896    2574 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/stop-feature-test-kserve-88fb48646-7mxhg"
Apr 24 14:44:17.315226 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:44:17.315194    2574 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/stop-feature-test-kserve-88fb48646-7mxhg"]
Apr 24 14:44:17.447309 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:44:17.447271    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/0e6568d3-aa47-482c-b6d2-ab032f34cc79-dshm\") pod \"stop-feature-test-kserve-88fb48646-7mxhg\" (UID: \"0e6568d3-aa47-482c-b6d2-ab032f34cc79\") " pod="kserve-ci-e2e-test/stop-feature-test-kserve-88fb48646-7mxhg"
Apr 24 14:44:17.447511 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:44:17.447326    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/0e6568d3-aa47-482c-b6d2-ab032f34cc79-kserve-provision-location\") pod \"stop-feature-test-kserve-88fb48646-7mxhg\" (UID: \"0e6568d3-aa47-482c-b6d2-ab032f34cc79\") " pod="kserve-ci-e2e-test/stop-feature-test-kserve-88fb48646-7mxhg"
Apr 24 14:44:17.447511 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:44:17.447410    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/0e6568d3-aa47-482c-b6d2-ab032f34cc79-tls-certs\") pod \"stop-feature-test-kserve-88fb48646-7mxhg\" (UID: \"0e6568d3-aa47-482c-b6d2-ab032f34cc79\") " pod="kserve-ci-e2e-test/stop-feature-test-kserve-88fb48646-7mxhg"
Apr 24 14:44:17.447511 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:44:17.447443    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/0e6568d3-aa47-482c-b6d2-ab032f34cc79-home\") pod \"stop-feature-test-kserve-88fb48646-7mxhg\" (UID: \"0e6568d3-aa47-482c-b6d2-ab032f34cc79\") " pod="kserve-ci-e2e-test/stop-feature-test-kserve-88fb48646-7mxhg"
Apr 24 14:44:17.447511 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:44:17.447483    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/0e6568d3-aa47-482c-b6d2-ab032f34cc79-model-cache\") pod \"stop-feature-test-kserve-88fb48646-7mxhg\" (UID: \"0e6568d3-aa47-482c-b6d2-ab032f34cc79\") " pod="kserve-ci-e2e-test/stop-feature-test-kserve-88fb48646-7mxhg"
Apr 24 14:44:17.447667 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:44:17.447514    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rpll9\" (UniqueName: \"kubernetes.io/projected/0e6568d3-aa47-482c-b6d2-ab032f34cc79-kube-api-access-rpll9\") pod \"stop-feature-test-kserve-88fb48646-7mxhg\" (UID: \"0e6568d3-aa47-482c-b6d2-ab032f34cc79\") " pod="kserve-ci-e2e-test/stop-feature-test-kserve-88fb48646-7mxhg"
Apr 24 14:44:17.539090 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:44:17.539057    2574 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["kserve-ci-e2e-test/stop-feature-test-kserve-router-scheduler-7cb4fb9c5d-4m59m"]
Apr 24 14:44:17.541740 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:44:17.541721    2574 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/stop-feature-test-kserve-router-scheduler-7cb4fb9c5d-4m59m"
Apr 24 14:44:17.543605 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:44:17.543585    2574 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"kserve-ci-e2e-test\"/\"stop-feature-test-epp-sa-dockercfg-smwdp\""
Apr 24 14:44:17.548186 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:44:17.548163    2574 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-rpll9\" (UniqueName: \"kubernetes.io/projected/0e6568d3-aa47-482c-b6d2-ab032f34cc79-kube-api-access-rpll9\") pod \"stop-feature-test-kserve-88fb48646-7mxhg\" (UID: \"0e6568d3-aa47-482c-b6d2-ab032f34cc79\") " pod="kserve-ci-e2e-test/stop-feature-test-kserve-88fb48646-7mxhg"
Apr 24 14:44:17.548307 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:44:17.548236    2574 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/0e6568d3-aa47-482c-b6d2-ab032f34cc79-dshm\") pod \"stop-feature-test-kserve-88fb48646-7mxhg\" (UID: \"0e6568d3-aa47-482c-b6d2-ab032f34cc79\") " pod="kserve-ci-e2e-test/stop-feature-test-kserve-88fb48646-7mxhg"
Apr 24 14:44:17.548307 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:44:17.548280    2574 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/0e6568d3-aa47-482c-b6d2-ab032f34cc79-kserve-provision-location\") pod \"stop-feature-test-kserve-88fb48646-7mxhg\" (UID: \"0e6568d3-aa47-482c-b6d2-ab032f34cc79\") " pod="kserve-ci-e2e-test/stop-feature-test-kserve-88fb48646-7mxhg"
Apr 24 14:44:17.548563 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:44:17.548310    2574 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/0e6568d3-aa47-482c-b6d2-ab032f34cc79-tls-certs\") pod \"stop-feature-test-kserve-88fb48646-7mxhg\" (UID: \"0e6568d3-aa47-482c-b6d2-ab032f34cc79\") " pod="kserve-ci-e2e-test/stop-feature-test-kserve-88fb48646-7mxhg"
Apr 24 14:44:17.548563 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:44:17.548372    2574 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/0e6568d3-aa47-482c-b6d2-ab032f34cc79-home\") pod \"stop-feature-test-kserve-88fb48646-7mxhg\" (UID: \"0e6568d3-aa47-482c-b6d2-ab032f34cc79\") " pod="kserve-ci-e2e-test/stop-feature-test-kserve-88fb48646-7mxhg"
Apr 24 14:44:17.548563 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:44:17.548421    2574 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/0e6568d3-aa47-482c-b6d2-ab032f34cc79-model-cache\") pod \"stop-feature-test-kserve-88fb48646-7mxhg\" (UID: \"0e6568d3-aa47-482c-b6d2-ab032f34cc79\") " pod="kserve-ci-e2e-test/stop-feature-test-kserve-88fb48646-7mxhg"
Apr 24 14:44:17.548881 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:44:17.548858    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/0e6568d3-aa47-482c-b6d2-ab032f34cc79-kserve-provision-location\") pod \"stop-feature-test-kserve-88fb48646-7mxhg\" (UID: \"0e6568d3-aa47-482c-b6d2-ab032f34cc79\") " pod="kserve-ci-e2e-test/stop-feature-test-kserve-88fb48646-7mxhg"
Apr 24 14:44:17.548989 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:44:17.548926    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/0e6568d3-aa47-482c-b6d2-ab032f34cc79-model-cache\") pod \"stop-feature-test-kserve-88fb48646-7mxhg\" (UID: \"0e6568d3-aa47-482c-b6d2-ab032f34cc79\") " pod="kserve-ci-e2e-test/stop-feature-test-kserve-88fb48646-7mxhg"
Apr 24 14:44:17.549177 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:44:17.549157    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/0e6568d3-aa47-482c-b6d2-ab032f34cc79-home\") pod \"stop-feature-test-kserve-88fb48646-7mxhg\" (UID: \"0e6568d3-aa47-482c-b6d2-ab032f34cc79\") " pod="kserve-ci-e2e-test/stop-feature-test-kserve-88fb48646-7mxhg"
Apr 24 14:44:17.551205 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:44:17.551184    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/0e6568d3-aa47-482c-b6d2-ab032f34cc79-dshm\") pod \"stop-feature-test-kserve-88fb48646-7mxhg\" (UID: \"0e6568d3-aa47-482c-b6d2-ab032f34cc79\") " pod="kserve-ci-e2e-test/stop-feature-test-kserve-88fb48646-7mxhg"
Apr 24 14:44:17.552019 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:44:17.551993    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/0e6568d3-aa47-482c-b6d2-ab032f34cc79-tls-certs\") pod \"stop-feature-test-kserve-88fb48646-7mxhg\" (UID: \"0e6568d3-aa47-482c-b6d2-ab032f34cc79\") " pod="kserve-ci-e2e-test/stop-feature-test-kserve-88fb48646-7mxhg"
Apr 24 14:44:17.553863 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:44:17.553818    2574 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/stop-feature-test-kserve-router-scheduler-7cb4fb9c5d-4m59m"]
Apr 24 14:44:17.555872 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:44:17.555850    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-rpll9\" (UniqueName: \"kubernetes.io/projected/0e6568d3-aa47-482c-b6d2-ab032f34cc79-kube-api-access-rpll9\") pod \"stop-feature-test-kserve-88fb48646-7mxhg\" (UID: \"0e6568d3-aa47-482c-b6d2-ab032f34cc79\") " pod="kserve-ci-e2e-test/stop-feature-test-kserve-88fb48646-7mxhg"
Apr 24 14:44:17.616093 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:44:17.616058    2574 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/stop-feature-test-kserve-88fb48646-7mxhg"
Apr 24 14:44:17.649273 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:44:17.649244    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tokenizer-uds\" (UniqueName: \"kubernetes.io/empty-dir/51b65675-57de-46c2-b8ff-398029ac53c7-tokenizer-uds\") pod \"stop-feature-test-kserve-router-scheduler-7cb4fb9c5d-4m59m\" (UID: \"51b65675-57de-46c2-b8ff-398029ac53c7\") " pod="kserve-ci-e2e-test/stop-feature-test-kserve-router-scheduler-7cb4fb9c5d-4m59m"
Apr 24 14:44:17.649441 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:44:17.649281    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/51b65675-57de-46c2-b8ff-398029ac53c7-kserve-provision-location\") pod \"stop-feature-test-kserve-router-scheduler-7cb4fb9c5d-4m59m\" (UID: \"51b65675-57de-46c2-b8ff-398029ac53c7\") " pod="kserve-ci-e2e-test/stop-feature-test-kserve-router-scheduler-7cb4fb9c5d-4m59m"
Apr 24 14:44:17.649441 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:44:17.649367    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bxwz2\" (UniqueName: \"kubernetes.io/projected/51b65675-57de-46c2-b8ff-398029ac53c7-kube-api-access-bxwz2\") pod \"stop-feature-test-kserve-router-scheduler-7cb4fb9c5d-4m59m\" (UID: \"51b65675-57de-46c2-b8ff-398029ac53c7\") " pod="kserve-ci-e2e-test/stop-feature-test-kserve-router-scheduler-7cb4fb9c5d-4m59m"
Apr 24 14:44:17.649441 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:44:17.649413    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/51b65675-57de-46c2-b8ff-398029ac53c7-tls-certs\") pod \"stop-feature-test-kserve-router-scheduler-7cb4fb9c5d-4m59m\" (UID: \"51b65675-57de-46c2-b8ff-398029ac53c7\") " pod="kserve-ci-e2e-test/stop-feature-test-kserve-router-scheduler-7cb4fb9c5d-4m59m"
Apr 24 14:44:17.649615 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:44:17.649518    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tokenizer-tmp\" (UniqueName: \"kubernetes.io/empty-dir/51b65675-57de-46c2-b8ff-398029ac53c7-tokenizer-tmp\") pod \"stop-feature-test-kserve-router-scheduler-7cb4fb9c5d-4m59m\" (UID: \"51b65675-57de-46c2-b8ff-398029ac53c7\") " pod="kserve-ci-e2e-test/stop-feature-test-kserve-router-scheduler-7cb4fb9c5d-4m59m"
Apr 24 14:44:17.649615 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:44:17.649569    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tokenizer-cache\" (UniqueName: \"kubernetes.io/empty-dir/51b65675-57de-46c2-b8ff-398029ac53c7-tokenizer-cache\") pod \"stop-feature-test-kserve-router-scheduler-7cb4fb9c5d-4m59m\" (UID: \"51b65675-57de-46c2-b8ff-398029ac53c7\") " pod="kserve-ci-e2e-test/stop-feature-test-kserve-router-scheduler-7cb4fb9c5d-4m59m"
Apr 24 14:44:17.741394 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:44:17.741365    2574 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/stop-feature-test-kserve-88fb48646-7mxhg"]
Apr 24 14:44:17.743638 ip-10-0-136-132 kubenswrapper[2574]: W0424 14:44:17.743599    2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0e6568d3_aa47_482c_b6d2_ab032f34cc79.slice/crio-e241dc709fd6371e2323a9f265870d768d82cb930ccca417431f11985efcceda WatchSource:0}: Error finding container e241dc709fd6371e2323a9f265870d768d82cb930ccca417431f11985efcceda: Status 404 returned error can't find the container with id e241dc709fd6371e2323a9f265870d768d82cb930ccca417431f11985efcceda
Apr 24 14:44:17.753084 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:44:17.751836    2574 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"tokenizer-tmp\" (UniqueName: \"kubernetes.io/empty-dir/51b65675-57de-46c2-b8ff-398029ac53c7-tokenizer-tmp\") pod \"stop-feature-test-kserve-router-scheduler-7cb4fb9c5d-4m59m\" (UID: \"51b65675-57de-46c2-b8ff-398029ac53c7\") " pod="kserve-ci-e2e-test/stop-feature-test-kserve-router-scheduler-7cb4fb9c5d-4m59m"
Apr 24 14:44:17.753084 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:44:17.751886    2574 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"tokenizer-cache\" (UniqueName: \"kubernetes.io/empty-dir/51b65675-57de-46c2-b8ff-398029ac53c7-tokenizer-cache\") pod \"stop-feature-test-kserve-router-scheduler-7cb4fb9c5d-4m59m\" (UID: \"51b65675-57de-46c2-b8ff-398029ac53c7\") " pod="kserve-ci-e2e-test/stop-feature-test-kserve-router-scheduler-7cb4fb9c5d-4m59m"
Apr 24 14:44:17.753084 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:44:17.751930    2574 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"tokenizer-uds\" (UniqueName: \"kubernetes.io/empty-dir/51b65675-57de-46c2-b8ff-398029ac53c7-tokenizer-uds\") pod \"stop-feature-test-kserve-router-scheduler-7cb4fb9c5d-4m59m\" (UID: \"51b65675-57de-46c2-b8ff-398029ac53c7\") " pod="kserve-ci-e2e-test/stop-feature-test-kserve-router-scheduler-7cb4fb9c5d-4m59m"
Apr 24 14:44:17.753084 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:44:17.751960    2574 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/51b65675-57de-46c2-b8ff-398029ac53c7-kserve-provision-location\") pod \"stop-feature-test-kserve-router-scheduler-7cb4fb9c5d-4m59m\" (UID: \"51b65675-57de-46c2-b8ff-398029ac53c7\") " pod="kserve-ci-e2e-test/stop-feature-test-kserve-router-scheduler-7cb4fb9c5d-4m59m"
Apr 24 14:44:17.753084 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:44:17.752017    2574 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-bxwz2\" (UniqueName: \"kubernetes.io/projected/51b65675-57de-46c2-b8ff-398029ac53c7-kube-api-access-bxwz2\") pod \"stop-feature-test-kserve-router-scheduler-7cb4fb9c5d-4m59m\" (UID: \"51b65675-57de-46c2-b8ff-398029ac53c7\") " pod="kserve-ci-e2e-test/stop-feature-test-kserve-router-scheduler-7cb4fb9c5d-4m59m"
Apr 24 14:44:17.753084 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:44:17.752078    2574 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/51b65675-57de-46c2-b8ff-398029ac53c7-tls-certs\") pod \"stop-feature-test-kserve-router-scheduler-7cb4fb9c5d-4m59m\" (UID: \"51b65675-57de-46c2-b8ff-398029ac53c7\") " pod="kserve-ci-e2e-test/stop-feature-test-kserve-router-scheduler-7cb4fb9c5d-4m59m"
Apr 24 14:44:17.753084 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:44:17.752310    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tokenizer-cache\" (UniqueName: \"kubernetes.io/empty-dir/51b65675-57de-46c2-b8ff-398029ac53c7-tokenizer-cache\") pod \"stop-feature-test-kserve-router-scheduler-7cb4fb9c5d-4m59m\" (UID: \"51b65675-57de-46c2-b8ff-398029ac53c7\") " pod="kserve-ci-e2e-test/stop-feature-test-kserve-router-scheduler-7cb4fb9c5d-4m59m"
Apr 24 14:44:17.753860 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:44:17.753792    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tokenizer-tmp\" (UniqueName: \"kubernetes.io/empty-dir/51b65675-57de-46c2-b8ff-398029ac53c7-tokenizer-tmp\") pod \"stop-feature-test-kserve-router-scheduler-7cb4fb9c5d-4m59m\" (UID: \"51b65675-57de-46c2-b8ff-398029ac53c7\") " pod="kserve-ci-e2e-test/stop-feature-test-kserve-router-scheduler-7cb4fb9c5d-4m59m"
Apr 24 14:44:17.754279 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:44:17.754247    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/51b65675-57de-46c2-b8ff-398029ac53c7-kserve-provision-location\") pod \"stop-feature-test-kserve-router-scheduler-7cb4fb9c5d-4m59m\" (UID: \"51b65675-57de-46c2-b8ff-398029ac53c7\") " pod="kserve-ci-e2e-test/stop-feature-test-kserve-router-scheduler-7cb4fb9c5d-4m59m"
Apr 24 14:44:17.754787 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:44:17.754745    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tokenizer-uds\" (UniqueName: \"kubernetes.io/empty-dir/51b65675-57de-46c2-b8ff-398029ac53c7-tokenizer-uds\") pod \"stop-feature-test-kserve-router-scheduler-7cb4fb9c5d-4m59m\" (UID: \"51b65675-57de-46c2-b8ff-398029ac53c7\") " pod="kserve-ci-e2e-test/stop-feature-test-kserve-router-scheduler-7cb4fb9c5d-4m59m"
Apr 24 14:44:17.755132 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:44:17.755064    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/51b65675-57de-46c2-b8ff-398029ac53c7-tls-certs\") pod \"stop-feature-test-kserve-router-scheduler-7cb4fb9c5d-4m59m\" (UID: \"51b65675-57de-46c2-b8ff-398029ac53c7\") " pod="kserve-ci-e2e-test/stop-feature-test-kserve-router-scheduler-7cb4fb9c5d-4m59m"
Apr 24 14:44:17.761331 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:44:17.761312    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-bxwz2\" (UniqueName: \"kubernetes.io/projected/51b65675-57de-46c2-b8ff-398029ac53c7-kube-api-access-bxwz2\") pod \"stop-feature-test-kserve-router-scheduler-7cb4fb9c5d-4m59m\" (UID: \"51b65675-57de-46c2-b8ff-398029ac53c7\") " pod="kserve-ci-e2e-test/stop-feature-test-kserve-router-scheduler-7cb4fb9c5d-4m59m"
Apr 24 14:44:17.852407 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:44:17.852373    2574 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/stop-feature-test-kserve-router-scheduler-7cb4fb9c5d-4m59m"
Apr 24 14:44:17.984386 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:44:17.984318    2574 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/stop-feature-test-kserve-router-scheduler-7cb4fb9c5d-4m59m"]
Apr 24 14:44:18.436909 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:44:18.436859    2574 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/stop-feature-test-kserve-router-scheduler-7cb4fb9c5d-4m59m" event={"ID":"51b65675-57de-46c2-b8ff-398029ac53c7","Type":"ContainerStarted","Data":"f379575de8b1a99f3b4171e971aaaa1ce800350b914485be4910fbc6e9c2973a"}
Apr 24 14:44:18.436909 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:44:18.436913    2574 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/stop-feature-test-kserve-router-scheduler-7cb4fb9c5d-4m59m" event={"ID":"51b65675-57de-46c2-b8ff-398029ac53c7","Type":"ContainerStarted","Data":"a9fec223473e140f9edec21089c29e7c60c8ab1e68109f2e86937479a381d2f9"}
Apr 24 14:44:18.438400 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:44:18.438371    2574 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/stop-feature-test-kserve-88fb48646-7mxhg" event={"ID":"0e6568d3-aa47-482c-b6d2-ab032f34cc79","Type":"ContainerStarted","Data":"98299b8ddd2f9e10c32a75b1a223f115a4f4a42c6394303606d24611c45c6f10"}
Apr 24 14:44:18.438509 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:44:18.438409    2574 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/stop-feature-test-kserve-88fb48646-7mxhg" event={"ID":"0e6568d3-aa47-482c-b6d2-ab032f34cc79","Type":"ContainerStarted","Data":"e241dc709fd6371e2323a9f265870d768d82cb930ccca417431f11985efcceda"}
Apr 24 14:44:19.443855 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:44:19.443809    2574 generic.go:358] "Generic (PLEG): container finished" podID="51b65675-57de-46c2-b8ff-398029ac53c7" containerID="f379575de8b1a99f3b4171e971aaaa1ce800350b914485be4910fbc6e9c2973a" exitCode=0
Apr 24 14:44:19.444305 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:44:19.443889    2574 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/stop-feature-test-kserve-router-scheduler-7cb4fb9c5d-4m59m" event={"ID":"51b65675-57de-46c2-b8ff-398029ac53c7","Type":"ContainerDied","Data":"f379575de8b1a99f3b4171e971aaaa1ce800350b914485be4910fbc6e9c2973a"}
Apr 24 14:44:20.450122 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:44:20.450075    2574 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/stop-feature-test-kserve-router-scheduler-7cb4fb9c5d-4m59m" event={"ID":"51b65675-57de-46c2-b8ff-398029ac53c7","Type":"ContainerStarted","Data":"4c1c922dd5365754272bc67c340cac6a2f714319afd9850df1cafcf3ccba7ae9"}
Apr 24 14:44:20.450122 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:44:20.450122    2574 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/stop-feature-test-kserve-router-scheduler-7cb4fb9c5d-4m59m" event={"ID":"51b65675-57de-46c2-b8ff-398029ac53c7","Type":"ContainerStarted","Data":"0a718d3f4431ec2090a1cc7fa0d815d590354811e52a3025a64e4a33be5078a5"}
Apr 24 14:44:20.450704 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:44:20.450206    2574 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve-ci-e2e-test/stop-feature-test-kserve-router-scheduler-7cb4fb9c5d-4m59m"
Apr 24 14:44:20.471357 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:44:20.471289    2574 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="kserve-ci-e2e-test/stop-feature-test-kserve-router-scheduler-7cb4fb9c5d-4m59m" podStartSLOduration=3.471273635 podStartE2EDuration="3.471273635s" podCreationTimestamp="2026-04-24 14:44:17 +0000 UTC" firstStartedPulling="0001-01-01 00:00:00 +0000 UTC" lastFinishedPulling="0001-01-01 00:00:00 +0000 UTC" observedRunningTime="2026-04-24 14:44:20.467633161 +0000 UTC m=+1223.199734844" watchObservedRunningTime="2026-04-24 14:44:20.471273635 +0000 UTC m=+1223.203375284"
Apr 24 14:44:27.853325 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:44:27.853288    2574 kubelet.go:2658] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="kserve-ci-e2e-test/stop-feature-test-kserve-router-scheduler-7cb4fb9c5d-4m59m"
Apr 24 14:44:27.853325 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:44:27.853326    2574 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve-ci-e2e-test/stop-feature-test-kserve-router-scheduler-7cb4fb9c5d-4m59m"
Apr 24 14:44:27.856395 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:44:27.856371    2574 kubelet.go:2658] "SyncLoop (probe)" probe="startup" status="started" pod="kserve-ci-e2e-test/stop-feature-test-kserve-router-scheduler-7cb4fb9c5d-4m59m"
Apr 24 14:44:28.486178 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:44:28.486142    2574 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="kserve-ci-e2e-test/stop-feature-test-kserve-router-scheduler-7cb4fb9c5d-4m59m"
Apr 24 14:44:29.491055 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:44:29.491026    2574 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_stop-feature-test-kserve-88fb48646-cq9c6_b9251f52-2109-4405-8651-0a0f450355d5/main/0.log"
Apr 24 14:44:29.491491 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:44:29.491409    2574 generic.go:358] "Generic (PLEG): container finished" podID="b9251f52-2109-4405-8651-0a0f450355d5" containerID="1185fbcac517f3f1a066a2111f2ed31485629deab077361f48e4c461e9799f09" exitCode=137
Apr 24 14:44:29.491535 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:44:29.491479    2574 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/stop-feature-test-kserve-88fb48646-cq9c6" event={"ID":"b9251f52-2109-4405-8651-0a0f450355d5","Type":"ContainerDied","Data":"1185fbcac517f3f1a066a2111f2ed31485629deab077361f48e4c461e9799f09"}
Apr 24 14:44:29.592929 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:44:29.592903    2574 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_stop-feature-test-kserve-88fb48646-cq9c6_b9251f52-2109-4405-8651-0a0f450355d5/main/0.log"
Apr 24 14:44:29.593320 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:44:29.593304    2574 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/stop-feature-test-kserve-88fb48646-cq9c6"
Apr 24 14:44:29.653313 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:44:29.653285    2574 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/b9251f52-2109-4405-8651-0a0f450355d5-tls-certs\") pod \"b9251f52-2109-4405-8651-0a0f450355d5\" (UID: \"b9251f52-2109-4405-8651-0a0f450355d5\") "
Apr 24 14:44:29.653313 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:44:29.653323    2574 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/b9251f52-2109-4405-8651-0a0f450355d5-dshm\") pod \"b9251f52-2109-4405-8651-0a0f450355d5\" (UID: \"b9251f52-2109-4405-8651-0a0f450355d5\") "
Apr 24 14:44:29.653545 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:44:29.653370    2574 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/b9251f52-2109-4405-8651-0a0f450355d5-kserve-provision-location\") pod \"b9251f52-2109-4405-8651-0a0f450355d5\" (UID: \"b9251f52-2109-4405-8651-0a0f450355d5\") "
Apr 24 14:44:29.653545 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:44:29.653413    2574 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"kube-api-access-7kdqx\" (UniqueName: \"kubernetes.io/projected/b9251f52-2109-4405-8651-0a0f450355d5-kube-api-access-7kdqx\") pod \"b9251f52-2109-4405-8651-0a0f450355d5\" (UID: \"b9251f52-2109-4405-8651-0a0f450355d5\") "
Apr 24 14:44:29.653545 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:44:29.653499    2574 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/b9251f52-2109-4405-8651-0a0f450355d5-home\") pod \"b9251f52-2109-4405-8651-0a0f450355d5\" (UID: \"b9251f52-2109-4405-8651-0a0f450355d5\") "
Apr 24 14:44:29.653545 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:44:29.653528    2574 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/b9251f52-2109-4405-8651-0a0f450355d5-model-cache\") pod \"b9251f52-2109-4405-8651-0a0f450355d5\" (UID: \"b9251f52-2109-4405-8651-0a0f450355d5\") "
Apr 24 14:44:29.653978 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:44:29.653893    2574 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b9251f52-2109-4405-8651-0a0f450355d5-home" (OuterVolumeSpecName: "home") pod "b9251f52-2109-4405-8651-0a0f450355d5" (UID: "b9251f52-2109-4405-8651-0a0f450355d5"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Apr 24 14:44:29.654064 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:44:29.653960    2574 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b9251f52-2109-4405-8651-0a0f450355d5-model-cache" (OuterVolumeSpecName: "model-cache") pod "b9251f52-2109-4405-8651-0a0f450355d5" (UID: "b9251f52-2109-4405-8651-0a0f450355d5"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Apr 24 14:44:29.655908 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:44:29.655884    2574 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b9251f52-2109-4405-8651-0a0f450355d5-dshm" (OuterVolumeSpecName: "dshm") pod "b9251f52-2109-4405-8651-0a0f450355d5" (UID: "b9251f52-2109-4405-8651-0a0f450355d5"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Apr 24 14:44:29.656010 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:44:29.655884    2574 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/b9251f52-2109-4405-8651-0a0f450355d5-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "b9251f52-2109-4405-8651-0a0f450355d5" (UID: "b9251f52-2109-4405-8651-0a0f450355d5"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Apr 24 14:44:29.656093 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:44:29.656077    2574 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/b9251f52-2109-4405-8651-0a0f450355d5-kube-api-access-7kdqx" (OuterVolumeSpecName: "kube-api-access-7kdqx") pod "b9251f52-2109-4405-8651-0a0f450355d5" (UID: "b9251f52-2109-4405-8651-0a0f450355d5"). InnerVolumeSpecName "kube-api-access-7kdqx". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Apr 24 14:44:29.710413 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:44:29.710362    2574 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b9251f52-2109-4405-8651-0a0f450355d5-kserve-provision-location" (OuterVolumeSpecName: "kserve-provision-location") pod "b9251f52-2109-4405-8651-0a0f450355d5" (UID: "b9251f52-2109-4405-8651-0a0f450355d5"). InnerVolumeSpecName "kserve-provision-location". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Apr 24 14:44:29.754741 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:44:29.754708    2574 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-7kdqx\" (UniqueName: \"kubernetes.io/projected/b9251f52-2109-4405-8651-0a0f450355d5-kube-api-access-7kdqx\") on node \"ip-10-0-136-132.ec2.internal\" DevicePath \"\""
Apr 24 14:44:29.754741 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:44:29.754737    2574 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/b9251f52-2109-4405-8651-0a0f450355d5-home\") on node \"ip-10-0-136-132.ec2.internal\" DevicePath \"\""
Apr 24 14:44:29.754741 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:44:29.754747    2574 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/b9251f52-2109-4405-8651-0a0f450355d5-model-cache\") on node \"ip-10-0-136-132.ec2.internal\" DevicePath \"\""
Apr 24 14:44:29.754994 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:44:29.754757    2574 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/b9251f52-2109-4405-8651-0a0f450355d5-tls-certs\") on node \"ip-10-0-136-132.ec2.internal\" DevicePath \"\""
Apr 24 14:44:29.754994 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:44:29.754765    2574 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/b9251f52-2109-4405-8651-0a0f450355d5-dshm\") on node \"ip-10-0-136-132.ec2.internal\" DevicePath \"\""
Apr 24 14:44:29.754994 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:44:29.754773    2574 reconciler_common.go:299] "Volume detached for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/b9251f52-2109-4405-8651-0a0f450355d5-kserve-provision-location\") on node \"ip-10-0-136-132.ec2.internal\" DevicePath \"\""
Apr 24 14:44:30.496967 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:44:30.496938    2574 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_stop-feature-test-kserve-88fb48646-cq9c6_b9251f52-2109-4405-8651-0a0f450355d5/main/0.log"
Apr 24 14:44:30.497423 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:44:30.497298    2574 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/stop-feature-test-kserve-88fb48646-cq9c6" event={"ID":"b9251f52-2109-4405-8651-0a0f450355d5","Type":"ContainerDied","Data":"ec35a851c26f1a5127008873945de8189db3e6cc37849fa5525348840e1de676"}
Apr 24 14:44:30.497423 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:44:30.497363    2574 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/stop-feature-test-kserve-88fb48646-cq9c6"
Apr 24 14:44:30.497505 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:44:30.497362    2574 scope.go:117] "RemoveContainer" containerID="1185fbcac517f3f1a066a2111f2ed31485629deab077361f48e4c461e9799f09"
Apr 24 14:44:30.516925 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:44:30.516905    2574 scope.go:117] "RemoveContainer" containerID="e22f33838bdab274b88160de954942ddcbbb8e2d8d0e9589d1eeb3ccacaab0f7"
Apr 24 14:44:30.523131 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:44:30.523102    2574 kubelet.go:2553] "SyncLoop DELETE" source="api" pods=["kserve-ci-e2e-test/stop-feature-test-kserve-88fb48646-cq9c6"]
Apr 24 14:44:30.526506 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:44:30.526484    2574 kubelet.go:2547] "SyncLoop REMOVE" source="api" pods=["kserve-ci-e2e-test/stop-feature-test-kserve-88fb48646-cq9c6"]
Apr 24 14:44:31.770494 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:44:31.770459    2574 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="b9251f52-2109-4405-8651-0a0f450355d5" path="/var/lib/kubelet/pods/b9251f52-2109-4405-8651-0a0f450355d5/volumes"
Apr 24 14:44:49.493090 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:44:49.493018    2574 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="kserve-ci-e2e-test/stop-feature-test-kserve-router-scheduler-7cb4fb9c5d-4m59m"
Apr 24 14:45:25.716767 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:45:25.716732    2574 generic.go:358] "Generic (PLEG): container finished" podID="0e6568d3-aa47-482c-b6d2-ab032f34cc79" containerID="98299b8ddd2f9e10c32a75b1a223f115a4f4a42c6394303606d24611c45c6f10" exitCode=0
Apr 24 14:45:25.717171 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:45:25.716808    2574 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/stop-feature-test-kserve-88fb48646-7mxhg" event={"ID":"0e6568d3-aa47-482c-b6d2-ab032f34cc79","Type":"ContainerDied","Data":"98299b8ddd2f9e10c32a75b1a223f115a4f4a42c6394303606d24611c45c6f10"}
Apr 24 14:45:25.717900 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:45:25.717883    2574 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Apr 24 14:45:26.723417 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:45:26.723369    2574 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/stop-feature-test-kserve-88fb48646-7mxhg" event={"ID":"0e6568d3-aa47-482c-b6d2-ab032f34cc79","Type":"ContainerStarted","Data":"f3614809bccf1ef3b90baf950e6552c818b6bcb298034f26ab3f0f9e622553c6"}
Apr 24 14:45:26.747004 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:45:26.746943    2574 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="kserve-ci-e2e-test/stop-feature-test-kserve-88fb48646-7mxhg" podStartSLOduration=69.746923851 podStartE2EDuration="1m9.746923851s" podCreationTimestamp="2026-04-24 14:44:17 +0000 UTC" firstStartedPulling="0001-01-01 00:00:00 +0000 UTC" lastFinishedPulling="0001-01-01 00:00:00 +0000 UTC" observedRunningTime="2026-04-24 14:45:26.740827363 +0000 UTC m=+1289.472929012" watchObservedRunningTime="2026-04-24 14:45:26.746923851 +0000 UTC m=+1289.479025500"
Apr 24 14:45:27.616201 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:45:27.616161    2574 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve-ci-e2e-test/stop-feature-test-kserve-88fb48646-7mxhg"
Apr 24 14:45:27.616412 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:45:27.616212    2574 kubelet.go:2658] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="kserve-ci-e2e-test/stop-feature-test-kserve-88fb48646-7mxhg"
Apr 24 14:45:27.617614 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:45:27.617580    2574 prober.go:120] "Probe failed" probeType="Startup" pod="kserve-ci-e2e-test/stop-feature-test-kserve-88fb48646-7mxhg" podUID="0e6568d3-aa47-482c-b6d2-ab032f34cc79" containerName="main" probeResult="failure" output="Get \"https://10.133.0.55:8000/health\": dial tcp 10.133.0.55:8000: connect: connection refused"
Apr 24 14:45:37.616916 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:45:37.616871    2574 prober.go:120] "Probe failed" probeType="Startup" pod="kserve-ci-e2e-test/stop-feature-test-kserve-88fb48646-7mxhg" podUID="0e6568d3-aa47-482c-b6d2-ab032f34cc79" containerName="main" probeResult="failure" output="Get \"https://10.133.0.55:8000/health\": dial tcp 10.133.0.55:8000: connect: connection refused"
Apr 24 14:45:47.617259 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:45:47.617212    2574 prober.go:120] "Probe failed" probeType="Startup" pod="kserve-ci-e2e-test/stop-feature-test-kserve-88fb48646-7mxhg" podUID="0e6568d3-aa47-482c-b6d2-ab032f34cc79" containerName="main" probeResult="failure" output="Get \"https://10.133.0.55:8000/health\": dial tcp 10.133.0.55:8000: connect: connection refused"
Apr 24 14:45:57.616982 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:45:57.616937    2574 prober.go:120] "Probe failed" probeType="Startup" pod="kserve-ci-e2e-test/stop-feature-test-kserve-88fb48646-7mxhg" podUID="0e6568d3-aa47-482c-b6d2-ab032f34cc79" containerName="main" probeResult="failure" output="Get \"https://10.133.0.55:8000/health\": dial tcp 10.133.0.55:8000: connect: connection refused"
Apr 24 14:46:07.617565 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:46:07.617520    2574 prober.go:120] "Probe failed" probeType="Startup" pod="kserve-ci-e2e-test/stop-feature-test-kserve-88fb48646-7mxhg" podUID="0e6568d3-aa47-482c-b6d2-ab032f34cc79" containerName="main" probeResult="failure" output="Get \"https://10.133.0.55:8000/health\": dial tcp 10.133.0.55:8000: connect: connection refused"
Apr 24 14:46:17.617281 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:46:17.617181    2574 prober.go:120] "Probe failed" probeType="Startup" pod="kserve-ci-e2e-test/stop-feature-test-kserve-88fb48646-7mxhg" podUID="0e6568d3-aa47-482c-b6d2-ab032f34cc79" containerName="main" probeResult="failure" output="Get \"https://10.133.0.55:8000/health\": dial tcp 10.133.0.55:8000: connect: connection refused"
Apr 24 14:46:27.616989 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:46:27.616933    2574 prober.go:120] "Probe failed" probeType="Startup" pod="kserve-ci-e2e-test/stop-feature-test-kserve-88fb48646-7mxhg" podUID="0e6568d3-aa47-482c-b6d2-ab032f34cc79" containerName="main" probeResult="failure" output="Get \"https://10.133.0.55:8000/health\": dial tcp 10.133.0.55:8000: connect: connection refused"
Apr 24 14:46:37.617143 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:46:37.617097    2574 prober.go:120] "Probe failed" probeType="Startup" pod="kserve-ci-e2e-test/stop-feature-test-kserve-88fb48646-7mxhg" podUID="0e6568d3-aa47-482c-b6d2-ab032f34cc79" containerName="main" probeResult="failure" output="Get \"https://10.133.0.55:8000/health\": dial tcp 10.133.0.55:8000: connect: connection refused"
Apr 24 14:46:47.616923 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:46:47.616873    2574 prober.go:120] "Probe failed" probeType="Startup" pod="kserve-ci-e2e-test/stop-feature-test-kserve-88fb48646-7mxhg" podUID="0e6568d3-aa47-482c-b6d2-ab032f34cc79" containerName="main" probeResult="failure" output="Get \"https://10.133.0.55:8000/health\": dial tcp 10.133.0.55:8000: connect: connection refused"
Apr 24 14:46:57.626359 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:46:57.626305    2574 kubelet.go:2658] "SyncLoop (probe)" probe="startup" status="started" pod="kserve-ci-e2e-test/stop-feature-test-kserve-88fb48646-7mxhg"
Apr 24 14:46:57.634085 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:46:57.634064    2574 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="kserve-ci-e2e-test/stop-feature-test-kserve-88fb48646-7mxhg"
Apr 24 14:47:11.431582 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:47:11.431548    2574 kubelet.go:2553] "SyncLoop DELETE" source="api" pods=["kserve-ci-e2e-test/stop-feature-test-kserve-router-scheduler-7cb4fb9c5d-4m59m"]
Apr 24 14:47:11.432144 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:47:11.431971    2574 kuberuntime_container.go:864] "Killing container with a grace period" pod="kserve-ci-e2e-test/stop-feature-test-kserve-router-scheduler-7cb4fb9c5d-4m59m" podUID="51b65675-57de-46c2-b8ff-398029ac53c7" containerName="main" containerID="cri-o://0a718d3f4431ec2090a1cc7fa0d815d590354811e52a3025a64e4a33be5078a5" gracePeriod=30
Apr 24 14:47:11.432144 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:47:11.432017    2574 kuberuntime_container.go:864] "Killing container with a grace period" pod="kserve-ci-e2e-test/stop-feature-test-kserve-router-scheduler-7cb4fb9c5d-4m59m" podUID="51b65675-57de-46c2-b8ff-398029ac53c7" containerName="tokenizer" containerID="cri-o://4c1c922dd5365754272bc67c340cac6a2f714319afd9850df1cafcf3ccba7ae9" gracePeriod=30
Apr 24 14:47:11.437190 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:47:11.437164    2574 kubelet.go:2553] "SyncLoop DELETE" source="api" pods=["kserve-ci-e2e-test/stop-feature-test-kserve-88fb48646-7mxhg"]
Apr 24 14:47:11.437470 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:47:11.437443    2574 kuberuntime_container.go:864] "Killing container with a grace period" pod="kserve-ci-e2e-test/stop-feature-test-kserve-88fb48646-7mxhg" podUID="0e6568d3-aa47-482c-b6d2-ab032f34cc79" containerName="main" containerID="cri-o://f3614809bccf1ef3b90baf950e6552c818b6bcb298034f26ab3f0f9e622553c6" gracePeriod=30
Apr 24 14:47:12.176755 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:47:12.176717    2574 generic.go:358] "Generic (PLEG): container finished" podID="51b65675-57de-46c2-b8ff-398029ac53c7" containerID="0a718d3f4431ec2090a1cc7fa0d815d590354811e52a3025a64e4a33be5078a5" exitCode=0
Apr 24 14:47:12.176944 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:47:12.176784    2574 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/stop-feature-test-kserve-router-scheduler-7cb4fb9c5d-4m59m" event={"ID":"51b65675-57de-46c2-b8ff-398029ac53c7","Type":"ContainerDied","Data":"0a718d3f4431ec2090a1cc7fa0d815d590354811e52a3025a64e4a33be5078a5"}
Apr 24 14:47:12.683959 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:47:12.683937    2574 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/stop-feature-test-kserve-router-scheduler-7cb4fb9c5d-4m59m"
Apr 24 14:47:12.794553 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:47:12.794477    2574 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/51b65675-57de-46c2-b8ff-398029ac53c7-kserve-provision-location\") pod \"51b65675-57de-46c2-b8ff-398029ac53c7\" (UID: \"51b65675-57de-46c2-b8ff-398029ac53c7\") "
Apr 24 14:47:12.794553 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:47:12.794515    2574 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/51b65675-57de-46c2-b8ff-398029ac53c7-tls-certs\") pod \"51b65675-57de-46c2-b8ff-398029ac53c7\" (UID: \"51b65675-57de-46c2-b8ff-398029ac53c7\") "
Apr 24 14:47:12.794553 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:47:12.794550    2574 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"kube-api-access-bxwz2\" (UniqueName: \"kubernetes.io/projected/51b65675-57de-46c2-b8ff-398029ac53c7-kube-api-access-bxwz2\") pod \"51b65675-57de-46c2-b8ff-398029ac53c7\" (UID: \"51b65675-57de-46c2-b8ff-398029ac53c7\") "
Apr 24 14:47:12.794837 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:47:12.794586    2574 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"tokenizer-uds\" (UniqueName: \"kubernetes.io/empty-dir/51b65675-57de-46c2-b8ff-398029ac53c7-tokenizer-uds\") pod \"51b65675-57de-46c2-b8ff-398029ac53c7\" (UID: \"51b65675-57de-46c2-b8ff-398029ac53c7\") "
Apr 24 14:47:12.794837 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:47:12.794625    2574 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"tokenizer-cache\" (UniqueName: \"kubernetes.io/empty-dir/51b65675-57de-46c2-b8ff-398029ac53c7-tokenizer-cache\") pod \"51b65675-57de-46c2-b8ff-398029ac53c7\" (UID: \"51b65675-57de-46c2-b8ff-398029ac53c7\") "
Apr 24 14:47:12.794837 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:47:12.794651    2574 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"tokenizer-tmp\" (UniqueName: \"kubernetes.io/empty-dir/51b65675-57de-46c2-b8ff-398029ac53c7-tokenizer-tmp\") pod \"51b65675-57de-46c2-b8ff-398029ac53c7\" (UID: \"51b65675-57de-46c2-b8ff-398029ac53c7\") "
Apr 24 14:47:12.795004 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:47:12.794898    2574 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/51b65675-57de-46c2-b8ff-398029ac53c7-tokenizer-cache" (OuterVolumeSpecName: "tokenizer-cache") pod "51b65675-57de-46c2-b8ff-398029ac53c7" (UID: "51b65675-57de-46c2-b8ff-398029ac53c7"). InnerVolumeSpecName "tokenizer-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Apr 24 14:47:12.795004 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:47:12.794906    2574 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/51b65675-57de-46c2-b8ff-398029ac53c7-tokenizer-uds" (OuterVolumeSpecName: "tokenizer-uds") pod "51b65675-57de-46c2-b8ff-398029ac53c7" (UID: "51b65675-57de-46c2-b8ff-398029ac53c7"). InnerVolumeSpecName "tokenizer-uds". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Apr 24 14:47:12.795111 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:47:12.795023    2574 reconciler_common.go:299] "Volume detached for volume \"tokenizer-uds\" (UniqueName: \"kubernetes.io/empty-dir/51b65675-57de-46c2-b8ff-398029ac53c7-tokenizer-uds\") on node \"ip-10-0-136-132.ec2.internal\" DevicePath \"\""
Apr 24 14:47:12.795111 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:47:12.795042    2574 reconciler_common.go:299] "Volume detached for volume \"tokenizer-cache\" (UniqueName: \"kubernetes.io/empty-dir/51b65675-57de-46c2-b8ff-398029ac53c7-tokenizer-cache\") on node \"ip-10-0-136-132.ec2.internal\" DevicePath \"\""
Apr 24 14:47:12.795326 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:47:12.795302    2574 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/51b65675-57de-46c2-b8ff-398029ac53c7-kserve-provision-location" (OuterVolumeSpecName: "kserve-provision-location") pod "51b65675-57de-46c2-b8ff-398029ac53c7" (UID: "51b65675-57de-46c2-b8ff-398029ac53c7"). InnerVolumeSpecName "kserve-provision-location". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Apr 24 14:47:12.795535 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:47:12.795511    2574 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/51b65675-57de-46c2-b8ff-398029ac53c7-tokenizer-tmp" (OuterVolumeSpecName: "tokenizer-tmp") pod "51b65675-57de-46c2-b8ff-398029ac53c7" (UID: "51b65675-57de-46c2-b8ff-398029ac53c7"). InnerVolumeSpecName "tokenizer-tmp". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Apr 24 14:47:12.796777 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:47:12.796756    2574 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/51b65675-57de-46c2-b8ff-398029ac53c7-kube-api-access-bxwz2" (OuterVolumeSpecName: "kube-api-access-bxwz2") pod "51b65675-57de-46c2-b8ff-398029ac53c7" (UID: "51b65675-57de-46c2-b8ff-398029ac53c7"). InnerVolumeSpecName "kube-api-access-bxwz2". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Apr 24 14:47:12.796883 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:47:12.796791    2574 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/51b65675-57de-46c2-b8ff-398029ac53c7-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "51b65675-57de-46c2-b8ff-398029ac53c7" (UID: "51b65675-57de-46c2-b8ff-398029ac53c7"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Apr 24 14:47:12.896290 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:47:12.896237    2574 reconciler_common.go:299] "Volume detached for volume \"tokenizer-tmp\" (UniqueName: \"kubernetes.io/empty-dir/51b65675-57de-46c2-b8ff-398029ac53c7-tokenizer-tmp\") on node \"ip-10-0-136-132.ec2.internal\" DevicePath \"\""
Apr 24 14:47:12.896290 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:47:12.896284    2574 reconciler_common.go:299] "Volume detached for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/51b65675-57de-46c2-b8ff-398029ac53c7-kserve-provision-location\") on node \"ip-10-0-136-132.ec2.internal\" DevicePath \"\""
Apr 24 14:47:12.896290 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:47:12.896296    2574 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/51b65675-57de-46c2-b8ff-398029ac53c7-tls-certs\") on node \"ip-10-0-136-132.ec2.internal\" DevicePath \"\""
Apr 24 14:47:12.896572 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:47:12.896305    2574 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-bxwz2\" (UniqueName: \"kubernetes.io/projected/51b65675-57de-46c2-b8ff-398029ac53c7-kube-api-access-bxwz2\") on node \"ip-10-0-136-132.ec2.internal\" DevicePath \"\""
Apr 24 14:47:13.188615 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:47:13.188578    2574 generic.go:358] "Generic (PLEG): container finished" podID="51b65675-57de-46c2-b8ff-398029ac53c7" containerID="4c1c922dd5365754272bc67c340cac6a2f714319afd9850df1cafcf3ccba7ae9" exitCode=0
Apr 24 14:47:13.188801 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:47:13.188672    2574 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/stop-feature-test-kserve-router-scheduler-7cb4fb9c5d-4m59m"
Apr 24 14:47:13.188801 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:47:13.188672    2574 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/stop-feature-test-kserve-router-scheduler-7cb4fb9c5d-4m59m" event={"ID":"51b65675-57de-46c2-b8ff-398029ac53c7","Type":"ContainerDied","Data":"4c1c922dd5365754272bc67c340cac6a2f714319afd9850df1cafcf3ccba7ae9"}
Apr 24 14:47:13.188801 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:47:13.188716    2574 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/stop-feature-test-kserve-router-scheduler-7cb4fb9c5d-4m59m" event={"ID":"51b65675-57de-46c2-b8ff-398029ac53c7","Type":"ContainerDied","Data":"a9fec223473e140f9edec21089c29e7c60c8ab1e68109f2e86937479a381d2f9"}
Apr 24 14:47:13.188801 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:47:13.188729    2574 scope.go:117] "RemoveContainer" containerID="4c1c922dd5365754272bc67c340cac6a2f714319afd9850df1cafcf3ccba7ae9"
Apr 24 14:47:13.198739 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:47:13.198721    2574 scope.go:117] "RemoveContainer" containerID="0a718d3f4431ec2090a1cc7fa0d815d590354811e52a3025a64e4a33be5078a5"
Apr 24 14:47:13.206831 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:47:13.206811    2574 scope.go:117] "RemoveContainer" containerID="f379575de8b1a99f3b4171e971aaaa1ce800350b914485be4910fbc6e9c2973a"
Apr 24 14:47:13.211946 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:47:13.211923    2574 kubelet.go:2553] "SyncLoop DELETE" source="api" pods=["kserve-ci-e2e-test/stop-feature-test-kserve-router-scheduler-7cb4fb9c5d-4m59m"]
Apr 24 14:47:13.215576 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:47:13.215555    2574 kubelet.go:2547] "SyncLoop REMOVE" source="api" pods=["kserve-ci-e2e-test/stop-feature-test-kserve-router-scheduler-7cb4fb9c5d-4m59m"]
Apr 24 14:47:13.215994 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:47:13.215965    2574 scope.go:117] "RemoveContainer" containerID="4c1c922dd5365754272bc67c340cac6a2f714319afd9850df1cafcf3ccba7ae9"
Apr 24 14:47:13.216202 ip-10-0-136-132 kubenswrapper[2574]: E0424 14:47:13.216183    2574 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"4c1c922dd5365754272bc67c340cac6a2f714319afd9850df1cafcf3ccba7ae9\": container with ID starting with 4c1c922dd5365754272bc67c340cac6a2f714319afd9850df1cafcf3ccba7ae9 not found: ID does not exist" containerID="4c1c922dd5365754272bc67c340cac6a2f714319afd9850df1cafcf3ccba7ae9"
Apr 24 14:47:13.216345 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:47:13.216213    2574 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"4c1c922dd5365754272bc67c340cac6a2f714319afd9850df1cafcf3ccba7ae9"} err="failed to get container status \"4c1c922dd5365754272bc67c340cac6a2f714319afd9850df1cafcf3ccba7ae9\": rpc error: code = NotFound desc = could not find container \"4c1c922dd5365754272bc67c340cac6a2f714319afd9850df1cafcf3ccba7ae9\": container with ID starting with 4c1c922dd5365754272bc67c340cac6a2f714319afd9850df1cafcf3ccba7ae9 not found: ID does not exist"
Apr 24 14:47:13.216345 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:47:13.216239    2574 scope.go:117] "RemoveContainer" containerID="0a718d3f4431ec2090a1cc7fa0d815d590354811e52a3025a64e4a33be5078a5"
Apr 24 14:47:13.216513 ip-10-0-136-132 kubenswrapper[2574]: E0424 14:47:13.216496    2574 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"0a718d3f4431ec2090a1cc7fa0d815d590354811e52a3025a64e4a33be5078a5\": container with ID starting with 0a718d3f4431ec2090a1cc7fa0d815d590354811e52a3025a64e4a33be5078a5 not found: ID does not exist" containerID="0a718d3f4431ec2090a1cc7fa0d815d590354811e52a3025a64e4a33be5078a5"
Apr 24 14:47:13.216554 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:47:13.216518    2574 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"0a718d3f4431ec2090a1cc7fa0d815d590354811e52a3025a64e4a33be5078a5"} err="failed to get container status \"0a718d3f4431ec2090a1cc7fa0d815d590354811e52a3025a64e4a33be5078a5\": rpc error: code = NotFound desc = could not find container \"0a718d3f4431ec2090a1cc7fa0d815d590354811e52a3025a64e4a33be5078a5\": container with ID starting with 0a718d3f4431ec2090a1cc7fa0d815d590354811e52a3025a64e4a33be5078a5 not found: ID does not exist"
Apr 24 14:47:13.216554 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:47:13.216533    2574 scope.go:117] "RemoveContainer" containerID="f379575de8b1a99f3b4171e971aaaa1ce800350b914485be4910fbc6e9c2973a"
Apr 24 14:47:13.216757 ip-10-0-136-132 kubenswrapper[2574]: E0424 14:47:13.216739    2574 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"f379575de8b1a99f3b4171e971aaaa1ce800350b914485be4910fbc6e9c2973a\": container with ID starting with f379575de8b1a99f3b4171e971aaaa1ce800350b914485be4910fbc6e9c2973a not found: ID does not exist" containerID="f379575de8b1a99f3b4171e971aaaa1ce800350b914485be4910fbc6e9c2973a"
Apr 24 14:47:13.216811 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:47:13.216765    2574 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"f379575de8b1a99f3b4171e971aaaa1ce800350b914485be4910fbc6e9c2973a"} err="failed to get container status \"f379575de8b1a99f3b4171e971aaaa1ce800350b914485be4910fbc6e9c2973a\": rpc error: code = NotFound desc = could not find container \"f379575de8b1a99f3b4171e971aaaa1ce800350b914485be4910fbc6e9c2973a\": container with ID starting with f379575de8b1a99f3b4171e971aaaa1ce800350b914485be4910fbc6e9c2973a not found: ID does not exist"
Apr 24 14:47:13.770512 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:47:13.770478    2574 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="51b65675-57de-46c2-b8ff-398029ac53c7" path="/var/lib/kubelet/pods/51b65675-57de-46c2-b8ff-398029ac53c7/volumes"
Apr 24 14:47:41.656164 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:47:41.656141    2574 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_stop-feature-test-kserve-88fb48646-7mxhg_0e6568d3-aa47-482c-b6d2-ab032f34cc79/main/0.log"
Apr 24 14:47:41.656571 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:47:41.656555    2574 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/stop-feature-test-kserve-88fb48646-7mxhg"
Apr 24 14:47:41.742407 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:47:41.742307    2574 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"kube-api-access-rpll9\" (UniqueName: \"kubernetes.io/projected/0e6568d3-aa47-482c-b6d2-ab032f34cc79-kube-api-access-rpll9\") pod \"0e6568d3-aa47-482c-b6d2-ab032f34cc79\" (UID: \"0e6568d3-aa47-482c-b6d2-ab032f34cc79\") "
Apr 24 14:47:41.742407 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:47:41.742365    2574 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/0e6568d3-aa47-482c-b6d2-ab032f34cc79-tls-certs\") pod \"0e6568d3-aa47-482c-b6d2-ab032f34cc79\" (UID: \"0e6568d3-aa47-482c-b6d2-ab032f34cc79\") "
Apr 24 14:47:41.742407 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:47:41.742392    2574 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/0e6568d3-aa47-482c-b6d2-ab032f34cc79-model-cache\") pod \"0e6568d3-aa47-482c-b6d2-ab032f34cc79\" (UID: \"0e6568d3-aa47-482c-b6d2-ab032f34cc79\") "
Apr 24 14:47:41.742691 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:47:41.742428    2574 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/0e6568d3-aa47-482c-b6d2-ab032f34cc79-kserve-provision-location\") pod \"0e6568d3-aa47-482c-b6d2-ab032f34cc79\" (UID: \"0e6568d3-aa47-482c-b6d2-ab032f34cc79\") "
Apr 24 14:47:41.742691 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:47:41.742467    2574 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/0e6568d3-aa47-482c-b6d2-ab032f34cc79-dshm\") pod \"0e6568d3-aa47-482c-b6d2-ab032f34cc79\" (UID: \"0e6568d3-aa47-482c-b6d2-ab032f34cc79\") "
Apr 24 14:47:41.742691 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:47:41.742494    2574 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/0e6568d3-aa47-482c-b6d2-ab032f34cc79-home\") pod \"0e6568d3-aa47-482c-b6d2-ab032f34cc79\" (UID: \"0e6568d3-aa47-482c-b6d2-ab032f34cc79\") "
Apr 24 14:47:41.742839 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:47:41.742708    2574 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/0e6568d3-aa47-482c-b6d2-ab032f34cc79-model-cache" (OuterVolumeSpecName: "model-cache") pod "0e6568d3-aa47-482c-b6d2-ab032f34cc79" (UID: "0e6568d3-aa47-482c-b6d2-ab032f34cc79"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Apr 24 14:47:41.742839 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:47:41.742813    2574 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/0e6568d3-aa47-482c-b6d2-ab032f34cc79-model-cache\") on node \"ip-10-0-136-132.ec2.internal\" DevicePath \"\""
Apr 24 14:47:41.742918 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:47:41.742874    2574 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/0e6568d3-aa47-482c-b6d2-ab032f34cc79-home" (OuterVolumeSpecName: "home") pod "0e6568d3-aa47-482c-b6d2-ab032f34cc79" (UID: "0e6568d3-aa47-482c-b6d2-ab032f34cc79"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Apr 24 14:47:41.744503 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:47:41.744470    2574 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/0e6568d3-aa47-482c-b6d2-ab032f34cc79-dshm" (OuterVolumeSpecName: "dshm") pod "0e6568d3-aa47-482c-b6d2-ab032f34cc79" (UID: "0e6568d3-aa47-482c-b6d2-ab032f34cc79"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Apr 24 14:47:41.744611 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:47:41.744524    2574 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/0e6568d3-aa47-482c-b6d2-ab032f34cc79-kube-api-access-rpll9" (OuterVolumeSpecName: "kube-api-access-rpll9") pod "0e6568d3-aa47-482c-b6d2-ab032f34cc79" (UID: "0e6568d3-aa47-482c-b6d2-ab032f34cc79"). InnerVolumeSpecName "kube-api-access-rpll9". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Apr 24 14:47:41.744826 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:47:41.744808    2574 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/0e6568d3-aa47-482c-b6d2-ab032f34cc79-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "0e6568d3-aa47-482c-b6d2-ab032f34cc79" (UID: "0e6568d3-aa47-482c-b6d2-ab032f34cc79"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Apr 24 14:47:41.802577 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:47:41.802521    2574 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/0e6568d3-aa47-482c-b6d2-ab032f34cc79-kserve-provision-location" (OuterVolumeSpecName: "kserve-provision-location") pod "0e6568d3-aa47-482c-b6d2-ab032f34cc79" (UID: "0e6568d3-aa47-482c-b6d2-ab032f34cc79"). InnerVolumeSpecName "kserve-provision-location". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Apr 24 14:47:41.844020 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:47:41.843979    2574 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/0e6568d3-aa47-482c-b6d2-ab032f34cc79-dshm\") on node \"ip-10-0-136-132.ec2.internal\" DevicePath \"\""
Apr 24 14:47:41.844020 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:47:41.844009    2574 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/0e6568d3-aa47-482c-b6d2-ab032f34cc79-home\") on node \"ip-10-0-136-132.ec2.internal\" DevicePath \"\""
Apr 24 14:47:41.844020 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:47:41.844023    2574 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-rpll9\" (UniqueName: \"kubernetes.io/projected/0e6568d3-aa47-482c-b6d2-ab032f34cc79-kube-api-access-rpll9\") on node \"ip-10-0-136-132.ec2.internal\" DevicePath \"\""
Apr 24 14:47:41.844274 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:47:41.844039    2574 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/0e6568d3-aa47-482c-b6d2-ab032f34cc79-tls-certs\") on node \"ip-10-0-136-132.ec2.internal\" DevicePath \"\""
Apr 24 14:47:41.844274 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:47:41.844054    2574 reconciler_common.go:299] "Volume detached for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/0e6568d3-aa47-482c-b6d2-ab032f34cc79-kserve-provision-location\") on node \"ip-10-0-136-132.ec2.internal\" DevicePath \"\""
Apr 24 14:47:42.305925 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:47:42.305897    2574 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_stop-feature-test-kserve-88fb48646-7mxhg_0e6568d3-aa47-482c-b6d2-ab032f34cc79/main/0.log"
Apr 24 14:47:42.306224 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:47:42.306200    2574 generic.go:358] "Generic (PLEG): container finished" podID="0e6568d3-aa47-482c-b6d2-ab032f34cc79" containerID="f3614809bccf1ef3b90baf950e6552c818b6bcb298034f26ab3f0f9e622553c6" exitCode=137
Apr 24 14:47:42.306360 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:47:42.306267    2574 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/stop-feature-test-kserve-88fb48646-7mxhg" event={"ID":"0e6568d3-aa47-482c-b6d2-ab032f34cc79","Type":"ContainerDied","Data":"f3614809bccf1ef3b90baf950e6552c818b6bcb298034f26ab3f0f9e622553c6"}
Apr 24 14:47:42.306360 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:47:42.306275    2574 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/stop-feature-test-kserve-88fb48646-7mxhg"
Apr 24 14:47:42.306360 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:47:42.306302    2574 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/stop-feature-test-kserve-88fb48646-7mxhg" event={"ID":"0e6568d3-aa47-482c-b6d2-ab032f34cc79","Type":"ContainerDied","Data":"e241dc709fd6371e2323a9f265870d768d82cb930ccca417431f11985efcceda"}
Apr 24 14:47:42.306360 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:47:42.306322    2574 scope.go:117] "RemoveContainer" containerID="f3614809bccf1ef3b90baf950e6552c818b6bcb298034f26ab3f0f9e622553c6"
Apr 24 14:47:42.325513 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:47:42.325494    2574 scope.go:117] "RemoveContainer" containerID="98299b8ddd2f9e10c32a75b1a223f115a4f4a42c6394303606d24611c45c6f10"
Apr 24 14:47:42.328533 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:47:42.328510    2574 kubelet.go:2553] "SyncLoop DELETE" source="api" pods=["kserve-ci-e2e-test/stop-feature-test-kserve-88fb48646-7mxhg"]
Apr 24 14:47:42.331495 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:47:42.331473    2574 kubelet.go:2547] "SyncLoop REMOVE" source="api" pods=["kserve-ci-e2e-test/stop-feature-test-kserve-88fb48646-7mxhg"]
Apr 24 14:47:42.392282 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:47:42.392262    2574 scope.go:117] "RemoveContainer" containerID="f3614809bccf1ef3b90baf950e6552c818b6bcb298034f26ab3f0f9e622553c6"
Apr 24 14:47:42.392681 ip-10-0-136-132 kubenswrapper[2574]: E0424 14:47:42.392657    2574 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"f3614809bccf1ef3b90baf950e6552c818b6bcb298034f26ab3f0f9e622553c6\": container with ID starting with f3614809bccf1ef3b90baf950e6552c818b6bcb298034f26ab3f0f9e622553c6 not found: ID does not exist" containerID="f3614809bccf1ef3b90baf950e6552c818b6bcb298034f26ab3f0f9e622553c6"
Apr 24 14:47:42.392779 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:47:42.392691    2574 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"f3614809bccf1ef3b90baf950e6552c818b6bcb298034f26ab3f0f9e622553c6"} err="failed to get container status \"f3614809bccf1ef3b90baf950e6552c818b6bcb298034f26ab3f0f9e622553c6\": rpc error: code = NotFound desc = could not find container \"f3614809bccf1ef3b90baf950e6552c818b6bcb298034f26ab3f0f9e622553c6\": container with ID starting with f3614809bccf1ef3b90baf950e6552c818b6bcb298034f26ab3f0f9e622553c6 not found: ID does not exist"
Apr 24 14:47:42.392779 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:47:42.392711    2574 scope.go:117] "RemoveContainer" containerID="98299b8ddd2f9e10c32a75b1a223f115a4f4a42c6394303606d24611c45c6f10"
Apr 24 14:47:42.393027 ip-10-0-136-132 kubenswrapper[2574]: E0424 14:47:42.393005    2574 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"98299b8ddd2f9e10c32a75b1a223f115a4f4a42c6394303606d24611c45c6f10\": container with ID starting with 98299b8ddd2f9e10c32a75b1a223f115a4f4a42c6394303606d24611c45c6f10 not found: ID does not exist" containerID="98299b8ddd2f9e10c32a75b1a223f115a4f4a42c6394303606d24611c45c6f10"
Apr 24 14:47:42.393069 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:47:42.393038    2574 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"98299b8ddd2f9e10c32a75b1a223f115a4f4a42c6394303606d24611c45c6f10"} err="failed to get container status \"98299b8ddd2f9e10c32a75b1a223f115a4f4a42c6394303606d24611c45c6f10\": rpc error: code = NotFound desc = could not find container \"98299b8ddd2f9e10c32a75b1a223f115a4f4a42c6394303606d24611c45c6f10\": container with ID starting with 98299b8ddd2f9e10c32a75b1a223f115a4f4a42c6394303606d24611c45c6f10 not found: ID does not exist"
Apr 24 14:47:43.770801 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:47:43.770768    2574 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="0e6568d3-aa47-482c-b6d2-ab032f34cc79" path="/var/lib/kubelet/pods/0e6568d3-aa47-482c-b6d2-ab032f34cc79/volumes"
Apr 24 14:48:17.758459 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:48:17.758427    2574 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["kserve-ci-e2e-test/llmisvc-model-fb-opt-125m-route-02ac88e7-kserve-7969569c4dsmh7x"]
Apr 24 14:48:17.758922 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:48:17.758797    2574 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b9251f52-2109-4405-8651-0a0f450355d5" containerName="main"
Apr 24 14:48:17.758922 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:48:17.758808    2574 state_mem.go:107] "Deleted CPUSet assignment" podUID="b9251f52-2109-4405-8651-0a0f450355d5" containerName="main"
Apr 24 14:48:17.758922 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:48:17.758822    2574 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="51b65675-57de-46c2-b8ff-398029ac53c7" containerName="tokenizer"
Apr 24 14:48:17.758922 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:48:17.758828    2574 state_mem.go:107] "Deleted CPUSet assignment" podUID="51b65675-57de-46c2-b8ff-398029ac53c7" containerName="tokenizer"
Apr 24 14:48:17.758922 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:48:17.758834    2574 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="51b65675-57de-46c2-b8ff-398029ac53c7" containerName="storage-initializer"
Apr 24 14:48:17.758922 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:48:17.758839    2574 state_mem.go:107] "Deleted CPUSet assignment" podUID="51b65675-57de-46c2-b8ff-398029ac53c7" containerName="storage-initializer"
Apr 24 14:48:17.758922 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:48:17.758846    2574 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="0e6568d3-aa47-482c-b6d2-ab032f34cc79" containerName="main"
Apr 24 14:48:17.758922 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:48:17.758851    2574 state_mem.go:107] "Deleted CPUSet assignment" podUID="0e6568d3-aa47-482c-b6d2-ab032f34cc79" containerName="main"
Apr 24 14:48:17.758922 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:48:17.758860    2574 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b9251f52-2109-4405-8651-0a0f450355d5" containerName="storage-initializer"
Apr 24 14:48:17.758922 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:48:17.758865    2574 state_mem.go:107] "Deleted CPUSet assignment" podUID="b9251f52-2109-4405-8651-0a0f450355d5" containerName="storage-initializer"
Apr 24 14:48:17.758922 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:48:17.758871    2574 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="51b65675-57de-46c2-b8ff-398029ac53c7" containerName="main"
Apr 24 14:48:17.758922 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:48:17.758876    2574 state_mem.go:107] "Deleted CPUSet assignment" podUID="51b65675-57de-46c2-b8ff-398029ac53c7" containerName="main"
Apr 24 14:48:17.758922 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:48:17.758886    2574 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="0e6568d3-aa47-482c-b6d2-ab032f34cc79" containerName="storage-initializer"
Apr 24 14:48:17.758922 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:48:17.758891    2574 state_mem.go:107] "Deleted CPUSet assignment" podUID="0e6568d3-aa47-482c-b6d2-ab032f34cc79" containerName="storage-initializer"
Apr 24 14:48:17.759357 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:48:17.758952    2574 memory_manager.go:356] "RemoveStaleState removing state" podUID="51b65675-57de-46c2-b8ff-398029ac53c7" containerName="tokenizer"
Apr 24 14:48:17.759357 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:48:17.758961    2574 memory_manager.go:356] "RemoveStaleState removing state" podUID="0e6568d3-aa47-482c-b6d2-ab032f34cc79" containerName="main"
Apr 24 14:48:17.759357 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:48:17.758968    2574 memory_manager.go:356] "RemoveStaleState removing state" podUID="b9251f52-2109-4405-8651-0a0f450355d5" containerName="main"
Apr 24 14:48:17.759357 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:48:17.758975    2574 memory_manager.go:356] "RemoveStaleState removing state" podUID="51b65675-57de-46c2-b8ff-398029ac53c7" containerName="main"
Apr 24 14:48:17.761514 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:48:17.761497    2574 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/llmisvc-model-fb-opt-125m-route-02ac88e7-kserve-7969569c4dsmh7x"
Apr 24 14:48:17.763581 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:48:17.763547    2574 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kserve-ci-e2e-test\"/\"kube-root-ca.crt\""
Apr 24 14:48:17.763581 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:48:17.763563    2574 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kserve-ci-e2e-test\"/\"openshift-service-ca.crt\""
Apr 24 14:48:17.763759 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:48:17.763593    2574 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"kserve-ci-e2e-test\"/\"default-dockercfg-92j8s\""
Apr 24 14:48:17.763811 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:48:17.763776    2574 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"kserve-ci-e2e-test\"/\"llmisv8de1d74aab16d9cabd8b5aafeb5248e8-kserve-self-signed-certs\""
Apr 24 14:48:17.771979 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:48:17.771957    2574 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/llmisvc-model-fb-opt-125m-route-02ac88e7-kserve-7969569c4dsmh7x"]
Apr 24 14:48:17.856188 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:48:17.856155    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/5fc35206-739e-42b8-b564-7a905fa24886-model-cache\") pod \"llmisvc-model-fb-opt-125m-route-02ac88e7-kserve-7969569c4dsmh7x\" (UID: \"5fc35206-739e-42b8-b564-7a905fa24886\") " pod="kserve-ci-e2e-test/llmisvc-model-fb-opt-125m-route-02ac88e7-kserve-7969569c4dsmh7x"
Apr 24 14:48:17.856376 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:48:17.856201    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/5fc35206-739e-42b8-b564-7a905fa24886-dshm\") pod \"llmisvc-model-fb-opt-125m-route-02ac88e7-kserve-7969569c4dsmh7x\" (UID: \"5fc35206-739e-42b8-b564-7a905fa24886\") " pod="kserve-ci-e2e-test/llmisvc-model-fb-opt-125m-route-02ac88e7-kserve-7969569c4dsmh7x"
Apr 24 14:48:17.856376 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:48:17.856224    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/5fc35206-739e-42b8-b564-7a905fa24886-tls-certs\") pod \"llmisvc-model-fb-opt-125m-route-02ac88e7-kserve-7969569c4dsmh7x\" (UID: \"5fc35206-739e-42b8-b564-7a905fa24886\") " pod="kserve-ci-e2e-test/llmisvc-model-fb-opt-125m-route-02ac88e7-kserve-7969569c4dsmh7x"
Apr 24 14:48:17.856376 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:48:17.856317    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/5fc35206-739e-42b8-b564-7a905fa24886-kserve-provision-location\") pod \"llmisvc-model-fb-opt-125m-route-02ac88e7-kserve-7969569c4dsmh7x\" (UID: \"5fc35206-739e-42b8-b564-7a905fa24886\") " pod="kserve-ci-e2e-test/llmisvc-model-fb-opt-125m-route-02ac88e7-kserve-7969569c4dsmh7x"
Apr 24 14:48:17.856489 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:48:17.856381    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/5fc35206-739e-42b8-b564-7a905fa24886-home\") pod \"llmisvc-model-fb-opt-125m-route-02ac88e7-kserve-7969569c4dsmh7x\" (UID: \"5fc35206-739e-42b8-b564-7a905fa24886\") " pod="kserve-ci-e2e-test/llmisvc-model-fb-opt-125m-route-02ac88e7-kserve-7969569c4dsmh7x"
Apr 24 14:48:17.856526 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:48:17.856491    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ll9zn\" (UniqueName: \"kubernetes.io/projected/5fc35206-739e-42b8-b564-7a905fa24886-kube-api-access-ll9zn\") pod \"llmisvc-model-fb-opt-125m-route-02ac88e7-kserve-7969569c4dsmh7x\" (UID: \"5fc35206-739e-42b8-b564-7a905fa24886\") " pod="kserve-ci-e2e-test/llmisvc-model-fb-opt-125m-route-02ac88e7-kserve-7969569c4dsmh7x"
Apr 24 14:48:17.957188 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:48:17.957160    2574 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-ll9zn\" (UniqueName: \"kubernetes.io/projected/5fc35206-739e-42b8-b564-7a905fa24886-kube-api-access-ll9zn\") pod \"llmisvc-model-fb-opt-125m-route-02ac88e7-kserve-7969569c4dsmh7x\" (UID: \"5fc35206-739e-42b8-b564-7a905fa24886\") " pod="kserve-ci-e2e-test/llmisvc-model-fb-opt-125m-route-02ac88e7-kserve-7969569c4dsmh7x"
Apr 24 14:48:17.957311 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:48:17.957195    2574 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/5fc35206-739e-42b8-b564-7a905fa24886-model-cache\") pod \"llmisvc-model-fb-opt-125m-route-02ac88e7-kserve-7969569c4dsmh7x\" (UID: \"5fc35206-739e-42b8-b564-7a905fa24886\") " pod="kserve-ci-e2e-test/llmisvc-model-fb-opt-125m-route-02ac88e7-kserve-7969569c4dsmh7x"
Apr 24 14:48:17.957311 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:48:17.957218    2574 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/5fc35206-739e-42b8-b564-7a905fa24886-dshm\") pod \"llmisvc-model-fb-opt-125m-route-02ac88e7-kserve-7969569c4dsmh7x\" (UID: \"5fc35206-739e-42b8-b564-7a905fa24886\") " pod="kserve-ci-e2e-test/llmisvc-model-fb-opt-125m-route-02ac88e7-kserve-7969569c4dsmh7x"
Apr 24 14:48:17.957311 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:48:17.957236    2574 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/5fc35206-739e-42b8-b564-7a905fa24886-tls-certs\") pod \"llmisvc-model-fb-opt-125m-route-02ac88e7-kserve-7969569c4dsmh7x\" (UID: \"5fc35206-739e-42b8-b564-7a905fa24886\") " pod="kserve-ci-e2e-test/llmisvc-model-fb-opt-125m-route-02ac88e7-kserve-7969569c4dsmh7x"
Apr 24 14:48:17.957311 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:48:17.957267    2574 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/5fc35206-739e-42b8-b564-7a905fa24886-kserve-provision-location\") pod \"llmisvc-model-fb-opt-125m-route-02ac88e7-kserve-7969569c4dsmh7x\" (UID: \"5fc35206-739e-42b8-b564-7a905fa24886\") " pod="kserve-ci-e2e-test/llmisvc-model-fb-opt-125m-route-02ac88e7-kserve-7969569c4dsmh7x"
Apr 24 14:48:17.957541 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:48:17.957372    2574 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/5fc35206-739e-42b8-b564-7a905fa24886-home\") pod \"llmisvc-model-fb-opt-125m-route-02ac88e7-kserve-7969569c4dsmh7x\" (UID: \"5fc35206-739e-42b8-b564-7a905fa24886\") " pod="kserve-ci-e2e-test/llmisvc-model-fb-opt-125m-route-02ac88e7-kserve-7969569c4dsmh7x"
Apr 24 14:48:17.957650 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:48:17.957619    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/5fc35206-739e-42b8-b564-7a905fa24886-model-cache\") pod \"llmisvc-model-fb-opt-125m-route-02ac88e7-kserve-7969569c4dsmh7x\" (UID: \"5fc35206-739e-42b8-b564-7a905fa24886\") " pod="kserve-ci-e2e-test/llmisvc-model-fb-opt-125m-route-02ac88e7-kserve-7969569c4dsmh7x"
Apr 24 14:48:17.957779 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:48:17.957726    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/5fc35206-739e-42b8-b564-7a905fa24886-kserve-provision-location\") pod \"llmisvc-model-fb-opt-125m-route-02ac88e7-kserve-7969569c4dsmh7x\" (UID: \"5fc35206-739e-42b8-b564-7a905fa24886\") " pod="kserve-ci-e2e-test/llmisvc-model-fb-opt-125m-route-02ac88e7-kserve-7969569c4dsmh7x"
Apr 24 14:48:17.957779 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:48:17.957757    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/5fc35206-739e-42b8-b564-7a905fa24886-home\") pod \"llmisvc-model-fb-opt-125m-route-02ac88e7-kserve-7969569c4dsmh7x\" (UID: \"5fc35206-739e-42b8-b564-7a905fa24886\") " pod="kserve-ci-e2e-test/llmisvc-model-fb-opt-125m-route-02ac88e7-kserve-7969569c4dsmh7x"
Apr 24 14:48:17.959459 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:48:17.959440    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/5fc35206-739e-42b8-b564-7a905fa24886-dshm\") pod \"llmisvc-model-fb-opt-125m-route-02ac88e7-kserve-7969569c4dsmh7x\" (UID: \"5fc35206-739e-42b8-b564-7a905fa24886\") " pod="kserve-ci-e2e-test/llmisvc-model-fb-opt-125m-route-02ac88e7-kserve-7969569c4dsmh7x"
Apr 24 14:48:17.959653 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:48:17.959638    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/5fc35206-739e-42b8-b564-7a905fa24886-tls-certs\") pod \"llmisvc-model-fb-opt-125m-route-02ac88e7-kserve-7969569c4dsmh7x\" (UID: \"5fc35206-739e-42b8-b564-7a905fa24886\") " pod="kserve-ci-e2e-test/llmisvc-model-fb-opt-125m-route-02ac88e7-kserve-7969569c4dsmh7x"
Apr 24 14:48:17.964664 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:48:17.964637    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-ll9zn\" (UniqueName: \"kubernetes.io/projected/5fc35206-739e-42b8-b564-7a905fa24886-kube-api-access-ll9zn\") pod \"llmisvc-model-fb-opt-125m-route-02ac88e7-kserve-7969569c4dsmh7x\" (UID: \"5fc35206-739e-42b8-b564-7a905fa24886\") " pod="kserve-ci-e2e-test/llmisvc-model-fb-opt-125m-route-02ac88e7-kserve-7969569c4dsmh7x"
Apr 24 14:48:18.074665 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:48:18.074603    2574 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/llmisvc-model-fb-opt-125m-route-02ac88e7-kserve-7969569c4dsmh7x"
Apr 24 14:48:18.211130 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:48:18.211093    2574 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/llmisvc-model-fb-opt-125m-route-02ac88e7-kserve-7969569c4dsmh7x"]
Apr 24 14:48:18.216540 ip-10-0-136-132 kubenswrapper[2574]: W0424 14:48:18.216513    2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5fc35206_739e_42b8_b564_7a905fa24886.slice/crio-1725ffd64f9b231d6fc04a9973c4da81ef20d7e2734705886ad2fac6e0814349 WatchSource:0}: Error finding container 1725ffd64f9b231d6fc04a9973c4da81ef20d7e2734705886ad2fac6e0814349: Status 404 returned error can't find the container with id 1725ffd64f9b231d6fc04a9973c4da81ef20d7e2734705886ad2fac6e0814349
Apr 24 14:48:18.451720 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:48:18.451685    2574 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/llmisvc-model-fb-opt-125m-route-02ac88e7-kserve-7969569c4dsmh7x" event={"ID":"5fc35206-739e-42b8-b564-7a905fa24886","Type":"ContainerStarted","Data":"16d60a6b728eb8469fc320b8c8adaebdfb431a2a000c2e475a6292f789e331f3"}
Apr 24 14:48:18.451720 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:48:18.451723    2574 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/llmisvc-model-fb-opt-125m-route-02ac88e7-kserve-7969569c4dsmh7x" event={"ID":"5fc35206-739e-42b8-b564-7a905fa24886","Type":"ContainerStarted","Data":"1725ffd64f9b231d6fc04a9973c4da81ef20d7e2734705886ad2fac6e0814349"}
Apr 24 14:48:57.837911 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:48:57.837877    2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-9d4b6777b-f6ngr_2571829f-666a-422a-9b02-2d2bb4c89b25/console-operator/1.log"
Apr 24 14:48:57.841894 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:48:57.841871    2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-9d4b6777b-f6ngr_2571829f-666a-422a-9b02-2d2bb4c89b25/console-operator/1.log"
Apr 24 14:48:57.842729 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:48:57.842707    2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-w5x5x_44036c3e-1fdc-40d4-899c-9009cd4589e3/ovn-acl-logging/0.log"
Apr 24 14:48:57.853175 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:48:57.853159    2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-w5x5x_44036c3e-1fdc-40d4-899c-9009cd4589e3/ovn-acl-logging/0.log"
Apr 24 14:49:54.822631 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:49:54.822597    2574 generic.go:358] "Generic (PLEG): container finished" podID="5fc35206-739e-42b8-b564-7a905fa24886" containerID="16d60a6b728eb8469fc320b8c8adaebdfb431a2a000c2e475a6292f789e331f3" exitCode=0
Apr 24 14:49:54.823003 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:49:54.822668    2574 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/llmisvc-model-fb-opt-125m-route-02ac88e7-kserve-7969569c4dsmh7x" event={"ID":"5fc35206-739e-42b8-b564-7a905fa24886","Type":"ContainerDied","Data":"16d60a6b728eb8469fc320b8c8adaebdfb431a2a000c2e475a6292f789e331f3"}
Apr 24 14:49:55.828993 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:49:55.828961    2574 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/llmisvc-model-fb-opt-125m-route-02ac88e7-kserve-7969569c4dsmh7x" event={"ID":"5fc35206-739e-42b8-b564-7a905fa24886","Type":"ContainerStarted","Data":"ba4585d4fff39b6bd21a01049332ac7850f81d492a5d190717c2d532fd95e766"}
Apr 24 14:49:55.847619 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:49:55.847560    2574 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="kserve-ci-e2e-test/llmisvc-model-fb-opt-125m-route-02ac88e7-kserve-7969569c4dsmh7x" podStartSLOduration=98.847540518 podStartE2EDuration="1m38.847540518s" podCreationTimestamp="2026-04-24 14:48:17 +0000 UTC" firstStartedPulling="0001-01-01 00:00:00 +0000 UTC" lastFinishedPulling="0001-01-01 00:00:00 +0000 UTC" observedRunningTime="2026-04-24 14:49:55.846260538 +0000 UTC m=+1558.578362180" watchObservedRunningTime="2026-04-24 14:49:55.847540518 +0000 UTC m=+1558.579642170"
Apr 24 14:49:58.075068 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:49:58.075033    2574 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve-ci-e2e-test/llmisvc-model-fb-opt-125m-route-02ac88e7-kserve-7969569c4dsmh7x"
Apr 24 14:49:58.075487 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:49:58.075082    2574 kubelet.go:2658] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="kserve-ci-e2e-test/llmisvc-model-fb-opt-125m-route-02ac88e7-kserve-7969569c4dsmh7x"
Apr 24 14:49:58.076789 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:49:58.076760    2574 prober.go:120] "Probe failed" probeType="Startup" pod="kserve-ci-e2e-test/llmisvc-model-fb-opt-125m-route-02ac88e7-kserve-7969569c4dsmh7x" podUID="5fc35206-739e-42b8-b564-7a905fa24886" containerName="main" probeResult="failure" output="Get \"https://10.133.0.57:8000/health\": dial tcp 10.133.0.57:8000: connect: connection refused"
Apr 24 14:50:08.075378 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:50:08.075310    2574 prober.go:120] "Probe failed" probeType="Startup" pod="kserve-ci-e2e-test/llmisvc-model-fb-opt-125m-route-02ac88e7-kserve-7969569c4dsmh7x" podUID="5fc35206-739e-42b8-b564-7a905fa24886" containerName="main" probeResult="failure" output="Get \"https://10.133.0.57:8000/health\": dial tcp 10.133.0.57:8000: connect: connection refused"
Apr 24 14:50:18.075448 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:50:18.075404    2574 prober.go:120] "Probe failed" probeType="Startup" pod="kserve-ci-e2e-test/llmisvc-model-fb-opt-125m-route-02ac88e7-kserve-7969569c4dsmh7x" podUID="5fc35206-739e-42b8-b564-7a905fa24886" containerName="main" probeResult="failure" output="Get \"https://10.133.0.57:8000/health\": dial tcp 10.133.0.57:8000: connect: connection refused"
Apr 24 14:50:28.076010 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:50:28.075969    2574 prober.go:120] "Probe failed" probeType="Startup" pod="kserve-ci-e2e-test/llmisvc-model-fb-opt-125m-route-02ac88e7-kserve-7969569c4dsmh7x" podUID="5fc35206-739e-42b8-b564-7a905fa24886" containerName="main" probeResult="failure" output="Get \"https://10.133.0.57:8000/health\": dial tcp 10.133.0.57:8000: connect: connection refused"
Apr 24 14:50:38.075704 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:50:38.075661    2574 prober.go:120] "Probe failed" probeType="Startup" pod="kserve-ci-e2e-test/llmisvc-model-fb-opt-125m-route-02ac88e7-kserve-7969569c4dsmh7x" podUID="5fc35206-739e-42b8-b564-7a905fa24886" containerName="main" probeResult="failure" output="Get \"https://10.133.0.57:8000/health\": dial tcp 10.133.0.57:8000: connect: connection refused"
Apr 24 14:50:48.075700 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:50:48.075612    2574 prober.go:120] "Probe failed" probeType="Startup" pod="kserve-ci-e2e-test/llmisvc-model-fb-opt-125m-route-02ac88e7-kserve-7969569c4dsmh7x" podUID="5fc35206-739e-42b8-b564-7a905fa24886" containerName="main" probeResult="failure" output="Get \"https://10.133.0.57:8000/health\": dial tcp 10.133.0.57:8000: connect: connection refused"
Apr 24 14:50:58.075634 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:50:58.075591    2574 prober.go:120] "Probe failed" probeType="Startup" pod="kserve-ci-e2e-test/llmisvc-model-fb-opt-125m-route-02ac88e7-kserve-7969569c4dsmh7x" podUID="5fc35206-739e-42b8-b564-7a905fa24886" containerName="main" probeResult="failure" output="Get \"https://10.133.0.57:8000/health\": dial tcp 10.133.0.57:8000: connect: connection refused"
Apr 24 14:51:08.075120 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:51:08.075081    2574 prober.go:120] "Probe failed" probeType="Startup" pod="kserve-ci-e2e-test/llmisvc-model-fb-opt-125m-route-02ac88e7-kserve-7969569c4dsmh7x" podUID="5fc35206-739e-42b8-b564-7a905fa24886" containerName="main" probeResult="failure" output="Get \"https://10.133.0.57:8000/health\": dial tcp 10.133.0.57:8000: connect: connection refused"
Apr 24 14:51:18.075562 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:51:18.075525    2574 prober.go:120] "Probe failed" probeType="Startup" pod="kserve-ci-e2e-test/llmisvc-model-fb-opt-125m-route-02ac88e7-kserve-7969569c4dsmh7x" podUID="5fc35206-739e-42b8-b564-7a905fa24886" containerName="main" probeResult="failure" output="Get \"https://10.133.0.57:8000/health\": dial tcp 10.133.0.57:8000: connect: connection refused"
Apr 24 14:51:28.084783 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:51:28.084754    2574 kubelet.go:2658] "SyncLoop (probe)" probe="startup" status="started" pod="kserve-ci-e2e-test/llmisvc-model-fb-opt-125m-route-02ac88e7-kserve-7969569c4dsmh7x"
Apr 24 14:51:28.092554 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:51:28.092529    2574 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="kserve-ci-e2e-test/llmisvc-model-fb-opt-125m-route-02ac88e7-kserve-7969569c4dsmh7x"
Apr 24 14:51:32.018611 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:51:32.018575    2574 kubelet.go:2553] "SyncLoop DELETE" source="api" pods=["kserve-ci-e2e-test/llmisvc-model-fb-opt-125m-route-02ac88e7-kserve-7969569c4dsmh7x"]
Apr 24 14:51:32.018979 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:51:32.018853    2574 kuberuntime_container.go:864] "Killing container with a grace period" pod="kserve-ci-e2e-test/llmisvc-model-fb-opt-125m-route-02ac88e7-kserve-7969569c4dsmh7x" podUID="5fc35206-739e-42b8-b564-7a905fa24886" containerName="main" containerID="cri-o://ba4585d4fff39b6bd21a01049332ac7850f81d492a5d190717c2d532fd95e766" gracePeriod=30
Apr 24 14:51:45.908511 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:51:45.908477    2574 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["kserve-ci-e2e-test/llmisvc-model-fb-opt-125m-route-f312f5ec-kserve-mn-0"]
Apr 24 14:51:45.913232 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:51:45.913217    2574 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/llmisvc-model-fb-opt-125m-route-f312f5ec-kserve-mn-0"
Apr 24 14:51:45.915190 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:51:45.915168    2574 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"kserve-ci-e2e-test\"/\"llmisv4e643bc258191ffc517a31cd1d0ddd27-kserve-self-signed-certs\""
Apr 24 14:51:45.915502 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:51:45.915486    2574 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"kserve-ci-e2e-test\"/\"llmisvc-model-fb-opt-125m-route-f312f5-cb7fb8cf-dockercfg-fbmng\""
Apr 24 14:51:45.923842 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:51:45.923818    2574 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/llmisvc-model-fb-opt-125m-route-f312f5ec-kserve-mn-0"]
Apr 24 14:51:45.934118 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:51:45.934098    2574 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["kserve-ci-e2e-test/llmisvc4e643bc258191ffc517a31cd1d0ddd27-kserve-router-sches594d"]
Apr 24 14:51:45.938451 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:51:45.938429    2574 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/llmisvc4e643bc258191ffc517a31cd1d0ddd27-kserve-router-sches594d"
Apr 24 14:51:45.940318 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:51:45.940299    2574 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"kserve-ci-e2e-test\"/\"llmisvc-model-fb-opt-125m-route-f312f5ec-epp-sa-dockercfg-m2qkc\""
Apr 24 14:51:45.951162 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:51:45.951140    2574 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/llmisvc4e643bc258191ffc517a31cd1d0ddd27-kserve-router-sches594d"]
Apr 24 14:51:45.983973 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:51:45.983942    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/abf93af5-6a94-4a56-9e04-e9ab739ef953-model-cache\") pod \"llmisvc-model-fb-opt-125m-route-f312f5ec-kserve-mn-0\" (UID: \"abf93af5-6a94-4a56-9e04-e9ab739ef953\") " pod="kserve-ci-e2e-test/llmisvc-model-fb-opt-125m-route-f312f5ec-kserve-mn-0"
Apr 24 14:51:45.984104 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:51:45.983999    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qx4rt\" (UniqueName: \"kubernetes.io/projected/abf93af5-6a94-4a56-9e04-e9ab739ef953-kube-api-access-qx4rt\") pod \"llmisvc-model-fb-opt-125m-route-f312f5ec-kserve-mn-0\" (UID: \"abf93af5-6a94-4a56-9e04-e9ab739ef953\") " pod="kserve-ci-e2e-test/llmisvc-model-fb-opt-125m-route-f312f5ec-kserve-mn-0"
Apr 24 14:51:45.984104 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:51:45.984067    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/abf93af5-6a94-4a56-9e04-e9ab739ef953-kserve-provision-location\") pod \"llmisvc-model-fb-opt-125m-route-f312f5ec-kserve-mn-0\" (UID: \"abf93af5-6a94-4a56-9e04-e9ab739ef953\") " pod="kserve-ci-e2e-test/llmisvc-model-fb-opt-125m-route-f312f5ec-kserve-mn-0"
Apr 24 14:51:45.984189 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:51:45.984105    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/abf93af5-6a94-4a56-9e04-e9ab739ef953-home\") pod \"llmisvc-model-fb-opt-125m-route-f312f5ec-kserve-mn-0\" (UID: \"abf93af5-6a94-4a56-9e04-e9ab739ef953\") " pod="kserve-ci-e2e-test/llmisvc-model-fb-opt-125m-route-f312f5ec-kserve-mn-0"
Apr 24 14:51:45.984189 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:51:45.984140    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/abf93af5-6a94-4a56-9e04-e9ab739ef953-tls-certs\") pod \"llmisvc-model-fb-opt-125m-route-f312f5ec-kserve-mn-0\" (UID: \"abf93af5-6a94-4a56-9e04-e9ab739ef953\") " pod="kserve-ci-e2e-test/llmisvc-model-fb-opt-125m-route-f312f5ec-kserve-mn-0"
Apr 24 14:51:45.984189 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:51:45.984170    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/abf93af5-6a94-4a56-9e04-e9ab739ef953-dshm\") pod \"llmisvc-model-fb-opt-125m-route-f312f5ec-kserve-mn-0\" (UID: \"abf93af5-6a94-4a56-9e04-e9ab739ef953\") " pod="kserve-ci-e2e-test/llmisvc-model-fb-opt-125m-route-f312f5ec-kserve-mn-0"
Apr 24 14:51:46.085410 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:51:46.085372    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9hcd4\" (UniqueName: \"kubernetes.io/projected/2e6b4d90-4724-4430-a0c6-78ab68774e58-kube-api-access-9hcd4\") pod \"llmisvc4e643bc258191ffc517a31cd1d0ddd27-kserve-router-sches594d\" (UID: \"2e6b4d90-4724-4430-a0c6-78ab68774e58\") " pod="kserve-ci-e2e-test/llmisvc4e643bc258191ffc517a31cd1d0ddd27-kserve-router-sches594d"
Apr 24 14:51:46.085582 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:51:46.085419    2574 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/abf93af5-6a94-4a56-9e04-e9ab739ef953-kserve-provision-location\") pod \"llmisvc-model-fb-opt-125m-route-f312f5ec-kserve-mn-0\" (UID: \"abf93af5-6a94-4a56-9e04-e9ab739ef953\") " pod="kserve-ci-e2e-test/llmisvc-model-fb-opt-125m-route-f312f5ec-kserve-mn-0"
Apr 24 14:51:46.085582 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:51:46.085474    2574 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/abf93af5-6a94-4a56-9e04-e9ab739ef953-home\") pod \"llmisvc-model-fb-opt-125m-route-f312f5ec-kserve-mn-0\" (UID: \"abf93af5-6a94-4a56-9e04-e9ab739ef953\") " pod="kserve-ci-e2e-test/llmisvc-model-fb-opt-125m-route-f312f5ec-kserve-mn-0"
Apr 24 14:51:46.085582 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:51:46.085515    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/2e6b4d90-4724-4430-a0c6-78ab68774e58-tls-certs\") pod \"llmisvc4e643bc258191ffc517a31cd1d0ddd27-kserve-router-sches594d\" (UID: \"2e6b4d90-4724-4430-a0c6-78ab68774e58\") " pod="kserve-ci-e2e-test/llmisvc4e643bc258191ffc517a31cd1d0ddd27-kserve-router-sches594d"
Apr 24 14:51:46.085582 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:51:46.085534    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tokenizer-uds\" (UniqueName: \"kubernetes.io/empty-dir/2e6b4d90-4724-4430-a0c6-78ab68774e58-tokenizer-uds\") pod \"llmisvc4e643bc258191ffc517a31cd1d0ddd27-kserve-router-sches594d\" (UID: \"2e6b4d90-4724-4430-a0c6-78ab68774e58\") " pod="kserve-ci-e2e-test/llmisvc4e643bc258191ffc517a31cd1d0ddd27-kserve-router-sches594d"
Apr 24 14:51:46.085582 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:51:46.085551    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/2e6b4d90-4724-4430-a0c6-78ab68774e58-kserve-provision-location\") pod \"llmisvc4e643bc258191ffc517a31cd1d0ddd27-kserve-router-sches594d\" (UID: \"2e6b4d90-4724-4430-a0c6-78ab68774e58\") " pod="kserve-ci-e2e-test/llmisvc4e643bc258191ffc517a31cd1d0ddd27-kserve-router-sches594d"
Apr 24 14:51:46.085916 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:51:46.085588    2574 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/abf93af5-6a94-4a56-9e04-e9ab739ef953-tls-certs\") pod \"llmisvc-model-fb-opt-125m-route-f312f5ec-kserve-mn-0\" (UID: \"abf93af5-6a94-4a56-9e04-e9ab739ef953\") " pod="kserve-ci-e2e-test/llmisvc-model-fb-opt-125m-route-f312f5ec-kserve-mn-0"
Apr 24 14:51:46.085916 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:51:46.085634    2574 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/abf93af5-6a94-4a56-9e04-e9ab739ef953-dshm\") pod \"llmisvc-model-fb-opt-125m-route-f312f5ec-kserve-mn-0\" (UID: \"abf93af5-6a94-4a56-9e04-e9ab739ef953\") " pod="kserve-ci-e2e-test/llmisvc-model-fb-opt-125m-route-f312f5ec-kserve-mn-0"
Apr 24 14:51:46.085916 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:51:46.085656    2574 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/abf93af5-6a94-4a56-9e04-e9ab739ef953-model-cache\") pod \"llmisvc-model-fb-opt-125m-route-f312f5ec-kserve-mn-0\" (UID: \"abf93af5-6a94-4a56-9e04-e9ab739ef953\") " pod="kserve-ci-e2e-test/llmisvc-model-fb-opt-125m-route-f312f5ec-kserve-mn-0"
Apr 24 14:51:46.085916 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:51:46.085687    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/abf93af5-6a94-4a56-9e04-e9ab739ef953-kserve-provision-location\") pod \"llmisvc-model-fb-opt-125m-route-f312f5ec-kserve-mn-0\" (UID: \"abf93af5-6a94-4a56-9e04-e9ab739ef953\") " pod="kserve-ci-e2e-test/llmisvc-model-fb-opt-125m-route-f312f5ec-kserve-mn-0"
Apr 24 14:51:46.085916 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:51:46.085724    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tokenizer-tmp\" (UniqueName: \"kubernetes.io/empty-dir/2e6b4d90-4724-4430-a0c6-78ab68774e58-tokenizer-tmp\") pod \"llmisvc4e643bc258191ffc517a31cd1d0ddd27-kserve-router-sches594d\" (UID: \"2e6b4d90-4724-4430-a0c6-78ab68774e58\") " pod="kserve-ci-e2e-test/llmisvc4e643bc258191ffc517a31cd1d0ddd27-kserve-router-sches594d"
Apr 24 14:51:46.085916 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:51:46.085758    2574 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-qx4rt\" (UniqueName: \"kubernetes.io/projected/abf93af5-6a94-4a56-9e04-e9ab739ef953-kube-api-access-qx4rt\") pod \"llmisvc-model-fb-opt-125m-route-f312f5ec-kserve-mn-0\" (UID: \"abf93af5-6a94-4a56-9e04-e9ab739ef953\") " pod="kserve-ci-e2e-test/llmisvc-model-fb-opt-125m-route-f312f5ec-kserve-mn-0"
Apr 24 14:51:46.085916 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:51:46.085781    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tokenizer-cache\" (UniqueName: \"kubernetes.io/empty-dir/2e6b4d90-4724-4430-a0c6-78ab68774e58-tokenizer-cache\") pod \"llmisvc4e643bc258191ffc517a31cd1d0ddd27-kserve-router-sches594d\" (UID: \"2e6b4d90-4724-4430-a0c6-78ab68774e58\") " pod="kserve-ci-e2e-test/llmisvc4e643bc258191ffc517a31cd1d0ddd27-kserve-router-sches594d"
Apr 24 14:51:46.085916 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:51:46.085854    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/abf93af5-6a94-4a56-9e04-e9ab739ef953-home\") pod \"llmisvc-model-fb-opt-125m-route-f312f5ec-kserve-mn-0\" (UID: \"abf93af5-6a94-4a56-9e04-e9ab739ef953\") " pod="kserve-ci-e2e-test/llmisvc-model-fb-opt-125m-route-f312f5ec-kserve-mn-0"
Apr 24 14:51:46.085916 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:51:46.085896    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/abf93af5-6a94-4a56-9e04-e9ab739ef953-model-cache\") pod \"llmisvc-model-fb-opt-125m-route-f312f5ec-kserve-mn-0\" (UID: \"abf93af5-6a94-4a56-9e04-e9ab739ef953\") " pod="kserve-ci-e2e-test/llmisvc-model-fb-opt-125m-route-f312f5ec-kserve-mn-0"
Apr 24 14:51:46.087856 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:51:46.087835    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/abf93af5-6a94-4a56-9e04-e9ab739ef953-dshm\") pod \"llmisvc-model-fb-opt-125m-route-f312f5ec-kserve-mn-0\" (UID: \"abf93af5-6a94-4a56-9e04-e9ab739ef953\") " pod="kserve-ci-e2e-test/llmisvc-model-fb-opt-125m-route-f312f5ec-kserve-mn-0"
Apr 24 14:51:46.087961 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:51:46.087908    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/abf93af5-6a94-4a56-9e04-e9ab739ef953-tls-certs\") pod \"llmisvc-model-fb-opt-125m-route-f312f5ec-kserve-mn-0\" (UID: \"abf93af5-6a94-4a56-9e04-e9ab739ef953\") " pod="kserve-ci-e2e-test/llmisvc-model-fb-opt-125m-route-f312f5ec-kserve-mn-0"
Apr 24 14:51:46.092813 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:51:46.092793    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-qx4rt\" (UniqueName: \"kubernetes.io/projected/abf93af5-6a94-4a56-9e04-e9ab739ef953-kube-api-access-qx4rt\") pod \"llmisvc-model-fb-opt-125m-route-f312f5ec-kserve-mn-0\" (UID: \"abf93af5-6a94-4a56-9e04-e9ab739ef953\") " pod="kserve-ci-e2e-test/llmisvc-model-fb-opt-125m-route-f312f5ec-kserve-mn-0"
Apr 24 14:51:46.186611 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:51:46.186535    2574 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/2e6b4d90-4724-4430-a0c6-78ab68774e58-tls-certs\") pod \"llmisvc4e643bc258191ffc517a31cd1d0ddd27-kserve-router-sches594d\" (UID: \"2e6b4d90-4724-4430-a0c6-78ab68774e58\") " pod="kserve-ci-e2e-test/llmisvc4e643bc258191ffc517a31cd1d0ddd27-kserve-router-sches594d"
Apr 24 14:51:46.186611 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:51:46.186575    2574 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"tokenizer-uds\" (UniqueName: \"kubernetes.io/empty-dir/2e6b4d90-4724-4430-a0c6-78ab68774e58-tokenizer-uds\") pod \"llmisvc4e643bc258191ffc517a31cd1d0ddd27-kserve-router-sches594d\" (UID: \"2e6b4d90-4724-4430-a0c6-78ab68774e58\") " pod="kserve-ci-e2e-test/llmisvc4e643bc258191ffc517a31cd1d0ddd27-kserve-router-sches594d"
Apr 24 14:51:46.186611 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:51:46.186602    2574 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/2e6b4d90-4724-4430-a0c6-78ab68774e58-kserve-provision-location\") pod \"llmisvc4e643bc258191ffc517a31cd1d0ddd27-kserve-router-sches594d\" (UID: \"2e6b4d90-4724-4430-a0c6-78ab68774e58\") " pod="kserve-ci-e2e-test/llmisvc4e643bc258191ffc517a31cd1d0ddd27-kserve-router-sches594d"
Apr 24 14:51:46.186855 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:51:46.186693    2574 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"tokenizer-tmp\" (UniqueName: \"kubernetes.io/empty-dir/2e6b4d90-4724-4430-a0c6-78ab68774e58-tokenizer-tmp\") pod \"llmisvc4e643bc258191ffc517a31cd1d0ddd27-kserve-router-sches594d\" (UID: \"2e6b4d90-4724-4430-a0c6-78ab68774e58\") " pod="kserve-ci-e2e-test/llmisvc4e643bc258191ffc517a31cd1d0ddd27-kserve-router-sches594d"
Apr 24 14:51:46.186855 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:51:46.186726    2574 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"tokenizer-cache\" (UniqueName: \"kubernetes.io/empty-dir/2e6b4d90-4724-4430-a0c6-78ab68774e58-tokenizer-cache\") pod \"llmisvc4e643bc258191ffc517a31cd1d0ddd27-kserve-router-sches594d\" (UID: \"2e6b4d90-4724-4430-a0c6-78ab68774e58\") " pod="kserve-ci-e2e-test/llmisvc4e643bc258191ffc517a31cd1d0ddd27-kserve-router-sches594d"
Apr 24 14:51:46.186855 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:51:46.186774    2574 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-9hcd4\" (UniqueName: \"kubernetes.io/projected/2e6b4d90-4724-4430-a0c6-78ab68774e58-kube-api-access-9hcd4\") pod \"llmisvc4e643bc258191ffc517a31cd1d0ddd27-kserve-router-sches594d\" (UID: \"2e6b4d90-4724-4430-a0c6-78ab68774e58\") " pod="kserve-ci-e2e-test/llmisvc4e643bc258191ffc517a31cd1d0ddd27-kserve-router-sches594d"
Apr 24 14:51:46.187027 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:51:46.187001    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tokenizer-uds\" (UniqueName: \"kubernetes.io/empty-dir/2e6b4d90-4724-4430-a0c6-78ab68774e58-tokenizer-uds\") pod \"llmisvc4e643bc258191ffc517a31cd1d0ddd27-kserve-router-sches594d\" (UID: \"2e6b4d90-4724-4430-a0c6-78ab68774e58\") " pod="kserve-ci-e2e-test/llmisvc4e643bc258191ffc517a31cd1d0ddd27-kserve-router-sches594d"
Apr 24 14:51:46.187090 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:51:46.187013    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/2e6b4d90-4724-4430-a0c6-78ab68774e58-kserve-provision-location\") pod \"llmisvc4e643bc258191ffc517a31cd1d0ddd27-kserve-router-sches594d\" (UID: \"2e6b4d90-4724-4430-a0c6-78ab68774e58\") " pod="kserve-ci-e2e-test/llmisvc4e643bc258191ffc517a31cd1d0ddd27-kserve-router-sches594d"
Apr 24 14:51:46.187137 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:51:46.187088    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tokenizer-cache\" (UniqueName: \"kubernetes.io/empty-dir/2e6b4d90-4724-4430-a0c6-78ab68774e58-tokenizer-cache\") pod \"llmisvc4e643bc258191ffc517a31cd1d0ddd27-kserve-router-sches594d\" (UID: \"2e6b4d90-4724-4430-a0c6-78ab68774e58\") " pod="kserve-ci-e2e-test/llmisvc4e643bc258191ffc517a31cd1d0ddd27-kserve-router-sches594d"
Apr 24 14:51:46.187137 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:51:46.187114    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tokenizer-tmp\" (UniqueName: \"kubernetes.io/empty-dir/2e6b4d90-4724-4430-a0c6-78ab68774e58-tokenizer-tmp\") pod \"llmisvc4e643bc258191ffc517a31cd1d0ddd27-kserve-router-sches594d\" (UID: \"2e6b4d90-4724-4430-a0c6-78ab68774e58\") " pod="kserve-ci-e2e-test/llmisvc4e643bc258191ffc517a31cd1d0ddd27-kserve-router-sches594d"
Apr 24 14:51:46.188882 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:51:46.188865    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/2e6b4d90-4724-4430-a0c6-78ab68774e58-tls-certs\") pod \"llmisvc4e643bc258191ffc517a31cd1d0ddd27-kserve-router-sches594d\" (UID: \"2e6b4d90-4724-4430-a0c6-78ab68774e58\") " pod="kserve-ci-e2e-test/llmisvc4e643bc258191ffc517a31cd1d0ddd27-kserve-router-sches594d"
Apr 24 14:51:46.193793 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:51:46.193773    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-9hcd4\" (UniqueName: \"kubernetes.io/projected/2e6b4d90-4724-4430-a0c6-78ab68774e58-kube-api-access-9hcd4\") pod \"llmisvc4e643bc258191ffc517a31cd1d0ddd27-kserve-router-sches594d\" (UID: \"2e6b4d90-4724-4430-a0c6-78ab68774e58\") " pod="kserve-ci-e2e-test/llmisvc4e643bc258191ffc517a31cd1d0ddd27-kserve-router-sches594d"
Apr 24 14:51:46.224688 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:51:46.224665    2574 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/llmisvc-model-fb-opt-125m-route-f312f5ec-kserve-mn-0"
Apr 24 14:51:46.250618 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:51:46.250590    2574 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/llmisvc4e643bc258191ffc517a31cd1d0ddd27-kserve-router-sches594d"
Apr 24 14:51:46.396585 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:51:46.396555    2574 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/llmisvc4e643bc258191ffc517a31cd1d0ddd27-kserve-router-sches594d"]
Apr 24 14:51:46.397569 ip-10-0-136-132 kubenswrapper[2574]: W0424 14:51:46.397542    2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2e6b4d90_4724_4430_a0c6_78ab68774e58.slice/crio-fd6fd5b9f30536b9f9d74d93089c076a78be7e832e7a1a2cabd388d3d04b0d21 WatchSource:0}: Error finding container fd6fd5b9f30536b9f9d74d93089c076a78be7e832e7a1a2cabd388d3d04b0d21: Status 404 returned error can't find the container with id fd6fd5b9f30536b9f9d74d93089c076a78be7e832e7a1a2cabd388d3d04b0d21
Apr 24 14:51:46.399725 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:51:46.399700    2574 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Apr 24 14:51:46.568757 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:51:46.568728    2574 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/llmisvc-model-fb-opt-125m-route-f312f5ec-kserve-mn-0"]
Apr 24 14:51:46.570159 ip-10-0-136-132 kubenswrapper[2574]: W0424 14:51:46.570136    2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podabf93af5_6a94_4a56_9e04_e9ab739ef953.slice/crio-6709ed13639ee6dabd6aaf75bfd1f467f71d43e31574ebab3ca7fe39abf8cca6 WatchSource:0}: Error finding container 6709ed13639ee6dabd6aaf75bfd1f467f71d43e31574ebab3ca7fe39abf8cca6: Status 404 returned error can't find the container with id 6709ed13639ee6dabd6aaf75bfd1f467f71d43e31574ebab3ca7fe39abf8cca6
Apr 24 14:51:47.267322 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:51:47.267291    2574 generic.go:358] "Generic (PLEG): container finished" podID="2e6b4d90-4724-4430-a0c6-78ab68774e58" containerID="4844b6a597af41e3f0ee4841f8c1ce925c2e382b0bd7d26ff7d15d3418767162" exitCode=0
Apr 24 14:51:47.267710 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:51:47.267373    2574 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/llmisvc4e643bc258191ffc517a31cd1d0ddd27-kserve-router-sches594d" event={"ID":"2e6b4d90-4724-4430-a0c6-78ab68774e58","Type":"ContainerDied","Data":"4844b6a597af41e3f0ee4841f8c1ce925c2e382b0bd7d26ff7d15d3418767162"}
Apr 24 14:51:47.267710 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:51:47.267398    2574 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/llmisvc4e643bc258191ffc517a31cd1d0ddd27-kserve-router-sches594d" event={"ID":"2e6b4d90-4724-4430-a0c6-78ab68774e58","Type":"ContainerStarted","Data":"fd6fd5b9f30536b9f9d74d93089c076a78be7e832e7a1a2cabd388d3d04b0d21"}
Apr 24 14:51:47.268952 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:51:47.268911    2574 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/llmisvc-model-fb-opt-125m-route-f312f5ec-kserve-mn-0" event={"ID":"abf93af5-6a94-4a56-9e04-e9ab739ef953","Type":"ContainerStarted","Data":"355fbe01f0a9622047d9cf3d679374aded099e09d1d5840fb19b37d7a758c814"}
Apr 24 14:51:47.269066 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:51:47.268959    2574 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/llmisvc-model-fb-opt-125m-route-f312f5ec-kserve-mn-0" event={"ID":"abf93af5-6a94-4a56-9e04-e9ab739ef953","Type":"ContainerStarted","Data":"6709ed13639ee6dabd6aaf75bfd1f467f71d43e31574ebab3ca7fe39abf8cca6"}
Apr 24 14:51:48.275494 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:51:48.275458    2574 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/llmisvc4e643bc258191ffc517a31cd1d0ddd27-kserve-router-sches594d" event={"ID":"2e6b4d90-4724-4430-a0c6-78ab68774e58","Type":"ContainerStarted","Data":"1b9d3724b82696fa337f496370d3936f733829e63d8296918b04a6ae0e4e21b8"}
Apr 24 14:51:48.275494 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:51:48.275499    2574 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/llmisvc4e643bc258191ffc517a31cd1d0ddd27-kserve-router-sches594d" event={"ID":"2e6b4d90-4724-4430-a0c6-78ab68774e58","Type":"ContainerStarted","Data":"cd8532e45668592121cf1a7b421534de0afe387326026693590a34b4d18c7495"}
Apr 24 14:51:48.275988 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:51:48.275767    2574 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve-ci-e2e-test/llmisvc4e643bc258191ffc517a31cd1d0ddd27-kserve-router-sches594d"
Apr 24 14:51:48.296444 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:51:48.296400    2574 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="kserve-ci-e2e-test/llmisvc4e643bc258191ffc517a31cd1d0ddd27-kserve-router-sches594d" podStartSLOduration=3.296386193 podStartE2EDuration="3.296386193s" podCreationTimestamp="2026-04-24 14:51:45 +0000 UTC" firstStartedPulling="0001-01-01 00:00:00 +0000 UTC" lastFinishedPulling="0001-01-01 00:00:00 +0000 UTC" observedRunningTime="2026-04-24 14:51:48.292605315 +0000 UTC m=+1671.024706988" watchObservedRunningTime="2026-04-24 14:51:48.296386193 +0000 UTC m=+1671.028487841"
Apr 24 14:51:52.295487 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:51:52.295441    2574 generic.go:358] "Generic (PLEG): container finished" podID="abf93af5-6a94-4a56-9e04-e9ab739ef953" containerID="355fbe01f0a9622047d9cf3d679374aded099e09d1d5840fb19b37d7a758c814" exitCode=0
Apr 24 14:51:52.295903 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:51:52.295511    2574 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/llmisvc-model-fb-opt-125m-route-f312f5ec-kserve-mn-0" event={"ID":"abf93af5-6a94-4a56-9e04-e9ab739ef953","Type":"ContainerDied","Data":"355fbe01f0a9622047d9cf3d679374aded099e09d1d5840fb19b37d7a758c814"}
Apr 24 14:51:53.302097 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:51:53.302061    2574 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/llmisvc-model-fb-opt-125m-route-f312f5ec-kserve-mn-0" event={"ID":"abf93af5-6a94-4a56-9e04-e9ab739ef953","Type":"ContainerStarted","Data":"88f06aaf57cc71f4fd14cafc31da217efa3c453749e50054e293acacdd57e9f3"}
Apr 24 14:51:53.320511 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:51:53.320447    2574 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="kserve-ci-e2e-test/llmisvc-model-fb-opt-125m-route-f312f5ec-kserve-mn-0" podStartSLOduration=8.320432808 podStartE2EDuration="8.320432808s" podCreationTimestamp="2026-04-24 14:51:45 +0000 UTC" firstStartedPulling="0001-01-01 00:00:00 +0000 UTC" lastFinishedPulling="0001-01-01 00:00:00 +0000 UTC" observedRunningTime="2026-04-24 14:51:53.318577413 +0000 UTC m=+1676.050679067" watchObservedRunningTime="2026-04-24 14:51:53.320432808 +0000 UTC m=+1676.052534457"
Apr 24 14:51:56.225779 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:51:56.225744    2574 kubelet.go:2658] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="kserve-ci-e2e-test/llmisvc-model-fb-opt-125m-route-f312f5ec-kserve-mn-0"
Apr 24 14:51:56.227564 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:51:56.227535    2574 prober.go:120] "Probe failed" probeType="Startup" pod="kserve-ci-e2e-test/llmisvc-model-fb-opt-125m-route-f312f5ec-kserve-mn-0" podUID="abf93af5-6a94-4a56-9e04-e9ab739ef953" containerName="main" probeResult="failure" output="Get \"https://10.133.0.58:8000/health\": dial tcp 10.133.0.58:8000: connect: connection refused"
Apr 24 14:51:56.251537 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:51:56.251509    2574 kubelet.go:2658] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="kserve-ci-e2e-test/llmisvc4e643bc258191ffc517a31cd1d0ddd27-kserve-router-sches594d"
Apr 24 14:51:56.251537 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:51:56.251541    2574 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve-ci-e2e-test/llmisvc4e643bc258191ffc517a31cd1d0ddd27-kserve-router-sches594d"
Apr 24 14:51:56.254304 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:51:56.254275    2574 kubelet.go:2658] "SyncLoop (probe)" probe="startup" status="started" pod="kserve-ci-e2e-test/llmisvc4e643bc258191ffc517a31cd1d0ddd27-kserve-router-sches594d"
Apr 24 14:51:56.315841 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:51:56.315806    2574 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="kserve-ci-e2e-test/llmisvc4e643bc258191ffc517a31cd1d0ddd27-kserve-router-sches594d"
Apr 24 14:52:02.289965 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:52:02.289942    2574 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_llmisvc-model-fb-opt-125m-route-02ac88e7-kserve-7969569c4dsmh7x_5fc35206-739e-42b8-b564-7a905fa24886/main/0.log"
Apr 24 14:52:02.290378 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:52:02.290362    2574 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/llmisvc-model-fb-opt-125m-route-02ac88e7-kserve-7969569c4dsmh7x"
Apr 24 14:52:02.339203 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:52:02.339175    2574 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_llmisvc-model-fb-opt-125m-route-02ac88e7-kserve-7969569c4dsmh7x_5fc35206-739e-42b8-b564-7a905fa24886/main/0.log"
Apr 24 14:52:02.339578 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:52:02.339550    2574 generic.go:358] "Generic (PLEG): container finished" podID="5fc35206-739e-42b8-b564-7a905fa24886" containerID="ba4585d4fff39b6bd21a01049332ac7850f81d492a5d190717c2d532fd95e766" exitCode=137
Apr 24 14:52:02.339648 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:52:02.339620    2574 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/llmisvc-model-fb-opt-125m-route-02ac88e7-kserve-7969569c4dsmh7x"
Apr 24 14:52:02.339694 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:52:02.339636    2574 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/llmisvc-model-fb-opt-125m-route-02ac88e7-kserve-7969569c4dsmh7x" event={"ID":"5fc35206-739e-42b8-b564-7a905fa24886","Type":"ContainerDied","Data":"ba4585d4fff39b6bd21a01049332ac7850f81d492a5d190717c2d532fd95e766"}
Apr 24 14:52:02.339694 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:52:02.339676    2574 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/llmisvc-model-fb-opt-125m-route-02ac88e7-kserve-7969569c4dsmh7x" event={"ID":"5fc35206-739e-42b8-b564-7a905fa24886","Type":"ContainerDied","Data":"1725ffd64f9b231d6fc04a9973c4da81ef20d7e2734705886ad2fac6e0814349"}
Apr 24 14:52:02.339770 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:52:02.339697    2574 scope.go:117] "RemoveContainer" containerID="ba4585d4fff39b6bd21a01049332ac7850f81d492a5d190717c2d532fd95e766"
Apr 24 14:52:02.359327 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:52:02.359308    2574 scope.go:117] "RemoveContainer" containerID="16d60a6b728eb8469fc320b8c8adaebdfb431a2a000c2e475a6292f789e331f3"
Apr 24 14:52:02.435055 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:52:02.435028    2574 scope.go:117] "RemoveContainer" containerID="ba4585d4fff39b6bd21a01049332ac7850f81d492a5d190717c2d532fd95e766"
Apr 24 14:52:02.435407 ip-10-0-136-132 kubenswrapper[2574]: E0424 14:52:02.435389    2574 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"ba4585d4fff39b6bd21a01049332ac7850f81d492a5d190717c2d532fd95e766\": container with ID starting with ba4585d4fff39b6bd21a01049332ac7850f81d492a5d190717c2d532fd95e766 not found: ID does not exist" containerID="ba4585d4fff39b6bd21a01049332ac7850f81d492a5d190717c2d532fd95e766"
Apr 24 14:52:02.435465 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:52:02.435418    2574 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"ba4585d4fff39b6bd21a01049332ac7850f81d492a5d190717c2d532fd95e766"} err="failed to get container status \"ba4585d4fff39b6bd21a01049332ac7850f81d492a5d190717c2d532fd95e766\": rpc error: code = NotFound desc = could not find container \"ba4585d4fff39b6bd21a01049332ac7850f81d492a5d190717c2d532fd95e766\": container with ID starting with ba4585d4fff39b6bd21a01049332ac7850f81d492a5d190717c2d532fd95e766 not found: ID does not exist"
Apr 24 14:52:02.435465 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:52:02.435437    2574 scope.go:117] "RemoveContainer" containerID="16d60a6b728eb8469fc320b8c8adaebdfb431a2a000c2e475a6292f789e331f3"
Apr 24 14:52:02.435766 ip-10-0-136-132 kubenswrapper[2574]: E0424 14:52:02.435726    2574 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"16d60a6b728eb8469fc320b8c8adaebdfb431a2a000c2e475a6292f789e331f3\": container with ID starting with 16d60a6b728eb8469fc320b8c8adaebdfb431a2a000c2e475a6292f789e331f3 not found: ID does not exist" containerID="16d60a6b728eb8469fc320b8c8adaebdfb431a2a000c2e475a6292f789e331f3"
Apr 24 14:52:02.435835 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:52:02.435770    2574 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"16d60a6b728eb8469fc320b8c8adaebdfb431a2a000c2e475a6292f789e331f3"} err="failed to get container status \"16d60a6b728eb8469fc320b8c8adaebdfb431a2a000c2e475a6292f789e331f3\": rpc error: code = NotFound desc = could not find container \"16d60a6b728eb8469fc320b8c8adaebdfb431a2a000c2e475a6292f789e331f3\": container with ID starting with 16d60a6b728eb8469fc320b8c8adaebdfb431a2a000c2e475a6292f789e331f3 not found: ID does not exist"
Apr 24 14:52:02.438089 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:52:02.438074    2574 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"kube-api-access-ll9zn\" (UniqueName: \"kubernetes.io/projected/5fc35206-739e-42b8-b564-7a905fa24886-kube-api-access-ll9zn\") pod \"5fc35206-739e-42b8-b564-7a905fa24886\" (UID: \"5fc35206-739e-42b8-b564-7a905fa24886\") "
Apr 24 14:52:02.438148 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:52:02.438117    2574 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/5fc35206-739e-42b8-b564-7a905fa24886-home\") pod \"5fc35206-739e-42b8-b564-7a905fa24886\" (UID: \"5fc35206-739e-42b8-b564-7a905fa24886\") "
Apr 24 14:52:02.438197 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:52:02.438149    2574 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/5fc35206-739e-42b8-b564-7a905fa24886-dshm\") pod \"5fc35206-739e-42b8-b564-7a905fa24886\" (UID: \"5fc35206-739e-42b8-b564-7a905fa24886\") "
Apr 24 14:52:02.438197 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:52:02.438167    2574 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/5fc35206-739e-42b8-b564-7a905fa24886-tls-certs\") pod \"5fc35206-739e-42b8-b564-7a905fa24886\" (UID: \"5fc35206-739e-42b8-b564-7a905fa24886\") "
Apr 24 14:52:02.438277 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:52:02.438218    2574 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/5fc35206-739e-42b8-b564-7a905fa24886-model-cache\") pod \"5fc35206-739e-42b8-b564-7a905fa24886\" (UID: \"5fc35206-739e-42b8-b564-7a905fa24886\") "
Apr 24 14:52:02.438277 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:52:02.438236    2574 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/5fc35206-739e-42b8-b564-7a905fa24886-kserve-provision-location\") pod \"5fc35206-739e-42b8-b564-7a905fa24886\" (UID: \"5fc35206-739e-42b8-b564-7a905fa24886\") "
Apr 24 14:52:02.438602 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:52:02.438502    2574 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/5fc35206-739e-42b8-b564-7a905fa24886-model-cache" (OuterVolumeSpecName: "model-cache") pod "5fc35206-739e-42b8-b564-7a905fa24886" (UID: "5fc35206-739e-42b8-b564-7a905fa24886"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Apr 24 14:52:02.438602 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:52:02.438556    2574 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/5fc35206-739e-42b8-b564-7a905fa24886-home" (OuterVolumeSpecName: "home") pod "5fc35206-739e-42b8-b564-7a905fa24886" (UID: "5fc35206-739e-42b8-b564-7a905fa24886"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Apr 24 14:52:02.440222 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:52:02.440195    2574 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/5fc35206-739e-42b8-b564-7a905fa24886-kube-api-access-ll9zn" (OuterVolumeSpecName: "kube-api-access-ll9zn") pod "5fc35206-739e-42b8-b564-7a905fa24886" (UID: "5fc35206-739e-42b8-b564-7a905fa24886"). InnerVolumeSpecName "kube-api-access-ll9zn". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Apr 24 14:52:02.440656 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:52:02.440628    2574 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/5fc35206-739e-42b8-b564-7a905fa24886-dshm" (OuterVolumeSpecName: "dshm") pod "5fc35206-739e-42b8-b564-7a905fa24886" (UID: "5fc35206-739e-42b8-b564-7a905fa24886"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Apr 24 14:52:02.440656 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:52:02.440645    2574 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/5fc35206-739e-42b8-b564-7a905fa24886-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "5fc35206-739e-42b8-b564-7a905fa24886" (UID: "5fc35206-739e-42b8-b564-7a905fa24886"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Apr 24 14:52:02.503668 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:52:02.503628    2574 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/5fc35206-739e-42b8-b564-7a905fa24886-kserve-provision-location" (OuterVolumeSpecName: "kserve-provision-location") pod "5fc35206-739e-42b8-b564-7a905fa24886" (UID: "5fc35206-739e-42b8-b564-7a905fa24886"). InnerVolumeSpecName "kserve-provision-location". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Apr 24 14:52:02.539484 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:52:02.539406    2574 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/5fc35206-739e-42b8-b564-7a905fa24886-model-cache\") on node \"ip-10-0-136-132.ec2.internal\" DevicePath \"\""
Apr 24 14:52:02.539484 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:52:02.539435    2574 reconciler_common.go:299] "Volume detached for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/5fc35206-739e-42b8-b564-7a905fa24886-kserve-provision-location\") on node \"ip-10-0-136-132.ec2.internal\" DevicePath \"\""
Apr 24 14:52:02.539484 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:52:02.539450    2574 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-ll9zn\" (UniqueName: \"kubernetes.io/projected/5fc35206-739e-42b8-b564-7a905fa24886-kube-api-access-ll9zn\") on node \"ip-10-0-136-132.ec2.internal\" DevicePath \"\""
Apr 24 14:52:02.539484 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:52:02.539463    2574 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/5fc35206-739e-42b8-b564-7a905fa24886-home\") on node \"ip-10-0-136-132.ec2.internal\" DevicePath \"\""
Apr 24 14:52:02.539484 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:52:02.539475    2574 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/5fc35206-739e-42b8-b564-7a905fa24886-dshm\") on node \"ip-10-0-136-132.ec2.internal\" DevicePath \"\""
Apr 24 14:52:02.539484 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:52:02.539487    2574 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/5fc35206-739e-42b8-b564-7a905fa24886-tls-certs\") on node \"ip-10-0-136-132.ec2.internal\" DevicePath \"\""
Apr 24 14:52:02.665108 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:52:02.665068    2574 kubelet.go:2553] "SyncLoop DELETE" source="api" pods=["kserve-ci-e2e-test/llmisvc-model-fb-opt-125m-route-02ac88e7-kserve-7969569c4dsmh7x"]
Apr 24 14:52:02.668438 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:52:02.668413    2574 kubelet.go:2547] "SyncLoop REMOVE" source="api" pods=["kserve-ci-e2e-test/llmisvc-model-fb-opt-125m-route-02ac88e7-kserve-7969569c4dsmh7x"]
Apr 24 14:52:03.771000 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:52:03.770957    2574 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="5fc35206-739e-42b8-b564-7a905fa24886" path="/var/lib/kubelet/pods/5fc35206-739e-42b8-b564-7a905fa24886/volumes"
Apr 24 14:52:06.225633 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:52:06.225585    2574 prober.go:120] "Probe failed" probeType="Startup" pod="kserve-ci-e2e-test/llmisvc-model-fb-opt-125m-route-f312f5ec-kserve-mn-0" podUID="abf93af5-6a94-4a56-9e04-e9ab739ef953" containerName="main" probeResult="failure" output="Get \"https://10.133.0.58:8000/health\": dial tcp 10.133.0.58:8000: connect: connection refused"
Apr 24 14:52:16.224987 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:52:16.224896    2574 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve-ci-e2e-test/llmisvc-model-fb-opt-125m-route-f312f5ec-kserve-mn-0"
Apr 24 14:52:16.225561 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:52:16.225212    2574 prober.go:120] "Probe failed" probeType="Startup" pod="kserve-ci-e2e-test/llmisvc-model-fb-opt-125m-route-f312f5ec-kserve-mn-0" podUID="abf93af5-6a94-4a56-9e04-e9ab739ef953" containerName="main" probeResult="failure" output="Get \"https://10.133.0.58:8000/health\": dial tcp 10.133.0.58:8000: connect: connection refused"
Apr 24 14:52:17.320488 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:52:17.320458    2574 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="kserve-ci-e2e-test/llmisvc4e643bc258191ffc517a31cd1d0ddd27-kserve-router-sches594d"
Apr 24 14:52:26.225604 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:52:26.225547    2574 prober.go:120] "Probe failed" probeType="Startup" pod="kserve-ci-e2e-test/llmisvc-model-fb-opt-125m-route-f312f5ec-kserve-mn-0" podUID="abf93af5-6a94-4a56-9e04-e9ab739ef953" containerName="main" probeResult="failure" output="Get \"https://10.133.0.58:8000/health\": dial tcp 10.133.0.58:8000: connect: connection refused"
Apr 24 14:52:36.226003 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:52:36.225943    2574 prober.go:120] "Probe failed" probeType="Startup" pod="kserve-ci-e2e-test/llmisvc-model-fb-opt-125m-route-f312f5ec-kserve-mn-0" podUID="abf93af5-6a94-4a56-9e04-e9ab739ef953" containerName="main" probeResult="failure" output="Get \"https://10.133.0.58:8000/health\": dial tcp 10.133.0.58:8000: connect: connection refused"
Apr 24 14:52:46.225545 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:52:46.225495    2574 prober.go:120] "Probe failed" probeType="Startup" pod="kserve-ci-e2e-test/llmisvc-model-fb-opt-125m-route-f312f5ec-kserve-mn-0" podUID="abf93af5-6a94-4a56-9e04-e9ab739ef953" containerName="main" probeResult="failure" output="Get \"https://10.133.0.58:8000/health\": dial tcp 10.133.0.58:8000: connect: connection refused"
Apr 24 14:52:56.225806 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:52:56.225753    2574 prober.go:120] "Probe failed" probeType="Startup" pod="kserve-ci-e2e-test/llmisvc-model-fb-opt-125m-route-f312f5ec-kserve-mn-0" podUID="abf93af5-6a94-4a56-9e04-e9ab739ef953" containerName="main" probeResult="failure" output="Get \"https://10.133.0.58:8000/health\": dial tcp 10.133.0.58:8000: connect: connection refused"
Apr 24 14:53:06.225279 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:53:06.225233    2574 prober.go:120] "Probe failed" probeType="Startup" pod="kserve-ci-e2e-test/llmisvc-model-fb-opt-125m-route-f312f5ec-kserve-mn-0" podUID="abf93af5-6a94-4a56-9e04-e9ab739ef953" containerName="main" probeResult="failure" output="Get \"https://10.133.0.58:8000/health\": dial tcp 10.133.0.58:8000: connect: connection refused"
Apr 24 14:53:16.225499 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:53:16.225458    2574 prober.go:120] "Probe failed" probeType="Startup" pod="kserve-ci-e2e-test/llmisvc-model-fb-opt-125m-route-f312f5ec-kserve-mn-0" podUID="abf93af5-6a94-4a56-9e04-e9ab739ef953" containerName="main" probeResult="failure" output="Get \"https://10.133.0.58:8000/health\": dial tcp 10.133.0.58:8000: connect: connection refused"
Apr 24 14:53:26.234460 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:53:26.234428    2574 kubelet.go:2658] "SyncLoop (probe)" probe="startup" status="started" pod="kserve-ci-e2e-test/llmisvc-model-fb-opt-125m-route-f312f5ec-kserve-mn-0"
Apr 24 14:53:26.241915 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:53:26.241885    2574 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="kserve-ci-e2e-test/llmisvc-model-fb-opt-125m-route-f312f5ec-kserve-mn-0"
Apr 24 14:53:33.308790 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:53:33.308754    2574 kubelet.go:2553] "SyncLoop DELETE" source="api" pods=["kserve-ci-e2e-test/llmisvc4e643bc258191ffc517a31cd1d0ddd27-kserve-router-sches594d"]
Apr 24 14:53:33.309298 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:53:33.309079    2574 kuberuntime_container.go:864] "Killing container with a grace period" pod="kserve-ci-e2e-test/llmisvc4e643bc258191ffc517a31cd1d0ddd27-kserve-router-sches594d" podUID="2e6b4d90-4724-4430-a0c6-78ab68774e58" containerName="main" containerID="cri-o://cd8532e45668592121cf1a7b421534de0afe387326026693590a34b4d18c7495" gracePeriod=30
Apr 24 14:53:33.309298 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:53:33.309209    2574 kuberuntime_container.go:864] "Killing container with a grace period" pod="kserve-ci-e2e-test/llmisvc4e643bc258191ffc517a31cd1d0ddd27-kserve-router-sches594d" podUID="2e6b4d90-4724-4430-a0c6-78ab68774e58" containerName="tokenizer" containerID="cri-o://1b9d3724b82696fa337f496370d3936f733829e63d8296918b04a6ae0e4e21b8" gracePeriod=30
Apr 24 14:53:33.317480 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:53:33.317455    2574 kubelet.go:2553] "SyncLoop DELETE" source="api" pods=["kserve-ci-e2e-test/llmisvc-model-fb-opt-125m-route-f312f5ec-kserve-mn-0"]
Apr 24 14:53:33.317860 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:53:33.317813    2574 kuberuntime_container.go:864] "Killing container with a grace period" pod="kserve-ci-e2e-test/llmisvc-model-fb-opt-125m-route-f312f5ec-kserve-mn-0" podUID="abf93af5-6a94-4a56-9e04-e9ab739ef953" containerName="main" containerID="cri-o://88f06aaf57cc71f4fd14cafc31da217efa3c453749e50054e293acacdd57e9f3" gracePeriod=30
Apr 24 14:53:33.726877 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:53:33.726841    2574 generic.go:358] "Generic (PLEG): container finished" podID="2e6b4d90-4724-4430-a0c6-78ab68774e58" containerID="cd8532e45668592121cf1a7b421534de0afe387326026693590a34b4d18c7495" exitCode=0
Apr 24 14:53:33.727095 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:53:33.726895    2574 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/llmisvc4e643bc258191ffc517a31cd1d0ddd27-kserve-router-sches594d" event={"ID":"2e6b4d90-4724-4430-a0c6-78ab68774e58","Type":"ContainerDied","Data":"cd8532e45668592121cf1a7b421534de0afe387326026693590a34b4d18c7495"}
Apr 24 14:53:34.180464 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:53:34.180441    2574 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/llmisvc-model-fb-opt-125m-route-f312f5ec-kserve-mn-0"
Apr 24 14:53:34.269605 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:53:34.269518    2574 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/abf93af5-6a94-4a56-9e04-e9ab739ef953-tls-certs\") pod \"abf93af5-6a94-4a56-9e04-e9ab739ef953\" (UID: \"abf93af5-6a94-4a56-9e04-e9ab739ef953\") "
Apr 24 14:53:34.269605 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:53:34.269552    2574 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/abf93af5-6a94-4a56-9e04-e9ab739ef953-home\") pod \"abf93af5-6a94-4a56-9e04-e9ab739ef953\" (UID: \"abf93af5-6a94-4a56-9e04-e9ab739ef953\") "
Apr 24 14:53:34.269605 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:53:34.269577    2574 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"kube-api-access-qx4rt\" (UniqueName: \"kubernetes.io/projected/abf93af5-6a94-4a56-9e04-e9ab739ef953-kube-api-access-qx4rt\") pod \"abf93af5-6a94-4a56-9e04-e9ab739ef953\" (UID: \"abf93af5-6a94-4a56-9e04-e9ab739ef953\") "
Apr 24 14:53:34.269896 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:53:34.269610    2574 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/abf93af5-6a94-4a56-9e04-e9ab739ef953-kserve-provision-location\") pod \"abf93af5-6a94-4a56-9e04-e9ab739ef953\" (UID: \"abf93af5-6a94-4a56-9e04-e9ab739ef953\") "
Apr 24 14:53:34.269896 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:53:34.269689    2574 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/abf93af5-6a94-4a56-9e04-e9ab739ef953-model-cache\") pod \"abf93af5-6a94-4a56-9e04-e9ab739ef953\" (UID: \"abf93af5-6a94-4a56-9e04-e9ab739ef953\") "
Apr 24 14:53:34.269896 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:53:34.269730    2574 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/abf93af5-6a94-4a56-9e04-e9ab739ef953-dshm\") pod \"abf93af5-6a94-4a56-9e04-e9ab739ef953\" (UID: \"abf93af5-6a94-4a56-9e04-e9ab739ef953\") "
Apr 24 14:53:34.270051 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:53:34.269952    2574 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/abf93af5-6a94-4a56-9e04-e9ab739ef953-model-cache" (OuterVolumeSpecName: "model-cache") pod "abf93af5-6a94-4a56-9e04-e9ab739ef953" (UID: "abf93af5-6a94-4a56-9e04-e9ab739ef953"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Apr 24 14:53:34.270051 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:53:34.270012    2574 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/abf93af5-6a94-4a56-9e04-e9ab739ef953-home" (OuterVolumeSpecName: "home") pod "abf93af5-6a94-4a56-9e04-e9ab739ef953" (UID: "abf93af5-6a94-4a56-9e04-e9ab739ef953"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Apr 24 14:53:34.270132 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:53:34.270059    2574 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/abf93af5-6a94-4a56-9e04-e9ab739ef953-model-cache\") on node \"ip-10-0-136-132.ec2.internal\" DevicePath \"\""
Apr 24 14:53:34.271705 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:53:34.271681    2574 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/abf93af5-6a94-4a56-9e04-e9ab739ef953-kube-api-access-qx4rt" (OuterVolumeSpecName: "kube-api-access-qx4rt") pod "abf93af5-6a94-4a56-9e04-e9ab739ef953" (UID: "abf93af5-6a94-4a56-9e04-e9ab739ef953"). InnerVolumeSpecName "kube-api-access-qx4rt". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Apr 24 14:53:34.271873 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:53:34.271855    2574 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/abf93af5-6a94-4a56-9e04-e9ab739ef953-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "abf93af5-6a94-4a56-9e04-e9ab739ef953" (UID: "abf93af5-6a94-4a56-9e04-e9ab739ef953"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Apr 24 14:53:34.271954 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:53:34.271887    2574 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/abf93af5-6a94-4a56-9e04-e9ab739ef953-dshm" (OuterVolumeSpecName: "dshm") pod "abf93af5-6a94-4a56-9e04-e9ab739ef953" (UID: "abf93af5-6a94-4a56-9e04-e9ab739ef953"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Apr 24 14:53:34.341506 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:53:34.341475    2574 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/abf93af5-6a94-4a56-9e04-e9ab739ef953-kserve-provision-location" (OuterVolumeSpecName: "kserve-provision-location") pod "abf93af5-6a94-4a56-9e04-e9ab739ef953" (UID: "abf93af5-6a94-4a56-9e04-e9ab739ef953"). InnerVolumeSpecName "kserve-provision-location". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Apr 24 14:53:34.371525 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:53:34.371499    2574 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/abf93af5-6a94-4a56-9e04-e9ab739ef953-dshm\") on node \"ip-10-0-136-132.ec2.internal\" DevicePath \"\""
Apr 24 14:53:34.371525 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:53:34.371524    2574 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/abf93af5-6a94-4a56-9e04-e9ab739ef953-tls-certs\") on node \"ip-10-0-136-132.ec2.internal\" DevicePath \"\""
Apr 24 14:53:34.371640 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:53:34.371533    2574 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/abf93af5-6a94-4a56-9e04-e9ab739ef953-home\") on node \"ip-10-0-136-132.ec2.internal\" DevicePath \"\""
Apr 24 14:53:34.371640 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:53:34.371542    2574 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-qx4rt\" (UniqueName: \"kubernetes.io/projected/abf93af5-6a94-4a56-9e04-e9ab739ef953-kube-api-access-qx4rt\") on node \"ip-10-0-136-132.ec2.internal\" DevicePath \"\""
Apr 24 14:53:34.371640 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:53:34.371551    2574 reconciler_common.go:299] "Volume detached for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/abf93af5-6a94-4a56-9e04-e9ab739ef953-kserve-provision-location\") on node \"ip-10-0-136-132.ec2.internal\" DevicePath \"\""
Apr 24 14:53:34.563964 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:53:34.563944    2574 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/llmisvc4e643bc258191ffc517a31cd1d0ddd27-kserve-router-sches594d"
Apr 24 14:53:34.675097 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:53:34.675072    2574 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/2e6b4d90-4724-4430-a0c6-78ab68774e58-tls-certs\") pod \"2e6b4d90-4724-4430-a0c6-78ab68774e58\" (UID: \"2e6b4d90-4724-4430-a0c6-78ab68774e58\") "
Apr 24 14:53:34.675224 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:53:34.675103    2574 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/2e6b4d90-4724-4430-a0c6-78ab68774e58-kserve-provision-location\") pod \"2e6b4d90-4724-4430-a0c6-78ab68774e58\" (UID: \"2e6b4d90-4724-4430-a0c6-78ab68774e58\") "
Apr 24 14:53:34.675224 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:53:34.675148    2574 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"tokenizer-uds\" (UniqueName: \"kubernetes.io/empty-dir/2e6b4d90-4724-4430-a0c6-78ab68774e58-tokenizer-uds\") pod \"2e6b4d90-4724-4430-a0c6-78ab68774e58\" (UID: \"2e6b4d90-4724-4430-a0c6-78ab68774e58\") "
Apr 24 14:53:34.675224 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:53:34.675169    2574 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"kube-api-access-9hcd4\" (UniqueName: \"kubernetes.io/projected/2e6b4d90-4724-4430-a0c6-78ab68774e58-kube-api-access-9hcd4\") pod \"2e6b4d90-4724-4430-a0c6-78ab68774e58\" (UID: \"2e6b4d90-4724-4430-a0c6-78ab68774e58\") "
Apr 24 14:53:34.675224 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:53:34.675185    2574 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"tokenizer-tmp\" (UniqueName: \"kubernetes.io/empty-dir/2e6b4d90-4724-4430-a0c6-78ab68774e58-tokenizer-tmp\") pod \"2e6b4d90-4724-4430-a0c6-78ab68774e58\" (UID: \"2e6b4d90-4724-4430-a0c6-78ab68774e58\") "
Apr 24 14:53:34.675224 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:53:34.675222    2574 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"tokenizer-cache\" (UniqueName: \"kubernetes.io/empty-dir/2e6b4d90-4724-4430-a0c6-78ab68774e58-tokenizer-cache\") pod \"2e6b4d90-4724-4430-a0c6-78ab68774e58\" (UID: \"2e6b4d90-4724-4430-a0c6-78ab68774e58\") "
Apr 24 14:53:34.675534 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:53:34.675447    2574 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/2e6b4d90-4724-4430-a0c6-78ab68774e58-tokenizer-uds" (OuterVolumeSpecName: "tokenizer-uds") pod "2e6b4d90-4724-4430-a0c6-78ab68774e58" (UID: "2e6b4d90-4724-4430-a0c6-78ab68774e58"). InnerVolumeSpecName "tokenizer-uds". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Apr 24 14:53:34.675587 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:53:34.675561    2574 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/2e6b4d90-4724-4430-a0c6-78ab68774e58-tokenizer-cache" (OuterVolumeSpecName: "tokenizer-cache") pod "2e6b4d90-4724-4430-a0c6-78ab68774e58" (UID: "2e6b4d90-4724-4430-a0c6-78ab68774e58"). InnerVolumeSpecName "tokenizer-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Apr 24 14:53:34.675633 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:53:34.675592    2574 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/2e6b4d90-4724-4430-a0c6-78ab68774e58-tokenizer-tmp" (OuterVolumeSpecName: "tokenizer-tmp") pod "2e6b4d90-4724-4430-a0c6-78ab68774e58" (UID: "2e6b4d90-4724-4430-a0c6-78ab68774e58"). InnerVolumeSpecName "tokenizer-tmp". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Apr 24 14:53:34.675843 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:53:34.675820    2574 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/2e6b4d90-4724-4430-a0c6-78ab68774e58-kserve-provision-location" (OuterVolumeSpecName: "kserve-provision-location") pod "2e6b4d90-4724-4430-a0c6-78ab68774e58" (UID: "2e6b4d90-4724-4430-a0c6-78ab68774e58"). InnerVolumeSpecName "kserve-provision-location". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Apr 24 14:53:34.677113 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:53:34.677095    2574 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/2e6b4d90-4724-4430-a0c6-78ab68774e58-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "2e6b4d90-4724-4430-a0c6-78ab68774e58" (UID: "2e6b4d90-4724-4430-a0c6-78ab68774e58"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Apr 24 14:53:34.677220 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:53:34.677199    2574 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/2e6b4d90-4724-4430-a0c6-78ab68774e58-kube-api-access-9hcd4" (OuterVolumeSpecName: "kube-api-access-9hcd4") pod "2e6b4d90-4724-4430-a0c6-78ab68774e58" (UID: "2e6b4d90-4724-4430-a0c6-78ab68774e58"). InnerVolumeSpecName "kube-api-access-9hcd4". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Apr 24 14:53:34.732784 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:53:34.732755    2574 generic.go:358] "Generic (PLEG): container finished" podID="2e6b4d90-4724-4430-a0c6-78ab68774e58" containerID="1b9d3724b82696fa337f496370d3936f733829e63d8296918b04a6ae0e4e21b8" exitCode=0
Apr 24 14:53:34.732879 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:53:34.732837    2574 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/llmisvc4e643bc258191ffc517a31cd1d0ddd27-kserve-router-sches594d"
Apr 24 14:53:34.732926 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:53:34.732834    2574 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/llmisvc4e643bc258191ffc517a31cd1d0ddd27-kserve-router-sches594d" event={"ID":"2e6b4d90-4724-4430-a0c6-78ab68774e58","Type":"ContainerDied","Data":"1b9d3724b82696fa337f496370d3936f733829e63d8296918b04a6ae0e4e21b8"}
Apr 24 14:53:34.732973 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:53:34.732947    2574 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/llmisvc4e643bc258191ffc517a31cd1d0ddd27-kserve-router-sches594d" event={"ID":"2e6b4d90-4724-4430-a0c6-78ab68774e58","Type":"ContainerDied","Data":"fd6fd5b9f30536b9f9d74d93089c076a78be7e832e7a1a2cabd388d3d04b0d21"}
Apr 24 14:53:34.733010 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:53:34.732970    2574 scope.go:117] "RemoveContainer" containerID="1b9d3724b82696fa337f496370d3936f733829e63d8296918b04a6ae0e4e21b8"
Apr 24 14:53:34.734298 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:53:34.734277    2574 generic.go:358] "Generic (PLEG): container finished" podID="abf93af5-6a94-4a56-9e04-e9ab739ef953" containerID="88f06aaf57cc71f4fd14cafc31da217efa3c453749e50054e293acacdd57e9f3" exitCode=0
Apr 24 14:53:34.734396 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:53:34.734363    2574 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/llmisvc-model-fb-opt-125m-route-f312f5ec-kserve-mn-0" event={"ID":"abf93af5-6a94-4a56-9e04-e9ab739ef953","Type":"ContainerDied","Data":"88f06aaf57cc71f4fd14cafc31da217efa3c453749e50054e293acacdd57e9f3"}
Apr 24 14:53:34.734441 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:53:34.734396    2574 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/llmisvc-model-fb-opt-125m-route-f312f5ec-kserve-mn-0" event={"ID":"abf93af5-6a94-4a56-9e04-e9ab739ef953","Type":"ContainerDied","Data":"6709ed13639ee6dabd6aaf75bfd1f467f71d43e31574ebab3ca7fe39abf8cca6"}
Apr 24 14:53:34.734486 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:53:34.734434    2574 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/llmisvc-model-fb-opt-125m-route-f312f5ec-kserve-mn-0"
Apr 24 14:53:34.747072 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:53:34.747047    2574 scope.go:117] "RemoveContainer" containerID="cd8532e45668592121cf1a7b421534de0afe387326026693590a34b4d18c7495"
Apr 24 14:53:34.755754 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:53:34.755739    2574 scope.go:117] "RemoveContainer" containerID="4844b6a597af41e3f0ee4841f8c1ce925c2e382b0bd7d26ff7d15d3418767162"
Apr 24 14:53:34.764309 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:53:34.764291    2574 scope.go:117] "RemoveContainer" containerID="1b9d3724b82696fa337f496370d3936f733829e63d8296918b04a6ae0e4e21b8"
Apr 24 14:53:34.764608 ip-10-0-136-132 kubenswrapper[2574]: E0424 14:53:34.764593    2574 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"1b9d3724b82696fa337f496370d3936f733829e63d8296918b04a6ae0e4e21b8\": container with ID starting with 1b9d3724b82696fa337f496370d3936f733829e63d8296918b04a6ae0e4e21b8 not found: ID does not exist" containerID="1b9d3724b82696fa337f496370d3936f733829e63d8296918b04a6ae0e4e21b8"
Apr 24 14:53:34.764659 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:53:34.764614    2574 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"1b9d3724b82696fa337f496370d3936f733829e63d8296918b04a6ae0e4e21b8"} err="failed to get container status \"1b9d3724b82696fa337f496370d3936f733829e63d8296918b04a6ae0e4e21b8\": rpc error: code = NotFound desc = could not find container \"1b9d3724b82696fa337f496370d3936f733829e63d8296918b04a6ae0e4e21b8\": container with ID starting with 1b9d3724b82696fa337f496370d3936f733829e63d8296918b04a6ae0e4e21b8 not found: ID does not exist"
Apr 24 14:53:34.764659 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:53:34.764630    2574 scope.go:117] "RemoveContainer" containerID="cd8532e45668592121cf1a7b421534de0afe387326026693590a34b4d18c7495"
Apr 24 14:53:34.764866 ip-10-0-136-132 kubenswrapper[2574]: E0424 14:53:34.764847    2574 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"cd8532e45668592121cf1a7b421534de0afe387326026693590a34b4d18c7495\": container with ID starting with cd8532e45668592121cf1a7b421534de0afe387326026693590a34b4d18c7495 not found: ID does not exist" containerID="cd8532e45668592121cf1a7b421534de0afe387326026693590a34b4d18c7495"
Apr 24 14:53:34.764909 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:53:34.764873    2574 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"cd8532e45668592121cf1a7b421534de0afe387326026693590a34b4d18c7495"} err="failed to get container status \"cd8532e45668592121cf1a7b421534de0afe387326026693590a34b4d18c7495\": rpc error: code = NotFound desc = could not find container \"cd8532e45668592121cf1a7b421534de0afe387326026693590a34b4d18c7495\": container with ID starting with cd8532e45668592121cf1a7b421534de0afe387326026693590a34b4d18c7495 not found: ID does not exist"
Apr 24 14:53:34.764909 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:53:34.764889    2574 scope.go:117] "RemoveContainer" containerID="4844b6a597af41e3f0ee4841f8c1ce925c2e382b0bd7d26ff7d15d3418767162"
Apr 24 14:53:34.765137 ip-10-0-136-132 kubenswrapper[2574]: E0424 14:53:34.765110    2574 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"4844b6a597af41e3f0ee4841f8c1ce925c2e382b0bd7d26ff7d15d3418767162\": container with ID starting with 4844b6a597af41e3f0ee4841f8c1ce925c2e382b0bd7d26ff7d15d3418767162 not found: ID does not exist" containerID="4844b6a597af41e3f0ee4841f8c1ce925c2e382b0bd7d26ff7d15d3418767162"
Apr 24 14:53:34.765215 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:53:34.765143    2574 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"4844b6a597af41e3f0ee4841f8c1ce925c2e382b0bd7d26ff7d15d3418767162"} err="failed to get container status \"4844b6a597af41e3f0ee4841f8c1ce925c2e382b0bd7d26ff7d15d3418767162\": rpc error: code = NotFound desc = could not find container \"4844b6a597af41e3f0ee4841f8c1ce925c2e382b0bd7d26ff7d15d3418767162\": container with ID starting with 4844b6a597af41e3f0ee4841f8c1ce925c2e382b0bd7d26ff7d15d3418767162 not found: ID does not exist"
Apr 24 14:53:34.765215 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:53:34.765164    2574 scope.go:117] "RemoveContainer" containerID="88f06aaf57cc71f4fd14cafc31da217efa3c453749e50054e293acacdd57e9f3"
Apr 24 14:53:34.766493 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:53:34.766473    2574 kubelet.go:2553] "SyncLoop DELETE" source="api" pods=["kserve-ci-e2e-test/llmisvc4e643bc258191ffc517a31cd1d0ddd27-kserve-router-sches594d"]
Apr 24 14:53:34.768857 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:53:34.768838    2574 kubelet.go:2547] "SyncLoop REMOVE" source="api" pods=["kserve-ci-e2e-test/llmisvc4e643bc258191ffc517a31cd1d0ddd27-kserve-router-sches594d"]
Apr 24 14:53:34.776199 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:53:34.776182    2574 reconciler_common.go:299] "Volume detached for volume \"tokenizer-uds\" (UniqueName: \"kubernetes.io/empty-dir/2e6b4d90-4724-4430-a0c6-78ab68774e58-tokenizer-uds\") on node \"ip-10-0-136-132.ec2.internal\" DevicePath \"\""
Apr 24 14:53:34.776281 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:53:34.776202    2574 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-9hcd4\" (UniqueName: \"kubernetes.io/projected/2e6b4d90-4724-4430-a0c6-78ab68774e58-kube-api-access-9hcd4\") on node \"ip-10-0-136-132.ec2.internal\" DevicePath \"\""
Apr 24 14:53:34.776281 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:53:34.776213    2574 reconciler_common.go:299] "Volume detached for volume \"tokenizer-tmp\" (UniqueName: \"kubernetes.io/empty-dir/2e6b4d90-4724-4430-a0c6-78ab68774e58-tokenizer-tmp\") on node \"ip-10-0-136-132.ec2.internal\" DevicePath \"\""
Apr 24 14:53:34.776281 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:53:34.776223    2574 reconciler_common.go:299] "Volume detached for volume \"tokenizer-cache\" (UniqueName: \"kubernetes.io/empty-dir/2e6b4d90-4724-4430-a0c6-78ab68774e58-tokenizer-cache\") on node \"ip-10-0-136-132.ec2.internal\" DevicePath \"\""
Apr 24 14:53:34.776281 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:53:34.776238    2574 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/2e6b4d90-4724-4430-a0c6-78ab68774e58-tls-certs\") on node \"ip-10-0-136-132.ec2.internal\" DevicePath \"\""
Apr 24 14:53:34.776281 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:53:34.776247    2574 reconciler_common.go:299] "Volume detached for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/2e6b4d90-4724-4430-a0c6-78ab68774e58-kserve-provision-location\") on node \"ip-10-0-136-132.ec2.internal\" DevicePath \"\""
Apr 24 14:53:34.779095 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:53:34.779074    2574 kubelet.go:2553] "SyncLoop DELETE" source="api" pods=["kserve-ci-e2e-test/llmisvc-model-fb-opt-125m-route-f312f5ec-kserve-mn-0"]
Apr 24 14:53:34.782893 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:53:34.782871    2574 kubelet.go:2547] "SyncLoop REMOVE" source="api" pods=["kserve-ci-e2e-test/llmisvc-model-fb-opt-125m-route-f312f5ec-kserve-mn-0"]
Apr 24 14:53:34.786185 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:53:34.786168    2574 scope.go:117] "RemoveContainer" containerID="355fbe01f0a9622047d9cf3d679374aded099e09d1d5840fb19b37d7a758c814"
Apr 24 14:53:34.847585 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:53:34.847567    2574 scope.go:117] "RemoveContainer" containerID="88f06aaf57cc71f4fd14cafc31da217efa3c453749e50054e293acacdd57e9f3"
Apr 24 14:53:34.847873 ip-10-0-136-132 kubenswrapper[2574]: E0424 14:53:34.847854    2574 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"88f06aaf57cc71f4fd14cafc31da217efa3c453749e50054e293acacdd57e9f3\": container with ID starting with 88f06aaf57cc71f4fd14cafc31da217efa3c453749e50054e293acacdd57e9f3 not found: ID does not exist" containerID="88f06aaf57cc71f4fd14cafc31da217efa3c453749e50054e293acacdd57e9f3"
Apr 24 14:53:34.847934 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:53:34.847884    2574 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"88f06aaf57cc71f4fd14cafc31da217efa3c453749e50054e293acacdd57e9f3"} err="failed to get container status \"88f06aaf57cc71f4fd14cafc31da217efa3c453749e50054e293acacdd57e9f3\": rpc error: code = NotFound desc = could not find container \"88f06aaf57cc71f4fd14cafc31da217efa3c453749e50054e293acacdd57e9f3\": container with ID starting with 88f06aaf57cc71f4fd14cafc31da217efa3c453749e50054e293acacdd57e9f3 not found: ID does not exist"
Apr 24 14:53:34.847934 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:53:34.847908    2574 scope.go:117] "RemoveContainer" containerID="355fbe01f0a9622047d9cf3d679374aded099e09d1d5840fb19b37d7a758c814"
Apr 24 14:53:34.848121 ip-10-0-136-132 kubenswrapper[2574]: E0424 14:53:34.848105    2574 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"355fbe01f0a9622047d9cf3d679374aded099e09d1d5840fb19b37d7a758c814\": container with ID starting with 355fbe01f0a9622047d9cf3d679374aded099e09d1d5840fb19b37d7a758c814 not found: ID does not exist" containerID="355fbe01f0a9622047d9cf3d679374aded099e09d1d5840fb19b37d7a758c814"
Apr 24 14:53:34.848162 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:53:34.848124    2574 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"355fbe01f0a9622047d9cf3d679374aded099e09d1d5840fb19b37d7a758c814"} err="failed to get container status \"355fbe01f0a9622047d9cf3d679374aded099e09d1d5840fb19b37d7a758c814\": rpc error: code = NotFound desc = could not find container \"355fbe01f0a9622047d9cf3d679374aded099e09d1d5840fb19b37d7a758c814\": container with ID starting with 355fbe01f0a9622047d9cf3d679374aded099e09d1d5840fb19b37d7a758c814 not found: ID does not exist"
Apr 24 14:53:35.770140 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:53:35.770107    2574 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="2e6b4d90-4724-4430-a0c6-78ab68774e58" path="/var/lib/kubelet/pods/2e6b4d90-4724-4430-a0c6-78ab68774e58/volumes"
Apr 24 14:53:35.770593 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:53:35.770579    2574 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="abf93af5-6a94-4a56-9e04-e9ab739ef953" path="/var/lib/kubelet/pods/abf93af5-6a94-4a56-9e04-e9ab739ef953/volumes"
Apr 24 14:53:53.455181 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:53:53.455151    2574 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["kserve-ci-e2e-test/scheduler-inline-config-test-kserve-65579b6fcf-5z89v"]
Apr 24 14:53:53.455579 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:53:53.455533    2574 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="5fc35206-739e-42b8-b564-7a905fa24886" containerName="main"
Apr 24 14:53:53.455579 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:53:53.455545    2574 state_mem.go:107] "Deleted CPUSet assignment" podUID="5fc35206-739e-42b8-b564-7a905fa24886" containerName="main"
Apr 24 14:53:53.455579 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:53:53.455554    2574 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="abf93af5-6a94-4a56-9e04-e9ab739ef953" containerName="storage-initializer"
Apr 24 14:53:53.455579 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:53:53.455560    2574 state_mem.go:107] "Deleted CPUSet assignment" podUID="abf93af5-6a94-4a56-9e04-e9ab739ef953" containerName="storage-initializer"
Apr 24 14:53:53.455579 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:53:53.455578    2574 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="5fc35206-739e-42b8-b564-7a905fa24886" containerName="storage-initializer"
Apr 24 14:53:53.455738 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:53:53.455583    2574 state_mem.go:107] "Deleted CPUSet assignment" podUID="5fc35206-739e-42b8-b564-7a905fa24886" containerName="storage-initializer"
Apr 24 14:53:53.455738 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:53:53.455589    2574 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="2e6b4d90-4724-4430-a0c6-78ab68774e58" containerName="tokenizer"
Apr 24 14:53:53.455738 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:53:53.455594    2574 state_mem.go:107] "Deleted CPUSet assignment" podUID="2e6b4d90-4724-4430-a0c6-78ab68774e58" containerName="tokenizer"
Apr 24 14:53:53.455738 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:53:53.455610    2574 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="2e6b4d90-4724-4430-a0c6-78ab68774e58" containerName="main"
Apr 24 14:53:53.455738 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:53:53.455614    2574 state_mem.go:107] "Deleted CPUSet assignment" podUID="2e6b4d90-4724-4430-a0c6-78ab68774e58" containerName="main"
Apr 24 14:53:53.455738 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:53:53.455621    2574 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="abf93af5-6a94-4a56-9e04-e9ab739ef953" containerName="main"
Apr 24 14:53:53.455738 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:53:53.455626    2574 state_mem.go:107] "Deleted CPUSet assignment" podUID="abf93af5-6a94-4a56-9e04-e9ab739ef953" containerName="main"
Apr 24 14:53:53.455738 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:53:53.455633    2574 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="2e6b4d90-4724-4430-a0c6-78ab68774e58" containerName="storage-initializer"
Apr 24 14:53:53.455738 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:53:53.455637    2574 state_mem.go:107] "Deleted CPUSet assignment" podUID="2e6b4d90-4724-4430-a0c6-78ab68774e58" containerName="storage-initializer"
Apr 24 14:53:53.455738 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:53:53.455689    2574 memory_manager.go:356] "RemoveStaleState removing state" podUID="5fc35206-739e-42b8-b564-7a905fa24886" containerName="main"
Apr 24 14:53:53.455738 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:53:53.455698    2574 memory_manager.go:356] "RemoveStaleState removing state" podUID="2e6b4d90-4724-4430-a0c6-78ab68774e58" containerName="main"
Apr 24 14:53:53.455738 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:53:53.455706    2574 memory_manager.go:356] "RemoveStaleState removing state" podUID="2e6b4d90-4724-4430-a0c6-78ab68774e58" containerName="tokenizer"
Apr 24 14:53:53.455738 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:53:53.455713    2574 memory_manager.go:356] "RemoveStaleState removing state" podUID="abf93af5-6a94-4a56-9e04-e9ab739ef953" containerName="main"
Apr 24 14:53:53.458773 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:53:53.458759    2574 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/scheduler-inline-config-test-kserve-65579b6fcf-5z89v"
Apr 24 14:53:53.461169 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:53:53.461145    2574 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"kserve-ci-e2e-test\"/\"default-dockercfg-92j8s\""
Apr 24 14:53:53.461303 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:53:53.461174    2574 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"kserve-ci-e2e-test\"/\"scheduler-inline-config-test-kserve-self-signed-certs\""
Apr 24 14:53:53.461303 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:53:53.461214    2574 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kserve-ci-e2e-test\"/\"openshift-service-ca.crt\""
Apr 24 14:53:53.461303 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:53:53.461228    2574 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kserve-ci-e2e-test\"/\"kube-root-ca.crt\""
Apr 24 14:53:53.467462 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:53:53.467439    2574 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/scheduler-inline-config-test-kserve-65579b6fcf-5z89v"]
Apr 24 14:53:53.515076 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:53:53.515045    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/d35f930f-7c4c-4018-a53c-8c15bbac6fa4-tls-certs\") pod \"scheduler-inline-config-test-kserve-65579b6fcf-5z89v\" (UID: \"d35f930f-7c4c-4018-a53c-8c15bbac6fa4\") " pod="kserve-ci-e2e-test/scheduler-inline-config-test-kserve-65579b6fcf-5z89v"
Apr 24 14:53:53.515218 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:53:53.515089    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/d35f930f-7c4c-4018-a53c-8c15bbac6fa4-dshm\") pod \"scheduler-inline-config-test-kserve-65579b6fcf-5z89v\" (UID: \"d35f930f-7c4c-4018-a53c-8c15bbac6fa4\") " pod="kserve-ci-e2e-test/scheduler-inline-config-test-kserve-65579b6fcf-5z89v"
Apr 24 14:53:53.515218 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:53:53.515185    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-s7gg6\" (UniqueName: \"kubernetes.io/projected/d35f930f-7c4c-4018-a53c-8c15bbac6fa4-kube-api-access-s7gg6\") pod \"scheduler-inline-config-test-kserve-65579b6fcf-5z89v\" (UID: \"d35f930f-7c4c-4018-a53c-8c15bbac6fa4\") " pod="kserve-ci-e2e-test/scheduler-inline-config-test-kserve-65579b6fcf-5z89v"
Apr 24 14:53:53.515218 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:53:53.515214    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/d35f930f-7c4c-4018-a53c-8c15bbac6fa4-model-cache\") pod \"scheduler-inline-config-test-kserve-65579b6fcf-5z89v\" (UID: \"d35f930f-7c4c-4018-a53c-8c15bbac6fa4\") " pod="kserve-ci-e2e-test/scheduler-inline-config-test-kserve-65579b6fcf-5z89v"
Apr 24 14:53:53.515482 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:53:53.515298    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/d35f930f-7c4c-4018-a53c-8c15bbac6fa4-kserve-provision-location\") pod \"scheduler-inline-config-test-kserve-65579b6fcf-5z89v\" (UID: \"d35f930f-7c4c-4018-a53c-8c15bbac6fa4\") " pod="kserve-ci-e2e-test/scheduler-inline-config-test-kserve-65579b6fcf-5z89v"
Apr 24 14:53:53.515482 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:53:53.515329    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/d35f930f-7c4c-4018-a53c-8c15bbac6fa4-home\") pod \"scheduler-inline-config-test-kserve-65579b6fcf-5z89v\" (UID: \"d35f930f-7c4c-4018-a53c-8c15bbac6fa4\") " pod="kserve-ci-e2e-test/scheduler-inline-config-test-kserve-65579b6fcf-5z89v"
Apr 24 14:53:53.616136 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:53:53.616111    2574 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/d35f930f-7c4c-4018-a53c-8c15bbac6fa4-kserve-provision-location\") pod \"scheduler-inline-config-test-kserve-65579b6fcf-5z89v\" (UID: \"d35f930f-7c4c-4018-a53c-8c15bbac6fa4\") " pod="kserve-ci-e2e-test/scheduler-inline-config-test-kserve-65579b6fcf-5z89v"
Apr 24 14:53:53.616250 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:53:53.616139    2574 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/d35f930f-7c4c-4018-a53c-8c15bbac6fa4-home\") pod \"scheduler-inline-config-test-kserve-65579b6fcf-5z89v\" (UID: \"d35f930f-7c4c-4018-a53c-8c15bbac6fa4\") " pod="kserve-ci-e2e-test/scheduler-inline-config-test-kserve-65579b6fcf-5z89v"
Apr 24 14:53:53.616250 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:53:53.616163    2574 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/d35f930f-7c4c-4018-a53c-8c15bbac6fa4-tls-certs\") pod \"scheduler-inline-config-test-kserve-65579b6fcf-5z89v\" (UID: \"d35f930f-7c4c-4018-a53c-8c15bbac6fa4\") " pod="kserve-ci-e2e-test/scheduler-inline-config-test-kserve-65579b6fcf-5z89v"
Apr 24 14:53:53.616250 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:53:53.616181    2574 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/d35f930f-7c4c-4018-a53c-8c15bbac6fa4-dshm\") pod \"scheduler-inline-config-test-kserve-65579b6fcf-5z89v\" (UID: \"d35f930f-7c4c-4018-a53c-8c15bbac6fa4\") " pod="kserve-ci-e2e-test/scheduler-inline-config-test-kserve-65579b6fcf-5z89v"
Apr 24 14:53:53.616408 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:53:53.616379    2574 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-s7gg6\" (UniqueName: \"kubernetes.io/projected/d35f930f-7c4c-4018-a53c-8c15bbac6fa4-kube-api-access-s7gg6\") pod \"scheduler-inline-config-test-kserve-65579b6fcf-5z89v\" (UID: \"d35f930f-7c4c-4018-a53c-8c15bbac6fa4\") " pod="kserve-ci-e2e-test/scheduler-inline-config-test-kserve-65579b6fcf-5z89v"
Apr 24 14:53:53.616447 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:53:53.616426    2574 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/d35f930f-7c4c-4018-a53c-8c15bbac6fa4-model-cache\") pod \"scheduler-inline-config-test-kserve-65579b6fcf-5z89v\" (UID: \"d35f930f-7c4c-4018-a53c-8c15bbac6fa4\") " pod="kserve-ci-e2e-test/scheduler-inline-config-test-kserve-65579b6fcf-5z89v"
Apr 24 14:53:53.616567 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:53:53.616547    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/d35f930f-7c4c-4018-a53c-8c15bbac6fa4-kserve-provision-location\") pod \"scheduler-inline-config-test-kserve-65579b6fcf-5z89v\" (UID: \"d35f930f-7c4c-4018-a53c-8c15bbac6fa4\") " pod="kserve-ci-e2e-test/scheduler-inline-config-test-kserve-65579b6fcf-5z89v"
Apr 24 14:53:53.616611 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:53:53.616592    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/d35f930f-7c4c-4018-a53c-8c15bbac6fa4-home\") pod \"scheduler-inline-config-test-kserve-65579b6fcf-5z89v\" (UID: \"d35f930f-7c4c-4018-a53c-8c15bbac6fa4\") " pod="kserve-ci-e2e-test/scheduler-inline-config-test-kserve-65579b6fcf-5z89v"
Apr 24 14:53:53.616785 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:53:53.616768    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/d35f930f-7c4c-4018-a53c-8c15bbac6fa4-model-cache\") pod \"scheduler-inline-config-test-kserve-65579b6fcf-5z89v\" (UID: \"d35f930f-7c4c-4018-a53c-8c15bbac6fa4\") " pod="kserve-ci-e2e-test/scheduler-inline-config-test-kserve-65579b6fcf-5z89v"
Apr 24 14:53:53.618452 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:53:53.618431    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/d35f930f-7c4c-4018-a53c-8c15bbac6fa4-dshm\") pod \"scheduler-inline-config-test-kserve-65579b6fcf-5z89v\" (UID: \"d35f930f-7c4c-4018-a53c-8c15bbac6fa4\") " pod="kserve-ci-e2e-test/scheduler-inline-config-test-kserve-65579b6fcf-5z89v"
Apr 24 14:53:53.618780 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:53:53.618762    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/d35f930f-7c4c-4018-a53c-8c15bbac6fa4-tls-certs\") pod \"scheduler-inline-config-test-kserve-65579b6fcf-5z89v\" (UID: \"d35f930f-7c4c-4018-a53c-8c15bbac6fa4\") " pod="kserve-ci-e2e-test/scheduler-inline-config-test-kserve-65579b6fcf-5z89v"
Apr 24 14:53:53.624104 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:53:53.624083    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-s7gg6\" (UniqueName: \"kubernetes.io/projected/d35f930f-7c4c-4018-a53c-8c15bbac6fa4-kube-api-access-s7gg6\") pod \"scheduler-inline-config-test-kserve-65579b6fcf-5z89v\" (UID: \"d35f930f-7c4c-4018-a53c-8c15bbac6fa4\") " pod="kserve-ci-e2e-test/scheduler-inline-config-test-kserve-65579b6fcf-5z89v"
Apr 24 14:53:53.678720 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:53:53.678695    2574 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["kserve-ci-e2e-test/scheduler-inline-config-test-kserve-router-scheduler-55cc9h69nc"]
Apr 24 14:53:53.682684 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:53:53.682668    2574 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/scheduler-inline-config-test-kserve-router-scheduler-55cc9h69nc"
Apr 24 14:53:53.684658 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:53:53.684632    2574 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"kserve-ci-e2e-test\"/\"scheduler-inline-config-test-epp-sa-dockercfg-csqtr\""
Apr 24 14:53:53.692974 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:53:53.692954    2574 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/scheduler-inline-config-test-kserve-router-scheduler-55cc9h69nc"]
Apr 24 14:53:53.717576 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:53:53.717525    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-r8n87\" (UniqueName: \"kubernetes.io/projected/b68197be-59b3-4668-9c73-4dcad99fe7e6-kube-api-access-r8n87\") pod \"scheduler-inline-config-test-kserve-router-scheduler-55cc9h69nc\" (UID: \"b68197be-59b3-4668-9c73-4dcad99fe7e6\") " pod="kserve-ci-e2e-test/scheduler-inline-config-test-kserve-router-scheduler-55cc9h69nc"
Apr 24 14:53:53.717576 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:53:53.717555    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/b68197be-59b3-4668-9c73-4dcad99fe7e6-tls-certs\") pod \"scheduler-inline-config-test-kserve-router-scheduler-55cc9h69nc\" (UID: \"b68197be-59b3-4668-9c73-4dcad99fe7e6\") " pod="kserve-ci-e2e-test/scheduler-inline-config-test-kserve-router-scheduler-55cc9h69nc"
Apr 24 14:53:53.717736 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:53:53.717584    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tokenizer-uds\" (UniqueName: \"kubernetes.io/empty-dir/b68197be-59b3-4668-9c73-4dcad99fe7e6-tokenizer-uds\") pod \"scheduler-inline-config-test-kserve-router-scheduler-55cc9h69nc\" (UID: \"b68197be-59b3-4668-9c73-4dcad99fe7e6\") " pod="kserve-ci-e2e-test/scheduler-inline-config-test-kserve-router-scheduler-55cc9h69nc"
Apr 24 14:53:53.717736 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:53:53.717657    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tokenizer-tmp\" (UniqueName: \"kubernetes.io/empty-dir/b68197be-59b3-4668-9c73-4dcad99fe7e6-tokenizer-tmp\") pod \"scheduler-inline-config-test-kserve-router-scheduler-55cc9h69nc\" (UID: \"b68197be-59b3-4668-9c73-4dcad99fe7e6\") " pod="kserve-ci-e2e-test/scheduler-inline-config-test-kserve-router-scheduler-55cc9h69nc"
Apr 24 14:53:53.717736 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:53:53.717689    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tokenizer-cache\" (UniqueName: \"kubernetes.io/empty-dir/b68197be-59b3-4668-9c73-4dcad99fe7e6-tokenizer-cache\") pod \"scheduler-inline-config-test-kserve-router-scheduler-55cc9h69nc\" (UID: \"b68197be-59b3-4668-9c73-4dcad99fe7e6\") " pod="kserve-ci-e2e-test/scheduler-inline-config-test-kserve-router-scheduler-55cc9h69nc"
Apr 24 14:53:53.717736 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:53:53.717706    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/b68197be-59b3-4668-9c73-4dcad99fe7e6-kserve-provision-location\") pod \"scheduler-inline-config-test-kserve-router-scheduler-55cc9h69nc\" (UID: \"b68197be-59b3-4668-9c73-4dcad99fe7e6\") " pod="kserve-ci-e2e-test/scheduler-inline-config-test-kserve-router-scheduler-55cc9h69nc"
Apr 24 14:53:53.770009 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:53:53.769990    2574 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/scheduler-inline-config-test-kserve-65579b6fcf-5z89v"
Apr 24 14:53:53.819840 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:53:53.819802    2574 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"tokenizer-uds\" (UniqueName: \"kubernetes.io/empty-dir/b68197be-59b3-4668-9c73-4dcad99fe7e6-tokenizer-uds\") pod \"scheduler-inline-config-test-kserve-router-scheduler-55cc9h69nc\" (UID: \"b68197be-59b3-4668-9c73-4dcad99fe7e6\") " pod="kserve-ci-e2e-test/scheduler-inline-config-test-kserve-router-scheduler-55cc9h69nc"
Apr 24 14:53:53.820759 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:53:53.820507    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tokenizer-uds\" (UniqueName: \"kubernetes.io/empty-dir/b68197be-59b3-4668-9c73-4dcad99fe7e6-tokenizer-uds\") pod \"scheduler-inline-config-test-kserve-router-scheduler-55cc9h69nc\" (UID: \"b68197be-59b3-4668-9c73-4dcad99fe7e6\") " pod="kserve-ci-e2e-test/scheduler-inline-config-test-kserve-router-scheduler-55cc9h69nc"
Apr 24 14:53:53.820925 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:53:53.820898    2574 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"tokenizer-tmp\" (UniqueName: \"kubernetes.io/empty-dir/b68197be-59b3-4668-9c73-4dcad99fe7e6-tokenizer-tmp\") pod \"scheduler-inline-config-test-kserve-router-scheduler-55cc9h69nc\" (UID: \"b68197be-59b3-4668-9c73-4dcad99fe7e6\") " pod="kserve-ci-e2e-test/scheduler-inline-config-test-kserve-router-scheduler-55cc9h69nc"
Apr 24 14:53:53.821041 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:53:53.820967    2574 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"tokenizer-cache\" (UniqueName: \"kubernetes.io/empty-dir/b68197be-59b3-4668-9c73-4dcad99fe7e6-tokenizer-cache\") pod \"scheduler-inline-config-test-kserve-router-scheduler-55cc9h69nc\" (UID: \"b68197be-59b3-4668-9c73-4dcad99fe7e6\") " pod="kserve-ci-e2e-test/scheduler-inline-config-test-kserve-router-scheduler-55cc9h69nc"
Apr 24 14:53:53.821041 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:53:53.820995    2574 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/b68197be-59b3-4668-9c73-4dcad99fe7e6-kserve-provision-location\") pod \"scheduler-inline-config-test-kserve-router-scheduler-55cc9h69nc\" (UID: \"b68197be-59b3-4668-9c73-4dcad99fe7e6\") " pod="kserve-ci-e2e-test/scheduler-inline-config-test-kserve-router-scheduler-55cc9h69nc"
Apr 24 14:53:53.821134 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:53:53.821099    2574 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-r8n87\" (UniqueName: \"kubernetes.io/projected/b68197be-59b3-4668-9c73-4dcad99fe7e6-kube-api-access-r8n87\") pod \"scheduler-inline-config-test-kserve-router-scheduler-55cc9h69nc\" (UID: \"b68197be-59b3-4668-9c73-4dcad99fe7e6\") " pod="kserve-ci-e2e-test/scheduler-inline-config-test-kserve-router-scheduler-55cc9h69nc"
Apr 24 14:53:53.821189 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:53:53.821134    2574 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/b68197be-59b3-4668-9c73-4dcad99fe7e6-tls-certs\") pod \"scheduler-inline-config-test-kserve-router-scheduler-55cc9h69nc\" (UID: \"b68197be-59b3-4668-9c73-4dcad99fe7e6\") " pod="kserve-ci-e2e-test/scheduler-inline-config-test-kserve-router-scheduler-55cc9h69nc"
Apr 24 14:53:53.821466 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:53:53.821444    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tokenizer-tmp\" (UniqueName: \"kubernetes.io/empty-dir/b68197be-59b3-4668-9c73-4dcad99fe7e6-tokenizer-tmp\") pod \"scheduler-inline-config-test-kserve-router-scheduler-55cc9h69nc\" (UID: \"b68197be-59b3-4668-9c73-4dcad99fe7e6\") " pod="kserve-ci-e2e-test/scheduler-inline-config-test-kserve-router-scheduler-55cc9h69nc"
Apr 24 14:53:53.821656 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:53:53.821632    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tokenizer-cache\" (UniqueName: \"kubernetes.io/empty-dir/b68197be-59b3-4668-9c73-4dcad99fe7e6-tokenizer-cache\") pod \"scheduler-inline-config-test-kserve-router-scheduler-55cc9h69nc\" (UID: \"b68197be-59b3-4668-9c73-4dcad99fe7e6\") " pod="kserve-ci-e2e-test/scheduler-inline-config-test-kserve-router-scheduler-55cc9h69nc"
Apr 24 14:53:53.822147 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:53:53.821883    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/b68197be-59b3-4668-9c73-4dcad99fe7e6-kserve-provision-location\") pod \"scheduler-inline-config-test-kserve-router-scheduler-55cc9h69nc\" (UID: \"b68197be-59b3-4668-9c73-4dcad99fe7e6\") " pod="kserve-ci-e2e-test/scheduler-inline-config-test-kserve-router-scheduler-55cc9h69nc"
Apr 24 14:53:53.824051 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:53:53.824007    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/b68197be-59b3-4668-9c73-4dcad99fe7e6-tls-certs\") pod \"scheduler-inline-config-test-kserve-router-scheduler-55cc9h69nc\" (UID: \"b68197be-59b3-4668-9c73-4dcad99fe7e6\") " pod="kserve-ci-e2e-test/scheduler-inline-config-test-kserve-router-scheduler-55cc9h69nc"
Apr 24 14:53:53.830082 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:53:53.830058    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-r8n87\" (UniqueName: \"kubernetes.io/projected/b68197be-59b3-4668-9c73-4dcad99fe7e6-kube-api-access-r8n87\") pod \"scheduler-inline-config-test-kserve-router-scheduler-55cc9h69nc\" (UID: \"b68197be-59b3-4668-9c73-4dcad99fe7e6\") " pod="kserve-ci-e2e-test/scheduler-inline-config-test-kserve-router-scheduler-55cc9h69nc"
Apr 24 14:53:53.901358 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:53:53.901315    2574 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/scheduler-inline-config-test-kserve-65579b6fcf-5z89v"]
Apr 24 14:53:53.902246 ip-10-0-136-132 kubenswrapper[2574]: W0424 14:53:53.902222    2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd35f930f_7c4c_4018_a53c_8c15bbac6fa4.slice/crio-24daa71ee10b4fe837e1d57d6f1b2e3f38d4b240aef1033c7447c3463d4193e5 WatchSource:0}: Error finding container 24daa71ee10b4fe837e1d57d6f1b2e3f38d4b240aef1033c7447c3463d4193e5: Status 404 returned error can't find the container with id 24daa71ee10b4fe837e1d57d6f1b2e3f38d4b240aef1033c7447c3463d4193e5
Apr 24 14:53:53.994248 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:53:53.994187    2574 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/scheduler-inline-config-test-kserve-router-scheduler-55cc9h69nc"
Apr 24 14:53:54.324738 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:53:54.324714    2574 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/scheduler-inline-config-test-kserve-router-scheduler-55cc9h69nc"]
Apr 24 14:53:54.325430 ip-10-0-136-132 kubenswrapper[2574]: W0424 14:53:54.325399    2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb68197be_59b3_4668_9c73_4dcad99fe7e6.slice/crio-4d1ce94fe00ddfef55387c87cd78cc350069b7d092f8c04e449c0c2770c02f90 WatchSource:0}: Error finding container 4d1ce94fe00ddfef55387c87cd78cc350069b7d092f8c04e449c0c2770c02f90: Status 404 returned error can't find the container with id 4d1ce94fe00ddfef55387c87cd78cc350069b7d092f8c04e449c0c2770c02f90
Apr 24 14:53:54.822367 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:53:54.822312    2574 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/scheduler-inline-config-test-kserve-router-scheduler-55cc9h69nc" event={"ID":"b68197be-59b3-4668-9c73-4dcad99fe7e6","Type":"ContainerStarted","Data":"14e67617011216aacbc1cd1c4ab32e4b4a55b1f796eb49fbd20b341a334391a7"}
Apr 24 14:53:54.822367 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:53:54.822373    2574 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/scheduler-inline-config-test-kserve-router-scheduler-55cc9h69nc" event={"ID":"b68197be-59b3-4668-9c73-4dcad99fe7e6","Type":"ContainerStarted","Data":"4d1ce94fe00ddfef55387c87cd78cc350069b7d092f8c04e449c0c2770c02f90"}
Apr 24 14:53:54.823847 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:53:54.823820    2574 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/scheduler-inline-config-test-kserve-65579b6fcf-5z89v" event={"ID":"d35f930f-7c4c-4018-a53c-8c15bbac6fa4","Type":"ContainerStarted","Data":"3f197441fbc3b5a59a6e954c67c91b95e12a93b952b9ce369a6dfff5cac44612"}
Apr 24 14:53:54.823847 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:53:54.823849    2574 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/scheduler-inline-config-test-kserve-65579b6fcf-5z89v" event={"ID":"d35f930f-7c4c-4018-a53c-8c15bbac6fa4","Type":"ContainerStarted","Data":"24daa71ee10b4fe837e1d57d6f1b2e3f38d4b240aef1033c7447c3463d4193e5"}
Apr 24 14:53:55.828407 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:53:55.828371    2574 generic.go:358] "Generic (PLEG): container finished" podID="b68197be-59b3-4668-9c73-4dcad99fe7e6" containerID="14e67617011216aacbc1cd1c4ab32e4b4a55b1f796eb49fbd20b341a334391a7" exitCode=0
Apr 24 14:53:55.828853 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:53:55.828455    2574 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/scheduler-inline-config-test-kserve-router-scheduler-55cc9h69nc" event={"ID":"b68197be-59b3-4668-9c73-4dcad99fe7e6","Type":"ContainerDied","Data":"14e67617011216aacbc1cd1c4ab32e4b4a55b1f796eb49fbd20b341a334391a7"}
Apr 24 14:53:56.834571 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:53:56.834487    2574 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/scheduler-inline-config-test-kserve-router-scheduler-55cc9h69nc" event={"ID":"b68197be-59b3-4668-9c73-4dcad99fe7e6","Type":"ContainerStarted","Data":"53cda8472102865068cac1162666ec8425df18a3c485c4f046bf374b5e61b141"}
Apr 24 14:53:56.834571 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:53:56.834530    2574 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/scheduler-inline-config-test-kserve-router-scheduler-55cc9h69nc" event={"ID":"b68197be-59b3-4668-9c73-4dcad99fe7e6","Type":"ContainerStarted","Data":"d8f48bb5ff95df5dc2839bfd507a0d85fa4fe3a6fe3fcf68cf1286642da1d13d"}
Apr 24 14:53:56.834987 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:53:56.834633    2574 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve-ci-e2e-test/scheduler-inline-config-test-kserve-router-scheduler-55cc9h69nc"
Apr 24 14:53:56.855015 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:53:56.854972    2574 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="kserve-ci-e2e-test/scheduler-inline-config-test-kserve-router-scheduler-55cc9h69nc" podStartSLOduration=3.8549591420000002 podStartE2EDuration="3.854959142s" podCreationTimestamp="2026-04-24 14:53:53 +0000 UTC" firstStartedPulling="0001-01-01 00:00:00 +0000 UTC" lastFinishedPulling="0001-01-01 00:00:00 +0000 UTC" observedRunningTime="2026-04-24 14:53:56.851914988 +0000 UTC m=+1799.584016665" watchObservedRunningTime="2026-04-24 14:53:56.854959142 +0000 UTC m=+1799.587060790"
Apr 24 14:53:57.883428 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:53:57.883399    2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-9d4b6777b-f6ngr_2571829f-666a-422a-9b02-2d2bb4c89b25/console-operator/1.log"
Apr 24 14:53:57.887961 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:53:57.887939    2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-9d4b6777b-f6ngr_2571829f-666a-422a-9b02-2d2bb4c89b25/console-operator/1.log"
Apr 24 14:53:57.888652 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:53:57.888636    2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-w5x5x_44036c3e-1fdc-40d4-899c-9009cd4589e3/ovn-acl-logging/0.log"
Apr 24 14:53:57.892877 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:53:57.892861    2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-w5x5x_44036c3e-1fdc-40d4-899c-9009cd4589e3/ovn-acl-logging/0.log"
Apr 24 14:54:03.863242 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:54:03.863157    2574 generic.go:358] "Generic (PLEG): container finished" podID="d35f930f-7c4c-4018-a53c-8c15bbac6fa4" containerID="3f197441fbc3b5a59a6e954c67c91b95e12a93b952b9ce369a6dfff5cac44612" exitCode=0
Apr 24 14:54:03.863242 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:54:03.863225    2574 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/scheduler-inline-config-test-kserve-65579b6fcf-5z89v" event={"ID":"d35f930f-7c4c-4018-a53c-8c15bbac6fa4","Type":"ContainerDied","Data":"3f197441fbc3b5a59a6e954c67c91b95e12a93b952b9ce369a6dfff5cac44612"}
Apr 24 14:54:03.995347 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:54:03.995304    2574 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve-ci-e2e-test/scheduler-inline-config-test-kserve-router-scheduler-55cc9h69nc"
Apr 24 14:54:03.995480 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:54:03.995374    2574 kubelet.go:2658] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="kserve-ci-e2e-test/scheduler-inline-config-test-kserve-router-scheduler-55cc9h69nc"
Apr 24 14:54:03.998109 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:54:03.998083    2574 kubelet.go:2658] "SyncLoop (probe)" probe="startup" status="started" pod="kserve-ci-e2e-test/scheduler-inline-config-test-kserve-router-scheduler-55cc9h69nc"
Apr 24 14:54:04.869245 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:54:04.869207    2574 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/scheduler-inline-config-test-kserve-65579b6fcf-5z89v" event={"ID":"d35f930f-7c4c-4018-a53c-8c15bbac6fa4","Type":"ContainerStarted","Data":"30ccfd910d93b7f02259cc7ebb35fcc50432d7d7553b7c46914484a204f1be31"}
Apr 24 14:54:04.870732 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:54:04.870710    2574 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="kserve-ci-e2e-test/scheduler-inline-config-test-kserve-router-scheduler-55cc9h69nc"
Apr 24 14:54:04.886747 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:54:04.886700    2574 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="kserve-ci-e2e-test/scheduler-inline-config-test-kserve-65579b6fcf-5z89v" podStartSLOduration=11.886686815000001 podStartE2EDuration="11.886686815s" podCreationTimestamp="2026-04-24 14:53:53 +0000 UTC" firstStartedPulling="0001-01-01 00:00:00 +0000 UTC" lastFinishedPulling="0001-01-01 00:00:00 +0000 UTC" observedRunningTime="2026-04-24 14:54:04.885211067 +0000 UTC m=+1807.617312730" watchObservedRunningTime="2026-04-24 14:54:04.886686815 +0000 UTC m=+1807.618788463"
Apr 24 14:54:13.770304 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:54:13.770273    2574 kubelet.go:2658] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="kserve-ci-e2e-test/scheduler-inline-config-test-kserve-65579b6fcf-5z89v"
Apr 24 14:54:13.770783 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:54:13.770314    2574 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve-ci-e2e-test/scheduler-inline-config-test-kserve-65579b6fcf-5z89v"
Apr 24 14:54:13.782903 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:54:13.782879    2574 kubelet.go:2658] "SyncLoop (probe)" probe="startup" status="started" pod="kserve-ci-e2e-test/scheduler-inline-config-test-kserve-65579b6fcf-5z89v"
Apr 24 14:54:13.915879 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:54:13.915849    2574 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="kserve-ci-e2e-test/scheduler-inline-config-test-kserve-65579b6fcf-5z89v"
Apr 24 14:54:25.875196 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:54:25.875167    2574 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="kserve-ci-e2e-test/scheduler-inline-config-test-kserve-router-scheduler-55cc9h69nc"
Apr 24 14:54:26.861760 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:54:26.861730    2574 kubelet.go:2553] "SyncLoop DELETE" source="api" pods=["kserve-ci-e2e-test/scheduler-inline-config-test-kserve-65579b6fcf-5z89v"]
Apr 24 14:54:26.862034 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:54:26.862009    2574 kuberuntime_container.go:864] "Killing container with a grace period" pod="kserve-ci-e2e-test/scheduler-inline-config-test-kserve-65579b6fcf-5z89v" podUID="d35f930f-7c4c-4018-a53c-8c15bbac6fa4" containerName="main" containerID="cri-o://30ccfd910d93b7f02259cc7ebb35fcc50432d7d7553b7c46914484a204f1be31" gracePeriod=30
Apr 24 14:54:26.865749 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:54:26.865723    2574 kubelet.go:2553] "SyncLoop DELETE" source="api" pods=["kserve-ci-e2e-test/scheduler-inline-config-test-kserve-router-scheduler-55cc9h69nc"]
Apr 24 14:54:26.866074 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:54:26.866049    2574 kuberuntime_container.go:864] "Killing container with a grace period" pod="kserve-ci-e2e-test/scheduler-inline-config-test-kserve-router-scheduler-55cc9h69nc" podUID="b68197be-59b3-4668-9c73-4dcad99fe7e6" containerName="main" containerID="cri-o://d8f48bb5ff95df5dc2839bfd507a0d85fa4fe3a6fe3fcf68cf1286642da1d13d" gracePeriod=30
Apr 24 14:54:26.866176 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:54:26.866079    2574 kuberuntime_container.go:864] "Killing container with a grace period" pod="kserve-ci-e2e-test/scheduler-inline-config-test-kserve-router-scheduler-55cc9h69nc" podUID="b68197be-59b3-4668-9c73-4dcad99fe7e6" containerName="tokenizer" containerID="cri-o://53cda8472102865068cac1162666ec8425df18a3c485c4f046bf374b5e61b141" gracePeriod=30
Apr 24 14:54:27.109829 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:54:27.109805    2574 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/scheduler-inline-config-test-kserve-65579b6fcf-5z89v"
Apr 24 14:54:27.206951 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:54:27.206919    2574 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/d35f930f-7c4c-4018-a53c-8c15bbac6fa4-tls-certs\") pod \"d35f930f-7c4c-4018-a53c-8c15bbac6fa4\" (UID: \"d35f930f-7c4c-4018-a53c-8c15bbac6fa4\") "
Apr 24 14:54:27.207128 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:54:27.206969    2574 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/d35f930f-7c4c-4018-a53c-8c15bbac6fa4-kserve-provision-location\") pod \"d35f930f-7c4c-4018-a53c-8c15bbac6fa4\" (UID: \"d35f930f-7c4c-4018-a53c-8c15bbac6fa4\") "
Apr 24 14:54:27.207128 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:54:27.206991    2574 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/d35f930f-7c4c-4018-a53c-8c15bbac6fa4-model-cache\") pod \"d35f930f-7c4c-4018-a53c-8c15bbac6fa4\" (UID: \"d35f930f-7c4c-4018-a53c-8c15bbac6fa4\") "
Apr 24 14:54:27.207238 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:54:27.207127    2574 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/d35f930f-7c4c-4018-a53c-8c15bbac6fa4-home\") pod \"d35f930f-7c4c-4018-a53c-8c15bbac6fa4\" (UID: \"d35f930f-7c4c-4018-a53c-8c15bbac6fa4\") "
Apr 24 14:54:27.207238 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:54:27.207187    2574 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"kube-api-access-s7gg6\" (UniqueName: \"kubernetes.io/projected/d35f930f-7c4c-4018-a53c-8c15bbac6fa4-kube-api-access-s7gg6\") pod \"d35f930f-7c4c-4018-a53c-8c15bbac6fa4\" (UID: \"d35f930f-7c4c-4018-a53c-8c15bbac6fa4\") "
Apr 24 14:54:27.207378 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:54:27.207243    2574 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d35f930f-7c4c-4018-a53c-8c15bbac6fa4-model-cache" (OuterVolumeSpecName: "model-cache") pod "d35f930f-7c4c-4018-a53c-8c15bbac6fa4" (UID: "d35f930f-7c4c-4018-a53c-8c15bbac6fa4"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Apr 24 14:54:27.207378 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:54:27.207272    2574 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/d35f930f-7c4c-4018-a53c-8c15bbac6fa4-dshm\") pod \"d35f930f-7c4c-4018-a53c-8c15bbac6fa4\" (UID: \"d35f930f-7c4c-4018-a53c-8c15bbac6fa4\") "
Apr 24 14:54:27.207494 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:54:27.207430    2574 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d35f930f-7c4c-4018-a53c-8c15bbac6fa4-home" (OuterVolumeSpecName: "home") pod "d35f930f-7c4c-4018-a53c-8c15bbac6fa4" (UID: "d35f930f-7c4c-4018-a53c-8c15bbac6fa4"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Apr 24 14:54:27.207638 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:54:27.207616    2574 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/d35f930f-7c4c-4018-a53c-8c15bbac6fa4-model-cache\") on node \"ip-10-0-136-132.ec2.internal\" DevicePath \"\""
Apr 24 14:54:27.207727 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:54:27.207645    2574 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/d35f930f-7c4c-4018-a53c-8c15bbac6fa4-home\") on node \"ip-10-0-136-132.ec2.internal\" DevicePath \"\""
Apr 24 14:54:27.209747 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:54:27.209718    2574 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d35f930f-7c4c-4018-a53c-8c15bbac6fa4-dshm" (OuterVolumeSpecName: "dshm") pod "d35f930f-7c4c-4018-a53c-8c15bbac6fa4" (UID: "d35f930f-7c4c-4018-a53c-8c15bbac6fa4"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Apr 24 14:54:27.209747 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:54:27.209718    2574 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d35f930f-7c4c-4018-a53c-8c15bbac6fa4-kube-api-access-s7gg6" (OuterVolumeSpecName: "kube-api-access-s7gg6") pod "d35f930f-7c4c-4018-a53c-8c15bbac6fa4" (UID: "d35f930f-7c4c-4018-a53c-8c15bbac6fa4"). InnerVolumeSpecName "kube-api-access-s7gg6". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Apr 24 14:54:27.209747 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:54:27.209724    2574 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/d35f930f-7c4c-4018-a53c-8c15bbac6fa4-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "d35f930f-7c4c-4018-a53c-8c15bbac6fa4" (UID: "d35f930f-7c4c-4018-a53c-8c15bbac6fa4"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Apr 24 14:54:27.267755 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:54:27.267711    2574 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d35f930f-7c4c-4018-a53c-8c15bbac6fa4-kserve-provision-location" (OuterVolumeSpecName: "kserve-provision-location") pod "d35f930f-7c4c-4018-a53c-8c15bbac6fa4" (UID: "d35f930f-7c4c-4018-a53c-8c15bbac6fa4"). InnerVolumeSpecName "kserve-provision-location". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Apr 24 14:54:27.308616 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:54:27.308579    2574 reconciler_common.go:299] "Volume detached for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/d35f930f-7c4c-4018-a53c-8c15bbac6fa4-kserve-provision-location\") on node \"ip-10-0-136-132.ec2.internal\" DevicePath \"\""
Apr 24 14:54:27.308616 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:54:27.308612    2574 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-s7gg6\" (UniqueName: \"kubernetes.io/projected/d35f930f-7c4c-4018-a53c-8c15bbac6fa4-kube-api-access-s7gg6\") on node \"ip-10-0-136-132.ec2.internal\" DevicePath \"\""
Apr 24 14:54:27.308616 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:54:27.308622    2574 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/d35f930f-7c4c-4018-a53c-8c15bbac6fa4-dshm\") on node \"ip-10-0-136-132.ec2.internal\" DevicePath \"\""
Apr 24 14:54:27.308843 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:54:27.308633    2574 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/d35f930f-7c4c-4018-a53c-8c15bbac6fa4-tls-certs\") on node \"ip-10-0-136-132.ec2.internal\" DevicePath \"\""
Apr 24 14:54:27.961760 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:54:27.961721    2574 generic.go:358] "Generic (PLEG): container finished" podID="b68197be-59b3-4668-9c73-4dcad99fe7e6" containerID="53cda8472102865068cac1162666ec8425df18a3c485c4f046bf374b5e61b141" exitCode=0
Apr 24 14:54:27.961760 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:54:27.961748    2574 generic.go:358] "Generic (PLEG): container finished" podID="b68197be-59b3-4668-9c73-4dcad99fe7e6" containerID="d8f48bb5ff95df5dc2839bfd507a0d85fa4fe3a6fe3fcf68cf1286642da1d13d" exitCode=0
Apr 24 14:54:27.961959 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:54:27.961799    2574 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/scheduler-inline-config-test-kserve-router-scheduler-55cc9h69nc" event={"ID":"b68197be-59b3-4668-9c73-4dcad99fe7e6","Type":"ContainerDied","Data":"53cda8472102865068cac1162666ec8425df18a3c485c4f046bf374b5e61b141"}
Apr 24 14:54:27.961959 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:54:27.961840    2574 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/scheduler-inline-config-test-kserve-router-scheduler-55cc9h69nc" event={"ID":"b68197be-59b3-4668-9c73-4dcad99fe7e6","Type":"ContainerDied","Data":"d8f48bb5ff95df5dc2839bfd507a0d85fa4fe3a6fe3fcf68cf1286642da1d13d"}
Apr 24 14:54:27.963256 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:54:27.963235    2574 generic.go:358] "Generic (PLEG): container finished" podID="d35f930f-7c4c-4018-a53c-8c15bbac6fa4" containerID="30ccfd910d93b7f02259cc7ebb35fcc50432d7d7553b7c46914484a204f1be31" exitCode=0
Apr 24 14:54:27.963395 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:54:27.963320    2574 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/scheduler-inline-config-test-kserve-65579b6fcf-5z89v"
Apr 24 14:54:27.963395 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:54:27.963322    2574 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/scheduler-inline-config-test-kserve-65579b6fcf-5z89v" event={"ID":"d35f930f-7c4c-4018-a53c-8c15bbac6fa4","Type":"ContainerDied","Data":"30ccfd910d93b7f02259cc7ebb35fcc50432d7d7553b7c46914484a204f1be31"}
Apr 24 14:54:27.963395 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:54:27.963379    2574 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/scheduler-inline-config-test-kserve-65579b6fcf-5z89v" event={"ID":"d35f930f-7c4c-4018-a53c-8c15bbac6fa4","Type":"ContainerDied","Data":"24daa71ee10b4fe837e1d57d6f1b2e3f38d4b240aef1033c7447c3463d4193e5"}
Apr 24 14:54:27.963553 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:54:27.963400    2574 scope.go:117] "RemoveContainer" containerID="30ccfd910d93b7f02259cc7ebb35fcc50432d7d7553b7c46914484a204f1be31"
Apr 24 14:54:27.972694 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:54:27.972676    2574 scope.go:117] "RemoveContainer" containerID="3f197441fbc3b5a59a6e954c67c91b95e12a93b952b9ce369a6dfff5cac44612"
Apr 24 14:54:27.981852 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:54:27.981830    2574 kubelet.go:2553] "SyncLoop DELETE" source="api" pods=["kserve-ci-e2e-test/scheduler-inline-config-test-kserve-65579b6fcf-5z89v"]
Apr 24 14:54:27.986266 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:54:27.986245    2574 kubelet.go:2547] "SyncLoop REMOVE" source="api" pods=["kserve-ci-e2e-test/scheduler-inline-config-test-kserve-65579b6fcf-5z89v"]
Apr 24 14:54:28.041043 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:54:28.041011    2574 scope.go:117] "RemoveContainer" containerID="30ccfd910d93b7f02259cc7ebb35fcc50432d7d7553b7c46914484a204f1be31"
Apr 24 14:54:28.041381 ip-10-0-136-132 kubenswrapper[2574]: E0424 14:54:28.041360    2574 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"30ccfd910d93b7f02259cc7ebb35fcc50432d7d7553b7c46914484a204f1be31\": container with ID starting with 30ccfd910d93b7f02259cc7ebb35fcc50432d7d7553b7c46914484a204f1be31 not found: ID does not exist" containerID="30ccfd910d93b7f02259cc7ebb35fcc50432d7d7553b7c46914484a204f1be31"
Apr 24 14:54:28.041477 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:54:28.041392    2574 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"30ccfd910d93b7f02259cc7ebb35fcc50432d7d7553b7c46914484a204f1be31"} err="failed to get container status \"30ccfd910d93b7f02259cc7ebb35fcc50432d7d7553b7c46914484a204f1be31\": rpc error: code = NotFound desc = could not find container \"30ccfd910d93b7f02259cc7ebb35fcc50432d7d7553b7c46914484a204f1be31\": container with ID starting with 30ccfd910d93b7f02259cc7ebb35fcc50432d7d7553b7c46914484a204f1be31 not found: ID does not exist"
Apr 24 14:54:28.041477 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:54:28.041420    2574 scope.go:117] "RemoveContainer" containerID="3f197441fbc3b5a59a6e954c67c91b95e12a93b952b9ce369a6dfff5cac44612"
Apr 24 14:54:28.041692 ip-10-0-136-132 kubenswrapper[2574]: E0424 14:54:28.041668    2574 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"3f197441fbc3b5a59a6e954c67c91b95e12a93b952b9ce369a6dfff5cac44612\": container with ID starting with 3f197441fbc3b5a59a6e954c67c91b95e12a93b952b9ce369a6dfff5cac44612 not found: ID does not exist" containerID="3f197441fbc3b5a59a6e954c67c91b95e12a93b952b9ce369a6dfff5cac44612"
Apr 24 14:54:28.041738 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:54:28.041701    2574 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"3f197441fbc3b5a59a6e954c67c91b95e12a93b952b9ce369a6dfff5cac44612"} err="failed to get container status \"3f197441fbc3b5a59a6e954c67c91b95e12a93b952b9ce369a6dfff5cac44612\": rpc error: code = NotFound desc = could not find container \"3f197441fbc3b5a59a6e954c67c91b95e12a93b952b9ce369a6dfff5cac44612\": container with ID starting with 3f197441fbc3b5a59a6e954c67c91b95e12a93b952b9ce369a6dfff5cac44612 not found: ID does not exist"
Apr 24 14:54:28.060534 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:54:28.060518    2574 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/scheduler-inline-config-test-kserve-router-scheduler-55cc9h69nc"
Apr 24 14:54:28.215752 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:54:28.215723    2574 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"tokenizer-uds\" (UniqueName: \"kubernetes.io/empty-dir/b68197be-59b3-4668-9c73-4dcad99fe7e6-tokenizer-uds\") pod \"b68197be-59b3-4668-9c73-4dcad99fe7e6\" (UID: \"b68197be-59b3-4668-9c73-4dcad99fe7e6\") "
Apr 24 14:54:28.216093 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:54:28.215817    2574 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"kube-api-access-r8n87\" (UniqueName: \"kubernetes.io/projected/b68197be-59b3-4668-9c73-4dcad99fe7e6-kube-api-access-r8n87\") pod \"b68197be-59b3-4668-9c73-4dcad99fe7e6\" (UID: \"b68197be-59b3-4668-9c73-4dcad99fe7e6\") "
Apr 24 14:54:28.216093 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:54:28.215857    2574 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/b68197be-59b3-4668-9c73-4dcad99fe7e6-kserve-provision-location\") pod \"b68197be-59b3-4668-9c73-4dcad99fe7e6\" (UID: \"b68197be-59b3-4668-9c73-4dcad99fe7e6\") "
Apr 24 14:54:28.216093 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:54:28.215885    2574 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"tokenizer-tmp\" (UniqueName: \"kubernetes.io/empty-dir/b68197be-59b3-4668-9c73-4dcad99fe7e6-tokenizer-tmp\") pod \"b68197be-59b3-4668-9c73-4dcad99fe7e6\" (UID: \"b68197be-59b3-4668-9c73-4dcad99fe7e6\") "
Apr 24 14:54:28.216093 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:54:28.215936    2574 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"tokenizer-cache\" (UniqueName: \"kubernetes.io/empty-dir/b68197be-59b3-4668-9c73-4dcad99fe7e6-tokenizer-cache\") pod \"b68197be-59b3-4668-9c73-4dcad99fe7e6\" (UID: \"b68197be-59b3-4668-9c73-4dcad99fe7e6\") "
Apr 24 14:54:28.216093 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:54:28.215965    2574 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/b68197be-59b3-4668-9c73-4dcad99fe7e6-tls-certs\") pod \"b68197be-59b3-4668-9c73-4dcad99fe7e6\" (UID: \"b68197be-59b3-4668-9c73-4dcad99fe7e6\") "
Apr 24 14:54:28.216093 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:54:28.216009    2574 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b68197be-59b3-4668-9c73-4dcad99fe7e6-tokenizer-uds" (OuterVolumeSpecName: "tokenizer-uds") pod "b68197be-59b3-4668-9c73-4dcad99fe7e6" (UID: "b68197be-59b3-4668-9c73-4dcad99fe7e6"). InnerVolumeSpecName "tokenizer-uds". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Apr 24 14:54:28.216391 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:54:28.216194    2574 reconciler_common.go:299] "Volume detached for volume \"tokenizer-uds\" (UniqueName: \"kubernetes.io/empty-dir/b68197be-59b3-4668-9c73-4dcad99fe7e6-tokenizer-uds\") on node \"ip-10-0-136-132.ec2.internal\" DevicePath \"\""
Apr 24 14:54:28.216391 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:54:28.216197    2574 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b68197be-59b3-4668-9c73-4dcad99fe7e6-tokenizer-cache" (OuterVolumeSpecName: "tokenizer-cache") pod "b68197be-59b3-4668-9c73-4dcad99fe7e6" (UID: "b68197be-59b3-4668-9c73-4dcad99fe7e6"). InnerVolumeSpecName "tokenizer-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Apr 24 14:54:28.216391 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:54:28.216257    2574 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b68197be-59b3-4668-9c73-4dcad99fe7e6-tokenizer-tmp" (OuterVolumeSpecName: "tokenizer-tmp") pod "b68197be-59b3-4668-9c73-4dcad99fe7e6" (UID: "b68197be-59b3-4668-9c73-4dcad99fe7e6"). InnerVolumeSpecName "tokenizer-tmp". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Apr 24 14:54:28.216590 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:54:28.216572    2574 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b68197be-59b3-4668-9c73-4dcad99fe7e6-kserve-provision-location" (OuterVolumeSpecName: "kserve-provision-location") pod "b68197be-59b3-4668-9c73-4dcad99fe7e6" (UID: "b68197be-59b3-4668-9c73-4dcad99fe7e6"). InnerVolumeSpecName "kserve-provision-location". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Apr 24 14:54:28.217923 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:54:28.217905    2574 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/b68197be-59b3-4668-9c73-4dcad99fe7e6-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "b68197be-59b3-4668-9c73-4dcad99fe7e6" (UID: "b68197be-59b3-4668-9c73-4dcad99fe7e6"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Apr 24 14:54:28.218013 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:54:28.217994    2574 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/b68197be-59b3-4668-9c73-4dcad99fe7e6-kube-api-access-r8n87" (OuterVolumeSpecName: "kube-api-access-r8n87") pod "b68197be-59b3-4668-9c73-4dcad99fe7e6" (UID: "b68197be-59b3-4668-9c73-4dcad99fe7e6"). InnerVolumeSpecName "kube-api-access-r8n87". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Apr 24 14:54:28.317516 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:54:28.317459    2574 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-r8n87\" (UniqueName: \"kubernetes.io/projected/b68197be-59b3-4668-9c73-4dcad99fe7e6-kube-api-access-r8n87\") on node \"ip-10-0-136-132.ec2.internal\" DevicePath \"\""
Apr 24 14:54:28.317516 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:54:28.317479    2574 reconciler_common.go:299] "Volume detached for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/b68197be-59b3-4668-9c73-4dcad99fe7e6-kserve-provision-location\") on node \"ip-10-0-136-132.ec2.internal\" DevicePath \"\""
Apr 24 14:54:28.317516 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:54:28.317489    2574 reconciler_common.go:299] "Volume detached for volume \"tokenizer-tmp\" (UniqueName: \"kubernetes.io/empty-dir/b68197be-59b3-4668-9c73-4dcad99fe7e6-tokenizer-tmp\") on node \"ip-10-0-136-132.ec2.internal\" DevicePath \"\""
Apr 24 14:54:28.317516 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:54:28.317498    2574 reconciler_common.go:299] "Volume detached for volume \"tokenizer-cache\" (UniqueName: \"kubernetes.io/empty-dir/b68197be-59b3-4668-9c73-4dcad99fe7e6-tokenizer-cache\") on node \"ip-10-0-136-132.ec2.internal\" DevicePath \"\""
Apr 24 14:54:28.317516 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:54:28.317507    2574 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/b68197be-59b3-4668-9c73-4dcad99fe7e6-tls-certs\") on node \"ip-10-0-136-132.ec2.internal\" DevicePath \"\""
Apr 24 14:54:28.969273 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:54:28.969233    2574 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/scheduler-inline-config-test-kserve-router-scheduler-55cc9h69nc" event={"ID":"b68197be-59b3-4668-9c73-4dcad99fe7e6","Type":"ContainerDied","Data":"4d1ce94fe00ddfef55387c87cd78cc350069b7d092f8c04e449c0c2770c02f90"}
Apr 24 14:54:28.969484 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:54:28.969290    2574 scope.go:117] "RemoveContainer" containerID="53cda8472102865068cac1162666ec8425df18a3c485c4f046bf374b5e61b141"
Apr 24 14:54:28.969484 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:54:28.969305    2574 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/scheduler-inline-config-test-kserve-router-scheduler-55cc9h69nc"
Apr 24 14:54:28.978674 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:54:28.978654    2574 scope.go:117] "RemoveContainer" containerID="d8f48bb5ff95df5dc2839bfd507a0d85fa4fe3a6fe3fcf68cf1286642da1d13d"
Apr 24 14:54:28.986138 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:54:28.986122    2574 scope.go:117] "RemoveContainer" containerID="14e67617011216aacbc1cd1c4ab32e4b4a55b1f796eb49fbd20b341a334391a7"
Apr 24 14:54:28.991319 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:54:28.991295    2574 kubelet.go:2553] "SyncLoop DELETE" source="api" pods=["kserve-ci-e2e-test/scheduler-inline-config-test-kserve-router-scheduler-55cc9h69nc"]
Apr 24 14:54:28.994312 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:54:28.994291    2574 kubelet.go:2547] "SyncLoop REMOVE" source="api" pods=["kserve-ci-e2e-test/scheduler-inline-config-test-kserve-router-scheduler-55cc9h69nc"]
Apr 24 14:54:29.770795 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:54:29.770758    2574 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="b68197be-59b3-4668-9c73-4dcad99fe7e6" path="/var/lib/kubelet/pods/b68197be-59b3-4668-9c73-4dcad99fe7e6/volumes"
Apr 24 14:54:29.771244 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:54:29.771228    2574 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="d35f930f-7c4c-4018-a53c-8c15bbac6fa4" path="/var/lib/kubelet/pods/d35f930f-7c4c-4018-a53c-8c15bbac6fa4/volumes"
Apr 24 14:54:35.887553 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:54:35.887522    2574 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["kserve-ci-e2e-test/custom-route-timeout-pd-test-kserve-7cbf8d89d-8j8f9"]
Apr 24 14:54:35.889823 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:54:35.887935    2574 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b68197be-59b3-4668-9c73-4dcad99fe7e6" containerName="storage-initializer"
Apr 24 14:54:35.889823 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:54:35.887948    2574 state_mem.go:107] "Deleted CPUSet assignment" podUID="b68197be-59b3-4668-9c73-4dcad99fe7e6" containerName="storage-initializer"
Apr 24 14:54:35.889823 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:54:35.887957    2574 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d35f930f-7c4c-4018-a53c-8c15bbac6fa4" containerName="storage-initializer"
Apr 24 14:54:35.889823 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:54:35.887964    2574 state_mem.go:107] "Deleted CPUSet assignment" podUID="d35f930f-7c4c-4018-a53c-8c15bbac6fa4" containerName="storage-initializer"
Apr 24 14:54:35.889823 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:54:35.887974    2574 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b68197be-59b3-4668-9c73-4dcad99fe7e6" containerName="main"
Apr 24 14:54:35.889823 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:54:35.887983    2574 state_mem.go:107] "Deleted CPUSet assignment" podUID="b68197be-59b3-4668-9c73-4dcad99fe7e6" containerName="main"
Apr 24 14:54:35.889823 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:54:35.887991    2574 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d35f930f-7c4c-4018-a53c-8c15bbac6fa4" containerName="main"
Apr 24 14:54:35.889823 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:54:35.887996    2574 state_mem.go:107] "Deleted CPUSet assignment" podUID="d35f930f-7c4c-4018-a53c-8c15bbac6fa4" containerName="main"
Apr 24 14:54:35.889823 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:54:35.888011    2574 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b68197be-59b3-4668-9c73-4dcad99fe7e6" containerName="tokenizer"
Apr 24 14:54:35.889823 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:54:35.888015    2574 state_mem.go:107] "Deleted CPUSet assignment" podUID="b68197be-59b3-4668-9c73-4dcad99fe7e6" containerName="tokenizer"
Apr 24 14:54:35.889823 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:54:35.888069    2574 memory_manager.go:356] "RemoveStaleState removing state" podUID="d35f930f-7c4c-4018-a53c-8c15bbac6fa4" containerName="main"
Apr 24 14:54:35.889823 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:54:35.888078    2574 memory_manager.go:356] "RemoveStaleState removing state" podUID="b68197be-59b3-4668-9c73-4dcad99fe7e6" containerName="main"
Apr 24 14:54:35.889823 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:54:35.888088    2574 memory_manager.go:356] "RemoveStaleState removing state" podUID="b68197be-59b3-4668-9c73-4dcad99fe7e6" containerName="tokenizer"
Apr 24 14:54:35.891078 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:54:35.891062    2574 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/custom-route-timeout-pd-test-kserve-7cbf8d89d-8j8f9"
Apr 24 14:54:35.893213 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:54:35.893185    2574 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kserve-ci-e2e-test\"/\"openshift-service-ca.crt\""
Apr 24 14:54:35.893359 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:54:35.893210    2574 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"kserve-ci-e2e-test\"/\"custom-route-timeout-pd-test-kserve-self-signed-certs\""
Apr 24 14:54:35.893359 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:54:35.893185    2574 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"kserve-ci-e2e-test\"/\"custom-route-timeout-pd-test-kserve-dockercfg-glkxv\""
Apr 24 14:54:35.893359 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:54:35.893238    2574 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kserve-ci-e2e-test\"/\"kube-root-ca.crt\""
Apr 24 14:54:35.893359 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:54:35.893264    2574 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"kserve-ci-e2e-test\"/\"default-dockercfg-92j8s\""
Apr 24 14:54:35.897109 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:54:35.897087    2574 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["kserve-ci-e2e-test/custom-route-timeout-pd-test-kserve-prefill-578d77b467-zfd6z"]
Apr 24 14:54:35.900764 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:54:35.900739    2574 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/custom-route-timeout-pd-test-kserve-prefill-578d77b467-zfd6z"
Apr 24 14:54:35.904025 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:54:35.904000    2574 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/custom-route-timeout-pd-test-kserve-7cbf8d89d-8j8f9"]
Apr 24 14:54:35.912528 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:54:35.912508    2574 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/custom-route-timeout-pd-test-kserve-prefill-578d77b467-zfd6z"]
Apr 24 14:54:36.074399 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:54:36.074363    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/8ad639c0-0908-443a-9a36-949955a89ee7-tls-certs\") pod \"custom-route-timeout-pd-test-kserve-7cbf8d89d-8j8f9\" (UID: \"8ad639c0-0908-443a-9a36-949955a89ee7\") " pod="kserve-ci-e2e-test/custom-route-timeout-pd-test-kserve-7cbf8d89d-8j8f9"
Apr 24 14:54:36.074399 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:54:36.074397    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/73d63045-437e-49a4-be0c-cfbd579d93f5-home\") pod \"custom-route-timeout-pd-test-kserve-prefill-578d77b467-zfd6z\" (UID: \"73d63045-437e-49a4-be0c-cfbd579d93f5\") " pod="kserve-ci-e2e-test/custom-route-timeout-pd-test-kserve-prefill-578d77b467-zfd6z"
Apr 24 14:54:36.074712 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:54:36.074518    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mw4fg\" (UniqueName: \"kubernetes.io/projected/8ad639c0-0908-443a-9a36-949955a89ee7-kube-api-access-mw4fg\") pod \"custom-route-timeout-pd-test-kserve-7cbf8d89d-8j8f9\" (UID: \"8ad639c0-0908-443a-9a36-949955a89ee7\") " pod="kserve-ci-e2e-test/custom-route-timeout-pd-test-kserve-7cbf8d89d-8j8f9"
Apr 24 14:54:36.074712 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:54:36.074579    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9h88x\" (UniqueName: \"kubernetes.io/projected/73d63045-437e-49a4-be0c-cfbd579d93f5-kube-api-access-9h88x\") pod \"custom-route-timeout-pd-test-kserve-prefill-578d77b467-zfd6z\" (UID: \"73d63045-437e-49a4-be0c-cfbd579d93f5\") " pod="kserve-ci-e2e-test/custom-route-timeout-pd-test-kserve-prefill-578d77b467-zfd6z"
Apr 24 14:54:36.074712 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:54:36.074608    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/73d63045-437e-49a4-be0c-cfbd579d93f5-model-cache\") pod \"custom-route-timeout-pd-test-kserve-prefill-578d77b467-zfd6z\" (UID: \"73d63045-437e-49a4-be0c-cfbd579d93f5\") " pod="kserve-ci-e2e-test/custom-route-timeout-pd-test-kserve-prefill-578d77b467-zfd6z"
Apr 24 14:54:36.074712 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:54:36.074632    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/73d63045-437e-49a4-be0c-cfbd579d93f5-tls-certs\") pod \"custom-route-timeout-pd-test-kserve-prefill-578d77b467-zfd6z\" (UID: \"73d63045-437e-49a4-be0c-cfbd579d93f5\") " pod="kserve-ci-e2e-test/custom-route-timeout-pd-test-kserve-prefill-578d77b467-zfd6z"
Apr 24 14:54:36.074712 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:54:36.074694    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/8ad639c0-0908-443a-9a36-949955a89ee7-home\") pod \"custom-route-timeout-pd-test-kserve-7cbf8d89d-8j8f9\" (UID: \"8ad639c0-0908-443a-9a36-949955a89ee7\") " pod="kserve-ci-e2e-test/custom-route-timeout-pd-test-kserve-7cbf8d89d-8j8f9"
Apr 24 14:54:36.074920 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:54:36.074719    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/73d63045-437e-49a4-be0c-cfbd579d93f5-kserve-provision-location\") pod \"custom-route-timeout-pd-test-kserve-prefill-578d77b467-zfd6z\" (UID: \"73d63045-437e-49a4-be0c-cfbd579d93f5\") " pod="kserve-ci-e2e-test/custom-route-timeout-pd-test-kserve-prefill-578d77b467-zfd6z"
Apr 24 14:54:36.074920 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:54:36.074767    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/8ad639c0-0908-443a-9a36-949955a89ee7-kserve-provision-location\") pod \"custom-route-timeout-pd-test-kserve-7cbf8d89d-8j8f9\" (UID: \"8ad639c0-0908-443a-9a36-949955a89ee7\") " pod="kserve-ci-e2e-test/custom-route-timeout-pd-test-kserve-7cbf8d89d-8j8f9"
Apr 24 14:54:36.074920 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:54:36.074797    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/73d63045-437e-49a4-be0c-cfbd579d93f5-dshm\") pod \"custom-route-timeout-pd-test-kserve-prefill-578d77b467-zfd6z\" (UID: \"73d63045-437e-49a4-be0c-cfbd579d93f5\") " pod="kserve-ci-e2e-test/custom-route-timeout-pd-test-kserve-prefill-578d77b467-zfd6z"
Apr 24 14:54:36.074920 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:54:36.074816    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/8ad639c0-0908-443a-9a36-949955a89ee7-dshm\") pod \"custom-route-timeout-pd-test-kserve-7cbf8d89d-8j8f9\" (UID: \"8ad639c0-0908-443a-9a36-949955a89ee7\") " pod="kserve-ci-e2e-test/custom-route-timeout-pd-test-kserve-7cbf8d89d-8j8f9"
Apr 24 14:54:36.074920 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:54:36.074852    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/8ad639c0-0908-443a-9a36-949955a89ee7-model-cache\") pod \"custom-route-timeout-pd-test-kserve-7cbf8d89d-8j8f9\" (UID: \"8ad639c0-0908-443a-9a36-949955a89ee7\") " pod="kserve-ci-e2e-test/custom-route-timeout-pd-test-kserve-7cbf8d89d-8j8f9"
Apr 24 14:54:36.176168 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:54:36.176090    2574 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/8ad639c0-0908-443a-9a36-949955a89ee7-kserve-provision-location\") pod \"custom-route-timeout-pd-test-kserve-7cbf8d89d-8j8f9\" (UID: \"8ad639c0-0908-443a-9a36-949955a89ee7\") " pod="kserve-ci-e2e-test/custom-route-timeout-pd-test-kserve-7cbf8d89d-8j8f9"
Apr 24 14:54:36.176168 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:54:36.176136    2574 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/73d63045-437e-49a4-be0c-cfbd579d93f5-dshm\") pod \"custom-route-timeout-pd-test-kserve-prefill-578d77b467-zfd6z\" (UID: \"73d63045-437e-49a4-be0c-cfbd579d93f5\") " pod="kserve-ci-e2e-test/custom-route-timeout-pd-test-kserve-prefill-578d77b467-zfd6z"
Apr 24 14:54:36.176413 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:54:36.176258    2574 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/8ad639c0-0908-443a-9a36-949955a89ee7-dshm\") pod \"custom-route-timeout-pd-test-kserve-7cbf8d89d-8j8f9\" (UID: \"8ad639c0-0908-443a-9a36-949955a89ee7\") " pod="kserve-ci-e2e-test/custom-route-timeout-pd-test-kserve-7cbf8d89d-8j8f9"
Apr 24 14:54:36.176413 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:54:36.176318    2574 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/8ad639c0-0908-443a-9a36-949955a89ee7-model-cache\") pod \"custom-route-timeout-pd-test-kserve-7cbf8d89d-8j8f9\" (UID: \"8ad639c0-0908-443a-9a36-949955a89ee7\") " pod="kserve-ci-e2e-test/custom-route-timeout-pd-test-kserve-7cbf8d89d-8j8f9"
Apr 24 14:54:36.176413 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:54:36.176372    2574 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/8ad639c0-0908-443a-9a36-949955a89ee7-tls-certs\") pod \"custom-route-timeout-pd-test-kserve-7cbf8d89d-8j8f9\" (UID: \"8ad639c0-0908-443a-9a36-949955a89ee7\") " pod="kserve-ci-e2e-test/custom-route-timeout-pd-test-kserve-7cbf8d89d-8j8f9"
Apr 24 14:54:36.176413 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:54:36.176401    2574 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/73d63045-437e-49a4-be0c-cfbd579d93f5-home\") pod \"custom-route-timeout-pd-test-kserve-prefill-578d77b467-zfd6z\" (UID: \"73d63045-437e-49a4-be0c-cfbd579d93f5\") " pod="kserve-ci-e2e-test/custom-route-timeout-pd-test-kserve-prefill-578d77b467-zfd6z"
Apr 24 14:54:36.176631 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:54:36.176513    2574 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-mw4fg\" (UniqueName: \"kubernetes.io/projected/8ad639c0-0908-443a-9a36-949955a89ee7-kube-api-access-mw4fg\") pod \"custom-route-timeout-pd-test-kserve-7cbf8d89d-8j8f9\" (UID: \"8ad639c0-0908-443a-9a36-949955a89ee7\") " pod="kserve-ci-e2e-test/custom-route-timeout-pd-test-kserve-7cbf8d89d-8j8f9"
Apr 24 14:54:36.176712 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:54:36.176692    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/8ad639c0-0908-443a-9a36-949955a89ee7-model-cache\") pod \"custom-route-timeout-pd-test-kserve-7cbf8d89d-8j8f9\" (UID: \"8ad639c0-0908-443a-9a36-949955a89ee7\") " pod="kserve-ci-e2e-test/custom-route-timeout-pd-test-kserve-7cbf8d89d-8j8f9"
Apr 24 14:54:36.176799 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:54:36.176748    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/73d63045-437e-49a4-be0c-cfbd579d93f5-home\") pod \"custom-route-timeout-pd-test-kserve-prefill-578d77b467-zfd6z\" (UID: \"73d63045-437e-49a4-be0c-cfbd579d93f5\") " pod="kserve-ci-e2e-test/custom-route-timeout-pd-test-kserve-prefill-578d77b467-zfd6z"
Apr 24 14:54:36.176851 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:54:36.176792    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/8ad639c0-0908-443a-9a36-949955a89ee7-kserve-provision-location\") pod \"custom-route-timeout-pd-test-kserve-7cbf8d89d-8j8f9\" (UID: \"8ad639c0-0908-443a-9a36-949955a89ee7\") " pod="kserve-ci-e2e-test/custom-route-timeout-pd-test-kserve-7cbf8d89d-8j8f9"
Apr 24 14:54:36.176908 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:54:36.176864    2574 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-9h88x\" (UniqueName: \"kubernetes.io/projected/73d63045-437e-49a4-be0c-cfbd579d93f5-kube-api-access-9h88x\") pod \"custom-route-timeout-pd-test-kserve-prefill-578d77b467-zfd6z\" (UID: \"73d63045-437e-49a4-be0c-cfbd579d93f5\") " pod="kserve-ci-e2e-test/custom-route-timeout-pd-test-kserve-prefill-578d77b467-zfd6z"
Apr 24 14:54:36.176908 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:54:36.176896    2574 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/73d63045-437e-49a4-be0c-cfbd579d93f5-model-cache\") pod \"custom-route-timeout-pd-test-kserve-prefill-578d77b467-zfd6z\" (UID: \"73d63045-437e-49a4-be0c-cfbd579d93f5\") " pod="kserve-ci-e2e-test/custom-route-timeout-pd-test-kserve-prefill-578d77b467-zfd6z"
Apr 24 14:54:36.177023 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:54:36.176924    2574 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/73d63045-437e-49a4-be0c-cfbd579d93f5-tls-certs\") pod \"custom-route-timeout-pd-test-kserve-prefill-578d77b467-zfd6z\" (UID: \"73d63045-437e-49a4-be0c-cfbd579d93f5\") " pod="kserve-ci-e2e-test/custom-route-timeout-pd-test-kserve-prefill-578d77b467-zfd6z"
Apr 24 14:54:36.177023 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:54:36.176961    2574 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/8ad639c0-0908-443a-9a36-949955a89ee7-home\") pod \"custom-route-timeout-pd-test-kserve-7cbf8d89d-8j8f9\" (UID: \"8ad639c0-0908-443a-9a36-949955a89ee7\") " pod="kserve-ci-e2e-test/custom-route-timeout-pd-test-kserve-7cbf8d89d-8j8f9"
Apr 24 14:54:36.177023 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:54:36.176986    2574 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/73d63045-437e-49a4-be0c-cfbd579d93f5-kserve-provision-location\") pod \"custom-route-timeout-pd-test-kserve-prefill-578d77b467-zfd6z\" (UID: \"73d63045-437e-49a4-be0c-cfbd579d93f5\") " pod="kserve-ci-e2e-test/custom-route-timeout-pd-test-kserve-prefill-578d77b467-zfd6z"
Apr 24 14:54:36.177390 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:54:36.177366    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/8ad639c0-0908-443a-9a36-949955a89ee7-home\") pod \"custom-route-timeout-pd-test-kserve-7cbf8d89d-8j8f9\" (UID: \"8ad639c0-0908-443a-9a36-949955a89ee7\") " pod="kserve-ci-e2e-test/custom-route-timeout-pd-test-kserve-7cbf8d89d-8j8f9"
Apr 24 14:54:36.177490 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:54:36.177387    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/73d63045-437e-49a4-be0c-cfbd579d93f5-kserve-provision-location\") pod \"custom-route-timeout-pd-test-kserve-prefill-578d77b467-zfd6z\" (UID: \"73d63045-437e-49a4-be0c-cfbd579d93f5\") " pod="kserve-ci-e2e-test/custom-route-timeout-pd-test-kserve-prefill-578d77b467-zfd6z"
Apr 24 14:54:36.177490 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:54:36.177368    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/73d63045-437e-49a4-be0c-cfbd579d93f5-model-cache\") pod \"custom-route-timeout-pd-test-kserve-prefill-578d77b467-zfd6z\" (UID: \"73d63045-437e-49a4-be0c-cfbd579d93f5\") " pod="kserve-ci-e2e-test/custom-route-timeout-pd-test-kserve-prefill-578d77b467-zfd6z"
Apr 24 14:54:36.178588 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:54:36.178568    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/73d63045-437e-49a4-be0c-cfbd579d93f5-dshm\") pod \"custom-route-timeout-pd-test-kserve-prefill-578d77b467-zfd6z\" (UID: \"73d63045-437e-49a4-be0c-cfbd579d93f5\") " pod="kserve-ci-e2e-test/custom-route-timeout-pd-test-kserve-prefill-578d77b467-zfd6z"
Apr 24 14:54:36.178666 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:54:36.178602    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/8ad639c0-0908-443a-9a36-949955a89ee7-dshm\") pod \"custom-route-timeout-pd-test-kserve-7cbf8d89d-8j8f9\" (UID: \"8ad639c0-0908-443a-9a36-949955a89ee7\") " pod="kserve-ci-e2e-test/custom-route-timeout-pd-test-kserve-7cbf8d89d-8j8f9"
Apr 24 14:54:36.178793 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:54:36.178776    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/8ad639c0-0908-443a-9a36-949955a89ee7-tls-certs\") pod \"custom-route-timeout-pd-test-kserve-7cbf8d89d-8j8f9\" (UID: \"8ad639c0-0908-443a-9a36-949955a89ee7\") " pod="kserve-ci-e2e-test/custom-route-timeout-pd-test-kserve-7cbf8d89d-8j8f9"
Apr 24 14:54:36.179421 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:54:36.179406    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/73d63045-437e-49a4-be0c-cfbd579d93f5-tls-certs\") pod \"custom-route-timeout-pd-test-kserve-prefill-578d77b467-zfd6z\" (UID: \"73d63045-437e-49a4-be0c-cfbd579d93f5\") " pod="kserve-ci-e2e-test/custom-route-timeout-pd-test-kserve-prefill-578d77b467-zfd6z"
Apr 24 14:54:36.184266 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:54:36.184240    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-mw4fg\" (UniqueName: \"kubernetes.io/projected/8ad639c0-0908-443a-9a36-949955a89ee7-kube-api-access-mw4fg\") pod \"custom-route-timeout-pd-test-kserve-7cbf8d89d-8j8f9\" (UID: \"8ad639c0-0908-443a-9a36-949955a89ee7\") " pod="kserve-ci-e2e-test/custom-route-timeout-pd-test-kserve-7cbf8d89d-8j8f9"
Apr 24 14:54:36.184360 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:54:36.184314    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-9h88x\" (UniqueName: \"kubernetes.io/projected/73d63045-437e-49a4-be0c-cfbd579d93f5-kube-api-access-9h88x\") pod \"custom-route-timeout-pd-test-kserve-prefill-578d77b467-zfd6z\" (UID: \"73d63045-437e-49a4-be0c-cfbd579d93f5\") " pod="kserve-ci-e2e-test/custom-route-timeout-pd-test-kserve-prefill-578d77b467-zfd6z"
Apr 24 14:54:36.201331 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:54:36.201312    2574 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/custom-route-timeout-pd-test-kserve-7cbf8d89d-8j8f9"
Apr 24 14:54:36.214060 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:54:36.214040    2574 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/custom-route-timeout-pd-test-kserve-prefill-578d77b467-zfd6z"
Apr 24 14:54:36.357911 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:54:36.357853    2574 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/custom-route-timeout-pd-test-kserve-7cbf8d89d-8j8f9"]
Apr 24 14:54:36.376411 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:54:36.376388    2574 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/custom-route-timeout-pd-test-kserve-prefill-578d77b467-zfd6z"]
Apr 24 14:54:36.376674 ip-10-0-136-132 kubenswrapper[2574]: W0424 14:54:36.376653    2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod73d63045_437e_49a4_be0c_cfbd579d93f5.slice/crio-6c067c9ceeb9cb6d3a0d5c977b41da4c093798ad05d41637c58ebf93c9d14135 WatchSource:0}: Error finding container 6c067c9ceeb9cb6d3a0d5c977b41da4c093798ad05d41637c58ebf93c9d14135: Status 404 returned error can't find the container with id 6c067c9ceeb9cb6d3a0d5c977b41da4c093798ad05d41637c58ebf93c9d14135
Apr 24 14:54:37.009388 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:54:37.009329    2574 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/custom-route-timeout-pd-test-kserve-7cbf8d89d-8j8f9" event={"ID":"8ad639c0-0908-443a-9a36-949955a89ee7","Type":"ContainerStarted","Data":"c689a0c777681dc25ace852028277ff620f19f5f6c4de3c1a3ea78a7fa2946cb"}
Apr 24 14:54:37.011086 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:54:37.011045    2574 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/custom-route-timeout-pd-test-kserve-prefill-578d77b467-zfd6z" event={"ID":"73d63045-437e-49a4-be0c-cfbd579d93f5","Type":"ContainerStarted","Data":"7dfbc3b0597d73f71629a6f375de4212a6a0a9e88145bd35b19461457699bfe5"}
Apr 24 14:54:37.011209 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:54:37.011089    2574 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/custom-route-timeout-pd-test-kserve-prefill-578d77b467-zfd6z" event={"ID":"73d63045-437e-49a4-be0c-cfbd579d93f5","Type":"ContainerStarted","Data":"6c067c9ceeb9cb6d3a0d5c977b41da4c093798ad05d41637c58ebf93c9d14135"}
Apr 24 14:54:38.017293 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:54:38.017258    2574 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/custom-route-timeout-pd-test-kserve-7cbf8d89d-8j8f9" event={"ID":"8ad639c0-0908-443a-9a36-949955a89ee7","Type":"ContainerStarted","Data":"c8966114c184684e9d75d4912eaf66b3ac39a2360529bed84f02affbd1cf68b3"}
Apr 24 14:54:39.025545 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:54:39.025504    2574 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/custom-route-timeout-pd-test-kserve-7cbf8d89d-8j8f9" event={"ID":"8ad639c0-0908-443a-9a36-949955a89ee7","Type":"ContainerStarted","Data":"ecea0f0dc3ac760a288d2737d765947fc28ad70473419edf52e3e831b0d2c1c7"}
Apr 24 14:54:39.025942 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:54:39.025690    2574 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve-ci-e2e-test/custom-route-timeout-pd-test-kserve-7cbf8d89d-8j8f9"
Apr 24 14:54:43.042557 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:54:43.042521    2574 generic.go:358] "Generic (PLEG): container finished" podID="8ad639c0-0908-443a-9a36-949955a89ee7" containerID="ecea0f0dc3ac760a288d2737d765947fc28ad70473419edf52e3e831b0d2c1c7" exitCode=0
Apr 24 14:54:43.043007 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:54:43.042596    2574 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/custom-route-timeout-pd-test-kserve-7cbf8d89d-8j8f9" event={"ID":"8ad639c0-0908-443a-9a36-949955a89ee7","Type":"ContainerDied","Data":"ecea0f0dc3ac760a288d2737d765947fc28ad70473419edf52e3e831b0d2c1c7"}
Apr 24 14:54:44.049561 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:54:44.049492    2574 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/custom-route-timeout-pd-test-kserve-7cbf8d89d-8j8f9" event={"ID":"8ad639c0-0908-443a-9a36-949955a89ee7","Type":"ContainerStarted","Data":"724ba3b78c6fbbed91ea92e813bfbf0334160fefccecb667f544c41d3297e574"}
Apr 24 14:54:44.070377 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:54:44.070318    2574 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="kserve-ci-e2e-test/custom-route-timeout-pd-test-kserve-7cbf8d89d-8j8f9" podStartSLOduration=8.25843748 podStartE2EDuration="9.070299594s" podCreationTimestamp="2026-04-24 14:54:35 +0000 UTC" firstStartedPulling="2026-04-24 14:54:36.360074019 +0000 UTC m=+1839.092175647" lastFinishedPulling="2026-04-24 14:54:37.171936125 +0000 UTC m=+1839.904037761" observedRunningTime="2026-04-24 14:54:44.067873981 +0000 UTC m=+1846.799975640" watchObservedRunningTime="2026-04-24 14:54:44.070299594 +0000 UTC m=+1846.802401243"
Apr 24 14:54:46.202252 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:54:46.202206    2574 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve-ci-e2e-test/custom-route-timeout-pd-test-kserve-7cbf8d89d-8j8f9"
Apr 24 14:54:46.202252 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:54:46.202258    2574 kubelet.go:2658] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="kserve-ci-e2e-test/custom-route-timeout-pd-test-kserve-7cbf8d89d-8j8f9"
Apr 24 14:54:46.203744 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:54:46.203716    2574 prober.go:120] "Probe failed" probeType="Startup" pod="kserve-ci-e2e-test/custom-route-timeout-pd-test-kserve-7cbf8d89d-8j8f9" podUID="8ad639c0-0908-443a-9a36-949955a89ee7" containerName="main" probeResult="failure" output="Get \"https://10.133.0.62:8001/health\": dial tcp 10.133.0.62:8001: connect: connection refused"
Apr 24 14:54:56.201898 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:54:56.201853    2574 prober.go:120] "Probe failed" probeType="Startup" pod="kserve-ci-e2e-test/custom-route-timeout-pd-test-kserve-7cbf8d89d-8j8f9" podUID="8ad639c0-0908-443a-9a36-949955a89ee7" containerName="main" probeResult="failure" output="Get \"https://10.133.0.62:8001/health\": dial tcp 10.133.0.62:8001: connect: connection refused"
Apr 24 14:54:56.215930 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:54:56.215901    2574 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="kserve-ci-e2e-test/custom-route-timeout-pd-test-kserve-7cbf8d89d-8j8f9"
Apr 24 14:55:06.201772 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:55:06.201731    2574 prober.go:120] "Probe failed" probeType="Startup" pod="kserve-ci-e2e-test/custom-route-timeout-pd-test-kserve-7cbf8d89d-8j8f9" podUID="8ad639c0-0908-443a-9a36-949955a89ee7" containerName="main" probeResult="failure" output="Get \"https://10.133.0.62:8001/health\": dial tcp 10.133.0.62:8001: connect: connection refused"
Apr 24 14:55:16.202838 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:55:16.202738    2574 prober.go:120] "Probe failed" probeType="Startup" pod="kserve-ci-e2e-test/custom-route-timeout-pd-test-kserve-7cbf8d89d-8j8f9" podUID="8ad639c0-0908-443a-9a36-949955a89ee7" containerName="main" probeResult="failure" output="Get \"https://10.133.0.62:8001/health\": dial tcp 10.133.0.62:8001: connect: connection refused"
Apr 24 14:55:26.201899 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:55:26.201845    2574 prober.go:120] "Probe failed" probeType="Startup" pod="kserve-ci-e2e-test/custom-route-timeout-pd-test-kserve-7cbf8d89d-8j8f9" podUID="8ad639c0-0908-443a-9a36-949955a89ee7" containerName="main" probeResult="failure" output="Get \"https://10.133.0.62:8001/health\": dial tcp 10.133.0.62:8001: connect: connection refused"
Apr 24 14:55:36.202600 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:55:36.202543    2574 prober.go:120] "Probe failed" probeType="Startup" pod="kserve-ci-e2e-test/custom-route-timeout-pd-test-kserve-7cbf8d89d-8j8f9" podUID="8ad639c0-0908-443a-9a36-949955a89ee7" containerName="main" probeResult="failure" output="Get \"https://10.133.0.62:8001/health\": dial tcp 10.133.0.62:8001: connect: connection refused"
Apr 24 14:55:46.201765 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:55:46.201721    2574 prober.go:120] "Probe failed" probeType="Startup" pod="kserve-ci-e2e-test/custom-route-timeout-pd-test-kserve-7cbf8d89d-8j8f9" podUID="8ad639c0-0908-443a-9a36-949955a89ee7" containerName="main" probeResult="failure" output="Get \"https://10.133.0.62:8001/health\": dial tcp 10.133.0.62:8001: connect: connection refused"
Apr 24 14:55:56.201726 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:55:56.201676    2574 prober.go:120] "Probe failed" probeType="Startup" pod="kserve-ci-e2e-test/custom-route-timeout-pd-test-kserve-7cbf8d89d-8j8f9" podUID="8ad639c0-0908-443a-9a36-949955a89ee7" containerName="main" probeResult="failure" output="Get \"https://10.133.0.62:8001/health\": dial tcp 10.133.0.62:8001: connect: connection refused"
Apr 24 14:56:06.201808 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:56:06.201763    2574 prober.go:120] "Probe failed" probeType="Startup" pod="kserve-ci-e2e-test/custom-route-timeout-pd-test-kserve-7cbf8d89d-8j8f9" podUID="8ad639c0-0908-443a-9a36-949955a89ee7" containerName="main" probeResult="failure" output="Get \"https://10.133.0.62:8001/health\": dial tcp 10.133.0.62:8001: connect: connection refused"
Apr 24 14:56:16.211511 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:56:16.211477    2574 kubelet.go:2658] "SyncLoop (probe)" probe="startup" status="started" pod="kserve-ci-e2e-test/custom-route-timeout-pd-test-kserve-7cbf8d89d-8j8f9"
Apr 24 14:56:16.224422 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:56:16.224398    2574 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="kserve-ci-e2e-test/custom-route-timeout-pd-test-kserve-7cbf8d89d-8j8f9"
Apr 24 14:58:18.914773 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:58:18.914742    2574 generic.go:358] "Generic (PLEG): container finished" podID="73d63045-437e-49a4-be0c-cfbd579d93f5" containerID="7dfbc3b0597d73f71629a6f375de4212a6a0a9e88145bd35b19461457699bfe5" exitCode=0
Apr 24 14:58:18.915151 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:58:18.914799    2574 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/custom-route-timeout-pd-test-kserve-prefill-578d77b467-zfd6z" event={"ID":"73d63045-437e-49a4-be0c-cfbd579d93f5","Type":"ContainerDied","Data":"7dfbc3b0597d73f71629a6f375de4212a6a0a9e88145bd35b19461457699bfe5"}
Apr 24 14:58:18.915853 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:58:18.915837    2574 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Apr 24 14:58:19.920901 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:58:19.920866    2574 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/custom-route-timeout-pd-test-kserve-prefill-578d77b467-zfd6z" event={"ID":"73d63045-437e-49a4-be0c-cfbd579d93f5","Type":"ContainerStarted","Data":"d5396dd69e4f9924570281c5031d973ff554aa478fd76c82d2b9d281de73f8fd"}
Apr 24 14:58:19.940989 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:58:19.940931    2574 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="kserve-ci-e2e-test/custom-route-timeout-pd-test-kserve-prefill-578d77b467-zfd6z" podStartSLOduration=224.940897077 podStartE2EDuration="3m44.940897077s" podCreationTimestamp="2026-04-24 14:54:35 +0000 UTC" firstStartedPulling="0001-01-01 00:00:00 +0000 UTC" lastFinishedPulling="0001-01-01 00:00:00 +0000 UTC" observedRunningTime="2026-04-24 14:58:19.938886849 +0000 UTC m=+2062.670988508" watchObservedRunningTime="2026-04-24 14:58:19.940897077 +0000 UTC m=+2062.672998726"
Apr 24 14:58:26.214225 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:58:26.214186    2574 kubelet.go:2658] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="kserve-ci-e2e-test/custom-route-timeout-pd-test-kserve-prefill-578d77b467-zfd6z"
Apr 24 14:58:26.214225 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:58:26.214231    2574 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve-ci-e2e-test/custom-route-timeout-pd-test-kserve-prefill-578d77b467-zfd6z"
Apr 24 14:58:26.215797 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:58:26.215763    2574 prober.go:120] "Probe failed" probeType="Startup" pod="kserve-ci-e2e-test/custom-route-timeout-pd-test-kserve-prefill-578d77b467-zfd6z" podUID="73d63045-437e-49a4-be0c-cfbd579d93f5" containerName="main" probeResult="failure" output="Get \"https://10.133.0.63:8000/health\": dial tcp 10.133.0.63:8000: connect: connection refused"
Apr 24 14:58:36.214357 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:58:36.214302    2574 prober.go:120] "Probe failed" probeType="Startup" pod="kserve-ci-e2e-test/custom-route-timeout-pd-test-kserve-prefill-578d77b467-zfd6z" podUID="73d63045-437e-49a4-be0c-cfbd579d93f5" containerName="main" probeResult="failure" output="Get \"https://10.133.0.63:8000/health\": dial tcp 10.133.0.63:8000: connect: connection refused"
Apr 24 14:58:46.214757 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:58:46.214717    2574 prober.go:120] "Probe failed" probeType="Startup" pod="kserve-ci-e2e-test/custom-route-timeout-pd-test-kserve-prefill-578d77b467-zfd6z" podUID="73d63045-437e-49a4-be0c-cfbd579d93f5" containerName="main" probeResult="failure" output="Get \"https://10.133.0.63:8000/health\": dial tcp 10.133.0.63:8000: connect: connection refused"
Apr 24 14:58:56.214622 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:58:56.214579    2574 prober.go:120] "Probe failed" probeType="Startup" pod="kserve-ci-e2e-test/custom-route-timeout-pd-test-kserve-prefill-578d77b467-zfd6z" podUID="73d63045-437e-49a4-be0c-cfbd579d93f5" containerName="main" probeResult="failure" output="Get \"https://10.133.0.63:8000/health\": dial tcp 10.133.0.63:8000: connect: connection refused"
Apr 24 14:58:57.920833 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:58:57.920805    2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-9d4b6777b-f6ngr_2571829f-666a-422a-9b02-2d2bb4c89b25/console-operator/1.log"
Apr 24 14:58:57.925696 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:58:57.925674    2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-w5x5x_44036c3e-1fdc-40d4-899c-9009cd4589e3/ovn-acl-logging/0.log"
Apr 24 14:58:57.926365 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:58:57.926326    2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-9d4b6777b-f6ngr_2571829f-666a-422a-9b02-2d2bb4c89b25/console-operator/1.log"
Apr 24 14:58:57.931275 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:58:57.931259    2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-w5x5x_44036c3e-1fdc-40d4-899c-9009cd4589e3/ovn-acl-logging/0.log"
Apr 24 14:59:06.215143 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:59:06.215087    2574 prober.go:120] "Probe failed" probeType="Startup" pod="kserve-ci-e2e-test/custom-route-timeout-pd-test-kserve-prefill-578d77b467-zfd6z" podUID="73d63045-437e-49a4-be0c-cfbd579d93f5" containerName="main" probeResult="failure" output="Get \"https://10.133.0.63:8000/health\": dial tcp 10.133.0.63:8000: connect: connection refused"
Apr 24 14:59:16.214530 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:59:16.214477    2574 prober.go:120] "Probe failed" probeType="Startup" pod="kserve-ci-e2e-test/custom-route-timeout-pd-test-kserve-prefill-578d77b467-zfd6z" podUID="73d63045-437e-49a4-be0c-cfbd579d93f5" containerName="main" probeResult="failure" output="Get \"https://10.133.0.63:8000/health\": dial tcp 10.133.0.63:8000: connect: connection refused"
Apr 24 14:59:26.215214 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:59:26.215171    2574 prober.go:120] "Probe failed" probeType="Startup" pod="kserve-ci-e2e-test/custom-route-timeout-pd-test-kserve-prefill-578d77b467-zfd6z" podUID="73d63045-437e-49a4-be0c-cfbd579d93f5" containerName="main" probeResult="failure" output="Get \"https://10.133.0.63:8000/health\": dial tcp 10.133.0.63:8000: connect: connection refused"
Apr 24 14:59:36.215168 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:59:36.215109    2574 prober.go:120] "Probe failed" probeType="Startup" pod="kserve-ci-e2e-test/custom-route-timeout-pd-test-kserve-prefill-578d77b467-zfd6z" podUID="73d63045-437e-49a4-be0c-cfbd579d93f5" containerName="main" probeResult="failure" output="Get \"https://10.133.0.63:8000/health\": dial tcp 10.133.0.63:8000: connect: connection refused"
Apr 24 14:59:46.214662 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:59:46.214580    2574 prober.go:120] "Probe failed" probeType="Startup" pod="kserve-ci-e2e-test/custom-route-timeout-pd-test-kserve-prefill-578d77b467-zfd6z" podUID="73d63045-437e-49a4-be0c-cfbd579d93f5" containerName="main" probeResult="failure" output="Get \"https://10.133.0.63:8000/health\": dial tcp 10.133.0.63:8000: connect: connection refused"
Apr 24 14:59:56.224327 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:59:56.224294    2574 kubelet.go:2658] "SyncLoop (probe)" probe="startup" status="started" pod="kserve-ci-e2e-test/custom-route-timeout-pd-test-kserve-prefill-578d77b467-zfd6z"
Apr 24 14:59:56.231897 ip-10-0-136-132 kubenswrapper[2574]: I0424 14:59:56.231877    2574 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="kserve-ci-e2e-test/custom-route-timeout-pd-test-kserve-prefill-578d77b467-zfd6z"
Apr 24 15:00:07.705492 ip-10-0-136-132 kubenswrapper[2574]: I0424 15:00:07.705456    2574 kubelet.go:2553] "SyncLoop DELETE" source="api" pods=["kserve-ci-e2e-test/custom-route-timeout-pd-test-kserve-prefill-578d77b467-zfd6z"]
Apr 24 15:00:07.706598 ip-10-0-136-132 kubenswrapper[2574]: I0424 15:00:07.706545    2574 kuberuntime_container.go:864] "Killing container with a grace period" pod="kserve-ci-e2e-test/custom-route-timeout-pd-test-kserve-prefill-578d77b467-zfd6z" podUID="73d63045-437e-49a4-be0c-cfbd579d93f5" containerName="main" containerID="cri-o://d5396dd69e4f9924570281c5031d973ff554aa478fd76c82d2b9d281de73f8fd" gracePeriod=30
Apr 24 15:00:07.716375 ip-10-0-136-132 kubenswrapper[2574]: I0424 15:00:07.711899    2574 kubelet.go:2553] "SyncLoop DELETE" source="api" pods=["kserve-ci-e2e-test/custom-route-timeout-pd-test-kserve-7cbf8d89d-8j8f9"]
Apr 24 15:00:07.717219 ip-10-0-136-132 kubenswrapper[2574]: I0424 15:00:07.717158    2574 kuberuntime_container.go:864] "Killing container with a grace period" pod="kserve-ci-e2e-test/custom-route-timeout-pd-test-kserve-7cbf8d89d-8j8f9" podUID="8ad639c0-0908-443a-9a36-949955a89ee7" containerName="main" containerID="cri-o://724ba3b78c6fbbed91ea92e813bfbf0334160fefccecb667f544c41d3297e574" gracePeriod=30
Apr 24 15:00:24.490399 ip-10-0-136-132 kubenswrapper[2574]: I0424 15:00:24.490315    2574 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["kserve-ci-e2e-test/router-with-refs-pd-test-kserve-prefill-5c8777dc-txbgd"]
Apr 24 15:00:24.494442 ip-10-0-136-132 kubenswrapper[2574]: I0424 15:00:24.494418    2574 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/router-with-refs-pd-test-kserve-prefill-5c8777dc-txbgd"
Apr 24 15:00:24.496547 ip-10-0-136-132 kubenswrapper[2574]: I0424 15:00:24.496522    2574 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"kserve-ci-e2e-test\"/\"router-with-refs-pd-test-kserve-self-signed-certs\""
Apr 24 15:00:24.510566 ip-10-0-136-132 kubenswrapper[2574]: I0424 15:00:24.510542    2574 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/router-with-refs-pd-test-kserve-prefill-5c8777dc-txbgd"]
Apr 24 15:00:24.549167 ip-10-0-136-132 kubenswrapper[2574]: I0424 15:00:24.549142    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/4a48929f-f66c-49b8-9d3d-08c90eb4c1c6-model-cache\") pod \"router-with-refs-pd-test-kserve-prefill-5c8777dc-txbgd\" (UID: \"4a48929f-f66c-49b8-9d3d-08c90eb4c1c6\") " pod="kserve-ci-e2e-test/router-with-refs-pd-test-kserve-prefill-5c8777dc-txbgd"
Apr 24 15:00:24.549301 ip-10-0-136-132 kubenswrapper[2574]: I0424 15:00:24.549185    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/4a48929f-f66c-49b8-9d3d-08c90eb4c1c6-kserve-provision-location\") pod \"router-with-refs-pd-test-kserve-prefill-5c8777dc-txbgd\" (UID: \"4a48929f-f66c-49b8-9d3d-08c90eb4c1c6\") " pod="kserve-ci-e2e-test/router-with-refs-pd-test-kserve-prefill-5c8777dc-txbgd"
Apr 24 15:00:24.549301 ip-10-0-136-132 kubenswrapper[2574]: I0424 15:00:24.549267    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/4a48929f-f66c-49b8-9d3d-08c90eb4c1c6-dshm\") pod \"router-with-refs-pd-test-kserve-prefill-5c8777dc-txbgd\" (UID: \"4a48929f-f66c-49b8-9d3d-08c90eb4c1c6\") " pod="kserve-ci-e2e-test/router-with-refs-pd-test-kserve-prefill-5c8777dc-txbgd"
Apr 24 15:00:24.549456 ip-10-0-136-132 kubenswrapper[2574]: I0424 15:00:24.549323    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/4a48929f-f66c-49b8-9d3d-08c90eb4c1c6-home\") pod \"router-with-refs-pd-test-kserve-prefill-5c8777dc-txbgd\" (UID: \"4a48929f-f66c-49b8-9d3d-08c90eb4c1c6\") " pod="kserve-ci-e2e-test/router-with-refs-pd-test-kserve-prefill-5c8777dc-txbgd"
Apr 24 15:00:24.549456 ip-10-0-136-132 kubenswrapper[2574]: I0424 15:00:24.549420    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-n8xmr\" (UniqueName: \"kubernetes.io/projected/4a48929f-f66c-49b8-9d3d-08c90eb4c1c6-kube-api-access-n8xmr\") pod \"router-with-refs-pd-test-kserve-prefill-5c8777dc-txbgd\" (UID: \"4a48929f-f66c-49b8-9d3d-08c90eb4c1c6\") " pod="kserve-ci-e2e-test/router-with-refs-pd-test-kserve-prefill-5c8777dc-txbgd"
Apr 24 15:00:24.549559 ip-10-0-136-132 kubenswrapper[2574]: I0424 15:00:24.549471    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/4a48929f-f66c-49b8-9d3d-08c90eb4c1c6-tls-certs\") pod \"router-with-refs-pd-test-kserve-prefill-5c8777dc-txbgd\" (UID: \"4a48929f-f66c-49b8-9d3d-08c90eb4c1c6\") " pod="kserve-ci-e2e-test/router-with-refs-pd-test-kserve-prefill-5c8777dc-txbgd"
Apr 24 15:00:24.650523 ip-10-0-136-132 kubenswrapper[2574]: I0424 15:00:24.650490    2574 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/4a48929f-f66c-49b8-9d3d-08c90eb4c1c6-kserve-provision-location\") pod \"router-with-refs-pd-test-kserve-prefill-5c8777dc-txbgd\" (UID: \"4a48929f-f66c-49b8-9d3d-08c90eb4c1c6\") " pod="kserve-ci-e2e-test/router-with-refs-pd-test-kserve-prefill-5c8777dc-txbgd"
Apr 24 15:00:24.650689 ip-10-0-136-132 kubenswrapper[2574]: I0424 15:00:24.650540    2574 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/4a48929f-f66c-49b8-9d3d-08c90eb4c1c6-dshm\") pod \"router-with-refs-pd-test-kserve-prefill-5c8777dc-txbgd\" (UID: \"4a48929f-f66c-49b8-9d3d-08c90eb4c1c6\") " pod="kserve-ci-e2e-test/router-with-refs-pd-test-kserve-prefill-5c8777dc-txbgd"
Apr 24 15:00:24.650689 ip-10-0-136-132 kubenswrapper[2574]: I0424 15:00:24.650569    2574 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/4a48929f-f66c-49b8-9d3d-08c90eb4c1c6-home\") pod \"router-with-refs-pd-test-kserve-prefill-5c8777dc-txbgd\" (UID: \"4a48929f-f66c-49b8-9d3d-08c90eb4c1c6\") " pod="kserve-ci-e2e-test/router-with-refs-pd-test-kserve-prefill-5c8777dc-txbgd"
Apr 24 15:00:24.650689 ip-10-0-136-132 kubenswrapper[2574]: I0424 15:00:24.650598    2574 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-n8xmr\" (UniqueName: \"kubernetes.io/projected/4a48929f-f66c-49b8-9d3d-08c90eb4c1c6-kube-api-access-n8xmr\") pod \"router-with-refs-pd-test-kserve-prefill-5c8777dc-txbgd\" (UID: \"4a48929f-f66c-49b8-9d3d-08c90eb4c1c6\") " pod="kserve-ci-e2e-test/router-with-refs-pd-test-kserve-prefill-5c8777dc-txbgd"
Apr 24 15:00:24.650689 ip-10-0-136-132 kubenswrapper[2574]: I0424 15:00:24.650623    2574 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/4a48929f-f66c-49b8-9d3d-08c90eb4c1c6-tls-certs\") pod \"router-with-refs-pd-test-kserve-prefill-5c8777dc-txbgd\" (UID: \"4a48929f-f66c-49b8-9d3d-08c90eb4c1c6\") " pod="kserve-ci-e2e-test/router-with-refs-pd-test-kserve-prefill-5c8777dc-txbgd"
Apr 24 15:00:24.650689 ip-10-0-136-132 kubenswrapper[2574]: I0424 15:00:24.650669    2574 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/4a48929f-f66c-49b8-9d3d-08c90eb4c1c6-model-cache\") pod \"router-with-refs-pd-test-kserve-prefill-5c8777dc-txbgd\" (UID: \"4a48929f-f66c-49b8-9d3d-08c90eb4c1c6\") " pod="kserve-ci-e2e-test/router-with-refs-pd-test-kserve-prefill-5c8777dc-txbgd"
Apr 24 15:00:24.650970 ip-10-0-136-132 kubenswrapper[2574]: I0424 15:00:24.650948    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/4a48929f-f66c-49b8-9d3d-08c90eb4c1c6-kserve-provision-location\") pod \"router-with-refs-pd-test-kserve-prefill-5c8777dc-txbgd\" (UID: \"4a48929f-f66c-49b8-9d3d-08c90eb4c1c6\") " pod="kserve-ci-e2e-test/router-with-refs-pd-test-kserve-prefill-5c8777dc-txbgd"
Apr 24 15:00:24.651038 ip-10-0-136-132 kubenswrapper[2574]: I0424 15:00:24.650995    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/4a48929f-f66c-49b8-9d3d-08c90eb4c1c6-home\") pod \"router-with-refs-pd-test-kserve-prefill-5c8777dc-txbgd\" (UID: \"4a48929f-f66c-49b8-9d3d-08c90eb4c1c6\") " pod="kserve-ci-e2e-test/router-with-refs-pd-test-kserve-prefill-5c8777dc-txbgd"
Apr 24 15:00:24.651105 ip-10-0-136-132 kubenswrapper[2574]: I0424 15:00:24.651088    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/4a48929f-f66c-49b8-9d3d-08c90eb4c1c6-model-cache\") pod \"router-with-refs-pd-test-kserve-prefill-5c8777dc-txbgd\" (UID: \"4a48929f-f66c-49b8-9d3d-08c90eb4c1c6\") " pod="kserve-ci-e2e-test/router-with-refs-pd-test-kserve-prefill-5c8777dc-txbgd"
Apr 24 15:00:24.652952 ip-10-0-136-132 kubenswrapper[2574]: I0424 15:00:24.652931    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/4a48929f-f66c-49b8-9d3d-08c90eb4c1c6-dshm\") pod \"router-with-refs-pd-test-kserve-prefill-5c8777dc-txbgd\" (UID: \"4a48929f-f66c-49b8-9d3d-08c90eb4c1c6\") " pod="kserve-ci-e2e-test/router-with-refs-pd-test-kserve-prefill-5c8777dc-txbgd"
Apr 24 15:00:24.653040 ip-10-0-136-132 kubenswrapper[2574]: I0424 15:00:24.653023    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/4a48929f-f66c-49b8-9d3d-08c90eb4c1c6-tls-certs\") pod \"router-with-refs-pd-test-kserve-prefill-5c8777dc-txbgd\" (UID: \"4a48929f-f66c-49b8-9d3d-08c90eb4c1c6\") " pod="kserve-ci-e2e-test/router-with-refs-pd-test-kserve-prefill-5c8777dc-txbgd"
Apr 24 15:00:24.658378 ip-10-0-136-132 kubenswrapper[2574]: I0424 15:00:24.658358    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-n8xmr\" (UniqueName: \"kubernetes.io/projected/4a48929f-f66c-49b8-9d3d-08c90eb4c1c6-kube-api-access-n8xmr\") pod \"router-with-refs-pd-test-kserve-prefill-5c8777dc-txbgd\" (UID: \"4a48929f-f66c-49b8-9d3d-08c90eb4c1c6\") " pod="kserve-ci-e2e-test/router-with-refs-pd-test-kserve-prefill-5c8777dc-txbgd"
Apr 24 15:00:24.805686 ip-10-0-136-132 kubenswrapper[2574]: I0424 15:00:24.805602    2574 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/router-with-refs-pd-test-kserve-prefill-5c8777dc-txbgd"
Apr 24 15:00:24.933642 ip-10-0-136-132 kubenswrapper[2574]: I0424 15:00:24.933618    2574 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/router-with-refs-pd-test-kserve-prefill-5c8777dc-txbgd"]
Apr 24 15:00:24.935544 ip-10-0-136-132 kubenswrapper[2574]: W0424 15:00:24.935515    2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4a48929f_f66c_49b8_9d3d_08c90eb4c1c6.slice/crio-170fca5a493884c06dc89ada3d355086add8d5927127df6a4dce2eaa8b16011f WatchSource:0}: Error finding container 170fca5a493884c06dc89ada3d355086add8d5927127df6a4dce2eaa8b16011f: Status 404 returned error can't find the container with id 170fca5a493884c06dc89ada3d355086add8d5927127df6a4dce2eaa8b16011f
Apr 24 15:00:25.451841 ip-10-0-136-132 kubenswrapper[2574]: I0424 15:00:25.451803    2574 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/router-with-refs-pd-test-kserve-prefill-5c8777dc-txbgd" event={"ID":"4a48929f-f66c-49b8-9d3d-08c90eb4c1c6","Type":"ContainerStarted","Data":"7c8769c79147fa7ef6d67b1691039909eb37177cdc54807c7940df4d4b6ea209"}
Apr 24 15:00:25.451841 ip-10-0-136-132 kubenswrapper[2574]: I0424 15:00:25.451843    2574 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/router-with-refs-pd-test-kserve-prefill-5c8777dc-txbgd" event={"ID":"4a48929f-f66c-49b8-9d3d-08c90eb4c1c6","Type":"ContainerStarted","Data":"170fca5a493884c06dc89ada3d355086add8d5927127df6a4dce2eaa8b16011f"}
Apr 24 15:00:37.717733 ip-10-0-136-132 kubenswrapper[2574]: I0424 15:00:37.717659    2574 kuberuntime_container.go:864] "Killing container with a grace period" pod="kserve-ci-e2e-test/custom-route-timeout-pd-test-kserve-7cbf8d89d-8j8f9" podUID="8ad639c0-0908-443a-9a36-949955a89ee7" containerName="llm-d-routing-sidecar" containerID="cri-o://c8966114c184684e9d75d4912eaf66b3ac39a2360529bed84f02affbd1cf68b3" gracePeriod=2
Apr 24 15:00:38.049271 ip-10-0-136-132 kubenswrapper[2574]: I0424 15:00:38.049251    2574 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_custom-route-timeout-pd-test-kserve-7cbf8d89d-8j8f9_8ad639c0-0908-443a-9a36-949955a89ee7/main/0.log"
Apr 24 15:00:38.049943 ip-10-0-136-132 kubenswrapper[2574]: I0424 15:00:38.049925    2574 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/custom-route-timeout-pd-test-kserve-7cbf8d89d-8j8f9"
Apr 24 15:00:38.052867 ip-10-0-136-132 kubenswrapper[2574]: I0424 15:00:38.052831    2574 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/custom-route-timeout-pd-test-kserve-prefill-578d77b467-zfd6z"
Apr 24 15:00:38.173383 ip-10-0-136-132 kubenswrapper[2574]: I0424 15:00:38.173331    2574 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/8ad639c0-0908-443a-9a36-949955a89ee7-tls-certs\") pod \"8ad639c0-0908-443a-9a36-949955a89ee7\" (UID: \"8ad639c0-0908-443a-9a36-949955a89ee7\") "
Apr 24 15:00:38.173573 ip-10-0-136-132 kubenswrapper[2574]: I0424 15:00:38.173411    2574 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/73d63045-437e-49a4-be0c-cfbd579d93f5-model-cache\") pod \"73d63045-437e-49a4-be0c-cfbd579d93f5\" (UID: \"73d63045-437e-49a4-be0c-cfbd579d93f5\") "
Apr 24 15:00:38.173573 ip-10-0-136-132 kubenswrapper[2574]: I0424 15:00:38.173432    2574 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/8ad639c0-0908-443a-9a36-949955a89ee7-home\") pod \"8ad639c0-0908-443a-9a36-949955a89ee7\" (UID: \"8ad639c0-0908-443a-9a36-949955a89ee7\") "
Apr 24 15:00:38.173573 ip-10-0-136-132 kubenswrapper[2574]: I0424 15:00:38.173477    2574 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"kube-api-access-9h88x\" (UniqueName: \"kubernetes.io/projected/73d63045-437e-49a4-be0c-cfbd579d93f5-kube-api-access-9h88x\") pod \"73d63045-437e-49a4-be0c-cfbd579d93f5\" (UID: \"73d63045-437e-49a4-be0c-cfbd579d93f5\") "
Apr 24 15:00:38.173573 ip-10-0-136-132 kubenswrapper[2574]: I0424 15:00:38.173493    2574 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/8ad639c0-0908-443a-9a36-949955a89ee7-model-cache\") pod \"8ad639c0-0908-443a-9a36-949955a89ee7\" (UID: \"8ad639c0-0908-443a-9a36-949955a89ee7\") "
Apr 24 15:00:38.173573 ip-10-0-136-132 kubenswrapper[2574]: I0424 15:00:38.173515    2574 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/73d63045-437e-49a4-be0c-cfbd579d93f5-tls-certs\") pod \"73d63045-437e-49a4-be0c-cfbd579d93f5\" (UID: \"73d63045-437e-49a4-be0c-cfbd579d93f5\") "
Apr 24 15:00:38.173854 ip-10-0-136-132 kubenswrapper[2574]: I0424 15:00:38.173725    2574 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/73d63045-437e-49a4-be0c-cfbd579d93f5-model-cache" (OuterVolumeSpecName: "model-cache") pod "73d63045-437e-49a4-be0c-cfbd579d93f5" (UID: "73d63045-437e-49a4-be0c-cfbd579d93f5"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Apr 24 15:00:38.173854 ip-10-0-136-132 kubenswrapper[2574]: I0424 15:00:38.173761    2574 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/73d63045-437e-49a4-be0c-cfbd579d93f5-home\") pod \"73d63045-437e-49a4-be0c-cfbd579d93f5\" (UID: \"73d63045-437e-49a4-be0c-cfbd579d93f5\") "
Apr 24 15:00:38.173854 ip-10-0-136-132 kubenswrapper[2574]: I0424 15:00:38.173797    2574 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/8ad639c0-0908-443a-9a36-949955a89ee7-dshm\") pod \"8ad639c0-0908-443a-9a36-949955a89ee7\" (UID: \"8ad639c0-0908-443a-9a36-949955a89ee7\") "
Apr 24 15:00:38.173854 ip-10-0-136-132 kubenswrapper[2574]: I0424 15:00:38.173822    2574 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/8ad639c0-0908-443a-9a36-949955a89ee7-home" (OuterVolumeSpecName: "home") pod "8ad639c0-0908-443a-9a36-949955a89ee7" (UID: "8ad639c0-0908-443a-9a36-949955a89ee7"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Apr 24 15:00:38.174125 ip-10-0-136-132 kubenswrapper[2574]: I0424 15:00:38.173853    2574 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/73d63045-437e-49a4-be0c-cfbd579d93f5-kserve-provision-location\") pod \"73d63045-437e-49a4-be0c-cfbd579d93f5\" (UID: \"73d63045-437e-49a4-be0c-cfbd579d93f5\") "
Apr 24 15:00:38.174125 ip-10-0-136-132 kubenswrapper[2574]: I0424 15:00:38.173886    2574 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/73d63045-437e-49a4-be0c-cfbd579d93f5-dshm\") pod \"73d63045-437e-49a4-be0c-cfbd579d93f5\" (UID: \"73d63045-437e-49a4-be0c-cfbd579d93f5\") "
Apr 24 15:00:38.174125 ip-10-0-136-132 kubenswrapper[2574]: I0424 15:00:38.173923    2574 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"kube-api-access-mw4fg\" (UniqueName: \"kubernetes.io/projected/8ad639c0-0908-443a-9a36-949955a89ee7-kube-api-access-mw4fg\") pod \"8ad639c0-0908-443a-9a36-949955a89ee7\" (UID: \"8ad639c0-0908-443a-9a36-949955a89ee7\") "
Apr 24 15:00:38.174125 ip-10-0-136-132 kubenswrapper[2574]: I0424 15:00:38.173951    2574 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/8ad639c0-0908-443a-9a36-949955a89ee7-kserve-provision-location\") pod \"8ad639c0-0908-443a-9a36-949955a89ee7\" (UID: \"8ad639c0-0908-443a-9a36-949955a89ee7\") "
Apr 24 15:00:38.174125 ip-10-0-136-132 kubenswrapper[2574]: I0424 15:00:38.174116    2574 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/73d63045-437e-49a4-be0c-cfbd579d93f5-home" (OuterVolumeSpecName: "home") pod "73d63045-437e-49a4-be0c-cfbd579d93f5" (UID: "73d63045-437e-49a4-be0c-cfbd579d93f5"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Apr 24 15:00:38.175056 ip-10-0-136-132 kubenswrapper[2574]: I0424 15:00:38.174766    2574 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/73d63045-437e-49a4-be0c-cfbd579d93f5-model-cache\") on node \"ip-10-0-136-132.ec2.internal\" DevicePath \"\""
Apr 24 15:00:38.175056 ip-10-0-136-132 kubenswrapper[2574]: I0424 15:00:38.174794    2574 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/8ad639c0-0908-443a-9a36-949955a89ee7-home\") on node \"ip-10-0-136-132.ec2.internal\" DevicePath \"\""
Apr 24 15:00:38.175056 ip-10-0-136-132 kubenswrapper[2574]: I0424 15:00:38.174809    2574 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/73d63045-437e-49a4-be0c-cfbd579d93f5-home\") on node \"ip-10-0-136-132.ec2.internal\" DevicePath \"\""
Apr 24 15:00:38.175056 ip-10-0-136-132 kubenswrapper[2574]: I0424 15:00:38.174951    2574 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/8ad639c0-0908-443a-9a36-949955a89ee7-model-cache" (OuterVolumeSpecName: "model-cache") pod "8ad639c0-0908-443a-9a36-949955a89ee7" (UID: "8ad639c0-0908-443a-9a36-949955a89ee7"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Apr 24 15:00:38.176126 ip-10-0-136-132 kubenswrapper[2574]: I0424 15:00:38.176080    2574 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/73d63045-437e-49a4-be0c-cfbd579d93f5-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "73d63045-437e-49a4-be0c-cfbd579d93f5" (UID: "73d63045-437e-49a4-be0c-cfbd579d93f5"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Apr 24 15:00:38.176248 ip-10-0-136-132 kubenswrapper[2574]: I0424 15:00:38.176119    2574 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/73d63045-437e-49a4-be0c-cfbd579d93f5-kube-api-access-9h88x" (OuterVolumeSpecName: "kube-api-access-9h88x") pod "73d63045-437e-49a4-be0c-cfbd579d93f5" (UID: "73d63045-437e-49a4-be0c-cfbd579d93f5"). InnerVolumeSpecName "kube-api-access-9h88x". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Apr 24 15:00:38.176419 ip-10-0-136-132 kubenswrapper[2574]: I0424 15:00:38.176392    2574 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/8ad639c0-0908-443a-9a36-949955a89ee7-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "8ad639c0-0908-443a-9a36-949955a89ee7" (UID: "8ad639c0-0908-443a-9a36-949955a89ee7"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Apr 24 15:00:38.176537 ip-10-0-136-132 kubenswrapper[2574]: I0424 15:00:38.176474    2574 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/8ad639c0-0908-443a-9a36-949955a89ee7-dshm" (OuterVolumeSpecName: "dshm") pod "8ad639c0-0908-443a-9a36-949955a89ee7" (UID: "8ad639c0-0908-443a-9a36-949955a89ee7"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Apr 24 15:00:38.177220 ip-10-0-136-132 kubenswrapper[2574]: I0424 15:00:38.177197    2574 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/73d63045-437e-49a4-be0c-cfbd579d93f5-dshm" (OuterVolumeSpecName: "dshm") pod "73d63045-437e-49a4-be0c-cfbd579d93f5" (UID: "73d63045-437e-49a4-be0c-cfbd579d93f5"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Apr 24 15:00:38.177385 ip-10-0-136-132 kubenswrapper[2574]: I0424 15:00:38.177367    2574 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/8ad639c0-0908-443a-9a36-949955a89ee7-kube-api-access-mw4fg" (OuterVolumeSpecName: "kube-api-access-mw4fg") pod "8ad639c0-0908-443a-9a36-949955a89ee7" (UID: "8ad639c0-0908-443a-9a36-949955a89ee7"). InnerVolumeSpecName "kube-api-access-mw4fg". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Apr 24 15:00:38.239719 ip-10-0-136-132 kubenswrapper[2574]: I0424 15:00:38.239627    2574 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/8ad639c0-0908-443a-9a36-949955a89ee7-kserve-provision-location" (OuterVolumeSpecName: "kserve-provision-location") pod "8ad639c0-0908-443a-9a36-949955a89ee7" (UID: "8ad639c0-0908-443a-9a36-949955a89ee7"). InnerVolumeSpecName "kserve-provision-location". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Apr 24 15:00:38.241499 ip-10-0-136-132 kubenswrapper[2574]: I0424 15:00:38.241463    2574 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/73d63045-437e-49a4-be0c-cfbd579d93f5-kserve-provision-location" (OuterVolumeSpecName: "kserve-provision-location") pod "73d63045-437e-49a4-be0c-cfbd579d93f5" (UID: "73d63045-437e-49a4-be0c-cfbd579d93f5"). InnerVolumeSpecName "kserve-provision-location". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Apr 24 15:00:38.275235 ip-10-0-136-132 kubenswrapper[2574]: I0424 15:00:38.275201    2574 reconciler_common.go:299] "Volume detached for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/73d63045-437e-49a4-be0c-cfbd579d93f5-kserve-provision-location\") on node \"ip-10-0-136-132.ec2.internal\" DevicePath \"\""
Apr 24 15:00:38.275235 ip-10-0-136-132 kubenswrapper[2574]: I0424 15:00:38.275233    2574 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/73d63045-437e-49a4-be0c-cfbd579d93f5-dshm\") on node \"ip-10-0-136-132.ec2.internal\" DevicePath \"\""
Apr 24 15:00:38.275465 ip-10-0-136-132 kubenswrapper[2574]: I0424 15:00:38.275244    2574 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-mw4fg\" (UniqueName: \"kubernetes.io/projected/8ad639c0-0908-443a-9a36-949955a89ee7-kube-api-access-mw4fg\") on node \"ip-10-0-136-132.ec2.internal\" DevicePath \"\""
Apr 24 15:00:38.275465 ip-10-0-136-132 kubenswrapper[2574]: I0424 15:00:38.275253    2574 reconciler_common.go:299] "Volume detached for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/8ad639c0-0908-443a-9a36-949955a89ee7-kserve-provision-location\") on node \"ip-10-0-136-132.ec2.internal\" DevicePath \"\""
Apr 24 15:00:38.275465 ip-10-0-136-132 kubenswrapper[2574]: I0424 15:00:38.275264    2574 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/8ad639c0-0908-443a-9a36-949955a89ee7-tls-certs\") on node \"ip-10-0-136-132.ec2.internal\" DevicePath \"\""
Apr 24 15:00:38.275465 ip-10-0-136-132 kubenswrapper[2574]: I0424 15:00:38.275272    2574 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-9h88x\" (UniqueName: \"kubernetes.io/projected/73d63045-437e-49a4-be0c-cfbd579d93f5-kube-api-access-9h88x\") on node \"ip-10-0-136-132.ec2.internal\" DevicePath \"\""
Apr 24 15:00:38.275465 ip-10-0-136-132 kubenswrapper[2574]: I0424 15:00:38.275281    2574 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/8ad639c0-0908-443a-9a36-949955a89ee7-model-cache\") on node \"ip-10-0-136-132.ec2.internal\" DevicePath \"\""
Apr 24 15:00:38.275465 ip-10-0-136-132 kubenswrapper[2574]: I0424 15:00:38.275289    2574 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/73d63045-437e-49a4-be0c-cfbd579d93f5-tls-certs\") on node \"ip-10-0-136-132.ec2.internal\" DevicePath \"\""
Apr 24 15:00:38.275465 ip-10-0-136-132 kubenswrapper[2574]: I0424 15:00:38.275298    2574 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/8ad639c0-0908-443a-9a36-949955a89ee7-dshm\") on node \"ip-10-0-136-132.ec2.internal\" DevicePath \"\""
Apr 24 15:00:38.504999 ip-10-0-136-132 kubenswrapper[2574]: I0424 15:00:38.504929    2574 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_custom-route-timeout-pd-test-kserve-7cbf8d89d-8j8f9_8ad639c0-0908-443a-9a36-949955a89ee7/main/0.log"
Apr 24 15:00:38.505564 ip-10-0-136-132 kubenswrapper[2574]: I0424 15:00:38.505541    2574 generic.go:358] "Generic (PLEG): container finished" podID="8ad639c0-0908-443a-9a36-949955a89ee7" containerID="724ba3b78c6fbbed91ea92e813bfbf0334160fefccecb667f544c41d3297e574" exitCode=137
Apr 24 15:00:38.505564 ip-10-0-136-132 kubenswrapper[2574]: I0424 15:00:38.505560    2574 generic.go:358] "Generic (PLEG): container finished" podID="8ad639c0-0908-443a-9a36-949955a89ee7" containerID="c8966114c184684e9d75d4912eaf66b3ac39a2360529bed84f02affbd1cf68b3" exitCode=0
Apr 24 15:00:38.505719 ip-10-0-136-132 kubenswrapper[2574]: I0424 15:00:38.505625    2574 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/custom-route-timeout-pd-test-kserve-7cbf8d89d-8j8f9" event={"ID":"8ad639c0-0908-443a-9a36-949955a89ee7","Type":"ContainerDied","Data":"724ba3b78c6fbbed91ea92e813bfbf0334160fefccecb667f544c41d3297e574"}
Apr 24 15:00:38.505719 ip-10-0-136-132 kubenswrapper[2574]: I0424 15:00:38.505668    2574 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/custom-route-timeout-pd-test-kserve-7cbf8d89d-8j8f9" event={"ID":"8ad639c0-0908-443a-9a36-949955a89ee7","Type":"ContainerDied","Data":"c8966114c184684e9d75d4912eaf66b3ac39a2360529bed84f02affbd1cf68b3"}
Apr 24 15:00:38.505719 ip-10-0-136-132 kubenswrapper[2574]: I0424 15:00:38.505633    2574 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/custom-route-timeout-pd-test-kserve-7cbf8d89d-8j8f9"
Apr 24 15:00:38.505719 ip-10-0-136-132 kubenswrapper[2574]: I0424 15:00:38.505701    2574 scope.go:117] "RemoveContainer" containerID="724ba3b78c6fbbed91ea92e813bfbf0334160fefccecb667f544c41d3297e574"
Apr 24 15:00:38.505897 ip-10-0-136-132 kubenswrapper[2574]: I0424 15:00:38.505687    2574 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/custom-route-timeout-pd-test-kserve-7cbf8d89d-8j8f9" event={"ID":"8ad639c0-0908-443a-9a36-949955a89ee7","Type":"ContainerDied","Data":"c689a0c777681dc25ace852028277ff620f19f5f6c4de3c1a3ea78a7fa2946cb"}
Apr 24 15:00:38.507154 ip-10-0-136-132 kubenswrapper[2574]: I0424 15:00:38.507132    2574 generic.go:358] "Generic (PLEG): container finished" podID="73d63045-437e-49a4-be0c-cfbd579d93f5" containerID="d5396dd69e4f9924570281c5031d973ff554aa478fd76c82d2b9d281de73f8fd" exitCode=137
Apr 24 15:00:38.507277 ip-10-0-136-132 kubenswrapper[2574]: I0424 15:00:38.507206    2574 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/custom-route-timeout-pd-test-kserve-prefill-578d77b467-zfd6z"
Apr 24 15:00:38.507277 ip-10-0-136-132 kubenswrapper[2574]: I0424 15:00:38.507211    2574 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/custom-route-timeout-pd-test-kserve-prefill-578d77b467-zfd6z" event={"ID":"73d63045-437e-49a4-be0c-cfbd579d93f5","Type":"ContainerDied","Data":"d5396dd69e4f9924570281c5031d973ff554aa478fd76c82d2b9d281de73f8fd"}
Apr 24 15:00:38.507277 ip-10-0-136-132 kubenswrapper[2574]: I0424 15:00:38.507256    2574 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/custom-route-timeout-pd-test-kserve-prefill-578d77b467-zfd6z" event={"ID":"73d63045-437e-49a4-be0c-cfbd579d93f5","Type":"ContainerDied","Data":"6c067c9ceeb9cb6d3a0d5c977b41da4c093798ad05d41637c58ebf93c9d14135"}
Apr 24 15:00:38.533444 ip-10-0-136-132 kubenswrapper[2574]: I0424 15:00:38.533418    2574 scope.go:117] "RemoveContainer" containerID="ecea0f0dc3ac760a288d2737d765947fc28ad70473419edf52e3e831b0d2c1c7"
Apr 24 15:00:38.533957 ip-10-0-136-132 kubenswrapper[2574]: I0424 15:00:38.533926    2574 kubelet.go:2553] "SyncLoop DELETE" source="api" pods=["kserve-ci-e2e-test/custom-route-timeout-pd-test-kserve-7cbf8d89d-8j8f9"]
Apr 24 15:00:38.538451 ip-10-0-136-132 kubenswrapper[2574]: I0424 15:00:38.538426    2574 kubelet.go:2547] "SyncLoop REMOVE" source="api" pods=["kserve-ci-e2e-test/custom-route-timeout-pd-test-kserve-7cbf8d89d-8j8f9"]
Apr 24 15:00:38.542983 ip-10-0-136-132 kubenswrapper[2574]: I0424 15:00:38.542963    2574 kubelet.go:2553] "SyncLoop DELETE" source="api" pods=["kserve-ci-e2e-test/custom-route-timeout-pd-test-kserve-prefill-578d77b467-zfd6z"]
Apr 24 15:00:38.545925 ip-10-0-136-132 kubenswrapper[2574]: I0424 15:00:38.545904    2574 kubelet.go:2547] "SyncLoop REMOVE" source="api" pods=["kserve-ci-e2e-test/custom-route-timeout-pd-test-kserve-prefill-578d77b467-zfd6z"]
Apr 24 15:00:38.548311 ip-10-0-136-132 kubenswrapper[2574]: I0424 15:00:38.548287    2574 scope.go:117] "RemoveContainer" containerID="c8966114c184684e9d75d4912eaf66b3ac39a2360529bed84f02affbd1cf68b3"
Apr 24 15:00:38.556286 ip-10-0-136-132 kubenswrapper[2574]: I0424 15:00:38.556264    2574 scope.go:117] "RemoveContainer" containerID="724ba3b78c6fbbed91ea92e813bfbf0334160fefccecb667f544c41d3297e574"
Apr 24 15:00:38.556589 ip-10-0-136-132 kubenswrapper[2574]: E0424 15:00:38.556570    2574 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"724ba3b78c6fbbed91ea92e813bfbf0334160fefccecb667f544c41d3297e574\": container with ID starting with 724ba3b78c6fbbed91ea92e813bfbf0334160fefccecb667f544c41d3297e574 not found: ID does not exist" containerID="724ba3b78c6fbbed91ea92e813bfbf0334160fefccecb667f544c41d3297e574"
Apr 24 15:00:38.556645 ip-10-0-136-132 kubenswrapper[2574]: I0424 15:00:38.556598    2574 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"724ba3b78c6fbbed91ea92e813bfbf0334160fefccecb667f544c41d3297e574"} err="failed to get container status \"724ba3b78c6fbbed91ea92e813bfbf0334160fefccecb667f544c41d3297e574\": rpc error: code = NotFound desc = could not find container \"724ba3b78c6fbbed91ea92e813bfbf0334160fefccecb667f544c41d3297e574\": container with ID starting with 724ba3b78c6fbbed91ea92e813bfbf0334160fefccecb667f544c41d3297e574 not found: ID does not exist"
Apr 24 15:00:38.556645 ip-10-0-136-132 kubenswrapper[2574]: I0424 15:00:38.556619    2574 scope.go:117] "RemoveContainer" containerID="ecea0f0dc3ac760a288d2737d765947fc28ad70473419edf52e3e831b0d2c1c7"
Apr 24 15:00:38.556872 ip-10-0-136-132 kubenswrapper[2574]: E0424 15:00:38.556854    2574 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"ecea0f0dc3ac760a288d2737d765947fc28ad70473419edf52e3e831b0d2c1c7\": container with ID starting with ecea0f0dc3ac760a288d2737d765947fc28ad70473419edf52e3e831b0d2c1c7 not found: ID does not exist" containerID="ecea0f0dc3ac760a288d2737d765947fc28ad70473419edf52e3e831b0d2c1c7"
Apr 24 15:00:38.556933 ip-10-0-136-132 kubenswrapper[2574]: I0424 15:00:38.556881    2574 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"ecea0f0dc3ac760a288d2737d765947fc28ad70473419edf52e3e831b0d2c1c7"} err="failed to get container status \"ecea0f0dc3ac760a288d2737d765947fc28ad70473419edf52e3e831b0d2c1c7\": rpc error: code = NotFound desc = could not find container \"ecea0f0dc3ac760a288d2737d765947fc28ad70473419edf52e3e831b0d2c1c7\": container with ID starting with ecea0f0dc3ac760a288d2737d765947fc28ad70473419edf52e3e831b0d2c1c7 not found: ID does not exist"
Apr 24 15:00:38.556933 ip-10-0-136-132 kubenswrapper[2574]: I0424 15:00:38.556904    2574 scope.go:117] "RemoveContainer" containerID="c8966114c184684e9d75d4912eaf66b3ac39a2360529bed84f02affbd1cf68b3"
Apr 24 15:00:38.557137 ip-10-0-136-132 kubenswrapper[2574]: E0424 15:00:38.557122    2574 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"c8966114c184684e9d75d4912eaf66b3ac39a2360529bed84f02affbd1cf68b3\": container with ID starting with c8966114c184684e9d75d4912eaf66b3ac39a2360529bed84f02affbd1cf68b3 not found: ID does not exist" containerID="c8966114c184684e9d75d4912eaf66b3ac39a2360529bed84f02affbd1cf68b3"
Apr 24 15:00:38.557197 ip-10-0-136-132 kubenswrapper[2574]: I0424 15:00:38.557144    2574 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"c8966114c184684e9d75d4912eaf66b3ac39a2360529bed84f02affbd1cf68b3"} err="failed to get container status \"c8966114c184684e9d75d4912eaf66b3ac39a2360529bed84f02affbd1cf68b3\": rpc error: code = NotFound desc = could not find container \"c8966114c184684e9d75d4912eaf66b3ac39a2360529bed84f02affbd1cf68b3\": container with ID starting with c8966114c184684e9d75d4912eaf66b3ac39a2360529bed84f02affbd1cf68b3 not found: ID does not exist"
Apr 24 15:00:38.557197 ip-10-0-136-132 kubenswrapper[2574]: I0424 15:00:38.557163    2574 scope.go:117] "RemoveContainer" containerID="724ba3b78c6fbbed91ea92e813bfbf0334160fefccecb667f544c41d3297e574"
Apr 24 15:00:38.557445 ip-10-0-136-132 kubenswrapper[2574]: I0424 15:00:38.557423    2574 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"724ba3b78c6fbbed91ea92e813bfbf0334160fefccecb667f544c41d3297e574"} err="failed to get container status \"724ba3b78c6fbbed91ea92e813bfbf0334160fefccecb667f544c41d3297e574\": rpc error: code = NotFound desc = could not find container \"724ba3b78c6fbbed91ea92e813bfbf0334160fefccecb667f544c41d3297e574\": container with ID starting with 724ba3b78c6fbbed91ea92e813bfbf0334160fefccecb667f544c41d3297e574 not found: ID does not exist"
Apr 24 15:00:38.557493 ip-10-0-136-132 kubenswrapper[2574]: I0424 15:00:38.557445    2574 scope.go:117] "RemoveContainer" containerID="ecea0f0dc3ac760a288d2737d765947fc28ad70473419edf52e3e831b0d2c1c7"
Apr 24 15:00:38.557700 ip-10-0-136-132 kubenswrapper[2574]: I0424 15:00:38.557684    2574 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"ecea0f0dc3ac760a288d2737d765947fc28ad70473419edf52e3e831b0d2c1c7"} err="failed to get container status \"ecea0f0dc3ac760a288d2737d765947fc28ad70473419edf52e3e831b0d2c1c7\": rpc error: code = NotFound desc = could not find container \"ecea0f0dc3ac760a288d2737d765947fc28ad70473419edf52e3e831b0d2c1c7\": container with ID starting with ecea0f0dc3ac760a288d2737d765947fc28ad70473419edf52e3e831b0d2c1c7 not found: ID does not exist"
Apr 24 15:00:38.557700 ip-10-0-136-132 kubenswrapper[2574]: I0424 15:00:38.557700    2574 scope.go:117] "RemoveContainer" containerID="c8966114c184684e9d75d4912eaf66b3ac39a2360529bed84f02affbd1cf68b3"
Apr 24 15:00:38.557910 ip-10-0-136-132 kubenswrapper[2574]: I0424 15:00:38.557888    2574 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"c8966114c184684e9d75d4912eaf66b3ac39a2360529bed84f02affbd1cf68b3"} err="failed to get container status \"c8966114c184684e9d75d4912eaf66b3ac39a2360529bed84f02affbd1cf68b3\": rpc error: code = NotFound desc = could not find container \"c8966114c184684e9d75d4912eaf66b3ac39a2360529bed84f02affbd1cf68b3\": container with ID starting with c8966114c184684e9d75d4912eaf66b3ac39a2360529bed84f02affbd1cf68b3 not found: ID does not exist"
Apr 24 15:00:38.557910 ip-10-0-136-132 kubenswrapper[2574]: I0424 15:00:38.557909    2574 scope.go:117] "RemoveContainer" containerID="d5396dd69e4f9924570281c5031d973ff554aa478fd76c82d2b9d281de73f8fd"
Apr 24 15:00:38.576674 ip-10-0-136-132 kubenswrapper[2574]: I0424 15:00:38.576653    2574 scope.go:117] "RemoveContainer" containerID="7dfbc3b0597d73f71629a6f375de4212a6a0a9e88145bd35b19461457699bfe5"
Apr 24 15:00:38.644848 ip-10-0-136-132 kubenswrapper[2574]: I0424 15:00:38.644826    2574 scope.go:117] "RemoveContainer" containerID="d5396dd69e4f9924570281c5031d973ff554aa478fd76c82d2b9d281de73f8fd"
Apr 24 15:00:38.645205 ip-10-0-136-132 kubenswrapper[2574]: E0424 15:00:38.645183    2574 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"d5396dd69e4f9924570281c5031d973ff554aa478fd76c82d2b9d281de73f8fd\": container with ID starting with d5396dd69e4f9924570281c5031d973ff554aa478fd76c82d2b9d281de73f8fd not found: ID does not exist" containerID="d5396dd69e4f9924570281c5031d973ff554aa478fd76c82d2b9d281de73f8fd"
Apr 24 15:00:38.645293 ip-10-0-136-132 kubenswrapper[2574]: I0424 15:00:38.645212    2574 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d5396dd69e4f9924570281c5031d973ff554aa478fd76c82d2b9d281de73f8fd"} err="failed to get container status \"d5396dd69e4f9924570281c5031d973ff554aa478fd76c82d2b9d281de73f8fd\": rpc error: code = NotFound desc = could not find container \"d5396dd69e4f9924570281c5031d973ff554aa478fd76c82d2b9d281de73f8fd\": container with ID starting with d5396dd69e4f9924570281c5031d973ff554aa478fd76c82d2b9d281de73f8fd not found: ID does not exist"
Apr 24 15:00:38.645293 ip-10-0-136-132 kubenswrapper[2574]: I0424 15:00:38.645231    2574 scope.go:117] "RemoveContainer" containerID="7dfbc3b0597d73f71629a6f375de4212a6a0a9e88145bd35b19461457699bfe5"
Apr 24 15:00:38.645527 ip-10-0-136-132 kubenswrapper[2574]: E0424 15:00:38.645509    2574 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"7dfbc3b0597d73f71629a6f375de4212a6a0a9e88145bd35b19461457699bfe5\": container with ID starting with 7dfbc3b0597d73f71629a6f375de4212a6a0a9e88145bd35b19461457699bfe5 not found: ID does not exist" containerID="7dfbc3b0597d73f71629a6f375de4212a6a0a9e88145bd35b19461457699bfe5"
Apr 24 15:00:38.645573 ip-10-0-136-132 kubenswrapper[2574]: I0424 15:00:38.645539    2574 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"7dfbc3b0597d73f71629a6f375de4212a6a0a9e88145bd35b19461457699bfe5"} err="failed to get container status \"7dfbc3b0597d73f71629a6f375de4212a6a0a9e88145bd35b19461457699bfe5\": rpc error: code = NotFound desc = could not find container \"7dfbc3b0597d73f71629a6f375de4212a6a0a9e88145bd35b19461457699bfe5\": container with ID starting with 7dfbc3b0597d73f71629a6f375de4212a6a0a9e88145bd35b19461457699bfe5 not found: ID does not exist"
Apr 24 15:00:39.771318 ip-10-0-136-132 kubenswrapper[2574]: I0424 15:00:39.771283    2574 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="73d63045-437e-49a4-be0c-cfbd579d93f5" path="/var/lib/kubelet/pods/73d63045-437e-49a4-be0c-cfbd579d93f5/volumes"
Apr 24 15:00:39.771809 ip-10-0-136-132 kubenswrapper[2574]: I0424 15:00:39.771792    2574 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="8ad639c0-0908-443a-9a36-949955a89ee7" path="/var/lib/kubelet/pods/8ad639c0-0908-443a-9a36-949955a89ee7/volumes"
Apr 24 15:00:55.578657 ip-10-0-136-132 kubenswrapper[2574]: I0424 15:00:55.578622    2574 generic.go:358] "Generic (PLEG): container finished" podID="4a48929f-f66c-49b8-9d3d-08c90eb4c1c6" containerID="7c8769c79147fa7ef6d67b1691039909eb37177cdc54807c7940df4d4b6ea209" exitCode=0
Apr 24 15:00:55.579104 ip-10-0-136-132 kubenswrapper[2574]: I0424 15:00:55.578705    2574 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/router-with-refs-pd-test-kserve-prefill-5c8777dc-txbgd" event={"ID":"4a48929f-f66c-49b8-9d3d-08c90eb4c1c6","Type":"ContainerDied","Data":"7c8769c79147fa7ef6d67b1691039909eb37177cdc54807c7940df4d4b6ea209"}
Apr 24 15:00:56.585255 ip-10-0-136-132 kubenswrapper[2574]: I0424 15:00:56.585221    2574 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/router-with-refs-pd-test-kserve-prefill-5c8777dc-txbgd" event={"ID":"4a48929f-f66c-49b8-9d3d-08c90eb4c1c6","Type":"ContainerStarted","Data":"a41b6c69d055b8a1a292068cf88530f7ead7ebb18368a93ffe1eb028fb59ae2f"}
Apr 24 15:00:56.602622 ip-10-0-136-132 kubenswrapper[2574]: I0424 15:00:56.602573    2574 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="kserve-ci-e2e-test/router-with-refs-pd-test-kserve-prefill-5c8777dc-txbgd" podStartSLOduration=32.602555658 podStartE2EDuration="32.602555658s" podCreationTimestamp="2026-04-24 15:00:24 +0000 UTC" firstStartedPulling="0001-01-01 00:00:00 +0000 UTC" lastFinishedPulling="0001-01-01 00:00:00 +0000 UTC" observedRunningTime="2026-04-24 15:00:56.601373151 +0000 UTC m=+2219.333474790" watchObservedRunningTime="2026-04-24 15:00:56.602555658 +0000 UTC m=+2219.334657308"
Apr 24 15:01:04.805709 ip-10-0-136-132 kubenswrapper[2574]: I0424 15:01:04.805670    2574 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve-ci-e2e-test/router-with-refs-pd-test-kserve-prefill-5c8777dc-txbgd"
Apr 24 15:01:04.806089 ip-10-0-136-132 kubenswrapper[2574]: I0424 15:01:04.805720    2574 kubelet.go:2658] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="kserve-ci-e2e-test/router-with-refs-pd-test-kserve-prefill-5c8777dc-txbgd"
Apr 24 15:01:04.807100 ip-10-0-136-132 kubenswrapper[2574]: I0424 15:01:04.807071    2574 prober.go:120] "Probe failed" probeType="Startup" pod="kserve-ci-e2e-test/router-with-refs-pd-test-kserve-prefill-5c8777dc-txbgd" podUID="4a48929f-f66c-49b8-9d3d-08c90eb4c1c6" containerName="main" probeResult="failure" output="Get \"https://10.133.0.64:8000/health\": dial tcp 10.133.0.64:8000: connect: connection refused"
Apr 24 15:01:14.806461 ip-10-0-136-132 kubenswrapper[2574]: I0424 15:01:14.806366    2574 prober.go:120] "Probe failed" probeType="Startup" pod="kserve-ci-e2e-test/router-with-refs-pd-test-kserve-prefill-5c8777dc-txbgd" podUID="4a48929f-f66c-49b8-9d3d-08c90eb4c1c6" containerName="main" probeResult="failure" output="Get \"https://10.133.0.64:8000/health\": dial tcp 10.133.0.64:8000: connect: connection refused"
Apr 24 15:01:24.806687 ip-10-0-136-132 kubenswrapper[2574]: I0424 15:01:24.806630    2574 prober.go:120] "Probe failed" probeType="Startup" pod="kserve-ci-e2e-test/router-with-refs-pd-test-kserve-prefill-5c8777dc-txbgd" podUID="4a48929f-f66c-49b8-9d3d-08c90eb4c1c6" containerName="main" probeResult="failure" output="Get \"https://10.133.0.64:8000/health\": dial tcp 10.133.0.64:8000: connect: connection refused"
Apr 24 15:01:34.806823 ip-10-0-136-132 kubenswrapper[2574]: I0424 15:01:34.806779    2574 prober.go:120] "Probe failed" probeType="Startup" pod="kserve-ci-e2e-test/router-with-refs-pd-test-kserve-prefill-5c8777dc-txbgd" podUID="4a48929f-f66c-49b8-9d3d-08c90eb4c1c6" containerName="main" probeResult="failure" output="Get \"https://10.133.0.64:8000/health\": dial tcp 10.133.0.64:8000: connect: connection refused"
Apr 24 15:01:44.806912 ip-10-0-136-132 kubenswrapper[2574]: I0424 15:01:44.806867    2574 prober.go:120] "Probe failed" probeType="Startup" pod="kserve-ci-e2e-test/router-with-refs-pd-test-kserve-prefill-5c8777dc-txbgd" podUID="4a48929f-f66c-49b8-9d3d-08c90eb4c1c6" containerName="main" probeResult="failure" output="Get \"https://10.133.0.64:8000/health\": dial tcp 10.133.0.64:8000: connect: connection refused"
Apr 24 15:01:51.842497 ip-10-0-136-132 kubenswrapper[2574]: I0424 15:01:51.842463    2574 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["kserve-ci-e2e-test/llmisvc-model-fb-opt-125m-route-05aa9bba-kserve-568864c59fhvjqg"]
Apr 24 15:01:51.843102 ip-10-0-136-132 kubenswrapper[2574]: I0424 15:01:51.843068    2574 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="73d63045-437e-49a4-be0c-cfbd579d93f5" containerName="storage-initializer"
Apr 24 15:01:51.843102 ip-10-0-136-132 kubenswrapper[2574]: I0424 15:01:51.843089    2574 state_mem.go:107] "Deleted CPUSet assignment" podUID="73d63045-437e-49a4-be0c-cfbd579d93f5" containerName="storage-initializer"
Apr 24 15:01:51.843190 ip-10-0-136-132 kubenswrapper[2574]: I0424 15:01:51.843106    2574 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="73d63045-437e-49a4-be0c-cfbd579d93f5" containerName="main"
Apr 24 15:01:51.843190 ip-10-0-136-132 kubenswrapper[2574]: I0424 15:01:51.843115    2574 state_mem.go:107] "Deleted CPUSet assignment" podUID="73d63045-437e-49a4-be0c-cfbd579d93f5" containerName="main"
Apr 24 15:01:51.843190 ip-10-0-136-132 kubenswrapper[2574]: I0424 15:01:51.843137    2574 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="8ad639c0-0908-443a-9a36-949955a89ee7" containerName="llm-d-routing-sidecar"
Apr 24 15:01:51.843190 ip-10-0-136-132 kubenswrapper[2574]: I0424 15:01:51.843143    2574 state_mem.go:107] "Deleted CPUSet assignment" podUID="8ad639c0-0908-443a-9a36-949955a89ee7" containerName="llm-d-routing-sidecar"
Apr 24 15:01:51.843190 ip-10-0-136-132 kubenswrapper[2574]: I0424 15:01:51.843149    2574 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="8ad639c0-0908-443a-9a36-949955a89ee7" containerName="main"
Apr 24 15:01:51.843190 ip-10-0-136-132 kubenswrapper[2574]: I0424 15:01:51.843155    2574 state_mem.go:107] "Deleted CPUSet assignment" podUID="8ad639c0-0908-443a-9a36-949955a89ee7" containerName="main"
Apr 24 15:01:51.843190 ip-10-0-136-132 kubenswrapper[2574]: I0424 15:01:51.843164    2574 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="8ad639c0-0908-443a-9a36-949955a89ee7" containerName="storage-initializer"
Apr 24 15:01:51.843190 ip-10-0-136-132 kubenswrapper[2574]: I0424 15:01:51.843169    2574 state_mem.go:107] "Deleted CPUSet assignment" podUID="8ad639c0-0908-443a-9a36-949955a89ee7" containerName="storage-initializer"
Apr 24 15:01:51.843456 ip-10-0-136-132 kubenswrapper[2574]: I0424 15:01:51.843237    2574 memory_manager.go:356] "RemoveStaleState removing state" podUID="8ad639c0-0908-443a-9a36-949955a89ee7" containerName="llm-d-routing-sidecar"
Apr 24 15:01:51.843456 ip-10-0-136-132 kubenswrapper[2574]: I0424 15:01:51.843246    2574 memory_manager.go:356] "RemoveStaleState removing state" podUID="8ad639c0-0908-443a-9a36-949955a89ee7" containerName="main"
Apr 24 15:01:51.843456 ip-10-0-136-132 kubenswrapper[2574]: I0424 15:01:51.843255    2574 memory_manager.go:356] "RemoveStaleState removing state" podUID="73d63045-437e-49a4-be0c-cfbd579d93f5" containerName="main"
Apr 24 15:01:51.847202 ip-10-0-136-132 kubenswrapper[2574]: I0424 15:01:51.847181    2574 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/llmisvc-model-fb-opt-125m-route-05aa9bba-kserve-568864c59fhvjqg"
Apr 24 15:01:51.849264 ip-10-0-136-132 kubenswrapper[2574]: I0424 15:01:51.849236    2574 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"kserve-ci-e2e-test\"/\"llmisvc-model-fb-opt-125m-route-05aa9bba-kserve-dockercfg-svpw5\""
Apr 24 15:01:51.849403 ip-10-0-136-132 kubenswrapper[2574]: I0424 15:01:51.849331    2574 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"kserve-ci-e2e-test\"/\"llmisv8f1a6f044e8c7a4d31a250e0c4861caf-kserve-self-signed-certs\""
Apr 24 15:01:51.855947 ip-10-0-136-132 kubenswrapper[2574]: I0424 15:01:51.855925    2574 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["kserve-ci-e2e-test/llmisvc-model-fb-opt-125m-route-05aa9bba-kserve-prefill-7f2szqt"]
Apr 24 15:01:51.862807 ip-10-0-136-132 kubenswrapper[2574]: I0424 15:01:51.862784    2574 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/llmisvc-model-fb-opt-125m-route-05aa9bba-kserve-568864c59fhvjqg"]
Apr 24 15:01:51.862934 ip-10-0-136-132 kubenswrapper[2574]: I0424 15:01:51.862909    2574 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/llmisvc-model-fb-opt-125m-route-05aa9bba-kserve-prefill-7f2szqt"
Apr 24 15:01:51.869437 ip-10-0-136-132 kubenswrapper[2574]: I0424 15:01:51.869416    2574 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/llmisvc-model-fb-opt-125m-route-05aa9bba-kserve-prefill-7f2szqt"]
Apr 24 15:01:51.944507 ip-10-0-136-132 kubenswrapper[2574]: I0424 15:01:51.944473    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6dgsc\" (UniqueName: \"kubernetes.io/projected/7a613b84-cbb1-4ab8-bc7c-2187ccaae8e6-kube-api-access-6dgsc\") pod \"llmisvc-model-fb-opt-125m-route-05aa9bba-kserve-568864c59fhvjqg\" (UID: \"7a613b84-cbb1-4ab8-bc7c-2187ccaae8e6\") " pod="kserve-ci-e2e-test/llmisvc-model-fb-opt-125m-route-05aa9bba-kserve-568864c59fhvjqg"
Apr 24 15:01:51.944507 ip-10-0-136-132 kubenswrapper[2574]: I0424 15:01:51.944509    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/7a613b84-cbb1-4ab8-bc7c-2187ccaae8e6-dshm\") pod \"llmisvc-model-fb-opt-125m-route-05aa9bba-kserve-568864c59fhvjqg\" (UID: \"7a613b84-cbb1-4ab8-bc7c-2187ccaae8e6\") " pod="kserve-ci-e2e-test/llmisvc-model-fb-opt-125m-route-05aa9bba-kserve-568864c59fhvjqg"
Apr 24 15:01:51.944750 ip-10-0-136-132 kubenswrapper[2574]: I0424 15:01:51.944532    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/7a613b84-cbb1-4ab8-bc7c-2187ccaae8e6-home\") pod \"llmisvc-model-fb-opt-125m-route-05aa9bba-kserve-568864c59fhvjqg\" (UID: \"7a613b84-cbb1-4ab8-bc7c-2187ccaae8e6\") " pod="kserve-ci-e2e-test/llmisvc-model-fb-opt-125m-route-05aa9bba-kserve-568864c59fhvjqg"
Apr 24 15:01:51.944750 ip-10-0-136-132 kubenswrapper[2574]: I0424 15:01:51.944574    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/7a613b84-cbb1-4ab8-bc7c-2187ccaae8e6-kserve-provision-location\") pod \"llmisvc-model-fb-opt-125m-route-05aa9bba-kserve-568864c59fhvjqg\" (UID: \"7a613b84-cbb1-4ab8-bc7c-2187ccaae8e6\") " pod="kserve-ci-e2e-test/llmisvc-model-fb-opt-125m-route-05aa9bba-kserve-568864c59fhvjqg"
Apr 24 15:01:51.944750 ip-10-0-136-132 kubenswrapper[2574]: I0424 15:01:51.944618    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/7a613b84-cbb1-4ab8-bc7c-2187ccaae8e6-tls-certs\") pod \"llmisvc-model-fb-opt-125m-route-05aa9bba-kserve-568864c59fhvjqg\" (UID: \"7a613b84-cbb1-4ab8-bc7c-2187ccaae8e6\") " pod="kserve-ci-e2e-test/llmisvc-model-fb-opt-125m-route-05aa9bba-kserve-568864c59fhvjqg"
Apr 24 15:01:51.944750 ip-10-0-136-132 kubenswrapper[2574]: I0424 15:01:51.944659    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/7a613b84-cbb1-4ab8-bc7c-2187ccaae8e6-model-cache\") pod \"llmisvc-model-fb-opt-125m-route-05aa9bba-kserve-568864c59fhvjqg\" (UID: \"7a613b84-cbb1-4ab8-bc7c-2187ccaae8e6\") " pod="kserve-ci-e2e-test/llmisvc-model-fb-opt-125m-route-05aa9bba-kserve-568864c59fhvjqg"
Apr 24 15:01:52.045613 ip-10-0-136-132 kubenswrapper[2574]: I0424 15:01:52.045578    2574 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/7a613b84-cbb1-4ab8-bc7c-2187ccaae8e6-tls-certs\") pod \"llmisvc-model-fb-opt-125m-route-05aa9bba-kserve-568864c59fhvjqg\" (UID: \"7a613b84-cbb1-4ab8-bc7c-2187ccaae8e6\") " pod="kserve-ci-e2e-test/llmisvc-model-fb-opt-125m-route-05aa9bba-kserve-568864c59fhvjqg"
Apr 24 15:01:52.045613 ip-10-0-136-132 kubenswrapper[2574]: I0424 15:01:52.045617    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/f0cfa28d-bba4-4e59-80a9-55145e2e51f8-kserve-provision-location\") pod \"llmisvc-model-fb-opt-125m-route-05aa9bba-kserve-prefill-7f2szqt\" (UID: \"f0cfa28d-bba4-4e59-80a9-55145e2e51f8\") " pod="kserve-ci-e2e-test/llmisvc-model-fb-opt-125m-route-05aa9bba-kserve-prefill-7f2szqt"
Apr 24 15:01:52.045808 ip-10-0-136-132 kubenswrapper[2574]: I0424 15:01:52.045641    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/f0cfa28d-bba4-4e59-80a9-55145e2e51f8-tls-certs\") pod \"llmisvc-model-fb-opt-125m-route-05aa9bba-kserve-prefill-7f2szqt\" (UID: \"f0cfa28d-bba4-4e59-80a9-55145e2e51f8\") " pod="kserve-ci-e2e-test/llmisvc-model-fb-opt-125m-route-05aa9bba-kserve-prefill-7f2szqt"
Apr 24 15:01:52.045808 ip-10-0-136-132 kubenswrapper[2574]: I0424 15:01:52.045745    2574 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/7a613b84-cbb1-4ab8-bc7c-2187ccaae8e6-model-cache\") pod \"llmisvc-model-fb-opt-125m-route-05aa9bba-kserve-568864c59fhvjqg\" (UID: \"7a613b84-cbb1-4ab8-bc7c-2187ccaae8e6\") " pod="kserve-ci-e2e-test/llmisvc-model-fb-opt-125m-route-05aa9bba-kserve-568864c59fhvjqg"
Apr 24 15:01:52.045876 ip-10-0-136-132 kubenswrapper[2574]: I0424 15:01:52.045820    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6k9w6\" (UniqueName: \"kubernetes.io/projected/f0cfa28d-bba4-4e59-80a9-55145e2e51f8-kube-api-access-6k9w6\") pod \"llmisvc-model-fb-opt-125m-route-05aa9bba-kserve-prefill-7f2szqt\" (UID: \"f0cfa28d-bba4-4e59-80a9-55145e2e51f8\") " pod="kserve-ci-e2e-test/llmisvc-model-fb-opt-125m-route-05aa9bba-kserve-prefill-7f2szqt"
Apr 24 15:01:52.045926 ip-10-0-136-132 kubenswrapper[2574]: I0424 15:01:52.045871    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/f0cfa28d-bba4-4e59-80a9-55145e2e51f8-dshm\") pod \"llmisvc-model-fb-opt-125m-route-05aa9bba-kserve-prefill-7f2szqt\" (UID: \"f0cfa28d-bba4-4e59-80a9-55145e2e51f8\") " pod="kserve-ci-e2e-test/llmisvc-model-fb-opt-125m-route-05aa9bba-kserve-prefill-7f2szqt"
Apr 24 15:01:52.046003 ip-10-0-136-132 kubenswrapper[2574]: I0424 15:01:52.045978    2574 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-6dgsc\" (UniqueName: \"kubernetes.io/projected/7a613b84-cbb1-4ab8-bc7c-2187ccaae8e6-kube-api-access-6dgsc\") pod \"llmisvc-model-fb-opt-125m-route-05aa9bba-kserve-568864c59fhvjqg\" (UID: \"7a613b84-cbb1-4ab8-bc7c-2187ccaae8e6\") " pod="kserve-ci-e2e-test/llmisvc-model-fb-opt-125m-route-05aa9bba-kserve-568864c59fhvjqg"
Apr 24 15:01:52.046058 ip-10-0-136-132 kubenswrapper[2574]: I0424 15:01:52.046032    2574 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/7a613b84-cbb1-4ab8-bc7c-2187ccaae8e6-dshm\") pod \"llmisvc-model-fb-opt-125m-route-05aa9bba-kserve-568864c59fhvjqg\" (UID: \"7a613b84-cbb1-4ab8-bc7c-2187ccaae8e6\") " pod="kserve-ci-e2e-test/llmisvc-model-fb-opt-125m-route-05aa9bba-kserve-568864c59fhvjqg"
Apr 24 15:01:52.046119 ip-10-0-136-132 kubenswrapper[2574]: I0424 15:01:52.046074    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/f0cfa28d-bba4-4e59-80a9-55145e2e51f8-home\") pod \"llmisvc-model-fb-opt-125m-route-05aa9bba-kserve-prefill-7f2szqt\" (UID: \"f0cfa28d-bba4-4e59-80a9-55145e2e51f8\") " pod="kserve-ci-e2e-test/llmisvc-model-fb-opt-125m-route-05aa9bba-kserve-prefill-7f2szqt"
Apr 24 15:01:52.046119 ip-10-0-136-132 kubenswrapper[2574]: I0424 15:01:52.046100    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/7a613b84-cbb1-4ab8-bc7c-2187ccaae8e6-model-cache\") pod \"llmisvc-model-fb-opt-125m-route-05aa9bba-kserve-568864c59fhvjqg\" (UID: \"7a613b84-cbb1-4ab8-bc7c-2187ccaae8e6\") " pod="kserve-ci-e2e-test/llmisvc-model-fb-opt-125m-route-05aa9bba-kserve-568864c59fhvjqg"
Apr 24 15:01:52.046119 ip-10-0-136-132 kubenswrapper[2574]: I0424 15:01:52.046108    2574 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/7a613b84-cbb1-4ab8-bc7c-2187ccaae8e6-home\") pod \"llmisvc-model-fb-opt-125m-route-05aa9bba-kserve-568864c59fhvjqg\" (UID: \"7a613b84-cbb1-4ab8-bc7c-2187ccaae8e6\") " pod="kserve-ci-e2e-test/llmisvc-model-fb-opt-125m-route-05aa9bba-kserve-568864c59fhvjqg"
Apr 24 15:01:52.046299 ip-10-0-136-132 kubenswrapper[2574]: I0424 15:01:52.046247    2574 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/7a613b84-cbb1-4ab8-bc7c-2187ccaae8e6-kserve-provision-location\") pod \"llmisvc-model-fb-opt-125m-route-05aa9bba-kserve-568864c59fhvjqg\" (UID: \"7a613b84-cbb1-4ab8-bc7c-2187ccaae8e6\") " pod="kserve-ci-e2e-test/llmisvc-model-fb-opt-125m-route-05aa9bba-kserve-568864c59fhvjqg"
Apr 24 15:01:52.046299 ip-10-0-136-132 kubenswrapper[2574]: I0424 15:01:52.046290    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/f0cfa28d-bba4-4e59-80a9-55145e2e51f8-model-cache\") pod \"llmisvc-model-fb-opt-125m-route-05aa9bba-kserve-prefill-7f2szqt\" (UID: \"f0cfa28d-bba4-4e59-80a9-55145e2e51f8\") " pod="kserve-ci-e2e-test/llmisvc-model-fb-opt-125m-route-05aa9bba-kserve-prefill-7f2szqt"
Apr 24 15:01:52.046476 ip-10-0-136-132 kubenswrapper[2574]: I0424 15:01:52.046451    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/7a613b84-cbb1-4ab8-bc7c-2187ccaae8e6-home\") pod \"llmisvc-model-fb-opt-125m-route-05aa9bba-kserve-568864c59fhvjqg\" (UID: \"7a613b84-cbb1-4ab8-bc7c-2187ccaae8e6\") " pod="kserve-ci-e2e-test/llmisvc-model-fb-opt-125m-route-05aa9bba-kserve-568864c59fhvjqg"
Apr 24 15:01:52.046620 ip-10-0-136-132 kubenswrapper[2574]: I0424 15:01:52.046601    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/7a613b84-cbb1-4ab8-bc7c-2187ccaae8e6-kserve-provision-location\") pod \"llmisvc-model-fb-opt-125m-route-05aa9bba-kserve-568864c59fhvjqg\" (UID: \"7a613b84-cbb1-4ab8-bc7c-2187ccaae8e6\") " pod="kserve-ci-e2e-test/llmisvc-model-fb-opt-125m-route-05aa9bba-kserve-568864c59fhvjqg"
Apr 24 15:01:52.048092 ip-10-0-136-132 kubenswrapper[2574]: I0424 15:01:52.048075    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/7a613b84-cbb1-4ab8-bc7c-2187ccaae8e6-tls-certs\") pod \"llmisvc-model-fb-opt-125m-route-05aa9bba-kserve-568864c59fhvjqg\" (UID: \"7a613b84-cbb1-4ab8-bc7c-2187ccaae8e6\") " pod="kserve-ci-e2e-test/llmisvc-model-fb-opt-125m-route-05aa9bba-kserve-568864c59fhvjqg"
Apr 24 15:01:52.048248 ip-10-0-136-132 kubenswrapper[2574]: I0424 15:01:52.048231    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/7a613b84-cbb1-4ab8-bc7c-2187ccaae8e6-dshm\") pod \"llmisvc-model-fb-opt-125m-route-05aa9bba-kserve-568864c59fhvjqg\" (UID: \"7a613b84-cbb1-4ab8-bc7c-2187ccaae8e6\") " pod="kserve-ci-e2e-test/llmisvc-model-fb-opt-125m-route-05aa9bba-kserve-568864c59fhvjqg"
Apr 24 15:01:52.060968 ip-10-0-136-132 kubenswrapper[2574]: I0424 15:01:52.060937    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-6dgsc\" (UniqueName: \"kubernetes.io/projected/7a613b84-cbb1-4ab8-bc7c-2187ccaae8e6-kube-api-access-6dgsc\") pod \"llmisvc-model-fb-opt-125m-route-05aa9bba-kserve-568864c59fhvjqg\" (UID: \"7a613b84-cbb1-4ab8-bc7c-2187ccaae8e6\") " pod="kserve-ci-e2e-test/llmisvc-model-fb-opt-125m-route-05aa9bba-kserve-568864c59fhvjqg"
Apr 24 15:01:52.147238 ip-10-0-136-132 kubenswrapper[2574]: I0424 15:01:52.147201    2574 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/f0cfa28d-bba4-4e59-80a9-55145e2e51f8-kserve-provision-location\") pod \"llmisvc-model-fb-opt-125m-route-05aa9bba-kserve-prefill-7f2szqt\" (UID: \"f0cfa28d-bba4-4e59-80a9-55145e2e51f8\") " pod="kserve-ci-e2e-test/llmisvc-model-fb-opt-125m-route-05aa9bba-kserve-prefill-7f2szqt"
Apr 24 15:01:52.147444 ip-10-0-136-132 kubenswrapper[2574]: I0424 15:01:52.147246    2574 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/f0cfa28d-bba4-4e59-80a9-55145e2e51f8-tls-certs\") pod \"llmisvc-model-fb-opt-125m-route-05aa9bba-kserve-prefill-7f2szqt\" (UID: \"f0cfa28d-bba4-4e59-80a9-55145e2e51f8\") " pod="kserve-ci-e2e-test/llmisvc-model-fb-opt-125m-route-05aa9bba-kserve-prefill-7f2szqt"
Apr 24 15:01:52.147444 ip-10-0-136-132 kubenswrapper[2574]: I0424 15:01:52.147307    2574 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-6k9w6\" (UniqueName: \"kubernetes.io/projected/f0cfa28d-bba4-4e59-80a9-55145e2e51f8-kube-api-access-6k9w6\") pod \"llmisvc-model-fb-opt-125m-route-05aa9bba-kserve-prefill-7f2szqt\" (UID: \"f0cfa28d-bba4-4e59-80a9-55145e2e51f8\") " pod="kserve-ci-e2e-test/llmisvc-model-fb-opt-125m-route-05aa9bba-kserve-prefill-7f2szqt"
Apr 24 15:01:52.147444 ip-10-0-136-132 kubenswrapper[2574]: I0424 15:01:52.147360    2574 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/f0cfa28d-bba4-4e59-80a9-55145e2e51f8-dshm\") pod \"llmisvc-model-fb-opt-125m-route-05aa9bba-kserve-prefill-7f2szqt\" (UID: \"f0cfa28d-bba4-4e59-80a9-55145e2e51f8\") " pod="kserve-ci-e2e-test/llmisvc-model-fb-opt-125m-route-05aa9bba-kserve-prefill-7f2szqt"
Apr 24 15:01:52.147444 ip-10-0-136-132 kubenswrapper[2574]: I0424 15:01:52.147411    2574 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/f0cfa28d-bba4-4e59-80a9-55145e2e51f8-home\") pod \"llmisvc-model-fb-opt-125m-route-05aa9bba-kserve-prefill-7f2szqt\" (UID: \"f0cfa28d-bba4-4e59-80a9-55145e2e51f8\") " pod="kserve-ci-e2e-test/llmisvc-model-fb-opt-125m-route-05aa9bba-kserve-prefill-7f2szqt"
Apr 24 15:01:52.147679 ip-10-0-136-132 kubenswrapper[2574]: I0424 15:01:52.147457    2574 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/f0cfa28d-bba4-4e59-80a9-55145e2e51f8-model-cache\") pod \"llmisvc-model-fb-opt-125m-route-05aa9bba-kserve-prefill-7f2szqt\" (UID: \"f0cfa28d-bba4-4e59-80a9-55145e2e51f8\") " pod="kserve-ci-e2e-test/llmisvc-model-fb-opt-125m-route-05aa9bba-kserve-prefill-7f2szqt"
Apr 24 15:01:52.147743 ip-10-0-136-132 kubenswrapper[2574]: I0424 15:01:52.147660    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/f0cfa28d-bba4-4e59-80a9-55145e2e51f8-kserve-provision-location\") pod \"llmisvc-model-fb-opt-125m-route-05aa9bba-kserve-prefill-7f2szqt\" (UID: \"f0cfa28d-bba4-4e59-80a9-55145e2e51f8\") " pod="kserve-ci-e2e-test/llmisvc-model-fb-opt-125m-route-05aa9bba-kserve-prefill-7f2szqt"
Apr 24 15:01:52.148024 ip-10-0-136-132 kubenswrapper[2574]: I0424 15:01:52.147977    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/f0cfa28d-bba4-4e59-80a9-55145e2e51f8-home\") pod \"llmisvc-model-fb-opt-125m-route-05aa9bba-kserve-prefill-7f2szqt\" (UID: \"f0cfa28d-bba4-4e59-80a9-55145e2e51f8\") " pod="kserve-ci-e2e-test/llmisvc-model-fb-opt-125m-route-05aa9bba-kserve-prefill-7f2szqt"
Apr 24 15:01:52.148024 ip-10-0-136-132 kubenswrapper[2574]: I0424 15:01:52.148002    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/f0cfa28d-bba4-4e59-80a9-55145e2e51f8-model-cache\") pod \"llmisvc-model-fb-opt-125m-route-05aa9bba-kserve-prefill-7f2szqt\" (UID: \"f0cfa28d-bba4-4e59-80a9-55145e2e51f8\") " pod="kserve-ci-e2e-test/llmisvc-model-fb-opt-125m-route-05aa9bba-kserve-prefill-7f2szqt"
Apr 24 15:01:52.149707 ip-10-0-136-132 kubenswrapper[2574]: I0424 15:01:52.149685    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/f0cfa28d-bba4-4e59-80a9-55145e2e51f8-dshm\") pod \"llmisvc-model-fb-opt-125m-route-05aa9bba-kserve-prefill-7f2szqt\" (UID: \"f0cfa28d-bba4-4e59-80a9-55145e2e51f8\") " pod="kserve-ci-e2e-test/llmisvc-model-fb-opt-125m-route-05aa9bba-kserve-prefill-7f2szqt"
Apr 24 15:01:52.149867 ip-10-0-136-132 kubenswrapper[2574]: I0424 15:01:52.149848    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/f0cfa28d-bba4-4e59-80a9-55145e2e51f8-tls-certs\") pod \"llmisvc-model-fb-opt-125m-route-05aa9bba-kserve-prefill-7f2szqt\" (UID: \"f0cfa28d-bba4-4e59-80a9-55145e2e51f8\") " pod="kserve-ci-e2e-test/llmisvc-model-fb-opt-125m-route-05aa9bba-kserve-prefill-7f2szqt"
Apr 24 15:01:52.154985 ip-10-0-136-132 kubenswrapper[2574]: I0424 15:01:52.154959    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-6k9w6\" (UniqueName: \"kubernetes.io/projected/f0cfa28d-bba4-4e59-80a9-55145e2e51f8-kube-api-access-6k9w6\") pod \"llmisvc-model-fb-opt-125m-route-05aa9bba-kserve-prefill-7f2szqt\" (UID: \"f0cfa28d-bba4-4e59-80a9-55145e2e51f8\") " pod="kserve-ci-e2e-test/llmisvc-model-fb-opt-125m-route-05aa9bba-kserve-prefill-7f2szqt"
Apr 24 15:01:52.161785 ip-10-0-136-132 kubenswrapper[2574]: I0424 15:01:52.161761    2574 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/llmisvc-model-fb-opt-125m-route-05aa9bba-kserve-568864c59fhvjqg"
Apr 24 15:01:52.175640 ip-10-0-136-132 kubenswrapper[2574]: I0424 15:01:52.175617    2574 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/llmisvc-model-fb-opt-125m-route-05aa9bba-kserve-prefill-7f2szqt"
Apr 24 15:01:52.306632 ip-10-0-136-132 kubenswrapper[2574]: I0424 15:01:52.306600    2574 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/llmisvc-model-fb-opt-125m-route-05aa9bba-kserve-568864c59fhvjqg"]
Apr 24 15:01:52.307654 ip-10-0-136-132 kubenswrapper[2574]: W0424 15:01:52.307628    2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7a613b84_cbb1_4ab8_bc7c_2187ccaae8e6.slice/crio-9df4703af3c8822cc5b231522c3f8922fc1116ca91aab7d3b349f16f37c25614 WatchSource:0}: Error finding container 9df4703af3c8822cc5b231522c3f8922fc1116ca91aab7d3b349f16f37c25614: Status 404 returned error can't find the container with id 9df4703af3c8822cc5b231522c3f8922fc1116ca91aab7d3b349f16f37c25614
Apr 24 15:01:52.335060 ip-10-0-136-132 kubenswrapper[2574]: I0424 15:01:52.335038    2574 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/llmisvc-model-fb-opt-125m-route-05aa9bba-kserve-prefill-7f2szqt"]
Apr 24 15:01:52.340525 ip-10-0-136-132 kubenswrapper[2574]: W0424 15:01:52.340500    2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf0cfa28d_bba4_4e59_80a9_55145e2e51f8.slice/crio-edc9eea73ba0dcf5062b1479a879bc99763c5754ec88eddd9f9698a55e250269 WatchSource:0}: Error finding container edc9eea73ba0dcf5062b1479a879bc99763c5754ec88eddd9f9698a55e250269: Status 404 returned error can't find the container with id edc9eea73ba0dcf5062b1479a879bc99763c5754ec88eddd9f9698a55e250269
Apr 24 15:01:52.815428 ip-10-0-136-132 kubenswrapper[2574]: I0424 15:01:52.815389    2574 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/llmisvc-model-fb-opt-125m-route-05aa9bba-kserve-prefill-7f2szqt" event={"ID":"f0cfa28d-bba4-4e59-80a9-55145e2e51f8","Type":"ContainerStarted","Data":"78cffda93bb61a573a2cd6e23085eb66ffc9337828bcafb41da8b3c47a9800dc"}
Apr 24 15:01:52.815428 ip-10-0-136-132 kubenswrapper[2574]: I0424 15:01:52.815432    2574 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/llmisvc-model-fb-opt-125m-route-05aa9bba-kserve-prefill-7f2szqt" event={"ID":"f0cfa28d-bba4-4e59-80a9-55145e2e51f8","Type":"ContainerStarted","Data":"edc9eea73ba0dcf5062b1479a879bc99763c5754ec88eddd9f9698a55e250269"}
Apr 24 15:01:52.817025 ip-10-0-136-132 kubenswrapper[2574]: I0424 15:01:52.816999    2574 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/llmisvc-model-fb-opt-125m-route-05aa9bba-kserve-568864c59fhvjqg" event={"ID":"7a613b84-cbb1-4ab8-bc7c-2187ccaae8e6","Type":"ContainerStarted","Data":"948389d6d5cb2378166ade76619c0e31c44c7062065e4e297f9a3f8e018424d8"}
Apr 24 15:01:52.817025 ip-10-0-136-132 kubenswrapper[2574]: I0424 15:01:52.817027    2574 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/llmisvc-model-fb-opt-125m-route-05aa9bba-kserve-568864c59fhvjqg" event={"ID":"7a613b84-cbb1-4ab8-bc7c-2187ccaae8e6","Type":"ContainerStarted","Data":"9df4703af3c8822cc5b231522c3f8922fc1116ca91aab7d3b349f16f37c25614"}
Apr 24 15:01:52.817219 ip-10-0-136-132 kubenswrapper[2574]: I0424 15:01:52.817071    2574 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve-ci-e2e-test/llmisvc-model-fb-opt-125m-route-05aa9bba-kserve-568864c59fhvjqg"
Apr 24 15:01:53.824026 ip-10-0-136-132 kubenswrapper[2574]: I0424 15:01:53.823987    2574 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/llmisvc-model-fb-opt-125m-route-05aa9bba-kserve-568864c59fhvjqg" event={"ID":"7a613b84-cbb1-4ab8-bc7c-2187ccaae8e6","Type":"ContainerStarted","Data":"e93ba8ed8c23b6f6e25e4148a6f4e457c7e3fb02f9539bf9096b34b9a5911672"}
Apr 24 15:01:54.806281 ip-10-0-136-132 kubenswrapper[2574]: I0424 15:01:54.806237    2574 prober.go:120] "Probe failed" probeType="Startup" pod="kserve-ci-e2e-test/router-with-refs-pd-test-kserve-prefill-5c8777dc-txbgd" podUID="4a48929f-f66c-49b8-9d3d-08c90eb4c1c6" containerName="main" probeResult="failure" output="Get \"https://10.133.0.64:8000/health\": dial tcp 10.133.0.64:8000: connect: connection refused"
Apr 24 15:01:57.843631 ip-10-0-136-132 kubenswrapper[2574]: I0424 15:01:57.843593    2574 generic.go:358] "Generic (PLEG): container finished" podID="7a613b84-cbb1-4ab8-bc7c-2187ccaae8e6" containerID="e93ba8ed8c23b6f6e25e4148a6f4e457c7e3fb02f9539bf9096b34b9a5911672" exitCode=0
Apr 24 15:01:57.844001 ip-10-0-136-132 kubenswrapper[2574]: I0424 15:01:57.843646    2574 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/llmisvc-model-fb-opt-125m-route-05aa9bba-kserve-568864c59fhvjqg" event={"ID":"7a613b84-cbb1-4ab8-bc7c-2187ccaae8e6","Type":"ContainerDied","Data":"e93ba8ed8c23b6f6e25e4148a6f4e457c7e3fb02f9539bf9096b34b9a5911672"}
Apr 24 15:01:58.850851 ip-10-0-136-132 kubenswrapper[2574]: I0424 15:01:58.850808    2574 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/llmisvc-model-fb-opt-125m-route-05aa9bba-kserve-568864c59fhvjqg" event={"ID":"7a613b84-cbb1-4ab8-bc7c-2187ccaae8e6","Type":"ContainerStarted","Data":"af3b02be99dc732bd4d00a99bc32e8944a80c854a921bcf888f85cb26d999cf0"}
Apr 24 15:01:58.875588 ip-10-0-136-132 kubenswrapper[2574]: I0424 15:01:58.875531    2574 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="kserve-ci-e2e-test/llmisvc-model-fb-opt-125m-route-05aa9bba-kserve-568864c59fhvjqg" podStartSLOduration=7.875510919 podStartE2EDuration="7.875510919s" podCreationTimestamp="2026-04-24 15:01:51 +0000 UTC" firstStartedPulling="0001-01-01 00:00:00 +0000 UTC" lastFinishedPulling="0001-01-01 00:00:00 +0000 UTC" observedRunningTime="2026-04-24 15:01:58.872577535 +0000 UTC m=+2281.604679187" watchObservedRunningTime="2026-04-24 15:01:58.875510919 +0000 UTC m=+2281.607612570"
Apr 24 15:02:02.162418 ip-10-0-136-132 kubenswrapper[2574]: I0424 15:02:02.162375    2574 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve-ci-e2e-test/llmisvc-model-fb-opt-125m-route-05aa9bba-kserve-568864c59fhvjqg"
Apr 24 15:02:02.162863 ip-10-0-136-132 kubenswrapper[2574]: I0424 15:02:02.162500    2574 kubelet.go:2658] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="kserve-ci-e2e-test/llmisvc-model-fb-opt-125m-route-05aa9bba-kserve-568864c59fhvjqg"
Apr 24 15:02:02.163875 ip-10-0-136-132 kubenswrapper[2574]: I0424 15:02:02.163838    2574 prober.go:120] "Probe failed" probeType="Startup" pod="kserve-ci-e2e-test/llmisvc-model-fb-opt-125m-route-05aa9bba-kserve-568864c59fhvjqg" podUID="7a613b84-cbb1-4ab8-bc7c-2187ccaae8e6" containerName="main" probeResult="failure" output="Get \"https://10.133.0.65:8001/health\": dial tcp 10.133.0.65:8001: connect: connection refused"
Apr 24 15:02:02.182820 ip-10-0-136-132 kubenswrapper[2574]: I0424 15:02:02.182789    2574 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="kserve-ci-e2e-test/llmisvc-model-fb-opt-125m-route-05aa9bba-kserve-568864c59fhvjqg"
Apr 24 15:02:04.806031 ip-10-0-136-132 kubenswrapper[2574]: I0424 15:02:04.805974    2574 prober.go:120] "Probe failed" probeType="Startup" pod="kserve-ci-e2e-test/router-with-refs-pd-test-kserve-prefill-5c8777dc-txbgd" podUID="4a48929f-f66c-49b8-9d3d-08c90eb4c1c6" containerName="main" probeResult="failure" output="Get \"https://10.133.0.64:8000/health\": dial tcp 10.133.0.64:8000: connect: connection refused"
Apr 24 15:02:12.162241 ip-10-0-136-132 kubenswrapper[2574]: I0424 15:02:12.162189    2574 prober.go:120] "Probe failed" probeType="Startup" pod="kserve-ci-e2e-test/llmisvc-model-fb-opt-125m-route-05aa9bba-kserve-568864c59fhvjqg" podUID="7a613b84-cbb1-4ab8-bc7c-2187ccaae8e6" containerName="main" probeResult="failure" output="Get \"https://10.133.0.65:8001/health\": dial tcp 10.133.0.65:8001: connect: connection refused"
Apr 24 15:02:14.806955 ip-10-0-136-132 kubenswrapper[2574]: I0424 15:02:14.806903    2574 prober.go:120] "Probe failed" probeType="Startup" pod="kserve-ci-e2e-test/router-with-refs-pd-test-kserve-prefill-5c8777dc-txbgd" podUID="4a48929f-f66c-49b8-9d3d-08c90eb4c1c6" containerName="main" probeResult="failure" output="Get \"https://10.133.0.64:8000/health\": dial tcp 10.133.0.64:8000: connect: connection refused"
Apr 24 15:02:22.162386 ip-10-0-136-132 kubenswrapper[2574]: I0424 15:02:22.162309    2574 prober.go:120] "Probe failed" probeType="Startup" pod="kserve-ci-e2e-test/llmisvc-model-fb-opt-125m-route-05aa9bba-kserve-568864c59fhvjqg" podUID="7a613b84-cbb1-4ab8-bc7c-2187ccaae8e6" containerName="main" probeResult="failure" output="Get \"https://10.133.0.65:8001/health\": dial tcp 10.133.0.65:8001: connect: connection refused"
Apr 24 15:02:24.806324 ip-10-0-136-132 kubenswrapper[2574]: I0424 15:02:24.806277    2574 prober.go:120] "Probe failed" probeType="Startup" pod="kserve-ci-e2e-test/router-with-refs-pd-test-kserve-prefill-5c8777dc-txbgd" podUID="4a48929f-f66c-49b8-9d3d-08c90eb4c1c6" containerName="main" probeResult="failure" output="Get \"https://10.133.0.64:8000/health\": dial tcp 10.133.0.64:8000: connect: connection refused"
Apr 24 15:02:32.162727 ip-10-0-136-132 kubenswrapper[2574]: I0424 15:02:32.162686    2574 prober.go:120] "Probe failed" probeType="Startup" pod="kserve-ci-e2e-test/llmisvc-model-fb-opt-125m-route-05aa9bba-kserve-568864c59fhvjqg" podUID="7a613b84-cbb1-4ab8-bc7c-2187ccaae8e6" containerName="main" probeResult="failure" output="Get \"https://10.133.0.65:8001/health\": dial tcp 10.133.0.65:8001: connect: connection refused"
Apr 24 15:02:34.815739 ip-10-0-136-132 kubenswrapper[2574]: I0424 15:02:34.815707    2574 kubelet.go:2658] "SyncLoop (probe)" probe="startup" status="started" pod="kserve-ci-e2e-test/router-with-refs-pd-test-kserve-prefill-5c8777dc-txbgd"
Apr 24 15:02:34.824032 ip-10-0-136-132 kubenswrapper[2574]: I0424 15:02:34.824009    2574 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="kserve-ci-e2e-test/router-with-refs-pd-test-kserve-prefill-5c8777dc-txbgd"
Apr 24 15:02:42.162602 ip-10-0-136-132 kubenswrapper[2574]: I0424 15:02:42.162497    2574 prober.go:120] "Probe failed" probeType="Startup" pod="kserve-ci-e2e-test/llmisvc-model-fb-opt-125m-route-05aa9bba-kserve-568864c59fhvjqg" podUID="7a613b84-cbb1-4ab8-bc7c-2187ccaae8e6" containerName="main" probeResult="failure" output="Get \"https://10.133.0.65:8001/health\": dial tcp 10.133.0.65:8001: connect: connection refused"
Apr 24 15:02:45.984878 ip-10-0-136-132 kubenswrapper[2574]: I0424 15:02:45.984841    2574 kubelet.go:2553] "SyncLoop DELETE" source="api" pods=["kserve-ci-e2e-test/router-with-refs-pd-test-kserve-prefill-5c8777dc-txbgd"]
Apr 24 15:02:45.985301 ip-10-0-136-132 kubenswrapper[2574]: I0424 15:02:45.985128    2574 kuberuntime_container.go:864] "Killing container with a grace period" pod="kserve-ci-e2e-test/router-with-refs-pd-test-kserve-prefill-5c8777dc-txbgd" podUID="4a48929f-f66c-49b8-9d3d-08c90eb4c1c6" containerName="main" containerID="cri-o://a41b6c69d055b8a1a292068cf88530f7ead7ebb18368a93ffe1eb028fb59ae2f" gracePeriod=30
Apr 24 15:02:52.093231 ip-10-0-136-132 kubenswrapper[2574]: I0424 15:02:52.093147    2574 generic.go:358] "Generic (PLEG): container finished" podID="f0cfa28d-bba4-4e59-80a9-55145e2e51f8" containerID="78cffda93bb61a573a2cd6e23085eb66ffc9337828bcafb41da8b3c47a9800dc" exitCode=0
Apr 24 15:02:52.093631 ip-10-0-136-132 kubenswrapper[2574]: I0424 15:02:52.093219    2574 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/llmisvc-model-fb-opt-125m-route-05aa9bba-kserve-prefill-7f2szqt" event={"ID":"f0cfa28d-bba4-4e59-80a9-55145e2e51f8","Type":"ContainerDied","Data":"78cffda93bb61a573a2cd6e23085eb66ffc9337828bcafb41da8b3c47a9800dc"}
Apr 24 15:02:52.163011 ip-10-0-136-132 kubenswrapper[2574]: I0424 15:02:52.162961    2574 prober.go:120] "Probe failed" probeType="Startup" pod="kserve-ci-e2e-test/llmisvc-model-fb-opt-125m-route-05aa9bba-kserve-568864c59fhvjqg" podUID="7a613b84-cbb1-4ab8-bc7c-2187ccaae8e6" containerName="main" probeResult="failure" output="Get \"https://10.133.0.65:8001/health\": dial tcp 10.133.0.65:8001: connect: connection refused"
Apr 24 15:02:53.100006 ip-10-0-136-132 kubenswrapper[2574]: I0424 15:02:53.099972    2574 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/llmisvc-model-fb-opt-125m-route-05aa9bba-kserve-prefill-7f2szqt" event={"ID":"f0cfa28d-bba4-4e59-80a9-55145e2e51f8","Type":"ContainerStarted","Data":"ee9b62b7762d126b08c542347ec2177afb8052b330b3f53eef2db7c490a98efa"}
Apr 24 15:02:53.119564 ip-10-0-136-132 kubenswrapper[2574]: I0424 15:02:53.119515    2574 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="kserve-ci-e2e-test/llmisvc-model-fb-opt-125m-route-05aa9bba-kserve-prefill-7f2szqt" podStartSLOduration=62.119499916 podStartE2EDuration="1m2.119499916s" podCreationTimestamp="2026-04-24 15:01:51 +0000 UTC" firstStartedPulling="0001-01-01 00:00:00 +0000 UTC" lastFinishedPulling="0001-01-01 00:00:00 +0000 UTC" observedRunningTime="2026-04-24 15:02:53.11840611 +0000 UTC m=+2335.850507759" watchObservedRunningTime="2026-04-24 15:02:53.119499916 +0000 UTC m=+2335.851601565"
Apr 24 15:03:02.162256 ip-10-0-136-132 kubenswrapper[2574]: I0424 15:03:02.162203    2574 prober.go:120] "Probe failed" probeType="Startup" pod="kserve-ci-e2e-test/llmisvc-model-fb-opt-125m-route-05aa9bba-kserve-568864c59fhvjqg" podUID="7a613b84-cbb1-4ab8-bc7c-2187ccaae8e6" containerName="main" probeResult="failure" output="Get \"https://10.133.0.65:8001/health\": dial tcp 10.133.0.65:8001: connect: connection refused"
Apr 24 15:03:02.176037 ip-10-0-136-132 kubenswrapper[2574]: I0424 15:03:02.176001    2574 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve-ci-e2e-test/llmisvc-model-fb-opt-125m-route-05aa9bba-kserve-prefill-7f2szqt"
Apr 24 15:03:02.176231 ip-10-0-136-132 kubenswrapper[2574]: I0424 15:03:02.176049    2574 kubelet.go:2658] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="kserve-ci-e2e-test/llmisvc-model-fb-opt-125m-route-05aa9bba-kserve-prefill-7f2szqt"
Apr 24 15:03:02.177491 ip-10-0-136-132 kubenswrapper[2574]: I0424 15:03:02.177464    2574 prober.go:120] "Probe failed" probeType="Startup" pod="kserve-ci-e2e-test/llmisvc-model-fb-opt-125m-route-05aa9bba-kserve-prefill-7f2szqt" podUID="f0cfa28d-bba4-4e59-80a9-55145e2e51f8" containerName="main" probeResult="failure" output="Get \"https://10.133.0.66:8000/health\": dial tcp 10.133.0.66:8000: connect: connection refused"
Apr 24 15:03:12.163070 ip-10-0-136-132 kubenswrapper[2574]: I0424 15:03:12.163017    2574 prober.go:120] "Probe failed" probeType="Startup" pod="kserve-ci-e2e-test/llmisvc-model-fb-opt-125m-route-05aa9bba-kserve-568864c59fhvjqg" podUID="7a613b84-cbb1-4ab8-bc7c-2187ccaae8e6" containerName="main" probeResult="failure" output="Get \"https://10.133.0.65:8001/health\": dial tcp 10.133.0.65:8001: connect: connection refused"
Apr 24 15:03:12.175965 ip-10-0-136-132 kubenswrapper[2574]: I0424 15:03:12.175929    2574 prober.go:120] "Probe failed" probeType="Startup" pod="kserve-ci-e2e-test/llmisvc-model-fb-opt-125m-route-05aa9bba-kserve-prefill-7f2szqt" podUID="f0cfa28d-bba4-4e59-80a9-55145e2e51f8" containerName="main" probeResult="failure" output="Get \"https://10.133.0.66:8000/health\": dial tcp 10.133.0.66:8000: connect: connection refused"
Apr 24 15:03:16.208613 ip-10-0-136-132 kubenswrapper[2574]: I0424 15:03:16.208573    2574 generic.go:358] "Generic (PLEG): container finished" podID="4a48929f-f66c-49b8-9d3d-08c90eb4c1c6" containerID="a41b6c69d055b8a1a292068cf88530f7ead7ebb18368a93ffe1eb028fb59ae2f" exitCode=137
Apr 24 15:03:16.208967 ip-10-0-136-132 kubenswrapper[2574]: I0424 15:03:16.208731    2574 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/router-with-refs-pd-test-kserve-prefill-5c8777dc-txbgd" event={"ID":"4a48929f-f66c-49b8-9d3d-08c90eb4c1c6","Type":"ContainerDied","Data":"a41b6c69d055b8a1a292068cf88530f7ead7ebb18368a93ffe1eb028fb59ae2f"}
Apr 24 15:03:16.261436 ip-10-0-136-132 kubenswrapper[2574]: I0424 15:03:16.261407    2574 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/router-with-refs-pd-test-kserve-prefill-5c8777dc-txbgd"
Apr 24 15:03:16.371465 ip-10-0-136-132 kubenswrapper[2574]: I0424 15:03:16.371425    2574 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/4a48929f-f66c-49b8-9d3d-08c90eb4c1c6-dshm\") pod \"4a48929f-f66c-49b8-9d3d-08c90eb4c1c6\" (UID: \"4a48929f-f66c-49b8-9d3d-08c90eb4c1c6\") "
Apr 24 15:03:16.371647 ip-10-0-136-132 kubenswrapper[2574]: I0424 15:03:16.371483    2574 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"kube-api-access-n8xmr\" (UniqueName: \"kubernetes.io/projected/4a48929f-f66c-49b8-9d3d-08c90eb4c1c6-kube-api-access-n8xmr\") pod \"4a48929f-f66c-49b8-9d3d-08c90eb4c1c6\" (UID: \"4a48929f-f66c-49b8-9d3d-08c90eb4c1c6\") "
Apr 24 15:03:16.371647 ip-10-0-136-132 kubenswrapper[2574]: I0424 15:03:16.371530    2574 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/4a48929f-f66c-49b8-9d3d-08c90eb4c1c6-model-cache\") pod \"4a48929f-f66c-49b8-9d3d-08c90eb4c1c6\" (UID: \"4a48929f-f66c-49b8-9d3d-08c90eb4c1c6\") "
Apr 24 15:03:16.371647 ip-10-0-136-132 kubenswrapper[2574]: I0424 15:03:16.371581    2574 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/4a48929f-f66c-49b8-9d3d-08c90eb4c1c6-kserve-provision-location\") pod \"4a48929f-f66c-49b8-9d3d-08c90eb4c1c6\" (UID: \"4a48929f-f66c-49b8-9d3d-08c90eb4c1c6\") "
Apr 24 15:03:16.371647 ip-10-0-136-132 kubenswrapper[2574]: I0424 15:03:16.371618    2574 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/4a48929f-f66c-49b8-9d3d-08c90eb4c1c6-home\") pod \"4a48929f-f66c-49b8-9d3d-08c90eb4c1c6\" (UID: \"4a48929f-f66c-49b8-9d3d-08c90eb4c1c6\") "
Apr 24 15:03:16.371867 ip-10-0-136-132 kubenswrapper[2574]: I0424 15:03:16.371666    2574 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/4a48929f-f66c-49b8-9d3d-08c90eb4c1c6-tls-certs\") pod \"4a48929f-f66c-49b8-9d3d-08c90eb4c1c6\" (UID: \"4a48929f-f66c-49b8-9d3d-08c90eb4c1c6\") "
Apr 24 15:03:16.372017 ip-10-0-136-132 kubenswrapper[2574]: I0424 15:03:16.371963    2574 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/4a48929f-f66c-49b8-9d3d-08c90eb4c1c6-model-cache" (OuterVolumeSpecName: "model-cache") pod "4a48929f-f66c-49b8-9d3d-08c90eb4c1c6" (UID: "4a48929f-f66c-49b8-9d3d-08c90eb4c1c6"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Apr 24 15:03:16.372382 ip-10-0-136-132 kubenswrapper[2574]: I0424 15:03:16.372358    2574 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/4a48929f-f66c-49b8-9d3d-08c90eb4c1c6-home" (OuterVolumeSpecName: "home") pod "4a48929f-f66c-49b8-9d3d-08c90eb4c1c6" (UID: "4a48929f-f66c-49b8-9d3d-08c90eb4c1c6"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Apr 24 15:03:16.374445 ip-10-0-136-132 kubenswrapper[2574]: I0424 15:03:16.374278    2574 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/4a48929f-f66c-49b8-9d3d-08c90eb4c1c6-dshm" (OuterVolumeSpecName: "dshm") pod "4a48929f-f66c-49b8-9d3d-08c90eb4c1c6" (UID: "4a48929f-f66c-49b8-9d3d-08c90eb4c1c6"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Apr 24 15:03:16.374564 ip-10-0-136-132 kubenswrapper[2574]: I0424 15:03:16.374486    2574 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/4a48929f-f66c-49b8-9d3d-08c90eb4c1c6-kube-api-access-n8xmr" (OuterVolumeSpecName: "kube-api-access-n8xmr") pod "4a48929f-f66c-49b8-9d3d-08c90eb4c1c6" (UID: "4a48929f-f66c-49b8-9d3d-08c90eb4c1c6"). InnerVolumeSpecName "kube-api-access-n8xmr". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Apr 24 15:03:16.374564 ip-10-0-136-132 kubenswrapper[2574]: I0424 15:03:16.374492    2574 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/4a48929f-f66c-49b8-9d3d-08c90eb4c1c6-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "4a48929f-f66c-49b8-9d3d-08c90eb4c1c6" (UID: "4a48929f-f66c-49b8-9d3d-08c90eb4c1c6"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Apr 24 15:03:16.439849 ip-10-0-136-132 kubenswrapper[2574]: I0424 15:03:16.439792    2574 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/4a48929f-f66c-49b8-9d3d-08c90eb4c1c6-kserve-provision-location" (OuterVolumeSpecName: "kserve-provision-location") pod "4a48929f-f66c-49b8-9d3d-08c90eb4c1c6" (UID: "4a48929f-f66c-49b8-9d3d-08c90eb4c1c6"). InnerVolumeSpecName "kserve-provision-location". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Apr 24 15:03:16.472826 ip-10-0-136-132 kubenswrapper[2574]: I0424 15:03:16.472792    2574 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/4a48929f-f66c-49b8-9d3d-08c90eb4c1c6-tls-certs\") on node \"ip-10-0-136-132.ec2.internal\" DevicePath \"\""
Apr 24 15:03:16.472826 ip-10-0-136-132 kubenswrapper[2574]: I0424 15:03:16.472828    2574 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/4a48929f-f66c-49b8-9d3d-08c90eb4c1c6-dshm\") on node \"ip-10-0-136-132.ec2.internal\" DevicePath \"\""
Apr 24 15:03:16.473007 ip-10-0-136-132 kubenswrapper[2574]: I0424 15:03:16.472840    2574 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-n8xmr\" (UniqueName: \"kubernetes.io/projected/4a48929f-f66c-49b8-9d3d-08c90eb4c1c6-kube-api-access-n8xmr\") on node \"ip-10-0-136-132.ec2.internal\" DevicePath \"\""
Apr 24 15:03:16.473007 ip-10-0-136-132 kubenswrapper[2574]: I0424 15:03:16.472854    2574 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/4a48929f-f66c-49b8-9d3d-08c90eb4c1c6-model-cache\") on node \"ip-10-0-136-132.ec2.internal\" DevicePath \"\""
Apr 24 15:03:16.473007 ip-10-0-136-132 kubenswrapper[2574]: I0424 15:03:16.472866    2574 reconciler_common.go:299] "Volume detached for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/4a48929f-f66c-49b8-9d3d-08c90eb4c1c6-kserve-provision-location\") on node \"ip-10-0-136-132.ec2.internal\" DevicePath \"\""
Apr 24 15:03:16.473007 ip-10-0-136-132 kubenswrapper[2574]: I0424 15:03:16.472877    2574 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/4a48929f-f66c-49b8-9d3d-08c90eb4c1c6-home\") on node \"ip-10-0-136-132.ec2.internal\" DevicePath \"\""
Apr 24 15:03:17.215161 ip-10-0-136-132 kubenswrapper[2574]: I0424 15:03:17.215127    2574 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/router-with-refs-pd-test-kserve-prefill-5c8777dc-txbgd"
Apr 24 15:03:17.215161 ip-10-0-136-132 kubenswrapper[2574]: I0424 15:03:17.215132    2574 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/router-with-refs-pd-test-kserve-prefill-5c8777dc-txbgd" event={"ID":"4a48929f-f66c-49b8-9d3d-08c90eb4c1c6","Type":"ContainerDied","Data":"170fca5a493884c06dc89ada3d355086add8d5927127df6a4dce2eaa8b16011f"}
Apr 24 15:03:17.215691 ip-10-0-136-132 kubenswrapper[2574]: I0424 15:03:17.215196    2574 scope.go:117] "RemoveContainer" containerID="a41b6c69d055b8a1a292068cf88530f7ead7ebb18368a93ffe1eb028fb59ae2f"
Apr 24 15:03:17.236817 ip-10-0-136-132 kubenswrapper[2574]: I0424 15:03:17.236793    2574 scope.go:117] "RemoveContainer" containerID="7c8769c79147fa7ef6d67b1691039909eb37177cdc54807c7940df4d4b6ea209"
Apr 24 15:03:17.239954 ip-10-0-136-132 kubenswrapper[2574]: I0424 15:03:17.239929    2574 kubelet.go:2553] "SyncLoop DELETE" source="api" pods=["kserve-ci-e2e-test/router-with-refs-pd-test-kserve-prefill-5c8777dc-txbgd"]
Apr 24 15:03:17.247395 ip-10-0-136-132 kubenswrapper[2574]: I0424 15:03:17.247367    2574 kubelet.go:2547] "SyncLoop REMOVE" source="api" pods=["kserve-ci-e2e-test/router-with-refs-pd-test-kserve-prefill-5c8777dc-txbgd"]
Apr 24 15:03:17.771314 ip-10-0-136-132 kubenswrapper[2574]: I0424 15:03:17.771277    2574 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="4a48929f-f66c-49b8-9d3d-08c90eb4c1c6" path="/var/lib/kubelet/pods/4a48929f-f66c-49b8-9d3d-08c90eb4c1c6/volumes"
Apr 24 15:03:22.162776 ip-10-0-136-132 kubenswrapper[2574]: I0424 15:03:22.162732    2574 prober.go:120] "Probe failed" probeType="Startup" pod="kserve-ci-e2e-test/llmisvc-model-fb-opt-125m-route-05aa9bba-kserve-568864c59fhvjqg" podUID="7a613b84-cbb1-4ab8-bc7c-2187ccaae8e6" containerName="main" probeResult="failure" output="Get \"https://10.133.0.65:8001/health\": dial tcp 10.133.0.65:8001: connect: connection refused"
Apr 24 15:03:22.176384 ip-10-0-136-132 kubenswrapper[2574]: I0424 15:03:22.176351    2574 prober.go:120] "Probe failed" probeType="Startup" pod="kserve-ci-e2e-test/llmisvc-model-fb-opt-125m-route-05aa9bba-kserve-prefill-7f2szqt" podUID="f0cfa28d-bba4-4e59-80a9-55145e2e51f8" containerName="main" probeResult="failure" output="Get \"https://10.133.0.66:8000/health\": dial tcp 10.133.0.66:8000: connect: connection refused"
Apr 24 15:03:32.177381 ip-10-0-136-132 kubenswrapper[2574]: I0424 15:03:32.177314    2574 prober.go:120] "Probe failed" probeType="Startup" pod="kserve-ci-e2e-test/llmisvc-model-fb-opt-125m-route-05aa9bba-kserve-prefill-7f2szqt" podUID="f0cfa28d-bba4-4e59-80a9-55145e2e51f8" containerName="main" probeResult="failure" output="Get \"https://10.133.0.66:8000/health\": dial tcp 10.133.0.66:8000: connect: connection refused"
Apr 24 15:03:32.179678 ip-10-0-136-132 kubenswrapper[2574]: I0424 15:03:32.179647    2574 kubelet.go:2658] "SyncLoop (probe)" probe="startup" status="started" pod="kserve-ci-e2e-test/llmisvc-model-fb-opt-125m-route-05aa9bba-kserve-568864c59fhvjqg"
Apr 24 15:03:32.192900 ip-10-0-136-132 kubenswrapper[2574]: I0424 15:03:32.192874    2574 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="kserve-ci-e2e-test/llmisvc-model-fb-opt-125m-route-05aa9bba-kserve-568864c59fhvjqg"
Apr 24 15:03:42.176982 ip-10-0-136-132 kubenswrapper[2574]: I0424 15:03:42.176931    2574 prober.go:120] "Probe failed" probeType="Startup" pod="kserve-ci-e2e-test/llmisvc-model-fb-opt-125m-route-05aa9bba-kserve-prefill-7f2szqt" podUID="f0cfa28d-bba4-4e59-80a9-55145e2e51f8" containerName="main" probeResult="failure" output="Get \"https://10.133.0.66:8000/health\": dial tcp 10.133.0.66:8000: connect: connection refused"
Apr 24 15:03:52.176581 ip-10-0-136-132 kubenswrapper[2574]: I0424 15:03:52.176534    2574 prober.go:120] "Probe failed" probeType="Startup" pod="kserve-ci-e2e-test/llmisvc-model-fb-opt-125m-route-05aa9bba-kserve-prefill-7f2szqt" podUID="f0cfa28d-bba4-4e59-80a9-55145e2e51f8" containerName="main" probeResult="failure" output="Get \"https://10.133.0.66:8000/health\": dial tcp 10.133.0.66:8000: connect: connection refused"
Apr 24 15:03:57.958581 ip-10-0-136-132 kubenswrapper[2574]: I0424 15:03:57.958553    2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-9d4b6777b-f6ngr_2571829f-666a-422a-9b02-2d2bb4c89b25/console-operator/1.log"
Apr 24 15:03:57.963530 ip-10-0-136-132 kubenswrapper[2574]: I0424 15:03:57.963507    2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-w5x5x_44036c3e-1fdc-40d4-899c-9009cd4589e3/ovn-acl-logging/0.log"
Apr 24 15:03:57.964384 ip-10-0-136-132 kubenswrapper[2574]: I0424 15:03:57.964360    2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-9d4b6777b-f6ngr_2571829f-666a-422a-9b02-2d2bb4c89b25/console-operator/1.log"
Apr 24 15:03:57.969254 ip-10-0-136-132 kubenswrapper[2574]: I0424 15:03:57.969234    2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-w5x5x_44036c3e-1fdc-40d4-899c-9009cd4589e3/ovn-acl-logging/0.log"
Apr 24 15:04:02.176412 ip-10-0-136-132 kubenswrapper[2574]: I0424 15:04:02.176354    2574 prober.go:120] "Probe failed" probeType="Startup" pod="kserve-ci-e2e-test/llmisvc-model-fb-opt-125m-route-05aa9bba-kserve-prefill-7f2szqt" podUID="f0cfa28d-bba4-4e59-80a9-55145e2e51f8" containerName="main" probeResult="failure" output="Get \"https://10.133.0.66:8000/health\": dial tcp 10.133.0.66:8000: connect: connection refused"
Apr 24 15:04:12.176817 ip-10-0-136-132 kubenswrapper[2574]: I0424 15:04:12.176710    2574 prober.go:120] "Probe failed" probeType="Startup" pod="kserve-ci-e2e-test/llmisvc-model-fb-opt-125m-route-05aa9bba-kserve-prefill-7f2szqt" podUID="f0cfa28d-bba4-4e59-80a9-55145e2e51f8" containerName="main" probeResult="failure" output="Get \"https://10.133.0.66:8000/health\": dial tcp 10.133.0.66:8000: connect: connection refused"
Apr 24 15:04:22.177163 ip-10-0-136-132 kubenswrapper[2574]: I0424 15:04:22.177112    2574 prober.go:120] "Probe failed" probeType="Startup" pod="kserve-ci-e2e-test/llmisvc-model-fb-opt-125m-route-05aa9bba-kserve-prefill-7f2szqt" podUID="f0cfa28d-bba4-4e59-80a9-55145e2e51f8" containerName="main" probeResult="failure" output="Get \"https://10.133.0.66:8000/health\": dial tcp 10.133.0.66:8000: connect: connection refused"
Apr 24 15:04:32.185847 ip-10-0-136-132 kubenswrapper[2574]: I0424 15:04:32.185811    2574 kubelet.go:2658] "SyncLoop (probe)" probe="startup" status="started" pod="kserve-ci-e2e-test/llmisvc-model-fb-opt-125m-route-05aa9bba-kserve-prefill-7f2szqt"
Apr 24 15:04:32.194480 ip-10-0-136-132 kubenswrapper[2574]: I0424 15:04:32.194452    2574 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="kserve-ci-e2e-test/llmisvc-model-fb-opt-125m-route-05aa9bba-kserve-prefill-7f2szqt"
Apr 24 15:04:43.586160 ip-10-0-136-132 kubenswrapper[2574]: I0424 15:04:43.586114    2574 kubelet.go:2553] "SyncLoop DELETE" source="api" pods=["kserve-ci-e2e-test/llmisvc-model-fb-opt-125m-route-05aa9bba-kserve-568864c59fhvjqg"]
Apr 24 15:04:43.589920 ip-10-0-136-132 kubenswrapper[2574]: I0424 15:04:43.586648    2574 kuberuntime_container.go:864] "Killing container with a grace period" pod="kserve-ci-e2e-test/llmisvc-model-fb-opt-125m-route-05aa9bba-kserve-568864c59fhvjqg" podUID="7a613b84-cbb1-4ab8-bc7c-2187ccaae8e6" containerName="main" containerID="cri-o://af3b02be99dc732bd4d00a99bc32e8944a80c854a921bcf888f85cb26d999cf0" gracePeriod=30
Apr 24 15:04:43.589920 ip-10-0-136-132 kubenswrapper[2574]: I0424 15:04:43.589128    2574 kubelet.go:2553] "SyncLoop DELETE" source="api" pods=["kserve-ci-e2e-test/llmisvc-model-fb-opt-125m-route-05aa9bba-kserve-prefill-7f2szqt"]
Apr 24 15:04:43.589920 ip-10-0-136-132 kubenswrapper[2574]: I0424 15:04:43.589425    2574 kuberuntime_container.go:864] "Killing container with a grace period" pod="kserve-ci-e2e-test/llmisvc-model-fb-opt-125m-route-05aa9bba-kserve-prefill-7f2szqt" podUID="f0cfa28d-bba4-4e59-80a9-55145e2e51f8" containerName="main" containerID="cri-o://ee9b62b7762d126b08c542347ec2177afb8052b330b3f53eef2db7c490a98efa" gracePeriod=30
Apr 24 15:04:58.189717 ip-10-0-136-132 kubenswrapper[2574]: I0424 15:04:58.189676    2574 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_llmisvc-model-fb-opt-125m-route-05aa9bba-kserve-568864c59fhvjqg_7a613b84-cbb1-4ab8-bc7c-2187ccaae8e6/main/0.log"
Apr 24 15:04:58.204486 ip-10-0-136-132 kubenswrapper[2574]: I0424 15:04:58.204462    2574 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_llmisvc-model-fb-opt-125m-route-05aa9bba-kserve-568864c59fhvjqg_7a613b84-cbb1-4ab8-bc7c-2187ccaae8e6/llm-d-routing-sidecar/0.log"
Apr 24 15:04:58.212550 ip-10-0-136-132 kubenswrapper[2574]: I0424 15:04:58.212524    2574 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_llmisvc-model-fb-opt-125m-route-05aa9bba-kserve-568864c59fhvjqg_7a613b84-cbb1-4ab8-bc7c-2187ccaae8e6/storage-initializer/0.log"
Apr 24 15:04:58.226671 ip-10-0-136-132 kubenswrapper[2574]: I0424 15:04:58.226649    2574 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_llmisvc-model-fb-opt-125m-route-05aa9bba-kserve-prefill-7f2szqt_f0cfa28d-bba4-4e59-80a9-55145e2e51f8/main/0.log"
Apr 24 15:04:58.232836 ip-10-0-136-132 kubenswrapper[2574]: I0424 15:04:58.232818    2574 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_llmisvc-model-fb-opt-125m-route-05aa9bba-kserve-prefill-7f2szqt_f0cfa28d-bba4-4e59-80a9-55145e2e51f8/storage-initializer/0.log"
Apr 24 15:04:59.214250 ip-10-0-136-132 kubenswrapper[2574]: I0424 15:04:59.214223    2574 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_llmisvc-model-fb-opt-125m-route-05aa9bba-kserve-568864c59fhvjqg_7a613b84-cbb1-4ab8-bc7c-2187ccaae8e6/main/0.log"
Apr 24 15:04:59.226587 ip-10-0-136-132 kubenswrapper[2574]: I0424 15:04:59.226564    2574 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_llmisvc-model-fb-opt-125m-route-05aa9bba-kserve-568864c59fhvjqg_7a613b84-cbb1-4ab8-bc7c-2187ccaae8e6/llm-d-routing-sidecar/0.log"
Apr 24 15:04:59.234486 ip-10-0-136-132 kubenswrapper[2574]: I0424 15:04:59.234467    2574 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_llmisvc-model-fb-opt-125m-route-05aa9bba-kserve-568864c59fhvjqg_7a613b84-cbb1-4ab8-bc7c-2187ccaae8e6/storage-initializer/0.log"
Apr 24 15:04:59.249525 ip-10-0-136-132 kubenswrapper[2574]: I0424 15:04:59.249504    2574 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_llmisvc-model-fb-opt-125m-route-05aa9bba-kserve-prefill-7f2szqt_f0cfa28d-bba4-4e59-80a9-55145e2e51f8/main/0.log"
Apr 24 15:04:59.263542 ip-10-0-136-132 kubenswrapper[2574]: I0424 15:04:59.263525    2574 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_llmisvc-model-fb-opt-125m-route-05aa9bba-kserve-prefill-7f2szqt_f0cfa28d-bba4-4e59-80a9-55145e2e51f8/storage-initializer/0.log"
Apr 24 15:05:00.186152 ip-10-0-136-132 kubenswrapper[2574]: I0424 15:05:00.186125    2574 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_llmisvc-model-fb-opt-125m-route-05aa9bba-kserve-568864c59fhvjqg_7a613b84-cbb1-4ab8-bc7c-2187ccaae8e6/main/0.log"
Apr 24 15:05:00.191295 ip-10-0-136-132 kubenswrapper[2574]: I0424 15:05:00.191271    2574 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_llmisvc-model-fb-opt-125m-route-05aa9bba-kserve-568864c59fhvjqg_7a613b84-cbb1-4ab8-bc7c-2187ccaae8e6/llm-d-routing-sidecar/0.log"
Apr 24 15:05:00.199082 ip-10-0-136-132 kubenswrapper[2574]: I0424 15:05:00.199063    2574 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_llmisvc-model-fb-opt-125m-route-05aa9bba-kserve-568864c59fhvjqg_7a613b84-cbb1-4ab8-bc7c-2187ccaae8e6/storage-initializer/0.log"
Apr 24 15:05:00.214832 ip-10-0-136-132 kubenswrapper[2574]: I0424 15:05:00.214813    2574 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_llmisvc-model-fb-opt-125m-route-05aa9bba-kserve-prefill-7f2szqt_f0cfa28d-bba4-4e59-80a9-55145e2e51f8/main/0.log"
Apr 24 15:05:00.220689 ip-10-0-136-132 kubenswrapper[2574]: I0424 15:05:00.220670    2574 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_llmisvc-model-fb-opt-125m-route-05aa9bba-kserve-prefill-7f2szqt_f0cfa28d-bba4-4e59-80a9-55145e2e51f8/storage-initializer/0.log"
Apr 24 15:05:01.116289 ip-10-0-136-132 kubenswrapper[2574]: I0424 15:05:01.116246    2574 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_llmisvc-model-fb-opt-125m-route-05aa9bba-kserve-568864c59fhvjqg_7a613b84-cbb1-4ab8-bc7c-2187ccaae8e6/main/0.log"
Apr 24 15:05:01.121589 ip-10-0-136-132 kubenswrapper[2574]: I0424 15:05:01.121568    2574 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_llmisvc-model-fb-opt-125m-route-05aa9bba-kserve-568864c59fhvjqg_7a613b84-cbb1-4ab8-bc7c-2187ccaae8e6/llm-d-routing-sidecar/0.log"
Apr 24 15:05:01.129778 ip-10-0-136-132 kubenswrapper[2574]: I0424 15:05:01.129754    2574 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_llmisvc-model-fb-opt-125m-route-05aa9bba-kserve-568864c59fhvjqg_7a613b84-cbb1-4ab8-bc7c-2187ccaae8e6/storage-initializer/0.log"
Apr 24 15:05:01.145117 ip-10-0-136-132 kubenswrapper[2574]: I0424 15:05:01.145097    2574 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_llmisvc-model-fb-opt-125m-route-05aa9bba-kserve-prefill-7f2szqt_f0cfa28d-bba4-4e59-80a9-55145e2e51f8/main/0.log"
Apr 24 15:05:01.152671 ip-10-0-136-132 kubenswrapper[2574]: I0424 15:05:01.152650    2574 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_llmisvc-model-fb-opt-125m-route-05aa9bba-kserve-prefill-7f2szqt_f0cfa28d-bba4-4e59-80a9-55145e2e51f8/storage-initializer/0.log"
Apr 24 15:05:02.059450 ip-10-0-136-132 kubenswrapper[2574]: I0424 15:05:02.059419    2574 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_llmisvc-model-fb-opt-125m-route-05aa9bba-kserve-568864c59fhvjqg_7a613b84-cbb1-4ab8-bc7c-2187ccaae8e6/main/0.log"
Apr 24 15:05:02.064927 ip-10-0-136-132 kubenswrapper[2574]: I0424 15:05:02.064908    2574 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_llmisvc-model-fb-opt-125m-route-05aa9bba-kserve-568864c59fhvjqg_7a613b84-cbb1-4ab8-bc7c-2187ccaae8e6/llm-d-routing-sidecar/0.log"
Apr 24 15:05:02.073584 ip-10-0-136-132 kubenswrapper[2574]: I0424 15:05:02.073568    2574 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_llmisvc-model-fb-opt-125m-route-05aa9bba-kserve-568864c59fhvjqg_7a613b84-cbb1-4ab8-bc7c-2187ccaae8e6/storage-initializer/0.log"
Apr 24 15:05:02.088919 ip-10-0-136-132 kubenswrapper[2574]: I0424 15:05:02.088898    2574 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_llmisvc-model-fb-opt-125m-route-05aa9bba-kserve-prefill-7f2szqt_f0cfa28d-bba4-4e59-80a9-55145e2e51f8/main/0.log"
Apr 24 15:05:02.094959 ip-10-0-136-132 kubenswrapper[2574]: I0424 15:05:02.094942    2574 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_llmisvc-model-fb-opt-125m-route-05aa9bba-kserve-prefill-7f2szqt_f0cfa28d-bba4-4e59-80a9-55145e2e51f8/storage-initializer/0.log"
Apr 24 15:05:02.999363 ip-10-0-136-132 kubenswrapper[2574]: I0424 15:05:02.999321    2574 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_llmisvc-model-fb-opt-125m-route-05aa9bba-kserve-568864c59fhvjqg_7a613b84-cbb1-4ab8-bc7c-2187ccaae8e6/main/0.log"
Apr 24 15:05:03.005223 ip-10-0-136-132 kubenswrapper[2574]: I0424 15:05:03.005197    2574 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_llmisvc-model-fb-opt-125m-route-05aa9bba-kserve-568864c59fhvjqg_7a613b84-cbb1-4ab8-bc7c-2187ccaae8e6/llm-d-routing-sidecar/0.log"
Apr 24 15:05:03.014199 ip-10-0-136-132 kubenswrapper[2574]: I0424 15:05:03.014179    2574 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_llmisvc-model-fb-opt-125m-route-05aa9bba-kserve-568864c59fhvjqg_7a613b84-cbb1-4ab8-bc7c-2187ccaae8e6/storage-initializer/0.log"
Apr 24 15:05:03.028997 ip-10-0-136-132 kubenswrapper[2574]: I0424 15:05:03.028979    2574 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_llmisvc-model-fb-opt-125m-route-05aa9bba-kserve-prefill-7f2szqt_f0cfa28d-bba4-4e59-80a9-55145e2e51f8/main/0.log"
Apr 24 15:05:03.036314 ip-10-0-136-132 kubenswrapper[2574]: I0424 15:05:03.036291    2574 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_llmisvc-model-fb-opt-125m-route-05aa9bba-kserve-prefill-7f2szqt_f0cfa28d-bba4-4e59-80a9-55145e2e51f8/storage-initializer/0.log"
Apr 24 15:05:03.938998 ip-10-0-136-132 kubenswrapper[2574]: I0424 15:05:03.938971    2574 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_llmisvc-model-fb-opt-125m-route-05aa9bba-kserve-568864c59fhvjqg_7a613b84-cbb1-4ab8-bc7c-2187ccaae8e6/main/0.log"
Apr 24 15:05:03.945033 ip-10-0-136-132 kubenswrapper[2574]: I0424 15:05:03.945007    2574 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_llmisvc-model-fb-opt-125m-route-05aa9bba-kserve-568864c59fhvjqg_7a613b84-cbb1-4ab8-bc7c-2187ccaae8e6/llm-d-routing-sidecar/0.log"
Apr 24 15:05:03.954653 ip-10-0-136-132 kubenswrapper[2574]: I0424 15:05:03.954635    2574 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_llmisvc-model-fb-opt-125m-route-05aa9bba-kserve-568864c59fhvjqg_7a613b84-cbb1-4ab8-bc7c-2187ccaae8e6/storage-initializer/0.log"
Apr 24 15:05:03.971152 ip-10-0-136-132 kubenswrapper[2574]: I0424 15:05:03.971133    2574 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_llmisvc-model-fb-opt-125m-route-05aa9bba-kserve-prefill-7f2szqt_f0cfa28d-bba4-4e59-80a9-55145e2e51f8/main/0.log"
Apr 24 15:05:03.978950 ip-10-0-136-132 kubenswrapper[2574]: I0424 15:05:03.978935    2574 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_llmisvc-model-fb-opt-125m-route-05aa9bba-kserve-prefill-7f2szqt_f0cfa28d-bba4-4e59-80a9-55145e2e51f8/storage-initializer/0.log"
Apr 24 15:05:04.880243 ip-10-0-136-132 kubenswrapper[2574]: I0424 15:05:04.880202    2574 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_llmisvc-model-fb-opt-125m-route-05aa9bba-kserve-568864c59fhvjqg_7a613b84-cbb1-4ab8-bc7c-2187ccaae8e6/main/0.log"
Apr 24 15:05:04.885590 ip-10-0-136-132 kubenswrapper[2574]: I0424 15:05:04.885569    2574 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_llmisvc-model-fb-opt-125m-route-05aa9bba-kserve-568864c59fhvjqg_7a613b84-cbb1-4ab8-bc7c-2187ccaae8e6/llm-d-routing-sidecar/0.log"
Apr 24 15:05:04.894791 ip-10-0-136-132 kubenswrapper[2574]: I0424 15:05:04.894776    2574 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_llmisvc-model-fb-opt-125m-route-05aa9bba-kserve-568864c59fhvjqg_7a613b84-cbb1-4ab8-bc7c-2187ccaae8e6/storage-initializer/0.log"
Apr 24 15:05:04.908776 ip-10-0-136-132 kubenswrapper[2574]: I0424 15:05:04.908759    2574 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_llmisvc-model-fb-opt-125m-route-05aa9bba-kserve-prefill-7f2szqt_f0cfa28d-bba4-4e59-80a9-55145e2e51f8/main/0.log"
Apr 24 15:05:04.915069 ip-10-0-136-132 kubenswrapper[2574]: I0424 15:05:04.915052    2574 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_llmisvc-model-fb-opt-125m-route-05aa9bba-kserve-prefill-7f2szqt_f0cfa28d-bba4-4e59-80a9-55145e2e51f8/storage-initializer/0.log"
Apr 24 15:05:05.797716 ip-10-0-136-132 kubenswrapper[2574]: I0424 15:05:05.797690    2574 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_llmisvc-model-fb-opt-125m-route-05aa9bba-kserve-568864c59fhvjqg_7a613b84-cbb1-4ab8-bc7c-2187ccaae8e6/main/0.log"
Apr 24 15:05:05.803044 ip-10-0-136-132 kubenswrapper[2574]: I0424 15:05:05.803023    2574 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_llmisvc-model-fb-opt-125m-route-05aa9bba-kserve-568864c59fhvjqg_7a613b84-cbb1-4ab8-bc7c-2187ccaae8e6/llm-d-routing-sidecar/0.log"
Apr 24 15:05:05.810767 ip-10-0-136-132 kubenswrapper[2574]: I0424 15:05:05.810748    2574 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_llmisvc-model-fb-opt-125m-route-05aa9bba-kserve-568864c59fhvjqg_7a613b84-cbb1-4ab8-bc7c-2187ccaae8e6/storage-initializer/0.log"
Apr 24 15:05:05.825514 ip-10-0-136-132 kubenswrapper[2574]: I0424 15:05:05.825490    2574 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_llmisvc-model-fb-opt-125m-route-05aa9bba-kserve-prefill-7f2szqt_f0cfa28d-bba4-4e59-80a9-55145e2e51f8/main/0.log"
Apr 24 15:05:05.831821 ip-10-0-136-132 kubenswrapper[2574]: I0424 15:05:05.831804    2574 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_llmisvc-model-fb-opt-125m-route-05aa9bba-kserve-prefill-7f2szqt_f0cfa28d-bba4-4e59-80a9-55145e2e51f8/storage-initializer/0.log"
Apr 24 15:05:06.742057 ip-10-0-136-132 kubenswrapper[2574]: I0424 15:05:06.742027    2574 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_llmisvc-model-fb-opt-125m-route-05aa9bba-kserve-568864c59fhvjqg_7a613b84-cbb1-4ab8-bc7c-2187ccaae8e6/main/0.log"
Apr 24 15:05:06.747404 ip-10-0-136-132 kubenswrapper[2574]: I0424 15:05:06.747374    2574 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_llmisvc-model-fb-opt-125m-route-05aa9bba-kserve-568864c59fhvjqg_7a613b84-cbb1-4ab8-bc7c-2187ccaae8e6/llm-d-routing-sidecar/0.log"
Apr 24 15:05:06.755688 ip-10-0-136-132 kubenswrapper[2574]: I0424 15:05:06.755668    2574 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_llmisvc-model-fb-opt-125m-route-05aa9bba-kserve-568864c59fhvjqg_7a613b84-cbb1-4ab8-bc7c-2187ccaae8e6/storage-initializer/0.log"
Apr 24 15:05:06.770614 ip-10-0-136-132 kubenswrapper[2574]: I0424 15:05:06.770590    2574 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_llmisvc-model-fb-opt-125m-route-05aa9bba-kserve-prefill-7f2szqt_f0cfa28d-bba4-4e59-80a9-55145e2e51f8/main/0.log"
Apr 24 15:05:06.778278 ip-10-0-136-132 kubenswrapper[2574]: I0424 15:05:06.778255    2574 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_llmisvc-model-fb-opt-125m-route-05aa9bba-kserve-prefill-7f2szqt_f0cfa28d-bba4-4e59-80a9-55145e2e51f8/storage-initializer/0.log"
Apr 24 15:05:07.679930 ip-10-0-136-132 kubenswrapper[2574]: I0424 15:05:07.679902    2574 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_llmisvc-model-fb-opt-125m-route-05aa9bba-kserve-568864c59fhvjqg_7a613b84-cbb1-4ab8-bc7c-2187ccaae8e6/main/0.log"
Apr 24 15:05:07.685067 ip-10-0-136-132 kubenswrapper[2574]: I0424 15:05:07.685047    2574 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_llmisvc-model-fb-opt-125m-route-05aa9bba-kserve-568864c59fhvjqg_7a613b84-cbb1-4ab8-bc7c-2187ccaae8e6/llm-d-routing-sidecar/0.log"
Apr 24 15:05:07.693066 ip-10-0-136-132 kubenswrapper[2574]: I0424 15:05:07.693047    2574 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_llmisvc-model-fb-opt-125m-route-05aa9bba-kserve-568864c59fhvjqg_7a613b84-cbb1-4ab8-bc7c-2187ccaae8e6/storage-initializer/0.log"
Apr 24 15:05:07.708569 ip-10-0-136-132 kubenswrapper[2574]: I0424 15:05:07.708544    2574 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_llmisvc-model-fb-opt-125m-route-05aa9bba-kserve-prefill-7f2szqt_f0cfa28d-bba4-4e59-80a9-55145e2e51f8/main/0.log"
Apr 24 15:05:07.715418 ip-10-0-136-132 kubenswrapper[2574]: I0424 15:05:07.715395    2574 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_llmisvc-model-fb-opt-125m-route-05aa9bba-kserve-prefill-7f2szqt_f0cfa28d-bba4-4e59-80a9-55145e2e51f8/storage-initializer/0.log"
Apr 24 15:05:08.621912 ip-10-0-136-132 kubenswrapper[2574]: I0424 15:05:08.621882    2574 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_llmisvc-model-fb-opt-125m-route-05aa9bba-kserve-568864c59fhvjqg_7a613b84-cbb1-4ab8-bc7c-2187ccaae8e6/main/0.log"
Apr 24 15:05:08.626675 ip-10-0-136-132 kubenswrapper[2574]: I0424 15:05:08.626653    2574 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_llmisvc-model-fb-opt-125m-route-05aa9bba-kserve-568864c59fhvjqg_7a613b84-cbb1-4ab8-bc7c-2187ccaae8e6/llm-d-routing-sidecar/0.log"
Apr 24 15:05:08.634946 ip-10-0-136-132 kubenswrapper[2574]: I0424 15:05:08.634927    2574 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_llmisvc-model-fb-opt-125m-route-05aa9bba-kserve-568864c59fhvjqg_7a613b84-cbb1-4ab8-bc7c-2187ccaae8e6/storage-initializer/0.log"
Apr 24 15:05:08.650758 ip-10-0-136-132 kubenswrapper[2574]: I0424 15:05:08.650734    2574 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_llmisvc-model-fb-opt-125m-route-05aa9bba-kserve-prefill-7f2szqt_f0cfa28d-bba4-4e59-80a9-55145e2e51f8/main/0.log"
Apr 24 15:05:08.656492 ip-10-0-136-132 kubenswrapper[2574]: I0424 15:05:08.656476    2574 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_llmisvc-model-fb-opt-125m-route-05aa9bba-kserve-prefill-7f2szqt_f0cfa28d-bba4-4e59-80a9-55145e2e51f8/storage-initializer/0.log"
Apr 24 15:05:09.562289 ip-10-0-136-132 kubenswrapper[2574]: I0424 15:05:09.562252    2574 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_llmisvc-model-fb-opt-125m-route-05aa9bba-kserve-568864c59fhvjqg_7a613b84-cbb1-4ab8-bc7c-2187ccaae8e6/main/0.log"
Apr 24 15:05:09.567736 ip-10-0-136-132 kubenswrapper[2574]: I0424 15:05:09.567716    2574 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_llmisvc-model-fb-opt-125m-route-05aa9bba-kserve-568864c59fhvjqg_7a613b84-cbb1-4ab8-bc7c-2187ccaae8e6/llm-d-routing-sidecar/0.log"
Apr 24 15:05:09.577899 ip-10-0-136-132 kubenswrapper[2574]: I0424 15:05:09.577882    2574 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_llmisvc-model-fb-opt-125m-route-05aa9bba-kserve-568864c59fhvjqg_7a613b84-cbb1-4ab8-bc7c-2187ccaae8e6/storage-initializer/0.log"
Apr 24 15:05:09.593052 ip-10-0-136-132 kubenswrapper[2574]: I0424 15:05:09.593021    2574 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_llmisvc-model-fb-opt-125m-route-05aa9bba-kserve-prefill-7f2szqt_f0cfa28d-bba4-4e59-80a9-55145e2e51f8/main/0.log"
Apr 24 15:05:09.599893 ip-10-0-136-132 kubenswrapper[2574]: I0424 15:05:09.599873    2574 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_llmisvc-model-fb-opt-125m-route-05aa9bba-kserve-prefill-7f2szqt_f0cfa28d-bba4-4e59-80a9-55145e2e51f8/storage-initializer/0.log"
Apr 24 15:05:10.574028 ip-10-0-136-132 kubenswrapper[2574]: I0424 15:05:10.573994    2574 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_llmisvc-model-fb-opt-125m-route-05aa9bba-kserve-568864c59fhvjqg_7a613b84-cbb1-4ab8-bc7c-2187ccaae8e6/main/0.log"
Apr 24 15:05:10.580697 ip-10-0-136-132 kubenswrapper[2574]: I0424 15:05:10.580670    2574 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_llmisvc-model-fb-opt-125m-route-05aa9bba-kserve-568864c59fhvjqg_7a613b84-cbb1-4ab8-bc7c-2187ccaae8e6/llm-d-routing-sidecar/0.log"
Apr 24 15:05:10.588806 ip-10-0-136-132 kubenswrapper[2574]: I0424 15:05:10.588788    2574 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_llmisvc-model-fb-opt-125m-route-05aa9bba-kserve-568864c59fhvjqg_7a613b84-cbb1-4ab8-bc7c-2187ccaae8e6/storage-initializer/0.log"
Apr 24 15:05:10.606375 ip-10-0-136-132 kubenswrapper[2574]: I0424 15:05:10.606351    2574 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_llmisvc-model-fb-opt-125m-route-05aa9bba-kserve-prefill-7f2szqt_f0cfa28d-bba4-4e59-80a9-55145e2e51f8/main/0.log"
Apr 24 15:05:10.613534 ip-10-0-136-132 kubenswrapper[2574]: I0424 15:05:10.613517    2574 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_llmisvc-model-fb-opt-125m-route-05aa9bba-kserve-prefill-7f2szqt_f0cfa28d-bba4-4e59-80a9-55145e2e51f8/storage-initializer/0.log"
Apr 24 15:05:13.089318 ip-10-0-136-132 kubenswrapper[2574]: I0424 15:05:13.089286    2574 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-68bd676465-8cn2c_98a9fc6f-299c-4729-8d6b-24d9464f30b7/authorino/0.log"
Apr 24 15:05:13.100015 ip-10-0-136-132 kubenswrapper[2574]: I0424 15:05:13.099980    2574 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-operator-7587b89b76-649lf_a5174eb2-b50f-486d-a789-823a9f147bde/manager/0.log"
Apr 24 15:05:13.111109 ip-10-0-136-132 kubenswrapper[2574]: I0424 15:05:13.111092    2574 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_dns-operator-controller-manager-844548ff4c-qzx86_323de2eb-8fc1-4879-92cb-5b7daa1c7a99/manager/0.log"
Apr 24 15:05:13.587151 ip-10-0-136-132 kubenswrapper[2574]: I0424 15:05:13.587112    2574 kuberuntime_container.go:864] "Killing container with a grace period" pod="kserve-ci-e2e-test/llmisvc-model-fb-opt-125m-route-05aa9bba-kserve-568864c59fhvjqg" podUID="7a613b84-cbb1-4ab8-bc7c-2187ccaae8e6" containerName="llm-d-routing-sidecar" containerID="cri-o://948389d6d5cb2378166ade76619c0e31c44c7062065e4e297f9a3f8e018424d8" gracePeriod=2
Apr 24 15:05:13.705974 ip-10-0-136-132 kubenswrapper[2574]: I0424 15:05:13.705948    2574 generic.go:358] "Generic (PLEG): container finished" podID="7a613b84-cbb1-4ab8-bc7c-2187ccaae8e6" containerID="948389d6d5cb2378166ade76619c0e31c44c7062065e4e297f9a3f8e018424d8" exitCode=0
Apr 24 15:05:13.706101 ip-10-0-136-132 kubenswrapper[2574]: I0424 15:05:13.706019    2574 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/llmisvc-model-fb-opt-125m-route-05aa9bba-kserve-568864c59fhvjqg" event={"ID":"7a613b84-cbb1-4ab8-bc7c-2187ccaae8e6","Type":"ContainerDied","Data":"948389d6d5cb2378166ade76619c0e31c44c7062065e4e297f9a3f8e018424d8"}
Apr 24 15:05:13.880248 ip-10-0-136-132 kubenswrapper[2574]: I0424 15:05:13.880219    2574 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_llmisvc-model-fb-opt-125m-route-05aa9bba-kserve-568864c59fhvjqg_7a613b84-cbb1-4ab8-bc7c-2187ccaae8e6/main/0.log"
Apr 24 15:05:13.880933 ip-10-0-136-132 kubenswrapper[2574]: I0424 15:05:13.880917    2574 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/llmisvc-model-fb-opt-125m-route-05aa9bba-kserve-568864c59fhvjqg"
Apr 24 15:05:13.883749 ip-10-0-136-132 kubenswrapper[2574]: I0424 15:05:13.883733    2574 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/llmisvc-model-fb-opt-125m-route-05aa9bba-kserve-prefill-7f2szqt"
Apr 24 15:05:13.927287 ip-10-0-136-132 kubenswrapper[2574]: I0424 15:05:13.927255    2574 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/7a613b84-cbb1-4ab8-bc7c-2187ccaae8e6-dshm\") pod \"7a613b84-cbb1-4ab8-bc7c-2187ccaae8e6\" (UID: \"7a613b84-cbb1-4ab8-bc7c-2187ccaae8e6\") "
Apr 24 15:05:13.927490 ip-10-0-136-132 kubenswrapper[2574]: I0424 15:05:13.927301    2574 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/f0cfa28d-bba4-4e59-80a9-55145e2e51f8-tls-certs\") pod \"f0cfa28d-bba4-4e59-80a9-55145e2e51f8\" (UID: \"f0cfa28d-bba4-4e59-80a9-55145e2e51f8\") "
Apr 24 15:05:13.927490 ip-10-0-136-132 kubenswrapper[2574]: I0424 15:05:13.927328    2574 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/f0cfa28d-bba4-4e59-80a9-55145e2e51f8-dshm\") pod \"f0cfa28d-bba4-4e59-80a9-55145e2e51f8\" (UID: \"f0cfa28d-bba4-4e59-80a9-55145e2e51f8\") "
Apr 24 15:05:13.927490 ip-10-0-136-132 kubenswrapper[2574]: I0424 15:05:13.927471    2574 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/7a613b84-cbb1-4ab8-bc7c-2187ccaae8e6-kserve-provision-location\") pod \"7a613b84-cbb1-4ab8-bc7c-2187ccaae8e6\" (UID: \"7a613b84-cbb1-4ab8-bc7c-2187ccaae8e6\") "
Apr 24 15:05:13.927782 ip-10-0-136-132 kubenswrapper[2574]: I0424 15:05:13.927508    2574 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"kube-api-access-6dgsc\" (UniqueName: \"kubernetes.io/projected/7a613b84-cbb1-4ab8-bc7c-2187ccaae8e6-kube-api-access-6dgsc\") pod \"7a613b84-cbb1-4ab8-bc7c-2187ccaae8e6\" (UID: \"7a613b84-cbb1-4ab8-bc7c-2187ccaae8e6\") "
Apr 24 15:05:13.927782 ip-10-0-136-132 kubenswrapper[2574]: I0424 15:05:13.927565    2574 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/f0cfa28d-bba4-4e59-80a9-55145e2e51f8-kserve-provision-location\") pod \"f0cfa28d-bba4-4e59-80a9-55145e2e51f8\" (UID: \"f0cfa28d-bba4-4e59-80a9-55145e2e51f8\") "
Apr 24 15:05:13.927782 ip-10-0-136-132 kubenswrapper[2574]: I0424 15:05:13.927588    2574 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/f0cfa28d-bba4-4e59-80a9-55145e2e51f8-model-cache\") pod \"f0cfa28d-bba4-4e59-80a9-55145e2e51f8\" (UID: \"f0cfa28d-bba4-4e59-80a9-55145e2e51f8\") "
Apr 24 15:05:13.927782 ip-10-0-136-132 kubenswrapper[2574]: I0424 15:05:13.927628    2574 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/7a613b84-cbb1-4ab8-bc7c-2187ccaae8e6-model-cache\") pod \"7a613b84-cbb1-4ab8-bc7c-2187ccaae8e6\" (UID: \"7a613b84-cbb1-4ab8-bc7c-2187ccaae8e6\") "
Apr 24 15:05:13.927782 ip-10-0-136-132 kubenswrapper[2574]: I0424 15:05:13.927668    2574 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/7a613b84-cbb1-4ab8-bc7c-2187ccaae8e6-home\") pod \"7a613b84-cbb1-4ab8-bc7c-2187ccaae8e6\" (UID: \"7a613b84-cbb1-4ab8-bc7c-2187ccaae8e6\") "
Apr 24 15:05:13.927782 ip-10-0-136-132 kubenswrapper[2574]: I0424 15:05:13.927691    2574 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/f0cfa28d-bba4-4e59-80a9-55145e2e51f8-home\") pod \"f0cfa28d-bba4-4e59-80a9-55145e2e51f8\" (UID: \"f0cfa28d-bba4-4e59-80a9-55145e2e51f8\") "
Apr 24 15:05:13.927782 ip-10-0-136-132 kubenswrapper[2574]: I0424 15:05:13.927719    2574 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/7a613b84-cbb1-4ab8-bc7c-2187ccaae8e6-tls-certs\") pod \"7a613b84-cbb1-4ab8-bc7c-2187ccaae8e6\" (UID: \"7a613b84-cbb1-4ab8-bc7c-2187ccaae8e6\") "
Apr 24 15:05:13.927782 ip-10-0-136-132 kubenswrapper[2574]: I0424 15:05:13.927748    2574 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"kube-api-access-6k9w6\" (UniqueName: \"kubernetes.io/projected/f0cfa28d-bba4-4e59-80a9-55145e2e51f8-kube-api-access-6k9w6\") pod \"f0cfa28d-bba4-4e59-80a9-55145e2e51f8\" (UID: \"f0cfa28d-bba4-4e59-80a9-55145e2e51f8\") "
Apr 24 15:05:13.928230 ip-10-0-136-132 kubenswrapper[2574]: I0424 15:05:13.927902    2574 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f0cfa28d-bba4-4e59-80a9-55145e2e51f8-model-cache" (OuterVolumeSpecName: "model-cache") pod "f0cfa28d-bba4-4e59-80a9-55145e2e51f8" (UID: "f0cfa28d-bba4-4e59-80a9-55145e2e51f8"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Apr 24 15:05:13.928230 ip-10-0-136-132 kubenswrapper[2574]: I0424 15:05:13.928066    2574 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/f0cfa28d-bba4-4e59-80a9-55145e2e51f8-model-cache\") on node \"ip-10-0-136-132.ec2.internal\" DevicePath \"\""
Apr 24 15:05:13.930869 ip-10-0-136-132 kubenswrapper[2574]: I0424 15:05:13.928606    2574 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f0cfa28d-bba4-4e59-80a9-55145e2e51f8-home" (OuterVolumeSpecName: "home") pod "f0cfa28d-bba4-4e59-80a9-55145e2e51f8" (UID: "f0cfa28d-bba4-4e59-80a9-55145e2e51f8"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Apr 24 15:05:13.930869 ip-10-0-136-132 kubenswrapper[2574]: I0424 15:05:13.928695    2574 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/7a613b84-cbb1-4ab8-bc7c-2187ccaae8e6-model-cache" (OuterVolumeSpecName: "model-cache") pod "7a613b84-cbb1-4ab8-bc7c-2187ccaae8e6" (UID: "7a613b84-cbb1-4ab8-bc7c-2187ccaae8e6"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Apr 24 15:05:13.930869 ip-10-0-136-132 kubenswrapper[2574]: I0424 15:05:13.929004    2574 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/7a613b84-cbb1-4ab8-bc7c-2187ccaae8e6-home" (OuterVolumeSpecName: "home") pod "7a613b84-cbb1-4ab8-bc7c-2187ccaae8e6" (UID: "7a613b84-cbb1-4ab8-bc7c-2187ccaae8e6"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Apr 24 15:05:13.930869 ip-10-0-136-132 kubenswrapper[2574]: I0424 15:05:13.929843    2574 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/f0cfa28d-bba4-4e59-80a9-55145e2e51f8-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "f0cfa28d-bba4-4e59-80a9-55145e2e51f8" (UID: "f0cfa28d-bba4-4e59-80a9-55145e2e51f8"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Apr 24 15:05:13.930869 ip-10-0-136-132 kubenswrapper[2574]: I0424 15:05:13.930149    2574 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/7a613b84-cbb1-4ab8-bc7c-2187ccaae8e6-dshm" (OuterVolumeSpecName: "dshm") pod "7a613b84-cbb1-4ab8-bc7c-2187ccaae8e6" (UID: "7a613b84-cbb1-4ab8-bc7c-2187ccaae8e6"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Apr 24 15:05:13.931218 ip-10-0-136-132 kubenswrapper[2574]: I0424 15:05:13.930925    2574 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/7a613b84-cbb1-4ab8-bc7c-2187ccaae8e6-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "7a613b84-cbb1-4ab8-bc7c-2187ccaae8e6" (UID: "7a613b84-cbb1-4ab8-bc7c-2187ccaae8e6"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Apr 24 15:05:13.931218 ip-10-0-136-132 kubenswrapper[2574]: I0424 15:05:13.930961    2574 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/7a613b84-cbb1-4ab8-bc7c-2187ccaae8e6-kube-api-access-6dgsc" (OuterVolumeSpecName: "kube-api-access-6dgsc") pod "7a613b84-cbb1-4ab8-bc7c-2187ccaae8e6" (UID: "7a613b84-cbb1-4ab8-bc7c-2187ccaae8e6"). InnerVolumeSpecName "kube-api-access-6dgsc". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Apr 24 15:05:13.931218 ip-10-0-136-132 kubenswrapper[2574]: I0424 15:05:13.931155    2574 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f0cfa28d-bba4-4e59-80a9-55145e2e51f8-dshm" (OuterVolumeSpecName: "dshm") pod "f0cfa28d-bba4-4e59-80a9-55145e2e51f8" (UID: "f0cfa28d-bba4-4e59-80a9-55145e2e51f8"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Apr 24 15:05:13.932053 ip-10-0-136-132 kubenswrapper[2574]: I0424 15:05:13.932024    2574 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f0cfa28d-bba4-4e59-80a9-55145e2e51f8-kube-api-access-6k9w6" (OuterVolumeSpecName: "kube-api-access-6k9w6") pod "f0cfa28d-bba4-4e59-80a9-55145e2e51f8" (UID: "f0cfa28d-bba4-4e59-80a9-55145e2e51f8"). InnerVolumeSpecName "kube-api-access-6k9w6". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Apr 24 15:05:13.987324 ip-10-0-136-132 kubenswrapper[2574]: I0424 15:05:13.987259    2574 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/7a613b84-cbb1-4ab8-bc7c-2187ccaae8e6-kserve-provision-location" (OuterVolumeSpecName: "kserve-provision-location") pod "7a613b84-cbb1-4ab8-bc7c-2187ccaae8e6" (UID: "7a613b84-cbb1-4ab8-bc7c-2187ccaae8e6"). InnerVolumeSpecName "kserve-provision-location". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Apr 24 15:05:13.988273 ip-10-0-136-132 kubenswrapper[2574]: I0424 15:05:13.988247    2574 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f0cfa28d-bba4-4e59-80a9-55145e2e51f8-kserve-provision-location" (OuterVolumeSpecName: "kserve-provision-location") pod "f0cfa28d-bba4-4e59-80a9-55145e2e51f8" (UID: "f0cfa28d-bba4-4e59-80a9-55145e2e51f8"). InnerVolumeSpecName "kserve-provision-location". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Apr 24 15:05:14.028548 ip-10-0-136-132 kubenswrapper[2574]: I0424 15:05:14.028518    2574 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/7a613b84-cbb1-4ab8-bc7c-2187ccaae8e6-home\") on node \"ip-10-0-136-132.ec2.internal\" DevicePath \"\""
Apr 24 15:05:14.028548 ip-10-0-136-132 kubenswrapper[2574]: I0424 15:05:14.028546    2574 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/f0cfa28d-bba4-4e59-80a9-55145e2e51f8-home\") on node \"ip-10-0-136-132.ec2.internal\" DevicePath \"\""
Apr 24 15:05:14.028737 ip-10-0-136-132 kubenswrapper[2574]: I0424 15:05:14.028560    2574 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/7a613b84-cbb1-4ab8-bc7c-2187ccaae8e6-tls-certs\") on node \"ip-10-0-136-132.ec2.internal\" DevicePath \"\""
Apr 24 15:05:14.028737 ip-10-0-136-132 kubenswrapper[2574]: I0424 15:05:14.028574    2574 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-6k9w6\" (UniqueName: \"kubernetes.io/projected/f0cfa28d-bba4-4e59-80a9-55145e2e51f8-kube-api-access-6k9w6\") on node \"ip-10-0-136-132.ec2.internal\" DevicePath \"\""
Apr 24 15:05:14.028737 ip-10-0-136-132 kubenswrapper[2574]: I0424 15:05:14.028587    2574 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/7a613b84-cbb1-4ab8-bc7c-2187ccaae8e6-dshm\") on node \"ip-10-0-136-132.ec2.internal\" DevicePath \"\""
Apr 24 15:05:14.028737 ip-10-0-136-132 kubenswrapper[2574]: I0424 15:05:14.028599    2574 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/f0cfa28d-bba4-4e59-80a9-55145e2e51f8-tls-certs\") on node \"ip-10-0-136-132.ec2.internal\" DevicePath \"\""
Apr 24 15:05:14.028737 ip-10-0-136-132 kubenswrapper[2574]: I0424 15:05:14.028611    2574 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/f0cfa28d-bba4-4e59-80a9-55145e2e51f8-dshm\") on node \"ip-10-0-136-132.ec2.internal\" DevicePath \"\""
Apr 24 15:05:14.028737 ip-10-0-136-132 kubenswrapper[2574]: I0424 15:05:14.028624    2574 reconciler_common.go:299] "Volume detached for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/7a613b84-cbb1-4ab8-bc7c-2187ccaae8e6-kserve-provision-location\") on node \"ip-10-0-136-132.ec2.internal\" DevicePath \"\""
Apr 24 15:05:14.028737 ip-10-0-136-132 kubenswrapper[2574]: I0424 15:05:14.028636    2574 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-6dgsc\" (UniqueName: \"kubernetes.io/projected/7a613b84-cbb1-4ab8-bc7c-2187ccaae8e6-kube-api-access-6dgsc\") on node \"ip-10-0-136-132.ec2.internal\" DevicePath \"\""
Apr 24 15:05:14.028737 ip-10-0-136-132 kubenswrapper[2574]: I0424 15:05:14.028649    2574 reconciler_common.go:299] "Volume detached for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/f0cfa28d-bba4-4e59-80a9-55145e2e51f8-kserve-provision-location\") on node \"ip-10-0-136-132.ec2.internal\" DevicePath \"\""
Apr 24 15:05:14.028737 ip-10-0-136-132 kubenswrapper[2574]: I0424 15:05:14.028662    2574 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/7a613b84-cbb1-4ab8-bc7c-2187ccaae8e6-model-cache\") on node \"ip-10-0-136-132.ec2.internal\" DevicePath \"\""
Apr 24 15:05:14.711577 ip-10-0-136-132 kubenswrapper[2574]: I0424 15:05:14.711538    2574 generic.go:358] "Generic (PLEG): container finished" podID="f0cfa28d-bba4-4e59-80a9-55145e2e51f8" containerID="ee9b62b7762d126b08c542347ec2177afb8052b330b3f53eef2db7c490a98efa" exitCode=137
Apr 24 15:05:14.711982 ip-10-0-136-132 kubenswrapper[2574]: I0424 15:05:14.711621    2574 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/llmisvc-model-fb-opt-125m-route-05aa9bba-kserve-prefill-7f2szqt" event={"ID":"f0cfa28d-bba4-4e59-80a9-55145e2e51f8","Type":"ContainerDied","Data":"ee9b62b7762d126b08c542347ec2177afb8052b330b3f53eef2db7c490a98efa"}
Apr 24 15:05:14.711982 ip-10-0-136-132 kubenswrapper[2574]: I0424 15:05:14.711644    2574 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/llmisvc-model-fb-opt-125m-route-05aa9bba-kserve-prefill-7f2szqt"
Apr 24 15:05:14.711982 ip-10-0-136-132 kubenswrapper[2574]: I0424 15:05:14.711663    2574 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/llmisvc-model-fb-opt-125m-route-05aa9bba-kserve-prefill-7f2szqt" event={"ID":"f0cfa28d-bba4-4e59-80a9-55145e2e51f8","Type":"ContainerDied","Data":"edc9eea73ba0dcf5062b1479a879bc99763c5754ec88eddd9f9698a55e250269"}
Apr 24 15:05:14.711982 ip-10-0-136-132 kubenswrapper[2574]: I0424 15:05:14.711685    2574 scope.go:117] "RemoveContainer" containerID="ee9b62b7762d126b08c542347ec2177afb8052b330b3f53eef2db7c490a98efa"
Apr 24 15:05:14.713117 ip-10-0-136-132 kubenswrapper[2574]: I0424 15:05:14.713000    2574 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_llmisvc-model-fb-opt-125m-route-05aa9bba-kserve-568864c59fhvjqg_7a613b84-cbb1-4ab8-bc7c-2187ccaae8e6/main/0.log"
Apr 24 15:05:14.713727 ip-10-0-136-132 kubenswrapper[2574]: I0424 15:05:14.713704    2574 generic.go:358] "Generic (PLEG): container finished" podID="7a613b84-cbb1-4ab8-bc7c-2187ccaae8e6" containerID="af3b02be99dc732bd4d00a99bc32e8944a80c854a921bcf888f85cb26d999cf0" exitCode=137
Apr 24 15:05:14.713868 ip-10-0-136-132 kubenswrapper[2574]: I0424 15:05:14.713787    2574 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/llmisvc-model-fb-opt-125m-route-05aa9bba-kserve-568864c59fhvjqg" event={"ID":"7a613b84-cbb1-4ab8-bc7c-2187ccaae8e6","Type":"ContainerDied","Data":"af3b02be99dc732bd4d00a99bc32e8944a80c854a921bcf888f85cb26d999cf0"}
Apr 24 15:05:14.713868 ip-10-0-136-132 kubenswrapper[2574]: I0424 15:05:14.713822    2574 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/llmisvc-model-fb-opt-125m-route-05aa9bba-kserve-568864c59fhvjqg" event={"ID":"7a613b84-cbb1-4ab8-bc7c-2187ccaae8e6","Type":"ContainerDied","Data":"9df4703af3c8822cc5b231522c3f8922fc1116ca91aab7d3b349f16f37c25614"}
Apr 24 15:05:14.713868 ip-10-0-136-132 kubenswrapper[2574]: I0424 15:05:14.713834    2574 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/llmisvc-model-fb-opt-125m-route-05aa9bba-kserve-568864c59fhvjqg"
Apr 24 15:05:14.734834 ip-10-0-136-132 kubenswrapper[2574]: I0424 15:05:14.734809    2574 scope.go:117] "RemoveContainer" containerID="78cffda93bb61a573a2cd6e23085eb66ffc9337828bcafb41da8b3c47a9800dc"
Apr 24 15:05:14.736813 ip-10-0-136-132 kubenswrapper[2574]: I0424 15:05:14.736766    2574 kubelet.go:2553] "SyncLoop DELETE" source="api" pods=["kserve-ci-e2e-test/llmisvc-model-fb-opt-125m-route-05aa9bba-kserve-568864c59fhvjqg"]
Apr 24 15:05:14.740465 ip-10-0-136-132 kubenswrapper[2574]: I0424 15:05:14.740445    2574 kubelet.go:2547] "SyncLoop REMOVE" source="api" pods=["kserve-ci-e2e-test/llmisvc-model-fb-opt-125m-route-05aa9bba-kserve-568864c59fhvjqg"]
Apr 24 15:05:14.745353 ip-10-0-136-132 kubenswrapper[2574]: I0424 15:05:14.745315    2574 scope.go:117] "RemoveContainer" containerID="ee9b62b7762d126b08c542347ec2177afb8052b330b3f53eef2db7c490a98efa"
Apr 24 15:05:14.745701 ip-10-0-136-132 kubenswrapper[2574]: E0424 15:05:14.745680    2574 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"ee9b62b7762d126b08c542347ec2177afb8052b330b3f53eef2db7c490a98efa\": container with ID starting with ee9b62b7762d126b08c542347ec2177afb8052b330b3f53eef2db7c490a98efa not found: ID does not exist" containerID="ee9b62b7762d126b08c542347ec2177afb8052b330b3f53eef2db7c490a98efa"
Apr 24 15:05:14.745757 ip-10-0-136-132 kubenswrapper[2574]: I0424 15:05:14.745707    2574 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"ee9b62b7762d126b08c542347ec2177afb8052b330b3f53eef2db7c490a98efa"} err="failed to get container status \"ee9b62b7762d126b08c542347ec2177afb8052b330b3f53eef2db7c490a98efa\": rpc error: code = NotFound desc = could not find container \"ee9b62b7762d126b08c542347ec2177afb8052b330b3f53eef2db7c490a98efa\": container with ID starting with ee9b62b7762d126b08c542347ec2177afb8052b330b3f53eef2db7c490a98efa not found: ID does not exist"
Apr 24 15:05:14.745757 ip-10-0-136-132 kubenswrapper[2574]: I0424 15:05:14.745726    2574 scope.go:117] "RemoveContainer" containerID="78cffda93bb61a573a2cd6e23085eb66ffc9337828bcafb41da8b3c47a9800dc"
Apr 24 15:05:14.745942 ip-10-0-136-132 kubenswrapper[2574]: E0424 15:05:14.745926    2574 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"78cffda93bb61a573a2cd6e23085eb66ffc9337828bcafb41da8b3c47a9800dc\": container with ID starting with 78cffda93bb61a573a2cd6e23085eb66ffc9337828bcafb41da8b3c47a9800dc not found: ID does not exist" containerID="78cffda93bb61a573a2cd6e23085eb66ffc9337828bcafb41da8b3c47a9800dc"
Apr 24 15:05:14.745985 ip-10-0-136-132 kubenswrapper[2574]: I0424 15:05:14.745946    2574 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"78cffda93bb61a573a2cd6e23085eb66ffc9337828bcafb41da8b3c47a9800dc"} err="failed to get container status \"78cffda93bb61a573a2cd6e23085eb66ffc9337828bcafb41da8b3c47a9800dc\": rpc error: code = NotFound desc = could not find container \"78cffda93bb61a573a2cd6e23085eb66ffc9337828bcafb41da8b3c47a9800dc\": container with ID starting with 78cffda93bb61a573a2cd6e23085eb66ffc9337828bcafb41da8b3c47a9800dc not found: ID does not exist"
Apr 24 15:05:14.745985 ip-10-0-136-132 kubenswrapper[2574]: I0424 15:05:14.745960    2574 scope.go:117] "RemoveContainer" containerID="af3b02be99dc732bd4d00a99bc32e8944a80c854a921bcf888f85cb26d999cf0"
Apr 24 15:05:14.749305 ip-10-0-136-132 kubenswrapper[2574]: I0424 15:05:14.749285    2574 kubelet.go:2553] "SyncLoop DELETE" source="api" pods=["kserve-ci-e2e-test/llmisvc-model-fb-opt-125m-route-05aa9bba-kserve-prefill-7f2szqt"]
Apr 24 15:05:14.752992 ip-10-0-136-132 kubenswrapper[2574]: I0424 15:05:14.752969    2574 kubelet.go:2547] "SyncLoop REMOVE" source="api" pods=["kserve-ci-e2e-test/llmisvc-model-fb-opt-125m-route-05aa9bba-kserve-prefill-7f2szqt"]
Apr 24 15:05:14.756354 ip-10-0-136-132 kubenswrapper[2574]: I0424 15:05:14.756321    2574 scope.go:117] "RemoveContainer" containerID="e93ba8ed8c23b6f6e25e4148a6f4e457c7e3fb02f9539bf9096b34b9a5911672"
Apr 24 15:05:14.768535 ip-10-0-136-132 kubenswrapper[2574]: I0424 15:05:14.768519    2574 scope.go:117] "RemoveContainer" containerID="948389d6d5cb2378166ade76619c0e31c44c7062065e4e297f9a3f8e018424d8"
Apr 24 15:05:14.776063 ip-10-0-136-132 kubenswrapper[2574]: I0424 15:05:14.776048    2574 scope.go:117] "RemoveContainer" containerID="af3b02be99dc732bd4d00a99bc32e8944a80c854a921bcf888f85cb26d999cf0"
Apr 24 15:05:14.776303 ip-10-0-136-132 kubenswrapper[2574]: E0424 15:05:14.776284    2574 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"af3b02be99dc732bd4d00a99bc32e8944a80c854a921bcf888f85cb26d999cf0\": container with ID starting with af3b02be99dc732bd4d00a99bc32e8944a80c854a921bcf888f85cb26d999cf0 not found: ID does not exist" containerID="af3b02be99dc732bd4d00a99bc32e8944a80c854a921bcf888f85cb26d999cf0"
Apr 24 15:05:14.776379 ip-10-0-136-132 kubenswrapper[2574]: I0424 15:05:14.776310    2574 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"af3b02be99dc732bd4d00a99bc32e8944a80c854a921bcf888f85cb26d999cf0"} err="failed to get container status \"af3b02be99dc732bd4d00a99bc32e8944a80c854a921bcf888f85cb26d999cf0\": rpc error: code = NotFound desc = could not find container \"af3b02be99dc732bd4d00a99bc32e8944a80c854a921bcf888f85cb26d999cf0\": container with ID starting with af3b02be99dc732bd4d00a99bc32e8944a80c854a921bcf888f85cb26d999cf0 not found: ID does not exist"
Apr 24 15:05:14.776379 ip-10-0-136-132 kubenswrapper[2574]: I0424 15:05:14.776328    2574 scope.go:117] "RemoveContainer" containerID="e93ba8ed8c23b6f6e25e4148a6f4e457c7e3fb02f9539bf9096b34b9a5911672"
Apr 24 15:05:14.776599 ip-10-0-136-132 kubenswrapper[2574]: E0424 15:05:14.776582    2574 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"e93ba8ed8c23b6f6e25e4148a6f4e457c7e3fb02f9539bf9096b34b9a5911672\": container with ID starting with e93ba8ed8c23b6f6e25e4148a6f4e457c7e3fb02f9539bf9096b34b9a5911672 not found: ID does not exist" containerID="e93ba8ed8c23b6f6e25e4148a6f4e457c7e3fb02f9539bf9096b34b9a5911672"
Apr 24 15:05:14.776664 ip-10-0-136-132 kubenswrapper[2574]: I0424 15:05:14.776609    2574 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e93ba8ed8c23b6f6e25e4148a6f4e457c7e3fb02f9539bf9096b34b9a5911672"} err="failed to get container status \"e93ba8ed8c23b6f6e25e4148a6f4e457c7e3fb02f9539bf9096b34b9a5911672\": rpc error: code = NotFound desc = could not find container \"e93ba8ed8c23b6f6e25e4148a6f4e457c7e3fb02f9539bf9096b34b9a5911672\": container with ID starting with e93ba8ed8c23b6f6e25e4148a6f4e457c7e3fb02f9539bf9096b34b9a5911672 not found: ID does not exist"
Apr 24 15:05:14.776664 ip-10-0-136-132 kubenswrapper[2574]: I0424 15:05:14.776633    2574 scope.go:117] "RemoveContainer" containerID="948389d6d5cb2378166ade76619c0e31c44c7062065e4e297f9a3f8e018424d8"
Apr 24 15:05:14.776859 ip-10-0-136-132 kubenswrapper[2574]: E0424 15:05:14.776844    2574 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"948389d6d5cb2378166ade76619c0e31c44c7062065e4e297f9a3f8e018424d8\": container with ID starting with 948389d6d5cb2378166ade76619c0e31c44c7062065e4e297f9a3f8e018424d8 not found: ID does not exist" containerID="948389d6d5cb2378166ade76619c0e31c44c7062065e4e297f9a3f8e018424d8"
Apr 24 15:05:14.776915 ip-10-0-136-132 kubenswrapper[2574]: I0424 15:05:14.776865    2574 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"948389d6d5cb2378166ade76619c0e31c44c7062065e4e297f9a3f8e018424d8"} err="failed to get container status \"948389d6d5cb2378166ade76619c0e31c44c7062065e4e297f9a3f8e018424d8\": rpc error: code = NotFound desc = could not find container \"948389d6d5cb2378166ade76619c0e31c44c7062065e4e297f9a3f8e018424d8\": container with ID starting with 948389d6d5cb2378166ade76619c0e31c44c7062065e4e297f9a3f8e018424d8 not found: ID does not exist"
Apr 24 15:05:15.770385 ip-10-0-136-132 kubenswrapper[2574]: I0424 15:05:15.770329    2574 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="7a613b84-cbb1-4ab8-bc7c-2187ccaae8e6" path="/var/lib/kubelet/pods/7a613b84-cbb1-4ab8-bc7c-2187ccaae8e6/volumes"
Apr 24 15:05:15.770846 ip-10-0-136-132 kubenswrapper[2574]: I0424 15:05:15.770832    2574 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="f0cfa28d-bba4-4e59-80a9-55145e2e51f8" path="/var/lib/kubelet/pods/f0cfa28d-bba4-4e59-80a9-55145e2e51f8/volumes"
Apr 24 15:05:18.385820 ip-10-0-136-132 kubenswrapper[2574]: I0424 15:05:18.385788    2574 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_global-pull-secret-syncer-nwx74_5e2a3789-4ab2-4df5-bbbf-e649d90f15a5/global-pull-secret-syncer/0.log"
Apr 24 15:05:18.521020 ip-10-0-136-132 kubenswrapper[2574]: I0424 15:05:18.520994    2574 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_konnectivity-agent-zz678_b1844926-4d01-4f6e-8970-a8d662acd7c6/konnectivity-agent/0.log"
Apr 24 15:05:18.558954 ip-10-0-136-132 kubenswrapper[2574]: I0424 15:05:18.558924    2574 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_kube-apiserver-proxy-ip-10-0-136-132.ec2.internal_682e7062ab8cd6aae842785f377acfd4/haproxy/0.log"
Apr 24 15:05:22.000796 ip-10-0-136-132 kubenswrapper[2574]: I0424 15:05:22.000766    2574 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-68bd676465-8cn2c_98a9fc6f-299c-4729-8d6b-24d9464f30b7/authorino/0.log"
Apr 24 15:05:22.031180 ip-10-0-136-132 kubenswrapper[2574]: I0424 15:05:22.031144    2574 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-operator-7587b89b76-649lf_a5174eb2-b50f-486d-a789-823a9f147bde/manager/0.log"
Apr 24 15:05:22.062596 ip-10-0-136-132 kubenswrapper[2574]: I0424 15:05:22.062576    2574 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_dns-operator-controller-manager-844548ff4c-qzx86_323de2eb-8fc1-4879-92cb-5b7daa1c7a99/manager/0.log"
Apr 24 15:05:23.372454 ip-10-0-136-132 kubenswrapper[2574]: I0424 15:05:23.372423    2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_51022bd6-a15d-4d4a-aa38-101a7f8e558f/alertmanager/0.log"
Apr 24 15:05:23.388740 ip-10-0-136-132 kubenswrapper[2574]: I0424 15:05:23.388717    2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_51022bd6-a15d-4d4a-aa38-101a7f8e558f/config-reloader/0.log"
Apr 24 15:05:23.416112 ip-10-0-136-132 kubenswrapper[2574]: I0424 15:05:23.416089    2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_51022bd6-a15d-4d4a-aa38-101a7f8e558f/kube-rbac-proxy-web/0.log"
Apr 24 15:05:23.436347 ip-10-0-136-132 kubenswrapper[2574]: I0424 15:05:23.436321    2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_51022bd6-a15d-4d4a-aa38-101a7f8e558f/kube-rbac-proxy/0.log"
Apr 24 15:05:23.453408 ip-10-0-136-132 kubenswrapper[2574]: I0424 15:05:23.453387    2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_51022bd6-a15d-4d4a-aa38-101a7f8e558f/kube-rbac-proxy-metric/0.log"
Apr 24 15:05:23.468967 ip-10-0-136-132 kubenswrapper[2574]: I0424 15:05:23.468951    2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_51022bd6-a15d-4d4a-aa38-101a7f8e558f/prom-label-proxy/0.log"
Apr 24 15:05:23.487216 ip-10-0-136-132 kubenswrapper[2574]: I0424 15:05:23.487198    2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_51022bd6-a15d-4d4a-aa38-101a7f8e558f/init-config-reloader/0.log"
Apr 24 15:05:23.516926 ip-10-0-136-132 kubenswrapper[2574]: I0424 15:05:23.516906    2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_cluster-monitoring-operator-75587bd455-q9jbv_0be19048-1b01-4c64-bcca-94c9bff7768d/cluster-monitoring-operator/0.log"
Apr 24 15:05:23.541140 ip-10-0-136-132 kubenswrapper[2574]: I0424 15:05:23.541122    2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_kube-state-metrics-69db897b98-spwlq_e22659f8-0faa-4d68-99a6-d7d61e263c6a/kube-state-metrics/0.log"
Apr 24 15:05:23.560914 ip-10-0-136-132 kubenswrapper[2574]: I0424 15:05:23.560892    2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_kube-state-metrics-69db897b98-spwlq_e22659f8-0faa-4d68-99a6-d7d61e263c6a/kube-rbac-proxy-main/0.log"
Apr 24 15:05:23.576612 ip-10-0-136-132 kubenswrapper[2574]: I0424 15:05:23.576587    2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_kube-state-metrics-69db897b98-spwlq_e22659f8-0faa-4d68-99a6-d7d61e263c6a/kube-rbac-proxy-self/0.log"
Apr 24 15:05:23.598572 ip-10-0-136-132 kubenswrapper[2574]: I0424 15:05:23.598548    2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_metrics-server-6cf575bddc-lztrh_9dda3a92-755c-4dee-a886-0f8519fe84af/metrics-server/0.log"
Apr 24 15:05:23.619071 ip-10-0-136-132 kubenswrapper[2574]: I0424 15:05:23.619052    2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_monitoring-plugin-7dccd58f55-4v2k7_9192b2ee-7b51-4184-9f8f-fd91ff3f3535/monitoring-plugin/0.log"
Apr 24 15:05:23.704330 ip-10-0-136-132 kubenswrapper[2574]: I0424 15:05:23.704263    2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-tprms_0694c1e6-ce19-4731-a344-f8558961d7d4/node-exporter/0.log"
Apr 24 15:05:23.725193 ip-10-0-136-132 kubenswrapper[2574]: I0424 15:05:23.725173    2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-tprms_0694c1e6-ce19-4731-a344-f8558961d7d4/kube-rbac-proxy/0.log"
Apr 24 15:05:23.739934 ip-10-0-136-132 kubenswrapper[2574]: I0424 15:05:23.739888    2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-tprms_0694c1e6-ce19-4731-a344-f8558961d7d4/init-textfile/0.log"
Apr 24 15:05:23.839182 ip-10-0-136-132 kubenswrapper[2574]: I0424 15:05:23.839155    2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_openshift-state-metrics-9d44df66c-hkfgq_80e82582-a13a-4c7b-a6ea-3074aff3a5f5/kube-rbac-proxy-main/0.log"
Apr 24 15:05:23.855715 ip-10-0-136-132 kubenswrapper[2574]: I0424 15:05:23.855691    2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_openshift-state-metrics-9d44df66c-hkfgq_80e82582-a13a-4c7b-a6ea-3074aff3a5f5/kube-rbac-proxy-self/0.log"
Apr 24 15:05:23.872735 ip-10-0-136-132 kubenswrapper[2574]: I0424 15:05:23.872715    2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_openshift-state-metrics-9d44df66c-hkfgq_80e82582-a13a-4c7b-a6ea-3074aff3a5f5/openshift-state-metrics/0.log"
Apr 24 15:05:24.165729 ip-10-0-136-132 kubenswrapper[2574]: I0424 15:05:24.165699    2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_telemeter-client-5cf48867f6-bv6xf_c978078e-58d2-4fb8-bb17-0913013b8d7a/telemeter-client/0.log"
Apr 24 15:05:24.186001 ip-10-0-136-132 kubenswrapper[2574]: I0424 15:05:24.185975    2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_telemeter-client-5cf48867f6-bv6xf_c978078e-58d2-4fb8-bb17-0913013b8d7a/reload/0.log"
Apr 24 15:05:24.203003 ip-10-0-136-132 kubenswrapper[2574]: I0424 15:05:24.202985    2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_telemeter-client-5cf48867f6-bv6xf_c978078e-58d2-4fb8-bb17-0913013b8d7a/kube-rbac-proxy/0.log"
Apr 24 15:05:25.844928 ip-10-0-136-132 kubenswrapper[2574]: I0424 15:05:25.844896    2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-network-console_networking-console-plugin-cb95c66f6-96ndb_6a47c891-13d9-4726-8204-e7559e6e6ff4/networking-console-plugin/0.log"
Apr 24 15:05:26.297330 ip-10-0-136-132 kubenswrapper[2574]: I0424 15:05:26.297300    2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-9d4b6777b-f6ngr_2571829f-666a-422a-9b02-2d2bb4c89b25/console-operator/1.log"
Apr 24 15:05:26.301788 ip-10-0-136-132 kubenswrapper[2574]: I0424 15:05:26.301764    2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-9d4b6777b-f6ngr_2571829f-666a-422a-9b02-2d2bb4c89b25/console-operator/2.log"
Apr 24 15:05:27.304266 ip-10-0-136-132 kubenswrapper[2574]: I0424 15:05:27.304237    2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-cluster-storage-operator_volume-data-source-validator-7c6cbb6c87-j2gr5_de9f9e0b-653f-481d-b23f-5c408b5cc75e/volume-data-source-validator/0.log"
Apr 24 15:05:27.745103 ip-10-0-136-132 kubenswrapper[2574]: I0424 15:05:27.745071    2574 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["openshift-must-gather-rpb9c/perf-node-gather-daemonset-fmz4h"]
Apr 24 15:05:27.745489 ip-10-0-136-132 kubenswrapper[2574]: I0424 15:05:27.745476    2574 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f0cfa28d-bba4-4e59-80a9-55145e2e51f8" containerName="storage-initializer"
Apr 24 15:05:27.745548 ip-10-0-136-132 kubenswrapper[2574]: I0424 15:05:27.745491    2574 state_mem.go:107] "Deleted CPUSet assignment" podUID="f0cfa28d-bba4-4e59-80a9-55145e2e51f8" containerName="storage-initializer"
Apr 24 15:05:27.745548 ip-10-0-136-132 kubenswrapper[2574]: I0424 15:05:27.745501    2574 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="7a613b84-cbb1-4ab8-bc7c-2187ccaae8e6" containerName="main"
Apr 24 15:05:27.745548 ip-10-0-136-132 kubenswrapper[2574]: I0424 15:05:27.745507    2574 state_mem.go:107] "Deleted CPUSet assignment" podUID="7a613b84-cbb1-4ab8-bc7c-2187ccaae8e6" containerName="main"
Apr 24 15:05:27.745548 ip-10-0-136-132 kubenswrapper[2574]: I0424 15:05:27.745524    2574 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="4a48929f-f66c-49b8-9d3d-08c90eb4c1c6" containerName="main"
Apr 24 15:05:27.745548 ip-10-0-136-132 kubenswrapper[2574]: I0424 15:05:27.745530    2574 state_mem.go:107] "Deleted CPUSet assignment" podUID="4a48929f-f66c-49b8-9d3d-08c90eb4c1c6" containerName="main"
Apr 24 15:05:27.745548 ip-10-0-136-132 kubenswrapper[2574]: I0424 15:05:27.745536    2574 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="7a613b84-cbb1-4ab8-bc7c-2187ccaae8e6" containerName="llm-d-routing-sidecar"
Apr 24 15:05:27.745548 ip-10-0-136-132 kubenswrapper[2574]: I0424 15:05:27.745542    2574 state_mem.go:107] "Deleted CPUSet assignment" podUID="7a613b84-cbb1-4ab8-bc7c-2187ccaae8e6" containerName="llm-d-routing-sidecar"
Apr 24 15:05:27.745757 ip-10-0-136-132 kubenswrapper[2574]: I0424 15:05:27.745551    2574 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="7a613b84-cbb1-4ab8-bc7c-2187ccaae8e6" containerName="storage-initializer"
Apr 24 15:05:27.745757 ip-10-0-136-132 kubenswrapper[2574]: I0424 15:05:27.745557    2574 state_mem.go:107] "Deleted CPUSet assignment" podUID="7a613b84-cbb1-4ab8-bc7c-2187ccaae8e6" containerName="storage-initializer"
Apr 24 15:05:27.745757 ip-10-0-136-132 kubenswrapper[2574]: I0424 15:05:27.745566    2574 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f0cfa28d-bba4-4e59-80a9-55145e2e51f8" containerName="main"
Apr 24 15:05:27.745757 ip-10-0-136-132 kubenswrapper[2574]: I0424 15:05:27.745571    2574 state_mem.go:107] "Deleted CPUSet assignment" podUID="f0cfa28d-bba4-4e59-80a9-55145e2e51f8" containerName="main"
Apr 24 15:05:27.745757 ip-10-0-136-132 kubenswrapper[2574]: I0424 15:05:27.745585    2574 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="4a48929f-f66c-49b8-9d3d-08c90eb4c1c6" containerName="storage-initializer"
Apr 24 15:05:27.745757 ip-10-0-136-132 kubenswrapper[2574]: I0424 15:05:27.745590    2574 state_mem.go:107] "Deleted CPUSet assignment" podUID="4a48929f-f66c-49b8-9d3d-08c90eb4c1c6" containerName="storage-initializer"
Apr 24 15:05:27.745757 ip-10-0-136-132 kubenswrapper[2574]: I0424 15:05:27.745649    2574 memory_manager.go:356] "RemoveStaleState removing state" podUID="f0cfa28d-bba4-4e59-80a9-55145e2e51f8" containerName="main"
Apr 24 15:05:27.745757 ip-10-0-136-132 kubenswrapper[2574]: I0424 15:05:27.745656    2574 memory_manager.go:356] "RemoveStaleState removing state" podUID="7a613b84-cbb1-4ab8-bc7c-2187ccaae8e6" containerName="llm-d-routing-sidecar"
Apr 24 15:05:27.745757 ip-10-0-136-132 kubenswrapper[2574]: I0424 15:05:27.745667    2574 memory_manager.go:356] "RemoveStaleState removing state" podUID="4a48929f-f66c-49b8-9d3d-08c90eb4c1c6" containerName="main"
Apr 24 15:05:27.745757 ip-10-0-136-132 kubenswrapper[2574]: I0424 15:05:27.745674    2574 memory_manager.go:356] "RemoveStaleState removing state" podUID="7a613b84-cbb1-4ab8-bc7c-2187ccaae8e6" containerName="main"
Apr 24 15:05:27.748691 ip-10-0-136-132 kubenswrapper[2574]: I0424 15:05:27.748675    2574 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-rpb9c/perf-node-gather-daemonset-fmz4h"
Apr 24 15:05:27.750466 ip-10-0-136-132 kubenswrapper[2574]: I0424 15:05:27.750445    2574 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-must-gather-rpb9c\"/\"default-dockercfg-rtxl9\""
Apr 24 15:05:27.750953 ip-10-0-136-132 kubenswrapper[2574]: I0424 15:05:27.750932    2574 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-rpb9c\"/\"openshift-service-ca.crt\""
Apr 24 15:05:27.751193 ip-10-0-136-132 kubenswrapper[2574]: I0424 15:05:27.751177    2574 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-rpb9c\"/\"kube-root-ca.crt\""
Apr 24 15:05:27.755838 ip-10-0-136-132 kubenswrapper[2574]: I0424 15:05:27.755618    2574 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-rpb9c/perf-node-gather-daemonset-fmz4h"]
Apr 24 15:05:27.843286 ip-10-0-136-132 kubenswrapper[2574]: I0424 15:05:27.843252    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/aef9c7f3-26ce-4e4c-af0b-6211f9383a62-lib-modules\") pod \"perf-node-gather-daemonset-fmz4h\" (UID: \"aef9c7f3-26ce-4e4c-af0b-6211f9383a62\") " pod="openshift-must-gather-rpb9c/perf-node-gather-daemonset-fmz4h"
Apr 24 15:05:27.843466 ip-10-0-136-132 kubenswrapper[2574]: I0424 15:05:27.843309    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/aef9c7f3-26ce-4e4c-af0b-6211f9383a62-proc\") pod \"perf-node-gather-daemonset-fmz4h\" (UID: \"aef9c7f3-26ce-4e4c-af0b-6211f9383a62\") " pod="openshift-must-gather-rpb9c/perf-node-gather-daemonset-fmz4h"
Apr 24 15:05:27.843466 ip-10-0-136-132 kubenswrapper[2574]: I0424 15:05:27.843409    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-f57lq\" (UniqueName: \"kubernetes.io/projected/aef9c7f3-26ce-4e4c-af0b-6211f9383a62-kube-api-access-f57lq\") pod \"perf-node-gather-daemonset-fmz4h\" (UID: \"aef9c7f3-26ce-4e4c-af0b-6211f9383a62\") " pod="openshift-must-gather-rpb9c/perf-node-gather-daemonset-fmz4h"
Apr 24 15:05:27.843466 ip-10-0-136-132 kubenswrapper[2574]: I0424 15:05:27.843446    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/aef9c7f3-26ce-4e4c-af0b-6211f9383a62-sys\") pod \"perf-node-gather-daemonset-fmz4h\" (UID: \"aef9c7f3-26ce-4e4c-af0b-6211f9383a62\") " pod="openshift-must-gather-rpb9c/perf-node-gather-daemonset-fmz4h"
Apr 24 15:05:27.843624 ip-10-0-136-132 kubenswrapper[2574]: I0424 15:05:27.843482    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/aef9c7f3-26ce-4e4c-af0b-6211f9383a62-podres\") pod \"perf-node-gather-daemonset-fmz4h\" (UID: \"aef9c7f3-26ce-4e4c-af0b-6211f9383a62\") " pod="openshift-must-gather-rpb9c/perf-node-gather-daemonset-fmz4h"
Apr 24 15:05:27.944514 ip-10-0-136-132 kubenswrapper[2574]: I0424 15:05:27.944475    2574 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-f57lq\" (UniqueName: \"kubernetes.io/projected/aef9c7f3-26ce-4e4c-af0b-6211f9383a62-kube-api-access-f57lq\") pod \"perf-node-gather-daemonset-fmz4h\" (UID: \"aef9c7f3-26ce-4e4c-af0b-6211f9383a62\") " pod="openshift-must-gather-rpb9c/perf-node-gather-daemonset-fmz4h"
Apr 24 15:05:27.944514 ip-10-0-136-132 kubenswrapper[2574]: I0424 15:05:27.944516    2574 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/aef9c7f3-26ce-4e4c-af0b-6211f9383a62-sys\") pod \"perf-node-gather-daemonset-fmz4h\" (UID: \"aef9c7f3-26ce-4e4c-af0b-6211f9383a62\") " pod="openshift-must-gather-rpb9c/perf-node-gather-daemonset-fmz4h"
Apr 24 15:05:27.944718 ip-10-0-136-132 kubenswrapper[2574]: I0424 15:05:27.944541    2574 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/aef9c7f3-26ce-4e4c-af0b-6211f9383a62-podres\") pod \"perf-node-gather-daemonset-fmz4h\" (UID: \"aef9c7f3-26ce-4e4c-af0b-6211f9383a62\") " pod="openshift-must-gather-rpb9c/perf-node-gather-daemonset-fmz4h"
Apr 24 15:05:27.944718 ip-10-0-136-132 kubenswrapper[2574]: I0424 15:05:27.944582    2574 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/aef9c7f3-26ce-4e4c-af0b-6211f9383a62-lib-modules\") pod \"perf-node-gather-daemonset-fmz4h\" (UID: \"aef9c7f3-26ce-4e4c-af0b-6211f9383a62\") " pod="openshift-must-gather-rpb9c/perf-node-gather-daemonset-fmz4h"
Apr 24 15:05:27.944718 ip-10-0-136-132 kubenswrapper[2574]: I0424 15:05:27.944609    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/aef9c7f3-26ce-4e4c-af0b-6211f9383a62-sys\") pod \"perf-node-gather-daemonset-fmz4h\" (UID: \"aef9c7f3-26ce-4e4c-af0b-6211f9383a62\") " pod="openshift-must-gather-rpb9c/perf-node-gather-daemonset-fmz4h"
Apr 24 15:05:27.944718 ip-10-0-136-132 kubenswrapper[2574]: I0424 15:05:27.944635    2574 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/aef9c7f3-26ce-4e4c-af0b-6211f9383a62-proc\") pod \"perf-node-gather-daemonset-fmz4h\" (UID: \"aef9c7f3-26ce-4e4c-af0b-6211f9383a62\") " pod="openshift-must-gather-rpb9c/perf-node-gather-daemonset-fmz4h"
Apr 24 15:05:27.944718 ip-10-0-136-132 kubenswrapper[2574]: I0424 15:05:27.944680    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/aef9c7f3-26ce-4e4c-af0b-6211f9383a62-proc\") pod \"perf-node-gather-daemonset-fmz4h\" (UID: \"aef9c7f3-26ce-4e4c-af0b-6211f9383a62\") " pod="openshift-must-gather-rpb9c/perf-node-gather-daemonset-fmz4h"
Apr 24 15:05:27.944890 ip-10-0-136-132 kubenswrapper[2574]: I0424 15:05:27.944742    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/aef9c7f3-26ce-4e4c-af0b-6211f9383a62-lib-modules\") pod \"perf-node-gather-daemonset-fmz4h\" (UID: \"aef9c7f3-26ce-4e4c-af0b-6211f9383a62\") " pod="openshift-must-gather-rpb9c/perf-node-gather-daemonset-fmz4h"
Apr 24 15:05:27.944890 ip-10-0-136-132 kubenswrapper[2574]: I0424 15:05:27.944747    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/aef9c7f3-26ce-4e4c-af0b-6211f9383a62-podres\") pod \"perf-node-gather-daemonset-fmz4h\" (UID: \"aef9c7f3-26ce-4e4c-af0b-6211f9383a62\") " pod="openshift-must-gather-rpb9c/perf-node-gather-daemonset-fmz4h"
Apr 24 15:05:27.951534 ip-10-0-136-132 kubenswrapper[2574]: I0424 15:05:27.951511    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-f57lq\" (UniqueName: \"kubernetes.io/projected/aef9c7f3-26ce-4e4c-af0b-6211f9383a62-kube-api-access-f57lq\") pod \"perf-node-gather-daemonset-fmz4h\" (UID: \"aef9c7f3-26ce-4e4c-af0b-6211f9383a62\") " pod="openshift-must-gather-rpb9c/perf-node-gather-daemonset-fmz4h"
Apr 24 15:05:27.989328 ip-10-0-136-132 kubenswrapper[2574]: I0424 15:05:27.989302    2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-nvz9x_b4b7957c-9f4f-4c2d-ab2b-85c958db734a/dns/0.log"
Apr 24 15:05:28.004173 ip-10-0-136-132 kubenswrapper[2574]: I0424 15:05:28.004096    2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-nvz9x_b4b7957c-9f4f-4c2d-ab2b-85c958db734a/kube-rbac-proxy/0.log"
Apr 24 15:05:28.059128 ip-10-0-136-132 kubenswrapper[2574]: I0424 15:05:28.059100    2574 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-rpb9c/perf-node-gather-daemonset-fmz4h"
Apr 24 15:05:28.129085 ip-10-0-136-132 kubenswrapper[2574]: I0424 15:05:28.129059    2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-bt99l_0b8fb5e7-eaa1-4b51-9f84-37bb0eb1a711/dns-node-resolver/0.log"
Apr 24 15:05:28.186584 ip-10-0-136-132 kubenswrapper[2574]: I0424 15:05:28.186554    2574 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-rpb9c/perf-node-gather-daemonset-fmz4h"]
Apr 24 15:05:28.188453 ip-10-0-136-132 kubenswrapper[2574]: W0424 15:05:28.188416    2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-podaef9c7f3_26ce_4e4c_af0b_6211f9383a62.slice/crio-26f01631377c598f37253d520b8e52577fed6e8bae49301229b23056e699104c WatchSource:0}: Error finding container 26f01631377c598f37253d520b8e52577fed6e8bae49301229b23056e699104c: Status 404 returned error can't find the container with id 26f01631377c598f37253d520b8e52577fed6e8bae49301229b23056e699104c
Apr 24 15:05:28.190063 ip-10-0-136-132 kubenswrapper[2574]: I0424 15:05:28.190039    2574 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Apr 24 15:05:28.606541 ip-10-0-136-132 kubenswrapper[2574]: I0424 15:05:28.606518    2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-lgh9n_5e4a7a90-d6ce-491b-ad65-52f0b2a127d6/node-ca/0.log"
Apr 24 15:05:28.769521 ip-10-0-136-132 kubenswrapper[2574]: I0424 15:05:28.769483    2574 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-rpb9c/perf-node-gather-daemonset-fmz4h" event={"ID":"aef9c7f3-26ce-4e4c-af0b-6211f9383a62","Type":"ContainerStarted","Data":"81a55d262905d59ef6a69ecd97dd20b612a881438e163689cfd19a749bcd0ab5"}
Apr 24 15:05:28.769521 ip-10-0-136-132 kubenswrapper[2574]: I0424 15:05:28.769521    2574 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-rpb9c/perf-node-gather-daemonset-fmz4h" event={"ID":"aef9c7f3-26ce-4e4c-af0b-6211f9383a62","Type":"ContainerStarted","Data":"26f01631377c598f37253d520b8e52577fed6e8bae49301229b23056e699104c"}
Apr 24 15:05:28.769740 ip-10-0-136-132 kubenswrapper[2574]: I0424 15:05:28.769605    2574 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-must-gather-rpb9c/perf-node-gather-daemonset-fmz4h"
Apr 24 15:05:28.784125 ip-10-0-136-132 kubenswrapper[2574]: I0424 15:05:28.784081    2574 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-must-gather-rpb9c/perf-node-gather-daemonset-fmz4h" podStartSLOduration=1.784070808 podStartE2EDuration="1.784070808s" podCreationTimestamp="2026-04-24 15:05:27 +0000 UTC" firstStartedPulling="0001-01-01 00:00:00 +0000 UTC" lastFinishedPulling="0001-01-01 00:00:00 +0000 UTC" observedRunningTime="2026-04-24 15:05:28.781928489 +0000 UTC m=+2491.514030137" watchObservedRunningTime="2026-04-24 15:05:28.784070808 +0000 UTC m=+2491.516172456"
Apr 24 15:05:29.871867 ip-10-0-136-132 kubenswrapper[2574]: I0424 15:05:29.871835    2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-tzbfv_4ed72386-1812-4697-9811-c3534a37dc95/serve-healthcheck-canary/0.log"
Apr 24 15:05:30.289497 ip-10-0-136-132 kubenswrapper[2574]: I0424 15:05:30.289403    2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-qgxfn_660bbad4-5502-418e-a02d-869defb74468/kube-rbac-proxy/0.log"
Apr 24 15:05:30.304281 ip-10-0-136-132 kubenswrapper[2574]: I0424 15:05:30.304256    2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-qgxfn_660bbad4-5502-418e-a02d-869defb74468/exporter/0.log"
Apr 24 15:05:30.319729 ip-10-0-136-132 kubenswrapper[2574]: I0424 15:05:30.319700    2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-qgxfn_660bbad4-5502-418e-a02d-869defb74468/extractor/0.log"
Apr 24 15:05:32.900413 ip-10-0-136-132 kubenswrapper[2574]: I0424 15:05:32.900380    2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-lws-operator_lws-controller-manager-6d967bb649-nc7mb_73112bb2-b649-468e-97cd-22458526fe47/manager/0.log"
Apr 24 15:05:34.782695 ip-10-0-136-132 kubenswrapper[2574]: I0424 15:05:34.782658    2574 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-must-gather-rpb9c/perf-node-gather-daemonset-fmz4h"
Apr 24 15:05:38.008101 ip-10-0-136-132 kubenswrapper[2574]: I0424 15:05:38.008072    2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-kube-storage-version-migrator_migrator-74bb7799d9-xcnvv_a1182186-de0f-42da-82d2-92fa7ed16130/migrator/0.log"
Apr 24 15:05:38.023549 ip-10-0-136-132 kubenswrapper[2574]: I0424 15:05:38.023525    2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-kube-storage-version-migrator_migrator-74bb7799d9-xcnvv_a1182186-de0f-42da-82d2-92fa7ed16130/graceful-termination/0.log"
Apr 24 15:05:38.317952 ip-10-0-136-132 kubenswrapper[2574]: I0424 15:05:38.317868    2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-kube-storage-version-migrator-operator_kube-storage-version-migrator-operator-6769c5d45-rmc8n_5bc2f6f9-dff9-48b0-bc63-5f778a1ed590/kube-storage-version-migrator-operator/1.log"
Apr 24 15:05:38.318921 ip-10-0-136-132 kubenswrapper[2574]: I0424 15:05:38.318889    2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-kube-storage-version-migrator-operator_kube-storage-version-migrator-operator-6769c5d45-rmc8n_5bc2f6f9-dff9-48b0-bc63-5f778a1ed590/kube-storage-version-migrator-operator/0.log"
Apr 24 15:05:39.462131 ip-10-0-136-132 kubenswrapper[2574]: I0424 15:05:39.462047    2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-j2jc7_c9814dbf-8db1-4d4a-9d40-d7a0fdc96b26/kube-multus-additional-cni-plugins/0.log"
Apr 24 15:05:39.478696 ip-10-0-136-132 kubenswrapper[2574]: I0424 15:05:39.478665    2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-j2jc7_c9814dbf-8db1-4d4a-9d40-d7a0fdc96b26/egress-router-binary-copy/0.log"
Apr 24 15:05:39.496143 ip-10-0-136-132 kubenswrapper[2574]: I0424 15:05:39.496124    2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-j2jc7_c9814dbf-8db1-4d4a-9d40-d7a0fdc96b26/cni-plugins/0.log"
Apr 24 15:05:39.512502 ip-10-0-136-132 kubenswrapper[2574]: I0424 15:05:39.512483    2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-j2jc7_c9814dbf-8db1-4d4a-9d40-d7a0fdc96b26/bond-cni-plugin/0.log"
Apr 24 15:05:39.528774 ip-10-0-136-132 kubenswrapper[2574]: I0424 15:05:39.528755    2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-j2jc7_c9814dbf-8db1-4d4a-9d40-d7a0fdc96b26/routeoverride-cni/0.log"
Apr 24 15:05:39.544647 ip-10-0-136-132 kubenswrapper[2574]: I0424 15:05:39.544628    2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-j2jc7_c9814dbf-8db1-4d4a-9d40-d7a0fdc96b26/whereabouts-cni-bincopy/0.log"
Apr 24 15:05:39.562652 ip-10-0-136-132 kubenswrapper[2574]: I0424 15:05:39.562635    2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-j2jc7_c9814dbf-8db1-4d4a-9d40-d7a0fdc96b26/whereabouts-cni/0.log"
Apr 24 15:05:39.791972 ip-10-0-136-132 kubenswrapper[2574]: I0424 15:05:39.791900    2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-rwdxf_e87608cb-741d-47f5-bb29-fc68c0dc8bc4/kube-multus/0.log"
Apr 24 15:05:39.882649 ip-10-0-136-132 kubenswrapper[2574]: I0424 15:05:39.882622    2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_network-metrics-daemon-zhkmq_3bc44c73-963f-4a9e-8a52-1e8b031a08dc/network-metrics-daemon/0.log"
Apr 24 15:05:39.898405 ip-10-0-136-132 kubenswrapper[2574]: I0424 15:05:39.898380    2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_network-metrics-daemon-zhkmq_3bc44c73-963f-4a9e-8a52-1e8b031a08dc/kube-rbac-proxy/0.log"
Apr 24 15:05:41.165560 ip-10-0-136-132 kubenswrapper[2574]: I0424 15:05:41.165494    2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-w5x5x_44036c3e-1fdc-40d4-899c-9009cd4589e3/ovn-controller/0.log"
Apr 24 15:05:41.178518 ip-10-0-136-132 kubenswrapper[2574]: I0424 15:05:41.178494    2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-w5x5x_44036c3e-1fdc-40d4-899c-9009cd4589e3/ovn-acl-logging/0.log"
Apr 24 15:05:41.188755 ip-10-0-136-132 kubenswrapper[2574]: I0424 15:05:41.188736    2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-w5x5x_44036c3e-1fdc-40d4-899c-9009cd4589e3/ovn-acl-logging/1.log"
Apr 24 15:05:41.204429 ip-10-0-136-132 kubenswrapper[2574]: I0424 15:05:41.204393    2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-w5x5x_44036c3e-1fdc-40d4-899c-9009cd4589e3/kube-rbac-proxy-node/0.log"
Apr 24 15:05:41.222270 ip-10-0-136-132 kubenswrapper[2574]: I0424 15:05:41.222251    2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-w5x5x_44036c3e-1fdc-40d4-899c-9009cd4589e3/kube-rbac-proxy-ovn-metrics/0.log"
Apr 24 15:05:41.237638 ip-10-0-136-132 kubenswrapper[2574]: I0424 15:05:41.237611    2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-w5x5x_44036c3e-1fdc-40d4-899c-9009cd4589e3/northd/0.log"
Apr 24 15:05:41.252024 ip-10-0-136-132 kubenswrapper[2574]: I0424 15:05:41.252000    2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-w5x5x_44036c3e-1fdc-40d4-899c-9009cd4589e3/nbdb/0.log"
Apr 24 15:05:41.270016 ip-10-0-136-132 kubenswrapper[2574]: I0424 15:05:41.269992    2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-w5x5x_44036c3e-1fdc-40d4-899c-9009cd4589e3/sbdb/0.log"
Apr 24 15:05:41.372950 ip-10-0-136-132 kubenswrapper[2574]: I0424 15:05:41.372918    2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-w5x5x_44036c3e-1fdc-40d4-899c-9009cd4589e3/ovnkube-controller/0.log"
Apr 24 15:05:42.400390 ip-10-0-136-132 kubenswrapper[2574]: I0424 15:05:42.400362    2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-network-diagnostics_network-check-source-8894fc9bd-kj5df_c429617c-d32e-4466-b249-d0a28cfd5b9d/check-endpoints/0.log"
Apr 24 15:05:42.417438 ip-10-0-136-132 kubenswrapper[2574]: I0424 15:05:42.417410    2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-network-diagnostics_network-check-target-6tsjh_3964207c-9c58-4994-a8a9-aa4e4feca773/network-check-target-container/0.log"
Apr 24 15:05:43.384557 ip-10-0-136-132 kubenswrapper[2574]: I0424 15:05:43.384531    2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-network-operator_iptables-alerter-r2pxq_95f221a8-f16d-4d5f-9638-29db9dd3a969/iptables-alerter/0.log"
Apr 24 15:05:44.051404 ip-10-0-136-132 kubenswrapper[2574]: I0424 15:05:44.051375    2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-cluster-node-tuning-operator_tuned-4kqkj_21ff8454-68c7-440e-9994-e8405b47919a/tuned/0.log"