Sep 17 17:34:08.756292 ip-10-0-137-20 systemd[1]: Starting Kubernetes Kubelet...
Sep 17 17:34:11.937409 ip-10-0-137-20 kubenswrapper[2565]: Flag --container-runtime-endpoint has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Sep 17 17:34:11.937409 ip-10-0-137-20 kubenswrapper[2565]: Flag --minimum-container-ttl-duration has been deprecated, Use --eviction-hard or --eviction-soft instead. Will be removed in a future version.
Sep 17 17:34:11.937409 ip-10-0-137-20 kubenswrapper[2565]: Flag --volume-plugin-dir has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Sep 17 17:34:11.937409 ip-10-0-137-20 kubenswrapper[2565]: Flag --pod-infra-container-image has been deprecated, will be removed in 1.35. Image garbage collector will get sandbox image information from CRI.
Sep 17 17:34:11.937409 ip-10-0-137-20 kubenswrapper[2565]: Flag --system-reserved has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Sep 17 17:34:11.938096 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:34:11.937486    2565 server.go:213] "--pod-infra-container-image will not be pruned by the image garbage collector in kubelet and should also be set in the remote runtime"
Sep 17 17:34:11.942928 ip-10-0-137-20 kubenswrapper[2565]: W0917 17:34:11.942906    2565 feature_gate.go:328] unrecognized feature gate: SignatureStores
Sep 17 17:34:11.942928 ip-10-0-137-20 kubenswrapper[2565]: W0917 17:34:11.942925    2565 feature_gate.go:328] unrecognized feature gate: PinnedImages
Sep 17 17:34:11.942928 ip-10-0-137-20 kubenswrapper[2565]: W0917 17:34:11.942929    2565 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Sep 17 17:34:11.942928 ip-10-0-137-20 kubenswrapper[2565]: W0917 17:34:11.942933    2565 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Sep 17 17:34:11.943110 ip-10-0-137-20 kubenswrapper[2565]: W0917 17:34:11.942936    2565 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Sep 17 17:34:11.943110 ip-10-0-137-20 kubenswrapper[2565]: W0917 17:34:11.942940    2565 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Sep 17 17:34:11.943110 ip-10-0-137-20 kubenswrapper[2565]: W0917 17:34:11.942943    2565 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Sep 17 17:34:11.943110 ip-10-0-137-20 kubenswrapper[2565]: W0917 17:34:11.942946    2565 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Sep 17 17:34:11.943110 ip-10-0-137-20 kubenswrapper[2565]: W0917 17:34:11.942949    2565 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Sep 17 17:34:11.943110 ip-10-0-137-20 kubenswrapper[2565]: W0917 17:34:11.942952    2565 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Sep 17 17:34:11.943110 ip-10-0-137-20 kubenswrapper[2565]: W0917 17:34:11.942954    2565 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Sep 17 17:34:11.943110 ip-10-0-137-20 kubenswrapper[2565]: W0917 17:34:11.942957    2565 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Sep 17 17:34:11.943110 ip-10-0-137-20 kubenswrapper[2565]: W0917 17:34:11.942960    2565 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Sep 17 17:34:11.943110 ip-10-0-137-20 kubenswrapper[2565]: W0917 17:34:11.942963    2565 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Sep 17 17:34:11.943110 ip-10-0-137-20 kubenswrapper[2565]: W0917 17:34:11.942965    2565 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Sep 17 17:34:11.943110 ip-10-0-137-20 kubenswrapper[2565]: W0917 17:34:11.942968    2565 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Sep 17 17:34:11.943110 ip-10-0-137-20 kubenswrapper[2565]: W0917 17:34:11.942970    2565 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Sep 17 17:34:11.943110 ip-10-0-137-20 kubenswrapper[2565]: W0917 17:34:11.942973    2565 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Sep 17 17:34:11.943110 ip-10-0-137-20 kubenswrapper[2565]: W0917 17:34:11.942975    2565 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Sep 17 17:34:11.943110 ip-10-0-137-20 kubenswrapper[2565]: W0917 17:34:11.942978    2565 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Sep 17 17:34:11.943110 ip-10-0-137-20 kubenswrapper[2565]: W0917 17:34:11.942981    2565 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Sep 17 17:34:11.943110 ip-10-0-137-20 kubenswrapper[2565]: W0917 17:34:11.942983    2565 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Sep 17 17:34:11.943110 ip-10-0-137-20 kubenswrapper[2565]: W0917 17:34:11.942986    2565 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Sep 17 17:34:11.943110 ip-10-0-137-20 kubenswrapper[2565]: W0917 17:34:11.942988    2565 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Sep 17 17:34:11.943630 ip-10-0-137-20 kubenswrapper[2565]: W0917 17:34:11.942991    2565 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Sep 17 17:34:11.943630 ip-10-0-137-20 kubenswrapper[2565]: W0917 17:34:11.942994    2565 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Sep 17 17:34:11.943630 ip-10-0-137-20 kubenswrapper[2565]: W0917 17:34:11.942996    2565 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Sep 17 17:34:11.943630 ip-10-0-137-20 kubenswrapper[2565]: W0917 17:34:11.942999    2565 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Sep 17 17:34:11.943630 ip-10-0-137-20 kubenswrapper[2565]: W0917 17:34:11.943005    2565 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Sep 17 17:34:11.943630 ip-10-0-137-20 kubenswrapper[2565]: W0917 17:34:11.943008    2565 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Sep 17 17:34:11.943630 ip-10-0-137-20 kubenswrapper[2565]: W0917 17:34:11.943011    2565 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Sep 17 17:34:11.943630 ip-10-0-137-20 kubenswrapper[2565]: W0917 17:34:11.943013    2565 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Sep 17 17:34:11.943630 ip-10-0-137-20 kubenswrapper[2565]: W0917 17:34:11.943016    2565 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Sep 17 17:34:11.943630 ip-10-0-137-20 kubenswrapper[2565]: W0917 17:34:11.943031    2565 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Sep 17 17:34:11.943630 ip-10-0-137-20 kubenswrapper[2565]: W0917 17:34:11.943034    2565 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Sep 17 17:34:11.943630 ip-10-0-137-20 kubenswrapper[2565]: W0917 17:34:11.943037    2565 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Sep 17 17:34:11.943630 ip-10-0-137-20 kubenswrapper[2565]: W0917 17:34:11.943039    2565 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Sep 17 17:34:11.943630 ip-10-0-137-20 kubenswrapper[2565]: W0917 17:34:11.943043    2565 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Sep 17 17:34:11.943630 ip-10-0-137-20 kubenswrapper[2565]: W0917 17:34:11.943046    2565 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Sep 17 17:34:11.943630 ip-10-0-137-20 kubenswrapper[2565]: W0917 17:34:11.943049    2565 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Sep 17 17:34:11.943630 ip-10-0-137-20 kubenswrapper[2565]: W0917 17:34:11.943051    2565 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Sep 17 17:34:11.943630 ip-10-0-137-20 kubenswrapper[2565]: W0917 17:34:11.943054    2565 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Sep 17 17:34:11.943630 ip-10-0-137-20 kubenswrapper[2565]: W0917 17:34:11.943057    2565 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Sep 17 17:34:11.944116 ip-10-0-137-20 kubenswrapper[2565]: W0917 17:34:11.943059    2565 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Sep 17 17:34:11.944116 ip-10-0-137-20 kubenswrapper[2565]: W0917 17:34:11.943062    2565 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Sep 17 17:34:11.944116 ip-10-0-137-20 kubenswrapper[2565]: W0917 17:34:11.943065    2565 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Sep 17 17:34:11.944116 ip-10-0-137-20 kubenswrapper[2565]: W0917 17:34:11.943069    2565 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Sep 17 17:34:11.944116 ip-10-0-137-20 kubenswrapper[2565]: W0917 17:34:11.943072    2565 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Sep 17 17:34:11.944116 ip-10-0-137-20 kubenswrapper[2565]: W0917 17:34:11.943075    2565 feature_gate.go:328] unrecognized feature gate: EventTTL
Sep 17 17:34:11.944116 ip-10-0-137-20 kubenswrapper[2565]: W0917 17:34:11.943077    2565 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Sep 17 17:34:11.944116 ip-10-0-137-20 kubenswrapper[2565]: W0917 17:34:11.943082    2565 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Sep 17 17:34:11.944116 ip-10-0-137-20 kubenswrapper[2565]: W0917 17:34:11.943085    2565 feature_gate.go:328] unrecognized feature gate: OSStreams
Sep 17 17:34:11.944116 ip-10-0-137-20 kubenswrapper[2565]: W0917 17:34:11.943089    2565 feature_gate.go:328] unrecognized feature gate: OVNObservability
Sep 17 17:34:11.944116 ip-10-0-137-20 kubenswrapper[2565]: W0917 17:34:11.943091    2565 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Sep 17 17:34:11.944116 ip-10-0-137-20 kubenswrapper[2565]: W0917 17:34:11.943094    2565 feature_gate.go:328] unrecognized feature gate: Example2
Sep 17 17:34:11.944116 ip-10-0-137-20 kubenswrapper[2565]: W0917 17:34:11.943097    2565 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Sep 17 17:34:11.944116 ip-10-0-137-20 kubenswrapper[2565]: W0917 17:34:11.943100    2565 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Sep 17 17:34:11.944116 ip-10-0-137-20 kubenswrapper[2565]: W0917 17:34:11.943103    2565 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Sep 17 17:34:11.944116 ip-10-0-137-20 kubenswrapper[2565]: W0917 17:34:11.943105    2565 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Sep 17 17:34:11.944116 ip-10-0-137-20 kubenswrapper[2565]: W0917 17:34:11.943108    2565 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Sep 17 17:34:11.944116 ip-10-0-137-20 kubenswrapper[2565]: W0917 17:34:11.943111    2565 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Sep 17 17:34:11.944116 ip-10-0-137-20 kubenswrapper[2565]: W0917 17:34:11.943113    2565 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Sep 17 17:34:11.944116 ip-10-0-137-20 kubenswrapper[2565]: W0917 17:34:11.943116    2565 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Sep 17 17:34:11.944596 ip-10-0-137-20 kubenswrapper[2565]: W0917 17:34:11.943119    2565 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Sep 17 17:34:11.944596 ip-10-0-137-20 kubenswrapper[2565]: W0917 17:34:11.943121    2565 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Sep 17 17:34:11.944596 ip-10-0-137-20 kubenswrapper[2565]: W0917 17:34:11.943124    2565 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Sep 17 17:34:11.944596 ip-10-0-137-20 kubenswrapper[2565]: W0917 17:34:11.943127    2565 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Sep 17 17:34:11.944596 ip-10-0-137-20 kubenswrapper[2565]: W0917 17:34:11.943130    2565 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Sep 17 17:34:11.944596 ip-10-0-137-20 kubenswrapper[2565]: W0917 17:34:11.943132    2565 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Sep 17 17:34:11.944596 ip-10-0-137-20 kubenswrapper[2565]: W0917 17:34:11.943136    2565 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Sep 17 17:34:11.944596 ip-10-0-137-20 kubenswrapper[2565]: W0917 17:34:11.943139    2565 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Sep 17 17:34:11.944596 ip-10-0-137-20 kubenswrapper[2565]: W0917 17:34:11.943142    2565 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Sep 17 17:34:11.944596 ip-10-0-137-20 kubenswrapper[2565]: W0917 17:34:11.943145    2565 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Sep 17 17:34:11.944596 ip-10-0-137-20 kubenswrapper[2565]: W0917 17:34:11.943147    2565 feature_gate.go:328] unrecognized feature gate: DualReplica
Sep 17 17:34:11.944596 ip-10-0-137-20 kubenswrapper[2565]: W0917 17:34:11.943150    2565 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Sep 17 17:34:11.944596 ip-10-0-137-20 kubenswrapper[2565]: W0917 17:34:11.943152    2565 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Sep 17 17:34:11.944596 ip-10-0-137-20 kubenswrapper[2565]: W0917 17:34:11.943155    2565 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Sep 17 17:34:11.944596 ip-10-0-137-20 kubenswrapper[2565]: W0917 17:34:11.943158    2565 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Sep 17 17:34:11.944596 ip-10-0-137-20 kubenswrapper[2565]: W0917 17:34:11.943161    2565 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Sep 17 17:34:11.944596 ip-10-0-137-20 kubenswrapper[2565]: W0917 17:34:11.943164    2565 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Sep 17 17:34:11.944596 ip-10-0-137-20 kubenswrapper[2565]: W0917 17:34:11.943166    2565 feature_gate.go:328] unrecognized feature gate: Example
Sep 17 17:34:11.944596 ip-10-0-137-20 kubenswrapper[2565]: W0917 17:34:11.943169    2565 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Sep 17 17:34:11.944596 ip-10-0-137-20 kubenswrapper[2565]: W0917 17:34:11.943172    2565 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Sep 17 17:34:11.945098 ip-10-0-137-20 kubenswrapper[2565]: W0917 17:34:11.943175    2565 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Sep 17 17:34:11.945098 ip-10-0-137-20 kubenswrapper[2565]: W0917 17:34:11.943178    2565 feature_gate.go:328] unrecognized feature gate: NewOLM
Sep 17 17:34:11.945098 ip-10-0-137-20 kubenswrapper[2565]: W0917 17:34:11.943181    2565 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Sep 17 17:34:11.945098 ip-10-0-137-20 kubenswrapper[2565]: W0917 17:34:11.943183    2565 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Sep 17 17:34:11.945098 ip-10-0-137-20 kubenswrapper[2565]: W0917 17:34:11.943187    2565 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Sep 17 17:34:11.945098 ip-10-0-137-20 kubenswrapper[2565]: W0917 17:34:11.943191    2565 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Sep 17 17:34:11.945098 ip-10-0-137-20 kubenswrapper[2565]: W0917 17:34:11.943195    2565 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Sep 17 17:34:11.945098 ip-10-0-137-20 kubenswrapper[2565]: W0917 17:34:11.943198    2565 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Sep 17 17:34:11.945098 ip-10-0-137-20 kubenswrapper[2565]: W0917 17:34:11.943201    2565 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Sep 17 17:34:11.945098 ip-10-0-137-20 kubenswrapper[2565]: W0917 17:34:11.943204    2565 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Sep 17 17:34:11.945098 ip-10-0-137-20 kubenswrapper[2565]: W0917 17:34:11.943206    2565 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Sep 17 17:34:11.945098 ip-10-0-137-20 kubenswrapper[2565]: W0917 17:34:11.943209    2565 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Sep 17 17:34:11.945098 ip-10-0-137-20 kubenswrapper[2565]: W0917 17:34:11.943211    2565 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Sep 17 17:34:11.945098 ip-10-0-137-20 kubenswrapper[2565]: W0917 17:34:11.943214    2565 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Sep 17 17:34:11.945098 ip-10-0-137-20 kubenswrapper[2565]: W0917 17:34:11.943217    2565 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Sep 17 17:34:11.945098 ip-10-0-137-20 kubenswrapper[2565]: W0917 17:34:11.943219    2565 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Sep 17 17:34:11.945098 ip-10-0-137-20 kubenswrapper[2565]: W0917 17:34:11.943903    2565 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Sep 17 17:34:11.945098 ip-10-0-137-20 kubenswrapper[2565]: W0917 17:34:11.943911    2565 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Sep 17 17:34:11.945098 ip-10-0-137-20 kubenswrapper[2565]: W0917 17:34:11.943916    2565 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Sep 17 17:34:11.945543 ip-10-0-137-20 kubenswrapper[2565]: W0917 17:34:11.943920    2565 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Sep 17 17:34:11.945543 ip-10-0-137-20 kubenswrapper[2565]: W0917 17:34:11.943923    2565 feature_gate.go:328] unrecognized feature gate: Example
Sep 17 17:34:11.945543 ip-10-0-137-20 kubenswrapper[2565]: W0917 17:34:11.943926    2565 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Sep 17 17:34:11.945543 ip-10-0-137-20 kubenswrapper[2565]: W0917 17:34:11.943928    2565 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Sep 17 17:34:11.945543 ip-10-0-137-20 kubenswrapper[2565]: W0917 17:34:11.943931    2565 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Sep 17 17:34:11.945543 ip-10-0-137-20 kubenswrapper[2565]: W0917 17:34:11.943934    2565 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Sep 17 17:34:11.945543 ip-10-0-137-20 kubenswrapper[2565]: W0917 17:34:11.943936    2565 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Sep 17 17:34:11.945543 ip-10-0-137-20 kubenswrapper[2565]: W0917 17:34:11.943939    2565 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Sep 17 17:34:11.945543 ip-10-0-137-20 kubenswrapper[2565]: W0917 17:34:11.943941    2565 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Sep 17 17:34:11.945543 ip-10-0-137-20 kubenswrapper[2565]: W0917 17:34:11.943944    2565 feature_gate.go:328] unrecognized feature gate: DualReplica
Sep 17 17:34:11.945543 ip-10-0-137-20 kubenswrapper[2565]: W0917 17:34:11.943947    2565 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Sep 17 17:34:11.945543 ip-10-0-137-20 kubenswrapper[2565]: W0917 17:34:11.943949    2565 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Sep 17 17:34:11.945543 ip-10-0-137-20 kubenswrapper[2565]: W0917 17:34:11.943952    2565 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Sep 17 17:34:11.945543 ip-10-0-137-20 kubenswrapper[2565]: W0917 17:34:11.943956    2565 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Sep 17 17:34:11.945543 ip-10-0-137-20 kubenswrapper[2565]: W0917 17:34:11.943960    2565 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Sep 17 17:34:11.945543 ip-10-0-137-20 kubenswrapper[2565]: W0917 17:34:11.943963    2565 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Sep 17 17:34:11.945543 ip-10-0-137-20 kubenswrapper[2565]: W0917 17:34:11.943966    2565 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Sep 17 17:34:11.945543 ip-10-0-137-20 kubenswrapper[2565]: W0917 17:34:11.943969    2565 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Sep 17 17:34:11.945543 ip-10-0-137-20 kubenswrapper[2565]: W0917 17:34:11.943971    2565 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Sep 17 17:34:11.945543 ip-10-0-137-20 kubenswrapper[2565]: W0917 17:34:11.943974    2565 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Sep 17 17:34:11.946045 ip-10-0-137-20 kubenswrapper[2565]: W0917 17:34:11.943977    2565 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Sep 17 17:34:11.946045 ip-10-0-137-20 kubenswrapper[2565]: W0917 17:34:11.943979    2565 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Sep 17 17:34:11.946045 ip-10-0-137-20 kubenswrapper[2565]: W0917 17:34:11.943982    2565 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Sep 17 17:34:11.946045 ip-10-0-137-20 kubenswrapper[2565]: W0917 17:34:11.943984    2565 feature_gate.go:328] unrecognized feature gate: OVNObservability
Sep 17 17:34:11.946045 ip-10-0-137-20 kubenswrapper[2565]: W0917 17:34:11.943987    2565 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Sep 17 17:34:11.946045 ip-10-0-137-20 kubenswrapper[2565]: W0917 17:34:11.943990    2565 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Sep 17 17:34:11.946045 ip-10-0-137-20 kubenswrapper[2565]: W0917 17:34:11.943993    2565 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Sep 17 17:34:11.946045 ip-10-0-137-20 kubenswrapper[2565]: W0917 17:34:11.943996    2565 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Sep 17 17:34:11.946045 ip-10-0-137-20 kubenswrapper[2565]: W0917 17:34:11.943999    2565 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Sep 17 17:34:11.946045 ip-10-0-137-20 kubenswrapper[2565]: W0917 17:34:11.944001    2565 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Sep 17 17:34:11.946045 ip-10-0-137-20 kubenswrapper[2565]: W0917 17:34:11.944004    2565 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Sep 17 17:34:11.946045 ip-10-0-137-20 kubenswrapper[2565]: W0917 17:34:11.944007    2565 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Sep 17 17:34:11.946045 ip-10-0-137-20 kubenswrapper[2565]: W0917 17:34:11.944010    2565 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Sep 17 17:34:11.946045 ip-10-0-137-20 kubenswrapper[2565]: W0917 17:34:11.944013    2565 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Sep 17 17:34:11.946045 ip-10-0-137-20 kubenswrapper[2565]: W0917 17:34:11.944015    2565 feature_gate.go:328] unrecognized feature gate: SignatureStores
Sep 17 17:34:11.946045 ip-10-0-137-20 kubenswrapper[2565]: W0917 17:34:11.944033    2565 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Sep 17 17:34:11.946045 ip-10-0-137-20 kubenswrapper[2565]: W0917 17:34:11.944036    2565 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Sep 17 17:34:11.946045 ip-10-0-137-20 kubenswrapper[2565]: W0917 17:34:11.944039    2565 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Sep 17 17:34:11.946045 ip-10-0-137-20 kubenswrapper[2565]: W0917 17:34:11.944042    2565 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Sep 17 17:34:11.946519 ip-10-0-137-20 kubenswrapper[2565]: W0917 17:34:11.944047    2565 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Sep 17 17:34:11.946519 ip-10-0-137-20 kubenswrapper[2565]: W0917 17:34:11.944049    2565 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Sep 17 17:34:11.946519 ip-10-0-137-20 kubenswrapper[2565]: W0917 17:34:11.944052    2565 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Sep 17 17:34:11.946519 ip-10-0-137-20 kubenswrapper[2565]: W0917 17:34:11.944055    2565 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Sep 17 17:34:11.946519 ip-10-0-137-20 kubenswrapper[2565]: W0917 17:34:11.944057    2565 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Sep 17 17:34:11.946519 ip-10-0-137-20 kubenswrapper[2565]: W0917 17:34:11.944060    2565 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Sep 17 17:34:11.946519 ip-10-0-137-20 kubenswrapper[2565]: W0917 17:34:11.944062    2565 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Sep 17 17:34:11.946519 ip-10-0-137-20 kubenswrapper[2565]: W0917 17:34:11.944065    2565 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Sep 17 17:34:11.946519 ip-10-0-137-20 kubenswrapper[2565]: W0917 17:34:11.944068    2565 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Sep 17 17:34:11.946519 ip-10-0-137-20 kubenswrapper[2565]: W0917 17:34:11.944071    2565 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Sep 17 17:34:11.946519 ip-10-0-137-20 kubenswrapper[2565]: W0917 17:34:11.944075    2565 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Sep 17 17:34:11.946519 ip-10-0-137-20 kubenswrapper[2565]: W0917 17:34:11.944080    2565 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Sep 17 17:34:11.946519 ip-10-0-137-20 kubenswrapper[2565]: W0917 17:34:11.944083    2565 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Sep 17 17:34:11.946519 ip-10-0-137-20 kubenswrapper[2565]: W0917 17:34:11.944085    2565 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Sep 17 17:34:11.946519 ip-10-0-137-20 kubenswrapper[2565]: W0917 17:34:11.944088    2565 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Sep 17 17:34:11.946519 ip-10-0-137-20 kubenswrapper[2565]: W0917 17:34:11.944090    2565 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Sep 17 17:34:11.946519 ip-10-0-137-20 kubenswrapper[2565]: W0917 17:34:11.944093    2565 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Sep 17 17:34:11.946519 ip-10-0-137-20 kubenswrapper[2565]: W0917 17:34:11.944095    2565 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Sep 17 17:34:11.946519 ip-10-0-137-20 kubenswrapper[2565]: W0917 17:34:11.944098    2565 feature_gate.go:328] unrecognized feature gate: PinnedImages
Sep 17 17:34:11.946986 ip-10-0-137-20 kubenswrapper[2565]: W0917 17:34:11.944101    2565 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Sep 17 17:34:11.946986 ip-10-0-137-20 kubenswrapper[2565]: W0917 17:34:11.944103    2565 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Sep 17 17:34:11.946986 ip-10-0-137-20 kubenswrapper[2565]: W0917 17:34:11.944106    2565 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Sep 17 17:34:11.946986 ip-10-0-137-20 kubenswrapper[2565]: W0917 17:34:11.944108    2565 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Sep 17 17:34:11.946986 ip-10-0-137-20 kubenswrapper[2565]: W0917 17:34:11.944111    2565 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Sep 17 17:34:11.946986 ip-10-0-137-20 kubenswrapper[2565]: W0917 17:34:11.944115    2565 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Sep 17 17:34:11.946986 ip-10-0-137-20 kubenswrapper[2565]: W0917 17:34:11.944118    2565 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Sep 17 17:34:11.946986 ip-10-0-137-20 kubenswrapper[2565]: W0917 17:34:11.944121    2565 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Sep 17 17:34:11.946986 ip-10-0-137-20 kubenswrapper[2565]: W0917 17:34:11.944123    2565 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Sep 17 17:34:11.946986 ip-10-0-137-20 kubenswrapper[2565]: W0917 17:34:11.944126    2565 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Sep 17 17:34:11.946986 ip-10-0-137-20 kubenswrapper[2565]: W0917 17:34:11.944129    2565 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Sep 17 17:34:11.946986 ip-10-0-137-20 kubenswrapper[2565]: W0917 17:34:11.944131    2565 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Sep 17 17:34:11.946986 ip-10-0-137-20 kubenswrapper[2565]: W0917 17:34:11.944134    2565 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Sep 17 17:34:11.946986 ip-10-0-137-20 kubenswrapper[2565]: W0917 17:34:11.944136    2565 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Sep 17 17:34:11.946986 ip-10-0-137-20 kubenswrapper[2565]: W0917 17:34:11.944139    2565 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Sep 17 17:34:11.946986 ip-10-0-137-20 kubenswrapper[2565]: W0917 17:34:11.944142    2565 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Sep 17 17:34:11.946986 ip-10-0-137-20 kubenswrapper[2565]: W0917 17:34:11.944144    2565 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Sep 17 17:34:11.946986 ip-10-0-137-20 kubenswrapper[2565]: W0917 17:34:11.944147    2565 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Sep 17 17:34:11.946986 ip-10-0-137-20 kubenswrapper[2565]: W0917 17:34:11.944149    2565 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Sep 17 17:34:11.946986 ip-10-0-137-20 kubenswrapper[2565]: W0917 17:34:11.944152    2565 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Sep 17 17:34:11.947512 ip-10-0-137-20 kubenswrapper[2565]: W0917 17:34:11.944154    2565 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Sep 17 17:34:11.947512 ip-10-0-137-20 kubenswrapper[2565]: W0917 17:34:11.944157    2565 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Sep 17 17:34:11.947512 ip-10-0-137-20 kubenswrapper[2565]: W0917 17:34:11.944160    2565 feature_gate.go:328] unrecognized feature gate: OSStreams
Sep 17 17:34:11.947512 ip-10-0-137-20 kubenswrapper[2565]: W0917 17:34:11.944162    2565 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Sep 17 17:34:11.947512 ip-10-0-137-20 kubenswrapper[2565]: W0917 17:34:11.944165    2565 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Sep 17 17:34:11.947512 ip-10-0-137-20 kubenswrapper[2565]: W0917 17:34:11.944168    2565 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Sep 17 17:34:11.947512 ip-10-0-137-20 kubenswrapper[2565]: W0917 17:34:11.944170    2565 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Sep 17 17:34:11.947512 ip-10-0-137-20 kubenswrapper[2565]: W0917 17:34:11.944172    2565 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Sep 17 17:34:11.947512 ip-10-0-137-20 kubenswrapper[2565]: W0917 17:34:11.944175    2565 feature_gate.go:328] unrecognized feature gate: NewOLM
Sep 17 17:34:11.947512 ip-10-0-137-20 kubenswrapper[2565]: W0917 17:34:11.944177    2565 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Sep 17 17:34:11.947512 ip-10-0-137-20 kubenswrapper[2565]: W0917 17:34:11.944180    2565 feature_gate.go:328] unrecognized feature gate: EventTTL
Sep 17 17:34:11.947512 ip-10-0-137-20 kubenswrapper[2565]: W0917 17:34:11.944183    2565 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Sep 17 17:34:11.947512 ip-10-0-137-20 kubenswrapper[2565]: W0917 17:34:11.944185    2565 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Sep 17 17:34:11.947512 ip-10-0-137-20 kubenswrapper[2565]: W0917 17:34:11.944187    2565 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Sep 17 17:34:11.947512 ip-10-0-137-20 kubenswrapper[2565]: W0917 17:34:11.944190    2565 feature_gate.go:328] unrecognized feature gate: Example2
Sep 17 17:34:11.947512 ip-10-0-137-20 kubenswrapper[2565]: W0917 17:34:11.944193    2565 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Sep 17 17:34:11.947512 ip-10-0-137-20 kubenswrapper[2565]: W0917 17:34:11.944195    2565 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Sep 17 17:34:11.947512 ip-10-0-137-20 kubenswrapper[2565]: W0917 17:34:11.944198    2565 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Sep 17 17:34:11.947512 ip-10-0-137-20 kubenswrapper[2565]: W0917 17:34:11.944563    2565 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Sep 17 17:34:11.947512 ip-10-0-137-20 kubenswrapper[2565]: W0917 17:34:11.944567    2565 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Sep 17 17:34:11.948043 ip-10-0-137-20 kubenswrapper[2565]: W0917 17:34:11.944570    2565 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Sep 17 17:34:11.948043 ip-10-0-137-20 kubenswrapper[2565]: W0917 17:34:11.944573    2565 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Sep 17 17:34:11.948043 ip-10-0-137-20 kubenswrapper[2565]: W0917 17:34:11.944575    2565 feature_gate.go:328] unrecognized feature gate: SignatureStores
Sep 17 17:34:11.948043 ip-10-0-137-20 kubenswrapper[2565]: W0917 17:34:11.944578    2565 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Sep 17 17:34:11.948043 ip-10-0-137-20 kubenswrapper[2565]: W0917 17:34:11.944581    2565 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Sep 17 17:34:11.948043 ip-10-0-137-20 kubenswrapper[2565]: W0917 17:34:11.944583    2565 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Sep 17 17:34:11.948043 ip-10-0-137-20 kubenswrapper[2565]: W0917 17:34:11.944586    2565 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Sep 17 17:34:11.948043 ip-10-0-137-20 kubenswrapper[2565]: W0917 17:34:11.944589    2565 feature_gate.go:328] unrecognized feature gate: DualReplica
Sep 17 17:34:11.948043 ip-10-0-137-20 kubenswrapper[2565]: W0917 17:34:11.944591    2565 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Sep 17 17:34:11.948043 ip-10-0-137-20 kubenswrapper[2565]: W0917 17:34:11.944594    2565 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Sep 17 17:34:11.948043 ip-10-0-137-20 kubenswrapper[2565]: W0917 17:34:11.944597    2565 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Sep 17 17:34:11.948043 ip-10-0-137-20 kubenswrapper[2565]: W0917 17:34:11.944600    2565 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Sep 17 17:34:11.948043 ip-10-0-137-20 kubenswrapper[2565]: W0917 17:34:11.944603    2565 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Sep 17 17:34:11.948043 ip-10-0-137-20 kubenswrapper[2565]: W0917 17:34:11.944605    2565 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Sep 17 17:34:11.948043 ip-10-0-137-20 kubenswrapper[2565]: W0917 17:34:11.944609    2565 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Sep 17 17:34:11.948043 ip-10-0-137-20 kubenswrapper[2565]: W0917 17:34:11.944612    2565 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Sep 17 17:34:11.948043 ip-10-0-137-20 kubenswrapper[2565]: W0917 17:34:11.944616    2565 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Sep 17 17:34:11.948043 ip-10-0-137-20 kubenswrapper[2565]: W0917 17:34:11.944619    2565 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Sep 17 17:34:11.948043 ip-10-0-137-20 kubenswrapper[2565]: W0917 17:34:11.944623    2565 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Sep 17 17:34:11.948043 ip-10-0-137-20 kubenswrapper[2565]: W0917 17:34:11.944625    2565 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Sep 17 17:34:11.948533 ip-10-0-137-20 kubenswrapper[2565]: W0917 17:34:11.944628    2565 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Sep 17 17:34:11.948533 ip-10-0-137-20 kubenswrapper[2565]: W0917 17:34:11.944630    2565 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Sep 17 17:34:11.948533 ip-10-0-137-20 kubenswrapper[2565]: W0917 17:34:11.944633    2565 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Sep 17 17:34:11.948533 ip-10-0-137-20 kubenswrapper[2565]: W0917 17:34:11.944635    2565 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Sep 17 17:34:11.948533 ip-10-0-137-20 kubenswrapper[2565]: W0917 17:34:11.944638    2565 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Sep 17 17:34:11.948533 ip-10-0-137-20 kubenswrapper[2565]: W0917 17:34:11.944641    2565 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Sep 17 17:34:11.948533 ip-10-0-137-20 kubenswrapper[2565]: W0917 17:34:11.944643    2565 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Sep 17 17:34:11.948533 ip-10-0-137-20 kubenswrapper[2565]: W0917 17:34:11.944646    2565 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Sep 17 17:34:11.948533 ip-10-0-137-20 kubenswrapper[2565]: W0917 17:34:11.944648    2565 feature_gate.go:328] unrecognized feature gate: Example2
Sep 17 17:34:11.948533 ip-10-0-137-20 kubenswrapper[2565]: W0917 17:34:11.944652    2565 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Sep 17 17:34:11.948533 ip-10-0-137-20 kubenswrapper[2565]: W0917 17:34:11.944654    2565 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Sep 17 17:34:11.948533 ip-10-0-137-20 kubenswrapper[2565]: W0917 17:34:11.944657    2565 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Sep 17 17:34:11.948533 ip-10-0-137-20 kubenswrapper[2565]: W0917 17:34:11.944660    2565 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Sep 17 17:34:11.948533 ip-10-0-137-20 kubenswrapper[2565]: W0917 17:34:11.944662    2565 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Sep 17 17:34:11.948533 ip-10-0-137-20 kubenswrapper[2565]: W0917 17:34:11.944666    2565 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Sep 17 17:34:11.948533 ip-10-0-137-20 kubenswrapper[2565]: W0917 17:34:11.944670    2565 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Sep 17 17:34:11.948533 ip-10-0-137-20 kubenswrapper[2565]: W0917 17:34:11.944673    2565 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Sep 17 17:34:11.948533 ip-10-0-137-20 kubenswrapper[2565]: W0917 17:34:11.944675    2565 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Sep 17 17:34:11.948533 ip-10-0-137-20 kubenswrapper[2565]: W0917 17:34:11.944678    2565 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Sep 17 17:34:11.948533 ip-10-0-137-20 kubenswrapper[2565]: W0917 17:34:11.944681    2565 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Sep 17 17:34:11.949060 ip-10-0-137-20 kubenswrapper[2565]: W0917 17:34:11.944684    2565 feature_gate.go:328] unrecognized feature gate: OVNObservability
Sep 17 17:34:11.949060 ip-10-0-137-20 kubenswrapper[2565]: W0917 17:34:11.944687    2565 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Sep 17 17:34:11.949060 ip-10-0-137-20 kubenswrapper[2565]: W0917 17:34:11.944690    2565 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Sep 17 17:34:11.949060 ip-10-0-137-20 kubenswrapper[2565]: W0917 17:34:11.944692    2565 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Sep 17 17:34:11.949060 ip-10-0-137-20 kubenswrapper[2565]: W0917 17:34:11.944695    2565 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Sep 17 17:34:11.949060 ip-10-0-137-20 kubenswrapper[2565]: W0917 17:34:11.944698    2565 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Sep 17 17:34:11.949060 ip-10-0-137-20 kubenswrapper[2565]: W0917 17:34:11.944700    2565 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Sep 17 17:34:11.949060 ip-10-0-137-20 kubenswrapper[2565]: W0917 17:34:11.944704    2565 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Sep 17 17:34:11.949060 ip-10-0-137-20 kubenswrapper[2565]: W0917 17:34:11.944706    2565 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Sep 17 17:34:11.949060 ip-10-0-137-20 kubenswrapper[2565]: W0917 17:34:11.944709    2565 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Sep 17 17:34:11.949060 ip-10-0-137-20 kubenswrapper[2565]: W0917 17:34:11.944711    2565 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Sep 17 17:34:11.949060 ip-10-0-137-20 kubenswrapper[2565]: W0917 17:34:11.944713    2565 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Sep 17 17:34:11.949060 ip-10-0-137-20 kubenswrapper[2565]: W0917 17:34:11.944719    2565 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Sep 17 17:34:11.949060 ip-10-0-137-20 kubenswrapper[2565]: W0917 17:34:11.944722    2565 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Sep 17 17:34:11.949060 ip-10-0-137-20 kubenswrapper[2565]: W0917 17:34:11.944724    2565 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Sep 17 17:34:11.949060 ip-10-0-137-20 kubenswrapper[2565]: W0917 17:34:11.944727    2565 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Sep 17 17:34:11.949060 ip-10-0-137-20 kubenswrapper[2565]: W0917 17:34:11.944729    2565 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Sep 17 17:34:11.949060 ip-10-0-137-20 kubenswrapper[2565]: W0917 17:34:11.944732    2565 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Sep 17 17:34:11.949060 ip-10-0-137-20 kubenswrapper[2565]: W0917 17:34:11.944734    2565 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Sep 17 17:34:11.949519 ip-10-0-137-20 kubenswrapper[2565]: W0917 17:34:11.944737    2565 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Sep 17 17:34:11.949519 ip-10-0-137-20 kubenswrapper[2565]: W0917 17:34:11.944739    2565 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Sep 17 17:34:11.949519 ip-10-0-137-20 kubenswrapper[2565]: W0917 17:34:11.944742    2565 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Sep 17 17:34:11.949519 ip-10-0-137-20 kubenswrapper[2565]: W0917 17:34:11.944745    2565 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Sep 17 17:34:11.949519 ip-10-0-137-20 kubenswrapper[2565]: W0917 17:34:11.944748    2565 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Sep 17 17:34:11.949519 ip-10-0-137-20 kubenswrapper[2565]: W0917 17:34:11.944750    2565 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Sep 17 17:34:11.949519 ip-10-0-137-20 kubenswrapper[2565]: W0917 17:34:11.944752    2565 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Sep 17 17:34:11.949519 ip-10-0-137-20 kubenswrapper[2565]: W0917 17:34:11.944755    2565 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Sep 17 17:34:11.949519 ip-10-0-137-20 kubenswrapper[2565]: W0917 17:34:11.944758    2565 feature_gate.go:328] unrecognized feature gate: NewOLM
Sep 17 17:34:11.949519 ip-10-0-137-20 kubenswrapper[2565]: W0917 17:34:11.944760    2565 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Sep 17 17:34:11.949519 ip-10-0-137-20 kubenswrapper[2565]: W0917 17:34:11.944763    2565 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Sep 17 17:34:11.949519 ip-10-0-137-20 kubenswrapper[2565]: W0917 17:34:11.944765    2565 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Sep 17 17:34:11.949519 ip-10-0-137-20 kubenswrapper[2565]: W0917 17:34:11.944769    2565 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Sep 17 17:34:11.949519 ip-10-0-137-20 kubenswrapper[2565]: W0917 17:34:11.944772    2565 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Sep 17 17:34:11.949519 ip-10-0-137-20 kubenswrapper[2565]: W0917 17:34:11.944775    2565 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Sep 17 17:34:11.949519 ip-10-0-137-20 kubenswrapper[2565]: W0917 17:34:11.944778    2565 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Sep 17 17:34:11.949519 ip-10-0-137-20 kubenswrapper[2565]: W0917 17:34:11.944781    2565 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Sep 17 17:34:11.949519 ip-10-0-137-20 kubenswrapper[2565]: W0917 17:34:11.944783    2565 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Sep 17 17:34:11.949519 ip-10-0-137-20 kubenswrapper[2565]: W0917 17:34:11.944786    2565 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Sep 17 17:34:11.949980 ip-10-0-137-20 kubenswrapper[2565]: W0917 17:34:11.944788    2565 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Sep 17 17:34:11.949980 ip-10-0-137-20 kubenswrapper[2565]: W0917 17:34:11.944791    2565 feature_gate.go:328] unrecognized feature gate: EventTTL
Sep 17 17:34:11.949980 ip-10-0-137-20 kubenswrapper[2565]: W0917 17:34:11.944796    2565 feature_gate.go:328] unrecognized feature gate: Example
Sep 17 17:34:11.949980 ip-10-0-137-20 kubenswrapper[2565]: W0917 17:34:11.944799    2565 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Sep 17 17:34:11.949980 ip-10-0-137-20 kubenswrapper[2565]: W0917 17:34:11.944801    2565 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Sep 17 17:34:11.949980 ip-10-0-137-20 kubenswrapper[2565]: W0917 17:34:11.944804    2565 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Sep 17 17:34:11.949980 ip-10-0-137-20 kubenswrapper[2565]: W0917 17:34:11.944808    2565 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Sep 17 17:34:11.949980 ip-10-0-137-20 kubenswrapper[2565]: W0917 17:34:11.944811    2565 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Sep 17 17:34:11.949980 ip-10-0-137-20 kubenswrapper[2565]: W0917 17:34:11.944813    2565 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Sep 17 17:34:11.949980 ip-10-0-137-20 kubenswrapper[2565]: W0917 17:34:11.944816    2565 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Sep 17 17:34:11.949980 ip-10-0-137-20 kubenswrapper[2565]: W0917 17:34:11.944818    2565 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Sep 17 17:34:11.949980 ip-10-0-137-20 kubenswrapper[2565]: W0917 17:34:11.944821    2565 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Sep 17 17:34:11.949980 ip-10-0-137-20 kubenswrapper[2565]: W0917 17:34:11.944823    2565 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Sep 17 17:34:11.949980 ip-10-0-137-20 kubenswrapper[2565]: W0917 17:34:11.944826    2565 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Sep 17 17:34:11.949980 ip-10-0-137-20 kubenswrapper[2565]: W0917 17:34:11.944828    2565 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Sep 17 17:34:11.949980 ip-10-0-137-20 kubenswrapper[2565]: W0917 17:34:11.944831    2565 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Sep 17 17:34:11.949980 ip-10-0-137-20 kubenswrapper[2565]: W0917 17:34:11.944834    2565 feature_gate.go:328] unrecognized feature gate: OSStreams
Sep 17 17:34:11.949980 ip-10-0-137-20 kubenswrapper[2565]: W0917 17:34:11.944837    2565 feature_gate.go:328] unrecognized feature gate: PinnedImages
Sep 17 17:34:11.949980 ip-10-0-137-20 kubenswrapper[2565]: W0917 17:34:11.944840    2565 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Sep 17 17:34:11.949980 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:34:11.944918    2565 flags.go:64] FLAG: --address="0.0.0.0"
Sep 17 17:34:11.949980 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:34:11.944954    2565 flags.go:64] FLAG: --allowed-unsafe-sysctls="[]"
Sep 17 17:34:11.950540 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:34:11.944960    2565 flags.go:64] FLAG: --anonymous-auth="true"
Sep 17 17:34:11.950540 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:34:11.944964    2565 flags.go:64] FLAG: --application-metrics-count-limit="100"
Sep 17 17:34:11.950540 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:34:11.944968    2565 flags.go:64] FLAG: --authentication-token-webhook="false"
Sep 17 17:34:11.950540 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:34:11.944971    2565 flags.go:64] FLAG: --authentication-token-webhook-cache-ttl="2m0s"
Sep 17 17:34:11.950540 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:34:11.944975    2565 flags.go:64] FLAG: --authorization-mode="AlwaysAllow"
Sep 17 17:34:11.950540 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:34:11.944980    2565 flags.go:64] FLAG: --authorization-webhook-cache-authorized-ttl="5m0s"
Sep 17 17:34:11.950540 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:34:11.944983    2565 flags.go:64] FLAG: --authorization-webhook-cache-unauthorized-ttl="30s"
Sep 17 17:34:11.950540 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:34:11.944986    2565 flags.go:64] FLAG: --boot-id-file="/proc/sys/kernel/random/boot_id"
Sep 17 17:34:11.950540 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:34:11.944989    2565 flags.go:64] FLAG: --bootstrap-kubeconfig="/etc/kubernetes/kubeconfig"
Sep 17 17:34:11.950540 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:34:11.944992    2565 flags.go:64] FLAG: --cert-dir="/var/lib/kubelet/pki"
Sep 17 17:34:11.950540 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:34:11.944995    2565 flags.go:64] FLAG: --cgroup-driver="cgroupfs"
Sep 17 17:34:11.950540 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:34:11.944999    2565 flags.go:64] FLAG: --cgroup-root=""
Sep 17 17:34:11.950540 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:34:11.945002    2565 flags.go:64] FLAG: --cgroups-per-qos="true"
Sep 17 17:34:11.950540 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:34:11.945005    2565 flags.go:64] FLAG: --client-ca-file=""
Sep 17 17:34:11.950540 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:34:11.945009    2565 flags.go:64] FLAG: --cloud-provider="external"
Sep 17 17:34:11.950540 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:34:11.945012    2565 flags.go:64] FLAG: --cluster-dns="[]"
Sep 17 17:34:11.950540 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:34:11.945017    2565 flags.go:64] FLAG: --cluster-domain=""
Sep 17 17:34:11.950540 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:34:11.945034    2565 flags.go:64] FLAG: --config="/etc/kubernetes/kubelet.conf"
Sep 17 17:34:11.950540 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:34:11.945037    2565 flags.go:64] FLAG: --config-dir="/etc/openshift/kubelet.conf.d"
Sep 17 17:34:11.950540 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:34:11.945041    2565 flags.go:64] FLAG: --container-hints="/etc/cadvisor/container_hints.json"
Sep 17 17:34:11.950540 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:34:11.945044    2565 flags.go:64] FLAG: --container-log-max-files="5"
Sep 17 17:34:11.950540 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:34:11.945048    2565 flags.go:64] FLAG: --container-log-max-size="10Mi"
Sep 17 17:34:11.950540 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:34:11.945051    2565 flags.go:64] FLAG: --container-runtime-endpoint="/var/run/crio/crio.sock"
Sep 17 17:34:11.951095 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:34:11.945054    2565 flags.go:64] FLAG: --containerd="/run/containerd/containerd.sock"
Sep 17 17:34:11.951095 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:34:11.945057    2565 flags.go:64] FLAG: --containerd-namespace="k8s.io"
Sep 17 17:34:11.951095 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:34:11.945060    2565 flags.go:64] FLAG: --contention-profiling="false"
Sep 17 17:34:11.951095 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:34:11.945064    2565 flags.go:64] FLAG: --cpu-cfs-quota="true"
Sep 17 17:34:11.951095 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:34:11.945067    2565 flags.go:64] FLAG: --cpu-cfs-quota-period="100ms"
Sep 17 17:34:11.951095 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:34:11.945070    2565 flags.go:64] FLAG: --cpu-manager-policy="none"
Sep 17 17:34:11.951095 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:34:11.945073    2565 flags.go:64] FLAG: --cpu-manager-policy-options=""
Sep 17 17:34:11.951095 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:34:11.945077    2565 flags.go:64] FLAG: --cpu-manager-reconcile-period="10s"
Sep 17 17:34:11.951095 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:34:11.945080    2565 flags.go:64] FLAG: --enable-controller-attach-detach="true"
Sep 17 17:34:11.951095 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:34:11.945083    2565 flags.go:64] FLAG: --enable-debugging-handlers="true"
Sep 17 17:34:11.951095 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:34:11.945086    2565 flags.go:64] FLAG: --enable-load-reader="false"
Sep 17 17:34:11.951095 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:34:11.945089    2565 flags.go:64] FLAG: --enable-server="true"
Sep 17 17:34:11.951095 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:34:11.945092    2565 flags.go:64] FLAG: --enforce-node-allocatable="[pods]"
Sep 17 17:34:11.951095 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:34:11.945096    2565 flags.go:64] FLAG: --event-burst="100"
Sep 17 17:34:11.951095 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:34:11.945100    2565 flags.go:64] FLAG: --event-qps="50"
Sep 17 17:34:11.951095 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:34:11.945103    2565 flags.go:64] FLAG: --event-storage-age-limit="default=0"
Sep 17 17:34:11.951095 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:34:11.945106    2565 flags.go:64] FLAG: --event-storage-event-limit="default=0"
Sep 17 17:34:11.951095 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:34:11.945109    2565 flags.go:64] FLAG: --eviction-hard=""
Sep 17 17:34:11.951095 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:34:11.945112    2565 flags.go:64] FLAG: --eviction-max-pod-grace-period="0"
Sep 17 17:34:11.951095 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:34:11.945115    2565 flags.go:64] FLAG: --eviction-minimum-reclaim=""
Sep 17 17:34:11.951095 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:34:11.945118    2565 flags.go:64] FLAG: --eviction-pressure-transition-period="5m0s"
Sep 17 17:34:11.951095 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:34:11.945121    2565 flags.go:64] FLAG: --eviction-soft=""
Sep 17 17:34:11.951095 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:34:11.945124    2565 flags.go:64] FLAG: --eviction-soft-grace-period=""
Sep 17 17:34:11.951095 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:34:11.945127    2565 flags.go:64] FLAG: --exit-on-lock-contention="false"
Sep 17 17:34:11.951095 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:34:11.945130    2565 flags.go:64] FLAG: --experimental-allocatable-ignore-eviction="false"
Sep 17 17:34:11.951711 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:34:11.945133    2565 flags.go:64] FLAG: --experimental-mounter-path=""
Sep 17 17:34:11.951711 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:34:11.945139    2565 flags.go:64] FLAG: --fail-cgroupv1="false"
Sep 17 17:34:11.951711 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:34:11.945142    2565 flags.go:64] FLAG: --fail-swap-on="true"
Sep 17 17:34:11.951711 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:34:11.945145    2565 flags.go:64] FLAG: --feature-gates=""
Sep 17 17:34:11.951711 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:34:11.945148    2565 flags.go:64] FLAG: --file-check-frequency="20s"
Sep 17 17:34:11.951711 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:34:11.945151    2565 flags.go:64] FLAG: --global-housekeeping-interval="1m0s"
Sep 17 17:34:11.951711 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:34:11.945154    2565 flags.go:64] FLAG: --hairpin-mode="promiscuous-bridge"
Sep 17 17:34:11.951711 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:34:11.945157    2565 flags.go:64] FLAG: --healthz-bind-address="127.0.0.1"
Sep 17 17:34:11.951711 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:34:11.945160    2565 flags.go:64] FLAG: --healthz-port="10248"
Sep 17 17:34:11.951711 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:34:11.945163    2565 flags.go:64] FLAG: --help="false"
Sep 17 17:34:11.951711 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:34:11.945168    2565 flags.go:64] FLAG: --hostname-override="ip-10-0-137-20.ec2.internal"
Sep 17 17:34:11.951711 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:34:11.945171    2565 flags.go:64] FLAG: --housekeeping-interval="10s"
Sep 17 17:34:11.951711 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:34:11.945174    2565 flags.go:64] FLAG: --http-check-frequency="20s"
Sep 17 17:34:11.951711 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:34:11.945176    2565 flags.go:64] FLAG: --image-credential-provider-bin-dir="/usr/libexec/kubelet-image-credential-provider-plugins"
Sep 17 17:34:11.951711 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:34:11.945180    2565 flags.go:64] FLAG: --image-credential-provider-config="/etc/kubernetes/credential-providers/ecr-credential-provider.yaml"
Sep 17 17:34:11.951711 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:34:11.945183    2565 flags.go:64] FLAG: --image-gc-high-threshold="85"
Sep 17 17:34:11.951711 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:34:11.945186    2565 flags.go:64] FLAG: --image-gc-low-threshold="80"
Sep 17 17:34:11.951711 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:34:11.945189    2565 flags.go:64] FLAG: --image-service-endpoint=""
Sep 17 17:34:11.951711 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:34:11.945192    2565 flags.go:64] FLAG: --kernel-memcg-notification="false"
Sep 17 17:34:11.951711 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:34:11.945195    2565 flags.go:64] FLAG: --kube-api-burst="100"
Sep 17 17:34:11.951711 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:34:11.945198    2565 flags.go:64] FLAG: --kube-api-content-type="application/vnd.kubernetes.protobuf"
Sep 17 17:34:11.951711 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:34:11.945201    2565 flags.go:64] FLAG: --kube-api-qps="50"
Sep 17 17:34:11.951711 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:34:11.945204    2565 flags.go:64] FLAG: --kube-reserved=""
Sep 17 17:34:11.951711 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:34:11.945207    2565 flags.go:64] FLAG: --kube-reserved-cgroup=""
Sep 17 17:34:11.952320 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:34:11.945209    2565 flags.go:64] FLAG: --kubeconfig="/var/lib/kubelet/kubeconfig"
Sep 17 17:34:11.952320 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:34:11.945212    2565 flags.go:64] FLAG: --kubelet-cgroups=""
Sep 17 17:34:11.952320 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:34:11.945215    2565 flags.go:64] FLAG: --local-storage-capacity-isolation="true"
Sep 17 17:34:11.952320 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:34:11.945218    2565 flags.go:64] FLAG: --lock-file=""
Sep 17 17:34:11.952320 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:34:11.945221    2565 flags.go:64] FLAG: --log-cadvisor-usage="false"
Sep 17 17:34:11.952320 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:34:11.945224    2565 flags.go:64] FLAG: --log-flush-frequency="5s"
Sep 17 17:34:11.952320 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:34:11.945226    2565 flags.go:64] FLAG: --log-json-info-buffer-size="0"
Sep 17 17:34:11.952320 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:34:11.945232    2565 flags.go:64] FLAG: --log-json-split-stream="false"
Sep 17 17:34:11.952320 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:34:11.945235    2565 flags.go:64] FLAG: --log-text-info-buffer-size="0"
Sep 17 17:34:11.952320 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:34:11.945239    2565 flags.go:64] FLAG: --log-text-split-stream="false"
Sep 17 17:34:11.952320 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:34:11.945242    2565 flags.go:64] FLAG: --logging-format="text"
Sep 17 17:34:11.952320 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:34:11.945245    2565 flags.go:64] FLAG: --machine-id-file="/etc/machine-id,/var/lib/dbus/machine-id"
Sep 17 17:34:11.952320 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:34:11.945248    2565 flags.go:64] FLAG: --make-iptables-util-chains="true"
Sep 17 17:34:11.952320 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:34:11.945251    2565 flags.go:64] FLAG: --manifest-url=""
Sep 17 17:34:11.952320 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:34:11.945254    2565 flags.go:64] FLAG: --manifest-url-header=""
Sep 17 17:34:11.952320 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:34:11.945258    2565 flags.go:64] FLAG: --max-housekeeping-interval="15s"
Sep 17 17:34:11.952320 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:34:11.945261    2565 flags.go:64] FLAG: --max-open-files="1000000"
Sep 17 17:34:11.952320 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:34:11.945265    2565 flags.go:64] FLAG: --max-pods="110"
Sep 17 17:34:11.952320 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:34:11.945269    2565 flags.go:64] FLAG: --maximum-dead-containers="-1"
Sep 17 17:34:11.952320 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:34:11.945272    2565 flags.go:64] FLAG: --maximum-dead-containers-per-container="1"
Sep 17 17:34:11.952320 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:34:11.945275    2565 flags.go:64] FLAG: --memory-manager-policy="None"
Sep 17 17:34:11.952320 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:34:11.945278    2565 flags.go:64] FLAG: --minimum-container-ttl-duration="6m0s"
Sep 17 17:34:11.952320 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:34:11.945281    2565 flags.go:64] FLAG: --minimum-image-ttl-duration="2m0s"
Sep 17 17:34:11.952320 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:34:11.945284    2565 flags.go:64] FLAG: --node-ip="0.0.0.0"
Sep 17 17:34:11.952320 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:34:11.945287    2565 flags.go:64] FLAG: --node-labels="node-role.kubernetes.io/worker=,node.openshift.io/os_id=rhel"
Sep 17 17:34:11.952919 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:34:11.945294    2565 flags.go:64] FLAG: --node-status-max-images="50"
Sep 17 17:34:11.952919 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:34:11.945297    2565 flags.go:64] FLAG: --node-status-update-frequency="10s"
Sep 17 17:34:11.952919 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:34:11.945300    2565 flags.go:64] FLAG: --oom-score-adj="-999"
Sep 17 17:34:11.952919 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:34:11.945303    2565 flags.go:64] FLAG: --pod-cidr=""
Sep 17 17:34:11.952919 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:34:11.945306    2565 flags.go:64] FLAG: --pod-infra-container-image="quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:a6abe77af61c5f2e3985a83e7d43538aefc38b1b0444127da14ee0e8cf8f76fc"
Sep 17 17:34:11.952919 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:34:11.945311    2565 flags.go:64] FLAG: --pod-manifest-path=""
Sep 17 17:34:11.952919 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:34:11.945313    2565 flags.go:64] FLAG: --pod-max-pids="-1"
Sep 17 17:34:11.952919 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:34:11.945316    2565 flags.go:64] FLAG: --pods-per-core="0"
Sep 17 17:34:11.952919 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:34:11.945319    2565 flags.go:64] FLAG: --port="10250"
Sep 17 17:34:11.952919 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:34:11.945322    2565 flags.go:64] FLAG: --protect-kernel-defaults="false"
Sep 17 17:34:11.952919 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:34:11.945325    2565 flags.go:64] FLAG: --provider-id="aws:///us-east-1a/i-0b72985e8514de2c4"
Sep 17 17:34:11.952919 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:34:11.945328    2565 flags.go:64] FLAG: --qos-reserved=""
Sep 17 17:34:11.952919 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:34:11.945331    2565 flags.go:64] FLAG: --read-only-port="10255"
Sep 17 17:34:11.952919 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:34:11.945334    2565 flags.go:64] FLAG: --register-node="true"
Sep 17 17:34:11.952919 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:34:11.945337    2565 flags.go:64] FLAG: --register-with-taints=""
Sep 17 17:34:11.952919 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:34:11.945340    2565 flags.go:64] FLAG: --registry-burst="10"
Sep 17 17:34:11.952919 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:34:11.945344    2565 flags.go:64] FLAG: --registry-qps="5"
Sep 17 17:34:11.952919 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:34:11.945348    2565 flags.go:64] FLAG: --reserved-cpus=""
Sep 17 17:34:11.952919 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:34:11.945351    2565 flags.go:64] FLAG: --reserved-memory=""
Sep 17 17:34:11.952919 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:34:11.945354    2565 flags.go:64] FLAG: --resolv-conf="/etc/resolv.conf"
Sep 17 17:34:11.952919 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:34:11.945357    2565 flags.go:64] FLAG: --root-dir="/var/lib/kubelet"
Sep 17 17:34:11.952919 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:34:11.945360    2565 flags.go:64] FLAG: --rotate-certificates="false"
Sep 17 17:34:11.952919 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:34:11.945363    2565 flags.go:64] FLAG: --rotate-server-certificates="false"
Sep 17 17:34:11.952919 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:34:11.945366    2565 flags.go:64] FLAG: --runonce="false"
Sep 17 17:34:11.952919 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:34:11.945369    2565 flags.go:64] FLAG: --runtime-cgroups="/system.slice/crio.service"
Sep 17 17:34:11.953571 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:34:11.945373    2565 flags.go:64] FLAG: --runtime-request-timeout="2m0s"
Sep 17 17:34:11.953571 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:34:11.945376    2565 flags.go:64] FLAG: --seccomp-default="false"
Sep 17 17:34:11.953571 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:34:11.945379    2565 flags.go:64] FLAG: --serialize-image-pulls="true"
Sep 17 17:34:11.953571 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:34:11.945382    2565 flags.go:64] FLAG: --storage-driver-buffer-duration="1m0s"
Sep 17 17:34:11.953571 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:34:11.945385    2565 flags.go:64] FLAG: --storage-driver-db="cadvisor"
Sep 17 17:34:11.953571 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:34:11.945388    2565 flags.go:64] FLAG: --storage-driver-host="localhost:8086"
Sep 17 17:34:11.953571 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:34:11.945391    2565 flags.go:64] FLAG: --storage-driver-password="root"
Sep 17 17:34:11.953571 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:34:11.945394    2565 flags.go:64] FLAG: --storage-driver-secure="false"
Sep 17 17:34:11.953571 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:34:11.945397    2565 flags.go:64] FLAG: --storage-driver-table="stats"
Sep 17 17:34:11.953571 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:34:11.945399    2565 flags.go:64] FLAG: --storage-driver-user="root"
Sep 17 17:34:11.953571 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:34:11.945402    2565 flags.go:64] FLAG: --streaming-connection-idle-timeout="4h0m0s"
Sep 17 17:34:11.953571 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:34:11.945405    2565 flags.go:64] FLAG: --sync-frequency="1m0s"
Sep 17 17:34:11.953571 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:34:11.945408    2565 flags.go:64] FLAG: --system-cgroups=""
Sep 17 17:34:11.953571 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:34:11.945411    2565 flags.go:64] FLAG: --system-reserved="cpu=500m,ephemeral-storage=1Gi,memory=1Gi"
Sep 17 17:34:11.953571 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:34:11.945416    2565 flags.go:64] FLAG: --system-reserved-cgroup=""
Sep 17 17:34:11.953571 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:34:11.945419    2565 flags.go:64] FLAG: --tls-cert-file=""
Sep 17 17:34:11.953571 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:34:11.945422    2565 flags.go:64] FLAG: --tls-cipher-suites="[]"
Sep 17 17:34:11.953571 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:34:11.945426    2565 flags.go:64] FLAG: --tls-min-version=""
Sep 17 17:34:11.953571 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:34:11.945428    2565 flags.go:64] FLAG: --tls-private-key-file=""
Sep 17 17:34:11.953571 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:34:11.945431    2565 flags.go:64] FLAG: --topology-manager-policy="none"
Sep 17 17:34:11.953571 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:34:11.945434    2565 flags.go:64] FLAG: --topology-manager-policy-options=""
Sep 17 17:34:11.953571 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:34:11.945437    2565 flags.go:64] FLAG: --topology-manager-scope="container"
Sep 17 17:34:11.953571 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:34:11.945440    2565 flags.go:64] FLAG: --v="2"
Sep 17 17:34:11.953571 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:34:11.945443    2565 flags.go:64] FLAG: --version="false"
Sep 17 17:34:11.953571 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:34:11.945450    2565 flags.go:64] FLAG: --vmodule=""
Sep 17 17:34:11.954187 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:34:11.945454    2565 flags.go:64] FLAG: --volume-plugin-dir="/etc/kubernetes/kubelet-plugins/volume/exec"
Sep 17 17:34:11.954187 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:34:11.945457    2565 flags.go:64] FLAG: --volume-stats-agg-period="1m0s"
Sep 17 17:34:11.954187 ip-10-0-137-20 kubenswrapper[2565]: W0917 17:34:11.945609    2565 feature_gate.go:328] unrecognized feature gate: NewOLM
Sep 17 17:34:11.954187 ip-10-0-137-20 kubenswrapper[2565]: W0917 17:34:11.945613    2565 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Sep 17 17:34:11.954187 ip-10-0-137-20 kubenswrapper[2565]: W0917 17:34:11.945617    2565 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Sep 17 17:34:11.954187 ip-10-0-137-20 kubenswrapper[2565]: W0917 17:34:11.945621    2565 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Sep 17 17:34:11.954187 ip-10-0-137-20 kubenswrapper[2565]: W0917 17:34:11.945624    2565 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Sep 17 17:34:11.954187 ip-10-0-137-20 kubenswrapper[2565]: W0917 17:34:11.945629    2565 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Sep 17 17:34:11.954187 ip-10-0-137-20 kubenswrapper[2565]: W0917 17:34:11.945632    2565 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Sep 17 17:34:11.954187 ip-10-0-137-20 kubenswrapper[2565]: W0917 17:34:11.945644    2565 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Sep 17 17:34:11.954187 ip-10-0-137-20 kubenswrapper[2565]: W0917 17:34:11.945647    2565 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Sep 17 17:34:11.954187 ip-10-0-137-20 kubenswrapper[2565]: W0917 17:34:11.945650    2565 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Sep 17 17:34:11.954187 ip-10-0-137-20 kubenswrapper[2565]: W0917 17:34:11.945653    2565 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Sep 17 17:34:11.954187 ip-10-0-137-20 kubenswrapper[2565]: W0917 17:34:11.945655    2565 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Sep 17 17:34:11.954187 ip-10-0-137-20 kubenswrapper[2565]: W0917 17:34:11.945658    2565 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Sep 17 17:34:11.954187 ip-10-0-137-20 kubenswrapper[2565]: W0917 17:34:11.945661    2565 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Sep 17 17:34:11.954187 ip-10-0-137-20 kubenswrapper[2565]: W0917 17:34:11.945663    2565 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Sep 17 17:34:11.954187 ip-10-0-137-20 kubenswrapper[2565]: W0917 17:34:11.945666    2565 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Sep 17 17:34:11.954187 ip-10-0-137-20 kubenswrapper[2565]: W0917 17:34:11.945669    2565 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Sep 17 17:34:11.954187 ip-10-0-137-20 kubenswrapper[2565]: W0917 17:34:11.945671    2565 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Sep 17 17:34:11.954670 ip-10-0-137-20 kubenswrapper[2565]: W0917 17:34:11.945674    2565 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Sep 17 17:34:11.954670 ip-10-0-137-20 kubenswrapper[2565]: W0917 17:34:11.945677    2565 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Sep 17 17:34:11.954670 ip-10-0-137-20 kubenswrapper[2565]: W0917 17:34:11.945680    2565 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Sep 17 17:34:11.954670 ip-10-0-137-20 kubenswrapper[2565]: W0917 17:34:11.945682    2565 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Sep 17 17:34:11.954670 ip-10-0-137-20 kubenswrapper[2565]: W0917 17:34:11.945685    2565 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Sep 17 17:34:11.954670 ip-10-0-137-20 kubenswrapper[2565]: W0917 17:34:11.945687    2565 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Sep 17 17:34:11.954670 ip-10-0-137-20 kubenswrapper[2565]: W0917 17:34:11.945690    2565 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Sep 17 17:34:11.954670 ip-10-0-137-20 kubenswrapper[2565]: W0917 17:34:11.945693    2565 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Sep 17 17:34:11.954670 ip-10-0-137-20 kubenswrapper[2565]: W0917 17:34:11.945695    2565 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Sep 17 17:34:11.954670 ip-10-0-137-20 kubenswrapper[2565]: W0917 17:34:11.945698    2565 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Sep 17 17:34:11.954670 ip-10-0-137-20 kubenswrapper[2565]: W0917 17:34:11.945700    2565 feature_gate.go:328] unrecognized feature gate: Example2
Sep 17 17:34:11.954670 ip-10-0-137-20 kubenswrapper[2565]: W0917 17:34:11.945704    2565 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Sep 17 17:34:11.954670 ip-10-0-137-20 kubenswrapper[2565]: W0917 17:34:11.945707    2565 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Sep 17 17:34:11.954670 ip-10-0-137-20 kubenswrapper[2565]: W0917 17:34:11.945710    2565 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Sep 17 17:34:11.954670 ip-10-0-137-20 kubenswrapper[2565]: W0917 17:34:11.945713    2565 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Sep 17 17:34:11.954670 ip-10-0-137-20 kubenswrapper[2565]: W0917 17:34:11.945716    2565 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Sep 17 17:34:11.954670 ip-10-0-137-20 kubenswrapper[2565]: W0917 17:34:11.945719    2565 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Sep 17 17:34:11.954670 ip-10-0-137-20 kubenswrapper[2565]: W0917 17:34:11.945721    2565 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Sep 17 17:34:11.954670 ip-10-0-137-20 kubenswrapper[2565]: W0917 17:34:11.945724    2565 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Sep 17 17:34:11.954670 ip-10-0-137-20 kubenswrapper[2565]: W0917 17:34:11.945728    2565 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Sep 17 17:34:11.955176 ip-10-0-137-20 kubenswrapper[2565]: W0917 17:34:11.945730    2565 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Sep 17 17:34:11.955176 ip-10-0-137-20 kubenswrapper[2565]: W0917 17:34:11.945733    2565 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Sep 17 17:34:11.955176 ip-10-0-137-20 kubenswrapper[2565]: W0917 17:34:11.945735    2565 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Sep 17 17:34:11.955176 ip-10-0-137-20 kubenswrapper[2565]: W0917 17:34:11.945738    2565 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Sep 17 17:34:11.955176 ip-10-0-137-20 kubenswrapper[2565]: W0917 17:34:11.945741    2565 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Sep 17 17:34:11.955176 ip-10-0-137-20 kubenswrapper[2565]: W0917 17:34:11.945743    2565 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Sep 17 17:34:11.955176 ip-10-0-137-20 kubenswrapper[2565]: W0917 17:34:11.945746    2565 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Sep 17 17:34:11.955176 ip-10-0-137-20 kubenswrapper[2565]: W0917 17:34:11.945749    2565 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Sep 17 17:34:11.955176 ip-10-0-137-20 kubenswrapper[2565]: W0917 17:34:11.945751    2565 feature_gate.go:328] unrecognized feature gate: SignatureStores
Sep 17 17:34:11.955176 ip-10-0-137-20 kubenswrapper[2565]: W0917 17:34:11.945754    2565 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Sep 17 17:34:11.955176 ip-10-0-137-20 kubenswrapper[2565]: W0917 17:34:11.945756    2565 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Sep 17 17:34:11.955176 ip-10-0-137-20 kubenswrapper[2565]: W0917 17:34:11.945759    2565 feature_gate.go:328] unrecognized feature gate: DualReplica
Sep 17 17:34:11.955176 ip-10-0-137-20 kubenswrapper[2565]: W0917 17:34:11.945762    2565 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Sep 17 17:34:11.955176 ip-10-0-137-20 kubenswrapper[2565]: W0917 17:34:11.945764    2565 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Sep 17 17:34:11.955176 ip-10-0-137-20 kubenswrapper[2565]: W0917 17:34:11.945767    2565 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Sep 17 17:34:11.955176 ip-10-0-137-20 kubenswrapper[2565]: W0917 17:34:11.945769    2565 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Sep 17 17:34:11.955176 ip-10-0-137-20 kubenswrapper[2565]: W0917 17:34:11.945772    2565 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Sep 17 17:34:11.955176 ip-10-0-137-20 kubenswrapper[2565]: W0917 17:34:11.945774    2565 feature_gate.go:328] unrecognized feature gate: Example
Sep 17 17:34:11.955176 ip-10-0-137-20 kubenswrapper[2565]: W0917 17:34:11.945777    2565 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Sep 17 17:34:11.955176 ip-10-0-137-20 kubenswrapper[2565]: W0917 17:34:11.945779    2565 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Sep 17 17:34:11.955681 ip-10-0-137-20 kubenswrapper[2565]: W0917 17:34:11.945782    2565 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Sep 17 17:34:11.955681 ip-10-0-137-20 kubenswrapper[2565]: W0917 17:34:11.945785    2565 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Sep 17 17:34:11.955681 ip-10-0-137-20 kubenswrapper[2565]: W0917 17:34:11.945787    2565 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Sep 17 17:34:11.955681 ip-10-0-137-20 kubenswrapper[2565]: W0917 17:34:11.945791    2565 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Sep 17 17:34:11.955681 ip-10-0-137-20 kubenswrapper[2565]: W0917 17:34:11.945794    2565 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Sep 17 17:34:11.955681 ip-10-0-137-20 kubenswrapper[2565]: W0917 17:34:11.945796    2565 feature_gate.go:328] unrecognized feature gate: OSStreams
Sep 17 17:34:11.955681 ip-10-0-137-20 kubenswrapper[2565]: W0917 17:34:11.945799    2565 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Sep 17 17:34:11.955681 ip-10-0-137-20 kubenswrapper[2565]: W0917 17:34:11.945802    2565 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Sep 17 17:34:11.955681 ip-10-0-137-20 kubenswrapper[2565]: W0917 17:34:11.945805    2565 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Sep 17 17:34:11.955681 ip-10-0-137-20 kubenswrapper[2565]: W0917 17:34:11.945807    2565 feature_gate.go:328] unrecognized feature gate: OVNObservability
Sep 17 17:34:11.955681 ip-10-0-137-20 kubenswrapper[2565]: W0917 17:34:11.945810    2565 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Sep 17 17:34:11.955681 ip-10-0-137-20 kubenswrapper[2565]: W0917 17:34:11.945814    2565 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Sep 17 17:34:11.955681 ip-10-0-137-20 kubenswrapper[2565]: W0917 17:34:11.945816    2565 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Sep 17 17:34:11.955681 ip-10-0-137-20 kubenswrapper[2565]: W0917 17:34:11.945819    2565 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Sep 17 17:34:11.955681 ip-10-0-137-20 kubenswrapper[2565]: W0917 17:34:11.945823    2565 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Sep 17 17:34:11.955681 ip-10-0-137-20 kubenswrapper[2565]: W0917 17:34:11.945826    2565 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Sep 17 17:34:11.955681 ip-10-0-137-20 kubenswrapper[2565]: W0917 17:34:11.945829    2565 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Sep 17 17:34:11.955681 ip-10-0-137-20 kubenswrapper[2565]: W0917 17:34:11.945832    2565 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Sep 17 17:34:11.955681 ip-10-0-137-20 kubenswrapper[2565]: W0917 17:34:11.945835    2565 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Sep 17 17:34:11.956169 ip-10-0-137-20 kubenswrapper[2565]: W0917 17:34:11.945838    2565 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Sep 17 17:34:11.956169 ip-10-0-137-20 kubenswrapper[2565]: W0917 17:34:11.945841    2565 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Sep 17 17:34:11.956169 ip-10-0-137-20 kubenswrapper[2565]: W0917 17:34:11.945843    2565 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Sep 17 17:34:11.956169 ip-10-0-137-20 kubenswrapper[2565]: W0917 17:34:11.945846    2565 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Sep 17 17:34:11.956169 ip-10-0-137-20 kubenswrapper[2565]: W0917 17:34:11.945849    2565 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Sep 17 17:34:11.956169 ip-10-0-137-20 kubenswrapper[2565]: W0917 17:34:11.945851    2565 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Sep 17 17:34:11.956169 ip-10-0-137-20 kubenswrapper[2565]: W0917 17:34:11.945854    2565 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Sep 17 17:34:11.956169 ip-10-0-137-20 kubenswrapper[2565]: W0917 17:34:11.945857    2565 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Sep 17 17:34:11.956169 ip-10-0-137-20 kubenswrapper[2565]: W0917 17:34:11.945859    2565 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Sep 17 17:34:11.956169 ip-10-0-137-20 kubenswrapper[2565]: W0917 17:34:11.945862    2565 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Sep 17 17:34:11.956169 ip-10-0-137-20 kubenswrapper[2565]: W0917 17:34:11.945864    2565 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Sep 17 17:34:11.956169 ip-10-0-137-20 kubenswrapper[2565]: W0917 17:34:11.945867    2565 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Sep 17 17:34:11.956169 ip-10-0-137-20 kubenswrapper[2565]: W0917 17:34:11.945869    2565 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Sep 17 17:34:11.956169 ip-10-0-137-20 kubenswrapper[2565]: W0917 17:34:11.945872    2565 feature_gate.go:328] unrecognized feature gate: EventTTL
Sep 17 17:34:11.956169 ip-10-0-137-20 kubenswrapper[2565]: W0917 17:34:11.945876    2565 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Sep 17 17:34:11.956169 ip-10-0-137-20 kubenswrapper[2565]: W0917 17:34:11.945879    2565 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Sep 17 17:34:11.956169 ip-10-0-137-20 kubenswrapper[2565]: W0917 17:34:11.945883    2565 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Sep 17 17:34:11.956169 ip-10-0-137-20 kubenswrapper[2565]: W0917 17:34:11.945886    2565 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Sep 17 17:34:11.956169 ip-10-0-137-20 kubenswrapper[2565]: W0917 17:34:11.945889    2565 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Sep 17 17:34:11.956634 ip-10-0-137-20 kubenswrapper[2565]: W0917 17:34:11.945892    2565 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Sep 17 17:34:11.956634 ip-10-0-137-20 kubenswrapper[2565]: W0917 17:34:11.945895    2565 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Sep 17 17:34:11.956634 ip-10-0-137-20 kubenswrapper[2565]: W0917 17:34:11.945898    2565 feature_gate.go:328] unrecognized feature gate: PinnedImages
Sep 17 17:34:11.956634 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:34:11.946527    2565 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]}
Sep 17 17:34:11.956634 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:34:11.953414    2565 server.go:529] "Kubelet version" kubeletVersion="v1.34.9"
Sep 17 17:34:11.956634 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:34:11.953428    2565 server.go:531] "Golang settings" GOGC="" GOMAXPROCS="" GOTRACEBACK=""
Sep 17 17:34:11.956634 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:34:11.953447    2565 watchdog_linux.go:95] "Systemd watchdog is not enabled"
Sep 17 17:34:11.956634 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:34:11.953452    2565 watchdog_linux.go:137] "Systemd watchdog is not enabled or the interval is invalid, so health checking will not be started."
Sep 17 17:34:11.956634 ip-10-0-137-20 kubenswrapper[2565]: W0917 17:34:11.953507    2565 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Sep 17 17:34:11.956634 ip-10-0-137-20 kubenswrapper[2565]: W0917 17:34:11.953511    2565 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Sep 17 17:34:11.956634 ip-10-0-137-20 kubenswrapper[2565]: W0917 17:34:11.953514    2565 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Sep 17 17:34:11.956634 ip-10-0-137-20 kubenswrapper[2565]: W0917 17:34:11.953517    2565 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Sep 17 17:34:11.956634 ip-10-0-137-20 kubenswrapper[2565]: W0917 17:34:11.953520    2565 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Sep 17 17:34:11.956634 ip-10-0-137-20 kubenswrapper[2565]: W0917 17:34:11.953522    2565 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Sep 17 17:34:11.956634 ip-10-0-137-20 kubenswrapper[2565]: W0917 17:34:11.953525    2565 feature_gate.go:328] unrecognized feature gate: DualReplica
Sep 17 17:34:11.957052 ip-10-0-137-20 kubenswrapper[2565]: W0917 17:34:11.953528    2565 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Sep 17 17:34:11.957052 ip-10-0-137-20 kubenswrapper[2565]: W0917 17:34:11.953531    2565 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Sep 17 17:34:11.957052 ip-10-0-137-20 kubenswrapper[2565]: W0917 17:34:11.953533    2565 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Sep 17 17:34:11.957052 ip-10-0-137-20 kubenswrapper[2565]: W0917 17:34:11.953536    2565 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Sep 17 17:34:11.957052 ip-10-0-137-20 kubenswrapper[2565]: W0917 17:34:11.953538    2565 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Sep 17 17:34:11.957052 ip-10-0-137-20 kubenswrapper[2565]: W0917 17:34:11.953541    2565 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Sep 17 17:34:11.957052 ip-10-0-137-20 kubenswrapper[2565]: W0917 17:34:11.953544    2565 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Sep 17 17:34:11.957052 ip-10-0-137-20 kubenswrapper[2565]: W0917 17:34:11.953547    2565 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Sep 17 17:34:11.957052 ip-10-0-137-20 kubenswrapper[2565]: W0917 17:34:11.953549    2565 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Sep 17 17:34:11.957052 ip-10-0-137-20 kubenswrapper[2565]: W0917 17:34:11.953552    2565 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Sep 17 17:34:11.957052 ip-10-0-137-20 kubenswrapper[2565]: W0917 17:34:11.953555    2565 feature_gate.go:328] unrecognized feature gate: PinnedImages
Sep 17 17:34:11.957052 ip-10-0-137-20 kubenswrapper[2565]: W0917 17:34:11.953558    2565 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Sep 17 17:34:11.957052 ip-10-0-137-20 kubenswrapper[2565]: W0917 17:34:11.953560    2565 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Sep 17 17:34:11.957052 ip-10-0-137-20 kubenswrapper[2565]: W0917 17:34:11.953563    2565 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Sep 17 17:34:11.957052 ip-10-0-137-20 kubenswrapper[2565]: W0917 17:34:11.953565    2565 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Sep 17 17:34:11.957052 ip-10-0-137-20 kubenswrapper[2565]: W0917 17:34:11.953568    2565 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Sep 17 17:34:11.957052 ip-10-0-137-20 kubenswrapper[2565]: W0917 17:34:11.953571    2565 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Sep 17 17:34:11.957052 ip-10-0-137-20 kubenswrapper[2565]: W0917 17:34:11.953574    2565 feature_gate.go:328] unrecognized feature gate: SignatureStores
Sep 17 17:34:11.957052 ip-10-0-137-20 kubenswrapper[2565]: W0917 17:34:11.953577    2565 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Sep 17 17:34:11.957052 ip-10-0-137-20 kubenswrapper[2565]: W0917 17:34:11.953580    2565 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Sep 17 17:34:11.957525 ip-10-0-137-20 kubenswrapper[2565]: W0917 17:34:11.953582    2565 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Sep 17 17:34:11.957525 ip-10-0-137-20 kubenswrapper[2565]: W0917 17:34:11.953585    2565 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Sep 17 17:34:11.957525 ip-10-0-137-20 kubenswrapper[2565]: W0917 17:34:11.953589    2565 feature_gate.go:328] unrecognized feature gate: EventTTL
Sep 17 17:34:11.957525 ip-10-0-137-20 kubenswrapper[2565]: W0917 17:34:11.953592    2565 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Sep 17 17:34:11.957525 ip-10-0-137-20 kubenswrapper[2565]: W0917 17:34:11.953594    2565 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Sep 17 17:34:11.957525 ip-10-0-137-20 kubenswrapper[2565]: W0917 17:34:11.953597    2565 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Sep 17 17:34:11.957525 ip-10-0-137-20 kubenswrapper[2565]: W0917 17:34:11.953599    2565 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Sep 17 17:34:11.957525 ip-10-0-137-20 kubenswrapper[2565]: W0917 17:34:11.953602    2565 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Sep 17 17:34:11.957525 ip-10-0-137-20 kubenswrapper[2565]: W0917 17:34:11.953605    2565 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Sep 17 17:34:11.957525 ip-10-0-137-20 kubenswrapper[2565]: W0917 17:34:11.953607    2565 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Sep 17 17:34:11.957525 ip-10-0-137-20 kubenswrapper[2565]: W0917 17:34:11.953611    2565 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Sep 17 17:34:11.957525 ip-10-0-137-20 kubenswrapper[2565]: W0917 17:34:11.953615    2565 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Sep 17 17:34:11.957525 ip-10-0-137-20 kubenswrapper[2565]: W0917 17:34:11.953618    2565 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Sep 17 17:34:11.957525 ip-10-0-137-20 kubenswrapper[2565]: W0917 17:34:11.953621    2565 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Sep 17 17:34:11.957525 ip-10-0-137-20 kubenswrapper[2565]: W0917 17:34:11.953623    2565 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Sep 17 17:34:11.957525 ip-10-0-137-20 kubenswrapper[2565]: W0917 17:34:11.953627    2565 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Sep 17 17:34:11.957525 ip-10-0-137-20 kubenswrapper[2565]: W0917 17:34:11.953631    2565 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Sep 17 17:34:11.957525 ip-10-0-137-20 kubenswrapper[2565]: W0917 17:34:11.953634    2565 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Sep 17 17:34:11.957525 ip-10-0-137-20 kubenswrapper[2565]: W0917 17:34:11.953637    2565 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Sep 17 17:34:11.957996 ip-10-0-137-20 kubenswrapper[2565]: W0917 17:34:11.953639    2565 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Sep 17 17:34:11.957996 ip-10-0-137-20 kubenswrapper[2565]: W0917 17:34:11.953642    2565 feature_gate.go:328] unrecognized feature gate: OSStreams
Sep 17 17:34:11.957996 ip-10-0-137-20 kubenswrapper[2565]: W0917 17:34:11.953645    2565 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Sep 17 17:34:11.957996 ip-10-0-137-20 kubenswrapper[2565]: W0917 17:34:11.953647    2565 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Sep 17 17:34:11.957996 ip-10-0-137-20 kubenswrapper[2565]: W0917 17:34:11.953650    2565 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Sep 17 17:34:11.957996 ip-10-0-137-20 kubenswrapper[2565]: W0917 17:34:11.953653    2565 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Sep 17 17:34:11.957996 ip-10-0-137-20 kubenswrapper[2565]: W0917 17:34:11.953655    2565 feature_gate.go:328] unrecognized feature gate: NewOLM
Sep 17 17:34:11.957996 ip-10-0-137-20 kubenswrapper[2565]: W0917 17:34:11.953658    2565 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Sep 17 17:34:11.957996 ip-10-0-137-20 kubenswrapper[2565]: W0917 17:34:11.953660    2565 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Sep 17 17:34:11.957996 ip-10-0-137-20 kubenswrapper[2565]: W0917 17:34:11.953664    2565 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Sep 17 17:34:11.957996 ip-10-0-137-20 kubenswrapper[2565]: W0917 17:34:11.953667    2565 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Sep 17 17:34:11.957996 ip-10-0-137-20 kubenswrapper[2565]: W0917 17:34:11.953671    2565 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Sep 17 17:34:11.957996 ip-10-0-137-20 kubenswrapper[2565]: W0917 17:34:11.953673    2565 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Sep 17 17:34:11.957996 ip-10-0-137-20 kubenswrapper[2565]: W0917 17:34:11.953676    2565 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Sep 17 17:34:11.957996 ip-10-0-137-20 kubenswrapper[2565]: W0917 17:34:11.953679    2565 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Sep 17 17:34:11.957996 ip-10-0-137-20 kubenswrapper[2565]: W0917 17:34:11.953682    2565 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Sep 17 17:34:11.957996 ip-10-0-137-20 kubenswrapper[2565]: W0917 17:34:11.953685    2565 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Sep 17 17:34:11.957996 ip-10-0-137-20 kubenswrapper[2565]: W0917 17:34:11.953687    2565 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Sep 17 17:34:11.957996 ip-10-0-137-20 kubenswrapper[2565]: W0917 17:34:11.953690    2565 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Sep 17 17:34:11.957996 ip-10-0-137-20 kubenswrapper[2565]: W0917 17:34:11.953692    2565 feature_gate.go:328] unrecognized feature gate: Example
Sep 17 17:34:11.958500 ip-10-0-137-20 kubenswrapper[2565]: W0917 17:34:11.953695    2565 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Sep 17 17:34:11.958500 ip-10-0-137-20 kubenswrapper[2565]: W0917 17:34:11.953698    2565 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Sep 17 17:34:11.958500 ip-10-0-137-20 kubenswrapper[2565]: W0917 17:34:11.953700    2565 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Sep 17 17:34:11.958500 ip-10-0-137-20 kubenswrapper[2565]: W0917 17:34:11.953703    2565 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Sep 17 17:34:11.958500 ip-10-0-137-20 kubenswrapper[2565]: W0917 17:34:11.953705    2565 feature_gate.go:328] unrecognized feature gate: OVNObservability
Sep 17 17:34:11.958500 ip-10-0-137-20 kubenswrapper[2565]: W0917 17:34:11.953708    2565 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Sep 17 17:34:11.958500 ip-10-0-137-20 kubenswrapper[2565]: W0917 17:34:11.953710    2565 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Sep 17 17:34:11.958500 ip-10-0-137-20 kubenswrapper[2565]: W0917 17:34:11.953713    2565 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Sep 17 17:34:11.958500 ip-10-0-137-20 kubenswrapper[2565]: W0917 17:34:11.953715    2565 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Sep 17 17:34:11.958500 ip-10-0-137-20 kubenswrapper[2565]: W0917 17:34:11.953718    2565 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Sep 17 17:34:11.958500 ip-10-0-137-20 kubenswrapper[2565]: W0917 17:34:11.953721    2565 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Sep 17 17:34:11.958500 ip-10-0-137-20 kubenswrapper[2565]: W0917 17:34:11.953723    2565 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Sep 17 17:34:11.958500 ip-10-0-137-20 kubenswrapper[2565]: W0917 17:34:11.953726    2565 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Sep 17 17:34:11.958500 ip-10-0-137-20 kubenswrapper[2565]: W0917 17:34:11.953729    2565 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Sep 17 17:34:11.958500 ip-10-0-137-20 kubenswrapper[2565]: W0917 17:34:11.953731    2565 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Sep 17 17:34:11.958500 ip-10-0-137-20 kubenswrapper[2565]: W0917 17:34:11.953734    2565 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Sep 17 17:34:11.958500 ip-10-0-137-20 kubenswrapper[2565]: W0917 17:34:11.953737    2565 feature_gate.go:328] unrecognized feature gate: Example2
Sep 17 17:34:11.958500 ip-10-0-137-20 kubenswrapper[2565]: W0917 17:34:11.953739    2565 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Sep 17 17:34:11.958500 ip-10-0-137-20 kubenswrapper[2565]: W0917 17:34:11.953742    2565 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Sep 17 17:34:11.958500 ip-10-0-137-20 kubenswrapper[2565]: W0917 17:34:11.953745    2565 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Sep 17 17:34:11.958985 ip-10-0-137-20 kubenswrapper[2565]: W0917 17:34:11.953747    2565 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Sep 17 17:34:11.958985 ip-10-0-137-20 kubenswrapper[2565]: W0917 17:34:11.953750    2565 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Sep 17 17:34:11.958985 ip-10-0-137-20 kubenswrapper[2565]: W0917 17:34:11.953752    2565 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Sep 17 17:34:11.958985 ip-10-0-137-20 kubenswrapper[2565]: W0917 17:34:11.953755    2565 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Sep 17 17:34:11.958985 ip-10-0-137-20 kubenswrapper[2565]: W0917 17:34:11.953758    2565 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Sep 17 17:34:11.958985 ip-10-0-137-20 kubenswrapper[2565]: W0917 17:34:11.953760    2565 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Sep 17 17:34:11.958985 ip-10-0-137-20 kubenswrapper[2565]: W0917 17:34:11.953763    2565 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Sep 17 17:34:11.958985 ip-10-0-137-20 kubenswrapper[2565]: W0917 17:34:11.953766    2565 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Sep 17 17:34:11.958985 ip-10-0-137-20 kubenswrapper[2565]: W0917 17:34:11.953769    2565 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Sep 17 17:34:11.958985 ip-10-0-137-20 kubenswrapper[2565]: W0917 17:34:11.953772    2565 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Sep 17 17:34:11.958985 ip-10-0-137-20 kubenswrapper[2565]: W0917 17:34:11.953774    2565 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Sep 17 17:34:11.958985 ip-10-0-137-20 kubenswrapper[2565]: W0917 17:34:11.953777    2565 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Sep 17 17:34:11.958985 ip-10-0-137-20 kubenswrapper[2565]: W0917 17:34:11.953780    2565 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Sep 17 17:34:11.958985 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:34:11.953784    2565 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]}
Sep 17 17:34:11.959435 ip-10-0-137-20 kubenswrapper[2565]: W0917 17:34:11.953886    2565 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Sep 17 17:34:11.959435 ip-10-0-137-20 kubenswrapper[2565]: W0917 17:34:11.953890    2565 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Sep 17 17:34:11.959435 ip-10-0-137-20 kubenswrapper[2565]: W0917 17:34:11.953893    2565 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Sep 17 17:34:11.959435 ip-10-0-137-20 kubenswrapper[2565]: W0917 17:34:11.953896    2565 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Sep 17 17:34:11.959435 ip-10-0-137-20 kubenswrapper[2565]: W0917 17:34:11.953899    2565 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Sep 17 17:34:11.959435 ip-10-0-137-20 kubenswrapper[2565]: W0917 17:34:11.953902    2565 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Sep 17 17:34:11.959435 ip-10-0-137-20 kubenswrapper[2565]: W0917 17:34:11.953905    2565 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Sep 17 17:34:11.959435 ip-10-0-137-20 kubenswrapper[2565]: W0917 17:34:11.953908    2565 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Sep 17 17:34:11.959435 ip-10-0-137-20 kubenswrapper[2565]: W0917 17:34:11.953911    2565 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Sep 17 17:34:11.959435 ip-10-0-137-20 kubenswrapper[2565]: W0917 17:34:11.953913    2565 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Sep 17 17:34:11.959435 ip-10-0-137-20 kubenswrapper[2565]: W0917 17:34:11.953916    2565 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Sep 17 17:34:11.959435 ip-10-0-137-20 kubenswrapper[2565]: W0917 17:34:11.953919    2565 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Sep 17 17:34:11.959435 ip-10-0-137-20 kubenswrapper[2565]: W0917 17:34:11.953922    2565 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Sep 17 17:34:11.959435 ip-10-0-137-20 kubenswrapper[2565]: W0917 17:34:11.953924    2565 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Sep 17 17:34:11.959435 ip-10-0-137-20 kubenswrapper[2565]: W0917 17:34:11.953927    2565 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Sep 17 17:34:11.959435 ip-10-0-137-20 kubenswrapper[2565]: W0917 17:34:11.953929    2565 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Sep 17 17:34:11.959435 ip-10-0-137-20 kubenswrapper[2565]: W0917 17:34:11.953931    2565 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Sep 17 17:34:11.959435 ip-10-0-137-20 kubenswrapper[2565]: W0917 17:34:11.953935    2565 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Sep 17 17:34:11.959435 ip-10-0-137-20 kubenswrapper[2565]: W0917 17:34:11.953937    2565 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Sep 17 17:34:11.959903 ip-10-0-137-20 kubenswrapper[2565]: W0917 17:34:11.953940    2565 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Sep 17 17:34:11.959903 ip-10-0-137-20 kubenswrapper[2565]: W0917 17:34:11.953942    2565 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Sep 17 17:34:11.959903 ip-10-0-137-20 kubenswrapper[2565]: W0917 17:34:11.953945    2565 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Sep 17 17:34:11.959903 ip-10-0-137-20 kubenswrapper[2565]: W0917 17:34:11.953948    2565 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Sep 17 17:34:11.959903 ip-10-0-137-20 kubenswrapper[2565]: W0917 17:34:11.953953    2565 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Sep 17 17:34:11.959903 ip-10-0-137-20 kubenswrapper[2565]: W0917 17:34:11.953956    2565 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Sep 17 17:34:11.959903 ip-10-0-137-20 kubenswrapper[2565]: W0917 17:34:11.953959    2565 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Sep 17 17:34:11.959903 ip-10-0-137-20 kubenswrapper[2565]: W0917 17:34:11.953962    2565 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Sep 17 17:34:11.959903 ip-10-0-137-20 kubenswrapper[2565]: W0917 17:34:11.953965    2565 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Sep 17 17:34:11.959903 ip-10-0-137-20 kubenswrapper[2565]: W0917 17:34:11.953968    2565 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Sep 17 17:34:11.959903 ip-10-0-137-20 kubenswrapper[2565]: W0917 17:34:11.953970    2565 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Sep 17 17:34:11.959903 ip-10-0-137-20 kubenswrapper[2565]: W0917 17:34:11.953973    2565 feature_gate.go:328] unrecognized feature gate: Example
Sep 17 17:34:11.959903 ip-10-0-137-20 kubenswrapper[2565]: W0917 17:34:11.953976    2565 feature_gate.go:328] unrecognized feature gate: NewOLM
Sep 17 17:34:11.959903 ip-10-0-137-20 kubenswrapper[2565]: W0917 17:34:11.953979    2565 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Sep 17 17:34:11.959903 ip-10-0-137-20 kubenswrapper[2565]: W0917 17:34:11.953981    2565 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Sep 17 17:34:11.959903 ip-10-0-137-20 kubenswrapper[2565]: W0917 17:34:11.953984    2565 feature_gate.go:328] unrecognized feature gate: OVNObservability
Sep 17 17:34:11.959903 ip-10-0-137-20 kubenswrapper[2565]: W0917 17:34:11.953986    2565 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Sep 17 17:34:11.959903 ip-10-0-137-20 kubenswrapper[2565]: W0917 17:34:11.953989    2565 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Sep 17 17:34:11.959903 ip-10-0-137-20 kubenswrapper[2565]: W0917 17:34:11.953991    2565 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Sep 17 17:34:11.960374 ip-10-0-137-20 kubenswrapper[2565]: W0917 17:34:11.953995    2565 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Sep 17 17:34:11.960374 ip-10-0-137-20 kubenswrapper[2565]: W0917 17:34:11.953998    2565 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Sep 17 17:34:11.960374 ip-10-0-137-20 kubenswrapper[2565]: W0917 17:34:11.954001    2565 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Sep 17 17:34:11.960374 ip-10-0-137-20 kubenswrapper[2565]: W0917 17:34:11.954004    2565 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Sep 17 17:34:11.960374 ip-10-0-137-20 kubenswrapper[2565]: W0917 17:34:11.954006    2565 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Sep 17 17:34:11.960374 ip-10-0-137-20 kubenswrapper[2565]: W0917 17:34:11.954009    2565 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Sep 17 17:34:11.960374 ip-10-0-137-20 kubenswrapper[2565]: W0917 17:34:11.954012    2565 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Sep 17 17:34:11.960374 ip-10-0-137-20 kubenswrapper[2565]: W0917 17:34:11.954014    2565 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Sep 17 17:34:11.960374 ip-10-0-137-20 kubenswrapper[2565]: W0917 17:34:11.954017    2565 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Sep 17 17:34:11.960374 ip-10-0-137-20 kubenswrapper[2565]: W0917 17:34:11.954038    2565 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Sep 17 17:34:11.960374 ip-10-0-137-20 kubenswrapper[2565]: W0917 17:34:11.954043    2565 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Sep 17 17:34:11.960374 ip-10-0-137-20 kubenswrapper[2565]: W0917 17:34:11.954046    2565 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Sep 17 17:34:11.960374 ip-10-0-137-20 kubenswrapper[2565]: W0917 17:34:11.954049    2565 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Sep 17 17:34:11.960374 ip-10-0-137-20 kubenswrapper[2565]: W0917 17:34:11.954051    2565 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Sep 17 17:34:11.960374 ip-10-0-137-20 kubenswrapper[2565]: W0917 17:34:11.954054    2565 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Sep 17 17:34:11.960374 ip-10-0-137-20 kubenswrapper[2565]: W0917 17:34:11.954056    2565 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Sep 17 17:34:11.960374 ip-10-0-137-20 kubenswrapper[2565]: W0917 17:34:11.954060    2565 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Sep 17 17:34:11.960374 ip-10-0-137-20 kubenswrapper[2565]: W0917 17:34:11.954062    2565 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Sep 17 17:34:11.960374 ip-10-0-137-20 kubenswrapper[2565]: W0917 17:34:11.954066    2565 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Sep 17 17:34:11.960828 ip-10-0-137-20 kubenswrapper[2565]: W0917 17:34:11.954068    2565 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Sep 17 17:34:11.960828 ip-10-0-137-20 kubenswrapper[2565]: W0917 17:34:11.954071    2565 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Sep 17 17:34:11.960828 ip-10-0-137-20 kubenswrapper[2565]: W0917 17:34:11.954074    2565 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Sep 17 17:34:11.960828 ip-10-0-137-20 kubenswrapper[2565]: W0917 17:34:11.954076    2565 feature_gate.go:328] unrecognized feature gate: DualReplica
Sep 17 17:34:11.960828 ip-10-0-137-20 kubenswrapper[2565]: W0917 17:34:11.954079    2565 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Sep 17 17:34:11.960828 ip-10-0-137-20 kubenswrapper[2565]: W0917 17:34:11.954081    2565 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Sep 17 17:34:11.960828 ip-10-0-137-20 kubenswrapper[2565]: W0917 17:34:11.954084    2565 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Sep 17 17:34:11.960828 ip-10-0-137-20 kubenswrapper[2565]: W0917 17:34:11.954086    2565 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Sep 17 17:34:11.960828 ip-10-0-137-20 kubenswrapper[2565]: W0917 17:34:11.954089    2565 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Sep 17 17:34:11.960828 ip-10-0-137-20 kubenswrapper[2565]: W0917 17:34:11.954091    2565 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Sep 17 17:34:11.960828 ip-10-0-137-20 kubenswrapper[2565]: W0917 17:34:11.954094    2565 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Sep 17 17:34:11.960828 ip-10-0-137-20 kubenswrapper[2565]: W0917 17:34:11.954096    2565 feature_gate.go:328] unrecognized feature gate: Example2
Sep 17 17:34:11.960828 ip-10-0-137-20 kubenswrapper[2565]: W0917 17:34:11.954099    2565 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Sep 17 17:34:11.960828 ip-10-0-137-20 kubenswrapper[2565]: W0917 17:34:11.954101    2565 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Sep 17 17:34:11.960828 ip-10-0-137-20 kubenswrapper[2565]: W0917 17:34:11.954104    2565 feature_gate.go:328] unrecognized feature gate: SignatureStores
Sep 17 17:34:11.960828 ip-10-0-137-20 kubenswrapper[2565]: W0917 17:34:11.954107    2565 feature_gate.go:328] unrecognized feature gate: PinnedImages
Sep 17 17:34:11.960828 ip-10-0-137-20 kubenswrapper[2565]: W0917 17:34:11.954109    2565 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Sep 17 17:34:11.960828 ip-10-0-137-20 kubenswrapper[2565]: W0917 17:34:11.954112    2565 feature_gate.go:328] unrecognized feature gate: EventTTL
Sep 17 17:34:11.960828 ip-10-0-137-20 kubenswrapper[2565]: W0917 17:34:11.954114    2565 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Sep 17 17:34:11.960828 ip-10-0-137-20 kubenswrapper[2565]: W0917 17:34:11.954117    2565 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Sep 17 17:34:11.961316 ip-10-0-137-20 kubenswrapper[2565]: W0917 17:34:11.954119    2565 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Sep 17 17:34:11.961316 ip-10-0-137-20 kubenswrapper[2565]: W0917 17:34:11.954122    2565 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Sep 17 17:34:11.961316 ip-10-0-137-20 kubenswrapper[2565]: W0917 17:34:11.954125    2565 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Sep 17 17:34:11.961316 ip-10-0-137-20 kubenswrapper[2565]: W0917 17:34:11.954127    2565 feature_gate.go:328] unrecognized feature gate: OSStreams
Sep 17 17:34:11.961316 ip-10-0-137-20 kubenswrapper[2565]: W0917 17:34:11.954130    2565 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Sep 17 17:34:11.961316 ip-10-0-137-20 kubenswrapper[2565]: W0917 17:34:11.954132    2565 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Sep 17 17:34:11.961316 ip-10-0-137-20 kubenswrapper[2565]: W0917 17:34:11.954135    2565 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Sep 17 17:34:11.961316 ip-10-0-137-20 kubenswrapper[2565]: W0917 17:34:11.954137    2565 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Sep 17 17:34:11.961316 ip-10-0-137-20 kubenswrapper[2565]: W0917 17:34:11.954140    2565 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Sep 17 17:34:11.961316 ip-10-0-137-20 kubenswrapper[2565]: W0917 17:34:11.954142    2565 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Sep 17 17:34:11.961316 ip-10-0-137-20 kubenswrapper[2565]: W0917 17:34:11.954145    2565 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Sep 17 17:34:11.961316 ip-10-0-137-20 kubenswrapper[2565]: W0917 17:34:11.954147    2565 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Sep 17 17:34:11.961316 ip-10-0-137-20 kubenswrapper[2565]: W0917 17:34:11.954150    2565 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Sep 17 17:34:11.961316 ip-10-0-137-20 kubenswrapper[2565]: W0917 17:34:11.954153    2565 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Sep 17 17:34:11.961316 ip-10-0-137-20 kubenswrapper[2565]: W0917 17:34:11.954156    2565 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Sep 17 17:34:11.961316 ip-10-0-137-20 kubenswrapper[2565]: W0917 17:34:11.954160    2565 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Sep 17 17:34:11.961316 ip-10-0-137-20 kubenswrapper[2565]: W0917 17:34:11.954163    2565 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Sep 17 17:34:11.961316 ip-10-0-137-20 kubenswrapper[2565]: W0917 17:34:11.954165    2565 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Sep 17 17:34:11.961316 ip-10-0-137-20 kubenswrapper[2565]: W0917 17:34:11.954168    2565 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Sep 17 17:34:11.961780 ip-10-0-137-20 kubenswrapper[2565]: W0917 17:34:11.954171    2565 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Sep 17 17:34:11.961780 ip-10-0-137-20 kubenswrapper[2565]: W0917 17:34:11.954173    2565 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Sep 17 17:34:11.961780 ip-10-0-137-20 kubenswrapper[2565]: W0917 17:34:11.954176    2565 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Sep 17 17:34:11.961780 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:34:11.954180    2565 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]}
Sep 17 17:34:11.961780 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:34:11.954271    2565 server.go:962] "Client rotation is on, will bootstrap in background"
Sep 17 17:34:11.961780 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:34:11.956311    2565 bootstrap.go:101] "Use the bootstrap credentials to request a cert, and set kubeconfig to point to the certificate dir"
Sep 17 17:34:11.961780 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:34:11.957094    2565 server.go:1019] "Starting client certificate rotation"
Sep 17 17:34:11.961780 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:34:11.957186    2565 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kube-apiserver-client-kubelet"
Sep 17 17:34:11.961780 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:34:11.957214    2565 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kube-apiserver-client-kubelet"
Sep 17 17:34:11.961780 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:34:11.959297    2565 dynamic_cafile_content.go:123] "Loaded a new CA Bundle and Verifier" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt"
Sep 17 17:34:11.968082 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:34:11.968042    2565 dynamic_cafile_content.go:161] "Starting controller" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt"
Sep 17 17:34:11.973965 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:34:11.973948    2565 log.go:25] "Validated CRI v1 runtime API"
Sep 17 17:34:11.975459 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:34:11.975445    2565 log.go:25] "Validated CRI v1 image API"
Sep 17 17:34:11.978894 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:34:11.978879    2565 server.go:1429] "Using cgroup driver setting received from the CRI runtime" cgroupDriver="systemd"
Sep 17 17:34:11.981044 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:34:11.981012    2565 fs.go:135] Filesystem UUIDs: map[551e6e10-1d6f-4e1e-bddb-c0399bbf6755:/dev/nvme0n1p4 7B77-95E7:/dev/nvme0n1p2 f57ad03a-9c9e-4805-ae89-c436674d23a6:/dev/nvme0n1p3]
Sep 17 17:34:11.981099 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:34:11.981043    2565 fs.go:136] Filesystem partitions: map[/dev/nvme0n1p3:{mountpoint:/boot major:259 minor:3 fsType:ext4 blockSize:0} /dev/nvme0n1p4:{mountpoint:/var major:259 minor:4 fsType:xfs blockSize:0} /dev/shm:{mountpoint:/dev/shm major:0 minor:22 fsType:tmpfs blockSize:0} /run:{mountpoint:/run major:0 minor:24 fsType:tmpfs blockSize:0} /tmp:{mountpoint:/tmp major:0 minor:31 fsType:tmpfs blockSize:0} composefs_0-33:{mountpoint:/ major:0 minor:33 fsType:overlay blockSize:0}]
Sep 17 17:34:11.985068 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:34:11.984940    2565 manager.go:216] Machine: {Timestamp:2026-09-17 17:34:11.984490501 +0000 UTC m=+1.861774907 CPUVendorID:GenuineIntel NumCores:8 NumPhysicalCores:4 NumSockets:1 CpuFrequency:3115827 MemoryCapacity:33164476416 SwapCapacity:0 MemoryByType:map[] NVMInfo:{MemoryModeCapacity:0 AppDirectModeCapacity:0 AvgPowerBudget:0} HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] MachineID:ec28a399e143ae866cab7b8707dd4e9c SystemUUID:ec28a399-e143-ae86-6cab-7b8707dd4e9c BootID:3a3dc3cc-f88c-4f7d-a3be-59ff89d8dd74 Filesystems:[{Device:/dev/nvme0n1p3 DeviceMajor:259 DeviceMinor:3 Capacity:366869504 Type:vfs Inodes:98304 HasInodes:true} {Device:/dev/shm DeviceMajor:0 DeviceMinor:22 Capacity:16582238208 Type:vfs Inodes:4048398 HasInodes:true} {Device:/run DeviceMajor:0 DeviceMinor:24 Capacity:6632898560 Type:vfs Inodes:819200 HasInodes:true} {Device:/dev/nvme0n1p4 DeviceMajor:259 DeviceMinor:4 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:composefs_0-33 DeviceMajor:0 DeviceMinor:33 Capacity:6111232 Type:vfs Inodes:18446744073709551615 HasInodes:true} {Device:/tmp DeviceMajor:0 DeviceMinor:31 Capacity:16582238208 Type:vfs Inodes:1048576 HasInodes:true}] DiskMap:map[259:0:{Name:nvme0n1 Major:259 Minor:0 Size:128849018880 Scheduler:none}] NetworkDevices:[{Name:br-ex MacAddress:02:ff:d0:f0:9f:eb Speed:0 Mtu:9001} {Name:ens5 MacAddress:02:ff:d0:f0:9f:eb Speed:0 Mtu:9001} {Name:ovs-system MacAddress:a2:cf:81:e9:94:31 Speed:0 Mtu:1500}] Topology:[{Id:0 Memory:33164476416 HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] Cores:[{Id:0 Threads:[0 4] Caches:[{Id:0 Size:32768 Type:Data Level:1} {Id:0 Size:32768 Type:Instruction Level:1} {Id:0 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:1 Threads:[1 5] Caches:[{Id:1 Size:32768 Type:Data Level:1} {Id:1 Size:32768 Type:Instruction Level:1} {Id:1 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:2 Threads:[2 6] Caches:[{Id:2 Size:32768 Type:Data Level:1} {Id:2 Size:32768 Type:Instruction Level:1} {Id:2 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:3 Threads:[3 7] Caches:[{Id:3 Size:32768 Type:Data Level:1} {Id:3 Size:32768 Type:Instruction Level:1} {Id:3 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:}] Caches:[{Id:0 Size:37486592 Type:Unified Level:3}] Distances:[10]}] CloudProvider:Unknown InstanceType:Unknown InstanceID:None}
Sep 17 17:34:11.985068 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:34:11.985055    2565 manager_no_libpfm.go:29] cAdvisor is build without cgo and/or libpfm support. Perf event counters are not available.
Sep 17 17:34:11.985213 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:34:11.985125    2565 manager.go:232] Version: {KernelVersion:5.14.0-570.141.1.el9_6.x86_64 ContainerOsVersion:Red Hat Enterprise Linux CoreOS 9.6.20260914-0 (Plow) DockerVersion: DockerAPIVersion: CadvisorVersion: CadvisorRevision:}
Sep 17 17:34:11.986567 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:34:11.986538    2565 container_manager_linux.go:273] "Container manager verified user specified cgroup-root exists" cgroupRoot=[]
Sep 17 17:34:11.986699 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:34:11.986568    2565 container_manager_linux.go:278] "Creating Container Manager object based on Node Config" nodeConfig={"NodeName":"ip-10-0-137-20.ec2.internal","RuntimeCgroupsName":"/system.slice/crio.service","SystemCgroupsName":"/system.slice","KubeletCgroupsName":"","KubeletOOMScoreAdj":-999,"ContainerRuntime":"","CgroupsPerQOS":true,"CgroupRoot":"/","CgroupDriver":"systemd","KubeletRootDir":"/var/lib/kubelet","ProtectKernelDefaults":true,"KubeReservedCgroupName":"","SystemReservedCgroupName":"","ReservedSystemCPUs":{},"EnforceNodeAllocatable":{"pods":{}},"KubeReserved":null,"SystemReserved":{"cpu":"500m","ephemeral-storage":"1Gi","memory":"1Gi"},"HardEvictionThresholds":[{"Signal":"imagefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.15},"GracePeriod":0,"MinReclaim":null},{"Signal":"imagefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null},{"Signal":"memory.available","Operator":"LessThan","Value":{"Quantity":"100Mi","Percentage":0},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.1},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null}],"QOSReserved":{},"CPUManagerPolicy":"none","CPUManagerPolicyOptions":null,"TopologyManagerScope":"container","CPUManagerReconcilePeriod":10000000000,"MemoryManagerPolicy":"None","MemoryManagerReservedMemory":null,"PodPidsLimit":4096,"EnforceCPULimits":true,"CPUCFSQuotaPeriod":100000000,"TopologyManagerPolicy":"none","TopologyManagerPolicyOptions":null,"CgroupVersion":2}
Sep 17 17:34:11.987976 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:34:11.987966    2565 topology_manager.go:138] "Creating topology manager with none policy"
Sep 17 17:34:11.988065 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:34:11.987979    2565 container_manager_linux.go:309] "Creating device plugin manager"
Sep 17 17:34:11.988065 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:34:11.987986    2565 manager.go:141] "Creating Device Plugin manager" path="/var/lib/kubelet/device-plugins/kubelet.sock"
Sep 17 17:34:11.988065 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:34:11.987996    2565 server.go:74] "Creating device plugin registration server" version="v1beta1" socket="/var/lib/kubelet/device-plugins/kubelet.sock"
Sep 17 17:34:11.988440 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:34:11.988424    2565 reflector.go:439] "Caches populated" logger="kubernetes.io/kube-apiserver-client-kubelet" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162"
Sep 17 17:34:11.989436 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:34:11.989426    2565 container_manager_linux.go:318] "Creating Dynamic Resource Allocation (DRA) manager"
Sep 17 17:34:11.993349 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:34:11.993336    2565 state_mem.go:36] "Initialized new in-memory state store"
Sep 17 17:34:11.993457 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:34:11.993448    2565 server.go:1240] "Using root directory" path="/var/lib/kubelet"
Sep 17 17:34:11.993506 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:34:11.993474    2565 kubelet.go:486] "Attempting to sync node with API server"
Sep 17 17:34:11.993506 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:34:11.993484    2565 kubelet.go:387] "Adding static pod path" path="/etc/kubernetes/manifests"
Sep 17 17:34:11.993506 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:34:11.993494    2565 file.go:69] "Watching path" path="/etc/kubernetes/manifests"
Sep 17 17:34:11.993506 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:34:11.993502    2565 kubelet.go:398] "Adding apiserver pod source"
Sep 17 17:34:11.993654 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:34:11.993510    2565 apiserver.go:42] "Waiting for node sync before watching apiserver pods"
Sep 17 17:34:11.993827 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:34:11.993817    2565 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking"
Sep 17 17:34:11.993867 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:34:11.993833    2565 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking"
Sep 17 17:34:11.999532 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:34:11.995686    2565 kuberuntime_manager.go:301] "Container runtime initialized" containerRuntime="cri-o" version="1.34.13-6.rhaos4.21.git6253162.el9" apiVersion="v1"
Sep 17 17:34:12.001822 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:34:12.001805    2565 kubelet.go:960] "Not starting ClusterTrustBundle informer because we are in static kubelet mode or the ClusterTrustBundleProjection featuregate is disabled"
Sep 17 17:34:12.001898 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:34:12.001843    2565 kubelet.go:984] "Not starting PodCertificateRequest manager because we are in static kubelet mode or the PodCertificateProjection feature gate is disabled"
Sep 17 17:34:12.003069 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:34:12.003054    2565 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/portworx-volume"
Sep 17 17:34:12.003163 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:34:12.003076    2565 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/empty-dir"
Sep 17 17:34:12.003163 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:34:12.003086    2565 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/git-repo"
Sep 17 17:34:12.003163 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:34:12.003093    2565 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/host-path"
Sep 17 17:34:12.003163 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:34:12.003104    2565 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/nfs"
Sep 17 17:34:12.003163 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:34:12.003112    2565 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/secret"
Sep 17 17:34:12.003163 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:34:12.003131    2565 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/iscsi"
Sep 17 17:34:12.003163 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:34:12.003139    2565 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/downward-api"
Sep 17 17:34:12.003163 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:34:12.003149    2565 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/fc"
Sep 17 17:34:12.003163 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:34:12.003157    2565 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/configmap"
Sep 17 17:34:12.003459 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:34:12.003170    2565 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/projected"
Sep 17 17:34:12.003459 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:34:12.003178    2565 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/local-volume"
Sep 17 17:34:12.003459 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:34:12.003208    2565 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/csi"
Sep 17 17:34:12.003459 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:34:12.003217    2565 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/image"
Sep 17 17:34:12.006557 ip-10-0-137-20 kubenswrapper[2565]: E0917 17:34:12.006534    2565 reflector.go:205] "Failed to watch" err="failed to list *v1.Service: services is forbidden: User \"system:anonymous\" cannot list resource \"services\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Service"
Sep 17 17:34:12.006557 ip-10-0-137-20 kubenswrapper[2565]: E0917 17:34:12.006541    2565 reflector.go:205] "Failed to watch" err="failed to list *v1.Node: nodes \"ip-10-0-137-20.ec2.internal\" is forbidden: User \"system:anonymous\" cannot list resource \"nodes\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Node"
Sep 17 17:34:12.006696 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:34:12.006585    2565 csi_plugin.go:990] Failed to contact API server when waiting for CSINode publishing: csinodes.storage.k8s.io "ip-10-0-137-20.ec2.internal" is forbidden: User "system:anonymous" cannot get resource "csinodes" in API group "storage.k8s.io" at the cluster scope
Sep 17 17:34:12.006816 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:34:12.006803    2565 server.go:1268] "Started kubelet"
Sep 17 17:34:12.006877 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:34:12.006857    2565 server.go:180] "Starting to listen" address="0.0.0.0" port=10250
Sep 17 17:34:12.040522 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:34:12.040474    2565 ratelimit.go:56] "Setting rate limiting for endpoint" service="podresources" qps=100 burstTokens=10
Sep 17 17:34:12.040621 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:34:12.040553    2565 server_v1.go:49] "podresources" method="list" useActivePods=true
Sep 17 17:34:12.040791 ip-10-0-137-20 systemd[1]: Started Kubernetes Kubelet.
Sep 17 17:34:12.041666 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:34:12.041611    2565 server.go:310] "Adding debug handlers to kubelet server"
Sep 17 17:34:12.042168 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:34:12.042149    2565 server.go:249] "Starting to serve the podresources API" endpoint="unix:/var/lib/kubelet/pod-resources/kubelet.sock"
Sep 17 17:34:12.044636 ip-10-0-137-20 kubenswrapper[2565]: E0917 17:34:12.044609    2565 kubelet.go:1638] "Image garbage collection failed once. Stats initialization may not have completed yet" err="failed to get imageFs info: unable to find data in memory cache"
Sep 17 17:34:12.050295 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:34:12.050280    2565 fs_resource_analyzer.go:67] "Starting FS ResourceAnalyzer"
Sep 17 17:34:12.050363 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:34:12.050302    2565 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kubelet-serving"
Sep 17 17:34:12.050425 ip-10-0-137-20 kubenswrapper[2565]: E0917 17:34:12.050410    2565 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-137-20.ec2.internal\" not found"
Sep 17 17:34:12.050848 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:34:12.050832    2565 volume_manager.go:311] "The desired_state_of_world populator starts"
Sep 17 17:34:12.050848 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:34:12.050848    2565 volume_manager.go:313] "Starting Kubelet Volume Manager"
Sep 17 17:34:12.050968 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:34:12.050896    2565 desired_state_of_world_populator.go:146] "Desired state populator starts to run"
Sep 17 17:34:12.050968 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:34:12.050928    2565 reconstruct.go:97] "Volume reconstruction finished"
Sep 17 17:34:12.050968 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:34:12.050932    2565 reconciler.go:29] "Reconciler: start to sync state"
Sep 17 17:34:12.051227 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:34:12.051208    2565 factory.go:55] Registering systemd factory
Sep 17 17:34:12.051227 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:34:12.051227    2565 factory.go:223] Registration of the systemd container factory successfully
Sep 17 17:34:12.051667 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:34:12.051651    2565 factory.go:153] Registering CRI-O factory
Sep 17 17:34:12.051667 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:34:12.051669    2565 factory.go:223] Registration of the crio container factory successfully
Sep 17 17:34:12.051768 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:34:12.051717    2565 factory.go:221] Registration of the containerd container factory failed: unable to create containerd client: containerd: cannot unix dial containerd api service: dial unix /run/containerd/containerd.sock: connect: no such file or directory
Sep 17 17:34:12.051768 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:34:12.051739    2565 factory.go:103] Registering Raw factory
Sep 17 17:34:12.051768 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:34:12.051753    2565 manager.go:1196] Started watching for new ooms in manager
Sep 17 17:34:12.052284 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:34:12.052268    2565 manager.go:349] Starting recovery of all containers
Sep 17 17:34:12.058454 ip-10-0-137-20 kubenswrapper[2565]: E0917 17:34:12.058429    2565 controller.go:145] "Failed to ensure lease exists, will retry" err="leases.coordination.k8s.io \"ip-10-0-137-20.ec2.internal\" is forbidden: User \"system:anonymous\" cannot get resource \"leases\" in API group \"coordination.k8s.io\" in the namespace \"kube-node-lease\"" interval="200ms"
Sep 17 17:34:12.058897 ip-10-0-137-20 kubenswrapper[2565]: E0917 17:34:12.058871    2565 reflector.go:205] "Failed to watch" err="failed to list *v1.CSIDriver: csidrivers.storage.k8s.io is forbidden: User \"system:anonymous\" cannot list resource \"csidrivers\" in API group \"storage.k8s.io\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.CSIDriver"
Sep 17 17:34:12.059396 ip-10-0-137-20 kubenswrapper[2565]: E0917 17:34:12.058521    2565 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-137-20.ec2.internal.18d62c27f24c3384  default    0 0001-01-01 00:00:00 +0000 UTC <nil> <nil> map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-137-20.ec2.internal,UID:ip-10-0-137-20.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:Starting,Message:Starting kubelet.,Source:EventSource{Component:kubelet,Host:ip-10-0-137-20.ec2.internal,},FirstTimestamp:2026-09-17 17:34:12.006777732 +0000 UTC m=+1.884062138,LastTimestamp:2026-09-17 17:34:12.006777732 +0000 UTC m=+1.884062138,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-137-20.ec2.internal,}"
Sep 17 17:34:12.061867 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:34:12.061734    2565 manager.go:354] Recovery completed
Sep 17 17:34:12.064212 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:34:12.064196    2565 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-9gfqh"
Sep 17 17:34:12.066409 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:34:12.066396    2565 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Sep 17 17:34:12.068002 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:34:12.067987    2565 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-137-20.ec2.internal" event="NodeHasSufficientMemory"
Sep 17 17:34:12.068075 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:34:12.068013    2565 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-137-20.ec2.internal" event="NodeHasNoDiskPressure"
Sep 17 17:34:12.068075 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:34:12.068038    2565 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-137-20.ec2.internal" event="NodeHasSufficientPID"
Sep 17 17:34:12.068542 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:34:12.068528    2565 cpu_manager.go:222] "Starting CPU manager" policy="none"
Sep 17 17:34:12.068542 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:34:12.068542    2565 cpu_manager.go:223] "Reconciling" reconcilePeriod="10s"
Sep 17 17:34:12.068665 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:34:12.068558    2565 state_mem.go:36] "Initialized new in-memory state store"
Sep 17 17:34:12.068665 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:34:12.068602    2565 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Sep 17 17:34:12.069249 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:34:12.069234    2565 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-137-20.ec2.internal" event="NodeHasSufficientMemory"
Sep 17 17:34:12.069312 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:34:12.069258    2565 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-137-20.ec2.internal" event="NodeHasNoDiskPressure"
Sep 17 17:34:12.069312 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:34:12.069269    2565 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-137-20.ec2.internal" event="NodeHasSufficientPID"
Sep 17 17:34:12.070767 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:34:12.070751    2565 policy_none.go:49] "None policy: Start"
Sep 17 17:34:12.070767 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:34:12.070767    2565 memory_manager.go:187] "Starting memorymanager" policy="None"
Sep 17 17:34:12.070904 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:34:12.070777    2565 state_mem.go:36] "Initializing new in-memory state store" logger="Memory Manager state checkpoint"
Sep 17 17:34:12.071229 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:34:12.071216    2565 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-9gfqh"
Sep 17 17:34:12.071561 ip-10-0-137-20 kubenswrapper[2565]: E0917 17:34:12.071506    2565 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-137-20.ec2.internal.18d62c27f5f264ae  default    0 0001-01-01 00:00:00 +0000 UTC <nil> <nil> map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-137-20.ec2.internal,UID:ip-10-0-137-20.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:NodeHasSufficientMemory,Message:Node ip-10-0-137-20.ec2.internal status is now: NodeHasSufficientMemory,Source:EventSource{Component:kubelet,Host:ip-10-0-137-20.ec2.internal,},FirstTimestamp:2026-09-17 17:34:12.068000942 +0000 UTC m=+1.945285348,LastTimestamp:2026-09-17 17:34:12.068000942 +0000 UTC m=+1.945285348,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-137-20.ec2.internal,}"
Sep 17 17:34:12.072759 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:34:12.072745    2565 policy_none.go:47] "Start"
Sep 17 17:34:12.078003 ip-10-0-137-20 kubenswrapper[2565]: E0917 17:34:12.077943    2565 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-137-20.ec2.internal.18d62c27f5f2d81d  default    0 0001-01-01 00:00:00 +0000 UTC <nil> <nil> map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-137-20.ec2.internal,UID:ip-10-0-137-20.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:NodeHasNoDiskPressure,Message:Node ip-10-0-137-20.ec2.internal status is now: NodeHasNoDiskPressure,Source:EventSource{Component:kubelet,Host:ip-10-0-137-20.ec2.internal,},FirstTimestamp:2026-09-17 17:34:12.068030493 +0000 UTC m=+1.945314900,LastTimestamp:2026-09-17 17:34:12.068030493 +0000 UTC m=+1.945314900,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-137-20.ec2.internal,}"
Sep 17 17:34:12.113579 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:34:12.113563    2565 manager.go:337] "Starting Device Plugin manager"
Sep 17 17:34:12.120950 ip-10-0-137-20 kubenswrapper[2565]: E0917 17:34:12.113598    2565 manager.go:513] "Failed to read data from checkpoint" err="checkpoint is not found" checkpoint="kubelet_internal_checkpoint"
Sep 17 17:34:12.120950 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:34:12.113610    2565 server.go:87] "Starting device plugin registration server"
Sep 17 17:34:12.120950 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:34:12.113811    2565 eviction_manager.go:189] "Eviction manager: starting control loop"
Sep 17 17:34:12.120950 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:34:12.113828    2565 container_log_manager.go:146] "Initializing container log rotate workers" workers=1 monitorPeriod="10s"
Sep 17 17:34:12.120950 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:34:12.113903    2565 plugin_watcher.go:51] "Plugin Watcher Start" path="/var/lib/kubelet/plugins_registry"
Sep 17 17:34:12.120950 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:34:12.113968    2565 plugin_manager.go:116] "The desired_state_of_world populator (plugin watcher) starts"
Sep 17 17:34:12.120950 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:34:12.113975    2565 plugin_manager.go:118] "Starting Kubelet Plugin Manager"
Sep 17 17:34:12.120950 ip-10-0-137-20 kubenswrapper[2565]: E0917 17:34:12.114493    2565 eviction_manager.go:267] "eviction manager: failed to check if we have separate container filesystem. Ignoring." err="non-existent label \"crio-containers\""
Sep 17 17:34:12.120950 ip-10-0-137-20 kubenswrapper[2565]: E0917 17:34:12.114530    2565 eviction_manager.go:292] "Eviction manager: failed to get summary stats" err="failed to get node info: node \"ip-10-0-137-20.ec2.internal\" not found"
Sep 17 17:34:12.148830 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:34:12.148807    2565 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv4"
Sep 17 17:34:12.149973 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:34:12.149957    2565 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv6"
Sep 17 17:34:12.150063 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:34:12.149995    2565 status_manager.go:244] "Starting to sync pod status with apiserver"
Sep 17 17:34:12.150063 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:34:12.150014    2565 kubelet.go:2448] "Starting kubelet main sync loop"
Sep 17 17:34:12.150146 ip-10-0-137-20 kubenswrapper[2565]: E0917 17:34:12.150071    2565 kubelet.go:2472] "Skipping pod synchronization" err="PLEG is not healthy: pleg has yet to be successful"
Sep 17 17:34:12.152133 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:34:12.152113    2565 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160"
Sep 17 17:34:12.213979 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:34:12.213933    2565 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Sep 17 17:34:12.214812 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:34:12.214794    2565 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-137-20.ec2.internal" event="NodeHasSufficientMemory"
Sep 17 17:34:12.214878 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:34:12.214821    2565 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-137-20.ec2.internal" event="NodeHasNoDiskPressure"
Sep 17 17:34:12.214878 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:34:12.214832    2565 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-137-20.ec2.internal" event="NodeHasSufficientPID"
Sep 17 17:34:12.214878 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:34:12.214851    2565 kubelet_node_status.go:78] "Attempting to register node" node="ip-10-0-137-20.ec2.internal"
Sep 17 17:34:12.221249 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:34:12.221236    2565 kubelet_node_status.go:81] "Successfully registered node" node="ip-10-0-137-20.ec2.internal"
Sep 17 17:34:12.221295 ip-10-0-137-20 kubenswrapper[2565]: E0917 17:34:12.221254    2565 kubelet_node_status.go:536] "Error updating node status, will retry" err="error getting node \"ip-10-0-137-20.ec2.internal\": node \"ip-10-0-137-20.ec2.internal\" not found"
Sep 17 17:34:12.238225 ip-10-0-137-20 kubenswrapper[2565]: E0917 17:34:12.238209    2565 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-137-20.ec2.internal\" not found"
Sep 17 17:34:12.250317 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:34:12.250283    2565 kubelet.go:2535] "SyncLoop ADD" source="file" pods=["openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-137-20.ec2.internal","kube-system/kube-apiserver-proxy-ip-10-0-137-20.ec2.internal"]
Sep 17 17:34:12.250363 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:34:12.250353    2565 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Sep 17 17:34:12.251671 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:34:12.251658    2565 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-137-20.ec2.internal" event="NodeHasSufficientMemory"
Sep 17 17:34:12.251734 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:34:12.251683    2565 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-137-20.ec2.internal" event="NodeHasNoDiskPressure"
Sep 17 17:34:12.251734 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:34:12.251698    2565 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-137-20.ec2.internal" event="NodeHasSufficientPID"
Sep 17 17:34:12.254056 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:34:12.254045    2565 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Sep 17 17:34:12.254182 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:34:12.254156    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-137-20.ec2.internal"
Sep 17 17:34:12.254182 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:34:12.254183    2565 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Sep 17 17:34:12.254980 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:34:12.254963    2565 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-137-20.ec2.internal" event="NodeHasSufficientMemory"
Sep 17 17:34:12.255079 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:34:12.254994    2565 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-137-20.ec2.internal" event="NodeHasNoDiskPressure"
Sep 17 17:34:12.255079 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:34:12.255009    2565 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-137-20.ec2.internal" event="NodeHasSufficientPID"
Sep 17 17:34:12.255653 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:34:12.255635    2565 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-137-20.ec2.internal" event="NodeHasSufficientMemory"
Sep 17 17:34:12.255739 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:34:12.255671    2565 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-137-20.ec2.internal" event="NodeHasNoDiskPressure"
Sep 17 17:34:12.255739 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:34:12.255687    2565 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-137-20.ec2.internal" event="NodeHasSufficientPID"
Sep 17 17:34:12.257405 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:34:12.257388    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-137-20.ec2.internal"
Sep 17 17:34:12.257479 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:34:12.257416    2565 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Sep 17 17:34:12.260128 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:34:12.260113    2565 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-137-20.ec2.internal" event="NodeHasSufficientMemory"
Sep 17 17:34:12.260217 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:34:12.260134    2565 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-137-20.ec2.internal" event="NodeHasNoDiskPressure"
Sep 17 17:34:12.260217 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:34:12.260142    2565 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-137-20.ec2.internal" event="NodeHasSufficientPID"
Sep 17 17:34:12.270542 ip-10-0-137-20 kubenswrapper[2565]: E0917 17:34:12.270526    2565 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-137-20.ec2.internal\" not found" node="ip-10-0-137-20.ec2.internal"
Sep 17 17:34:12.275478 ip-10-0-137-20 kubenswrapper[2565]: E0917 17:34:12.275465    2565 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-137-20.ec2.internal\" not found" node="ip-10-0-137-20.ec2.internal"
Sep 17 17:34:12.338696 ip-10-0-137-20 kubenswrapper[2565]: E0917 17:34:12.338683    2565 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-137-20.ec2.internal\" not found"
Sep 17 17:34:12.352472 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:34:12.352458    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/cda455bc0aed3ae7411ab9a4c6df01be-config\") pod \"kube-apiserver-proxy-ip-10-0-137-20.ec2.internal\" (UID: \"cda455bc0aed3ae7411ab9a4c6df01be\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-137-20.ec2.internal"
Sep 17 17:34:12.352532 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:34:12.352480    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/aae62a2aa7c494e1d00d19cbc1cec29c-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-137-20.ec2.internal\" (UID: \"aae62a2aa7c494e1d00d19cbc1cec29c\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-137-20.ec2.internal"
Sep 17 17:34:12.352532 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:34:12.352496    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/aae62a2aa7c494e1d00d19cbc1cec29c-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-137-20.ec2.internal\" (UID: \"aae62a2aa7c494e1d00d19cbc1cec29c\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-137-20.ec2.internal"
Sep 17 17:34:12.439107 ip-10-0-137-20 kubenswrapper[2565]: E0917 17:34:12.439091    2565 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-137-20.ec2.internal\" not found"
Sep 17 17:34:12.453452 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:34:12.453437    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/aae62a2aa7c494e1d00d19cbc1cec29c-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-137-20.ec2.internal\" (UID: \"aae62a2aa7c494e1d00d19cbc1cec29c\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-137-20.ec2.internal"
Sep 17 17:34:12.453505 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:34:12.453392    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/aae62a2aa7c494e1d00d19cbc1cec29c-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-137-20.ec2.internal\" (UID: \"aae62a2aa7c494e1d00d19cbc1cec29c\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-137-20.ec2.internal"
Sep 17 17:34:12.453505 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:34:12.453491    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/cda455bc0aed3ae7411ab9a4c6df01be-config\") pod \"kube-apiserver-proxy-ip-10-0-137-20.ec2.internal\" (UID: \"cda455bc0aed3ae7411ab9a4c6df01be\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-137-20.ec2.internal"
Sep 17 17:34:12.453568 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:34:12.453508    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/aae62a2aa7c494e1d00d19cbc1cec29c-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-137-20.ec2.internal\" (UID: \"aae62a2aa7c494e1d00d19cbc1cec29c\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-137-20.ec2.internal"
Sep 17 17:34:12.453568 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:34:12.453541    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/aae62a2aa7c494e1d00d19cbc1cec29c-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-137-20.ec2.internal\" (UID: \"aae62a2aa7c494e1d00d19cbc1cec29c\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-137-20.ec2.internal"
Sep 17 17:34:12.453626 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:34:12.453578    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/host-path/cda455bc0aed3ae7411ab9a4c6df01be-config\") pod \"kube-apiserver-proxy-ip-10-0-137-20.ec2.internal\" (UID: \"cda455bc0aed3ae7411ab9a4c6df01be\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-137-20.ec2.internal"
Sep 17 17:34:12.539882 ip-10-0-137-20 kubenswrapper[2565]: E0917 17:34:12.539822    2565 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-137-20.ec2.internal\" not found"
Sep 17 17:34:12.574281 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:34:12.574267    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-137-20.ec2.internal"
Sep 17 17:34:12.580011 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:34:12.579754    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-137-20.ec2.internal"
Sep 17 17:34:12.640673 ip-10-0-137-20 kubenswrapper[2565]: E0917 17:34:12.640640    2565 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-137-20.ec2.internal\" not found"
Sep 17 17:34:12.741192 ip-10-0-137-20 kubenswrapper[2565]: E0917 17:34:12.741167    2565 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-137-20.ec2.internal\" not found"
Sep 17 17:34:12.841655 ip-10-0-137-20 kubenswrapper[2565]: E0917 17:34:12.841601    2565 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-137-20.ec2.internal\" not found"
Sep 17 17:34:12.942164 ip-10-0-137-20 kubenswrapper[2565]: E0917 17:34:12.942135    2565 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-137-20.ec2.internal\" not found"
Sep 17 17:34:12.957516 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:34:12.957497    2565 transport.go:147] "Certificate rotation detected, shutting down client connections to start using new credentials"
Sep 17 17:34:12.957626 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:34:12.957609    2565 reflector.go:571] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.RuntimeClass" err="very short watch: k8s.io/client-go/informers/factory.go:160: Unexpected watch close - watch lasted less than a second and no items received"
Sep 17 17:34:13.043069 ip-10-0-137-20 kubenswrapper[2565]: E0917 17:34:13.043053    2565 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-137-20.ec2.internal\" not found"
Sep 17 17:34:13.076547 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:34:13.076498    2565 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kube-apiserver-client-kubelet" expiration="2028-09-16 17:29:12 +0000 UTC" deadline="2028-03-14 12:20:47.406427309 +0000 UTC"
Sep 17 17:34:13.076547 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:34:13.076546    2565 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kube-apiserver-client-kubelet" sleep="13050h46m34.329884885s"
Sep 17 17:34:13.143831 ip-10-0-137-20 kubenswrapper[2565]: E0917 17:34:13.143779    2565 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-137-20.ec2.internal\" not found"
Sep 17 17:34:13.157570 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:34:13.157552    2565 reflector.go:439] "Caches populated" type="*v1.Service" reflector="k8s.io/client-go/informers/factory.go:160"
Sep 17 17:34:13.202865 ip-10-0-137-20 kubenswrapper[2565]: W0917 17:34:13.202842    2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podaae62a2aa7c494e1d00d19cbc1cec29c.slice/crio-4ae390b848b1c2d5b45a5df2906c7417a5590db8ee27dec841cac8761feb9f40 WatchSource:0}: Error finding container 4ae390b848b1c2d5b45a5df2906c7417a5590db8ee27dec841cac8761feb9f40: Status 404 returned error can't find the container with id 4ae390b848b1c2d5b45a5df2906c7417a5590db8ee27dec841cac8761feb9f40
Sep 17 17:34:13.203833 ip-10-0-137-20 kubenswrapper[2565]: W0917 17:34:13.203814    2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podcda455bc0aed3ae7411ab9a4c6df01be.slice/crio-264f94d70f70fad78cc9fb1ce08abb03dd3312ff99b5b1bbe4d7ad3e3b5f5baa WatchSource:0}: Error finding container 264f94d70f70fad78cc9fb1ce08abb03dd3312ff99b5b1bbe4d7ad3e3b5f5baa: Status 404 returned error can't find the container with id 264f94d70f70fad78cc9fb1ce08abb03dd3312ff99b5b1bbe4d7ad3e3b5f5baa
Sep 17 17:34:13.207252 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:34:13.207231    2565 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Sep 17 17:34:13.244057 ip-10-0-137-20 kubenswrapper[2565]: E0917 17:34:13.244036    2565 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-137-20.ec2.internal\" not found"
Sep 17 17:34:13.267465 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:34:13.267437    2565 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Sep 17 17:34:13.268468 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:34:13.268453    2565 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-137-20.ec2.internal" event="NodeHasSufficientMemory"
Sep 17 17:34:13.268536 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:34:13.268480    2565 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-137-20.ec2.internal" event="NodeHasNoDiskPressure"
Sep 17 17:34:13.268536 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:34:13.268492    2565 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-137-20.ec2.internal" event="NodeHasSufficientPID"
Sep 17 17:34:13.344549 ip-10-0-137-20 kubenswrapper[2565]: E0917 17:34:13.344528    2565 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-137-20.ec2.internal\" not found"
Sep 17 17:34:13.438573 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:34:13.438528    2565 reflector.go:439] "Caches populated" type="*v1.Node" reflector="k8s.io/client-go/informers/factory.go:160"
Sep 17 17:34:13.449494 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:34:13.449477    2565 reflector.go:439] "Caches populated" type="*v1.CSIDriver" reflector="k8s.io/client-go/informers/factory.go:160"
Sep 17 17:34:13.451526 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:34:13.451510    2565 kubelet.go:3248] "Creating a mirror pod for static pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-137-20.ec2.internal"
Sep 17 17:34:13.463791 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:34:13.463773    2565 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]"
Sep 17 17:34:13.464677 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:34:13.464665    2565 kubelet.go:3248] "Creating a mirror pod for static pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-137-20.ec2.internal"
Sep 17 17:34:13.474220 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:34:13.474206    2565 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]"
Sep 17 17:34:13.994755 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:34:13.994723    2565 apiserver.go:52] "Watching apiserver"
Sep 17 17:34:14.006701 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:34:14.006515    2565 reflector.go:439] "Caches populated" type="*v1.Pod" reflector="pkg/kubelet/config/apiserver.go:66"
Sep 17 17:34:14.006954 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:34:14.006926    2565 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kube-system/kube-apiserver-proxy-ip-10-0-137-20.ec2.internal","openshift-cluster-node-tuning-operator/tuned-mf8sz","openshift-image-registry/node-ca-5blw5","openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-137-20.ec2.internal","openshift-multus/multus-additional-cni-plugins-msv4b","openshift-multus/network-metrics-daemon-rmlqx","kube-system/global-pull-secret-syncer-74xd8","kube-system/konnectivity-agent-p8jd8","openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-fsbkf","openshift-dns/node-resolver-vs64q","openshift-multus/multus-5l5k7","openshift-network-diagnostics/network-check-target-jsc2p","openshift-network-operator/iptables-alerter-g8kpq","openshift-ovn-kubernetes/ovnkube-node-d9927"]
Sep 17 17:34:14.009508 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:34:14.009471    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-vs64q"
Sep 17 17:34:14.011557 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:34:14.011536    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-g8kpq"
Sep 17 17:34:14.012726 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:34:14.012707    2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"openshift-service-ca.crt\""
Sep 17 17:34:14.012937 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:34:14.012922    2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"node-resolver-dockercfg-szphp\""
Sep 17 17:34:14.013046 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:34:14.012979    2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"kube-root-ca.crt\""
Sep 17 17:34:14.013752 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:34:14.013735    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-5blw5"
Sep 17 17:34:14.014361 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:34:14.014183    2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"openshift-service-ca.crt\""
Sep 17 17:34:14.014496 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:34:14.014476    2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-script\""
Sep 17 17:34:14.014588 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:34:14.014570    2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"kube-root-ca.crt\""
Sep 17 17:34:14.014694 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:34:14.014679    2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-dockercfg-gnnqt\""
Sep 17 17:34:14.016089 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:34:14.016072    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-d9927"
Sep 17 17:34:14.016969 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:34:14.016340    2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"image-registry-certificates\""
Sep 17 17:34:14.016969 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:34:14.016664    2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"kube-root-ca.crt\""
Sep 17 17:34:14.016969 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:34:14.016850    2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"openshift-service-ca.crt\""
Sep 17 17:34:14.017151 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:34:14.017100    2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"node-ca-dockercfg-zt5hz\""
Sep 17 17:34:14.019135 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:34:14.018420    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-msv4b"
Sep 17 17:34:14.020670 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:34:14.020139    2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-kubernetes-node-dockercfg-rvvsn\""
Sep 17 17:34:14.020670 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:34:14.020150    2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"openshift-service-ca.crt\""
Sep 17 17:34:14.020670 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:34:14.020142    2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-node-metrics-cert\""
Sep 17 17:34:14.020670 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:34:14.020228    2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-script-lib\""
Sep 17 17:34:14.020670 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:34:14.020277    2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"kube-root-ca.crt\""
Sep 17 17:34:14.020670 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:34:14.020365    2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-config\""
Sep 17 17:34:14.020670 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:34:14.020449    2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"env-overrides\""
Sep 17 17:34:14.020909 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:34:14.020748    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-rmlqx"
Sep 17 17:34:14.020909 ip-10-0-137-20 kubenswrapper[2565]: E0917 17:34:14.020857    2565 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-rmlqx" podUID="e39c5e1e-113d-42f2-9a00-8a6d14cb6d67"
Sep 17 17:34:14.021507 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:34:14.021057    2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"default-cni-sysctl-allowlist\""
Sep 17 17:34:14.021507 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:34:14.021130    2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"kube-root-ca.crt\""
Sep 17 17:34:14.021507 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:34:14.021241    2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"multus-ancillary-tools-dockercfg-6c2fl\""
Sep 17 17:34:14.021507 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:34:14.021375    2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"whereabouts-flatfile-config\""
Sep 17 17:34:14.021507 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:34:14.021418    2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"openshift-service-ca.crt\""
Sep 17 17:34:14.021507 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:34:14.021431    2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"cni-copy-resources\""
Sep 17 17:34:14.023079 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:34:14.023062    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-fsbkf"
Sep 17 17:34:14.025646 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:34:14.025530    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-74xd8"
Sep 17 17:34:14.025646 ip-10-0-137-20 kubenswrapper[2565]: E0917 17:34:14.025600    2565 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-74xd8" podUID="3d1c780d-6713-495e-91fb-54a74668e6cc"
Sep 17 17:34:14.025787 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:34:14.025683    2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-sa-dockercfg-z7kbm\""
Sep 17 17:34:14.025787 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:34:14.025705    2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"openshift-service-ca.crt\""
Sep 17 17:34:14.025884 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:34:14.025866    2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"kube-root-ca.crt\""
Sep 17 17:34:14.026039 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:34:14.026007    2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-metrics-serving-cert\""
Sep 17 17:34:14.027964 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:34:14.027943    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-mf8sz"
Sep 17 17:34:14.030248 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:34:14.030227    2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"openshift-service-ca.crt\""
Sep 17 17:34:14.030607 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:34:14.030587    2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"kube-root-ca.crt\""
Sep 17 17:34:14.030778 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:34:14.030764    2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"tuned-dockercfg-zq7ph\""
Sep 17 17:34:14.030873 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:34:14.030858    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-p8jd8"
Sep 17 17:34:14.033342 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:34:14.033244    2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kube-system\"/\"konnectivity-ca-bundle\""
Sep 17 17:34:14.033585 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:34:14.033566    2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"konnectivity-agent\""
Sep 17 17:34:14.033585 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:34:14.033579    2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"default-dockercfg-9k88f\""
Sep 17 17:34:14.034161 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:34:14.034089    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-5l5k7"
Sep 17 17:34:14.036618 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:34:14.036481    2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"default-dockercfg-mcvcq\""
Sep 17 17:34:14.037534 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:34:14.037158    2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"multus-daemon-config\""
Sep 17 17:34:14.038406 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:34:14.038170    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-jsc2p"
Sep 17 17:34:14.038406 ip-10-0-137-20 kubenswrapper[2565]: E0917 17:34:14.038278    2565 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-jsc2p" podUID="574f2e00-d361-423b-946b-e1e9ea3a5db5"
Sep 17 17:34:14.051274 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:34:14.051257    2565 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kubelet-serving"
Sep 17 17:34:14.052214 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:34:14.052194    2565 desired_state_of_world_populator.go:154] "Finished populating initial desired state of world"
Sep 17 17:34:14.063225 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:34:14.063205    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/27b114f9-fcc4-463b-897f-9e4561ad5540-host\") pod \"node-ca-5blw5\" (UID: \"27b114f9-fcc4-463b-897f-9e4561ad5540\") " pod="openshift-image-registry/node-ca-5blw5"
Sep 17 17:34:14.063354 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:34:14.063239    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-nd6qt\" (UniqueName: \"kubernetes.io/projected/30837de7-1447-4de2-ac62-c809813b3338-kube-api-access-nd6qt\") pod \"ovnkube-node-d9927\" (UID: \"30837de7-1447-4de2-ac62-c809813b3338\") " pod="openshift-ovn-kubernetes/ovnkube-node-d9927"
Sep 17 17:34:14.063354 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:34:14.063290    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/1958175e-39bb-4dc8-a5fb-5cff5b7e85f5-registration-dir\") pod \"aws-ebs-csi-driver-node-fsbkf\" (UID: \"1958175e-39bb-4dc8-a5fb-5cff5b7e85f5\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-fsbkf"
Sep 17 17:34:14.063354 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:34:14.063316    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/d4013629-eb04-40d7-bdb1-11e1cafa9394-etc-systemd\") pod \"tuned-mf8sz\" (UID: \"d4013629-eb04-40d7-bdb1-11e1cafa9394\") " pod="openshift-cluster-node-tuning-operator/tuned-mf8sz"
Sep 17 17:34:14.063557 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:34:14.063354    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-s8xpr\" (UniqueName: \"kubernetes.io/projected/574f2e00-d361-423b-946b-e1e9ea3a5db5-kube-api-access-s8xpr\") pod \"network-check-target-jsc2p\" (UID: \"574f2e00-d361-423b-946b-e1e9ea3a5db5\") " pod="openshift-network-diagnostics/network-check-target-jsc2p"
Sep 17 17:34:14.063557 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:34:14.063382    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-p7fds\" (UniqueName: \"kubernetes.io/projected/27b114f9-fcc4-463b-897f-9e4561ad5540-kube-api-access-p7fds\") pod \"node-ca-5blw5\" (UID: \"27b114f9-fcc4-463b-897f-9e4561ad5540\") " pod="openshift-image-registry/node-ca-5blw5"
Sep 17 17:34:14.063557 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:34:14.063427    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/048bee56-5e42-45f0-98a8-213ff62de69e-system-cni-dir\") pod \"multus-additional-cni-plugins-msv4b\" (UID: \"048bee56-5e42-45f0-98a8-213ff62de69e\") " pod="openshift-multus/multus-additional-cni-plugins-msv4b"
Sep 17 17:34:14.063557 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:34:14.063450    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/2feeabe5-1136-4f30-9a14-491423e838a3-cnibin\") pod \"multus-5l5k7\" (UID: \"2feeabe5-1136-4f30-9a14-491423e838a3\") " pod="openshift-multus/multus-5l5k7"
Sep 17 17:34:14.063557 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:34:14.063481    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/2feeabe5-1136-4f30-9a14-491423e838a3-hostroot\") pod \"multus-5l5k7\" (UID: \"2feeabe5-1136-4f30-9a14-491423e838a3\") " pod="openshift-multus/multus-5l5k7"
Sep 17 17:34:14.063557 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:34:14.063523    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/048bee56-5e42-45f0-98a8-213ff62de69e-tuning-conf-dir\") pod \"multus-additional-cni-plugins-msv4b\" (UID: \"048bee56-5e42-45f0-98a8-213ff62de69e\") " pod="openshift-multus/multus-additional-cni-plugins-msv4b"
Sep 17 17:34:14.063557 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:34:14.063554    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/30837de7-1447-4de2-ac62-c809813b3338-log-socket\") pod \"ovnkube-node-d9927\" (UID: \"30837de7-1447-4de2-ac62-c809813b3338\") " pod="openshift-ovn-kubernetes/ovnkube-node-d9927"
Sep 17 17:34:14.063966 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:34:14.063589    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/d4013629-eb04-40d7-bdb1-11e1cafa9394-run\") pod \"tuned-mf8sz\" (UID: \"d4013629-eb04-40d7-bdb1-11e1cafa9394\") " pod="openshift-cluster-node-tuning-operator/tuned-mf8sz"
Sep 17 17:34:14.063966 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:34:14.063614    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/3d1c780d-6713-495e-91fb-54a74668e6cc-original-pull-secret\") pod \"global-pull-secret-syncer-74xd8\" (UID: \"3d1c780d-6713-495e-91fb-54a74668e6cc\") " pod="kube-system/global-pull-secret-syncer-74xd8"
Sep 17 17:34:14.063966 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:34:14.063653    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/d4013629-eb04-40d7-bdb1-11e1cafa9394-lib-modules\") pod \"tuned-mf8sz\" (UID: \"d4013629-eb04-40d7-bdb1-11e1cafa9394\") " pod="openshift-cluster-node-tuning-operator/tuned-mf8sz"
Sep 17 17:34:14.063966 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:34:14.063677    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/af1cc6fe-9695-4e2e-9244-6505022f6c50-iptables-alerter-script\") pod \"iptables-alerter-g8kpq\" (UID: \"af1cc6fe-9695-4e2e-9244-6505022f6c50\") " pod="openshift-network-operator/iptables-alerter-g8kpq"
Sep 17 17:34:14.063966 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:34:14.063691    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/30837de7-1447-4de2-ac62-c809813b3338-host-kubelet\") pod \"ovnkube-node-d9927\" (UID: \"30837de7-1447-4de2-ac62-c809813b3338\") " pod="openshift-ovn-kubernetes/ovnkube-node-d9927"
Sep 17 17:34:14.063966 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:34:14.063712    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/30837de7-1447-4de2-ac62-c809813b3338-run-systemd\") pod \"ovnkube-node-d9927\" (UID: \"30837de7-1447-4de2-ac62-c809813b3338\") " pod="openshift-ovn-kubernetes/ovnkube-node-d9927"
Sep 17 17:34:14.063966 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:34:14.063734    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/30837de7-1447-4de2-ac62-c809813b3338-host-cni-bin\") pod \"ovnkube-node-d9927\" (UID: \"30837de7-1447-4de2-ac62-c809813b3338\") " pod="openshift-ovn-kubernetes/ovnkube-node-d9927"
Sep 17 17:34:14.063966 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:34:14.063764    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/2feeabe5-1136-4f30-9a14-491423e838a3-system-cni-dir\") pod \"multus-5l5k7\" (UID: \"2feeabe5-1136-4f30-9a14-491423e838a3\") " pod="openshift-multus/multus-5l5k7"
Sep 17 17:34:14.063966 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:34:14.063801    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/2feeabe5-1136-4f30-9a14-491423e838a3-multus-cni-dir\") pod \"multus-5l5k7\" (UID: \"2feeabe5-1136-4f30-9a14-491423e838a3\") " pod="openshift-multus/multus-5l5k7"
Sep 17 17:34:14.063966 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:34:14.063828    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/e39c5e1e-113d-42f2-9a00-8a6d14cb6d67-metrics-certs\") pod \"network-metrics-daemon-rmlqx\" (UID: \"e39c5e1e-113d-42f2-9a00-8a6d14cb6d67\") " pod="openshift-multus/network-metrics-daemon-rmlqx"
Sep 17 17:34:14.063966 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:34:14.063852    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/30837de7-1447-4de2-ac62-c809813b3338-var-lib-openvswitch\") pod \"ovnkube-node-d9927\" (UID: \"30837de7-1447-4de2-ac62-c809813b3338\") " pod="openshift-ovn-kubernetes/ovnkube-node-d9927"
Sep 17 17:34:14.063966 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:34:14.063876    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/30837de7-1447-4de2-ac62-c809813b3338-node-log\") pod \"ovnkube-node-d9927\" (UID: \"30837de7-1447-4de2-ac62-c809813b3338\") " pod="openshift-ovn-kubernetes/ovnkube-node-d9927"
Sep 17 17:34:14.063966 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:34:14.063902    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/30837de7-1447-4de2-ac62-c809813b3338-ovnkube-config\") pod \"ovnkube-node-d9927\" (UID: \"30837de7-1447-4de2-ac62-c809813b3338\") " pod="openshift-ovn-kubernetes/ovnkube-node-d9927"
Sep 17 17:34:14.063966 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:34:14.063932    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/2feeabe5-1136-4f30-9a14-491423e838a3-host-run-netns\") pod \"multus-5l5k7\" (UID: \"2feeabe5-1136-4f30-9a14-491423e838a3\") " pod="openshift-multus/multus-5l5k7"
Sep 17 17:34:14.063966 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:34:14.063967    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/048bee56-5e42-45f0-98a8-213ff62de69e-cni-binary-copy\") pod \"multus-additional-cni-plugins-msv4b\" (UID: \"048bee56-5e42-45f0-98a8-213ff62de69e\") " pod="openshift-multus/multus-additional-cni-plugins-msv4b"
Sep 17 17:34:14.064784 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:34:14.063992    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/caa60d4e-b46b-4fff-8a66-3a5b4f1a1602-tmp-dir\") pod \"node-resolver-vs64q\" (UID: \"caa60d4e-b46b-4fff-8a66-3a5b4f1a1602\") " pod="openshift-dns/node-resolver-vs64q"
Sep 17 17:34:14.064784 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:34:14.064014    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/30837de7-1447-4de2-ac62-c809813b3338-ovnkube-script-lib\") pod \"ovnkube-node-d9927\" (UID: \"30837de7-1447-4de2-ac62-c809813b3338\") " pod="openshift-ovn-kubernetes/ovnkube-node-d9927"
Sep 17 17:34:14.064784 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:34:14.064072    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/d4013629-eb04-40d7-bdb1-11e1cafa9394-etc-sysconfig\") pod \"tuned-mf8sz\" (UID: \"d4013629-eb04-40d7-bdb1-11e1cafa9394\") " pod="openshift-cluster-node-tuning-operator/tuned-mf8sz"
Sep 17 17:34:14.064784 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:34:14.064112    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/d4013629-eb04-40d7-bdb1-11e1cafa9394-tmp\") pod \"tuned-mf8sz\" (UID: \"d4013629-eb04-40d7-bdb1-11e1cafa9394\") " pod="openshift-cluster-node-tuning-operator/tuned-mf8sz"
Sep 17 17:34:14.064784 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:34:14.064142    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/1958175e-39bb-4dc8-a5fb-5cff5b7e85f5-socket-dir\") pod \"aws-ebs-csi-driver-node-fsbkf\" (UID: \"1958175e-39bb-4dc8-a5fb-5cff5b7e85f5\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-fsbkf"
Sep 17 17:34:14.064784 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:34:14.064168    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/d4013629-eb04-40d7-bdb1-11e1cafa9394-etc-kubernetes\") pod \"tuned-mf8sz\" (UID: \"d4013629-eb04-40d7-bdb1-11e1cafa9394\") " pod="openshift-cluster-node-tuning-operator/tuned-mf8sz"
Sep 17 17:34:14.064784 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:34:14.064190    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/d4013629-eb04-40d7-bdb1-11e1cafa9394-etc-sysctl-conf\") pod \"tuned-mf8sz\" (UID: \"d4013629-eb04-40d7-bdb1-11e1cafa9394\") " pod="openshift-cluster-node-tuning-operator/tuned-mf8sz"
Sep 17 17:34:14.064784 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:34:14.064212    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/2feeabe5-1136-4f30-9a14-491423e838a3-multus-conf-dir\") pod \"multus-5l5k7\" (UID: \"2feeabe5-1136-4f30-9a14-491423e838a3\") " pod="openshift-multus/multus-5l5k7"
Sep 17 17:34:14.064784 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:34:14.064235    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xkgxr\" (UniqueName: \"kubernetes.io/projected/2feeabe5-1136-4f30-9a14-491423e838a3-kube-api-access-xkgxr\") pod \"multus-5l5k7\" (UID: \"2feeabe5-1136-4f30-9a14-491423e838a3\") " pod="openshift-multus/multus-5l5k7"
Sep 17 17:34:14.064784 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:34:14.064256    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-98nk4\" (UniqueName: \"kubernetes.io/projected/caa60d4e-b46b-4fff-8a66-3a5b4f1a1602-kube-api-access-98nk4\") pod \"node-resolver-vs64q\" (UID: \"caa60d4e-b46b-4fff-8a66-3a5b4f1a1602\") " pod="openshift-dns/node-resolver-vs64q"
Sep 17 17:34:14.064784 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:34:14.064279    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vx4pk\" (UniqueName: \"kubernetes.io/projected/e39c5e1e-113d-42f2-9a00-8a6d14cb6d67-kube-api-access-vx4pk\") pod \"network-metrics-daemon-rmlqx\" (UID: \"e39c5e1e-113d-42f2-9a00-8a6d14cb6d67\") " pod="openshift-multus/network-metrics-daemon-rmlqx"
Sep 17 17:34:14.064784 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:34:14.064300    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/30837de7-1447-4de2-ac62-c809813b3338-host-run-ovn-kubernetes\") pod \"ovnkube-node-d9927\" (UID: \"30837de7-1447-4de2-ac62-c809813b3338\") " pod="openshift-ovn-kubernetes/ovnkube-node-d9927"
Sep 17 17:34:14.064784 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:34:14.064314    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/1958175e-39bb-4dc8-a5fb-5cff5b7e85f5-kubelet-dir\") pod \"aws-ebs-csi-driver-node-fsbkf\" (UID: \"1958175e-39bb-4dc8-a5fb-5cff5b7e85f5\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-fsbkf"
Sep 17 17:34:14.064784 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:34:14.064332    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/2feeabe5-1136-4f30-9a14-491423e838a3-multus-socket-dir-parent\") pod \"multus-5l5k7\" (UID: \"2feeabe5-1136-4f30-9a14-491423e838a3\") " pod="openshift-multus/multus-5l5k7"
Sep 17 17:34:14.064784 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:34:14.064355    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/d4013629-eb04-40d7-bdb1-11e1cafa9394-sys\") pod \"tuned-mf8sz\" (UID: \"d4013629-eb04-40d7-bdb1-11e1cafa9394\") " pod="openshift-cluster-node-tuning-operator/tuned-mf8sz"
Sep 17 17:34:14.064784 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:34:14.064390    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4rxt7\" (UniqueName: \"kubernetes.io/projected/d4013629-eb04-40d7-bdb1-11e1cafa9394-kube-api-access-4rxt7\") pod \"tuned-mf8sz\" (UID: \"d4013629-eb04-40d7-bdb1-11e1cafa9394\") " pod="openshift-cluster-node-tuning-operator/tuned-mf8sz"
Sep 17 17:34:14.065708 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:34:14.064431    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/2feeabe5-1136-4f30-9a14-491423e838a3-cni-binary-copy\") pod \"multus-5l5k7\" (UID: \"2feeabe5-1136-4f30-9a14-491423e838a3\") " pod="openshift-multus/multus-5l5k7"
Sep 17 17:34:14.065708 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:34:14.064456    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/2feeabe5-1136-4f30-9a14-491423e838a3-host-var-lib-cni-multus\") pod \"multus-5l5k7\" (UID: \"2feeabe5-1136-4f30-9a14-491423e838a3\") " pod="openshift-multus/multus-5l5k7"
Sep 17 17:34:14.065708 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:34:14.064514    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/27b114f9-fcc4-463b-897f-9e4561ad5540-serviceca\") pod \"node-ca-5blw5\" (UID: \"27b114f9-fcc4-463b-897f-9e4561ad5540\") " pod="openshift-image-registry/node-ca-5blw5"
Sep 17 17:34:14.065708 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:34:14.064541    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/30837de7-1447-4de2-ac62-c809813b3338-host-slash\") pod \"ovnkube-node-d9927\" (UID: \"30837de7-1447-4de2-ac62-c809813b3338\") " pod="openshift-ovn-kubernetes/ovnkube-node-d9927"
Sep 17 17:34:14.065708 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:34:14.064578    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/30837de7-1447-4de2-ac62-c809813b3338-run-openvswitch\") pod \"ovnkube-node-d9927\" (UID: \"30837de7-1447-4de2-ac62-c809813b3338\") " pod="openshift-ovn-kubernetes/ovnkube-node-d9927"
Sep 17 17:34:14.065708 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:34:14.064600    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/d4013629-eb04-40d7-bdb1-11e1cafa9394-var-lib-kubelet\") pod \"tuned-mf8sz\" (UID: \"d4013629-eb04-40d7-bdb1-11e1cafa9394\") " pod="openshift-cluster-node-tuning-operator/tuned-mf8sz"
Sep 17 17:34:14.065708 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:34:14.064623    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/048bee56-5e42-45f0-98a8-213ff62de69e-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-msv4b\" (UID: \"048bee56-5e42-45f0-98a8-213ff62de69e\") " pod="openshift-multus/multus-additional-cni-plugins-msv4b"
Sep 17 17:34:14.065708 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:34:14.064647    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/048bee56-5e42-45f0-98a8-213ff62de69e-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-msv4b\" (UID: \"048bee56-5e42-45f0-98a8-213ff62de69e\") " pod="openshift-multus/multus-additional-cni-plugins-msv4b"
Sep 17 17:34:14.065708 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:34:14.064680    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/2e847ec5-02d2-4210-a6ce-89d926b050f3-konnectivity-ca\") pod \"konnectivity-agent-p8jd8\" (UID: \"2e847ec5-02d2-4210-a6ce-89d926b050f3\") " pod="kube-system/konnectivity-agent-p8jd8"
Sep 17 17:34:14.065708 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:34:14.064704    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/30837de7-1447-4de2-ac62-c809813b3338-ovn-node-metrics-cert\") pod \"ovnkube-node-d9927\" (UID: \"30837de7-1447-4de2-ac62-c809813b3338\") " pod="openshift-ovn-kubernetes/ovnkube-node-d9927"
Sep 17 17:34:14.065708 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:34:14.064725    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/1958175e-39bb-4dc8-a5fb-5cff5b7e85f5-device-dir\") pod \"aws-ebs-csi-driver-node-fsbkf\" (UID: \"1958175e-39bb-4dc8-a5fb-5cff5b7e85f5\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-fsbkf"
Sep 17 17:34:14.065708 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:34:14.064748    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/1958175e-39bb-4dc8-a5fb-5cff5b7e85f5-etc-selinux\") pod \"aws-ebs-csi-driver-node-fsbkf\" (UID: \"1958175e-39bb-4dc8-a5fb-5cff5b7e85f5\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-fsbkf"
Sep 17 17:34:14.065708 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:34:14.064770    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/1958175e-39bb-4dc8-a5fb-5cff5b7e85f5-sys-fs\") pod \"aws-ebs-csi-driver-node-fsbkf\" (UID: \"1958175e-39bb-4dc8-a5fb-5cff5b7e85f5\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-fsbkf"
Sep 17 17:34:14.065708 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:34:14.064836    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/caa60d4e-b46b-4fff-8a66-3a5b4f1a1602-hosts-file\") pod \"node-resolver-vs64q\" (UID: \"caa60d4e-b46b-4fff-8a66-3a5b4f1a1602\") " pod="openshift-dns/node-resolver-vs64q"
Sep 17 17:34:14.065708 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:34:14.064861    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/30837de7-1447-4de2-ac62-c809813b3338-run-ovn\") pod \"ovnkube-node-d9927\" (UID: \"30837de7-1447-4de2-ac62-c809813b3338\") " pod="openshift-ovn-kubernetes/ovnkube-node-d9927"
Sep 17 17:34:14.065708 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:34:14.064888    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/30837de7-1447-4de2-ac62-c809813b3338-env-overrides\") pod \"ovnkube-node-d9927\" (UID: \"30837de7-1447-4de2-ac62-c809813b3338\") " pod="openshift-ovn-kubernetes/ovnkube-node-d9927"
Sep 17 17:34:14.066599 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:34:14.064945    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4vqpv\" (UniqueName: \"kubernetes.io/projected/1958175e-39bb-4dc8-a5fb-5cff5b7e85f5-kube-api-access-4vqpv\") pod \"aws-ebs-csi-driver-node-fsbkf\" (UID: \"1958175e-39bb-4dc8-a5fb-5cff5b7e85f5\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-fsbkf"
Sep 17 17:34:14.066599 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:34:14.064988    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/2feeabe5-1136-4f30-9a14-491423e838a3-host-run-multus-certs\") pod \"multus-5l5k7\" (UID: \"2feeabe5-1136-4f30-9a14-491423e838a3\") " pod="openshift-multus/multus-5l5k7"
Sep 17 17:34:14.066599 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:34:14.065014    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/30837de7-1447-4de2-ac62-c809813b3338-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-d9927\" (UID: \"30837de7-1447-4de2-ac62-c809813b3338\") " pod="openshift-ovn-kubernetes/ovnkube-node-d9927"
Sep 17 17:34:14.066599 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:34:14.065069    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/3d1c780d-6713-495e-91fb-54a74668e6cc-kubelet-config\") pod \"global-pull-secret-syncer-74xd8\" (UID: \"3d1c780d-6713-495e-91fb-54a74668e6cc\") " pod="kube-system/global-pull-secret-syncer-74xd8"
Sep 17 17:34:14.066599 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:34:14.065099    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/3d1c780d-6713-495e-91fb-54a74668e6cc-dbus\") pod \"global-pull-secret-syncer-74xd8\" (UID: \"3d1c780d-6713-495e-91fb-54a74668e6cc\") " pod="kube-system/global-pull-secret-syncer-74xd8"
Sep 17 17:34:14.066599 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:34:14.065134    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/d4013629-eb04-40d7-bdb1-11e1cafa9394-etc-tuned\") pod \"tuned-mf8sz\" (UID: \"d4013629-eb04-40d7-bdb1-11e1cafa9394\") " pod="openshift-cluster-node-tuning-operator/tuned-mf8sz"
Sep 17 17:34:14.066599 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:34:14.065167    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/af1cc6fe-9695-4e2e-9244-6505022f6c50-host-slash\") pod \"iptables-alerter-g8kpq\" (UID: \"af1cc6fe-9695-4e2e-9244-6505022f6c50\") " pod="openshift-network-operator/iptables-alerter-g8kpq"
Sep 17 17:34:14.066599 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:34:14.065183    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/2feeabe5-1136-4f30-9a14-491423e838a3-host-var-lib-cni-bin\") pod \"multus-5l5k7\" (UID: \"2feeabe5-1136-4f30-9a14-491423e838a3\") " pod="openshift-multus/multus-5l5k7"
Sep 17 17:34:14.066599 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:34:14.065202    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/2feeabe5-1136-4f30-9a14-491423e838a3-multus-daemon-config\") pod \"multus-5l5k7\" (UID: \"2feeabe5-1136-4f30-9a14-491423e838a3\") " pod="openshift-multus/multus-5l5k7"
Sep 17 17:34:14.066599 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:34:14.065227    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-877s8\" (UniqueName: \"kubernetes.io/projected/af1cc6fe-9695-4e2e-9244-6505022f6c50-kube-api-access-877s8\") pod \"iptables-alerter-g8kpq\" (UID: \"af1cc6fe-9695-4e2e-9244-6505022f6c50\") " pod="openshift-network-operator/iptables-alerter-g8kpq"
Sep 17 17:34:14.066599 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:34:14.065251    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/048bee56-5e42-45f0-98a8-213ff62de69e-cnibin\") pod \"multus-additional-cni-plugins-msv4b\" (UID: \"048bee56-5e42-45f0-98a8-213ff62de69e\") " pod="openshift-multus/multus-additional-cni-plugins-msv4b"
Sep 17 17:34:14.066599 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:34:14.065274    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-n9xm8\" (UniqueName: \"kubernetes.io/projected/048bee56-5e42-45f0-98a8-213ff62de69e-kube-api-access-n9xm8\") pod \"multus-additional-cni-plugins-msv4b\" (UID: \"048bee56-5e42-45f0-98a8-213ff62de69e\") " pod="openshift-multus/multus-additional-cni-plugins-msv4b"
Sep 17 17:34:14.066599 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:34:14.065298    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/30837de7-1447-4de2-ac62-c809813b3338-systemd-units\") pod \"ovnkube-node-d9927\" (UID: \"30837de7-1447-4de2-ac62-c809813b3338\") " pod="openshift-ovn-kubernetes/ovnkube-node-d9927"
Sep 17 17:34:14.066599 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:34:14.065320    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/30837de7-1447-4de2-ac62-c809813b3338-etc-openvswitch\") pod \"ovnkube-node-d9927\" (UID: \"30837de7-1447-4de2-ac62-c809813b3338\") " pod="openshift-ovn-kubernetes/ovnkube-node-d9927"
Sep 17 17:34:14.066599 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:34:14.065341    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/d4013629-eb04-40d7-bdb1-11e1cafa9394-etc-modprobe-d\") pod \"tuned-mf8sz\" (UID: \"d4013629-eb04-40d7-bdb1-11e1cafa9394\") " pod="openshift-cluster-node-tuning-operator/tuned-mf8sz"
Sep 17 17:34:14.066599 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:34:14.065361    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/d4013629-eb04-40d7-bdb1-11e1cafa9394-host\") pod \"tuned-mf8sz\" (UID: \"d4013629-eb04-40d7-bdb1-11e1cafa9394\") " pod="openshift-cluster-node-tuning-operator/tuned-mf8sz"
Sep 17 17:34:14.067607 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:34:14.065382    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/2feeabe5-1136-4f30-9a14-491423e838a3-host-run-k8s-cni-cncf-io\") pod \"multus-5l5k7\" (UID: \"2feeabe5-1136-4f30-9a14-491423e838a3\") " pod="openshift-multus/multus-5l5k7"
Sep 17 17:34:14.067607 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:34:14.065405    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/2feeabe5-1136-4f30-9a14-491423e838a3-host-var-lib-kubelet\") pod \"multus-5l5k7\" (UID: \"2feeabe5-1136-4f30-9a14-491423e838a3\") " pod="openshift-multus/multus-5l5k7"
Sep 17 17:34:14.067607 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:34:14.065434    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/2feeabe5-1136-4f30-9a14-491423e838a3-etc-kubernetes\") pod \"multus-5l5k7\" (UID: \"2feeabe5-1136-4f30-9a14-491423e838a3\") " pod="openshift-multus/multus-5l5k7"
Sep 17 17:34:14.067607 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:34:14.065455    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/30837de7-1447-4de2-ac62-c809813b3338-host-run-netns\") pod \"ovnkube-node-d9927\" (UID: \"30837de7-1447-4de2-ac62-c809813b3338\") " pod="openshift-ovn-kubernetes/ovnkube-node-d9927"
Sep 17 17:34:14.067607 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:34:14.065476    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/30837de7-1447-4de2-ac62-c809813b3338-host-cni-netd\") pod \"ovnkube-node-d9927\" (UID: \"30837de7-1447-4de2-ac62-c809813b3338\") " pod="openshift-ovn-kubernetes/ovnkube-node-d9927"
Sep 17 17:34:14.067607 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:34:14.065522    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/2e847ec5-02d2-4210-a6ce-89d926b050f3-agent-certs\") pod \"konnectivity-agent-p8jd8\" (UID: \"2e847ec5-02d2-4210-a6ce-89d926b050f3\") " pod="kube-system/konnectivity-agent-p8jd8"
Sep 17 17:34:14.067607 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:34:14.065556    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/d4013629-eb04-40d7-bdb1-11e1cafa9394-etc-sysctl-d\") pod \"tuned-mf8sz\" (UID: \"d4013629-eb04-40d7-bdb1-11e1cafa9394\") " pod="openshift-cluster-node-tuning-operator/tuned-mf8sz"
Sep 17 17:34:14.067607 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:34:14.066974    2565 reflector.go:439] "Caches populated" logger="kubernetes.io/kubelet-serving" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162"
Sep 17 17:34:14.087442 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:34:14.087413    2565 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kubelet-serving" csr="csr-tbr25"
Sep 17 17:34:14.095535 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:34:14.095516    2565 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kubelet-serving" csr="csr-tbr25"
Sep 17 17:34:14.155210 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:34:14.155165    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-137-20.ec2.internal" event={"ID":"aae62a2aa7c494e1d00d19cbc1cec29c","Type":"ContainerStarted","Data":"4ae390b848b1c2d5b45a5df2906c7417a5590db8ee27dec841cac8761feb9f40"}
Sep 17 17:34:14.156566 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:34:14.156543    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-137-20.ec2.internal" event={"ID":"cda455bc0aed3ae7411ab9a4c6df01be","Type":"ContainerStarted","Data":"264f94d70f70fad78cc9fb1ce08abb03dd3312ff99b5b1bbe4d7ad3e3b5f5baa"}
Sep 17 17:34:14.166788 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:34:14.166769    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/30837de7-1447-4de2-ac62-c809813b3338-run-systemd\") pod \"ovnkube-node-d9927\" (UID: \"30837de7-1447-4de2-ac62-c809813b3338\") " pod="openshift-ovn-kubernetes/ovnkube-node-d9927"
Sep 17 17:34:14.166885 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:34:14.166793    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/30837de7-1447-4de2-ac62-c809813b3338-host-cni-bin\") pod \"ovnkube-node-d9927\" (UID: \"30837de7-1447-4de2-ac62-c809813b3338\") " pod="openshift-ovn-kubernetes/ovnkube-node-d9927"
Sep 17 17:34:14.166885 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:34:14.166808    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/2feeabe5-1136-4f30-9a14-491423e838a3-system-cni-dir\") pod \"multus-5l5k7\" (UID: \"2feeabe5-1136-4f30-9a14-491423e838a3\") " pod="openshift-multus/multus-5l5k7"
Sep 17 17:34:14.166885 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:34:14.166839    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/2feeabe5-1136-4f30-9a14-491423e838a3-multus-cni-dir\") pod \"multus-5l5k7\" (UID: \"2feeabe5-1136-4f30-9a14-491423e838a3\") " pod="openshift-multus/multus-5l5k7"
Sep 17 17:34:14.167038 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:34:14.166880    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/e39c5e1e-113d-42f2-9a00-8a6d14cb6d67-metrics-certs\") pod \"network-metrics-daemon-rmlqx\" (UID: \"e39c5e1e-113d-42f2-9a00-8a6d14cb6d67\") " pod="openshift-multus/network-metrics-daemon-rmlqx"
Sep 17 17:34:14.167038 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:34:14.166911    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/2feeabe5-1136-4f30-9a14-491423e838a3-system-cni-dir\") pod \"multus-5l5k7\" (UID: \"2feeabe5-1136-4f30-9a14-491423e838a3\") " pod="openshift-multus/multus-5l5k7"
Sep 17 17:34:14.167038 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:34:14.166916    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/30837de7-1447-4de2-ac62-c809813b3338-run-systemd\") pod \"ovnkube-node-d9927\" (UID: \"30837de7-1447-4de2-ac62-c809813b3338\") " pod="openshift-ovn-kubernetes/ovnkube-node-d9927"
Sep 17 17:34:14.167038 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:34:14.166880    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/30837de7-1447-4de2-ac62-c809813b3338-host-cni-bin\") pod \"ovnkube-node-d9927\" (UID: \"30837de7-1447-4de2-ac62-c809813b3338\") " pod="openshift-ovn-kubernetes/ovnkube-node-d9927"
Sep 17 17:34:14.167038 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:34:14.166921    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/30837de7-1447-4de2-ac62-c809813b3338-var-lib-openvswitch\") pod \"ovnkube-node-d9927\" (UID: \"30837de7-1447-4de2-ac62-c809813b3338\") " pod="openshift-ovn-kubernetes/ovnkube-node-d9927"
Sep 17 17:34:14.167038 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:34:14.166965    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/30837de7-1447-4de2-ac62-c809813b3338-var-lib-openvswitch\") pod \"ovnkube-node-d9927\" (UID: \"30837de7-1447-4de2-ac62-c809813b3338\") " pod="openshift-ovn-kubernetes/ovnkube-node-d9927"
Sep 17 17:34:14.167038 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:34:14.166977    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/2feeabe5-1136-4f30-9a14-491423e838a3-multus-cni-dir\") pod \"multus-5l5k7\" (UID: \"2feeabe5-1136-4f30-9a14-491423e838a3\") " pod="openshift-multus/multus-5l5k7"
Sep 17 17:34:14.167038 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:34:14.167015    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/30837de7-1447-4de2-ac62-c809813b3338-node-log\") pod \"ovnkube-node-d9927\" (UID: \"30837de7-1447-4de2-ac62-c809813b3338\") " pod="openshift-ovn-kubernetes/ovnkube-node-d9927"
Sep 17 17:34:14.167426 ip-10-0-137-20 kubenswrapper[2565]: E0917 17:34:14.167052    2565 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 17 17:34:14.167426 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:34:14.167068    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/30837de7-1447-4de2-ac62-c809813b3338-ovnkube-config\") pod \"ovnkube-node-d9927\" (UID: \"30837de7-1447-4de2-ac62-c809813b3338\") " pod="openshift-ovn-kubernetes/ovnkube-node-d9927"
Sep 17 17:34:14.167426 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:34:14.167084    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/30837de7-1447-4de2-ac62-c809813b3338-node-log\") pod \"ovnkube-node-d9927\" (UID: \"30837de7-1447-4de2-ac62-c809813b3338\") " pod="openshift-ovn-kubernetes/ovnkube-node-d9927"
Sep 17 17:34:14.167426 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:34:14.167091    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/2feeabe5-1136-4f30-9a14-491423e838a3-host-run-netns\") pod \"multus-5l5k7\" (UID: \"2feeabe5-1136-4f30-9a14-491423e838a3\") " pod="openshift-multus/multus-5l5k7"
Sep 17 17:34:14.167426 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:34:14.167137    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/2feeabe5-1136-4f30-9a14-491423e838a3-host-run-netns\") pod \"multus-5l5k7\" (UID: \"2feeabe5-1136-4f30-9a14-491423e838a3\") " pod="openshift-multus/multus-5l5k7"
Sep 17 17:34:14.167426 ip-10-0-137-20 kubenswrapper[2565]: E0917 17:34:14.167146    2565 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/e39c5e1e-113d-42f2-9a00-8a6d14cb6d67-metrics-certs podName:e39c5e1e-113d-42f2-9a00-8a6d14cb6d67 nodeName:}" failed. No retries permitted until 2026-09-17 17:34:14.66710242 +0000 UTC m=+4.544386830 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/e39c5e1e-113d-42f2-9a00-8a6d14cb6d67-metrics-certs") pod "network-metrics-daemon-rmlqx" (UID: "e39c5e1e-113d-42f2-9a00-8a6d14cb6d67") : object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 17 17:34:14.167426 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:34:14.167163    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/048bee56-5e42-45f0-98a8-213ff62de69e-cni-binary-copy\") pod \"multus-additional-cni-plugins-msv4b\" (UID: \"048bee56-5e42-45f0-98a8-213ff62de69e\") " pod="openshift-multus/multus-additional-cni-plugins-msv4b"
Sep 17 17:34:14.167426 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:34:14.167185    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/caa60d4e-b46b-4fff-8a66-3a5b4f1a1602-tmp-dir\") pod \"node-resolver-vs64q\" (UID: \"caa60d4e-b46b-4fff-8a66-3a5b4f1a1602\") " pod="openshift-dns/node-resolver-vs64q"
Sep 17 17:34:14.167426 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:34:14.167208    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/30837de7-1447-4de2-ac62-c809813b3338-ovnkube-script-lib\") pod \"ovnkube-node-d9927\" (UID: \"30837de7-1447-4de2-ac62-c809813b3338\") " pod="openshift-ovn-kubernetes/ovnkube-node-d9927"
Sep 17 17:34:14.167426 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:34:14.167230    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/d4013629-eb04-40d7-bdb1-11e1cafa9394-etc-sysconfig\") pod \"tuned-mf8sz\" (UID: \"d4013629-eb04-40d7-bdb1-11e1cafa9394\") " pod="openshift-cluster-node-tuning-operator/tuned-mf8sz"
Sep 17 17:34:14.167426 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:34:14.167264    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/d4013629-eb04-40d7-bdb1-11e1cafa9394-tmp\") pod \"tuned-mf8sz\" (UID: \"d4013629-eb04-40d7-bdb1-11e1cafa9394\") " pod="openshift-cluster-node-tuning-operator/tuned-mf8sz"
Sep 17 17:34:14.167426 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:34:14.167285    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/1958175e-39bb-4dc8-a5fb-5cff5b7e85f5-socket-dir\") pod \"aws-ebs-csi-driver-node-fsbkf\" (UID: \"1958175e-39bb-4dc8-a5fb-5cff5b7e85f5\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-fsbkf"
Sep 17 17:34:14.167426 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:34:14.167328    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/d4013629-eb04-40d7-bdb1-11e1cafa9394-etc-kubernetes\") pod \"tuned-mf8sz\" (UID: \"d4013629-eb04-40d7-bdb1-11e1cafa9394\") " pod="openshift-cluster-node-tuning-operator/tuned-mf8sz"
Sep 17 17:34:14.167426 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:34:14.167377    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/d4013629-eb04-40d7-bdb1-11e1cafa9394-etc-sysctl-conf\") pod \"tuned-mf8sz\" (UID: \"d4013629-eb04-40d7-bdb1-11e1cafa9394\") " pod="openshift-cluster-node-tuning-operator/tuned-mf8sz"
Sep 17 17:34:14.167426 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:34:14.167402    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/2feeabe5-1136-4f30-9a14-491423e838a3-multus-conf-dir\") pod \"multus-5l5k7\" (UID: \"2feeabe5-1136-4f30-9a14-491423e838a3\") " pod="openshift-multus/multus-5l5k7"
Sep 17 17:34:14.168143 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:34:14.167520    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xkgxr\" (UniqueName: \"kubernetes.io/projected/2feeabe5-1136-4f30-9a14-491423e838a3-kube-api-access-xkgxr\") pod \"multus-5l5k7\" (UID: \"2feeabe5-1136-4f30-9a14-491423e838a3\") " pod="openshift-multus/multus-5l5k7"
Sep 17 17:34:14.168143 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:34:14.167565    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-98nk4\" (UniqueName: \"kubernetes.io/projected/caa60d4e-b46b-4fff-8a66-3a5b4f1a1602-kube-api-access-98nk4\") pod \"node-resolver-vs64q\" (UID: \"caa60d4e-b46b-4fff-8a66-3a5b4f1a1602\") " pod="openshift-dns/node-resolver-vs64q"
Sep 17 17:34:14.168143 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:34:14.167589    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vx4pk\" (UniqueName: \"kubernetes.io/projected/e39c5e1e-113d-42f2-9a00-8a6d14cb6d67-kube-api-access-vx4pk\") pod \"network-metrics-daemon-rmlqx\" (UID: \"e39c5e1e-113d-42f2-9a00-8a6d14cb6d67\") " pod="openshift-multus/network-metrics-daemon-rmlqx"
Sep 17 17:34:14.168143 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:34:14.167611    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/30837de7-1447-4de2-ac62-c809813b3338-host-run-ovn-kubernetes\") pod \"ovnkube-node-d9927\" (UID: \"30837de7-1447-4de2-ac62-c809813b3338\") " pod="openshift-ovn-kubernetes/ovnkube-node-d9927"
Sep 17 17:34:14.168143 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:34:14.167641    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/048bee56-5e42-45f0-98a8-213ff62de69e-cni-binary-copy\") pod \"multus-additional-cni-plugins-msv4b\" (UID: \"048bee56-5e42-45f0-98a8-213ff62de69e\") " pod="openshift-multus/multus-additional-cni-plugins-msv4b"
Sep 17 17:34:14.168143 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:34:14.167633    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/1958175e-39bb-4dc8-a5fb-5cff5b7e85f5-kubelet-dir\") pod \"aws-ebs-csi-driver-node-fsbkf\" (UID: \"1958175e-39bb-4dc8-a5fb-5cff5b7e85f5\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-fsbkf"
Sep 17 17:34:14.168143 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:34:14.167709    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/2feeabe5-1136-4f30-9a14-491423e838a3-multus-socket-dir-parent\") pod \"multus-5l5k7\" (UID: \"2feeabe5-1136-4f30-9a14-491423e838a3\") " pod="openshift-multus/multus-5l5k7"
Sep 17 17:34:14.168143 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:34:14.167716    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/1958175e-39bb-4dc8-a5fb-5cff5b7e85f5-kubelet-dir\") pod \"aws-ebs-csi-driver-node-fsbkf\" (UID: \"1958175e-39bb-4dc8-a5fb-5cff5b7e85f5\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-fsbkf"
Sep 17 17:34:14.168143 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:34:14.167729    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/d4013629-eb04-40d7-bdb1-11e1cafa9394-sys\") pod \"tuned-mf8sz\" (UID: \"d4013629-eb04-40d7-bdb1-11e1cafa9394\") " pod="openshift-cluster-node-tuning-operator/tuned-mf8sz"
Sep 17 17:34:14.168143 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:34:14.167757    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/d4013629-eb04-40d7-bdb1-11e1cafa9394-sys\") pod \"tuned-mf8sz\" (UID: \"d4013629-eb04-40d7-bdb1-11e1cafa9394\") " pod="openshift-cluster-node-tuning-operator/tuned-mf8sz"
Sep 17 17:34:14.168143 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:34:14.167789    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4rxt7\" (UniqueName: \"kubernetes.io/projected/d4013629-eb04-40d7-bdb1-11e1cafa9394-kube-api-access-4rxt7\") pod \"tuned-mf8sz\" (UID: \"d4013629-eb04-40d7-bdb1-11e1cafa9394\") " pod="openshift-cluster-node-tuning-operator/tuned-mf8sz"
Sep 17 17:34:14.168143 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:34:14.167797    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/2feeabe5-1136-4f30-9a14-491423e838a3-multus-socket-dir-parent\") pod \"multus-5l5k7\" (UID: \"2feeabe5-1136-4f30-9a14-491423e838a3\") " pod="openshift-multus/multus-5l5k7"
Sep 17 17:34:14.168143 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:34:14.167794    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/30837de7-1447-4de2-ac62-c809813b3338-ovnkube-config\") pod \"ovnkube-node-d9927\" (UID: \"30837de7-1447-4de2-ac62-c809813b3338\") " pod="openshift-ovn-kubernetes/ovnkube-node-d9927"
Sep 17 17:34:14.168143 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:34:14.167815    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/2feeabe5-1136-4f30-9a14-491423e838a3-cni-binary-copy\") pod \"multus-5l5k7\" (UID: \"2feeabe5-1136-4f30-9a14-491423e838a3\") " pod="openshift-multus/multus-5l5k7"
Sep 17 17:34:14.168143 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:34:14.167832    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/2feeabe5-1136-4f30-9a14-491423e838a3-multus-conf-dir\") pod \"multus-5l5k7\" (UID: \"2feeabe5-1136-4f30-9a14-491423e838a3\") " pod="openshift-multus/multus-5l5k7"
Sep 17 17:34:14.168143 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:34:14.167839    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/2feeabe5-1136-4f30-9a14-491423e838a3-host-var-lib-cni-multus\") pod \"multus-5l5k7\" (UID: \"2feeabe5-1136-4f30-9a14-491423e838a3\") " pod="openshift-multus/multus-5l5k7"
Sep 17 17:34:14.168143 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:34:14.167863    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/d4013629-eb04-40d7-bdb1-11e1cafa9394-etc-sysconfig\") pod \"tuned-mf8sz\" (UID: \"d4013629-eb04-40d7-bdb1-11e1cafa9394\") " pod="openshift-cluster-node-tuning-operator/tuned-mf8sz"
Sep 17 17:34:14.168884 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:34:14.167864    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/27b114f9-fcc4-463b-897f-9e4561ad5540-serviceca\") pod \"node-ca-5blw5\" (UID: \"27b114f9-fcc4-463b-897f-9e4561ad5540\") " pod="openshift-image-registry/node-ca-5blw5"
Sep 17 17:34:14.168884 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:34:14.167887    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/30837de7-1447-4de2-ac62-c809813b3338-host-slash\") pod \"ovnkube-node-d9927\" (UID: \"30837de7-1447-4de2-ac62-c809813b3338\") " pod="openshift-ovn-kubernetes/ovnkube-node-d9927"
Sep 17 17:34:14.168884 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:34:14.167898    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/30837de7-1447-4de2-ac62-c809813b3338-ovnkube-script-lib\") pod \"ovnkube-node-d9927\" (UID: \"30837de7-1447-4de2-ac62-c809813b3338\") " pod="openshift-ovn-kubernetes/ovnkube-node-d9927"
Sep 17 17:34:14.168884 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:34:14.167905    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/30837de7-1447-4de2-ac62-c809813b3338-run-openvswitch\") pod \"ovnkube-node-d9927\" (UID: \"30837de7-1447-4de2-ac62-c809813b3338\") " pod="openshift-ovn-kubernetes/ovnkube-node-d9927"
Sep 17 17:34:14.168884 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:34:14.167927    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/d4013629-eb04-40d7-bdb1-11e1cafa9394-var-lib-kubelet\") pod \"tuned-mf8sz\" (UID: \"d4013629-eb04-40d7-bdb1-11e1cafa9394\") " pod="openshift-cluster-node-tuning-operator/tuned-mf8sz"
Sep 17 17:34:14.168884 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:34:14.167951    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/048bee56-5e42-45f0-98a8-213ff62de69e-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-msv4b\" (UID: \"048bee56-5e42-45f0-98a8-213ff62de69e\") " pod="openshift-multus/multus-additional-cni-plugins-msv4b"
Sep 17 17:34:14.168884 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:34:14.167977    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/048bee56-5e42-45f0-98a8-213ff62de69e-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-msv4b\" (UID: \"048bee56-5e42-45f0-98a8-213ff62de69e\") " pod="openshift-multus/multus-additional-cni-plugins-msv4b"
Sep 17 17:34:14.168884 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:34:14.168066    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/30837de7-1447-4de2-ac62-c809813b3338-run-openvswitch\") pod \"ovnkube-node-d9927\" (UID: \"30837de7-1447-4de2-ac62-c809813b3338\") " pod="openshift-ovn-kubernetes/ovnkube-node-d9927"
Sep 17 17:34:14.168884 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:34:14.168092    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/caa60d4e-b46b-4fff-8a66-3a5b4f1a1602-tmp-dir\") pod \"node-resolver-vs64q\" (UID: \"caa60d4e-b46b-4fff-8a66-3a5b4f1a1602\") " pod="openshift-dns/node-resolver-vs64q"
Sep 17 17:34:14.169339 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:34:14.169143    2565 swap_util.go:74] "error creating dir to test if tmpfs noswap is enabled. Assuming not supported" mount path="" error="stat /var/lib/kubelet/plugins/kubernetes.io/empty-dir: no such file or directory"
Sep 17 17:34:14.169339 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:34:14.169226    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/2e847ec5-02d2-4210-a6ce-89d926b050f3-konnectivity-ca\") pod \"konnectivity-agent-p8jd8\" (UID: \"2e847ec5-02d2-4210-a6ce-89d926b050f3\") " pod="kube-system/konnectivity-agent-p8jd8"
Sep 17 17:34:14.169432 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:34:14.169409    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/2e847ec5-02d2-4210-a6ce-89d926b050f3-konnectivity-ca\") pod \"konnectivity-agent-p8jd8\" (UID: \"2e847ec5-02d2-4210-a6ce-89d926b050f3\") " pod="kube-system/konnectivity-agent-p8jd8"
Sep 17 17:34:14.169476 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:34:14.169466    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/30837de7-1447-4de2-ac62-c809813b3338-ovn-node-metrics-cert\") pod \"ovnkube-node-d9927\" (UID: \"30837de7-1447-4de2-ac62-c809813b3338\") " pod="openshift-ovn-kubernetes/ovnkube-node-d9927"
Sep 17 17:34:14.169523 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:34:14.169505    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/1958175e-39bb-4dc8-a5fb-5cff5b7e85f5-device-dir\") pod \"aws-ebs-csi-driver-node-fsbkf\" (UID: \"1958175e-39bb-4dc8-a5fb-5cff5b7e85f5\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-fsbkf"
Sep 17 17:34:14.169574 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:34:14.169538    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/1958175e-39bb-4dc8-a5fb-5cff5b7e85f5-etc-selinux\") pod \"aws-ebs-csi-driver-node-fsbkf\" (UID: \"1958175e-39bb-4dc8-a5fb-5cff5b7e85f5\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-fsbkf"
Sep 17 17:34:14.169625 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:34:14.169572    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/1958175e-39bb-4dc8-a5fb-5cff5b7e85f5-sys-fs\") pod \"aws-ebs-csi-driver-node-fsbkf\" (UID: \"1958175e-39bb-4dc8-a5fb-5cff5b7e85f5\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-fsbkf"
Sep 17 17:34:14.169625 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:34:14.169599    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/caa60d4e-b46b-4fff-8a66-3a5b4f1a1602-hosts-file\") pod \"node-resolver-vs64q\" (UID: \"caa60d4e-b46b-4fff-8a66-3a5b4f1a1602\") " pod="openshift-dns/node-resolver-vs64q"
Sep 17 17:34:14.169712 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:34:14.169633    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/30837de7-1447-4de2-ac62-c809813b3338-run-ovn\") pod \"ovnkube-node-d9927\" (UID: \"30837de7-1447-4de2-ac62-c809813b3338\") " pod="openshift-ovn-kubernetes/ovnkube-node-d9927"
Sep 17 17:34:14.169712 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:34:14.169668    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/30837de7-1447-4de2-ac62-c809813b3338-env-overrides\") pod \"ovnkube-node-d9927\" (UID: \"30837de7-1447-4de2-ac62-c809813b3338\") " pod="openshift-ovn-kubernetes/ovnkube-node-d9927"
Sep 17 17:34:14.169712 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:34:14.169701    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4vqpv\" (UniqueName: \"kubernetes.io/projected/1958175e-39bb-4dc8-a5fb-5cff5b7e85f5-kube-api-access-4vqpv\") pod \"aws-ebs-csi-driver-node-fsbkf\" (UID: \"1958175e-39bb-4dc8-a5fb-5cff5b7e85f5\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-fsbkf"
Sep 17 17:34:14.169848 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:34:14.169700    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/048bee56-5e42-45f0-98a8-213ff62de69e-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-msv4b\" (UID: \"048bee56-5e42-45f0-98a8-213ff62de69e\") " pod="openshift-multus/multus-additional-cni-plugins-msv4b"
Sep 17 17:34:14.169848 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:34:14.169735    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/2feeabe5-1136-4f30-9a14-491423e838a3-host-run-multus-certs\") pod \"multus-5l5k7\" (UID: \"2feeabe5-1136-4f30-9a14-491423e838a3\") " pod="openshift-multus/multus-5l5k7"
Sep 17 17:34:14.169848 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:34:14.169762    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/30837de7-1447-4de2-ac62-c809813b3338-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-d9927\" (UID: \"30837de7-1447-4de2-ac62-c809813b3338\") " pod="openshift-ovn-kubernetes/ovnkube-node-d9927"
Sep 17 17:34:14.169848 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:34:14.169797    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/3d1c780d-6713-495e-91fb-54a74668e6cc-kubelet-config\") pod \"global-pull-secret-syncer-74xd8\" (UID: \"3d1c780d-6713-495e-91fb-54a74668e6cc\") " pod="kube-system/global-pull-secret-syncer-74xd8"
Sep 17 17:34:14.169848 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:34:14.169826    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/3d1c780d-6713-495e-91fb-54a74668e6cc-dbus\") pod \"global-pull-secret-syncer-74xd8\" (UID: \"3d1c780d-6713-495e-91fb-54a74668e6cc\") " pod="kube-system/global-pull-secret-syncer-74xd8"
Sep 17 17:34:14.170095 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:34:14.169856    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/d4013629-eb04-40d7-bdb1-11e1cafa9394-etc-tuned\") pod \"tuned-mf8sz\" (UID: \"d4013629-eb04-40d7-bdb1-11e1cafa9394\") " pod="openshift-cluster-node-tuning-operator/tuned-mf8sz"
Sep 17 17:34:14.170095 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:34:14.169884    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/af1cc6fe-9695-4e2e-9244-6505022f6c50-host-slash\") pod \"iptables-alerter-g8kpq\" (UID: \"af1cc6fe-9695-4e2e-9244-6505022f6c50\") " pod="openshift-network-operator/iptables-alerter-g8kpq"
Sep 17 17:34:14.170095 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:34:14.169918    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/2feeabe5-1136-4f30-9a14-491423e838a3-host-var-lib-cni-bin\") pod \"multus-5l5k7\" (UID: \"2feeabe5-1136-4f30-9a14-491423e838a3\") " pod="openshift-multus/multus-5l5k7"
Sep 17 17:34:14.170095 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:34:14.169952    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/2feeabe5-1136-4f30-9a14-491423e838a3-multus-daemon-config\") pod \"multus-5l5k7\" (UID: \"2feeabe5-1136-4f30-9a14-491423e838a3\") " pod="openshift-multus/multus-5l5k7"
Sep 17 17:34:14.170095 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:34:14.169983    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-877s8\" (UniqueName: \"kubernetes.io/projected/af1cc6fe-9695-4e2e-9244-6505022f6c50-kube-api-access-877s8\") pod \"iptables-alerter-g8kpq\" (UID: \"af1cc6fe-9695-4e2e-9244-6505022f6c50\") " pod="openshift-network-operator/iptables-alerter-g8kpq"
Sep 17 17:34:14.170095 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:34:14.170013    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/048bee56-5e42-45f0-98a8-213ff62de69e-cnibin\") pod \"multus-additional-cni-plugins-msv4b\" (UID: \"048bee56-5e42-45f0-98a8-213ff62de69e\") " pod="openshift-multus/multus-additional-cni-plugins-msv4b"
Sep 17 17:34:14.170095 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:34:14.170068    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-n9xm8\" (UniqueName: \"kubernetes.io/projected/048bee56-5e42-45f0-98a8-213ff62de69e-kube-api-access-n9xm8\") pod \"multus-additional-cni-plugins-msv4b\" (UID: \"048bee56-5e42-45f0-98a8-213ff62de69e\") " pod="openshift-multus/multus-additional-cni-plugins-msv4b"
Sep 17 17:34:14.170400 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:34:14.170098    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/30837de7-1447-4de2-ac62-c809813b3338-systemd-units\") pod \"ovnkube-node-d9927\" (UID: \"30837de7-1447-4de2-ac62-c809813b3338\") " pod="openshift-ovn-kubernetes/ovnkube-node-d9927"
Sep 17 17:34:14.170400 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:34:14.170126    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/30837de7-1447-4de2-ac62-c809813b3338-etc-openvswitch\") pod \"ovnkube-node-d9927\" (UID: \"30837de7-1447-4de2-ac62-c809813b3338\") " pod="openshift-ovn-kubernetes/ovnkube-node-d9927"
Sep 17 17:34:14.170400 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:34:14.170128    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/1958175e-39bb-4dc8-a5fb-5cff5b7e85f5-socket-dir\") pod \"aws-ebs-csi-driver-node-fsbkf\" (UID: \"1958175e-39bb-4dc8-a5fb-5cff5b7e85f5\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-fsbkf"
Sep 17 17:34:14.170400 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:34:14.170157    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/d4013629-eb04-40d7-bdb1-11e1cafa9394-etc-modprobe-d\") pod \"tuned-mf8sz\" (UID: \"d4013629-eb04-40d7-bdb1-11e1cafa9394\") " pod="openshift-cluster-node-tuning-operator/tuned-mf8sz"
Sep 17 17:34:14.170400 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:34:14.170186    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/d4013629-eb04-40d7-bdb1-11e1cafa9394-host\") pod \"tuned-mf8sz\" (UID: \"d4013629-eb04-40d7-bdb1-11e1cafa9394\") " pod="openshift-cluster-node-tuning-operator/tuned-mf8sz"
Sep 17 17:34:14.170400 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:34:14.170218    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/2feeabe5-1136-4f30-9a14-491423e838a3-host-run-k8s-cni-cncf-io\") pod \"multus-5l5k7\" (UID: \"2feeabe5-1136-4f30-9a14-491423e838a3\") " pod="openshift-multus/multus-5l5k7"
Sep 17 17:34:14.170400 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:34:14.170257    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/2feeabe5-1136-4f30-9a14-491423e838a3-host-var-lib-kubelet\") pod \"multus-5l5k7\" (UID: \"2feeabe5-1136-4f30-9a14-491423e838a3\") " pod="openshift-multus/multus-5l5k7"
Sep 17 17:34:14.170400 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:34:14.170284    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/2feeabe5-1136-4f30-9a14-491423e838a3-etc-kubernetes\") pod \"multus-5l5k7\" (UID: \"2feeabe5-1136-4f30-9a14-491423e838a3\") " pod="openshift-multus/multus-5l5k7"
Sep 17 17:34:14.170400 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:34:14.170319    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/30837de7-1447-4de2-ac62-c809813b3338-host-run-netns\") pod \"ovnkube-node-d9927\" (UID: \"30837de7-1447-4de2-ac62-c809813b3338\") " pod="openshift-ovn-kubernetes/ovnkube-node-d9927"
Sep 17 17:34:14.170400 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:34:14.170351    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/30837de7-1447-4de2-ac62-c809813b3338-host-cni-netd\") pod \"ovnkube-node-d9927\" (UID: \"30837de7-1447-4de2-ac62-c809813b3338\") " pod="openshift-ovn-kubernetes/ovnkube-node-d9927"
Sep 17 17:34:14.170400 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:34:14.170360    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/048bee56-5e42-45f0-98a8-213ff62de69e-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-msv4b\" (UID: \"048bee56-5e42-45f0-98a8-213ff62de69e\") " pod="openshift-multus/multus-additional-cni-plugins-msv4b"
Sep 17 17:34:14.170400 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:34:14.170384    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/2e847ec5-02d2-4210-a6ce-89d926b050f3-agent-certs\") pod \"konnectivity-agent-p8jd8\" (UID: \"2e847ec5-02d2-4210-a6ce-89d926b050f3\") " pod="kube-system/konnectivity-agent-p8jd8"
Sep 17 17:34:14.170932 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:34:14.170412    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/d4013629-eb04-40d7-bdb1-11e1cafa9394-etc-sysctl-d\") pod \"tuned-mf8sz\" (UID: \"d4013629-eb04-40d7-bdb1-11e1cafa9394\") " pod="openshift-cluster-node-tuning-operator/tuned-mf8sz"
Sep 17 17:34:14.170932 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:34:14.170442    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/27b114f9-fcc4-463b-897f-9e4561ad5540-host\") pod \"node-ca-5blw5\" (UID: \"27b114f9-fcc4-463b-897f-9e4561ad5540\") " pod="openshift-image-registry/node-ca-5blw5"
Sep 17 17:34:14.170932 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:34:14.170475    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-nd6qt\" (UniqueName: \"kubernetes.io/projected/30837de7-1447-4de2-ac62-c809813b3338-kube-api-access-nd6qt\") pod \"ovnkube-node-d9927\" (UID: \"30837de7-1447-4de2-ac62-c809813b3338\") " pod="openshift-ovn-kubernetes/ovnkube-node-d9927"
Sep 17 17:34:14.170932 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:34:14.170509    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/1958175e-39bb-4dc8-a5fb-5cff5b7e85f5-registration-dir\") pod \"aws-ebs-csi-driver-node-fsbkf\" (UID: \"1958175e-39bb-4dc8-a5fb-5cff5b7e85f5\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-fsbkf"
Sep 17 17:34:14.170932 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:34:14.170545    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/d4013629-eb04-40d7-bdb1-11e1cafa9394-etc-systemd\") pod \"tuned-mf8sz\" (UID: \"d4013629-eb04-40d7-bdb1-11e1cafa9394\") " pod="openshift-cluster-node-tuning-operator/tuned-mf8sz"
Sep 17 17:34:14.170932 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:34:14.170571    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/2feeabe5-1136-4f30-9a14-491423e838a3-cni-binary-copy\") pod \"multus-5l5k7\" (UID: \"2feeabe5-1136-4f30-9a14-491423e838a3\") " pod="openshift-multus/multus-5l5k7"
Sep 17 17:34:14.170932 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:34:14.170574    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-s8xpr\" (UniqueName: \"kubernetes.io/projected/574f2e00-d361-423b-946b-e1e9ea3a5db5-kube-api-access-s8xpr\") pod \"network-check-target-jsc2p\" (UID: \"574f2e00-d361-423b-946b-e1e9ea3a5db5\") " pod="openshift-network-diagnostics/network-check-target-jsc2p"
Sep 17 17:34:14.170932 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:34:14.170606    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-p7fds\" (UniqueName: \"kubernetes.io/projected/27b114f9-fcc4-463b-897f-9e4561ad5540-kube-api-access-p7fds\") pod \"node-ca-5blw5\" (UID: \"27b114f9-fcc4-463b-897f-9e4561ad5540\") " pod="openshift-image-registry/node-ca-5blw5"
Sep 17 17:34:14.170932 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:34:14.170644    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/048bee56-5e42-45f0-98a8-213ff62de69e-system-cni-dir\") pod \"multus-additional-cni-plugins-msv4b\" (UID: \"048bee56-5e42-45f0-98a8-213ff62de69e\") " pod="openshift-multus/multus-additional-cni-plugins-msv4b"
Sep 17 17:34:14.170932 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:34:14.170674    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/2feeabe5-1136-4f30-9a14-491423e838a3-cnibin\") pod \"multus-5l5k7\" (UID: \"2feeabe5-1136-4f30-9a14-491423e838a3\") " pod="openshift-multus/multus-5l5k7"
Sep 17 17:34:14.170932 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:34:14.170707    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/2feeabe5-1136-4f30-9a14-491423e838a3-hostroot\") pod \"multus-5l5k7\" (UID: \"2feeabe5-1136-4f30-9a14-491423e838a3\") " pod="openshift-multus/multus-5l5k7"
Sep 17 17:34:14.170932 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:34:14.170736    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/048bee56-5e42-45f0-98a8-213ff62de69e-tuning-conf-dir\") pod \"multus-additional-cni-plugins-msv4b\" (UID: \"048bee56-5e42-45f0-98a8-213ff62de69e\") " pod="openshift-multus/multus-additional-cni-plugins-msv4b"
Sep 17 17:34:14.170932 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:34:14.170777    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/30837de7-1447-4de2-ac62-c809813b3338-log-socket\") pod \"ovnkube-node-d9927\" (UID: \"30837de7-1447-4de2-ac62-c809813b3338\") " pod="openshift-ovn-kubernetes/ovnkube-node-d9927"
Sep 17 17:34:14.170932 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:34:14.170807    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/d4013629-eb04-40d7-bdb1-11e1cafa9394-run\") pod \"tuned-mf8sz\" (UID: \"d4013629-eb04-40d7-bdb1-11e1cafa9394\") " pod="openshift-cluster-node-tuning-operator/tuned-mf8sz"
Sep 17 17:34:14.170932 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:34:14.170840    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/3d1c780d-6713-495e-91fb-54a74668e6cc-original-pull-secret\") pod \"global-pull-secret-syncer-74xd8\" (UID: \"3d1c780d-6713-495e-91fb-54a74668e6cc\") " pod="kube-system/global-pull-secret-syncer-74xd8"
Sep 17 17:34:14.170932 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:34:14.170870    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/d4013629-eb04-40d7-bdb1-11e1cafa9394-lib-modules\") pod \"tuned-mf8sz\" (UID: \"d4013629-eb04-40d7-bdb1-11e1cafa9394\") " pod="openshift-cluster-node-tuning-operator/tuned-mf8sz"
Sep 17 17:34:14.170932 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:34:14.170899    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/af1cc6fe-9695-4e2e-9244-6505022f6c50-iptables-alerter-script\") pod \"iptables-alerter-g8kpq\" (UID: \"af1cc6fe-9695-4e2e-9244-6505022f6c50\") " pod="openshift-network-operator/iptables-alerter-g8kpq"
Sep 17 17:34:14.171679 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:34:14.170931    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/30837de7-1447-4de2-ac62-c809813b3338-host-kubelet\") pod \"ovnkube-node-d9927\" (UID: \"30837de7-1447-4de2-ac62-c809813b3338\") " pod="openshift-ovn-kubernetes/ovnkube-node-d9927"
Sep 17 17:34:14.171679 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:34:14.171092    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/2feeabe5-1136-4f30-9a14-491423e838a3-host-var-lib-cni-multus\") pod \"multus-5l5k7\" (UID: \"2feeabe5-1136-4f30-9a14-491423e838a3\") " pod="openshift-multus/multus-5l5k7"
Sep 17 17:34:14.171679 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:34:14.171100    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/30837de7-1447-4de2-ac62-c809813b3338-host-kubelet\") pod \"ovnkube-node-d9927\" (UID: \"30837de7-1447-4de2-ac62-c809813b3338\") " pod="openshift-ovn-kubernetes/ovnkube-node-d9927"
Sep 17 17:34:14.171679 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:34:14.171227    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/1958175e-39bb-4dc8-a5fb-5cff5b7e85f5-etc-selinux\") pod \"aws-ebs-csi-driver-node-fsbkf\" (UID: \"1958175e-39bb-4dc8-a5fb-5cff5b7e85f5\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-fsbkf"
Sep 17 17:34:14.171679 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:34:14.171304    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/1958175e-39bb-4dc8-a5fb-5cff5b7e85f5-sys-fs\") pod \"aws-ebs-csi-driver-node-fsbkf\" (UID: \"1958175e-39bb-4dc8-a5fb-5cff5b7e85f5\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-fsbkf"
Sep 17 17:34:14.171679 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:34:14.171362    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/caa60d4e-b46b-4fff-8a66-3a5b4f1a1602-hosts-file\") pod \"node-resolver-vs64q\" (UID: \"caa60d4e-b46b-4fff-8a66-3a5b4f1a1602\") " pod="openshift-dns/node-resolver-vs64q"
Sep 17 17:34:14.171679 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:34:14.171423    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/30837de7-1447-4de2-ac62-c809813b3338-run-ovn\") pod \"ovnkube-node-d9927\" (UID: \"30837de7-1447-4de2-ac62-c809813b3338\") " pod="openshift-ovn-kubernetes/ovnkube-node-d9927"
Sep 17 17:34:14.171985 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:34:14.171810    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/30837de7-1447-4de2-ac62-c809813b3338-env-overrides\") pod \"ovnkube-node-d9927\" (UID: \"30837de7-1447-4de2-ac62-c809813b3338\") " pod="openshift-ovn-kubernetes/ovnkube-node-d9927"
Sep 17 17:34:14.172349 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:34:14.172193    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/2feeabe5-1136-4f30-9a14-491423e838a3-host-run-multus-certs\") pod \"multus-5l5k7\" (UID: \"2feeabe5-1136-4f30-9a14-491423e838a3\") " pod="openshift-multus/multus-5l5k7"
Sep 17 17:34:14.172349 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:34:14.172268    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/30837de7-1447-4de2-ac62-c809813b3338-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-d9927\" (UID: \"30837de7-1447-4de2-ac62-c809813b3338\") " pod="openshift-ovn-kubernetes/ovnkube-node-d9927"
Sep 17 17:34:14.172349 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:34:14.172345    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/3d1c780d-6713-495e-91fb-54a74668e6cc-kubelet-config\") pod \"global-pull-secret-syncer-74xd8\" (UID: \"3d1c780d-6713-495e-91fb-54a74668e6cc\") " pod="kube-system/global-pull-secret-syncer-74xd8"
Sep 17 17:34:14.172515 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:34:14.169797    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/d4013629-eb04-40d7-bdb1-11e1cafa9394-etc-kubernetes\") pod \"tuned-mf8sz\" (UID: \"d4013629-eb04-40d7-bdb1-11e1cafa9394\") " pod="openshift-cluster-node-tuning-operator/tuned-mf8sz"
Sep 17 17:34:14.172515 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:34:14.172437    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/3d1c780d-6713-495e-91fb-54a74668e6cc-dbus\") pod \"global-pull-secret-syncer-74xd8\" (UID: \"3d1c780d-6713-495e-91fb-54a74668e6cc\") " pod="kube-system/global-pull-secret-syncer-74xd8"
Sep 17 17:34:14.172815 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:34:14.172777    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/d4013629-eb04-40d7-bdb1-11e1cafa9394-var-lib-kubelet\") pod \"tuned-mf8sz\" (UID: \"d4013629-eb04-40d7-bdb1-11e1cafa9394\") " pod="openshift-cluster-node-tuning-operator/tuned-mf8sz"
Sep 17 17:34:14.172910 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:34:14.172891    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/30837de7-1447-4de2-ac62-c809813b3338-etc-openvswitch\") pod \"ovnkube-node-d9927\" (UID: \"30837de7-1447-4de2-ac62-c809813b3338\") " pod="openshift-ovn-kubernetes/ovnkube-node-d9927"
Sep 17 17:34:14.172990 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:34:14.172966    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/af1cc6fe-9695-4e2e-9244-6505022f6c50-host-slash\") pod \"iptables-alerter-g8kpq\" (UID: \"af1cc6fe-9695-4e2e-9244-6505022f6c50\") " pod="openshift-network-operator/iptables-alerter-g8kpq"
Sep 17 17:34:14.173097 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:34:14.173061    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/1958175e-39bb-4dc8-a5fb-5cff5b7e85f5-device-dir\") pod \"aws-ebs-csi-driver-node-fsbkf\" (UID: \"1958175e-39bb-4dc8-a5fb-5cff5b7e85f5\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-fsbkf"
Sep 17 17:34:14.173164 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:34:14.173145    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/2feeabe5-1136-4f30-9a14-491423e838a3-host-var-lib-cni-bin\") pod \"multus-5l5k7\" (UID: \"2feeabe5-1136-4f30-9a14-491423e838a3\") " pod="openshift-multus/multus-5l5k7"
Sep 17 17:34:14.173237 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:34:14.173216    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/048bee56-5e42-45f0-98a8-213ff62de69e-cnibin\") pod \"multus-additional-cni-plugins-msv4b\" (UID: \"048bee56-5e42-45f0-98a8-213ff62de69e\") " pod="openshift-multus/multus-additional-cni-plugins-msv4b"
Sep 17 17:34:14.173499 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:34:14.173425    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/048bee56-5e42-45f0-98a8-213ff62de69e-system-cni-dir\") pod \"multus-additional-cni-plugins-msv4b\" (UID: \"048bee56-5e42-45f0-98a8-213ff62de69e\") " pod="openshift-multus/multus-additional-cni-plugins-msv4b"
Sep 17 17:34:14.173608 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:34:14.173577    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/d4013629-eb04-40d7-bdb1-11e1cafa9394-host\") pod \"tuned-mf8sz\" (UID: \"d4013629-eb04-40d7-bdb1-11e1cafa9394\") " pod="openshift-cluster-node-tuning-operator/tuned-mf8sz"
Sep 17 17:34:14.173663 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:34:14.173618    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/2feeabe5-1136-4f30-9a14-491423e838a3-multus-daemon-config\") pod \"multus-5l5k7\" (UID: \"2feeabe5-1136-4f30-9a14-491423e838a3\") " pod="openshift-multus/multus-5l5k7"
Sep 17 17:34:14.173716 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:34:14.173673    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/27b114f9-fcc4-463b-897f-9e4561ad5540-host\") pod \"node-ca-5blw5\" (UID: \"27b114f9-fcc4-463b-897f-9e4561ad5540\") " pod="openshift-image-registry/node-ca-5blw5"
Sep 17 17:34:14.173785 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:34:14.173755    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/2feeabe5-1136-4f30-9a14-491423e838a3-host-run-k8s-cni-cncf-io\") pod \"multus-5l5k7\" (UID: \"2feeabe5-1136-4f30-9a14-491423e838a3\") " pod="openshift-multus/multus-5l5k7"
Sep 17 17:34:14.173785 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:34:14.173780    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/1958175e-39bb-4dc8-a5fb-5cff5b7e85f5-registration-dir\") pod \"aws-ebs-csi-driver-node-fsbkf\" (UID: \"1958175e-39bb-4dc8-a5fb-5cff5b7e85f5\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-fsbkf"
Sep 17 17:34:14.173897 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:34:14.173882    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/d4013629-eb04-40d7-bdb1-11e1cafa9394-etc-modprobe-d\") pod \"tuned-mf8sz\" (UID: \"d4013629-eb04-40d7-bdb1-11e1cafa9394\") " pod="openshift-cluster-node-tuning-operator/tuned-mf8sz"
Sep 17 17:34:14.173957 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:34:14.173923    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/d4013629-eb04-40d7-bdb1-11e1cafa9394-etc-systemd\") pod \"tuned-mf8sz\" (UID: \"d4013629-eb04-40d7-bdb1-11e1cafa9394\") " pod="openshift-cluster-node-tuning-operator/tuned-mf8sz"
Sep 17 17:34:14.174003 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:34:14.173983    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/d4013629-eb04-40d7-bdb1-11e1cafa9394-etc-sysctl-d\") pod \"tuned-mf8sz\" (UID: \"d4013629-eb04-40d7-bdb1-11e1cafa9394\") " pod="openshift-cluster-node-tuning-operator/tuned-mf8sz"
Sep 17 17:34:14.174071 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:34:14.174004    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/30837de7-1447-4de2-ac62-c809813b3338-systemd-units\") pod \"ovnkube-node-d9927\" (UID: \"30837de7-1447-4de2-ac62-c809813b3338\") " pod="openshift-ovn-kubernetes/ovnkube-node-d9927"
Sep 17 17:34:14.174156 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:34:14.174079    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/2feeabe5-1136-4f30-9a14-491423e838a3-host-var-lib-kubelet\") pod \"multus-5l5k7\" (UID: \"2feeabe5-1136-4f30-9a14-491423e838a3\") " pod="openshift-multus/multus-5l5k7"
Sep 17 17:34:14.174238 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:34:14.174220    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/30837de7-1447-4de2-ac62-c809813b3338-log-socket\") pod \"ovnkube-node-d9927\" (UID: \"30837de7-1447-4de2-ac62-c809813b3338\") " pod="openshift-ovn-kubernetes/ovnkube-node-d9927"
Sep 17 17:34:14.174306 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:34:14.174291    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/2feeabe5-1136-4f30-9a14-491423e838a3-cnibin\") pod \"multus-5l5k7\" (UID: \"2feeabe5-1136-4f30-9a14-491423e838a3\") " pod="openshift-multus/multus-5l5k7"
Sep 17 17:34:14.174354 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:34:14.174317    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/30837de7-1447-4de2-ac62-c809813b3338-host-run-netns\") pod \"ovnkube-node-d9927\" (UID: \"30837de7-1447-4de2-ac62-c809813b3338\") " pod="openshift-ovn-kubernetes/ovnkube-node-d9927"
Sep 17 17:34:14.174405 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:34:14.174397    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/2feeabe5-1136-4f30-9a14-491423e838a3-hostroot\") pod \"multus-5l5k7\" (UID: \"2feeabe5-1136-4f30-9a14-491423e838a3\") " pod="openshift-multus/multus-5l5k7"
Sep 17 17:34:14.174454 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:34:14.174420    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/2feeabe5-1136-4f30-9a14-491423e838a3-etc-kubernetes\") pod \"multus-5l5k7\" (UID: \"2feeabe5-1136-4f30-9a14-491423e838a3\") " pod="openshift-multus/multus-5l5k7"
Sep 17 17:34:14.174454 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:34:14.167952    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/30837de7-1447-4de2-ac62-c809813b3338-host-run-ovn-kubernetes\") pod \"ovnkube-node-d9927\" (UID: \"30837de7-1447-4de2-ac62-c809813b3338\") " pod="openshift-ovn-kubernetes/ovnkube-node-d9927"
Sep 17 17:34:14.174551 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:34:14.174467    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/30837de7-1447-4de2-ac62-c809813b3338-host-slash\") pod \"ovnkube-node-d9927\" (UID: \"30837de7-1447-4de2-ac62-c809813b3338\") " pod="openshift-ovn-kubernetes/ovnkube-node-d9927"
Sep 17 17:34:14.174600 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:34:14.174548    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/30837de7-1447-4de2-ac62-c809813b3338-host-cni-netd\") pod \"ovnkube-node-d9927\" (UID: \"30837de7-1447-4de2-ac62-c809813b3338\") " pod="openshift-ovn-kubernetes/ovnkube-node-d9927"
Sep 17 17:34:14.174652 ip-10-0-137-20 kubenswrapper[2565]: E0917 17:34:14.174602    2565 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Sep 17 17:34:14.174702 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:34:14.174666    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run\" (UniqueName: \"kubernetes.io/host-path/d4013629-eb04-40d7-bdb1-11e1cafa9394-run\") pod \"tuned-mf8sz\" (UID: \"d4013629-eb04-40d7-bdb1-11e1cafa9394\") " pod="openshift-cluster-node-tuning-operator/tuned-mf8sz"
Sep 17 17:34:14.174808 ip-10-0-137-20 kubenswrapper[2565]: E0917 17:34:14.174784    2565 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/3d1c780d-6713-495e-91fb-54a74668e6cc-original-pull-secret podName:3d1c780d-6713-495e-91fb-54a74668e6cc nodeName:}" failed. No retries permitted until 2026-09-17 17:34:14.674766707 +0000 UTC m=+4.552051122 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/3d1c780d-6713-495e-91fb-54a74668e6cc-original-pull-secret") pod "global-pull-secret-syncer-74xd8" (UID: "3d1c780d-6713-495e-91fb-54a74668e6cc") : object "kube-system"/"original-pull-secret" not registered
Sep 17 17:34:14.174873 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:34:14.174852    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/d4013629-eb04-40d7-bdb1-11e1cafa9394-lib-modules\") pod \"tuned-mf8sz\" (UID: \"d4013629-eb04-40d7-bdb1-11e1cafa9394\") " pod="openshift-cluster-node-tuning-operator/tuned-mf8sz"
Sep 17 17:34:14.176012 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:34:14.175058    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/30837de7-1447-4de2-ac62-c809813b3338-ovn-node-metrics-cert\") pod \"ovnkube-node-d9927\" (UID: \"30837de7-1447-4de2-ac62-c809813b3338\") " pod="openshift-ovn-kubernetes/ovnkube-node-d9927"
Sep 17 17:34:14.176012 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:34:14.175165    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/27b114f9-fcc4-463b-897f-9e4561ad5540-serviceca\") pod \"node-ca-5blw5\" (UID: \"27b114f9-fcc4-463b-897f-9e4561ad5540\") " pod="openshift-image-registry/node-ca-5blw5"
Sep 17 17:34:14.176012 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:34:14.168282    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/d4013629-eb04-40d7-bdb1-11e1cafa9394-etc-sysctl-conf\") pod \"tuned-mf8sz\" (UID: \"d4013629-eb04-40d7-bdb1-11e1cafa9394\") " pod="openshift-cluster-node-tuning-operator/tuned-mf8sz"
Sep 17 17:34:14.176012 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:34:14.175571    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/af1cc6fe-9695-4e2e-9244-6505022f6c50-iptables-alerter-script\") pod \"iptables-alerter-g8kpq\" (UID: \"af1cc6fe-9695-4e2e-9244-6505022f6c50\") " pod="openshift-network-operator/iptables-alerter-g8kpq"
Sep 17 17:34:14.176550 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:34:14.176450    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/048bee56-5e42-45f0-98a8-213ff62de69e-tuning-conf-dir\") pod \"multus-additional-cni-plugins-msv4b\" (UID: \"048bee56-5e42-45f0-98a8-213ff62de69e\") " pod="openshift-multus/multus-additional-cni-plugins-msv4b"
Sep 17 17:34:14.177938 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:34:14.177230    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/d4013629-eb04-40d7-bdb1-11e1cafa9394-etc-tuned\") pod \"tuned-mf8sz\" (UID: \"d4013629-eb04-40d7-bdb1-11e1cafa9394\") " pod="openshift-cluster-node-tuning-operator/tuned-mf8sz"
Sep 17 17:34:14.178652 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:34:14.178633    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/d4013629-eb04-40d7-bdb1-11e1cafa9394-tmp\") pod \"tuned-mf8sz\" (UID: \"d4013629-eb04-40d7-bdb1-11e1cafa9394\") " pod="openshift-cluster-node-tuning-operator/tuned-mf8sz"
Sep 17 17:34:14.179400 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:34:14.179331    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/2e847ec5-02d2-4210-a6ce-89d926b050f3-agent-certs\") pod \"konnectivity-agent-p8jd8\" (UID: \"2e847ec5-02d2-4210-a6ce-89d926b050f3\") " pod="kube-system/konnectivity-agent-p8jd8"
Sep 17 17:34:14.180947 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:34:14.180925    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-98nk4\" (UniqueName: \"kubernetes.io/projected/caa60d4e-b46b-4fff-8a66-3a5b4f1a1602-kube-api-access-98nk4\") pod \"node-resolver-vs64q\" (UID: \"caa60d4e-b46b-4fff-8a66-3a5b4f1a1602\") " pod="openshift-dns/node-resolver-vs64q"
Sep 17 17:34:14.186039 ip-10-0-137-20 kubenswrapper[2565]: E0917 17:34:14.185783    2565 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Sep 17 17:34:14.186039 ip-10-0-137-20 kubenswrapper[2565]: E0917 17:34:14.185804    2565 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Sep 17 17:34:14.186039 ip-10-0-137-20 kubenswrapper[2565]: E0917 17:34:14.185818    2565 projected.go:196] Error preparing data for projected volume kube-api-access-s8xpr for pod openshift-network-diagnostics/network-check-target-jsc2p: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 17 17:34:14.186039 ip-10-0-137-20 kubenswrapper[2565]: E0917 17:34:14.185877    2565 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/574f2e00-d361-423b-946b-e1e9ea3a5db5-kube-api-access-s8xpr podName:574f2e00-d361-423b-946b-e1e9ea3a5db5 nodeName:}" failed. No retries permitted until 2026-09-17 17:34:14.685862843 +0000 UTC m=+4.563147258 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-api-access-s8xpr" (UniqueName: "kubernetes.io/projected/574f2e00-d361-423b-946b-e1e9ea3a5db5-kube-api-access-s8xpr") pod "network-check-target-jsc2p" (UID: "574f2e00-d361-423b-946b-e1e9ea3a5db5") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 17 17:34:14.186039 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:34:14.185881    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-xkgxr\" (UniqueName: \"kubernetes.io/projected/2feeabe5-1136-4f30-9a14-491423e838a3-kube-api-access-xkgxr\") pod \"multus-5l5k7\" (UID: \"2feeabe5-1136-4f30-9a14-491423e838a3\") " pod="openshift-multus/multus-5l5k7"
Sep 17 17:34:14.188538 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:34:14.188343    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-nd6qt\" (UniqueName: \"kubernetes.io/projected/30837de7-1447-4de2-ac62-c809813b3338-kube-api-access-nd6qt\") pod \"ovnkube-node-d9927\" (UID: \"30837de7-1447-4de2-ac62-c809813b3338\") " pod="openshift-ovn-kubernetes/ovnkube-node-d9927"
Sep 17 17:34:14.188538 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:34:14.188362    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-p7fds\" (UniqueName: \"kubernetes.io/projected/27b114f9-fcc4-463b-897f-9e4561ad5540-kube-api-access-p7fds\") pod \"node-ca-5blw5\" (UID: \"27b114f9-fcc4-463b-897f-9e4561ad5540\") " pod="openshift-image-registry/node-ca-5blw5"
Sep 17 17:34:14.188538 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:34:14.188464    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-877s8\" (UniqueName: \"kubernetes.io/projected/af1cc6fe-9695-4e2e-9244-6505022f6c50-kube-api-access-877s8\") pod \"iptables-alerter-g8kpq\" (UID: \"af1cc6fe-9695-4e2e-9244-6505022f6c50\") " pod="openshift-network-operator/iptables-alerter-g8kpq"
Sep 17 17:34:14.188858 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:34:14.188811    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-vx4pk\" (UniqueName: \"kubernetes.io/projected/e39c5e1e-113d-42f2-9a00-8a6d14cb6d67-kube-api-access-vx4pk\") pod \"network-metrics-daemon-rmlqx\" (UID: \"e39c5e1e-113d-42f2-9a00-8a6d14cb6d67\") " pod="openshift-multus/network-metrics-daemon-rmlqx"
Sep 17 17:34:14.189491 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:34:14.189473    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-n9xm8\" (UniqueName: \"kubernetes.io/projected/048bee56-5e42-45f0-98a8-213ff62de69e-kube-api-access-n9xm8\") pod \"multus-additional-cni-plugins-msv4b\" (UID: \"048bee56-5e42-45f0-98a8-213ff62de69e\") " pod="openshift-multus/multus-additional-cni-plugins-msv4b"
Sep 17 17:34:14.190055 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:34:14.190037    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-4rxt7\" (UniqueName: \"kubernetes.io/projected/d4013629-eb04-40d7-bdb1-11e1cafa9394-kube-api-access-4rxt7\") pod \"tuned-mf8sz\" (UID: \"d4013629-eb04-40d7-bdb1-11e1cafa9394\") " pod="openshift-cluster-node-tuning-operator/tuned-mf8sz"
Sep 17 17:34:14.190157 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:34:14.190138    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-4vqpv\" (UniqueName: \"kubernetes.io/projected/1958175e-39bb-4dc8-a5fb-5cff5b7e85f5-kube-api-access-4vqpv\") pod \"aws-ebs-csi-driver-node-fsbkf\" (UID: \"1958175e-39bb-4dc8-a5fb-5cff5b7e85f5\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-fsbkf"
Sep 17 17:34:14.322195 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:34:14.322122    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-vs64q"
Sep 17 17:34:14.330532 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:34:14.330508    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-g8kpq"
Sep 17 17:34:14.342044 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:34:14.342007    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-5blw5"
Sep 17 17:34:14.347604 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:34:14.347586    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-d9927"
Sep 17 17:34:14.355282 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:34:14.355266    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-msv4b"
Sep 17 17:34:14.363287 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:34:14.363266    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-fsbkf"
Sep 17 17:34:14.370181 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:34:14.370158    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-mf8sz"
Sep 17 17:34:14.377588 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:34:14.377570    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-p8jd8"
Sep 17 17:34:14.384228 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:34:14.384083    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-5l5k7"
Sep 17 17:34:14.465777 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:34:14.465753    2565 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160"
Sep 17 17:34:14.674389 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:34:14.674319    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/e39c5e1e-113d-42f2-9a00-8a6d14cb6d67-metrics-certs\") pod \"network-metrics-daemon-rmlqx\" (UID: \"e39c5e1e-113d-42f2-9a00-8a6d14cb6d67\") " pod="openshift-multus/network-metrics-daemon-rmlqx"
Sep 17 17:34:14.674513 ip-10-0-137-20 kubenswrapper[2565]: E0917 17:34:14.674417    2565 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 17 17:34:14.674513 ip-10-0-137-20 kubenswrapper[2565]: E0917 17:34:14.674471    2565 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/e39c5e1e-113d-42f2-9a00-8a6d14cb6d67-metrics-certs podName:e39c5e1e-113d-42f2-9a00-8a6d14cb6d67 nodeName:}" failed. No retries permitted until 2026-09-17 17:34:15.674456597 +0000 UTC m=+5.551740991 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/e39c5e1e-113d-42f2-9a00-8a6d14cb6d67-metrics-certs") pod "network-metrics-daemon-rmlqx" (UID: "e39c5e1e-113d-42f2-9a00-8a6d14cb6d67") : object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 17 17:34:14.775567 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:34:14.775537    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-s8xpr\" (UniqueName: \"kubernetes.io/projected/574f2e00-d361-423b-946b-e1e9ea3a5db5-kube-api-access-s8xpr\") pod \"network-check-target-jsc2p\" (UID: \"574f2e00-d361-423b-946b-e1e9ea3a5db5\") " pod="openshift-network-diagnostics/network-check-target-jsc2p"
Sep 17 17:34:14.775729 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:34:14.775649    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/3d1c780d-6713-495e-91fb-54a74668e6cc-original-pull-secret\") pod \"global-pull-secret-syncer-74xd8\" (UID: \"3d1c780d-6713-495e-91fb-54a74668e6cc\") " pod="kube-system/global-pull-secret-syncer-74xd8"
Sep 17 17:34:14.775729 ip-10-0-137-20 kubenswrapper[2565]: E0917 17:34:14.775670    2565 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Sep 17 17:34:14.775729 ip-10-0-137-20 kubenswrapper[2565]: E0917 17:34:14.775691    2565 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Sep 17 17:34:14.775729 ip-10-0-137-20 kubenswrapper[2565]: E0917 17:34:14.775701    2565 projected.go:196] Error preparing data for projected volume kube-api-access-s8xpr for pod openshift-network-diagnostics/network-check-target-jsc2p: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 17 17:34:14.775938 ip-10-0-137-20 kubenswrapper[2565]: E0917 17:34:14.775747    2565 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/574f2e00-d361-423b-946b-e1e9ea3a5db5-kube-api-access-s8xpr podName:574f2e00-d361-423b-946b-e1e9ea3a5db5 nodeName:}" failed. No retries permitted until 2026-09-17 17:34:15.775733436 +0000 UTC m=+5.653017845 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-api-access-s8xpr" (UniqueName: "kubernetes.io/projected/574f2e00-d361-423b-946b-e1e9ea3a5db5-kube-api-access-s8xpr") pod "network-check-target-jsc2p" (UID: "574f2e00-d361-423b-946b-e1e9ea3a5db5") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 17 17:34:14.775938 ip-10-0-137-20 kubenswrapper[2565]: E0917 17:34:14.775789    2565 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Sep 17 17:34:14.775938 ip-10-0-137-20 kubenswrapper[2565]: E0917 17:34:14.775836    2565 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/3d1c780d-6713-495e-91fb-54a74668e6cc-original-pull-secret podName:3d1c780d-6713-495e-91fb-54a74668e6cc nodeName:}" failed. No retries permitted until 2026-09-17 17:34:15.775823694 +0000 UTC m=+5.653108102 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/3d1c780d-6713-495e-91fb-54a74668e6cc-original-pull-secret") pod "global-pull-secret-syncer-74xd8" (UID: "3d1c780d-6713-495e-91fb-54a74668e6cc") : object "kube-system"/"original-pull-secret" not registered
Sep 17 17:34:15.038690 ip-10-0-137-20 kubenswrapper[2565]: W0917 17:34:15.038603    2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1958175e_39bb_4dc8_a5fb_5cff5b7e85f5.slice/crio-d892d444aaa3e592189b380d18a7f261702e41858283871514d75ee1a5db8f2b WatchSource:0}: Error finding container d892d444aaa3e592189b380d18a7f261702e41858283871514d75ee1a5db8f2b: Status 404 returned error can't find the container with id d892d444aaa3e592189b380d18a7f261702e41858283871514d75ee1a5db8f2b
Sep 17 17:34:15.040840 ip-10-0-137-20 kubenswrapper[2565]: W0917 17:34:15.040666    2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2e847ec5_02d2_4210_a6ce_89d926b050f3.slice/crio-bec02130a260ec285fd6ab1e522e407fa189630a88acbc1ed5a4cfdc57bde091 WatchSource:0}: Error finding container bec02130a260ec285fd6ab1e522e407fa189630a88acbc1ed5a4cfdc57bde091: Status 404 returned error can't find the container with id bec02130a260ec285fd6ab1e522e407fa189630a88acbc1ed5a4cfdc57bde091
Sep 17 17:34:15.044384 ip-10-0-137-20 kubenswrapper[2565]: W0917 17:34:15.044352    2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2feeabe5_1136_4f30_9a14_491423e838a3.slice/crio-7b1d1d94184143c46ac2471b5ecce204c690c961e5b0e99b5321f5ef14865fe7 WatchSource:0}: Error finding container 7b1d1d94184143c46ac2471b5ecce204c690c961e5b0e99b5321f5ef14865fe7: Status 404 returned error can't find the container with id 7b1d1d94184143c46ac2471b5ecce204c690c961e5b0e99b5321f5ef14865fe7
Sep 17 17:34:15.045006 ip-10-0-137-20 kubenswrapper[2565]: W0917 17:34:15.044989    2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podcaa60d4e_b46b_4fff_8a66_3a5b4f1a1602.slice/crio-04b3d6891047201079e487e3e9b4beec37e33b347b6fbbde97106349ddebb694 WatchSource:0}: Error finding container 04b3d6891047201079e487e3e9b4beec37e33b347b6fbbde97106349ddebb694: Status 404 returned error can't find the container with id 04b3d6891047201079e487e3e9b4beec37e33b347b6fbbde97106349ddebb694
Sep 17 17:34:15.046599 ip-10-0-137-20 kubenswrapper[2565]: W0917 17:34:15.046503    2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod30837de7_1447_4de2_ac62_c809813b3338.slice/crio-4a484b2a3fdc965243bbfa66d4a52d89152bd28000ff6c5593c4c73d789e1174 WatchSource:0}: Error finding container 4a484b2a3fdc965243bbfa66d4a52d89152bd28000ff6c5593c4c73d789e1174: Status 404 returned error can't find the container with id 4a484b2a3fdc965243bbfa66d4a52d89152bd28000ff6c5593c4c73d789e1174
Sep 17 17:34:15.048565 ip-10-0-137-20 kubenswrapper[2565]: W0917 17:34:15.047888    2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod048bee56_5e42_45f0_98a8_213ff62de69e.slice/crio-5eb173532069c126c155ebdc8ae93cac4d821956a039c2fe05adc480a2c5a135 WatchSource:0}: Error finding container 5eb173532069c126c155ebdc8ae93cac4d821956a039c2fe05adc480a2c5a135: Status 404 returned error can't find the container with id 5eb173532069c126c155ebdc8ae93cac4d821956a039c2fe05adc480a2c5a135
Sep 17 17:34:15.048827 ip-10-0-137-20 kubenswrapper[2565]: W0917 17:34:15.048576    2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podaf1cc6fe_9695_4e2e_9244_6505022f6c50.slice/crio-c4b2883411999b053537fc195cb68266e7a909418cd14b47c139c77073d3c960 WatchSource:0}: Error finding container c4b2883411999b053537fc195cb68266e7a909418cd14b47c139c77073d3c960: Status 404 returned error can't find the container with id c4b2883411999b053537fc195cb68266e7a909418cd14b47c139c77073d3c960
Sep 17 17:34:15.049428 ip-10-0-137-20 kubenswrapper[2565]: W0917 17:34:15.049323    2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd4013629_eb04_40d7_bdb1_11e1cafa9394.slice/crio-a2cc9f29e15860c596932e45a707a0c7efccd42dbcc45f6f3e9d197c2997fe89 WatchSource:0}: Error finding container a2cc9f29e15860c596932e45a707a0c7efccd42dbcc45f6f3e9d197c2997fe89: Status 404 returned error can't find the container with id a2cc9f29e15860c596932e45a707a0c7efccd42dbcc45f6f3e9d197c2997fe89
Sep 17 17:34:15.050238 ip-10-0-137-20 kubenswrapper[2565]: W0917 17:34:15.050178    2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod27b114f9_fcc4_463b_897f_9e4561ad5540.slice/crio-7340afb819a559157bf5681439953002b630dbf6d2611a766f635f42e31c1f74 WatchSource:0}: Error finding container 7340afb819a559157bf5681439953002b630dbf6d2611a766f635f42e31c1f74: Status 404 returned error can't find the container with id 7340afb819a559157bf5681439953002b630dbf6d2611a766f635f42e31c1f74
Sep 17 17:34:15.098895 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:34:15.098850    2565 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-09-16 17:29:14 +0000 UTC" deadline="2028-05-29 17:35:46.761578756 +0000 UTC"
Sep 17 17:34:15.098975 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:34:15.098898    2565 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="14880h1m31.662686209s"
Sep 17 17:34:15.162810 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:34:15.162783    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-g8kpq" event={"ID":"af1cc6fe-9695-4e2e-9244-6505022f6c50","Type":"ContainerStarted","Data":"c4b2883411999b053537fc195cb68266e7a909418cd14b47c139c77073d3c960"}
Sep 17 17:34:15.163912 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:34:15.163889    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-mf8sz" event={"ID":"d4013629-eb04-40d7-bdb1-11e1cafa9394","Type":"ContainerStarted","Data":"a2cc9f29e15860c596932e45a707a0c7efccd42dbcc45f6f3e9d197c2997fe89"}
Sep 17 17:34:15.165039 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:34:15.164989    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-msv4b" event={"ID":"048bee56-5e42-45f0-98a8-213ff62de69e","Type":"ContainerStarted","Data":"5eb173532069c126c155ebdc8ae93cac4d821956a039c2fe05adc480a2c5a135"}
Sep 17 17:34:15.167855 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:34:15.167836    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-d9927" event={"ID":"30837de7-1447-4de2-ac62-c809813b3338","Type":"ContainerStarted","Data":"4a484b2a3fdc965243bbfa66d4a52d89152bd28000ff6c5593c4c73d789e1174"}
Sep 17 17:34:15.168839 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:34:15.168819    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-vs64q" event={"ID":"caa60d4e-b46b-4fff-8a66-3a5b4f1a1602","Type":"ContainerStarted","Data":"04b3d6891047201079e487e3e9b4beec37e33b347b6fbbde97106349ddebb694"}
Sep 17 17:34:15.170287 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:34:15.170267    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-5l5k7" event={"ID":"2feeabe5-1136-4f30-9a14-491423e838a3","Type":"ContainerStarted","Data":"7b1d1d94184143c46ac2471b5ecce204c690c961e5b0e99b5321f5ef14865fe7"}
Sep 17 17:34:15.173997 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:34:15.173979    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-5blw5" event={"ID":"27b114f9-fcc4-463b-897f-9e4561ad5540","Type":"ContainerStarted","Data":"7340afb819a559157bf5681439953002b630dbf6d2611a766f635f42e31c1f74"}
Sep 17 17:34:15.175746 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:34:15.175723    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-p8jd8" event={"ID":"2e847ec5-02d2-4210-a6ce-89d926b050f3","Type":"ContainerStarted","Data":"bec02130a260ec285fd6ab1e522e407fa189630a88acbc1ed5a4cfdc57bde091"}
Sep 17 17:34:15.177415 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:34:15.177397    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-fsbkf" event={"ID":"1958175e-39bb-4dc8-a5fb-5cff5b7e85f5","Type":"ContainerStarted","Data":"d892d444aaa3e592189b380d18a7f261702e41858283871514d75ee1a5db8f2b"}
Sep 17 17:34:15.180241 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:34:15.180219    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-137-20.ec2.internal" event={"ID":"cda455bc0aed3ae7411ab9a4c6df01be","Type":"ContainerStarted","Data":"70d4c8089bee0d94fa59a01186eccac740646e3ccec440032af6af510b95236f"}
Sep 17 17:34:15.204833 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:34:15.204795    2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/kube-apiserver-proxy-ip-10-0-137-20.ec2.internal" podStartSLOduration=2.204782309 podStartE2EDuration="2.204782309s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-17 17:34:13 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-17 17:34:15.204166537 +0000 UTC m=+5.081450951" watchObservedRunningTime="2026-09-17 17:34:15.204782309 +0000 UTC m=+5.082066723"
Sep 17 17:34:15.681195 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:34:15.681059    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/e39c5e1e-113d-42f2-9a00-8a6d14cb6d67-metrics-certs\") pod \"network-metrics-daemon-rmlqx\" (UID: \"e39c5e1e-113d-42f2-9a00-8a6d14cb6d67\") " pod="openshift-multus/network-metrics-daemon-rmlqx"
Sep 17 17:34:15.681350 ip-10-0-137-20 kubenswrapper[2565]: E0917 17:34:15.681196    2565 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 17 17:34:15.681350 ip-10-0-137-20 kubenswrapper[2565]: E0917 17:34:15.681264    2565 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/e39c5e1e-113d-42f2-9a00-8a6d14cb6d67-metrics-certs podName:e39c5e1e-113d-42f2-9a00-8a6d14cb6d67 nodeName:}" failed. No retries permitted until 2026-09-17 17:34:17.68124526 +0000 UTC m=+7.558529668 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/e39c5e1e-113d-42f2-9a00-8a6d14cb6d67-metrics-certs") pod "network-metrics-daemon-rmlqx" (UID: "e39c5e1e-113d-42f2-9a00-8a6d14cb6d67") : object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 17 17:34:15.781502 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:34:15.781464    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/3d1c780d-6713-495e-91fb-54a74668e6cc-original-pull-secret\") pod \"global-pull-secret-syncer-74xd8\" (UID: \"3d1c780d-6713-495e-91fb-54a74668e6cc\") " pod="kube-system/global-pull-secret-syncer-74xd8"
Sep 17 17:34:15.781671 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:34:15.781569    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-s8xpr\" (UniqueName: \"kubernetes.io/projected/574f2e00-d361-423b-946b-e1e9ea3a5db5-kube-api-access-s8xpr\") pod \"network-check-target-jsc2p\" (UID: \"574f2e00-d361-423b-946b-e1e9ea3a5db5\") " pod="openshift-network-diagnostics/network-check-target-jsc2p"
Sep 17 17:34:15.781735 ip-10-0-137-20 kubenswrapper[2565]: E0917 17:34:15.781708    2565 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Sep 17 17:34:15.781735 ip-10-0-137-20 kubenswrapper[2565]: E0917 17:34:15.781726    2565 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Sep 17 17:34:15.781834 ip-10-0-137-20 kubenswrapper[2565]: E0917 17:34:15.781741    2565 projected.go:196] Error preparing data for projected volume kube-api-access-s8xpr for pod openshift-network-diagnostics/network-check-target-jsc2p: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 17 17:34:15.781834 ip-10-0-137-20 kubenswrapper[2565]: E0917 17:34:15.781798    2565 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/574f2e00-d361-423b-946b-e1e9ea3a5db5-kube-api-access-s8xpr podName:574f2e00-d361-423b-946b-e1e9ea3a5db5 nodeName:}" failed. No retries permitted until 2026-09-17 17:34:17.781779322 +0000 UTC m=+7.659063728 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-api-access-s8xpr" (UniqueName: "kubernetes.io/projected/574f2e00-d361-423b-946b-e1e9ea3a5db5-kube-api-access-s8xpr") pod "network-check-target-jsc2p" (UID: "574f2e00-d361-423b-946b-e1e9ea3a5db5") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 17 17:34:15.782221 ip-10-0-137-20 kubenswrapper[2565]: E0917 17:34:15.782200    2565 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Sep 17 17:34:15.782318 ip-10-0-137-20 kubenswrapper[2565]: E0917 17:34:15.782257    2565 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/3d1c780d-6713-495e-91fb-54a74668e6cc-original-pull-secret podName:3d1c780d-6713-495e-91fb-54a74668e6cc nodeName:}" failed. No retries permitted until 2026-09-17 17:34:17.78224309 +0000 UTC m=+7.659527488 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/3d1c780d-6713-495e-91fb-54a74668e6cc-original-pull-secret") pod "global-pull-secret-syncer-74xd8" (UID: "3d1c780d-6713-495e-91fb-54a74668e6cc") : object "kube-system"/"original-pull-secret" not registered
Sep 17 17:34:16.099252 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:34:16.099165    2565 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-09-16 17:29:14 +0000 UTC" deadline="2028-04-17 15:53:56.059020323 +0000 UTC"
Sep 17 17:34:16.099252 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:34:16.099203    2565 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="13870h19m39.959821506s"
Sep 17 17:34:16.160772 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:34:16.157130    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-jsc2p"
Sep 17 17:34:16.160772 ip-10-0-137-20 kubenswrapper[2565]: E0917 17:34:16.157338    2565 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-jsc2p" podUID="574f2e00-d361-423b-946b-e1e9ea3a5db5"
Sep 17 17:34:16.160772 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:34:16.157348    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-74xd8"
Sep 17 17:34:16.160772 ip-10-0-137-20 kubenswrapper[2565]: E0917 17:34:16.157496    2565 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-74xd8" podUID="3d1c780d-6713-495e-91fb-54a74668e6cc"
Sep 17 17:34:16.160772 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:34:16.157527    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-rmlqx"
Sep 17 17:34:16.160772 ip-10-0-137-20 kubenswrapper[2565]: E0917 17:34:16.157659    2565 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-rmlqx" podUID="e39c5e1e-113d-42f2-9a00-8a6d14cb6d67"
Sep 17 17:34:16.201009 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:34:16.200902    2565 generic.go:356] "Generic (PLEG): container finished" podID="aae62a2aa7c494e1d00d19cbc1cec29c" containerID="28f88c158753d52f519cd6eeee56d2054329ed68f538f4a947b61d19cbaaf150" exitCode=0
Sep 17 17:34:16.201845 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:34:16.201709    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-137-20.ec2.internal" event={"ID":"aae62a2aa7c494e1d00d19cbc1cec29c","Type":"ContainerDied","Data":"28f88c158753d52f519cd6eeee56d2054329ed68f538f4a947b61d19cbaaf150"}
Sep 17 17:34:17.212234 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:34:17.212098    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-137-20.ec2.internal" event={"ID":"aae62a2aa7c494e1d00d19cbc1cec29c","Type":"ContainerStarted","Data":"440d5c10c696e930eb65a538245f0144e396121f045a7cbb8122b14e2336ddb3"}
Sep 17 17:34:17.695535 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:34:17.694922    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/e39c5e1e-113d-42f2-9a00-8a6d14cb6d67-metrics-certs\") pod \"network-metrics-daemon-rmlqx\" (UID: \"e39c5e1e-113d-42f2-9a00-8a6d14cb6d67\") " pod="openshift-multus/network-metrics-daemon-rmlqx"
Sep 17 17:34:17.695535 ip-10-0-137-20 kubenswrapper[2565]: E0917 17:34:17.695083    2565 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 17 17:34:17.695535 ip-10-0-137-20 kubenswrapper[2565]: E0917 17:34:17.695147    2565 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/e39c5e1e-113d-42f2-9a00-8a6d14cb6d67-metrics-certs podName:e39c5e1e-113d-42f2-9a00-8a6d14cb6d67 nodeName:}" failed. No retries permitted until 2026-09-17 17:34:21.695129535 +0000 UTC m=+11.572413928 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/e39c5e1e-113d-42f2-9a00-8a6d14cb6d67-metrics-certs") pod "network-metrics-daemon-rmlqx" (UID: "e39c5e1e-113d-42f2-9a00-8a6d14cb6d67") : object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 17 17:34:17.796740 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:34:17.795769    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/3d1c780d-6713-495e-91fb-54a74668e6cc-original-pull-secret\") pod \"global-pull-secret-syncer-74xd8\" (UID: \"3d1c780d-6713-495e-91fb-54a74668e6cc\") " pod="kube-system/global-pull-secret-syncer-74xd8"
Sep 17 17:34:17.796740 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:34:17.795867    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-s8xpr\" (UniqueName: \"kubernetes.io/projected/574f2e00-d361-423b-946b-e1e9ea3a5db5-kube-api-access-s8xpr\") pod \"network-check-target-jsc2p\" (UID: \"574f2e00-d361-423b-946b-e1e9ea3a5db5\") " pod="openshift-network-diagnostics/network-check-target-jsc2p"
Sep 17 17:34:17.796740 ip-10-0-137-20 kubenswrapper[2565]: E0917 17:34:17.796052    2565 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Sep 17 17:34:17.796740 ip-10-0-137-20 kubenswrapper[2565]: E0917 17:34:17.796074    2565 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Sep 17 17:34:17.796740 ip-10-0-137-20 kubenswrapper[2565]: E0917 17:34:17.796086    2565 projected.go:196] Error preparing data for projected volume kube-api-access-s8xpr for pod openshift-network-diagnostics/network-check-target-jsc2p: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 17 17:34:17.796740 ip-10-0-137-20 kubenswrapper[2565]: E0917 17:34:17.796142    2565 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/574f2e00-d361-423b-946b-e1e9ea3a5db5-kube-api-access-s8xpr podName:574f2e00-d361-423b-946b-e1e9ea3a5db5 nodeName:}" failed. No retries permitted until 2026-09-17 17:34:21.796124365 +0000 UTC m=+11.673408761 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-api-access-s8xpr" (UniqueName: "kubernetes.io/projected/574f2e00-d361-423b-946b-e1e9ea3a5db5-kube-api-access-s8xpr") pod "network-check-target-jsc2p" (UID: "574f2e00-d361-423b-946b-e1e9ea3a5db5") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 17 17:34:17.796740 ip-10-0-137-20 kubenswrapper[2565]: E0917 17:34:17.796523    2565 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Sep 17 17:34:17.796740 ip-10-0-137-20 kubenswrapper[2565]: E0917 17:34:17.796569    2565 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/3d1c780d-6713-495e-91fb-54a74668e6cc-original-pull-secret podName:3d1c780d-6713-495e-91fb-54a74668e6cc nodeName:}" failed. No retries permitted until 2026-09-17 17:34:21.796555072 +0000 UTC m=+11.673839480 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/3d1c780d-6713-495e-91fb-54a74668e6cc-original-pull-secret") pod "global-pull-secret-syncer-74xd8" (UID: "3d1c780d-6713-495e-91fb-54a74668e6cc") : object "kube-system"/"original-pull-secret" not registered
Sep 17 17:34:18.150690 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:34:18.150606    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-rmlqx"
Sep 17 17:34:18.150852 ip-10-0-137-20 kubenswrapper[2565]: E0917 17:34:18.150754    2565 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-rmlqx" podUID="e39c5e1e-113d-42f2-9a00-8a6d14cb6d67"
Sep 17 17:34:18.151261 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:34:18.151117    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-jsc2p"
Sep 17 17:34:18.151261 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:34:18.151231    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-74xd8"
Sep 17 17:34:18.151465 ip-10-0-137-20 kubenswrapper[2565]: E0917 17:34:18.151308    2565 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-74xd8" podUID="3d1c780d-6713-495e-91fb-54a74668e6cc"
Sep 17 17:34:18.151465 ip-10-0-137-20 kubenswrapper[2565]: E0917 17:34:18.151227    2565 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-jsc2p" podUID="574f2e00-d361-423b-946b-e1e9ea3a5db5"
Sep 17 17:34:20.151229 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:34:20.151195    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-rmlqx"
Sep 17 17:34:20.151747 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:34:20.151232    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-jsc2p"
Sep 17 17:34:20.151747 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:34:20.151197    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-74xd8"
Sep 17 17:34:20.151747 ip-10-0-137-20 kubenswrapper[2565]: E0917 17:34:20.151344    2565 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-rmlqx" podUID="e39c5e1e-113d-42f2-9a00-8a6d14cb6d67"
Sep 17 17:34:20.151747 ip-10-0-137-20 kubenswrapper[2565]: E0917 17:34:20.151442    2565 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-74xd8" podUID="3d1c780d-6713-495e-91fb-54a74668e6cc"
Sep 17 17:34:20.151747 ip-10-0-137-20 kubenswrapper[2565]: E0917 17:34:20.151525    2565 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-jsc2p" podUID="574f2e00-d361-423b-946b-e1e9ea3a5db5"
Sep 17 17:34:21.730927 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:34:21.730825    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/e39c5e1e-113d-42f2-9a00-8a6d14cb6d67-metrics-certs\") pod \"network-metrics-daemon-rmlqx\" (UID: \"e39c5e1e-113d-42f2-9a00-8a6d14cb6d67\") " pod="openshift-multus/network-metrics-daemon-rmlqx"
Sep 17 17:34:21.731445 ip-10-0-137-20 kubenswrapper[2565]: E0917 17:34:21.731001    2565 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 17 17:34:21.731445 ip-10-0-137-20 kubenswrapper[2565]: E0917 17:34:21.731091    2565 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/e39c5e1e-113d-42f2-9a00-8a6d14cb6d67-metrics-certs podName:e39c5e1e-113d-42f2-9a00-8a6d14cb6d67 nodeName:}" failed. No retries permitted until 2026-09-17 17:34:29.73107071 +0000 UTC m=+19.608355107 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/e39c5e1e-113d-42f2-9a00-8a6d14cb6d67-metrics-certs") pod "network-metrics-daemon-rmlqx" (UID: "e39c5e1e-113d-42f2-9a00-8a6d14cb6d67") : object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 17 17:34:21.831542 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:34:21.831505    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-s8xpr\" (UniqueName: \"kubernetes.io/projected/574f2e00-d361-423b-946b-e1e9ea3a5db5-kube-api-access-s8xpr\") pod \"network-check-target-jsc2p\" (UID: \"574f2e00-d361-423b-946b-e1e9ea3a5db5\") " pod="openshift-network-diagnostics/network-check-target-jsc2p"
Sep 17 17:34:21.831697 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:34:21.831561    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/3d1c780d-6713-495e-91fb-54a74668e6cc-original-pull-secret\") pod \"global-pull-secret-syncer-74xd8\" (UID: \"3d1c780d-6713-495e-91fb-54a74668e6cc\") " pod="kube-system/global-pull-secret-syncer-74xd8"
Sep 17 17:34:21.831752 ip-10-0-137-20 kubenswrapper[2565]: E0917 17:34:21.831707    2565 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Sep 17 17:34:21.831803 ip-10-0-137-20 kubenswrapper[2565]: E0917 17:34:21.831769    2565 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/3d1c780d-6713-495e-91fb-54a74668e6cc-original-pull-secret podName:3d1c780d-6713-495e-91fb-54a74668e6cc nodeName:}" failed. No retries permitted until 2026-09-17 17:34:29.831750626 +0000 UTC m=+19.709035033 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/3d1c780d-6713-495e-91fb-54a74668e6cc-original-pull-secret") pod "global-pull-secret-syncer-74xd8" (UID: "3d1c780d-6713-495e-91fb-54a74668e6cc") : object "kube-system"/"original-pull-secret" not registered
Sep 17 17:34:21.832053 ip-10-0-137-20 kubenswrapper[2565]: E0917 17:34:21.831997    2565 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Sep 17 17:34:21.832053 ip-10-0-137-20 kubenswrapper[2565]: E0917 17:34:21.832038    2565 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Sep 17 17:34:21.832053 ip-10-0-137-20 kubenswrapper[2565]: E0917 17:34:21.832053    2565 projected.go:196] Error preparing data for projected volume kube-api-access-s8xpr for pod openshift-network-diagnostics/network-check-target-jsc2p: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 17 17:34:21.832252 ip-10-0-137-20 kubenswrapper[2565]: E0917 17:34:21.832126    2565 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/574f2e00-d361-423b-946b-e1e9ea3a5db5-kube-api-access-s8xpr podName:574f2e00-d361-423b-946b-e1e9ea3a5db5 nodeName:}" failed. No retries permitted until 2026-09-17 17:34:29.832100376 +0000 UTC m=+19.709384781 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "kube-api-access-s8xpr" (UniqueName: "kubernetes.io/projected/574f2e00-d361-423b-946b-e1e9ea3a5db5-kube-api-access-s8xpr") pod "network-check-target-jsc2p" (UID: "574f2e00-d361-423b-946b-e1e9ea3a5db5") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 17 17:34:22.153260 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:34:22.153185    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-rmlqx"
Sep 17 17:34:22.153260 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:34:22.153228    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-74xd8"
Sep 17 17:34:22.153465 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:34:22.153195    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-jsc2p"
Sep 17 17:34:22.153465 ip-10-0-137-20 kubenswrapper[2565]: E0917 17:34:22.153312    2565 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-rmlqx" podUID="e39c5e1e-113d-42f2-9a00-8a6d14cb6d67"
Sep 17 17:34:22.153465 ip-10-0-137-20 kubenswrapper[2565]: E0917 17:34:22.153435    2565 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-jsc2p" podUID="574f2e00-d361-423b-946b-e1e9ea3a5db5"
Sep 17 17:34:22.153618 ip-10-0-137-20 kubenswrapper[2565]: E0917 17:34:22.153505    2565 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-74xd8" podUID="3d1c780d-6713-495e-91fb-54a74668e6cc"
Sep 17 17:34:24.150836 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:34:24.150803    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-jsc2p"
Sep 17 17:34:24.151322 ip-10-0-137-20 kubenswrapper[2565]: E0917 17:34:24.150923    2565 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-jsc2p" podUID="574f2e00-d361-423b-946b-e1e9ea3a5db5"
Sep 17 17:34:24.151322 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:34:24.150981    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-rmlqx"
Sep 17 17:34:24.151322 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:34:24.151051    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-74xd8"
Sep 17 17:34:24.151322 ip-10-0-137-20 kubenswrapper[2565]: E0917 17:34:24.151157    2565 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-74xd8" podUID="3d1c780d-6713-495e-91fb-54a74668e6cc"
Sep 17 17:34:24.151322 ip-10-0-137-20 kubenswrapper[2565]: E0917 17:34:24.151247    2565 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-rmlqx" podUID="e39c5e1e-113d-42f2-9a00-8a6d14cb6d67"
Sep 17 17:34:26.153612 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:34:26.153575    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-74xd8"
Sep 17 17:34:26.154097 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:34:26.153681    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-jsc2p"
Sep 17 17:34:26.154097 ip-10-0-137-20 kubenswrapper[2565]: E0917 17:34:26.153694    2565 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-74xd8" podUID="3d1c780d-6713-495e-91fb-54a74668e6cc"
Sep 17 17:34:26.154097 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:34:26.153715    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-rmlqx"
Sep 17 17:34:26.154097 ip-10-0-137-20 kubenswrapper[2565]: E0917 17:34:26.153795    2565 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-jsc2p" podUID="574f2e00-d361-423b-946b-e1e9ea3a5db5"
Sep 17 17:34:26.154097 ip-10-0-137-20 kubenswrapper[2565]: E0917 17:34:26.153870    2565 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-rmlqx" podUID="e39c5e1e-113d-42f2-9a00-8a6d14cb6d67"
Sep 17 17:34:28.150415 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:34:28.150390    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-rmlqx"
Sep 17 17:34:28.150753 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:34:28.150425    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-74xd8"
Sep 17 17:34:28.150753 ip-10-0-137-20 kubenswrapper[2565]: E0917 17:34:28.150493    2565 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-rmlqx" podUID="e39c5e1e-113d-42f2-9a00-8a6d14cb6d67"
Sep 17 17:34:28.150753 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:34:28.150556    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-jsc2p"
Sep 17 17:34:28.150753 ip-10-0-137-20 kubenswrapper[2565]: E0917 17:34:28.150675    2565 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-jsc2p" podUID="574f2e00-d361-423b-946b-e1e9ea3a5db5"
Sep 17 17:34:28.150967 ip-10-0-137-20 kubenswrapper[2565]: E0917 17:34:28.150757    2565 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-74xd8" podUID="3d1c780d-6713-495e-91fb-54a74668e6cc"
Sep 17 17:34:29.786487 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:34:29.786445    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/e39c5e1e-113d-42f2-9a00-8a6d14cb6d67-metrics-certs\") pod \"network-metrics-daemon-rmlqx\" (UID: \"e39c5e1e-113d-42f2-9a00-8a6d14cb6d67\") " pod="openshift-multus/network-metrics-daemon-rmlqx"
Sep 17 17:34:29.786966 ip-10-0-137-20 kubenswrapper[2565]: E0917 17:34:29.786632    2565 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 17 17:34:29.786966 ip-10-0-137-20 kubenswrapper[2565]: E0917 17:34:29.786723    2565 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/e39c5e1e-113d-42f2-9a00-8a6d14cb6d67-metrics-certs podName:e39c5e1e-113d-42f2-9a00-8a6d14cb6d67 nodeName:}" failed. No retries permitted until 2026-09-17 17:34:45.786701306 +0000 UTC m=+35.663985713 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/e39c5e1e-113d-42f2-9a00-8a6d14cb6d67-metrics-certs") pod "network-metrics-daemon-rmlqx" (UID: "e39c5e1e-113d-42f2-9a00-8a6d14cb6d67") : object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 17 17:34:29.887815 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:34:29.887773    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/3d1c780d-6713-495e-91fb-54a74668e6cc-original-pull-secret\") pod \"global-pull-secret-syncer-74xd8\" (UID: \"3d1c780d-6713-495e-91fb-54a74668e6cc\") " pod="kube-system/global-pull-secret-syncer-74xd8"
Sep 17 17:34:29.887980 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:34:29.887881    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-s8xpr\" (UniqueName: \"kubernetes.io/projected/574f2e00-d361-423b-946b-e1e9ea3a5db5-kube-api-access-s8xpr\") pod \"network-check-target-jsc2p\" (UID: \"574f2e00-d361-423b-946b-e1e9ea3a5db5\") " pod="openshift-network-diagnostics/network-check-target-jsc2p"
Sep 17 17:34:29.888302 ip-10-0-137-20 kubenswrapper[2565]: E0917 17:34:29.888221    2565 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Sep 17 17:34:29.888419 ip-10-0-137-20 kubenswrapper[2565]: E0917 17:34:29.888329    2565 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/3d1c780d-6713-495e-91fb-54a74668e6cc-original-pull-secret podName:3d1c780d-6713-495e-91fb-54a74668e6cc nodeName:}" failed. No retries permitted until 2026-09-17 17:34:45.888294789 +0000 UTC m=+35.765579182 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/3d1c780d-6713-495e-91fb-54a74668e6cc-original-pull-secret") pod "global-pull-secret-syncer-74xd8" (UID: "3d1c780d-6713-495e-91fb-54a74668e6cc") : object "kube-system"/"original-pull-secret" not registered
Sep 17 17:34:29.888703 ip-10-0-137-20 kubenswrapper[2565]: E0917 17:34:29.888681    2565 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Sep 17 17:34:29.888810 ip-10-0-137-20 kubenswrapper[2565]: E0917 17:34:29.888710    2565 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Sep 17 17:34:29.888810 ip-10-0-137-20 kubenswrapper[2565]: E0917 17:34:29.888725    2565 projected.go:196] Error preparing data for projected volume kube-api-access-s8xpr for pod openshift-network-diagnostics/network-check-target-jsc2p: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 17 17:34:29.888810 ip-10-0-137-20 kubenswrapper[2565]: E0917 17:34:29.888784    2565 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/574f2e00-d361-423b-946b-e1e9ea3a5db5-kube-api-access-s8xpr podName:574f2e00-d361-423b-946b-e1e9ea3a5db5 nodeName:}" failed. No retries permitted until 2026-09-17 17:34:45.888767798 +0000 UTC m=+35.766052191 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "kube-api-access-s8xpr" (UniqueName: "kubernetes.io/projected/574f2e00-d361-423b-946b-e1e9ea3a5db5-kube-api-access-s8xpr") pod "network-check-target-jsc2p" (UID: "574f2e00-d361-423b-946b-e1e9ea3a5db5") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 17 17:34:30.150912 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:34:30.150881    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-jsc2p"
Sep 17 17:34:30.151117 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:34:30.150884    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-rmlqx"
Sep 17 17:34:30.151117 ip-10-0-137-20 kubenswrapper[2565]: E0917 17:34:30.150993    2565 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-jsc2p" podUID="574f2e00-d361-423b-946b-e1e9ea3a5db5"
Sep 17 17:34:30.151117 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:34:30.151037    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-74xd8"
Sep 17 17:34:30.151284 ip-10-0-137-20 kubenswrapper[2565]: E0917 17:34:30.151171    2565 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-rmlqx" podUID="e39c5e1e-113d-42f2-9a00-8a6d14cb6d67"
Sep 17 17:34:30.151284 ip-10-0-137-20 kubenswrapper[2565]: E0917 17:34:30.151236    2565 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-74xd8" podUID="3d1c780d-6713-495e-91fb-54a74668e6cc"
Sep 17 17:34:32.151003 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:34:32.150977    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-rmlqx"
Sep 17 17:34:32.151386 ip-10-0-137-20 kubenswrapper[2565]: E0917 17:34:32.151094    2565 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-rmlqx" podUID="e39c5e1e-113d-42f2-9a00-8a6d14cb6d67"
Sep 17 17:34:32.151386 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:34:32.151160    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-74xd8"
Sep 17 17:34:32.151386 ip-10-0-137-20 kubenswrapper[2565]: E0917 17:34:32.151242    2565 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-74xd8" podUID="3d1c780d-6713-495e-91fb-54a74668e6cc"
Sep 17 17:34:32.151386 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:34:32.151276    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-jsc2p"
Sep 17 17:34:32.151386 ip-10-0-137-20 kubenswrapper[2565]: E0917 17:34:32.151338    2565 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-jsc2p" podUID="574f2e00-d361-423b-946b-e1e9ea3a5db5"
Sep 17 17:34:33.242003 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:34:33.241622    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-5blw5" event={"ID":"27b114f9-fcc4-463b-897f-9e4561ad5540","Type":"ContainerStarted","Data":"db06181c477f7b45c605e0082ac76d21989b0ad5420006d1afc89077cc7ca699"}
Sep 17 17:34:33.243774 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:34:33.243750    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-p8jd8" event={"ID":"2e847ec5-02d2-4210-a6ce-89d926b050f3","Type":"ContainerStarted","Data":"b9a8206130721b6cb3350e4a975ee210832df476b65e68c627980e6dd5e5d126"}
Sep 17 17:34:33.245421 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:34:33.245392    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-fsbkf" event={"ID":"1958175e-39bb-4dc8-a5fb-5cff5b7e85f5","Type":"ContainerStarted","Data":"3cd1ae3ab69ac7da2ad725c10dcc316ebbb841b80db81c3b95087a84211c452a"}
Sep 17 17:34:33.246948 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:34:33.246924    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-mf8sz" event={"ID":"d4013629-eb04-40d7-bdb1-11e1cafa9394","Type":"ContainerStarted","Data":"f452e086bdc60669d9893bc6bcb341b0dbd8a9ac86b141c6d5a5266564c7930b"}
Sep 17 17:34:33.248567 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:34:33.248454    2565 generic.go:356] "Generic (PLEG): container finished" podID="048bee56-5e42-45f0-98a8-213ff62de69e" containerID="e944b849967daf9f54e49b3865227f3a85367746dfbad7446f8e63cf2348d540" exitCode=0
Sep 17 17:34:33.248567 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:34:33.248551    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-msv4b" event={"ID":"048bee56-5e42-45f0-98a8-213ff62de69e","Type":"ContainerDied","Data":"e944b849967daf9f54e49b3865227f3a85367746dfbad7446f8e63cf2348d540"}
Sep 17 17:34:33.251699 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:34:33.251676    2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-d9927_30837de7-1447-4de2-ac62-c809813b3338/ovn-acl-logging/0.log"
Sep 17 17:34:33.252092 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:34:33.252071    2565 generic.go:356] "Generic (PLEG): container finished" podID="30837de7-1447-4de2-ac62-c809813b3338" containerID="c4dfb61c4b2dcc4a406ba7f2573b0afa36d5dbe5919ce133e3abc09657c0a9fa" exitCode=1
Sep 17 17:34:33.252181 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:34:33.252096    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-d9927" event={"ID":"30837de7-1447-4de2-ac62-c809813b3338","Type":"ContainerStarted","Data":"59d0bfaa8d643c746b330eaec041dfd196b543e2a6763628cbc464e969456d25"}
Sep 17 17:34:33.252181 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:34:33.252156    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-d9927" event={"ID":"30837de7-1447-4de2-ac62-c809813b3338","Type":"ContainerStarted","Data":"9e893ec52158727e87305103f6521554c359f27becbc32c5db7348d27429619a"}
Sep 17 17:34:33.252282 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:34:33.252209    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-d9927" event={"ID":"30837de7-1447-4de2-ac62-c809813b3338","Type":"ContainerStarted","Data":"e8114d469f9a1664b8cdbdd3e8f74e91b10426b0c187382ae48a83615c5e22dd"}
Sep 17 17:34:33.252282 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:34:33.252257    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-d9927" event={"ID":"30837de7-1447-4de2-ac62-c809813b3338","Type":"ContainerDied","Data":"c4dfb61c4b2dcc4a406ba7f2573b0afa36d5dbe5919ce133e3abc09657c0a9fa"}
Sep 17 17:34:33.252380 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:34:33.252305    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-d9927" event={"ID":"30837de7-1447-4de2-ac62-c809813b3338","Type":"ContainerStarted","Data":"46d89ee3e69fe2594ee0214fd2a0a716d30afbbf120019fe8367926825468f19"}
Sep 17 17:34:33.254094 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:34:33.254073    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-vs64q" event={"ID":"caa60d4e-b46b-4fff-8a66-3a5b4f1a1602","Type":"ContainerStarted","Data":"b7cce0870b760233ad24f34ae5c79120f45db966b4b35023a908e5fca93fe289"}
Sep 17 17:34:33.256088 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:34:33.256061    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-5l5k7" event={"ID":"2feeabe5-1136-4f30-9a14-491423e838a3","Type":"ContainerStarted","Data":"6de83bb01df0f3e39f2354ec2244911ef50cc28991caa792733f567af1e8fa8f"}
Sep 17 17:34:33.256320 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:34:33.256255    2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/node-ca-5blw5" podStartSLOduration=3.9170234539999997 podStartE2EDuration="21.256237922s" totalImagesPullingTime="17.339214468s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-17 17:34:12 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-17 17:34:33.255472954 +0000 UTC m=+23.132757369" watchObservedRunningTime="2026-09-17 17:34:33.256237922 +0000 UTC m=+23.133522341"
Sep 17 17:34:33.256434 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:34:33.256348    2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-137-20.ec2.internal" podStartSLOduration=20.256341414 podStartE2EDuration="20.256341414s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-17 17:34:13 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-17 17:34:17.228658954 +0000 UTC m=+7.105943370" watchObservedRunningTime="2026-09-17 17:34:33.256341414 +0000 UTC m=+23.133625830"
Sep 17 17:34:33.358915 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:34:33.358855    2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/node-resolver-vs64q" podStartSLOduration=3.937965771 podStartE2EDuration="21.358833363s" totalImagesPullingTime="17.420867592s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-17 17:34:12 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-17 17:34:33.336838344 +0000 UTC m=+23.214122760" watchObservedRunningTime="2026-09-17 17:34:33.358833363 +0000 UTC m=+23.236117779"
Sep 17 17:34:33.381190 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:34:33.381143    2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-node-tuning-operator/tuned-mf8sz" podStartSLOduration=3.991898869 podStartE2EDuration="21.381123942s" totalImagesPullingTime="17.389225073s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-17 17:34:12 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-17 17:34:33.358348833 +0000 UTC m=+23.235633250" watchObservedRunningTime="2026-09-17 17:34:33.381123942 +0000 UTC m=+23.258408360"
Sep 17 17:34:33.381584 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:34:33.381548    2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/konnectivity-agent-p8jd8" podStartSLOduration=4.030748356 podStartE2EDuration="21.38153834s" totalImagesPullingTime="17.350789984s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-17 17:34:12 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-17 17:34:33.381265581 +0000 UTC m=+23.258549998" watchObservedRunningTime="2026-09-17 17:34:33.38153834 +0000 UTC m=+23.258822757"
Sep 17 17:34:33.403167 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:34:33.403127    2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-5l5k7" podStartSLOduration=3.965873232 podStartE2EDuration="21.40311343s" totalImagesPullingTime="17.437240198s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-17 17:34:12 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-17 17:34:33.402868444 +0000 UTC m=+23.280152859" watchObservedRunningTime="2026-09-17 17:34:33.40311343 +0000 UTC m=+23.280397845"
Sep 17 17:34:33.821619 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:34:33.821594    2565 plugin_watcher.go:194] "Adding socket path or updating timestamp to desired state cache" path="/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock"
Sep 17 17:34:34.125531 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:34:34.125431    2565 reconciler.go:161] "OperationExecutor.RegisterPlugin started" plugin={"SocketPath":"/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock","Timestamp":"2026-09-17T17:34:33.821614861Z","UUID":"40499e07-1347-4d2e-bab6-f89ab2544c6e","Handler":null,"Name":"","Endpoint":""}
Sep 17 17:34:34.127010 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:34:34.126952    2565 csi_plugin.go:106] kubernetes.io/csi: Trying to validate a new CSI Driver with name: ebs.csi.aws.com endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock versions: 1.0.0
Sep 17 17:34:34.127010 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:34:34.126987    2565 csi_plugin.go:119] kubernetes.io/csi: Register new plugin with name: ebs.csi.aws.com at endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock
Sep 17 17:34:34.151261 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:34:34.151234    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-74xd8"
Sep 17 17:34:34.151419 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:34:34.151234    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-rmlqx"
Sep 17 17:34:34.151419 ip-10-0-137-20 kubenswrapper[2565]: E0917 17:34:34.151356    2565 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-74xd8" podUID="3d1c780d-6713-495e-91fb-54a74668e6cc"
Sep 17 17:34:34.151538 ip-10-0-137-20 kubenswrapper[2565]: E0917 17:34:34.151458    2565 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-rmlqx" podUID="e39c5e1e-113d-42f2-9a00-8a6d14cb6d67"
Sep 17 17:34:34.151538 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:34:34.151234    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-jsc2p"
Sep 17 17:34:34.151637 ip-10-0-137-20 kubenswrapper[2565]: E0917 17:34:34.151551    2565 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-jsc2p" podUID="574f2e00-d361-423b-946b-e1e9ea3a5db5"
Sep 17 17:34:34.261722 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:34:34.261679    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-fsbkf" event={"ID":"1958175e-39bb-4dc8-a5fb-5cff5b7e85f5","Type":"ContainerStarted","Data":"38a7cc45f1977c89327cbb60be334d4675c107a3e523e7380502af6061e3dc9f"}
Sep 17 17:34:34.263998 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:34:34.263799    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-g8kpq" event={"ID":"af1cc6fe-9695-4e2e-9244-6505022f6c50","Type":"ContainerStarted","Data":"70ff81850eef466f58e2da502029a042b08c33b834a57955b1a4731341cbaf02"}
Sep 17 17:34:34.267752 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:34:34.267732    2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-d9927_30837de7-1447-4de2-ac62-c809813b3338/ovn-acl-logging/0.log"
Sep 17 17:34:34.268514 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:34:34.268484    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-d9927" event={"ID":"30837de7-1447-4de2-ac62-c809813b3338","Type":"ContainerStarted","Data":"44b46080091293ecf4667796e0aeef7057e8267fc44994ec364debf916b547c9"}
Sep 17 17:34:34.289154 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:34:34.289110    2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-operator/iptables-alerter-g8kpq" podStartSLOduration=4.868391667 podStartE2EDuration="22.289095709s" totalImagesPullingTime="17.420704042s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-17 17:34:12 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-17 17:34:34.289069669 +0000 UTC m=+24.166354085" watchObservedRunningTime="2026-09-17 17:34:34.289095709 +0000 UTC m=+24.166380124"
Sep 17 17:34:35.242601 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:34:35.242520    2565 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="kube-system/konnectivity-agent-p8jd8"
Sep 17 17:34:35.243120 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:34:35.243097    2565 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="kube-system/konnectivity-agent-p8jd8"
Sep 17 17:34:35.273138 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:34:35.273098    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-fsbkf" event={"ID":"1958175e-39bb-4dc8-a5fb-5cff5b7e85f5","Type":"ContainerStarted","Data":"1e067b6bea26ec53c62d2d042c8c3df8ac90fac244697d676ceea52c3290c584"}
Sep 17 17:34:36.150607 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:34:36.150434    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-rmlqx"
Sep 17 17:34:36.150785 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:34:36.150438    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-74xd8"
Sep 17 17:34:36.150785 ip-10-0-137-20 kubenswrapper[2565]: E0917 17:34:36.150724    2565 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-rmlqx" podUID="e39c5e1e-113d-42f2-9a00-8a6d14cb6d67"
Sep 17 17:34:36.150785 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:34:36.150446    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-jsc2p"
Sep 17 17:34:36.150949 ip-10-0-137-20 kubenswrapper[2565]: E0917 17:34:36.150799    2565 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-74xd8" podUID="3d1c780d-6713-495e-91fb-54a74668e6cc"
Sep 17 17:34:36.150949 ip-10-0-137-20 kubenswrapper[2565]: E0917 17:34:36.150854    2565 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-jsc2p" podUID="574f2e00-d361-423b-946b-e1e9ea3a5db5"
Sep 17 17:34:36.278902 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:34:36.278877    2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-d9927_30837de7-1447-4de2-ac62-c809813b3338/ovn-acl-logging/0.log"
Sep 17 17:34:36.279390 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:34:36.279348    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-d9927" event={"ID":"30837de7-1447-4de2-ac62-c809813b3338","Type":"ContainerStarted","Data":"874a930fe07e97e823c25544a6ba7481a29eeef763281766e23a06add53369c5"}
Sep 17 17:34:36.279599 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:34:36.279566    2565 prober_manager.go:312] "Failed to trigger a manual run" probe="Readiness"
Sep 17 17:34:38.150548 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:34:38.150517    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-74xd8"
Sep 17 17:34:38.150548 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:34:38.150535    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-rmlqx"
Sep 17 17:34:38.151499 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:34:38.150532    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-jsc2p"
Sep 17 17:34:38.151499 ip-10-0-137-20 kubenswrapper[2565]: E0917 17:34:38.150618    2565 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-74xd8" podUID="3d1c780d-6713-495e-91fb-54a74668e6cc"
Sep 17 17:34:38.151499 ip-10-0-137-20 kubenswrapper[2565]: E0917 17:34:38.150699    2565 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-rmlqx" podUID="e39c5e1e-113d-42f2-9a00-8a6d14cb6d67"
Sep 17 17:34:38.151499 ip-10-0-137-20 kubenswrapper[2565]: E0917 17:34:38.150755    2565 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-jsc2p" podUID="574f2e00-d361-423b-946b-e1e9ea3a5db5"
Sep 17 17:34:38.285081 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:34:38.285046    2565 generic.go:356] "Generic (PLEG): container finished" podID="048bee56-5e42-45f0-98a8-213ff62de69e" containerID="1d5a9d606866d6b6b5bc02a7138e5e1a871a1ca5ec0a78ab56370e4b90fdcf2f" exitCode=0
Sep 17 17:34:38.285257 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:34:38.285136    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-msv4b" event={"ID":"048bee56-5e42-45f0-98a8-213ff62de69e","Type":"ContainerDied","Data":"1d5a9d606866d6b6b5bc02a7138e5e1a871a1ca5ec0a78ab56370e4b90fdcf2f"}
Sep 17 17:34:38.288790 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:34:38.288774    2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-d9927_30837de7-1447-4de2-ac62-c809813b3338/ovn-acl-logging/0.log"
Sep 17 17:34:38.289127 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:34:38.289107    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-d9927" event={"ID":"30837de7-1447-4de2-ac62-c809813b3338","Type":"ContainerStarted","Data":"77b09456a5adcb7808ddac7136ebbcd764c59d854b772513adae647cf39c5910"}
Sep 17 17:34:38.289396 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:34:38.289380    2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-d9927"
Sep 17 17:34:38.289458 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:34:38.289446    2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-d9927"
Sep 17 17:34:38.289558 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:34:38.289546    2565 scope.go:117] "RemoveContainer" containerID="c4dfb61c4b2dcc4a406ba7f2573b0afa36d5dbe5919ce133e3abc09657c0a9fa"
Sep 17 17:34:38.305949 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:34:38.305932    2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-d9927"
Sep 17 17:34:38.313128 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:34:38.313086    2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-fsbkf" podStartSLOduration=6.852320706 podStartE2EDuration="26.313071733s" totalImagesPullingTime="19.460751027s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-17 17:34:12 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-17 17:34:35.291542733 +0000 UTC m=+25.168827149" watchObservedRunningTime="2026-09-17 17:34:38.313071733 +0000 UTC m=+28.190356167"
Sep 17 17:34:39.295115 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:34:39.295088    2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-d9927_30837de7-1447-4de2-ac62-c809813b3338/ovn-acl-logging/0.log"
Sep 17 17:34:39.295502 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:34:39.295480    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-d9927" event={"ID":"30837de7-1447-4de2-ac62-c809813b3338","Type":"ContainerStarted","Data":"b762cc59fb42e194dc0fdd236ec84d13200bd84db3f5c0f692a90476b4b157d1"}
Sep 17 17:34:39.295720 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:34:39.295702    2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-d9927"
Sep 17 17:34:39.309751 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:34:39.309730    2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-d9927"
Sep 17 17:34:39.331550 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:34:39.331508    2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ovn-kubernetes/ovnkube-node-d9927" podStartSLOduration=9.911546286 podStartE2EDuration="27.331493943s" totalImagesPullingTime="17.419947657s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-17 17:34:12 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-17 17:34:39.329588581 +0000 UTC m=+29.206872996" watchObservedRunningTime="2026-09-17 17:34:39.331493943 +0000 UTC m=+29.208778358"
Sep 17 17:34:39.712897 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:34:39.712722    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-jsc2p"]
Sep 17 17:34:39.713077 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:34:39.713041    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-jsc2p"
Sep 17 17:34:39.713148 ip-10-0-137-20 kubenswrapper[2565]: E0917 17:34:39.713127    2565 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-jsc2p" podUID="574f2e00-d361-423b-946b-e1e9ea3a5db5"
Sep 17 17:34:39.716041 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:34:39.715997    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-rmlqx"]
Sep 17 17:34:39.716167 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:34:39.716150    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-rmlqx"
Sep 17 17:34:39.716275 ip-10-0-137-20 kubenswrapper[2565]: E0917 17:34:39.716257    2565 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-rmlqx" podUID="e39c5e1e-113d-42f2-9a00-8a6d14cb6d67"
Sep 17 17:34:39.716668 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:34:39.716647    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-74xd8"]
Sep 17 17:34:39.716769 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:34:39.716755    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-74xd8"
Sep 17 17:34:39.716877 ip-10-0-137-20 kubenswrapper[2565]: E0917 17:34:39.716856    2565 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-74xd8" podUID="3d1c780d-6713-495e-91fb-54a74668e6cc"
Sep 17 17:34:40.300079 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:34:40.300042    2565 generic.go:356] "Generic (PLEG): container finished" podID="048bee56-5e42-45f0-98a8-213ff62de69e" containerID="9acc93829c156ddbde77026359031d0efa1168f83096bc2581feb7b2a4b52221" exitCode=0
Sep 17 17:34:40.300433 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:34:40.300120    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-msv4b" event={"ID":"048bee56-5e42-45f0-98a8-213ff62de69e","Type":"ContainerDied","Data":"9acc93829c156ddbde77026359031d0efa1168f83096bc2581feb7b2a4b52221"}
Sep 17 17:34:41.150629 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:34:41.150592    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-74xd8"
Sep 17 17:34:41.150629 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:34:41.150636    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-rmlqx"
Sep 17 17:34:41.150808 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:34:41.150693    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-jsc2p"
Sep 17 17:34:41.150808 ip-10-0-137-20 kubenswrapper[2565]: E0917 17:34:41.150711    2565 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-74xd8" podUID="3d1c780d-6713-495e-91fb-54a74668e6cc"
Sep 17 17:34:41.150808 ip-10-0-137-20 kubenswrapper[2565]: E0917 17:34:41.150757    2565 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-jsc2p" podUID="574f2e00-d361-423b-946b-e1e9ea3a5db5"
Sep 17 17:34:41.150917 ip-10-0-137-20 kubenswrapper[2565]: E0917 17:34:41.150832    2565 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-rmlqx" podUID="e39c5e1e-113d-42f2-9a00-8a6d14cb6d67"
Sep 17 17:34:42.307717 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:34:42.307677    2565 generic.go:356] "Generic (PLEG): container finished" podID="048bee56-5e42-45f0-98a8-213ff62de69e" containerID="be7b3db4ed25a5657120bdfbf7edef8617e512db6125efa6264054be314367f2" exitCode=0
Sep 17 17:34:42.308309 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:34:42.307744    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-msv4b" event={"ID":"048bee56-5e42-45f0-98a8-213ff62de69e","Type":"ContainerDied","Data":"be7b3db4ed25a5657120bdfbf7edef8617e512db6125efa6264054be314367f2"}
Sep 17 17:34:42.423188 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:34:42.423151    2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kube-system/konnectivity-agent-p8jd8"
Sep 17 17:34:42.423332 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:34:42.423285    2565 prober_manager.go:312] "Failed to trigger a manual run" probe="Readiness"
Sep 17 17:34:42.423721 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:34:42.423694    2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kube-system/konnectivity-agent-p8jd8"
Sep 17 17:34:43.150968 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:34:43.150939    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-jsc2p"
Sep 17 17:34:43.151138 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:34:43.150978    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-rmlqx"
Sep 17 17:34:43.151138 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:34:43.150977    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-74xd8"
Sep 17 17:34:43.151138 ip-10-0-137-20 kubenswrapper[2565]: E0917 17:34:43.151073    2565 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-jsc2p" podUID="574f2e00-d361-423b-946b-e1e9ea3a5db5"
Sep 17 17:34:43.151260 ip-10-0-137-20 kubenswrapper[2565]: E0917 17:34:43.151149    2565 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-rmlqx" podUID="e39c5e1e-113d-42f2-9a00-8a6d14cb6d67"
Sep 17 17:34:43.151260 ip-10-0-137-20 kubenswrapper[2565]: E0917 17:34:43.151211    2565 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-74xd8" podUID="3d1c780d-6713-495e-91fb-54a74668e6cc"
Sep 17 17:34:45.150864 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:34:45.150819    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-rmlqx"
Sep 17 17:34:45.150864 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:34:45.150846    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-jsc2p"
Sep 17 17:34:45.150864 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:34:45.150857    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-74xd8"
Sep 17 17:34:45.151678 ip-10-0-137-20 kubenswrapper[2565]: E0917 17:34:45.150972    2565 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-rmlqx" podUID="e39c5e1e-113d-42f2-9a00-8a6d14cb6d67"
Sep 17 17:34:45.151678 ip-10-0-137-20 kubenswrapper[2565]: E0917 17:34:45.151051    2565 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-74xd8" podUID="3d1c780d-6713-495e-91fb-54a74668e6cc"
Sep 17 17:34:45.151678 ip-10-0-137-20 kubenswrapper[2565]: E0917 17:34:45.151113    2565 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-jsc2p" podUID="574f2e00-d361-423b-946b-e1e9ea3a5db5"
Sep 17 17:34:45.677746 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:34:45.677711    2565 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-137-20.ec2.internal" event="NodeReady"
Sep 17 17:34:45.677910 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:34:45.677859    2565 kubelet_node_status.go:489] "Fast updating node status as it just became ready"
Sep 17 17:34:45.718418 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:34:45.718386    2565 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-image-registry/image-registry-b79cf788d-x7pgt"]
Sep 17 17:34:45.757618 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:34:45.757586    2565 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress-canary/ingress-canary-cqzrc"]
Sep 17 17:34:45.757791 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:34:45.757710    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-b79cf788d-x7pgt"
Sep 17 17:34:45.760662 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:34:45.760632    2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-private-configuration\""
Sep 17 17:34:45.760662 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:34:45.760651    2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"registry-dockercfg-xqbcm\""
Sep 17 17:34:45.760921 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:34:45.760907    2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"installation-pull-secrets\""
Sep 17 17:34:45.761438 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:34:45.761419    2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-tls\""
Sep 17 17:34:45.768811 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:34:45.768791    2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"trusted-ca\""
Sep 17 17:34:45.773680 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:34:45.773658    2565 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-dns/dns-default-mj76m"]
Sep 17 17:34:45.773797 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:34:45.773779    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-cqzrc"
Sep 17 17:34:45.776510 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:34:45.776489    2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"default-dockercfg-7dqqw\""
Sep 17 17:34:45.776624 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:34:45.776559    2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"kube-root-ca.crt\""
Sep 17 17:34:45.776800 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:34:45.776779    2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"openshift-service-ca.crt\""
Sep 17 17:34:45.777349 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:34:45.777315    2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"canary-serving-cert\""
Sep 17 17:34:45.793585 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:34:45.793562    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-b79cf788d-x7pgt"]
Sep 17 17:34:45.793663 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:34:45.793598    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-cqzrc"]
Sep 17 17:34:45.793663 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:34:45.793615    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-mj76m"]
Sep 17 17:34:45.793746 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:34:45.793684    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-mj76m"
Sep 17 17:34:45.796479 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:34:45.796458    2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-gdbm5\""
Sep 17 17:34:45.796574 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:34:45.796491    2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-default-metrics-tls\""
Sep 17 17:34:45.796574 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:34:45.796468    2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"dns-default\""
Sep 17 17:34:45.809771 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:34:45.809751    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/e39c5e1e-113d-42f2-9a00-8a6d14cb6d67-metrics-certs\") pod \"network-metrics-daemon-rmlqx\" (UID: \"e39c5e1e-113d-42f2-9a00-8a6d14cb6d67\") " pod="openshift-multus/network-metrics-daemon-rmlqx"
Sep 17 17:34:45.809945 ip-10-0-137-20 kubenswrapper[2565]: E0917 17:34:45.809929    2565 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 17 17:34:45.810007 ip-10-0-137-20 kubenswrapper[2565]: E0917 17:34:45.809997    2565 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/e39c5e1e-113d-42f2-9a00-8a6d14cb6d67-metrics-certs podName:e39c5e1e-113d-42f2-9a00-8a6d14cb6d67 nodeName:}" failed. No retries permitted until 2026-09-17 17:35:17.809975967 +0000 UTC m=+67.687260383 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/e39c5e1e-113d-42f2-9a00-8a6d14cb6d67-metrics-certs") pod "network-metrics-daemon-rmlqx" (UID: "e39c5e1e-113d-42f2-9a00-8a6d14cb6d67") : object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 17 17:34:45.911036 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:34:45.910996    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/054b367d-f6ce-4337-ab2b-cab945fc918e-config-volume\") pod \"dns-default-mj76m\" (UID: \"054b367d-f6ce-4337-ab2b-cab945fc918e\") " pod="openshift-dns/dns-default-mj76m"
Sep 17 17:34:45.911192 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:34:45.911061    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/9ab56083-4127-4bbb-af70-4ef55006fe47-ca-trust-extracted\") pod \"image-registry-b79cf788d-x7pgt\" (UID: \"9ab56083-4127-4bbb-af70-4ef55006fe47\") " pod="openshift-image-registry/image-registry-b79cf788d-x7pgt"
Sep 17 17:34:45.911192 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:34:45.911092    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/9ab56083-4127-4bbb-af70-4ef55006fe47-installation-pull-secrets\") pod \"image-registry-b79cf788d-x7pgt\" (UID: \"9ab56083-4127-4bbb-af70-4ef55006fe47\") " pod="openshift-image-registry/image-registry-b79cf788d-x7pgt"
Sep 17 17:34:45.911192 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:34:45.911114    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/054b367d-f6ce-4337-ab2b-cab945fc918e-tmp-dir\") pod \"dns-default-mj76m\" (UID: \"054b367d-f6ce-4337-ab2b-cab945fc918e\") " pod="openshift-dns/dns-default-mj76m"
Sep 17 17:34:45.911192 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:34:45.911145    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-s8xpr\" (UniqueName: \"kubernetes.io/projected/574f2e00-d361-423b-946b-e1e9ea3a5db5-kube-api-access-s8xpr\") pod \"network-check-target-jsc2p\" (UID: \"574f2e00-d361-423b-946b-e1e9ea3a5db5\") " pod="openshift-network-diagnostics/network-check-target-jsc2p"
Sep 17 17:34:45.911192 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:34:45.911170    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7fvvr\" (UniqueName: \"kubernetes.io/projected/f8f9debb-8e49-4e10-b950-bc0e3305c30e-kube-api-access-7fvvr\") pod \"ingress-canary-cqzrc\" (UID: \"f8f9debb-8e49-4e10-b950-bc0e3305c30e\") " pod="openshift-ingress-canary/ingress-canary-cqzrc"
Sep 17 17:34:45.911465 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:34:45.911195    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/9ab56083-4127-4bbb-af70-4ef55006fe47-image-registry-private-configuration\") pod \"image-registry-b79cf788d-x7pgt\" (UID: \"9ab56083-4127-4bbb-af70-4ef55006fe47\") " pod="openshift-image-registry/image-registry-b79cf788d-x7pgt"
Sep 17 17:34:45.911465 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:34:45.911219    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mhswd\" (UniqueName: \"kubernetes.io/projected/9ab56083-4127-4bbb-af70-4ef55006fe47-kube-api-access-mhswd\") pod \"image-registry-b79cf788d-x7pgt\" (UID: \"9ab56083-4127-4bbb-af70-4ef55006fe47\") " pod="openshift-image-registry/image-registry-b79cf788d-x7pgt"
Sep 17 17:34:45.911465 ip-10-0-137-20 kubenswrapper[2565]: E0917 17:34:45.911264    2565 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Sep 17 17:34:45.911465 ip-10-0-137-20 kubenswrapper[2565]: E0917 17:34:45.911289    2565 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Sep 17 17:34:45.911465 ip-10-0-137-20 kubenswrapper[2565]: E0917 17:34:45.911303    2565 projected.go:196] Error preparing data for projected volume kube-api-access-s8xpr for pod openshift-network-diagnostics/network-check-target-jsc2p: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 17 17:34:45.911465 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:34:45.911263    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/3d1c780d-6713-495e-91fb-54a74668e6cc-original-pull-secret\") pod \"global-pull-secret-syncer-74xd8\" (UID: \"3d1c780d-6713-495e-91fb-54a74668e6cc\") " pod="kube-system/global-pull-secret-syncer-74xd8"
Sep 17 17:34:45.911465 ip-10-0-137-20 kubenswrapper[2565]: E0917 17:34:45.911361    2565 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Sep 17 17:34:45.911465 ip-10-0-137-20 kubenswrapper[2565]: E0917 17:34:45.911450    2565 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/574f2e00-d361-423b-946b-e1e9ea3a5db5-kube-api-access-s8xpr podName:574f2e00-d361-423b-946b-e1e9ea3a5db5 nodeName:}" failed. No retries permitted until 2026-09-17 17:35:17.911426186 +0000 UTC m=+67.788710588 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "kube-api-access-s8xpr" (UniqueName: "kubernetes.io/projected/574f2e00-d361-423b-946b-e1e9ea3a5db5-kube-api-access-s8xpr") pod "network-check-target-jsc2p" (UID: "574f2e00-d361-423b-946b-e1e9ea3a5db5") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 17 17:34:45.911751 ip-10-0-137-20 kubenswrapper[2565]: E0917 17:34:45.911495    2565 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/3d1c780d-6713-495e-91fb-54a74668e6cc-original-pull-secret podName:3d1c780d-6713-495e-91fb-54a74668e6cc nodeName:}" failed. No retries permitted until 2026-09-17 17:35:17.911483296 +0000 UTC m=+67.788767689 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/3d1c780d-6713-495e-91fb-54a74668e6cc-original-pull-secret") pod "global-pull-secret-syncer-74xd8" (UID: "3d1c780d-6713-495e-91fb-54a74668e6cc") : object "kube-system"/"original-pull-secret" not registered
Sep 17 17:34:45.911751 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:34:45.911526    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/9ab56083-4127-4bbb-af70-4ef55006fe47-bound-sa-token\") pod \"image-registry-b79cf788d-x7pgt\" (UID: \"9ab56083-4127-4bbb-af70-4ef55006fe47\") " pod="openshift-image-registry/image-registry-b79cf788d-x7pgt"
Sep 17 17:34:45.911751 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:34:45.911548    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/f8f9debb-8e49-4e10-b950-bc0e3305c30e-cert\") pod \"ingress-canary-cqzrc\" (UID: \"f8f9debb-8e49-4e10-b950-bc0e3305c30e\") " pod="openshift-ingress-canary/ingress-canary-cqzrc"
Sep 17 17:34:45.911751 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:34:45.911584    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/9ab56083-4127-4bbb-af70-4ef55006fe47-registry-tls\") pod \"image-registry-b79cf788d-x7pgt\" (UID: \"9ab56083-4127-4bbb-af70-4ef55006fe47\") " pod="openshift-image-registry/image-registry-b79cf788d-x7pgt"
Sep 17 17:34:45.911751 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:34:45.911599    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/9ab56083-4127-4bbb-af70-4ef55006fe47-registry-certificates\") pod \"image-registry-b79cf788d-x7pgt\" (UID: \"9ab56083-4127-4bbb-af70-4ef55006fe47\") " pod="openshift-image-registry/image-registry-b79cf788d-x7pgt"
Sep 17 17:34:45.911751 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:34:45.911621    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/9ab56083-4127-4bbb-af70-4ef55006fe47-trusted-ca\") pod \"image-registry-b79cf788d-x7pgt\" (UID: \"9ab56083-4127-4bbb-af70-4ef55006fe47\") " pod="openshift-image-registry/image-registry-b79cf788d-x7pgt"
Sep 17 17:34:45.911751 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:34:45.911644    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/054b367d-f6ce-4337-ab2b-cab945fc918e-metrics-tls\") pod \"dns-default-mj76m\" (UID: \"054b367d-f6ce-4337-ab2b-cab945fc918e\") " pod="openshift-dns/dns-default-mj76m"
Sep 17 17:34:45.911751 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:34:45.911665    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-nhsng\" (UniqueName: \"kubernetes.io/projected/054b367d-f6ce-4337-ab2b-cab945fc918e-kube-api-access-nhsng\") pod \"dns-default-mj76m\" (UID: \"054b367d-f6ce-4337-ab2b-cab945fc918e\") " pod="openshift-dns/dns-default-mj76m"
Sep 17 17:34:46.012639 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:34:46.012603    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/9ab56083-4127-4bbb-af70-4ef55006fe47-registry-tls\") pod \"image-registry-b79cf788d-x7pgt\" (UID: \"9ab56083-4127-4bbb-af70-4ef55006fe47\") " pod="openshift-image-registry/image-registry-b79cf788d-x7pgt"
Sep 17 17:34:46.012639 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:34:46.012643    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/9ab56083-4127-4bbb-af70-4ef55006fe47-registry-certificates\") pod \"image-registry-b79cf788d-x7pgt\" (UID: \"9ab56083-4127-4bbb-af70-4ef55006fe47\") " pod="openshift-image-registry/image-registry-b79cf788d-x7pgt"
Sep 17 17:34:46.012870 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:34:46.012665    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/9ab56083-4127-4bbb-af70-4ef55006fe47-trusted-ca\") pod \"image-registry-b79cf788d-x7pgt\" (UID: \"9ab56083-4127-4bbb-af70-4ef55006fe47\") " pod="openshift-image-registry/image-registry-b79cf788d-x7pgt"
Sep 17 17:34:46.012870 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:34:46.012685    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/054b367d-f6ce-4337-ab2b-cab945fc918e-metrics-tls\") pod \"dns-default-mj76m\" (UID: \"054b367d-f6ce-4337-ab2b-cab945fc918e\") " pod="openshift-dns/dns-default-mj76m"
Sep 17 17:34:46.012870 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:34:46.012708    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-nhsng\" (UniqueName: \"kubernetes.io/projected/054b367d-f6ce-4337-ab2b-cab945fc918e-kube-api-access-nhsng\") pod \"dns-default-mj76m\" (UID: \"054b367d-f6ce-4337-ab2b-cab945fc918e\") " pod="openshift-dns/dns-default-mj76m"
Sep 17 17:34:46.012870 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:34:46.012736    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/054b367d-f6ce-4337-ab2b-cab945fc918e-config-volume\") pod \"dns-default-mj76m\" (UID: \"054b367d-f6ce-4337-ab2b-cab945fc918e\") " pod="openshift-dns/dns-default-mj76m"
Sep 17 17:34:46.012870 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:34:46.012764    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/9ab56083-4127-4bbb-af70-4ef55006fe47-ca-trust-extracted\") pod \"image-registry-b79cf788d-x7pgt\" (UID: \"9ab56083-4127-4bbb-af70-4ef55006fe47\") " pod="openshift-image-registry/image-registry-b79cf788d-x7pgt"
Sep 17 17:34:46.012870 ip-10-0-137-20 kubenswrapper[2565]: E0917 17:34:46.012782    2565 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Sep 17 17:34:46.012870 ip-10-0-137-20 kubenswrapper[2565]: E0917 17:34:46.012804    2565 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-b79cf788d-x7pgt: secret "image-registry-tls" not found
Sep 17 17:34:46.012870 ip-10-0-137-20 kubenswrapper[2565]: E0917 17:34:46.012869    2565 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/9ab56083-4127-4bbb-af70-4ef55006fe47-registry-tls podName:9ab56083-4127-4bbb-af70-4ef55006fe47 nodeName:}" failed. No retries permitted until 2026-09-17 17:34:46.512847805 +0000 UTC m=+36.390132198 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/9ab56083-4127-4bbb-af70-4ef55006fe47-registry-tls") pod "image-registry-b79cf788d-x7pgt" (UID: "9ab56083-4127-4bbb-af70-4ef55006fe47") : secret "image-registry-tls" not found
Sep 17 17:34:46.013272 ip-10-0-137-20 kubenswrapper[2565]: E0917 17:34:46.013134    2565 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Sep 17 17:34:46.013272 ip-10-0-137-20 kubenswrapper[2565]: E0917 17:34:46.013198    2565 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/054b367d-f6ce-4337-ab2b-cab945fc918e-metrics-tls podName:054b367d-f6ce-4337-ab2b-cab945fc918e nodeName:}" failed. No retries permitted until 2026-09-17 17:34:46.51317166 +0000 UTC m=+36.390456070 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/054b367d-f6ce-4337-ab2b-cab945fc918e-metrics-tls") pod "dns-default-mj76m" (UID: "054b367d-f6ce-4337-ab2b-cab945fc918e") : secret "dns-default-metrics-tls" not found
Sep 17 17:34:46.013272 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:34:46.013222    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/9ab56083-4127-4bbb-af70-4ef55006fe47-installation-pull-secrets\") pod \"image-registry-b79cf788d-x7pgt\" (UID: \"9ab56083-4127-4bbb-af70-4ef55006fe47\") " pod="openshift-image-registry/image-registry-b79cf788d-x7pgt"
Sep 17 17:34:46.013272 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:34:46.013252    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/054b367d-f6ce-4337-ab2b-cab945fc918e-tmp-dir\") pod \"dns-default-mj76m\" (UID: \"054b367d-f6ce-4337-ab2b-cab945fc918e\") " pod="openshift-dns/dns-default-mj76m"
Sep 17 17:34:46.013272 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:34:46.013257    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/9ab56083-4127-4bbb-af70-4ef55006fe47-ca-trust-extracted\") pod \"image-registry-b79cf788d-x7pgt\" (UID: \"9ab56083-4127-4bbb-af70-4ef55006fe47\") " pod="openshift-image-registry/image-registry-b79cf788d-x7pgt"
Sep 17 17:34:46.013520 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:34:46.013294    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7fvvr\" (UniqueName: \"kubernetes.io/projected/f8f9debb-8e49-4e10-b950-bc0e3305c30e-kube-api-access-7fvvr\") pod \"ingress-canary-cqzrc\" (UID: \"f8f9debb-8e49-4e10-b950-bc0e3305c30e\") " pod="openshift-ingress-canary/ingress-canary-cqzrc"
Sep 17 17:34:46.013520 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:34:46.013323    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/9ab56083-4127-4bbb-af70-4ef55006fe47-image-registry-private-configuration\") pod \"image-registry-b79cf788d-x7pgt\" (UID: \"9ab56083-4127-4bbb-af70-4ef55006fe47\") " pod="openshift-image-registry/image-registry-b79cf788d-x7pgt"
Sep 17 17:34:46.013520 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:34:46.013351    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mhswd\" (UniqueName: \"kubernetes.io/projected/9ab56083-4127-4bbb-af70-4ef55006fe47-kube-api-access-mhswd\") pod \"image-registry-b79cf788d-x7pgt\" (UID: \"9ab56083-4127-4bbb-af70-4ef55006fe47\") " pod="openshift-image-registry/image-registry-b79cf788d-x7pgt"
Sep 17 17:34:46.013520 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:34:46.013398    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/054b367d-f6ce-4337-ab2b-cab945fc918e-config-volume\") pod \"dns-default-mj76m\" (UID: \"054b367d-f6ce-4337-ab2b-cab945fc918e\") " pod="openshift-dns/dns-default-mj76m"
Sep 17 17:34:46.013520 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:34:46.013409    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/9ab56083-4127-4bbb-af70-4ef55006fe47-bound-sa-token\") pod \"image-registry-b79cf788d-x7pgt\" (UID: \"9ab56083-4127-4bbb-af70-4ef55006fe47\") " pod="openshift-image-registry/image-registry-b79cf788d-x7pgt"
Sep 17 17:34:46.013520 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:34:46.013409    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/9ab56083-4127-4bbb-af70-4ef55006fe47-registry-certificates\") pod \"image-registry-b79cf788d-x7pgt\" (UID: \"9ab56083-4127-4bbb-af70-4ef55006fe47\") " pod="openshift-image-registry/image-registry-b79cf788d-x7pgt"
Sep 17 17:34:46.013520 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:34:46.013434    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/f8f9debb-8e49-4e10-b950-bc0e3305c30e-cert\") pod \"ingress-canary-cqzrc\" (UID: \"f8f9debb-8e49-4e10-b950-bc0e3305c30e\") " pod="openshift-ingress-canary/ingress-canary-cqzrc"
Sep 17 17:34:46.013520 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:34:46.013512    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/054b367d-f6ce-4337-ab2b-cab945fc918e-tmp-dir\") pod \"dns-default-mj76m\" (UID: \"054b367d-f6ce-4337-ab2b-cab945fc918e\") " pod="openshift-dns/dns-default-mj76m"
Sep 17 17:34:46.013937 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:34:46.013776    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/9ab56083-4127-4bbb-af70-4ef55006fe47-trusted-ca\") pod \"image-registry-b79cf788d-x7pgt\" (UID: \"9ab56083-4127-4bbb-af70-4ef55006fe47\") " pod="openshift-image-registry/image-registry-b79cf788d-x7pgt"
Sep 17 17:34:46.017642 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:34:46.017502    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/9ab56083-4127-4bbb-af70-4ef55006fe47-image-registry-private-configuration\") pod \"image-registry-b79cf788d-x7pgt\" (UID: \"9ab56083-4127-4bbb-af70-4ef55006fe47\") " pod="openshift-image-registry/image-registry-b79cf788d-x7pgt"
Sep 17 17:34:46.017753 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:34:46.017510    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/9ab56083-4127-4bbb-af70-4ef55006fe47-installation-pull-secrets\") pod \"image-registry-b79cf788d-x7pgt\" (UID: \"9ab56083-4127-4bbb-af70-4ef55006fe47\") " pod="openshift-image-registry/image-registry-b79cf788d-x7pgt"
Sep 17 17:34:46.017753 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:34:46.017521    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/f8f9debb-8e49-4e10-b950-bc0e3305c30e-cert\") pod \"ingress-canary-cqzrc\" (UID: \"f8f9debb-8e49-4e10-b950-bc0e3305c30e\") " pod="openshift-ingress-canary/ingress-canary-cqzrc"
Sep 17 17:34:46.027845 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:34:46.027824    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-mhswd\" (UniqueName: \"kubernetes.io/projected/9ab56083-4127-4bbb-af70-4ef55006fe47-kube-api-access-mhswd\") pod \"image-registry-b79cf788d-x7pgt\" (UID: \"9ab56083-4127-4bbb-af70-4ef55006fe47\") " pod="openshift-image-registry/image-registry-b79cf788d-x7pgt"
Sep 17 17:34:46.028472 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:34:46.028450    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-7fvvr\" (UniqueName: \"kubernetes.io/projected/f8f9debb-8e49-4e10-b950-bc0e3305c30e-kube-api-access-7fvvr\") pod \"ingress-canary-cqzrc\" (UID: \"f8f9debb-8e49-4e10-b950-bc0e3305c30e\") " pod="openshift-ingress-canary/ingress-canary-cqzrc"
Sep 17 17:34:46.029850 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:34:46.029829    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/9ab56083-4127-4bbb-af70-4ef55006fe47-bound-sa-token\") pod \"image-registry-b79cf788d-x7pgt\" (UID: \"9ab56083-4127-4bbb-af70-4ef55006fe47\") " pod="openshift-image-registry/image-registry-b79cf788d-x7pgt"
Sep 17 17:34:46.030192 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:34:46.030167    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-nhsng\" (UniqueName: \"kubernetes.io/projected/054b367d-f6ce-4337-ab2b-cab945fc918e-kube-api-access-nhsng\") pod \"dns-default-mj76m\" (UID: \"054b367d-f6ce-4337-ab2b-cab945fc918e\") " pod="openshift-dns/dns-default-mj76m"
Sep 17 17:34:46.085952 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:34:46.085919    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-cqzrc"
Sep 17 17:34:46.517098 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:34:46.517067    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/9ab56083-4127-4bbb-af70-4ef55006fe47-registry-tls\") pod \"image-registry-b79cf788d-x7pgt\" (UID: \"9ab56083-4127-4bbb-af70-4ef55006fe47\") " pod="openshift-image-registry/image-registry-b79cf788d-x7pgt"
Sep 17 17:34:46.517098 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:34:46.517100    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/054b367d-f6ce-4337-ab2b-cab945fc918e-metrics-tls\") pod \"dns-default-mj76m\" (UID: \"054b367d-f6ce-4337-ab2b-cab945fc918e\") " pod="openshift-dns/dns-default-mj76m"
Sep 17 17:34:46.517540 ip-10-0-137-20 kubenswrapper[2565]: E0917 17:34:46.517230    2565 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Sep 17 17:34:46.517540 ip-10-0-137-20 kubenswrapper[2565]: E0917 17:34:46.517246    2565 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Sep 17 17:34:46.517540 ip-10-0-137-20 kubenswrapper[2565]: E0917 17:34:46.517268    2565 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-b79cf788d-x7pgt: secret "image-registry-tls" not found
Sep 17 17:34:46.517540 ip-10-0-137-20 kubenswrapper[2565]: E0917 17:34:46.517284    2565 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/054b367d-f6ce-4337-ab2b-cab945fc918e-metrics-tls podName:054b367d-f6ce-4337-ab2b-cab945fc918e nodeName:}" failed. No retries permitted until 2026-09-17 17:34:47.517271564 +0000 UTC m=+37.394555957 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/054b367d-f6ce-4337-ab2b-cab945fc918e-metrics-tls") pod "dns-default-mj76m" (UID: "054b367d-f6ce-4337-ab2b-cab945fc918e") : secret "dns-default-metrics-tls" not found
Sep 17 17:34:46.517540 ip-10-0-137-20 kubenswrapper[2565]: E0917 17:34:46.517321    2565 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/9ab56083-4127-4bbb-af70-4ef55006fe47-registry-tls podName:9ab56083-4127-4bbb-af70-4ef55006fe47 nodeName:}" failed. No retries permitted until 2026-09-17 17:34:47.517303557 +0000 UTC m=+37.394587972 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/9ab56083-4127-4bbb-af70-4ef55006fe47-registry-tls") pod "image-registry-b79cf788d-x7pgt" (UID: "9ab56083-4127-4bbb-af70-4ef55006fe47") : secret "image-registry-tls" not found
Sep 17 17:34:46.796494 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:34:46.796468    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-cqzrc"]
Sep 17 17:34:46.800724 ip-10-0-137-20 kubenswrapper[2565]: W0917 17:34:46.800695    2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf8f9debb_8e49_4e10_b950_bc0e3305c30e.slice/crio-3353b89f2c49242bb6f41ade7ed3345bed343671cb84efb210eaae713e7e7bbf WatchSource:0}: Error finding container 3353b89f2c49242bb6f41ade7ed3345bed343671cb84efb210eaae713e7e7bbf: Status 404 returned error can't find the container with id 3353b89f2c49242bb6f41ade7ed3345bed343671cb84efb210eaae713e7e7bbf
Sep 17 17:34:47.150961 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:34:47.150937    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-74xd8"
Sep 17 17:34:47.151105 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:34:47.150937    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-rmlqx"
Sep 17 17:34:47.151154 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:34:47.150937    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-jsc2p"
Sep 17 17:34:47.154112 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:34:47.154093    2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\""
Sep 17 17:34:47.155121 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:34:47.155102    2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\""
Sep 17 17:34:47.155235 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:34:47.155142    2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\""
Sep 17 17:34:47.155235 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:34:47.155179    2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-hwnj6\""
Sep 17 17:34:47.155353 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:34:47.155327    2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\""
Sep 17 17:34:47.155445 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:34:47.155426    2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-thgr8\""
Sep 17 17:34:47.325728 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:34:47.325692    2565 generic.go:356] "Generic (PLEG): container finished" podID="048bee56-5e42-45f0-98a8-213ff62de69e" containerID="b0929455735a9b1b37ed857ec2a0b30ebccae7c622d00326df90e50495a0fb09" exitCode=0
Sep 17 17:34:47.325907 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:34:47.325783    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-msv4b" event={"ID":"048bee56-5e42-45f0-98a8-213ff62de69e","Type":"ContainerDied","Data":"b0929455735a9b1b37ed857ec2a0b30ebccae7c622d00326df90e50495a0fb09"}
Sep 17 17:34:47.327540 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:34:47.327518    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-cqzrc" event={"ID":"f8f9debb-8e49-4e10-b950-bc0e3305c30e","Type":"ContainerStarted","Data":"3353b89f2c49242bb6f41ade7ed3345bed343671cb84efb210eaae713e7e7bbf"}
Sep 17 17:34:47.524391 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:34:47.524351    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/9ab56083-4127-4bbb-af70-4ef55006fe47-registry-tls\") pod \"image-registry-b79cf788d-x7pgt\" (UID: \"9ab56083-4127-4bbb-af70-4ef55006fe47\") " pod="openshift-image-registry/image-registry-b79cf788d-x7pgt"
Sep 17 17:34:47.524831 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:34:47.524400    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/054b367d-f6ce-4337-ab2b-cab945fc918e-metrics-tls\") pod \"dns-default-mj76m\" (UID: \"054b367d-f6ce-4337-ab2b-cab945fc918e\") " pod="openshift-dns/dns-default-mj76m"
Sep 17 17:34:47.524831 ip-10-0-137-20 kubenswrapper[2565]: E0917 17:34:47.524521    2565 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Sep 17 17:34:47.524831 ip-10-0-137-20 kubenswrapper[2565]: E0917 17:34:47.524593    2565 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-b79cf788d-x7pgt: secret "image-registry-tls" not found
Sep 17 17:34:47.524831 ip-10-0-137-20 kubenswrapper[2565]: E0917 17:34:47.524564    2565 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Sep 17 17:34:47.524831 ip-10-0-137-20 kubenswrapper[2565]: E0917 17:34:47.524657    2565 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/054b367d-f6ce-4337-ab2b-cab945fc918e-metrics-tls podName:054b367d-f6ce-4337-ab2b-cab945fc918e nodeName:}" failed. No retries permitted until 2026-09-17 17:34:49.524641499 +0000 UTC m=+39.401925891 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/054b367d-f6ce-4337-ab2b-cab945fc918e-metrics-tls") pod "dns-default-mj76m" (UID: "054b367d-f6ce-4337-ab2b-cab945fc918e") : secret "dns-default-metrics-tls" not found
Sep 17 17:34:47.524831 ip-10-0-137-20 kubenswrapper[2565]: E0917 17:34:47.524670    2565 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/9ab56083-4127-4bbb-af70-4ef55006fe47-registry-tls podName:9ab56083-4127-4bbb-af70-4ef55006fe47 nodeName:}" failed. No retries permitted until 2026-09-17 17:34:49.524664136 +0000 UTC m=+39.401948528 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/9ab56083-4127-4bbb-af70-4ef55006fe47-registry-tls") pod "image-registry-b79cf788d-x7pgt" (UID: "9ab56083-4127-4bbb-af70-4ef55006fe47") : secret "image-registry-tls" not found
Sep 17 17:34:48.333423 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:34:48.333390    2565 generic.go:356] "Generic (PLEG): container finished" podID="048bee56-5e42-45f0-98a8-213ff62de69e" containerID="350884af776aa8e0ee6e60ed06eb462122c18a7b24457c096f81cf35b46ffaf8" exitCode=0
Sep 17 17:34:48.333595 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:34:48.333463    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-msv4b" event={"ID":"048bee56-5e42-45f0-98a8-213ff62de69e","Type":"ContainerDied","Data":"350884af776aa8e0ee6e60ed06eb462122c18a7b24457c096f81cf35b46ffaf8"}
Sep 17 17:34:49.338933 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:34:49.338757    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-msv4b" event={"ID":"048bee56-5e42-45f0-98a8-213ff62de69e","Type":"ContainerStarted","Data":"35e0b53c8dc3ba8bed64e17f1a131720edb04ffed13684e216561d1245abeb13"}
Sep 17 17:34:49.340149 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:34:49.340127    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-cqzrc" event={"ID":"f8f9debb-8e49-4e10-b950-bc0e3305c30e","Type":"ContainerStarted","Data":"6fff1e7d599171ecf0cfc277180b0f4c7ca7bdc5c13cca53e6822762f3930d0f"}
Sep 17 17:34:49.366440 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:34:49.366397    2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-additional-cni-plugins-msv4b" podStartSLOduration=9.228628987 podStartE2EDuration="37.366383276s" totalImagesPullingTime="28.137754289s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-17 17:34:12 +0000 UTC" imagePullSessionsCount=5 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-17 17:34:49.364995686 +0000 UTC m=+39.242280113" watchObservedRunningTime="2026-09-17 17:34:49.366383276 +0000 UTC m=+39.243667691"
Sep 17 17:34:49.384546 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:34:49.384501    2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress-canary/ingress-canary-cqzrc" podStartSLOduration=2.373025618 podStartE2EDuration="4.38448999s" totalImagesPullingTime="2.011464372s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-17 17:34:45 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-17 17:34:49.384251886 +0000 UTC m=+39.261536300" watchObservedRunningTime="2026-09-17 17:34:49.38448999 +0000 UTC m=+39.261774405"
Sep 17 17:34:49.541476 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:34:49.541406    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/9ab56083-4127-4bbb-af70-4ef55006fe47-registry-tls\") pod \"image-registry-b79cf788d-x7pgt\" (UID: \"9ab56083-4127-4bbb-af70-4ef55006fe47\") " pod="openshift-image-registry/image-registry-b79cf788d-x7pgt"
Sep 17 17:34:49.541476 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:34:49.541441    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/054b367d-f6ce-4337-ab2b-cab945fc918e-metrics-tls\") pod \"dns-default-mj76m\" (UID: \"054b367d-f6ce-4337-ab2b-cab945fc918e\") " pod="openshift-dns/dns-default-mj76m"
Sep 17 17:34:49.541691 ip-10-0-137-20 kubenswrapper[2565]: E0917 17:34:49.541530    2565 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Sep 17 17:34:49.541691 ip-10-0-137-20 kubenswrapper[2565]: E0917 17:34:49.541548    2565 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-b79cf788d-x7pgt: secret "image-registry-tls" not found
Sep 17 17:34:49.541691 ip-10-0-137-20 kubenswrapper[2565]: E0917 17:34:49.541599    2565 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/9ab56083-4127-4bbb-af70-4ef55006fe47-registry-tls podName:9ab56083-4127-4bbb-af70-4ef55006fe47 nodeName:}" failed. No retries permitted until 2026-09-17 17:34:53.541582664 +0000 UTC m=+43.418867058 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/9ab56083-4127-4bbb-af70-4ef55006fe47-registry-tls") pod "image-registry-b79cf788d-x7pgt" (UID: "9ab56083-4127-4bbb-af70-4ef55006fe47") : secret "image-registry-tls" not found
Sep 17 17:34:49.541691 ip-10-0-137-20 kubenswrapper[2565]: E0917 17:34:49.541606    2565 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Sep 17 17:34:49.541691 ip-10-0-137-20 kubenswrapper[2565]: E0917 17:34:49.541652    2565 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/054b367d-f6ce-4337-ab2b-cab945fc918e-metrics-tls podName:054b367d-f6ce-4337-ab2b-cab945fc918e nodeName:}" failed. No retries permitted until 2026-09-17 17:34:53.541638253 +0000 UTC m=+43.418922662 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/054b367d-f6ce-4337-ab2b-cab945fc918e-metrics-tls") pod "dns-default-mj76m" (UID: "054b367d-f6ce-4337-ab2b-cab945fc918e") : secret "dns-default-metrics-tls" not found
Sep 17 17:34:53.569752 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:34:53.569718    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/9ab56083-4127-4bbb-af70-4ef55006fe47-registry-tls\") pod \"image-registry-b79cf788d-x7pgt\" (UID: \"9ab56083-4127-4bbb-af70-4ef55006fe47\") " pod="openshift-image-registry/image-registry-b79cf788d-x7pgt"
Sep 17 17:34:53.569752 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:34:53.569751    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/054b367d-f6ce-4337-ab2b-cab945fc918e-metrics-tls\") pod \"dns-default-mj76m\" (UID: \"054b367d-f6ce-4337-ab2b-cab945fc918e\") " pod="openshift-dns/dns-default-mj76m"
Sep 17 17:34:53.570242 ip-10-0-137-20 kubenswrapper[2565]: E0917 17:34:53.569833    2565 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Sep 17 17:34:53.570242 ip-10-0-137-20 kubenswrapper[2565]: E0917 17:34:53.569849    2565 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Sep 17 17:34:53.570242 ip-10-0-137-20 kubenswrapper[2565]: E0917 17:34:53.569867    2565 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-b79cf788d-x7pgt: secret "image-registry-tls" not found
Sep 17 17:34:53.570242 ip-10-0-137-20 kubenswrapper[2565]: E0917 17:34:53.569877    2565 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/054b367d-f6ce-4337-ab2b-cab945fc918e-metrics-tls podName:054b367d-f6ce-4337-ab2b-cab945fc918e nodeName:}" failed. No retries permitted until 2026-09-17 17:35:01.569865835 +0000 UTC m=+51.447150227 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/054b367d-f6ce-4337-ab2b-cab945fc918e-metrics-tls") pod "dns-default-mj76m" (UID: "054b367d-f6ce-4337-ab2b-cab945fc918e") : secret "dns-default-metrics-tls" not found
Sep 17 17:34:53.570242 ip-10-0-137-20 kubenswrapper[2565]: E0917 17:34:53.569909    2565 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/9ab56083-4127-4bbb-af70-4ef55006fe47-registry-tls podName:9ab56083-4127-4bbb-af70-4ef55006fe47 nodeName:}" failed. No retries permitted until 2026-09-17 17:35:01.569897137 +0000 UTC m=+51.447181544 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/9ab56083-4127-4bbb-af70-4ef55006fe47-registry-tls") pod "image-registry-b79cf788d-x7pgt" (UID: "9ab56083-4127-4bbb-af70-4ef55006fe47") : secret "image-registry-tls" not found
Sep 17 17:35:01.627575 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:35:01.627538    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/9ab56083-4127-4bbb-af70-4ef55006fe47-registry-tls\") pod \"image-registry-b79cf788d-x7pgt\" (UID: \"9ab56083-4127-4bbb-af70-4ef55006fe47\") " pod="openshift-image-registry/image-registry-b79cf788d-x7pgt"
Sep 17 17:35:01.627575 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:35:01.627577    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/054b367d-f6ce-4337-ab2b-cab945fc918e-metrics-tls\") pod \"dns-default-mj76m\" (UID: \"054b367d-f6ce-4337-ab2b-cab945fc918e\") " pod="openshift-dns/dns-default-mj76m"
Sep 17 17:35:01.628105 ip-10-0-137-20 kubenswrapper[2565]: E0917 17:35:01.627670    2565 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Sep 17 17:35:01.628105 ip-10-0-137-20 kubenswrapper[2565]: E0917 17:35:01.627718    2565 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Sep 17 17:35:01.628105 ip-10-0-137-20 kubenswrapper[2565]: E0917 17:35:01.627742    2565 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-b79cf788d-x7pgt: secret "image-registry-tls" not found
Sep 17 17:35:01.628105 ip-10-0-137-20 kubenswrapper[2565]: E0917 17:35:01.627724    2565 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/054b367d-f6ce-4337-ab2b-cab945fc918e-metrics-tls podName:054b367d-f6ce-4337-ab2b-cab945fc918e nodeName:}" failed. No retries permitted until 2026-09-17 17:35:17.627709124 +0000 UTC m=+67.504993517 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/054b367d-f6ce-4337-ab2b-cab945fc918e-metrics-tls") pod "dns-default-mj76m" (UID: "054b367d-f6ce-4337-ab2b-cab945fc918e") : secret "dns-default-metrics-tls" not found
Sep 17 17:35:01.628105 ip-10-0-137-20 kubenswrapper[2565]: E0917 17:35:01.627815    2565 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/9ab56083-4127-4bbb-af70-4ef55006fe47-registry-tls podName:9ab56083-4127-4bbb-af70-4ef55006fe47 nodeName:}" failed. No retries permitted until 2026-09-17 17:35:17.627791167 +0000 UTC m=+67.505075571 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/9ab56083-4127-4bbb-af70-4ef55006fe47-registry-tls") pod "image-registry-b79cf788d-x7pgt" (UID: "9ab56083-4127-4bbb-af70-4ef55006fe47") : secret "image-registry-tls" not found
Sep 17 17:35:02.505623 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:35:02.505589    2565 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-insights/insights-runtime-extractor-6mqqs"]
Sep 17 17:35:02.540611 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:35:02.540574    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-6mqqs"]
Sep 17 17:35:02.540784 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:35:02.540684    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-6mqqs"
Sep 17 17:35:02.544280 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:35:02.544256    2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-tls\""
Sep 17 17:35:02.544582 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:35:02.544566    2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"openshift-service-ca.crt\""
Sep 17 17:35:02.544863 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:35:02.544847    2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-rbac-proxy\""
Sep 17 17:35:02.545349 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:35:02.545336    2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-sa-dockercfg-2xjm4\""
Sep 17 17:35:02.545401 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:35:02.545334    2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-root-ca.crt\""
Sep 17 17:35:02.635197 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:35:02.635172    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2d2sm\" (UniqueName: \"kubernetes.io/projected/fd16e5c0-6ecb-473a-84c0-bbee28f44e68-kube-api-access-2d2sm\") pod \"insights-runtime-extractor-6mqqs\" (UID: \"fd16e5c0-6ecb-473a-84c0-bbee28f44e68\") " pod="openshift-insights/insights-runtime-extractor-6mqqs"
Sep 17 17:35:02.635547 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:35:02.635217    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/fd16e5c0-6ecb-473a-84c0-bbee28f44e68-data-volume\") pod \"insights-runtime-extractor-6mqqs\" (UID: \"fd16e5c0-6ecb-473a-84c0-bbee28f44e68\") " pod="openshift-insights/insights-runtime-extractor-6mqqs"
Sep 17 17:35:02.635547 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:35:02.635312    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/fd16e5c0-6ecb-473a-84c0-bbee28f44e68-crio-socket\") pod \"insights-runtime-extractor-6mqqs\" (UID: \"fd16e5c0-6ecb-473a-84c0-bbee28f44e68\") " pod="openshift-insights/insights-runtime-extractor-6mqqs"
Sep 17 17:35:02.635547 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:35:02.635348    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/fd16e5c0-6ecb-473a-84c0-bbee28f44e68-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-6mqqs\" (UID: \"fd16e5c0-6ecb-473a-84c0-bbee28f44e68\") " pod="openshift-insights/insights-runtime-extractor-6mqqs"
Sep 17 17:35:02.635547 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:35:02.635385    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/fd16e5c0-6ecb-473a-84c0-bbee28f44e68-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-6mqqs\" (UID: \"fd16e5c0-6ecb-473a-84c0-bbee28f44e68\") " pod="openshift-insights/insights-runtime-extractor-6mqqs"
Sep 17 17:35:02.736449 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:35:02.736425    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/fd16e5c0-6ecb-473a-84c0-bbee28f44e68-data-volume\") pod \"insights-runtime-extractor-6mqqs\" (UID: \"fd16e5c0-6ecb-473a-84c0-bbee28f44e68\") " pod="openshift-insights/insights-runtime-extractor-6mqqs"
Sep 17 17:35:02.736598 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:35:02.736488    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/fd16e5c0-6ecb-473a-84c0-bbee28f44e68-crio-socket\") pod \"insights-runtime-extractor-6mqqs\" (UID: \"fd16e5c0-6ecb-473a-84c0-bbee28f44e68\") " pod="openshift-insights/insights-runtime-extractor-6mqqs"
Sep 17 17:35:02.736598 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:35:02.736516    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/fd16e5c0-6ecb-473a-84c0-bbee28f44e68-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-6mqqs\" (UID: \"fd16e5c0-6ecb-473a-84c0-bbee28f44e68\") " pod="openshift-insights/insights-runtime-extractor-6mqqs"
Sep 17 17:35:02.736598 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:35:02.736541    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/fd16e5c0-6ecb-473a-84c0-bbee28f44e68-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-6mqqs\" (UID: \"fd16e5c0-6ecb-473a-84c0-bbee28f44e68\") " pod="openshift-insights/insights-runtime-extractor-6mqqs"
Sep 17 17:35:02.736748 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:35:02.736626    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2d2sm\" (UniqueName: \"kubernetes.io/projected/fd16e5c0-6ecb-473a-84c0-bbee28f44e68-kube-api-access-2d2sm\") pod \"insights-runtime-extractor-6mqqs\" (UID: \"fd16e5c0-6ecb-473a-84c0-bbee28f44e68\") " pod="openshift-insights/insights-runtime-extractor-6mqqs"
Sep 17 17:35:02.736748 ip-10-0-137-20 kubenswrapper[2565]: E0917 17:35:02.736663    2565 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Sep 17 17:35:02.736748 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:35:02.736702    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/fd16e5c0-6ecb-473a-84c0-bbee28f44e68-data-volume\") pod \"insights-runtime-extractor-6mqqs\" (UID: \"fd16e5c0-6ecb-473a-84c0-bbee28f44e68\") " pod="openshift-insights/insights-runtime-extractor-6mqqs"
Sep 17 17:35:02.736748 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:35:02.736732    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/fd16e5c0-6ecb-473a-84c0-bbee28f44e68-crio-socket\") pod \"insights-runtime-extractor-6mqqs\" (UID: \"fd16e5c0-6ecb-473a-84c0-bbee28f44e68\") " pod="openshift-insights/insights-runtime-extractor-6mqqs"
Sep 17 17:35:02.736900 ip-10-0-137-20 kubenswrapper[2565]: E0917 17:35:02.736740    2565 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/fd16e5c0-6ecb-473a-84c0-bbee28f44e68-insights-runtime-extractor-tls podName:fd16e5c0-6ecb-473a-84c0-bbee28f44e68 nodeName:}" failed. No retries permitted until 2026-09-17 17:35:03.236715364 +0000 UTC m=+53.113999785 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/fd16e5c0-6ecb-473a-84c0-bbee28f44e68-insights-runtime-extractor-tls") pod "insights-runtime-extractor-6mqqs" (UID: "fd16e5c0-6ecb-473a-84c0-bbee28f44e68") : secret "insights-runtime-extractor-tls" not found
Sep 17 17:35:02.736900 ip-10-0-137-20 kubenswrapper[2565]: E0917 17:35:02.736765    2565 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Sep 17 17:35:02.736900 ip-10-0-137-20 kubenswrapper[2565]: E0917 17:35:02.736887    2565 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/fd16e5c0-6ecb-473a-84c0-bbee28f44e68-kube-rbac-proxy-cm podName:fd16e5c0-6ecb-473a-84c0-bbee28f44e68 nodeName:}" failed. No retries permitted until 2026-09-17 17:35:03.236875166 +0000 UTC m=+53.114159564 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/fd16e5c0-6ecb-473a-84c0-bbee28f44e68-kube-rbac-proxy-cm") pod "insights-runtime-extractor-6mqqs" (UID: "fd16e5c0-6ecb-473a-84c0-bbee28f44e68") : configmap "kube-rbac-proxy" not found
Sep 17 17:35:02.748170 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:35:02.748150    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-2d2sm\" (UniqueName: \"kubernetes.io/projected/fd16e5c0-6ecb-473a-84c0-bbee28f44e68-kube-api-access-2d2sm\") pod \"insights-runtime-extractor-6mqqs\" (UID: \"fd16e5c0-6ecb-473a-84c0-bbee28f44e68\") " pod="openshift-insights/insights-runtime-extractor-6mqqs"
Sep 17 17:35:03.240380 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:35:03.240347    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/fd16e5c0-6ecb-473a-84c0-bbee28f44e68-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-6mqqs\" (UID: \"fd16e5c0-6ecb-473a-84c0-bbee28f44e68\") " pod="openshift-insights/insights-runtime-extractor-6mqqs"
Sep 17 17:35:03.240514 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:35:03.240384    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/fd16e5c0-6ecb-473a-84c0-bbee28f44e68-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-6mqqs\" (UID: \"fd16e5c0-6ecb-473a-84c0-bbee28f44e68\") " pod="openshift-insights/insights-runtime-extractor-6mqqs"
Sep 17 17:35:03.240514 ip-10-0-137-20 kubenswrapper[2565]: E0917 17:35:03.240457    2565 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Sep 17 17:35:03.240514 ip-10-0-137-20 kubenswrapper[2565]: E0917 17:35:03.240499    2565 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Sep 17 17:35:03.240651 ip-10-0-137-20 kubenswrapper[2565]: E0917 17:35:03.240510    2565 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/fd16e5c0-6ecb-473a-84c0-bbee28f44e68-kube-rbac-proxy-cm podName:fd16e5c0-6ecb-473a-84c0-bbee28f44e68 nodeName:}" failed. No retries permitted until 2026-09-17 17:35:04.24049865 +0000 UTC m=+54.117783043 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/fd16e5c0-6ecb-473a-84c0-bbee28f44e68-kube-rbac-proxy-cm") pod "insights-runtime-extractor-6mqqs" (UID: "fd16e5c0-6ecb-473a-84c0-bbee28f44e68") : configmap "kube-rbac-proxy" not found
Sep 17 17:35:03.240651 ip-10-0-137-20 kubenswrapper[2565]: E0917 17:35:03.240571    2565 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/fd16e5c0-6ecb-473a-84c0-bbee28f44e68-insights-runtime-extractor-tls podName:fd16e5c0-6ecb-473a-84c0-bbee28f44e68 nodeName:}" failed. No retries permitted until 2026-09-17 17:35:04.240553182 +0000 UTC m=+54.117837584 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/fd16e5c0-6ecb-473a-84c0-bbee28f44e68-insights-runtime-extractor-tls") pod "insights-runtime-extractor-6mqqs" (UID: "fd16e5c0-6ecb-473a-84c0-bbee28f44e68") : secret "insights-runtime-extractor-tls" not found
Sep 17 17:35:04.248853 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:35:04.248820    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/fd16e5c0-6ecb-473a-84c0-bbee28f44e68-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-6mqqs\" (UID: \"fd16e5c0-6ecb-473a-84c0-bbee28f44e68\") " pod="openshift-insights/insights-runtime-extractor-6mqqs"
Sep 17 17:35:04.248853 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:35:04.248852    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/fd16e5c0-6ecb-473a-84c0-bbee28f44e68-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-6mqqs\" (UID: \"fd16e5c0-6ecb-473a-84c0-bbee28f44e68\") " pod="openshift-insights/insights-runtime-extractor-6mqqs"
Sep 17 17:35:04.249287 ip-10-0-137-20 kubenswrapper[2565]: E0917 17:35:04.248933    2565 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Sep 17 17:35:04.249287 ip-10-0-137-20 kubenswrapper[2565]: E0917 17:35:04.248958    2565 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Sep 17 17:35:04.249287 ip-10-0-137-20 kubenswrapper[2565]: E0917 17:35:04.248986    2565 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/fd16e5c0-6ecb-473a-84c0-bbee28f44e68-kube-rbac-proxy-cm podName:fd16e5c0-6ecb-473a-84c0-bbee28f44e68 nodeName:}" failed. No retries permitted until 2026-09-17 17:35:06.248970063 +0000 UTC m=+56.126254474 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/fd16e5c0-6ecb-473a-84c0-bbee28f44e68-kube-rbac-proxy-cm") pod "insights-runtime-extractor-6mqqs" (UID: "fd16e5c0-6ecb-473a-84c0-bbee28f44e68") : configmap "kube-rbac-proxy" not found
Sep 17 17:35:04.249287 ip-10-0-137-20 kubenswrapper[2565]: E0917 17:35:04.249004    2565 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/fd16e5c0-6ecb-473a-84c0-bbee28f44e68-insights-runtime-extractor-tls podName:fd16e5c0-6ecb-473a-84c0-bbee28f44e68 nodeName:}" failed. No retries permitted until 2026-09-17 17:35:06.24899256 +0000 UTC m=+56.126276953 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/fd16e5c0-6ecb-473a-84c0-bbee28f44e68-insights-runtime-extractor-tls") pod "insights-runtime-extractor-6mqqs" (UID: "fd16e5c0-6ecb-473a-84c0-bbee28f44e68") : secret "insights-runtime-extractor-tls" not found
Sep 17 17:35:05.226537 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:35:05.226505    2565 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-service-ca/service-ca-55f6fc7b86-2g76t"]
Sep 17 17:35:05.256940 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:35:05.256913    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-service-ca/service-ca-55f6fc7b86-2g76t"]
Sep 17 17:35:05.257311 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:35:05.257005    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-service-ca/service-ca-55f6fc7b86-2g76t"
Sep 17 17:35:05.260186 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:35:05.260168    2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-service-ca\"/\"service-ca-dockercfg-bjr6g\""
Sep 17 17:35:05.260415 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:35:05.260399    2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca\"/\"kube-root-ca.crt\""
Sep 17 17:35:05.261316 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:35:05.261301    2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca\"/\"signing-cabundle\""
Sep 17 17:35:05.261387 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:35:05.261321    2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca\"/\"openshift-service-ca.crt\""
Sep 17 17:35:05.261387 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:35:05.261342    2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-service-ca\"/\"signing-key\""
Sep 17 17:35:05.356162 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:35:05.356138    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2sszf\" (UniqueName: \"kubernetes.io/projected/5cb94a8e-04d3-4cdc-9eb8-ec82a3c14c36-kube-api-access-2sszf\") pod \"service-ca-55f6fc7b86-2g76t\" (UID: \"5cb94a8e-04d3-4cdc-9eb8-ec82a3c14c36\") " pod="openshift-service-ca/service-ca-55f6fc7b86-2g76t"
Sep 17 17:35:05.356251 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:35:05.356184    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"signing-key\" (UniqueName: \"kubernetes.io/secret/5cb94a8e-04d3-4cdc-9eb8-ec82a3c14c36-signing-key\") pod \"service-ca-55f6fc7b86-2g76t\" (UID: \"5cb94a8e-04d3-4cdc-9eb8-ec82a3c14c36\") " pod="openshift-service-ca/service-ca-55f6fc7b86-2g76t"
Sep 17 17:35:05.356251 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:35:05.356200    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"signing-cabundle\" (UniqueName: \"kubernetes.io/configmap/5cb94a8e-04d3-4cdc-9eb8-ec82a3c14c36-signing-cabundle\") pod \"service-ca-55f6fc7b86-2g76t\" (UID: \"5cb94a8e-04d3-4cdc-9eb8-ec82a3c14c36\") " pod="openshift-service-ca/service-ca-55f6fc7b86-2g76t"
Sep 17 17:35:05.449095 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:35:05.449075    2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-vs64q_caa60d4e-b46b-4fff-8a66-3a5b4f1a1602/dns-node-resolver/0.log"
Sep 17 17:35:05.457272 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:35:05.457253    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2sszf\" (UniqueName: \"kubernetes.io/projected/5cb94a8e-04d3-4cdc-9eb8-ec82a3c14c36-kube-api-access-2sszf\") pod \"service-ca-55f6fc7b86-2g76t\" (UID: \"5cb94a8e-04d3-4cdc-9eb8-ec82a3c14c36\") " pod="openshift-service-ca/service-ca-55f6fc7b86-2g76t"
Sep 17 17:35:05.457335 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:35:05.457296    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"signing-key\" (UniqueName: \"kubernetes.io/secret/5cb94a8e-04d3-4cdc-9eb8-ec82a3c14c36-signing-key\") pod \"service-ca-55f6fc7b86-2g76t\" (UID: \"5cb94a8e-04d3-4cdc-9eb8-ec82a3c14c36\") " pod="openshift-service-ca/service-ca-55f6fc7b86-2g76t"
Sep 17 17:35:05.457335 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:35:05.457313    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"signing-cabundle\" (UniqueName: \"kubernetes.io/configmap/5cb94a8e-04d3-4cdc-9eb8-ec82a3c14c36-signing-cabundle\") pod \"service-ca-55f6fc7b86-2g76t\" (UID: \"5cb94a8e-04d3-4cdc-9eb8-ec82a3c14c36\") " pod="openshift-service-ca/service-ca-55f6fc7b86-2g76t"
Sep 17 17:35:05.457932 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:35:05.457916    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"signing-cabundle\" (UniqueName: \"kubernetes.io/configmap/5cb94a8e-04d3-4cdc-9eb8-ec82a3c14c36-signing-cabundle\") pod \"service-ca-55f6fc7b86-2g76t\" (UID: \"5cb94a8e-04d3-4cdc-9eb8-ec82a3c14c36\") " pod="openshift-service-ca/service-ca-55f6fc7b86-2g76t"
Sep 17 17:35:05.459535 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:35:05.459518    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"signing-key\" (UniqueName: \"kubernetes.io/secret/5cb94a8e-04d3-4cdc-9eb8-ec82a3c14c36-signing-key\") pod \"service-ca-55f6fc7b86-2g76t\" (UID: \"5cb94a8e-04d3-4cdc-9eb8-ec82a3c14c36\") " pod="openshift-service-ca/service-ca-55f6fc7b86-2g76t"
Sep 17 17:35:05.466540 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:35:05.466522    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-2sszf\" (UniqueName: \"kubernetes.io/projected/5cb94a8e-04d3-4cdc-9eb8-ec82a3c14c36-kube-api-access-2sszf\") pod \"service-ca-55f6fc7b86-2g76t\" (UID: \"5cb94a8e-04d3-4cdc-9eb8-ec82a3c14c36\") " pod="openshift-service-ca/service-ca-55f6fc7b86-2g76t"
Sep 17 17:35:05.565665 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:35:05.565612    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-service-ca/service-ca-55f6fc7b86-2g76t"
Sep 17 17:35:05.860161 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:35:05.860134    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-service-ca/service-ca-55f6fc7b86-2g76t"]
Sep 17 17:35:05.863564 ip-10-0-137-20 kubenswrapper[2565]: W0917 17:35:05.863530    2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5cb94a8e_04d3_4cdc_9eb8_ec82a3c14c36.slice/crio-41eef9a03d9b821e804af2f7677a2ca00efd4b710a19ab570196997893bc73f2 WatchSource:0}: Error finding container 41eef9a03d9b821e804af2f7677a2ca00efd4b710a19ab570196997893bc73f2: Status 404 returned error can't find the container with id 41eef9a03d9b821e804af2f7677a2ca00efd4b710a19ab570196997893bc73f2
Sep 17 17:35:06.054113 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:35:06.054090    2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-5blw5_27b114f9-fcc4-463b-897f-9e4561ad5540/node-ca/0.log"
Sep 17 17:35:06.264713 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:35:06.264644    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/fd16e5c0-6ecb-473a-84c0-bbee28f44e68-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-6mqqs\" (UID: \"fd16e5c0-6ecb-473a-84c0-bbee28f44e68\") " pod="openshift-insights/insights-runtime-extractor-6mqqs"
Sep 17 17:35:06.265057 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:35:06.264746    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/fd16e5c0-6ecb-473a-84c0-bbee28f44e68-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-6mqqs\" (UID: \"fd16e5c0-6ecb-473a-84c0-bbee28f44e68\") " pod="openshift-insights/insights-runtime-extractor-6mqqs"
Sep 17 17:35:06.265057 ip-10-0-137-20 kubenswrapper[2565]: E0917 17:35:06.264765    2565 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Sep 17 17:35:06.265057 ip-10-0-137-20 kubenswrapper[2565]: E0917 17:35:06.264827    2565 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/fd16e5c0-6ecb-473a-84c0-bbee28f44e68-kube-rbac-proxy-cm podName:fd16e5c0-6ecb-473a-84c0-bbee28f44e68 nodeName:}" failed. No retries permitted until 2026-09-17 17:35:10.264811345 +0000 UTC m=+60.142095737 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/fd16e5c0-6ecb-473a-84c0-bbee28f44e68-kube-rbac-proxy-cm") pod "insights-runtime-extractor-6mqqs" (UID: "fd16e5c0-6ecb-473a-84c0-bbee28f44e68") : configmap "kube-rbac-proxy" not found
Sep 17 17:35:06.265057 ip-10-0-137-20 kubenswrapper[2565]: E0917 17:35:06.264836    2565 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Sep 17 17:35:06.265057 ip-10-0-137-20 kubenswrapper[2565]: E0917 17:35:06.264887    2565 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/fd16e5c0-6ecb-473a-84c0-bbee28f44e68-insights-runtime-extractor-tls podName:fd16e5c0-6ecb-473a-84c0-bbee28f44e68 nodeName:}" failed. No retries permitted until 2026-09-17 17:35:10.264871951 +0000 UTC m=+60.142156365 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/fd16e5c0-6ecb-473a-84c0-bbee28f44e68-insights-runtime-extractor-tls") pod "insights-runtime-extractor-6mqqs" (UID: "fd16e5c0-6ecb-473a-84c0-bbee28f44e68") : secret "insights-runtime-extractor-tls" not found
Sep 17 17:35:06.381014 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:35:06.380990    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca/service-ca-55f6fc7b86-2g76t" event={"ID":"5cb94a8e-04d3-4cdc-9eb8-ec82a3c14c36","Type":"ContainerStarted","Data":"41eef9a03d9b821e804af2f7677a2ca00efd4b710a19ab570196997893bc73f2"}
Sep 17 17:35:07.051659 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:35:07.051631    2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-cqzrc_f8f9debb-8e49-4e10-b950-bc0e3305c30e/serve-healthcheck-canary/0.log"
Sep 17 17:35:08.387344 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:35:08.387310    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca/service-ca-55f6fc7b86-2g76t" event={"ID":"5cb94a8e-04d3-4cdc-9eb8-ec82a3c14c36","Type":"ContainerStarted","Data":"6968a7b5566146e3a8425f87cca78cbc06f634e1cb30a9374f1b3eb741dd8a51"}
Sep 17 17:35:08.405896 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:35:08.405855    2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-service-ca/service-ca-55f6fc7b86-2g76t" podStartSLOduration=1.194517748 podStartE2EDuration="3.405840662s" totalImagesPullingTime="2.211322914s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-17 17:35:05 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-17 17:35:08.404299324 +0000 UTC m=+58.281583741" watchObservedRunningTime="2026-09-17 17:35:08.405840662 +0000 UTC m=+58.283125077"
Sep 17 17:35:10.294701 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:35:10.294662    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/fd16e5c0-6ecb-473a-84c0-bbee28f44e68-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-6mqqs\" (UID: \"fd16e5c0-6ecb-473a-84c0-bbee28f44e68\") " pod="openshift-insights/insights-runtime-extractor-6mqqs"
Sep 17 17:35:10.294701 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:35:10.294703    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/fd16e5c0-6ecb-473a-84c0-bbee28f44e68-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-6mqqs\" (UID: \"fd16e5c0-6ecb-473a-84c0-bbee28f44e68\") " pod="openshift-insights/insights-runtime-extractor-6mqqs"
Sep 17 17:35:10.295171 ip-10-0-137-20 kubenswrapper[2565]: E0917 17:35:10.294804    2565 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Sep 17 17:35:10.295171 ip-10-0-137-20 kubenswrapper[2565]: E0917 17:35:10.294856    2565 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Sep 17 17:35:10.295171 ip-10-0-137-20 kubenswrapper[2565]: E0917 17:35:10.294868    2565 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/fd16e5c0-6ecb-473a-84c0-bbee28f44e68-insights-runtime-extractor-tls podName:fd16e5c0-6ecb-473a-84c0-bbee28f44e68 nodeName:}" failed. No retries permitted until 2026-09-17 17:35:18.294851101 +0000 UTC m=+68.172135499 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/fd16e5c0-6ecb-473a-84c0-bbee28f44e68-insights-runtime-extractor-tls") pod "insights-runtime-extractor-6mqqs" (UID: "fd16e5c0-6ecb-473a-84c0-bbee28f44e68") : secret "insights-runtime-extractor-tls" not found
Sep 17 17:35:10.295171 ip-10-0-137-20 kubenswrapper[2565]: E0917 17:35:10.294908    2565 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/fd16e5c0-6ecb-473a-84c0-bbee28f44e68-kube-rbac-proxy-cm podName:fd16e5c0-6ecb-473a-84c0-bbee28f44e68 nodeName:}" failed. No retries permitted until 2026-09-17 17:35:18.294891683 +0000 UTC m=+68.172176092 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/fd16e5c0-6ecb-473a-84c0-bbee28f44e68-kube-rbac-proxy-cm") pod "insights-runtime-extractor-6mqqs" (UID: "fd16e5c0-6ecb-473a-84c0-bbee28f44e68") : configmap "kube-rbac-proxy" not found
Sep 17 17:35:11.312924 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:35:11.312899    2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-d9927"
Sep 17 17:35:17.651836 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:35:17.651794    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/9ab56083-4127-4bbb-af70-4ef55006fe47-registry-tls\") pod \"image-registry-b79cf788d-x7pgt\" (UID: \"9ab56083-4127-4bbb-af70-4ef55006fe47\") " pod="openshift-image-registry/image-registry-b79cf788d-x7pgt"
Sep 17 17:35:17.651836 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:35:17.651836    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/054b367d-f6ce-4337-ab2b-cab945fc918e-metrics-tls\") pod \"dns-default-mj76m\" (UID: \"054b367d-f6ce-4337-ab2b-cab945fc918e\") " pod="openshift-dns/dns-default-mj76m"
Sep 17 17:35:17.654157 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:35:17.654127    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/054b367d-f6ce-4337-ab2b-cab945fc918e-metrics-tls\") pod \"dns-default-mj76m\" (UID: \"054b367d-f6ce-4337-ab2b-cab945fc918e\") " pod="openshift-dns/dns-default-mj76m"
Sep 17 17:35:17.654265 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:35:17.654237    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/9ab56083-4127-4bbb-af70-4ef55006fe47-registry-tls\") pod \"image-registry-b79cf788d-x7pgt\" (UID: \"9ab56083-4127-4bbb-af70-4ef55006fe47\") " pod="openshift-image-registry/image-registry-b79cf788d-x7pgt"
Sep 17 17:35:17.853600 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:35:17.853563    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/e39c5e1e-113d-42f2-9a00-8a6d14cb6d67-metrics-certs\") pod \"network-metrics-daemon-rmlqx\" (UID: \"e39c5e1e-113d-42f2-9a00-8a6d14cb6d67\") " pod="openshift-multus/network-metrics-daemon-rmlqx"
Sep 17 17:35:17.857149 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:35:17.857130    2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\""
Sep 17 17:35:17.866967 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:35:17.866934    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/e39c5e1e-113d-42f2-9a00-8a6d14cb6d67-metrics-certs\") pod \"network-metrics-daemon-rmlqx\" (UID: \"e39c5e1e-113d-42f2-9a00-8a6d14cb6d67\") " pod="openshift-multus/network-metrics-daemon-rmlqx"
Sep 17 17:35:17.871859 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:35:17.871835    2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"registry-dockercfg-xqbcm\""
Sep 17 17:35:17.879739 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:35:17.879724    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-b79cf788d-x7pgt"
Sep 17 17:35:17.906229 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:35:17.906175    2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-gdbm5\""
Sep 17 17:35:17.913734 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:35:17.913716    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-mj76m"
Sep 17 17:35:17.954901 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:35:17.954870    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/3d1c780d-6713-495e-91fb-54a74668e6cc-original-pull-secret\") pod \"global-pull-secret-syncer-74xd8\" (UID: \"3d1c780d-6713-495e-91fb-54a74668e6cc\") " pod="kube-system/global-pull-secret-syncer-74xd8"
Sep 17 17:35:17.955076 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:35:17.954947    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-s8xpr\" (UniqueName: \"kubernetes.io/projected/574f2e00-d361-423b-946b-e1e9ea3a5db5-kube-api-access-s8xpr\") pod \"network-check-target-jsc2p\" (UID: \"574f2e00-d361-423b-946b-e1e9ea3a5db5\") " pod="openshift-network-diagnostics/network-check-target-jsc2p"
Sep 17 17:35:17.958258 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:35:17.958231    2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\""
Sep 17 17:35:17.958258 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:35:17.958231    2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\""
Sep 17 17:35:17.968213 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:35:17.968185    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/3d1c780d-6713-495e-91fb-54a74668e6cc-original-pull-secret\") pod \"global-pull-secret-syncer-74xd8\" (UID: \"3d1c780d-6713-495e-91fb-54a74668e6cc\") " pod="kube-system/global-pull-secret-syncer-74xd8"
Sep 17 17:35:17.968846 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:35:17.968513    2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\""
Sep 17 17:35:17.978034 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:35:17.978007    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-s8xpr\" (UniqueName: \"kubernetes.io/projected/574f2e00-d361-423b-946b-e1e9ea3a5db5-kube-api-access-s8xpr\") pod \"network-check-target-jsc2p\" (UID: \"574f2e00-d361-423b-946b-e1e9ea3a5db5\") " pod="openshift-network-diagnostics/network-check-target-jsc2p"
Sep 17 17:35:18.063397 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:35:18.063365    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-74xd8"
Sep 17 17:35:18.073293 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:35:18.073273    2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-hwnj6\""
Sep 17 17:35:18.079627 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:35:18.079608    2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-thgr8\""
Sep 17 17:35:18.080592 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:35:18.080577    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-rmlqx"
Sep 17 17:35:18.087408 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:35:18.087347    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-jsc2p"
Sep 17 17:35:18.164578 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:35:18.164552    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-b79cf788d-x7pgt"]
Sep 17 17:35:18.168482 ip-10-0-137-20 kubenswrapper[2565]: W0917 17:35:18.168462    2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9ab56083_4127_4bbb_af70_4ef55006fe47.slice/crio-db3867a0e90cec74c6bb6c2542e5f7b604a65777be5f50bd0f28eb96e943696f WatchSource:0}: Error finding container db3867a0e90cec74c6bb6c2542e5f7b604a65777be5f50bd0f28eb96e943696f: Status 404 returned error can't find the container with id db3867a0e90cec74c6bb6c2542e5f7b604a65777be5f50bd0f28eb96e943696f
Sep 17 17:35:18.200610 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:35:18.200589    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-mj76m"]
Sep 17 17:35:18.205880 ip-10-0-137-20 kubenswrapper[2565]: W0917 17:35:18.205860    2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod054b367d_f6ce_4337_ab2b_cab945fc918e.slice/crio-fdb6190fc26cb5c5a8dba0df6adff4b686f4f83fa95e72da8d991150490a48bf WatchSource:0}: Error finding container fdb6190fc26cb5c5a8dba0df6adff4b686f4f83fa95e72da8d991150490a48bf: Status 404 returned error can't find the container with id fdb6190fc26cb5c5a8dba0df6adff4b686f4f83fa95e72da8d991150490a48bf
Sep 17 17:35:18.338280 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:35:18.338250    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-74xd8"]
Sep 17 17:35:18.341991 ip-10-0-137-20 kubenswrapper[2565]: W0917 17:35:18.341964    2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3d1c780d_6713_495e_91fb_54a74668e6cc.slice/crio-d244544dbd6cadeee08b93743d232473bf7ee765b58489009ab4a34604a52159 WatchSource:0}: Error finding container d244544dbd6cadeee08b93743d232473bf7ee765b58489009ab4a34604a52159: Status 404 returned error can't find the container with id d244544dbd6cadeee08b93743d232473bf7ee765b58489009ab4a34604a52159
Sep 17 17:35:18.357121 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:35:18.357097    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/fd16e5c0-6ecb-473a-84c0-bbee28f44e68-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-6mqqs\" (UID: \"fd16e5c0-6ecb-473a-84c0-bbee28f44e68\") " pod="openshift-insights/insights-runtime-extractor-6mqqs"
Sep 17 17:35:18.357227 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:35:18.357135    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/fd16e5c0-6ecb-473a-84c0-bbee28f44e68-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-6mqqs\" (UID: \"fd16e5c0-6ecb-473a-84c0-bbee28f44e68\") " pod="openshift-insights/insights-runtime-extractor-6mqqs"
Sep 17 17:35:18.357285 ip-10-0-137-20 kubenswrapper[2565]: E0917 17:35:18.357262    2565 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Sep 17 17:35:18.357372 ip-10-0-137-20 kubenswrapper[2565]: E0917 17:35:18.357335    2565 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/fd16e5c0-6ecb-473a-84c0-bbee28f44e68-insights-runtime-extractor-tls podName:fd16e5c0-6ecb-473a-84c0-bbee28f44e68 nodeName:}" failed. No retries permitted until 2026-09-17 17:35:34.357313592 +0000 UTC m=+84.234598008 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/fd16e5c0-6ecb-473a-84c0-bbee28f44e68-insights-runtime-extractor-tls") pod "insights-runtime-extractor-6mqqs" (UID: "fd16e5c0-6ecb-473a-84c0-bbee28f44e68") : secret "insights-runtime-extractor-tls" not found
Sep 17 17:35:18.357695 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:35:18.357675    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/fd16e5c0-6ecb-473a-84c0-bbee28f44e68-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-6mqqs\" (UID: \"fd16e5c0-6ecb-473a-84c0-bbee28f44e68\") " pod="openshift-insights/insights-runtime-extractor-6mqqs"
Sep 17 17:35:18.362131 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:35:18.362096    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-rmlqx"]
Sep 17 17:35:18.364557 ip-10-0-137-20 kubenswrapper[2565]: W0917 17:35:18.364534    2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode39c5e1e_113d_42f2_9a00_8a6d14cb6d67.slice/crio-f342d72946d6b0cd7dba395e9842fb2084c8b2e3328fe92bc82852db434f768c WatchSource:0}: Error finding container f342d72946d6b0cd7dba395e9842fb2084c8b2e3328fe92bc82852db434f768c: Status 404 returned error can't find the container with id f342d72946d6b0cd7dba395e9842fb2084c8b2e3328fe92bc82852db434f768c
Sep 17 17:35:18.366538 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:35:18.366507    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-jsc2p"]
Sep 17 17:35:18.370060 ip-10-0-137-20 kubenswrapper[2565]: W0917 17:35:18.369996    2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod574f2e00_d361_423b_946b_e1e9ea3a5db5.slice/crio-c993bd59a71890e29728478cf32e91911c992e9694161db489e108ecfa6fd710 WatchSource:0}: Error finding container c993bd59a71890e29728478cf32e91911c992e9694161db489e108ecfa6fd710: Status 404 returned error can't find the container with id c993bd59a71890e29728478cf32e91911c992e9694161db489e108ecfa6fd710
Sep 17 17:35:18.412656 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:35:18.412601    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-jsc2p" event={"ID":"574f2e00-d361-423b-946b-e1e9ea3a5db5","Type":"ContainerStarted","Data":"c993bd59a71890e29728478cf32e91911c992e9694161db489e108ecfa6fd710"}
Sep 17 17:35:18.413749 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:35:18.413724    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-rmlqx" event={"ID":"e39c5e1e-113d-42f2-9a00-8a6d14cb6d67","Type":"ContainerStarted","Data":"f342d72946d6b0cd7dba395e9842fb2084c8b2e3328fe92bc82852db434f768c"}
Sep 17 17:35:18.414941 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:35:18.414922    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-74xd8" event={"ID":"3d1c780d-6713-495e-91fb-54a74668e6cc","Type":"ContainerStarted","Data":"d244544dbd6cadeee08b93743d232473bf7ee765b58489009ab4a34604a52159"}
Sep 17 17:35:18.416508 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:35:18.416485    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-b79cf788d-x7pgt" event={"ID":"9ab56083-4127-4bbb-af70-4ef55006fe47","Type":"ContainerStarted","Data":"dee6ce132d7c702c88549217cc6afaa0793e14697e844bc6ba3ba5c3d1e21ea4"}
Sep 17 17:35:18.416600 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:35:18.416525    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-b79cf788d-x7pgt" event={"ID":"9ab56083-4127-4bbb-af70-4ef55006fe47","Type":"ContainerStarted","Data":"db3867a0e90cec74c6bb6c2542e5f7b604a65777be5f50bd0f28eb96e943696f"}
Sep 17 17:35:18.416600 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:35:18.416589    2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-image-registry/image-registry-b79cf788d-x7pgt"
Sep 17 17:35:18.417649 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:35:18.417630    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-mj76m" event={"ID":"054b367d-f6ce-4337-ab2b-cab945fc918e","Type":"ContainerStarted","Data":"fdb6190fc26cb5c5a8dba0df6adff4b686f4f83fa95e72da8d991150490a48bf"}
Sep 17 17:35:18.440439 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:35:18.440393    2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/image-registry-b79cf788d-x7pgt" podStartSLOduration=53.440382879 podStartE2EDuration="53.440382879s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-17 17:34:25 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-17 17:35:18.439901814 +0000 UTC m=+68.317186232" watchObservedRunningTime="2026-09-17 17:35:18.440382879 +0000 UTC m=+68.317667316"
Sep 17 17:35:20.428363 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:35:20.428326    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-rmlqx" event={"ID":"e39c5e1e-113d-42f2-9a00-8a6d14cb6d67","Type":"ContainerStarted","Data":"efaa0dee6421fef3c3a92189316f196223f0fed16102d46b28c8325c15e8daa3"}
Sep 17 17:35:20.432672 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:35:20.432275    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-mj76m" event={"ID":"054b367d-f6ce-4337-ab2b-cab945fc918e","Type":"ContainerStarted","Data":"be8200af903b4e62f00d641060219aabc7aff5c9a62ea396db18ffaa8298317d"}
Sep 17 17:35:21.439646 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:35:21.439610    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-rmlqx" event={"ID":"e39c5e1e-113d-42f2-9a00-8a6d14cb6d67","Type":"ContainerStarted","Data":"70069dfe1df37b2238927da8b517cfde29f871ca20dec97c7e36f37b6de2ae21"}
Sep 17 17:35:21.441876 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:35:21.441854    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-mj76m" event={"ID":"054b367d-f6ce-4337-ab2b-cab945fc918e","Type":"ContainerStarted","Data":"19fdc590795c1822107d2eb0b10de4823fae10189cf9ec187bcc449779ffa355"}
Sep 17 17:35:21.442008 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:35:21.441985    2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-dns/dns-default-mj76m"
Sep 17 17:35:21.481744 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:35:21.481701    2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/network-metrics-daemon-rmlqx" podStartSLOduration=67.665276262 podStartE2EDuration="1m9.481688724s" totalImagesPullingTime="1.816412462s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-17 17:34:12 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-17 17:35:21.480797093 +0000 UTC m=+71.358081510" watchObservedRunningTime="2026-09-17 17:35:21.481688724 +0000 UTC m=+71.358973139"
Sep 17 17:35:21.525766 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:35:21.525718    2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/dns-default-mj76m" podStartSLOduration=34.54229922 podStartE2EDuration="36.52570685s" totalImagesPullingTime="1.98340763s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-17 17:34:45 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-17 17:35:21.525654992 +0000 UTC m=+71.402939406" watchObservedRunningTime="2026-09-17 17:35:21.52570685 +0000 UTC m=+71.402991264"
Sep 17 17:35:23.451284 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:35:23.451227    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-74xd8" event={"ID":"3d1c780d-6713-495e-91fb-54a74668e6cc","Type":"ContainerStarted","Data":"7a6b498be4280d7e49502eec520ee6fea85b147a3c1c30f206fa400d6133c4fb"}
Sep 17 17:35:23.452901 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:35:23.452876    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-jsc2p" event={"ID":"574f2e00-d361-423b-946b-e1e9ea3a5db5","Type":"ContainerStarted","Data":"af5270e2df06995764808e979b901f590b453afb5e1525695e654936f16497e2"}
Sep 17 17:35:23.453036 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:35:23.452999    2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-network-diagnostics/network-check-target-jsc2p"
Sep 17 17:35:23.471607 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:35:23.471538    2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/global-pull-secret-syncer-74xd8" podStartSLOduration=66.601985008 podStartE2EDuration="1m11.471526678s" totalImagesPullingTime="4.86954167s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-17 17:34:12 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-17 17:35:23.471186669 +0000 UTC m=+73.348471083" watchObservedRunningTime="2026-09-17 17:35:23.471526678 +0000 UTC m=+73.348811093"
Sep 17 17:35:23.488673 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:35:23.488634    2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-diagnostics/network-check-target-jsc2p" podStartSLOduration=66.651653073 podStartE2EDuration="1m11.488617593s" totalImagesPullingTime="4.83696452s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-17 17:34:12 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-17 17:35:23.488079151 +0000 UTC m=+73.365363569" watchObservedRunningTime="2026-09-17 17:35:23.488617593 +0000 UTC m=+73.365902005"
Sep 17 17:35:31.166981 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:35:31.166850    2565 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-image-registry/image-registry-b79cf788d-x7pgt"]
Sep 17 17:35:31.447771 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:35:31.447693    2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-dns/dns-default-mj76m"
Sep 17 17:35:34.365076 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:35:34.365037    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/fd16e5c0-6ecb-473a-84c0-bbee28f44e68-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-6mqqs\" (UID: \"fd16e5c0-6ecb-473a-84c0-bbee28f44e68\") " pod="openshift-insights/insights-runtime-extractor-6mqqs"
Sep 17 17:35:34.367323 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:35:34.367299    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/fd16e5c0-6ecb-473a-84c0-bbee28f44e68-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-6mqqs\" (UID: \"fd16e5c0-6ecb-473a-84c0-bbee28f44e68\") " pod="openshift-insights/insights-runtime-extractor-6mqqs"
Sep 17 17:35:34.585430 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:35:34.585400    2565 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/prometheus-operator-7bd5d56d6-lfpcg"]
Sep 17 17:35:34.589806 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:35:34.589791    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-operator-7bd5d56d6-lfpcg"
Sep 17 17:35:34.593898 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:35:34.593874    2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-tls\""
Sep 17 17:35:34.594016 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:35:34.593898    2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kube-root-ca.crt\""
Sep 17 17:35:34.594016 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:35:34.593929    2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"openshift-service-ca.crt\""
Sep 17 17:35:34.594016 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:35:34.593879    2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"metrics-client-ca\""
Sep 17 17:35:34.594196 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:35:34.594182    2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-kube-rbac-proxy-config\""
Sep 17 17:35:34.594280 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:35:34.594265    2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-dockercfg-xt99g\""
Sep 17 17:35:34.599286 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:35:34.599266    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-operator-7bd5d56d6-lfpcg"]
Sep 17 17:35:34.652460 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:35:34.652411    2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-sa-dockercfg-2xjm4\""
Sep 17 17:35:34.660886 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:35:34.660866    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-6mqqs"
Sep 17 17:35:34.666822 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:35:34.666792    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/14c8fb7d-dbe8-4246-b56e-cbded2092d3a-metrics-client-ca\") pod \"prometheus-operator-7bd5d56d6-lfpcg\" (UID: \"14c8fb7d-dbe8-4246-b56e-cbded2092d3a\") " pod="openshift-monitoring/prometheus-operator-7bd5d56d6-lfpcg"
Sep 17 17:35:34.666916 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:35:34.666825    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-operator-tls\" (UniqueName: \"kubernetes.io/secret/14c8fb7d-dbe8-4246-b56e-cbded2092d3a-prometheus-operator-tls\") pod \"prometheus-operator-7bd5d56d6-lfpcg\" (UID: \"14c8fb7d-dbe8-4246-b56e-cbded2092d3a\") " pod="openshift-monitoring/prometheus-operator-7bd5d56d6-lfpcg"
Sep 17 17:35:34.666916 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:35:34.666866    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-operator-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/14c8fb7d-dbe8-4246-b56e-cbded2092d3a-prometheus-operator-kube-rbac-proxy-config\") pod \"prometheus-operator-7bd5d56d6-lfpcg\" (UID: \"14c8fb7d-dbe8-4246-b56e-cbded2092d3a\") " pod="openshift-monitoring/prometheus-operator-7bd5d56d6-lfpcg"
Sep 17 17:35:34.666916 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:35:34.666892    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9j4bz\" (UniqueName: \"kubernetes.io/projected/14c8fb7d-dbe8-4246-b56e-cbded2092d3a-kube-api-access-9j4bz\") pod \"prometheus-operator-7bd5d56d6-lfpcg\" (UID: \"14c8fb7d-dbe8-4246-b56e-cbded2092d3a\") " pod="openshift-monitoring/prometheus-operator-7bd5d56d6-lfpcg"
Sep 17 17:35:34.669360 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:35:34.669341    2565 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-87747d4df-tk8xw"]
Sep 17 17:35:34.672803 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:35:34.672766    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-87747d4df-tk8xw"
Sep 17 17:35:34.677158 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:35:34.677138    2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-oauth-config\""
Sep 17 17:35:34.677327 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:35:34.677309    2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"openshift-service-ca.crt\""
Sep 17 17:35:34.677589 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:35:34.677507    2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"oauth-serving-cert\""
Sep 17 17:35:34.677803 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:35:34.677783    2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"kube-root-ca.crt\""
Sep 17 17:35:34.678084 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:35:34.678065    2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"console-config\""
Sep 17 17:35:34.678151 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:35:34.678108    2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"service-ca\""
Sep 17 17:35:34.678201 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:35:34.678065    2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-serving-cert\""
Sep 17 17:35:34.678260 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:35:34.678066    2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-dockercfg-j8glf\""
Sep 17 17:35:34.692204 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:35:34.692175    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-87747d4df-tk8xw"]
Sep 17 17:35:34.767559 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:35:34.767520    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-operator-tls\" (UniqueName: \"kubernetes.io/secret/14c8fb7d-dbe8-4246-b56e-cbded2092d3a-prometheus-operator-tls\") pod \"prometheus-operator-7bd5d56d6-lfpcg\" (UID: \"14c8fb7d-dbe8-4246-b56e-cbded2092d3a\") " pod="openshift-monitoring/prometheus-operator-7bd5d56d6-lfpcg"
Sep 17 17:35:34.767559 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:35:34.767555    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4pb2j\" (UniqueName: \"kubernetes.io/projected/cedae061-1174-4bc8-b7b9-0d034929ddad-kube-api-access-4pb2j\") pod \"console-87747d4df-tk8xw\" (UID: \"cedae061-1174-4bc8-b7b9-0d034929ddad\") " pod="openshift-console/console-87747d4df-tk8xw"
Sep 17 17:35:34.767726 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:35:34.767571    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/cedae061-1174-4bc8-b7b9-0d034929ddad-service-ca\") pod \"console-87747d4df-tk8xw\" (UID: \"cedae061-1174-4bc8-b7b9-0d034929ddad\") " pod="openshift-console/console-87747d4df-tk8xw"
Sep 17 17:35:34.767726 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:35:34.767594    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/cedae061-1174-4bc8-b7b9-0d034929ddad-console-oauth-config\") pod \"console-87747d4df-tk8xw\" (UID: \"cedae061-1174-4bc8-b7b9-0d034929ddad\") " pod="openshift-console/console-87747d4df-tk8xw"
Sep 17 17:35:34.767726 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:35:34.767613    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/cedae061-1174-4bc8-b7b9-0d034929ddad-oauth-serving-cert\") pod \"console-87747d4df-tk8xw\" (UID: \"cedae061-1174-4bc8-b7b9-0d034929ddad\") " pod="openshift-console/console-87747d4df-tk8xw"
Sep 17 17:35:34.767726 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:35:34.767659    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-operator-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/14c8fb7d-dbe8-4246-b56e-cbded2092d3a-prometheus-operator-kube-rbac-proxy-config\") pod \"prometheus-operator-7bd5d56d6-lfpcg\" (UID: \"14c8fb7d-dbe8-4246-b56e-cbded2092d3a\") " pod="openshift-monitoring/prometheus-operator-7bd5d56d6-lfpcg"
Sep 17 17:35:34.767726 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:35:34.767703    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9j4bz\" (UniqueName: \"kubernetes.io/projected/14c8fb7d-dbe8-4246-b56e-cbded2092d3a-kube-api-access-9j4bz\") pod \"prometheus-operator-7bd5d56d6-lfpcg\" (UID: \"14c8fb7d-dbe8-4246-b56e-cbded2092d3a\") " pod="openshift-monitoring/prometheus-operator-7bd5d56d6-lfpcg"
Sep 17 17:35:34.767949 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:35:34.767813    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/14c8fb7d-dbe8-4246-b56e-cbded2092d3a-metrics-client-ca\") pod \"prometheus-operator-7bd5d56d6-lfpcg\" (UID: \"14c8fb7d-dbe8-4246-b56e-cbded2092d3a\") " pod="openshift-monitoring/prometheus-operator-7bd5d56d6-lfpcg"
Sep 17 17:35:34.767949 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:35:34.767845    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/cedae061-1174-4bc8-b7b9-0d034929ddad-console-config\") pod \"console-87747d4df-tk8xw\" (UID: \"cedae061-1174-4bc8-b7b9-0d034929ddad\") " pod="openshift-console/console-87747d4df-tk8xw"
Sep 17 17:35:34.767949 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:35:34.767871    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/cedae061-1174-4bc8-b7b9-0d034929ddad-console-serving-cert\") pod \"console-87747d4df-tk8xw\" (UID: \"cedae061-1174-4bc8-b7b9-0d034929ddad\") " pod="openshift-console/console-87747d4df-tk8xw"
Sep 17 17:35:34.768419 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:35:34.768403    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/14c8fb7d-dbe8-4246-b56e-cbded2092d3a-metrics-client-ca\") pod \"prometheus-operator-7bd5d56d6-lfpcg\" (UID: \"14c8fb7d-dbe8-4246-b56e-cbded2092d3a\") " pod="openshift-monitoring/prometheus-operator-7bd5d56d6-lfpcg"
Sep 17 17:35:34.769800 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:35:34.769782    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-operator-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/14c8fb7d-dbe8-4246-b56e-cbded2092d3a-prometheus-operator-kube-rbac-proxy-config\") pod \"prometheus-operator-7bd5d56d6-lfpcg\" (UID: \"14c8fb7d-dbe8-4246-b56e-cbded2092d3a\") " pod="openshift-monitoring/prometheus-operator-7bd5d56d6-lfpcg"
Sep 17 17:35:34.769945 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:35:34.769929    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-operator-tls\" (UniqueName: \"kubernetes.io/secret/14c8fb7d-dbe8-4246-b56e-cbded2092d3a-prometheus-operator-tls\") pod \"prometheus-operator-7bd5d56d6-lfpcg\" (UID: \"14c8fb7d-dbe8-4246-b56e-cbded2092d3a\") " pod="openshift-monitoring/prometheus-operator-7bd5d56d6-lfpcg"
Sep 17 17:35:34.776744 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:35:34.776721    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-9j4bz\" (UniqueName: \"kubernetes.io/projected/14c8fb7d-dbe8-4246-b56e-cbded2092d3a-kube-api-access-9j4bz\") pod \"prometheus-operator-7bd5d56d6-lfpcg\" (UID: \"14c8fb7d-dbe8-4246-b56e-cbded2092d3a\") " pod="openshift-monitoring/prometheus-operator-7bd5d56d6-lfpcg"
Sep 17 17:35:34.868492 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:35:34.868464    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4pb2j\" (UniqueName: \"kubernetes.io/projected/cedae061-1174-4bc8-b7b9-0d034929ddad-kube-api-access-4pb2j\") pod \"console-87747d4df-tk8xw\" (UID: \"cedae061-1174-4bc8-b7b9-0d034929ddad\") " pod="openshift-console/console-87747d4df-tk8xw"
Sep 17 17:35:34.868492 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:35:34.868494    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/cedae061-1174-4bc8-b7b9-0d034929ddad-service-ca\") pod \"console-87747d4df-tk8xw\" (UID: \"cedae061-1174-4bc8-b7b9-0d034929ddad\") " pod="openshift-console/console-87747d4df-tk8xw"
Sep 17 17:35:34.868639 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:35:34.868515    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/cedae061-1174-4bc8-b7b9-0d034929ddad-console-oauth-config\") pod \"console-87747d4df-tk8xw\" (UID: \"cedae061-1174-4bc8-b7b9-0d034929ddad\") " pod="openshift-console/console-87747d4df-tk8xw"
Sep 17 17:35:34.868639 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:35:34.868531    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/cedae061-1174-4bc8-b7b9-0d034929ddad-oauth-serving-cert\") pod \"console-87747d4df-tk8xw\" (UID: \"cedae061-1174-4bc8-b7b9-0d034929ddad\") " pod="openshift-console/console-87747d4df-tk8xw"
Sep 17 17:35:34.868639 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:35:34.868565    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/cedae061-1174-4bc8-b7b9-0d034929ddad-console-config\") pod \"console-87747d4df-tk8xw\" (UID: \"cedae061-1174-4bc8-b7b9-0d034929ddad\") " pod="openshift-console/console-87747d4df-tk8xw"
Sep 17 17:35:34.868639 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:35:34.868583    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/cedae061-1174-4bc8-b7b9-0d034929ddad-console-serving-cert\") pod \"console-87747d4df-tk8xw\" (UID: \"cedae061-1174-4bc8-b7b9-0d034929ddad\") " pod="openshift-console/console-87747d4df-tk8xw"
Sep 17 17:35:34.869252 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:35:34.869230    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/cedae061-1174-4bc8-b7b9-0d034929ddad-oauth-serving-cert\") pod \"console-87747d4df-tk8xw\" (UID: \"cedae061-1174-4bc8-b7b9-0d034929ddad\") " pod="openshift-console/console-87747d4df-tk8xw"
Sep 17 17:35:34.869350 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:35:34.869265    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/cedae061-1174-4bc8-b7b9-0d034929ddad-service-ca\") pod \"console-87747d4df-tk8xw\" (UID: \"cedae061-1174-4bc8-b7b9-0d034929ddad\") " pod="openshift-console/console-87747d4df-tk8xw"
Sep 17 17:35:34.869350 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:35:34.869292    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/cedae061-1174-4bc8-b7b9-0d034929ddad-console-config\") pod \"console-87747d4df-tk8xw\" (UID: \"cedae061-1174-4bc8-b7b9-0d034929ddad\") " pod="openshift-console/console-87747d4df-tk8xw"
Sep 17 17:35:34.870883 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:35:34.870863    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/cedae061-1174-4bc8-b7b9-0d034929ddad-console-oauth-config\") pod \"console-87747d4df-tk8xw\" (UID: \"cedae061-1174-4bc8-b7b9-0d034929ddad\") " pod="openshift-console/console-87747d4df-tk8xw"
Sep 17 17:35:34.870936 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:35:34.870868    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/cedae061-1174-4bc8-b7b9-0d034929ddad-console-serving-cert\") pod \"console-87747d4df-tk8xw\" (UID: \"cedae061-1174-4bc8-b7b9-0d034929ddad\") " pod="openshift-console/console-87747d4df-tk8xw"
Sep 17 17:35:34.876323 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:35:34.876302    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-4pb2j\" (UniqueName: \"kubernetes.io/projected/cedae061-1174-4bc8-b7b9-0d034929ddad-kube-api-access-4pb2j\") pod \"console-87747d4df-tk8xw\" (UID: \"cedae061-1174-4bc8-b7b9-0d034929ddad\") " pod="openshift-console/console-87747d4df-tk8xw"
Sep 17 17:35:34.899327 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:35:34.899106    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-operator-7bd5d56d6-lfpcg"
Sep 17 17:35:34.933264 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:35:34.933242    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-6mqqs"]
Sep 17 17:35:34.935791 ip-10-0-137-20 kubenswrapper[2565]: W0917 17:35:34.935762    2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podfd16e5c0_6ecb_473a_84c0_bbee28f44e68.slice/crio-177c3c75a782a7b19302189f534a780b6a6ae2db4fb6fbd8d6d53b842585f3b8 WatchSource:0}: Error finding container 177c3c75a782a7b19302189f534a780b6a6ae2db4fb6fbd8d6d53b842585f3b8: Status 404 returned error can't find the container with id 177c3c75a782a7b19302189f534a780b6a6ae2db4fb6fbd8d6d53b842585f3b8
Sep 17 17:35:34.986805 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:35:34.986780    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-87747d4df-tk8xw"
Sep 17 17:35:35.170562 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:35:35.170536    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-operator-7bd5d56d6-lfpcg"]
Sep 17 17:35:35.172616 ip-10-0-137-20 kubenswrapper[2565]: W0917 17:35:35.172591    2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod14c8fb7d_dbe8_4246_b56e_cbded2092d3a.slice/crio-2bafcb8b5599c90981f5691af6d06003af0851a70cb9e772e300411d2dff5bb1 WatchSource:0}: Error finding container 2bafcb8b5599c90981f5691af6d06003af0851a70cb9e772e300411d2dff5bb1: Status 404 returned error can't find the container with id 2bafcb8b5599c90981f5691af6d06003af0851a70cb9e772e300411d2dff5bb1
Sep 17 17:35:35.261795 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:35:35.261724    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-87747d4df-tk8xw"]
Sep 17 17:35:35.263955 ip-10-0-137-20 kubenswrapper[2565]: W0917 17:35:35.263930    2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podcedae061_1174_4bc8_b7b9_0d034929ddad.slice/crio-8d6fcc892e090b6d00361162e09aca3454f6a166ea0781b71e5c38b4d917eeaa WatchSource:0}: Error finding container 8d6fcc892e090b6d00361162e09aca3454f6a166ea0781b71e5c38b4d917eeaa: Status 404 returned error can't find the container with id 8d6fcc892e090b6d00361162e09aca3454f6a166ea0781b71e5c38b4d917eeaa
Sep 17 17:35:35.491722 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:35:35.491641    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-87747d4df-tk8xw" event={"ID":"cedae061-1174-4bc8-b7b9-0d034929ddad","Type":"ContainerStarted","Data":"8d6fcc892e090b6d00361162e09aca3454f6a166ea0781b71e5c38b4d917eeaa"}
Sep 17 17:35:35.493428 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:35:35.493402    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-6mqqs" event={"ID":"fd16e5c0-6ecb-473a-84c0-bbee28f44e68","Type":"ContainerStarted","Data":"5f06e290b14c8938c13ee5f5607f4f103ea64ee219dc1e2b76c7221d23d1bd21"}
Sep 17 17:35:35.493566 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:35:35.493460    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-6mqqs" event={"ID":"fd16e5c0-6ecb-473a-84c0-bbee28f44e68","Type":"ContainerStarted","Data":"177c3c75a782a7b19302189f534a780b6a6ae2db4fb6fbd8d6d53b842585f3b8"}
Sep 17 17:35:35.494547 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:35:35.494529    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-7bd5d56d6-lfpcg" event={"ID":"14c8fb7d-dbe8-4246-b56e-cbded2092d3a","Type":"ContainerStarted","Data":"2bafcb8b5599c90981f5691af6d06003af0851a70cb9e772e300411d2dff5bb1"}
Sep 17 17:35:36.512845 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:35:36.512797    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-6mqqs" event={"ID":"fd16e5c0-6ecb-473a-84c0-bbee28f44e68","Type":"ContainerStarted","Data":"39eaf9bfbcb375cc773260f115e37eb30b89ac6677736ada92082fa0e389587d"}
Sep 17 17:35:40.528914 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:35:40.528876    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-7bd5d56d6-lfpcg" event={"ID":"14c8fb7d-dbe8-4246-b56e-cbded2092d3a","Type":"ContainerStarted","Data":"3d9362b9235766ac7338e071b12eb62f0a4dc94156771783a904e467e9304549"}
Sep 17 17:35:40.529371 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:35:40.528933    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-7bd5d56d6-lfpcg" event={"ID":"14c8fb7d-dbe8-4246-b56e-cbded2092d3a","Type":"ContainerStarted","Data":"16ffe278ce2b950ec39eb6c1f0b9cb958d0353ff9214bbbd643577164089edd0"}
Sep 17 17:35:40.530402 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:35:40.530376    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-87747d4df-tk8xw" event={"ID":"cedae061-1174-4bc8-b7b9-0d034929ddad","Type":"ContainerStarted","Data":"3789759b52cab6aafcf0ce91d69c2d69ccd11246729fd93665b9a16d715537b0"}
Sep 17 17:35:40.532336 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:35:40.532315    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-6mqqs" event={"ID":"fd16e5c0-6ecb-473a-84c0-bbee28f44e68","Type":"ContainerStarted","Data":"cb6784da528370461bd60e0e49b75abbbd0d46f1acbd0fb024d565846e873342"}
Sep 17 17:35:40.554082 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:35:40.554044    2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/prometheus-operator-7bd5d56d6-lfpcg" podStartSLOduration=1.99644994 podStartE2EDuration="6.554009879s" totalImagesPullingTime="4.557559939s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-17 17:35:34 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-17 17:35:40.553462399 +0000 UTC m=+90.430746816" watchObservedRunningTime="2026-09-17 17:35:40.554009879 +0000 UTC m=+90.431294297"
Sep 17 17:35:40.579708 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:35:40.579666    2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-insights/insights-runtime-extractor-6mqqs" podStartSLOduration=33.913192521 podStartE2EDuration="38.579651656s" totalImagesPullingTime="4.666459135s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-17 17:35:02 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-17 17:35:40.578943265 +0000 UTC m=+90.456227680" watchObservedRunningTime="2026-09-17 17:35:40.579651656 +0000 UTC m=+90.456936072"
Sep 17 17:35:40.597368 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:35:40.597329    2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-87747d4df-tk8xw" podStartSLOduration=2.065624171 podStartE2EDuration="6.597317978s" totalImagesPullingTime="4.531693807s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-17 17:35:34 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-17 17:35:40.597128231 +0000 UTC m=+90.474412641" watchObservedRunningTime="2026-09-17 17:35:40.597317978 +0000 UTC m=+90.474602393"
Sep 17 17:35:41.172133 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:35:41.172112    2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-image-registry/image-registry-b79cf788d-x7pgt"
Sep 17 17:35:43.046918 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:35:43.046884    2565 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/openshift-state-metrics-86c5bc79d6-28pph"]
Sep 17 17:35:43.053901 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:35:43.053880    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/openshift-state-metrics-86c5bc79d6-28pph"
Sep 17 17:35:43.057050 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:35:43.057016    2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"openshift-state-metrics-kube-rbac-proxy-config\""
Sep 17 17:35:43.057200 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:35:43.057180    2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"openshift-state-metrics-tls\""
Sep 17 17:35:43.057889 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:35:43.057869    2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"openshift-state-metrics-dockercfg-5hwr2\""
Sep 17 17:35:43.089571 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:35:43.089549    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/openshift-state-metrics-86c5bc79d6-28pph"]
Sep 17 17:35:43.130814 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:35:43.130791    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/d4a0864d-cd66-4cd1-9e93-4f39b40bf3c3-metrics-client-ca\") pod \"openshift-state-metrics-86c5bc79d6-28pph\" (UID: \"d4a0864d-cd66-4cd1-9e93-4f39b40bf3c3\") " pod="openshift-monitoring/openshift-state-metrics-86c5bc79d6-28pph"
Sep 17 17:35:43.130913 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:35:43.130838    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-z6db9\" (UniqueName: \"kubernetes.io/projected/d4a0864d-cd66-4cd1-9e93-4f39b40bf3c3-kube-api-access-z6db9\") pod \"openshift-state-metrics-86c5bc79d6-28pph\" (UID: \"d4a0864d-cd66-4cd1-9e93-4f39b40bf3c3\") " pod="openshift-monitoring/openshift-state-metrics-86c5bc79d6-28pph"
Sep 17 17:35:43.130913 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:35:43.130873    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/d4a0864d-cd66-4cd1-9e93-4f39b40bf3c3-openshift-state-metrics-tls\") pod \"openshift-state-metrics-86c5bc79d6-28pph\" (UID: \"d4a0864d-cd66-4cd1-9e93-4f39b40bf3c3\") " pod="openshift-monitoring/openshift-state-metrics-86c5bc79d6-28pph"
Sep 17 17:35:43.130999 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:35:43.130921    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/d4a0864d-cd66-4cd1-9e93-4f39b40bf3c3-openshift-state-metrics-kube-rbac-proxy-config\") pod \"openshift-state-metrics-86c5bc79d6-28pph\" (UID: \"d4a0864d-cd66-4cd1-9e93-4f39b40bf3c3\") " pod="openshift-monitoring/openshift-state-metrics-86c5bc79d6-28pph"
Sep 17 17:35:43.192264 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:35:43.192241    2565 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/node-exporter-rkh9d"]
Sep 17 17:35:43.196371 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:35:43.196358    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-rkh9d"
Sep 17 17:35:43.201340 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:35:43.201322    2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"node-exporter-accelerators-collector-config\""
Sep 17 17:35:43.201713 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:35:43.201699    2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-kube-rbac-proxy-config\""
Sep 17 17:35:43.202075 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:35:43.202058    2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-tls\""
Sep 17 17:35:43.203262 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:35:43.203237    2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-dockercfg-vptsg\""
Sep 17 17:35:43.231337 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:35:43.231314    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/d4a0864d-cd66-4cd1-9e93-4f39b40bf3c3-openshift-state-metrics-kube-rbac-proxy-config\") pod \"openshift-state-metrics-86c5bc79d6-28pph\" (UID: \"d4a0864d-cd66-4cd1-9e93-4f39b40bf3c3\") " pod="openshift-monitoring/openshift-state-metrics-86c5bc79d6-28pph"
Sep 17 17:35:43.231440 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:35:43.231348    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/d4a0864d-cd66-4cd1-9e93-4f39b40bf3c3-metrics-client-ca\") pod \"openshift-state-metrics-86c5bc79d6-28pph\" (UID: \"d4a0864d-cd66-4cd1-9e93-4f39b40bf3c3\") " pod="openshift-monitoring/openshift-state-metrics-86c5bc79d6-28pph"
Sep 17 17:35:43.231440 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:35:43.231373    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-z6db9\" (UniqueName: \"kubernetes.io/projected/d4a0864d-cd66-4cd1-9e93-4f39b40bf3c3-kube-api-access-z6db9\") pod \"openshift-state-metrics-86c5bc79d6-28pph\" (UID: \"d4a0864d-cd66-4cd1-9e93-4f39b40bf3c3\") " pod="openshift-monitoring/openshift-state-metrics-86c5bc79d6-28pph"
Sep 17 17:35:43.231440 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:35:43.231430    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/d4a0864d-cd66-4cd1-9e93-4f39b40bf3c3-openshift-state-metrics-tls\") pod \"openshift-state-metrics-86c5bc79d6-28pph\" (UID: \"d4a0864d-cd66-4cd1-9e93-4f39b40bf3c3\") " pod="openshift-monitoring/openshift-state-metrics-86c5bc79d6-28pph"
Sep 17 17:35:43.231585 ip-10-0-137-20 kubenswrapper[2565]: E0917 17:35:43.231524    2565 secret.go:189] Couldn't get secret openshift-monitoring/openshift-state-metrics-tls: secret "openshift-state-metrics-tls" not found
Sep 17 17:35:43.231645 ip-10-0-137-20 kubenswrapper[2565]: E0917 17:35:43.231627    2565 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d4a0864d-cd66-4cd1-9e93-4f39b40bf3c3-openshift-state-metrics-tls podName:d4a0864d-cd66-4cd1-9e93-4f39b40bf3c3 nodeName:}" failed. No retries permitted until 2026-09-17 17:35:43.73160769 +0000 UTC m=+93.608892100 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "openshift-state-metrics-tls" (UniqueName: "kubernetes.io/secret/d4a0864d-cd66-4cd1-9e93-4f39b40bf3c3-openshift-state-metrics-tls") pod "openshift-state-metrics-86c5bc79d6-28pph" (UID: "d4a0864d-cd66-4cd1-9e93-4f39b40bf3c3") : secret "openshift-state-metrics-tls" not found
Sep 17 17:35:43.231955 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:35:43.231938    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/d4a0864d-cd66-4cd1-9e93-4f39b40bf3c3-metrics-client-ca\") pod \"openshift-state-metrics-86c5bc79d6-28pph\" (UID: \"d4a0864d-cd66-4cd1-9e93-4f39b40bf3c3\") " pod="openshift-monitoring/openshift-state-metrics-86c5bc79d6-28pph"
Sep 17 17:35:43.233696 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:35:43.233679    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/d4a0864d-cd66-4cd1-9e93-4f39b40bf3c3-openshift-state-metrics-kube-rbac-proxy-config\") pod \"openshift-state-metrics-86c5bc79d6-28pph\" (UID: \"d4a0864d-cd66-4cd1-9e93-4f39b40bf3c3\") " pod="openshift-monitoring/openshift-state-metrics-86c5bc79d6-28pph"
Sep 17 17:35:43.257372 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:35:43.257354    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-z6db9\" (UniqueName: \"kubernetes.io/projected/d4a0864d-cd66-4cd1-9e93-4f39b40bf3c3-kube-api-access-z6db9\") pod \"openshift-state-metrics-86c5bc79d6-28pph\" (UID: \"d4a0864d-cd66-4cd1-9e93-4f39b40bf3c3\") " pod="openshift-monitoring/openshift-state-metrics-86c5bc79d6-28pph"
Sep 17 17:35:43.332258 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:35:43.332206    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/2e8b2e4c-5079-4454-a69a-9a183b1c1014-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-rkh9d\" (UID: \"2e8b2e4c-5079-4454-a69a-9a183b1c1014\") " pod="openshift-monitoring/node-exporter-rkh9d"
Sep 17 17:35:43.332258 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:35:43.332241    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/2e8b2e4c-5079-4454-a69a-9a183b1c1014-node-exporter-tls\") pod \"node-exporter-rkh9d\" (UID: \"2e8b2e4c-5079-4454-a69a-9a183b1c1014\") " pod="openshift-monitoring/node-exporter-rkh9d"
Sep 17 17:35:43.332393 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:35:43.332261    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/2e8b2e4c-5079-4454-a69a-9a183b1c1014-node-exporter-wtmp\") pod \"node-exporter-rkh9d\" (UID: \"2e8b2e4c-5079-4454-a69a-9a183b1c1014\") " pod="openshift-monitoring/node-exporter-rkh9d"
Sep 17 17:35:43.332393 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:35:43.332278    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/2e8b2e4c-5079-4454-a69a-9a183b1c1014-metrics-client-ca\") pod \"node-exporter-rkh9d\" (UID: \"2e8b2e4c-5079-4454-a69a-9a183b1c1014\") " pod="openshift-monitoring/node-exporter-rkh9d"
Sep 17 17:35:43.332393 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:35:43.332327    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6xmlv\" (UniqueName: \"kubernetes.io/projected/2e8b2e4c-5079-4454-a69a-9a183b1c1014-kube-api-access-6xmlv\") pod \"node-exporter-rkh9d\" (UID: \"2e8b2e4c-5079-4454-a69a-9a183b1c1014\") " pod="openshift-monitoring/node-exporter-rkh9d"
Sep 17 17:35:43.332489 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:35:43.332407    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/2e8b2e4c-5079-4454-a69a-9a183b1c1014-node-exporter-textfile\") pod \"node-exporter-rkh9d\" (UID: \"2e8b2e4c-5079-4454-a69a-9a183b1c1014\") " pod="openshift-monitoring/node-exporter-rkh9d"
Sep 17 17:35:43.332489 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:35:43.332426    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/2e8b2e4c-5079-4454-a69a-9a183b1c1014-root\") pod \"node-exporter-rkh9d\" (UID: \"2e8b2e4c-5079-4454-a69a-9a183b1c1014\") " pod="openshift-monitoring/node-exporter-rkh9d"
Sep 17 17:35:43.332489 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:35:43.332443    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/2e8b2e4c-5079-4454-a69a-9a183b1c1014-sys\") pod \"node-exporter-rkh9d\" (UID: \"2e8b2e4c-5079-4454-a69a-9a183b1c1014\") " pod="openshift-monitoring/node-exporter-rkh9d"
Sep 17 17:35:43.332489 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:35:43.332473    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/2e8b2e4c-5079-4454-a69a-9a183b1c1014-node-exporter-accelerators-collector-config\") pod \"node-exporter-rkh9d\" (UID: \"2e8b2e4c-5079-4454-a69a-9a183b1c1014\") " pod="openshift-monitoring/node-exporter-rkh9d"
Sep 17 17:35:43.433586 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:35:43.433563    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/2e8b2e4c-5079-4454-a69a-9a183b1c1014-sys\") pod \"node-exporter-rkh9d\" (UID: \"2e8b2e4c-5079-4454-a69a-9a183b1c1014\") " pod="openshift-monitoring/node-exporter-rkh9d"
Sep 17 17:35:43.433689 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:35:43.433590    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/2e8b2e4c-5079-4454-a69a-9a183b1c1014-node-exporter-accelerators-collector-config\") pod \"node-exporter-rkh9d\" (UID: \"2e8b2e4c-5079-4454-a69a-9a183b1c1014\") " pod="openshift-monitoring/node-exporter-rkh9d"
Sep 17 17:35:43.433689 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:35:43.433617    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/2e8b2e4c-5079-4454-a69a-9a183b1c1014-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-rkh9d\" (UID: \"2e8b2e4c-5079-4454-a69a-9a183b1c1014\") " pod="openshift-monitoring/node-exporter-rkh9d"
Sep 17 17:35:43.433689 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:35:43.433655    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/2e8b2e4c-5079-4454-a69a-9a183b1c1014-node-exporter-tls\") pod \"node-exporter-rkh9d\" (UID: \"2e8b2e4c-5079-4454-a69a-9a183b1c1014\") " pod="openshift-monitoring/node-exporter-rkh9d"
Sep 17 17:35:43.433689 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:35:43.433668    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/2e8b2e4c-5079-4454-a69a-9a183b1c1014-sys\") pod \"node-exporter-rkh9d\" (UID: \"2e8b2e4c-5079-4454-a69a-9a183b1c1014\") " pod="openshift-monitoring/node-exporter-rkh9d"
Sep 17 17:35:43.433689 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:35:43.433686    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/2e8b2e4c-5079-4454-a69a-9a183b1c1014-node-exporter-wtmp\") pod \"node-exporter-rkh9d\" (UID: \"2e8b2e4c-5079-4454-a69a-9a183b1c1014\") " pod="openshift-monitoring/node-exporter-rkh9d"
Sep 17 17:35:43.433933 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:35:43.433711    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/2e8b2e4c-5079-4454-a69a-9a183b1c1014-metrics-client-ca\") pod \"node-exporter-rkh9d\" (UID: \"2e8b2e4c-5079-4454-a69a-9a183b1c1014\") " pod="openshift-monitoring/node-exporter-rkh9d"
Sep 17 17:35:43.433933 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:35:43.433737    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6xmlv\" (UniqueName: \"kubernetes.io/projected/2e8b2e4c-5079-4454-a69a-9a183b1c1014-kube-api-access-6xmlv\") pod \"node-exporter-rkh9d\" (UID: \"2e8b2e4c-5079-4454-a69a-9a183b1c1014\") " pod="openshift-monitoring/node-exporter-rkh9d"
Sep 17 17:35:43.433933 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:35:43.433793    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/2e8b2e4c-5079-4454-a69a-9a183b1c1014-node-exporter-textfile\") pod \"node-exporter-rkh9d\" (UID: \"2e8b2e4c-5079-4454-a69a-9a183b1c1014\") " pod="openshift-monitoring/node-exporter-rkh9d"
Sep 17 17:35:43.433933 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:35:43.433810    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/2e8b2e4c-5079-4454-a69a-9a183b1c1014-root\") pod \"node-exporter-rkh9d\" (UID: \"2e8b2e4c-5079-4454-a69a-9a183b1c1014\") " pod="openshift-monitoring/node-exporter-rkh9d"
Sep 17 17:35:43.433933 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:35:43.433866    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"root\" (UniqueName: \"kubernetes.io/host-path/2e8b2e4c-5079-4454-a69a-9a183b1c1014-root\") pod \"node-exporter-rkh9d\" (UID: \"2e8b2e4c-5079-4454-a69a-9a183b1c1014\") " pod="openshift-monitoring/node-exporter-rkh9d"
Sep 17 17:35:43.433933 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:35:43.433866    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/2e8b2e4c-5079-4454-a69a-9a183b1c1014-node-exporter-wtmp\") pod \"node-exporter-rkh9d\" (UID: \"2e8b2e4c-5079-4454-a69a-9a183b1c1014\") " pod="openshift-monitoring/node-exporter-rkh9d"
Sep 17 17:35:43.434221 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:35:43.434151    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/2e8b2e4c-5079-4454-a69a-9a183b1c1014-node-exporter-textfile\") pod \"node-exporter-rkh9d\" (UID: \"2e8b2e4c-5079-4454-a69a-9a183b1c1014\") " pod="openshift-monitoring/node-exporter-rkh9d"
Sep 17 17:35:43.434257 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:35:43.434234    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/2e8b2e4c-5079-4454-a69a-9a183b1c1014-node-exporter-accelerators-collector-config\") pod \"node-exporter-rkh9d\" (UID: \"2e8b2e4c-5079-4454-a69a-9a183b1c1014\") " pod="openshift-monitoring/node-exporter-rkh9d"
Sep 17 17:35:43.434306 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:35:43.434282    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/2e8b2e4c-5079-4454-a69a-9a183b1c1014-metrics-client-ca\") pod \"node-exporter-rkh9d\" (UID: \"2e8b2e4c-5079-4454-a69a-9a183b1c1014\") " pod="openshift-monitoring/node-exporter-rkh9d"
Sep 17 17:35:43.436396 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:35:43.436374    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/2e8b2e4c-5079-4454-a69a-9a183b1c1014-node-exporter-tls\") pod \"node-exporter-rkh9d\" (UID: \"2e8b2e4c-5079-4454-a69a-9a183b1c1014\") " pod="openshift-monitoring/node-exporter-rkh9d"
Sep 17 17:35:43.436465 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:35:43.436423    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/2e8b2e4c-5079-4454-a69a-9a183b1c1014-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-rkh9d\" (UID: \"2e8b2e4c-5079-4454-a69a-9a183b1c1014\") " pod="openshift-monitoring/node-exporter-rkh9d"
Sep 17 17:35:43.451090 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:35:43.451073    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-6xmlv\" (UniqueName: \"kubernetes.io/projected/2e8b2e4c-5079-4454-a69a-9a183b1c1014-kube-api-access-6xmlv\") pod \"node-exporter-rkh9d\" (UID: \"2e8b2e4c-5079-4454-a69a-9a183b1c1014\") " pod="openshift-monitoring/node-exporter-rkh9d"
Sep 17 17:35:43.505403 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:35:43.505383    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-rkh9d"
Sep 17 17:35:43.518185 ip-10-0-137-20 kubenswrapper[2565]: W0917 17:35:43.518160    2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2e8b2e4c_5079_4454_a69a_9a183b1c1014.slice/crio-7763fb91414c674e4885488a127483d2d25627eef91556cac39639a002bc10cb WatchSource:0}: Error finding container 7763fb91414c674e4885488a127483d2d25627eef91556cac39639a002bc10cb: Status 404 returned error can't find the container with id 7763fb91414c674e4885488a127483d2d25627eef91556cac39639a002bc10cb
Sep 17 17:35:43.543230 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:35:43.543203    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-rkh9d" event={"ID":"2e8b2e4c-5079-4454-a69a-9a183b1c1014","Type":"ContainerStarted","Data":"7763fb91414c674e4885488a127483d2d25627eef91556cac39639a002bc10cb"}
Sep 17 17:35:43.735815 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:35:43.735795    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/d4a0864d-cd66-4cd1-9e93-4f39b40bf3c3-openshift-state-metrics-tls\") pod \"openshift-state-metrics-86c5bc79d6-28pph\" (UID: \"d4a0864d-cd66-4cd1-9e93-4f39b40bf3c3\") " pod="openshift-monitoring/openshift-state-metrics-86c5bc79d6-28pph"
Sep 17 17:35:43.738092 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:35:43.738070    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/d4a0864d-cd66-4cd1-9e93-4f39b40bf3c3-openshift-state-metrics-tls\") pod \"openshift-state-metrics-86c5bc79d6-28pph\" (UID: \"d4a0864d-cd66-4cd1-9e93-4f39b40bf3c3\") " pod="openshift-monitoring/openshift-state-metrics-86c5bc79d6-28pph"
Sep 17 17:35:43.963821 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:35:43.963798    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/openshift-state-metrics-86c5bc79d6-28pph"
Sep 17 17:35:44.087092 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:35:44.083834    2565 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/alertmanager-main-0"]
Sep 17 17:35:44.092210 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:35:44.092187    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0"
Sep 17 17:35:44.095707 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:35:44.095669    2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-tls\""
Sep 17 17:35:44.095707 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:35:44.095683    2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-generated\""
Sep 17 17:35:44.095866 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:35:44.095709    2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-cluster-tls-config\""
Sep 17 17:35:44.095866 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:35:44.095674    2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy-web\""
Sep 17 17:35:44.118109 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:35:44.118092    2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"alertmanager-trusted-ca-bundle\""
Sep 17 17:35:44.123414 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:35:44.123389    2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy-metric\""
Sep 17 17:35:44.123526 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:35:44.123432    2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-tls-assets-0\""
Sep 17 17:35:44.124560 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:35:44.124540    2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy\""
Sep 17 17:35:44.124706 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:35:44.124577    2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-dockercfg-q65bv\""
Sep 17 17:35:44.124706 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:35:44.124622    2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-web-config\""
Sep 17 17:35:44.125967 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:35:44.125949    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/alertmanager-main-0"]
Sep 17 17:35:44.139553 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:35:44.139530    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/16a894e3-8971-45a7-9f5e-c1da4d5b932f-web-config\") pod \"alertmanager-main-0\" (UID: \"16a894e3-8971-45a7-9f5e-c1da4d5b932f\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 17 17:35:44.139640 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:35:44.139563    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/16a894e3-8971-45a7-9f5e-c1da4d5b932f-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"16a894e3-8971-45a7-9f5e-c1da4d5b932f\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 17 17:35:44.139640 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:35:44.139585    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/16a894e3-8971-45a7-9f5e-c1da4d5b932f-config-out\") pod \"alertmanager-main-0\" (UID: \"16a894e3-8971-45a7-9f5e-c1da4d5b932f\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 17 17:35:44.139640 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:35:44.139632    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/16a894e3-8971-45a7-9f5e-c1da4d5b932f-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"16a894e3-8971-45a7-9f5e-c1da4d5b932f\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 17 17:35:44.139776 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:35:44.139654    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/16a894e3-8971-45a7-9f5e-c1da4d5b932f-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"16a894e3-8971-45a7-9f5e-c1da4d5b932f\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 17 17:35:44.139776 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:35:44.139674    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/16a894e3-8971-45a7-9f5e-c1da4d5b932f-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"16a894e3-8971-45a7-9f5e-c1da4d5b932f\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 17 17:35:44.139776 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:35:44.139702    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/16a894e3-8971-45a7-9f5e-c1da4d5b932f-tls-assets\") pod \"alertmanager-main-0\" (UID: \"16a894e3-8971-45a7-9f5e-c1da4d5b932f\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 17 17:35:44.139776 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:35:44.139750    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5m6k9\" (UniqueName: \"kubernetes.io/projected/16a894e3-8971-45a7-9f5e-c1da4d5b932f-kube-api-access-5m6k9\") pod \"alertmanager-main-0\" (UID: \"16a894e3-8971-45a7-9f5e-c1da4d5b932f\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 17 17:35:44.139968 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:35:44.139791    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/16a894e3-8971-45a7-9f5e-c1da4d5b932f-config-volume\") pod \"alertmanager-main-0\" (UID: \"16a894e3-8971-45a7-9f5e-c1da4d5b932f\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 17 17:35:44.139968 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:35:44.139842    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/16a894e3-8971-45a7-9f5e-c1da4d5b932f-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"16a894e3-8971-45a7-9f5e-c1da4d5b932f\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 17 17:35:44.139968 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:35:44.139883    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/16a894e3-8971-45a7-9f5e-c1da4d5b932f-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"16a894e3-8971-45a7-9f5e-c1da4d5b932f\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 17 17:35:44.139968 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:35:44.139912    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/16a894e3-8971-45a7-9f5e-c1da4d5b932f-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"16a894e3-8971-45a7-9f5e-c1da4d5b932f\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 17 17:35:44.139968 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:35:44.139937    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/16a894e3-8971-45a7-9f5e-c1da4d5b932f-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"16a894e3-8971-45a7-9f5e-c1da4d5b932f\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 17 17:35:44.239945 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:35:44.239922    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/openshift-state-metrics-86c5bc79d6-28pph"]
Sep 17 17:35:44.240300 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:35:44.240279    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/16a894e3-8971-45a7-9f5e-c1da4d5b932f-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"16a894e3-8971-45a7-9f5e-c1da4d5b932f\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 17 17:35:44.240413 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:35:44.240308    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/16a894e3-8971-45a7-9f5e-c1da4d5b932f-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"16a894e3-8971-45a7-9f5e-c1da4d5b932f\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 17 17:35:44.240413 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:35:44.240333    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/16a894e3-8971-45a7-9f5e-c1da4d5b932f-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"16a894e3-8971-45a7-9f5e-c1da4d5b932f\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 17 17:35:44.240413 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:35:44.240357    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/16a894e3-8971-45a7-9f5e-c1da4d5b932f-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"16a894e3-8971-45a7-9f5e-c1da4d5b932f\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 17 17:35:44.240413 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:35:44.240395    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/16a894e3-8971-45a7-9f5e-c1da4d5b932f-web-config\") pod \"alertmanager-main-0\" (UID: \"16a894e3-8971-45a7-9f5e-c1da4d5b932f\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 17 17:35:44.240583 ip-10-0-137-20 kubenswrapper[2565]: E0917 17:35:44.240423    2565 secret.go:189] Couldn't get secret openshift-monitoring/alertmanager-main-tls: secret "alertmanager-main-tls" not found
Sep 17 17:35:44.240583 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:35:44.240438    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/16a894e3-8971-45a7-9f5e-c1da4d5b932f-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"16a894e3-8971-45a7-9f5e-c1da4d5b932f\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 17 17:35:44.240583 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:35:44.240473    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/16a894e3-8971-45a7-9f5e-c1da4d5b932f-config-out\") pod \"alertmanager-main-0\" (UID: \"16a894e3-8971-45a7-9f5e-c1da4d5b932f\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 17 17:35:44.240583 ip-10-0-137-20 kubenswrapper[2565]: E0917 17:35:44.240489    2565 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/16a894e3-8971-45a7-9f5e-c1da4d5b932f-secret-alertmanager-main-tls podName:16a894e3-8971-45a7-9f5e-c1da4d5b932f nodeName:}" failed. No retries permitted until 2026-09-17 17:35:44.740470089 +0000 UTC m=+94.617754496 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "secret-alertmanager-main-tls" (UniqueName: "kubernetes.io/secret/16a894e3-8971-45a7-9f5e-c1da4d5b932f-secret-alertmanager-main-tls") pod "alertmanager-main-0" (UID: "16a894e3-8971-45a7-9f5e-c1da4d5b932f") : secret "alertmanager-main-tls" not found
Sep 17 17:35:44.241193 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:35:44.240864    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/16a894e3-8971-45a7-9f5e-c1da4d5b932f-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"16a894e3-8971-45a7-9f5e-c1da4d5b932f\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 17 17:35:44.241368 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:35:44.241349    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/16a894e3-8971-45a7-9f5e-c1da4d5b932f-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"16a894e3-8971-45a7-9f5e-c1da4d5b932f\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 17 17:35:44.241492 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:35:44.241468    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/16a894e3-8971-45a7-9f5e-c1da4d5b932f-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"16a894e3-8971-45a7-9f5e-c1da4d5b932f\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 17 17:35:44.241555 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:35:44.241496    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/16a894e3-8971-45a7-9f5e-c1da4d5b932f-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"16a894e3-8971-45a7-9f5e-c1da4d5b932f\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 17 17:35:44.241610 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:35:44.241550    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/16a894e3-8971-45a7-9f5e-c1da4d5b932f-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"16a894e3-8971-45a7-9f5e-c1da4d5b932f\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 17 17:35:44.241610 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:35:44.241600    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/16a894e3-8971-45a7-9f5e-c1da4d5b932f-tls-assets\") pod \"alertmanager-main-0\" (UID: \"16a894e3-8971-45a7-9f5e-c1da4d5b932f\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 17 17:35:44.241705 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:35:44.241627    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5m6k9\" (UniqueName: \"kubernetes.io/projected/16a894e3-8971-45a7-9f5e-c1da4d5b932f-kube-api-access-5m6k9\") pod \"alertmanager-main-0\" (UID: \"16a894e3-8971-45a7-9f5e-c1da4d5b932f\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 17 17:35:44.241705 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:35:44.241653    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/16a894e3-8971-45a7-9f5e-c1da4d5b932f-config-volume\") pod \"alertmanager-main-0\" (UID: \"16a894e3-8971-45a7-9f5e-c1da4d5b932f\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 17 17:35:44.243890 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:35:44.243860    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/16a894e3-8971-45a7-9f5e-c1da4d5b932f-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"16a894e3-8971-45a7-9f5e-c1da4d5b932f\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 17 17:35:44.243970 ip-10-0-137-20 kubenswrapper[2565]: W0917 17:35:44.243950    2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd4a0864d_cd66_4cd1_9e93_4f39b40bf3c3.slice/crio-167baff96f2713fa88176b250b7ccde78b33f5eeccddc0848630ec899ffa2c1b WatchSource:0}: Error finding container 167baff96f2713fa88176b250b7ccde78b33f5eeccddc0848630ec899ffa2c1b: Status 404 returned error can't find the container with id 167baff96f2713fa88176b250b7ccde78b33f5eeccddc0848630ec899ffa2c1b
Sep 17 17:35:44.244712 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:35:44.244690    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/16a894e3-8971-45a7-9f5e-c1da4d5b932f-tls-assets\") pod \"alertmanager-main-0\" (UID: \"16a894e3-8971-45a7-9f5e-c1da4d5b932f\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 17 17:35:44.245067 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:35:44.245046    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/16a894e3-8971-45a7-9f5e-c1da4d5b932f-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"16a894e3-8971-45a7-9f5e-c1da4d5b932f\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 17 17:35:44.245225 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:35:44.245185    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/16a894e3-8971-45a7-9f5e-c1da4d5b932f-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"16a894e3-8971-45a7-9f5e-c1da4d5b932f\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 17 17:35:44.246059 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:35:44.245985    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/16a894e3-8971-45a7-9f5e-c1da4d5b932f-config-out\") pod \"alertmanager-main-0\" (UID: \"16a894e3-8971-45a7-9f5e-c1da4d5b932f\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 17 17:35:44.246158 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:35:44.246096    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/16a894e3-8971-45a7-9f5e-c1da4d5b932f-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"16a894e3-8971-45a7-9f5e-c1da4d5b932f\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 17 17:35:44.246392 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:35:44.246369    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/16a894e3-8971-45a7-9f5e-c1da4d5b932f-web-config\") pod \"alertmanager-main-0\" (UID: \"16a894e3-8971-45a7-9f5e-c1da4d5b932f\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 17 17:35:44.246646 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:35:44.246625    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/16a894e3-8971-45a7-9f5e-c1da4d5b932f-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"16a894e3-8971-45a7-9f5e-c1da4d5b932f\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 17 17:35:44.247356 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:35:44.247336    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/16a894e3-8971-45a7-9f5e-c1da4d5b932f-config-volume\") pod \"alertmanager-main-0\" (UID: \"16a894e3-8971-45a7-9f5e-c1da4d5b932f\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 17 17:35:44.254456 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:35:44.254440    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-5m6k9\" (UniqueName: \"kubernetes.io/projected/16a894e3-8971-45a7-9f5e-c1da4d5b932f-kube-api-access-5m6k9\") pod \"alertmanager-main-0\" (UID: \"16a894e3-8971-45a7-9f5e-c1da4d5b932f\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 17 17:35:44.550038 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:35:44.549944    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/openshift-state-metrics-86c5bc79d6-28pph" event={"ID":"d4a0864d-cd66-4cd1-9e93-4f39b40bf3c3","Type":"ContainerStarted","Data":"dadd90604643f57d2635daadeb57efd53b1ec08fcd3b43d66e427f1ca91c6cc1"}
Sep 17 17:35:44.550038 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:35:44.550001    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/openshift-state-metrics-86c5bc79d6-28pph" event={"ID":"d4a0864d-cd66-4cd1-9e93-4f39b40bf3c3","Type":"ContainerStarted","Data":"ef08db25ca64fbfacdd3091c15ee62d5bac2e7ac2c277750fd86ef76e1ab6878"}
Sep 17 17:35:44.550038 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:35:44.550037    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/openshift-state-metrics-86c5bc79d6-28pph" event={"ID":"d4a0864d-cd66-4cd1-9e93-4f39b40bf3c3","Type":"ContainerStarted","Data":"167baff96f2713fa88176b250b7ccde78b33f5eeccddc0848630ec899ffa2c1b"}
Sep 17 17:35:44.746139 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:35:44.746108    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/16a894e3-8971-45a7-9f5e-c1da4d5b932f-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"16a894e3-8971-45a7-9f5e-c1da4d5b932f\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 17 17:35:44.748821 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:35:44.748798    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/16a894e3-8971-45a7-9f5e-c1da4d5b932f-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"16a894e3-8971-45a7-9f5e-c1da4d5b932f\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 17 17:35:44.987518 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:35:44.987490    2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-87747d4df-tk8xw"
Sep 17 17:35:44.987665 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:35:44.987564    2565 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-87747d4df-tk8xw"
Sep 17 17:35:44.994099 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:35:44.994074    2565 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-87747d4df-tk8xw"
Sep 17 17:35:45.001148 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:35:45.001126    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0"
Sep 17 17:35:45.084296 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:35:45.084232    2565 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-6d8f854898-t959r"]
Sep 17 17:35:45.088567 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:35:45.088547    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-6d8f854898-t959r"
Sep 17 17:35:45.097128 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:35:45.097103    2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"trusted-ca-bundle\""
Sep 17 17:35:45.100691 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:35:45.100662    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-6d8f854898-t959r"]
Sep 17 17:35:45.148961 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:35:45.148935    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/a416f0c6-706f-48ae-a806-48c7fc5b0171-trusted-ca-bundle\") pod \"console-6d8f854898-t959r\" (UID: \"a416f0c6-706f-48ae-a806-48c7fc5b0171\") " pod="openshift-console/console-6d8f854898-t959r"
Sep 17 17:35:45.149136 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:35:45.148970    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/a416f0c6-706f-48ae-a806-48c7fc5b0171-service-ca\") pod \"console-6d8f854898-t959r\" (UID: \"a416f0c6-706f-48ae-a806-48c7fc5b0171\") " pod="openshift-console/console-6d8f854898-t959r"
Sep 17 17:35:45.149136 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:35:45.149012    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/a416f0c6-706f-48ae-a806-48c7fc5b0171-console-serving-cert\") pod \"console-6d8f854898-t959r\" (UID: \"a416f0c6-706f-48ae-a806-48c7fc5b0171\") " pod="openshift-console/console-6d8f854898-t959r"
Sep 17 17:35:45.149136 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:35:45.149056    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/a416f0c6-706f-48ae-a806-48c7fc5b0171-console-config\") pod \"console-6d8f854898-t959r\" (UID: \"a416f0c6-706f-48ae-a806-48c7fc5b0171\") " pod="openshift-console/console-6d8f854898-t959r"
Sep 17 17:35:45.149136 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:35:45.149076    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-pvlqr\" (UniqueName: \"kubernetes.io/projected/a416f0c6-706f-48ae-a806-48c7fc5b0171-kube-api-access-pvlqr\") pod \"console-6d8f854898-t959r\" (UID: \"a416f0c6-706f-48ae-a806-48c7fc5b0171\") " pod="openshift-console/console-6d8f854898-t959r"
Sep 17 17:35:45.149136 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:35:45.149099    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/a416f0c6-706f-48ae-a806-48c7fc5b0171-console-oauth-config\") pod \"console-6d8f854898-t959r\" (UID: \"a416f0c6-706f-48ae-a806-48c7fc5b0171\") " pod="openshift-console/console-6d8f854898-t959r"
Sep 17 17:35:45.149136 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:35:45.149122    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/a416f0c6-706f-48ae-a806-48c7fc5b0171-oauth-serving-cert\") pod \"console-6d8f854898-t959r\" (UID: \"a416f0c6-706f-48ae-a806-48c7fc5b0171\") " pod="openshift-console/console-6d8f854898-t959r"
Sep 17 17:35:45.249690 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:35:45.249652    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/a416f0c6-706f-48ae-a806-48c7fc5b0171-console-serving-cert\") pod \"console-6d8f854898-t959r\" (UID: \"a416f0c6-706f-48ae-a806-48c7fc5b0171\") " pod="openshift-console/console-6d8f854898-t959r"
Sep 17 17:35:45.249690 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:35:45.249700    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/a416f0c6-706f-48ae-a806-48c7fc5b0171-console-config\") pod \"console-6d8f854898-t959r\" (UID: \"a416f0c6-706f-48ae-a806-48c7fc5b0171\") " pod="openshift-console/console-6d8f854898-t959r"
Sep 17 17:35:45.249914 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:35:45.249723    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-pvlqr\" (UniqueName: \"kubernetes.io/projected/a416f0c6-706f-48ae-a806-48c7fc5b0171-kube-api-access-pvlqr\") pod \"console-6d8f854898-t959r\" (UID: \"a416f0c6-706f-48ae-a806-48c7fc5b0171\") " pod="openshift-console/console-6d8f854898-t959r"
Sep 17 17:35:45.249914 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:35:45.249754    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/a416f0c6-706f-48ae-a806-48c7fc5b0171-console-oauth-config\") pod \"console-6d8f854898-t959r\" (UID: \"a416f0c6-706f-48ae-a806-48c7fc5b0171\") " pod="openshift-console/console-6d8f854898-t959r"
Sep 17 17:35:45.249914 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:35:45.249772    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/a416f0c6-706f-48ae-a806-48c7fc5b0171-oauth-serving-cert\") pod \"console-6d8f854898-t959r\" (UID: \"a416f0c6-706f-48ae-a806-48c7fc5b0171\") " pod="openshift-console/console-6d8f854898-t959r"
Sep 17 17:35:45.249914 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:35:45.249814    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/a416f0c6-706f-48ae-a806-48c7fc5b0171-trusted-ca-bundle\") pod \"console-6d8f854898-t959r\" (UID: \"a416f0c6-706f-48ae-a806-48c7fc5b0171\") " pod="openshift-console/console-6d8f854898-t959r"
Sep 17 17:35:45.249914 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:35:45.249838    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/a416f0c6-706f-48ae-a806-48c7fc5b0171-service-ca\") pod \"console-6d8f854898-t959r\" (UID: \"a416f0c6-706f-48ae-a806-48c7fc5b0171\") " pod="openshift-console/console-6d8f854898-t959r"
Sep 17 17:35:45.250582 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:35:45.250519    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/a416f0c6-706f-48ae-a806-48c7fc5b0171-console-config\") pod \"console-6d8f854898-t959r\" (UID: \"a416f0c6-706f-48ae-a806-48c7fc5b0171\") " pod="openshift-console/console-6d8f854898-t959r"
Sep 17 17:35:45.250582 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:35:45.250547    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/a416f0c6-706f-48ae-a806-48c7fc5b0171-oauth-serving-cert\") pod \"console-6d8f854898-t959r\" (UID: \"a416f0c6-706f-48ae-a806-48c7fc5b0171\") " pod="openshift-console/console-6d8f854898-t959r"
Sep 17 17:35:45.250732 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:35:45.250605    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/a416f0c6-706f-48ae-a806-48c7fc5b0171-service-ca\") pod \"console-6d8f854898-t959r\" (UID: \"a416f0c6-706f-48ae-a806-48c7fc5b0171\") " pod="openshift-console/console-6d8f854898-t959r"
Sep 17 17:35:45.250835 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:35:45.250802    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/a416f0c6-706f-48ae-a806-48c7fc5b0171-trusted-ca-bundle\") pod \"console-6d8f854898-t959r\" (UID: \"a416f0c6-706f-48ae-a806-48c7fc5b0171\") " pod="openshift-console/console-6d8f854898-t959r"
Sep 17 17:35:45.253150 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:35:45.253126    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/a416f0c6-706f-48ae-a806-48c7fc5b0171-console-oauth-config\") pod \"console-6d8f854898-t959r\" (UID: \"a416f0c6-706f-48ae-a806-48c7fc5b0171\") " pod="openshift-console/console-6d8f854898-t959r"
Sep 17 17:35:45.253242 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:35:45.253133    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/a416f0c6-706f-48ae-a806-48c7fc5b0171-console-serving-cert\") pod \"console-6d8f854898-t959r\" (UID: \"a416f0c6-706f-48ae-a806-48c7fc5b0171\") " pod="openshift-console/console-6d8f854898-t959r"
Sep 17 17:35:45.259518 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:35:45.259496    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-pvlqr\" (UniqueName: \"kubernetes.io/projected/a416f0c6-706f-48ae-a806-48c7fc5b0171-kube-api-access-pvlqr\") pod \"console-6d8f854898-t959r\" (UID: \"a416f0c6-706f-48ae-a806-48c7fc5b0171\") " pod="openshift-console/console-6d8f854898-t959r"
Sep 17 17:35:45.293237 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:35:45.293215    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/alertmanager-main-0"]
Sep 17 17:35:45.347395 ip-10-0-137-20 kubenswrapper[2565]: W0917 17:35:45.347333    2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod16a894e3_8971_45a7_9f5e_c1da4d5b932f.slice/crio-2383f71041ca396917229686da03a57e468e9a524324ee334cf1452d4f9098d7 WatchSource:0}: Error finding container 2383f71041ca396917229686da03a57e468e9a524324ee334cf1452d4f9098d7: Status 404 returned error can't find the container with id 2383f71041ca396917229686da03a57e468e9a524324ee334cf1452d4f9098d7
Sep 17 17:35:45.400044 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:35:45.400006    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-6d8f854898-t959r"
Sep 17 17:35:45.556865 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:35:45.556820    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/openshift-state-metrics-86c5bc79d6-28pph" event={"ID":"d4a0864d-cd66-4cd1-9e93-4f39b40bf3c3","Type":"ContainerStarted","Data":"1f3cd5046b4c0a8eccc481c32857fccdae1147e858a3193568961d29b829118e"}
Sep 17 17:35:45.558421 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:35:45.558396    2565 generic.go:356] "Generic (PLEG): container finished" podID="2e8b2e4c-5079-4454-a69a-9a183b1c1014" containerID="c404640d84902f4ebcbe1a0dd2ec7fd0d1a3c9367dcb2b7dc02c94cf59cc0e04" exitCode=0
Sep 17 17:35:45.558539 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:35:45.558458    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-rkh9d" event={"ID":"2e8b2e4c-5079-4454-a69a-9a183b1c1014","Type":"ContainerDied","Data":"c404640d84902f4ebcbe1a0dd2ec7fd0d1a3c9367dcb2b7dc02c94cf59cc0e04"}
Sep 17 17:35:45.559769 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:35:45.559751    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"16a894e3-8971-45a7-9f5e-c1da4d5b932f","Type":"ContainerStarted","Data":"2383f71041ca396917229686da03a57e468e9a524324ee334cf1452d4f9098d7"}
Sep 17 17:35:45.563837 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:35:45.563820    2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-87747d4df-tk8xw"
Sep 17 17:35:45.579210 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:35:45.579169    2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/openshift-state-metrics-86c5bc79d6-28pph" podStartSLOduration=2.580112864 podStartE2EDuration="3.579152761s" totalImagesPullingTime="999.039897ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-17 17:35:42 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-17 17:35:45.578622618 +0000 UTC m=+95.455907033" watchObservedRunningTime="2026-09-17 17:35:45.579152761 +0000 UTC m=+95.456437176"
Sep 17 17:35:45.677939 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:35:45.677915    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-6d8f854898-t959r"]
Sep 17 17:35:45.680256 ip-10-0-137-20 kubenswrapper[2565]: W0917 17:35:45.680225    2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda416f0c6_706f_48ae_a806_48c7fc5b0171.slice/crio-ef05f077f87c7fd92fdd0c046a9bd14b0a0fde2452ad47e8012c221abc1dbb9c WatchSource:0}: Error finding container ef05f077f87c7fd92fdd0c046a9bd14b0a0fde2452ad47e8012c221abc1dbb9c: Status 404 returned error can't find the container with id ef05f077f87c7fd92fdd0c046a9bd14b0a0fde2452ad47e8012c221abc1dbb9c
Sep 17 17:35:46.564979 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:35:46.564941    2565 generic.go:356] "Generic (PLEG): container finished" podID="16a894e3-8971-45a7-9f5e-c1da4d5b932f" containerID="39869e0624da33959e6dd5262faea8c3f150bfd6fa4b8e932781017c7e556cf3" exitCode=0
Sep 17 17:35:46.565425 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:35:46.565052    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"16a894e3-8971-45a7-9f5e-c1da4d5b932f","Type":"ContainerDied","Data":"39869e0624da33959e6dd5262faea8c3f150bfd6fa4b8e932781017c7e556cf3"}
Sep 17 17:35:46.566852 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:35:46.566830    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-6d8f854898-t959r" event={"ID":"a416f0c6-706f-48ae-a806-48c7fc5b0171","Type":"ContainerStarted","Data":"58a3238da0d06cda597df7483c2b2a388acb957633e3d857a8fb38eecea390b0"}
Sep 17 17:35:46.566941 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:35:46.566871    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-6d8f854898-t959r" event={"ID":"a416f0c6-706f-48ae-a806-48c7fc5b0171","Type":"ContainerStarted","Data":"ef05f077f87c7fd92fdd0c046a9bd14b0a0fde2452ad47e8012c221abc1dbb9c"}
Sep 17 17:35:46.571890 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:35:46.571869    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-rkh9d" event={"ID":"2e8b2e4c-5079-4454-a69a-9a183b1c1014","Type":"ContainerStarted","Data":"a395dbbb9d2b72b7d45ee853a61e8b9734a50887a108c70636d4e05f3ccbc97c"}
Sep 17 17:35:46.571990 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:35:46.571918    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-rkh9d" event={"ID":"2e8b2e4c-5079-4454-a69a-9a183b1c1014","Type":"ContainerStarted","Data":"c3f49e0e1e76aaf3dfa2810fd014434b1e98941114d153a68a748842a50a190f"}
Sep 17 17:35:46.634820 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:35:46.634764    2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-6d8f854898-t959r" podStartSLOduration=1.6347478469999999 podStartE2EDuration="1.634747847s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-17 17:35:45 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-17 17:35:46.629527101 +0000 UTC m=+96.506811516" watchObservedRunningTime="2026-09-17 17:35:46.634747847 +0000 UTC m=+96.512032261"
Sep 17 17:35:46.661798 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:35:46.661709    2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/node-exporter-rkh9d" podStartSLOduration=2.287266069 podStartE2EDuration="3.661695318s" totalImagesPullingTime="1.309423174s" totalInitContainerRuntime="65.006075ms" isStatefulPod=true podCreationTimestamp="2026-09-17 17:35:43 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-17 17:35:46.660640036 +0000 UTC m=+96.537924450" watchObservedRunningTime="2026-09-17 17:35:46.661695318 +0000 UTC m=+96.538979733"
Sep 17 17:35:48.583330 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:35:48.583248    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"16a894e3-8971-45a7-9f5e-c1da4d5b932f","Type":"ContainerStarted","Data":"a5e3d87b9a19c1adb40dbef70d3e60284afaf4a6b69bf01cac006949ec723835"}
Sep 17 17:35:48.583330 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:35:48.583309    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"16a894e3-8971-45a7-9f5e-c1da4d5b932f","Type":"ContainerStarted","Data":"134ec427f4263789d20f4763dcb06daf17337a7ca9916131c5f088039ad48699"}
Sep 17 17:35:48.583789 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:35:48.583335    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"16a894e3-8971-45a7-9f5e-c1da4d5b932f","Type":"ContainerStarted","Data":"b5e5e1a34891738d9b7dd1ad20f72ed0aa99dfeca3ac4b8c31c59db926925785"}
Sep 17 17:35:48.583789 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:35:48.583358    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"16a894e3-8971-45a7-9f5e-c1da4d5b932f","Type":"ContainerStarted","Data":"a30feba4b2557cb4bec23b611e8fa094a26c018639d581f94fbb2d866ee19089"}
Sep 17 17:35:48.583789 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:35:48.583381    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"16a894e3-8971-45a7-9f5e-c1da4d5b932f","Type":"ContainerStarted","Data":"664bc8312cf37b1f15381bb1251f7da8cec4ed651408d41becdbef57f97230bd"}
Sep 17 17:35:49.591982 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:35:49.591881    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"16a894e3-8971-45a7-9f5e-c1da4d5b932f","Type":"ContainerStarted","Data":"071a64b8ea3daeaea63a644fa08c4629e8aed942101e35e2a19a5125ea9264e0"}
Sep 17 17:35:49.629890 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:35:49.629833    2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/alertmanager-main-0" podStartSLOduration=2.237075593 podStartE2EDuration="5.629817773s" totalImagesPullingTime="3.377363433s" totalInitContainerRuntime="15.378747ms" isStatefulPod=false podCreationTimestamp="2026-09-17 17:35:44 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-17 17:35:49.627929894 +0000 UTC m=+99.505214310" watchObservedRunningTime="2026-09-17 17:35:49.629817773 +0000 UTC m=+99.507102189"
Sep 17 17:35:54.458243 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:35:54.458212    2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-network-diagnostics/network-check-target-jsc2p"
Sep 17 17:35:55.093878 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:35:55.093832    2565 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-6d8f854898-t959r"]
Sep 17 17:35:55.134083 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:35:55.134052    2565 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-7b8db48b-rv5v5"]
Sep 17 17:35:55.154743 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:35:55.154716    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-7b8db48b-rv5v5"]
Sep 17 17:35:55.154897 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:35:55.154820    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-7b8db48b-rv5v5"
Sep 17 17:35:55.235395 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:35:55.235358    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/7e66f8c1-3440-4ab4-8cef-1c5f2fe893c4-trusted-ca-bundle\") pod \"console-7b8db48b-rv5v5\" (UID: \"7e66f8c1-3440-4ab4-8cef-1c5f2fe893c4\") " pod="openshift-console/console-7b8db48b-rv5v5"
Sep 17 17:35:55.235395 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:35:55.235399    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/7e66f8c1-3440-4ab4-8cef-1c5f2fe893c4-oauth-serving-cert\") pod \"console-7b8db48b-rv5v5\" (UID: \"7e66f8c1-3440-4ab4-8cef-1c5f2fe893c4\") " pod="openshift-console/console-7b8db48b-rv5v5"
Sep 17 17:35:55.235638 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:35:55.235472    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/7e66f8c1-3440-4ab4-8cef-1c5f2fe893c4-service-ca\") pod \"console-7b8db48b-rv5v5\" (UID: \"7e66f8c1-3440-4ab4-8cef-1c5f2fe893c4\") " pod="openshift-console/console-7b8db48b-rv5v5"
Sep 17 17:35:55.235638 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:35:55.235513    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-77h6g\" (UniqueName: \"kubernetes.io/projected/7e66f8c1-3440-4ab4-8cef-1c5f2fe893c4-kube-api-access-77h6g\") pod \"console-7b8db48b-rv5v5\" (UID: \"7e66f8c1-3440-4ab4-8cef-1c5f2fe893c4\") " pod="openshift-console/console-7b8db48b-rv5v5"
Sep 17 17:35:55.235638 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:35:55.235534    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/7e66f8c1-3440-4ab4-8cef-1c5f2fe893c4-console-config\") pod \"console-7b8db48b-rv5v5\" (UID: \"7e66f8c1-3440-4ab4-8cef-1c5f2fe893c4\") " pod="openshift-console/console-7b8db48b-rv5v5"
Sep 17 17:35:55.235638 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:35:55.235551    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/7e66f8c1-3440-4ab4-8cef-1c5f2fe893c4-console-oauth-config\") pod \"console-7b8db48b-rv5v5\" (UID: \"7e66f8c1-3440-4ab4-8cef-1c5f2fe893c4\") " pod="openshift-console/console-7b8db48b-rv5v5"
Sep 17 17:35:55.235638 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:35:55.235572    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/7e66f8c1-3440-4ab4-8cef-1c5f2fe893c4-console-serving-cert\") pod \"console-7b8db48b-rv5v5\" (UID: \"7e66f8c1-3440-4ab4-8cef-1c5f2fe893c4\") " pod="openshift-console/console-7b8db48b-rv5v5"
Sep 17 17:35:55.336509 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:35:55.336473    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-77h6g\" (UniqueName: \"kubernetes.io/projected/7e66f8c1-3440-4ab4-8cef-1c5f2fe893c4-kube-api-access-77h6g\") pod \"console-7b8db48b-rv5v5\" (UID: \"7e66f8c1-3440-4ab4-8cef-1c5f2fe893c4\") " pod="openshift-console/console-7b8db48b-rv5v5"
Sep 17 17:35:55.336509 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:35:55.336510    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/7e66f8c1-3440-4ab4-8cef-1c5f2fe893c4-console-config\") pod \"console-7b8db48b-rv5v5\" (UID: \"7e66f8c1-3440-4ab4-8cef-1c5f2fe893c4\") " pod="openshift-console/console-7b8db48b-rv5v5"
Sep 17 17:35:55.336759 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:35:55.336529    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/7e66f8c1-3440-4ab4-8cef-1c5f2fe893c4-console-oauth-config\") pod \"console-7b8db48b-rv5v5\" (UID: \"7e66f8c1-3440-4ab4-8cef-1c5f2fe893c4\") " pod="openshift-console/console-7b8db48b-rv5v5"
Sep 17 17:35:55.336759 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:35:55.336547    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/7e66f8c1-3440-4ab4-8cef-1c5f2fe893c4-console-serving-cert\") pod \"console-7b8db48b-rv5v5\" (UID: \"7e66f8c1-3440-4ab4-8cef-1c5f2fe893c4\") " pod="openshift-console/console-7b8db48b-rv5v5"
Sep 17 17:35:55.336759 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:35:55.336590    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/7e66f8c1-3440-4ab4-8cef-1c5f2fe893c4-trusted-ca-bundle\") pod \"console-7b8db48b-rv5v5\" (UID: \"7e66f8c1-3440-4ab4-8cef-1c5f2fe893c4\") " pod="openshift-console/console-7b8db48b-rv5v5"
Sep 17 17:35:55.336759 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:35:55.336605    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/7e66f8c1-3440-4ab4-8cef-1c5f2fe893c4-oauth-serving-cert\") pod \"console-7b8db48b-rv5v5\" (UID: \"7e66f8c1-3440-4ab4-8cef-1c5f2fe893c4\") " pod="openshift-console/console-7b8db48b-rv5v5"
Sep 17 17:35:55.336759 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:35:55.336634    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/7e66f8c1-3440-4ab4-8cef-1c5f2fe893c4-service-ca\") pod \"console-7b8db48b-rv5v5\" (UID: \"7e66f8c1-3440-4ab4-8cef-1c5f2fe893c4\") " pod="openshift-console/console-7b8db48b-rv5v5"
Sep 17 17:35:55.337473 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:35:55.337432    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/7e66f8c1-3440-4ab4-8cef-1c5f2fe893c4-oauth-serving-cert\") pod \"console-7b8db48b-rv5v5\" (UID: \"7e66f8c1-3440-4ab4-8cef-1c5f2fe893c4\") " pod="openshift-console/console-7b8db48b-rv5v5"
Sep 17 17:35:55.337609 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:35:55.337431    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/7e66f8c1-3440-4ab4-8cef-1c5f2fe893c4-console-config\") pod \"console-7b8db48b-rv5v5\" (UID: \"7e66f8c1-3440-4ab4-8cef-1c5f2fe893c4\") " pod="openshift-console/console-7b8db48b-rv5v5"
Sep 17 17:35:55.337609 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:35:55.337595    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/7e66f8c1-3440-4ab4-8cef-1c5f2fe893c4-trusted-ca-bundle\") pod \"console-7b8db48b-rv5v5\" (UID: \"7e66f8c1-3440-4ab4-8cef-1c5f2fe893c4\") " pod="openshift-console/console-7b8db48b-rv5v5"
Sep 17 17:35:55.337723 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:35:55.337633    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/7e66f8c1-3440-4ab4-8cef-1c5f2fe893c4-service-ca\") pod \"console-7b8db48b-rv5v5\" (UID: \"7e66f8c1-3440-4ab4-8cef-1c5f2fe893c4\") " pod="openshift-console/console-7b8db48b-rv5v5"
Sep 17 17:35:55.339235 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:35:55.339203    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/7e66f8c1-3440-4ab4-8cef-1c5f2fe893c4-console-serving-cert\") pod \"console-7b8db48b-rv5v5\" (UID: \"7e66f8c1-3440-4ab4-8cef-1c5f2fe893c4\") " pod="openshift-console/console-7b8db48b-rv5v5"
Sep 17 17:35:55.339347 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:35:55.339262    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/7e66f8c1-3440-4ab4-8cef-1c5f2fe893c4-console-oauth-config\") pod \"console-7b8db48b-rv5v5\" (UID: \"7e66f8c1-3440-4ab4-8cef-1c5f2fe893c4\") " pod="openshift-console/console-7b8db48b-rv5v5"
Sep 17 17:35:55.367707 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:35:55.367683    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-77h6g\" (UniqueName: \"kubernetes.io/projected/7e66f8c1-3440-4ab4-8cef-1c5f2fe893c4-kube-api-access-77h6g\") pod \"console-7b8db48b-rv5v5\" (UID: \"7e66f8c1-3440-4ab4-8cef-1c5f2fe893c4\") " pod="openshift-console/console-7b8db48b-rv5v5"
Sep 17 17:35:55.400510 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:35:55.400479    2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-6d8f854898-t959r"
Sep 17 17:35:55.464297 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:35:55.464268    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-7b8db48b-rv5v5"
Sep 17 17:35:55.777820 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:35:55.777746    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-7b8db48b-rv5v5"]
Sep 17 17:35:55.779895 ip-10-0-137-20 kubenswrapper[2565]: W0917 17:35:55.779866    2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7e66f8c1_3440_4ab4_8cef_1c5f2fe893c4.slice/crio-9d6ebde33baacfa32cb81bd556604001fb843cd576044d849086d4a992431bfd WatchSource:0}: Error finding container 9d6ebde33baacfa32cb81bd556604001fb843cd576044d849086d4a992431bfd: Status 404 returned error can't find the container with id 9d6ebde33baacfa32cb81bd556604001fb843cd576044d849086d4a992431bfd
Sep 17 17:35:56.186037 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:35:56.185967    2565 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-image-registry/image-registry-b79cf788d-x7pgt" podUID="9ab56083-4127-4bbb-af70-4ef55006fe47" containerName="registry" containerID="cri-o://dee6ce132d7c702c88549217cc6afaa0793e14697e844bc6ba3ba5c3d1e21ea4" gracePeriod=30
Sep 17 17:35:56.420364 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:35:56.420341    2565 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-b79cf788d-x7pgt"
Sep 17 17:35:56.549313 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:35:56.549227    2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/9ab56083-4127-4bbb-af70-4ef55006fe47-image-registry-private-configuration\") pod \"9ab56083-4127-4bbb-af70-4ef55006fe47\" (UID: \"9ab56083-4127-4bbb-af70-4ef55006fe47\") "
Sep 17 17:35:56.549313 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:35:56.549263    2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/9ab56083-4127-4bbb-af70-4ef55006fe47-bound-sa-token\") pod \"9ab56083-4127-4bbb-af70-4ef55006fe47\" (UID: \"9ab56083-4127-4bbb-af70-4ef55006fe47\") "
Sep 17 17:35:56.549313 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:35:56.549289    2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/9ab56083-4127-4bbb-af70-4ef55006fe47-registry-tls\") pod \"9ab56083-4127-4bbb-af70-4ef55006fe47\" (UID: \"9ab56083-4127-4bbb-af70-4ef55006fe47\") "
Sep 17 17:35:56.549313 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:35:56.549315    2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/9ab56083-4127-4bbb-af70-4ef55006fe47-trusted-ca\") pod \"9ab56083-4127-4bbb-af70-4ef55006fe47\" (UID: \"9ab56083-4127-4bbb-af70-4ef55006fe47\") "
Sep 17 17:35:56.549925 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:35:56.549341    2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/9ab56083-4127-4bbb-af70-4ef55006fe47-registry-certificates\") pod \"9ab56083-4127-4bbb-af70-4ef55006fe47\" (UID: \"9ab56083-4127-4bbb-af70-4ef55006fe47\") "
Sep 17 17:35:56.549925 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:35:56.549363    2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-mhswd\" (UniqueName: \"kubernetes.io/projected/9ab56083-4127-4bbb-af70-4ef55006fe47-kube-api-access-mhswd\") pod \"9ab56083-4127-4bbb-af70-4ef55006fe47\" (UID: \"9ab56083-4127-4bbb-af70-4ef55006fe47\") "
Sep 17 17:35:56.549925 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:35:56.549398    2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/9ab56083-4127-4bbb-af70-4ef55006fe47-ca-trust-extracted\") pod \"9ab56083-4127-4bbb-af70-4ef55006fe47\" (UID: \"9ab56083-4127-4bbb-af70-4ef55006fe47\") "
Sep 17 17:35:56.549925 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:35:56.549505    2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/9ab56083-4127-4bbb-af70-4ef55006fe47-installation-pull-secrets\") pod \"9ab56083-4127-4bbb-af70-4ef55006fe47\" (UID: \"9ab56083-4127-4bbb-af70-4ef55006fe47\") "
Sep 17 17:35:56.549925 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:35:56.549756    2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/9ab56083-4127-4bbb-af70-4ef55006fe47-trusted-ca" (OuterVolumeSpecName: "trusted-ca") pod "9ab56083-4127-4bbb-af70-4ef55006fe47" (UID: "9ab56083-4127-4bbb-af70-4ef55006fe47"). InnerVolumeSpecName "trusted-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 17 17:35:56.549925 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:35:56.549848    2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/9ab56083-4127-4bbb-af70-4ef55006fe47-registry-certificates" (OuterVolumeSpecName: "registry-certificates") pod "9ab56083-4127-4bbb-af70-4ef55006fe47" (UID: "9ab56083-4127-4bbb-af70-4ef55006fe47"). InnerVolumeSpecName "registry-certificates". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 17 17:35:56.551957 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:35:56.551932    2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/9ab56083-4127-4bbb-af70-4ef55006fe47-image-registry-private-configuration" (OuterVolumeSpecName: "image-registry-private-configuration") pod "9ab56083-4127-4bbb-af70-4ef55006fe47" (UID: "9ab56083-4127-4bbb-af70-4ef55006fe47"). InnerVolumeSpecName "image-registry-private-configuration". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 17 17:35:56.552110 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:35:56.552070    2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/9ab56083-4127-4bbb-af70-4ef55006fe47-installation-pull-secrets" (OuterVolumeSpecName: "installation-pull-secrets") pod "9ab56083-4127-4bbb-af70-4ef55006fe47" (UID: "9ab56083-4127-4bbb-af70-4ef55006fe47"). InnerVolumeSpecName "installation-pull-secrets". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 17 17:35:56.552179 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:35:56.552109    2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/9ab56083-4127-4bbb-af70-4ef55006fe47-bound-sa-token" (OuterVolumeSpecName: "bound-sa-token") pod "9ab56083-4127-4bbb-af70-4ef55006fe47" (UID: "9ab56083-4127-4bbb-af70-4ef55006fe47"). InnerVolumeSpecName "bound-sa-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 17 17:35:56.552179 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:35:56.552173    2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/9ab56083-4127-4bbb-af70-4ef55006fe47-registry-tls" (OuterVolumeSpecName: "registry-tls") pod "9ab56083-4127-4bbb-af70-4ef55006fe47" (UID: "9ab56083-4127-4bbb-af70-4ef55006fe47"). InnerVolumeSpecName "registry-tls". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 17 17:35:56.552299 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:35:56.552274    2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/9ab56083-4127-4bbb-af70-4ef55006fe47-kube-api-access-mhswd" (OuterVolumeSpecName: "kube-api-access-mhswd") pod "9ab56083-4127-4bbb-af70-4ef55006fe47" (UID: "9ab56083-4127-4bbb-af70-4ef55006fe47"). InnerVolumeSpecName "kube-api-access-mhswd". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 17 17:35:56.558296 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:35:56.558274    2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/9ab56083-4127-4bbb-af70-4ef55006fe47-ca-trust-extracted" (OuterVolumeSpecName: "ca-trust-extracted") pod "9ab56083-4127-4bbb-af70-4ef55006fe47" (UID: "9ab56083-4127-4bbb-af70-4ef55006fe47"). InnerVolumeSpecName "ca-trust-extracted". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 17 17:35:56.623558 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:35:56.623522    2565 generic.go:356] "Generic (PLEG): container finished" podID="9ab56083-4127-4bbb-af70-4ef55006fe47" containerID="dee6ce132d7c702c88549217cc6afaa0793e14697e844bc6ba3ba5c3d1e21ea4" exitCode=0
Sep 17 17:35:56.623751 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:35:56.623677    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-b79cf788d-x7pgt" event={"ID":"9ab56083-4127-4bbb-af70-4ef55006fe47","Type":"ContainerDied","Data":"dee6ce132d7c702c88549217cc6afaa0793e14697e844bc6ba3ba5c3d1e21ea4"}
Sep 17 17:35:56.623818 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:35:56.623767    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-b79cf788d-x7pgt" event={"ID":"9ab56083-4127-4bbb-af70-4ef55006fe47","Type":"ContainerDied","Data":"db3867a0e90cec74c6bb6c2542e5f7b604a65777be5f50bd0f28eb96e943696f"}
Sep 17 17:35:56.623818 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:35:56.623806    2565 scope.go:117] "RemoveContainer" containerID="dee6ce132d7c702c88549217cc6afaa0793e14697e844bc6ba3ba5c3d1e21ea4"
Sep 17 17:35:56.624441 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:35:56.623999    2565 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-b79cf788d-x7pgt"
Sep 17 17:35:56.628205 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:35:56.628175    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-7b8db48b-rv5v5" event={"ID":"7e66f8c1-3440-4ab4-8cef-1c5f2fe893c4","Type":"ContainerStarted","Data":"a5f3cbc93af022831ef9950368b40b5296cbca3b5287d6672a89425cb84331f5"}
Sep 17 17:35:56.628309 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:35:56.628219    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-7b8db48b-rv5v5" event={"ID":"7e66f8c1-3440-4ab4-8cef-1c5f2fe893c4","Type":"ContainerStarted","Data":"9d6ebde33baacfa32cb81bd556604001fb843cd576044d849086d4a992431bfd"}
Sep 17 17:35:56.636134 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:35:56.636117    2565 scope.go:117] "RemoveContainer" containerID="dee6ce132d7c702c88549217cc6afaa0793e14697e844bc6ba3ba5c3d1e21ea4"
Sep 17 17:35:56.636413 ip-10-0-137-20 kubenswrapper[2565]: E0917 17:35:56.636393    2565 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"dee6ce132d7c702c88549217cc6afaa0793e14697e844bc6ba3ba5c3d1e21ea4\": container with ID starting with dee6ce132d7c702c88549217cc6afaa0793e14697e844bc6ba3ba5c3d1e21ea4 not found: ID does not exist" containerID="dee6ce132d7c702c88549217cc6afaa0793e14697e844bc6ba3ba5c3d1e21ea4"
Sep 17 17:35:56.636472 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:35:56.636421    2565 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"dee6ce132d7c702c88549217cc6afaa0793e14697e844bc6ba3ba5c3d1e21ea4"} err="failed to get container status \"dee6ce132d7c702c88549217cc6afaa0793e14697e844bc6ba3ba5c3d1e21ea4\": rpc error: code = NotFound desc = could not find container \"dee6ce132d7c702c88549217cc6afaa0793e14697e844bc6ba3ba5c3d1e21ea4\": container with ID starting with dee6ce132d7c702c88549217cc6afaa0793e14697e844bc6ba3ba5c3d1e21ea4 not found: ID does not exist"
Sep 17 17:35:56.650848 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:35:56.650825    2565 reconciler_common.go:299] "Volume detached for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/9ab56083-4127-4bbb-af70-4ef55006fe47-installation-pull-secrets\") on node \"ip-10-0-137-20.ec2.internal\" DevicePath \"\""
Sep 17 17:35:56.650848 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:35:56.650849    2565 reconciler_common.go:299] "Volume detached for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/9ab56083-4127-4bbb-af70-4ef55006fe47-image-registry-private-configuration\") on node \"ip-10-0-137-20.ec2.internal\" DevicePath \"\""
Sep 17 17:35:56.650981 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:35:56.650862    2565 reconciler_common.go:299] "Volume detached for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/9ab56083-4127-4bbb-af70-4ef55006fe47-bound-sa-token\") on node \"ip-10-0-137-20.ec2.internal\" DevicePath \"\""
Sep 17 17:35:56.650981 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:35:56.650876    2565 reconciler_common.go:299] "Volume detached for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/9ab56083-4127-4bbb-af70-4ef55006fe47-registry-tls\") on node \"ip-10-0-137-20.ec2.internal\" DevicePath \"\""
Sep 17 17:35:56.650981 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:35:56.650883    2565 reconciler_common.go:299] "Volume detached for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/9ab56083-4127-4bbb-af70-4ef55006fe47-trusted-ca\") on node \"ip-10-0-137-20.ec2.internal\" DevicePath \"\""
Sep 17 17:35:56.650981 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:35:56.650892    2565 reconciler_common.go:299] "Volume detached for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/9ab56083-4127-4bbb-af70-4ef55006fe47-registry-certificates\") on node \"ip-10-0-137-20.ec2.internal\" DevicePath \"\""
Sep 17 17:35:56.650981 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:35:56.650899    2565 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-mhswd\" (UniqueName: \"kubernetes.io/projected/9ab56083-4127-4bbb-af70-4ef55006fe47-kube-api-access-mhswd\") on node \"ip-10-0-137-20.ec2.internal\" DevicePath \"\""
Sep 17 17:35:56.650981 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:35:56.650907    2565 reconciler_common.go:299] "Volume detached for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/9ab56083-4127-4bbb-af70-4ef55006fe47-ca-trust-extracted\") on node \"ip-10-0-137-20.ec2.internal\" DevicePath \"\""
Sep 17 17:35:56.653722 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:35:56.653681    2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-7b8db48b-rv5v5" podStartSLOduration=1.653666518 podStartE2EDuration="1.653666518s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-17 17:35:55 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-17 17:35:56.652149732 +0000 UTC m=+106.529434149" watchObservedRunningTime="2026-09-17 17:35:56.653666518 +0000 UTC m=+106.530950932"
Sep 17 17:35:56.673491 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:35:56.673470    2565 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-image-registry/image-registry-b79cf788d-x7pgt"]
Sep 17 17:35:56.676982 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:35:56.676963    2565 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-image-registry/image-registry-b79cf788d-x7pgt"]
Sep 17 17:35:58.155581 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:35:58.155549    2565 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="9ab56083-4127-4bbb-af70-4ef55006fe47" path="/var/lib/kubelet/pods/9ab56083-4127-4bbb-af70-4ef55006fe47/volumes"
Sep 17 17:36:05.464502 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:36:05.464465    2565 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-7b8db48b-rv5v5"
Sep 17 17:36:05.464890 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:36:05.464527    2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-7b8db48b-rv5v5"
Sep 17 17:36:05.469046 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:36:05.469006    2565 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-7b8db48b-rv5v5"
Sep 17 17:36:05.663321 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:36:05.663299    2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-7b8db48b-rv5v5"
Sep 17 17:36:05.720520 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:36:05.720459    2565 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-87747d4df-tk8xw"]
Sep 17 17:36:20.116297 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:36:20.116257    2565 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-console/console-6d8f854898-t959r" podUID="a416f0c6-706f-48ae-a806-48c7fc5b0171" containerName="console" containerID="cri-o://58a3238da0d06cda597df7483c2b2a388acb957633e3d857a8fb38eecea390b0" gracePeriod=15
Sep 17 17:36:20.351939 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:36:20.351917    2565 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-6d8f854898-t959r"
Sep 17 17:36:20.408838 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:36:20.408775    2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/a416f0c6-706f-48ae-a806-48c7fc5b0171-trusted-ca-bundle\") pod \"a416f0c6-706f-48ae-a806-48c7fc5b0171\" (UID: \"a416f0c6-706f-48ae-a806-48c7fc5b0171\") "
Sep 17 17:36:20.408838 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:36:20.408804    2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/a416f0c6-706f-48ae-a806-48c7fc5b0171-console-config\") pod \"a416f0c6-706f-48ae-a806-48c7fc5b0171\" (UID: \"a416f0c6-706f-48ae-a806-48c7fc5b0171\") "
Sep 17 17:36:20.409040 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:36:20.408853    2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/a416f0c6-706f-48ae-a806-48c7fc5b0171-console-oauth-config\") pod \"a416f0c6-706f-48ae-a806-48c7fc5b0171\" (UID: \"a416f0c6-706f-48ae-a806-48c7fc5b0171\") "
Sep 17 17:36:20.409040 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:36:20.408961    2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-pvlqr\" (UniqueName: \"kubernetes.io/projected/a416f0c6-706f-48ae-a806-48c7fc5b0171-kube-api-access-pvlqr\") pod \"a416f0c6-706f-48ae-a806-48c7fc5b0171\" (UID: \"a416f0c6-706f-48ae-a806-48c7fc5b0171\") "
Sep 17 17:36:20.409040 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:36:20.409017    2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/a416f0c6-706f-48ae-a806-48c7fc5b0171-service-ca\") pod \"a416f0c6-706f-48ae-a806-48c7fc5b0171\" (UID: \"a416f0c6-706f-48ae-a806-48c7fc5b0171\") "
Sep 17 17:36:20.409201 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:36:20.409062    2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/a416f0c6-706f-48ae-a806-48c7fc5b0171-console-serving-cert\") pod \"a416f0c6-706f-48ae-a806-48c7fc5b0171\" (UID: \"a416f0c6-706f-48ae-a806-48c7fc5b0171\") "
Sep 17 17:36:20.409201 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:36:20.409092    2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/a416f0c6-706f-48ae-a806-48c7fc5b0171-oauth-serving-cert\") pod \"a416f0c6-706f-48ae-a806-48c7fc5b0171\" (UID: \"a416f0c6-706f-48ae-a806-48c7fc5b0171\") "
Sep 17 17:36:20.409300 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:36:20.409242    2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/a416f0c6-706f-48ae-a806-48c7fc5b0171-console-config" (OuterVolumeSpecName: "console-config") pod "a416f0c6-706f-48ae-a806-48c7fc5b0171" (UID: "a416f0c6-706f-48ae-a806-48c7fc5b0171"). InnerVolumeSpecName "console-config". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 17 17:36:20.409300 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:36:20.409254    2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/a416f0c6-706f-48ae-a806-48c7fc5b0171-trusted-ca-bundle" (OuterVolumeSpecName: "trusted-ca-bundle") pod "a416f0c6-706f-48ae-a806-48c7fc5b0171" (UID: "a416f0c6-706f-48ae-a806-48c7fc5b0171"). InnerVolumeSpecName "trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 17 17:36:20.409385 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:36:20.409325    2565 reconciler_common.go:299] "Volume detached for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/a416f0c6-706f-48ae-a806-48c7fc5b0171-trusted-ca-bundle\") on node \"ip-10-0-137-20.ec2.internal\" DevicePath \"\""
Sep 17 17:36:20.409385 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:36:20.409342    2565 reconciler_common.go:299] "Volume detached for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/a416f0c6-706f-48ae-a806-48c7fc5b0171-console-config\") on node \"ip-10-0-137-20.ec2.internal\" DevicePath \"\""
Sep 17 17:36:20.409480 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:36:20.409449    2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/a416f0c6-706f-48ae-a806-48c7fc5b0171-service-ca" (OuterVolumeSpecName: "service-ca") pod "a416f0c6-706f-48ae-a806-48c7fc5b0171" (UID: "a416f0c6-706f-48ae-a806-48c7fc5b0171"). InnerVolumeSpecName "service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 17 17:36:20.409626 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:36:20.409596    2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/a416f0c6-706f-48ae-a806-48c7fc5b0171-oauth-serving-cert" (OuterVolumeSpecName: "oauth-serving-cert") pod "a416f0c6-706f-48ae-a806-48c7fc5b0171" (UID: "a416f0c6-706f-48ae-a806-48c7fc5b0171"). InnerVolumeSpecName "oauth-serving-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 17 17:36:20.411068 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:36:20.411043    2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/a416f0c6-706f-48ae-a806-48c7fc5b0171-console-oauth-config" (OuterVolumeSpecName: "console-oauth-config") pod "a416f0c6-706f-48ae-a806-48c7fc5b0171" (UID: "a416f0c6-706f-48ae-a806-48c7fc5b0171"). InnerVolumeSpecName "console-oauth-config". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 17 17:36:20.411138 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:36:20.411093    2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/a416f0c6-706f-48ae-a806-48c7fc5b0171-console-serving-cert" (OuterVolumeSpecName: "console-serving-cert") pod "a416f0c6-706f-48ae-a806-48c7fc5b0171" (UID: "a416f0c6-706f-48ae-a806-48c7fc5b0171"). InnerVolumeSpecName "console-serving-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 17 17:36:20.411138 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:36:20.411108    2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a416f0c6-706f-48ae-a806-48c7fc5b0171-kube-api-access-pvlqr" (OuterVolumeSpecName: "kube-api-access-pvlqr") pod "a416f0c6-706f-48ae-a806-48c7fc5b0171" (UID: "a416f0c6-706f-48ae-a806-48c7fc5b0171"). InnerVolumeSpecName "kube-api-access-pvlqr". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 17 17:36:20.509840 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:36:20.509808    2565 reconciler_common.go:299] "Volume detached for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/a416f0c6-706f-48ae-a806-48c7fc5b0171-console-oauth-config\") on node \"ip-10-0-137-20.ec2.internal\" DevicePath \"\""
Sep 17 17:36:20.509840 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:36:20.509836    2565 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-pvlqr\" (UniqueName: \"kubernetes.io/projected/a416f0c6-706f-48ae-a806-48c7fc5b0171-kube-api-access-pvlqr\") on node \"ip-10-0-137-20.ec2.internal\" DevicePath \"\""
Sep 17 17:36:20.510007 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:36:20.509850    2565 reconciler_common.go:299] "Volume detached for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/a416f0c6-706f-48ae-a806-48c7fc5b0171-service-ca\") on node \"ip-10-0-137-20.ec2.internal\" DevicePath \"\""
Sep 17 17:36:20.510007 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:36:20.509862    2565 reconciler_common.go:299] "Volume detached for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/a416f0c6-706f-48ae-a806-48c7fc5b0171-console-serving-cert\") on node \"ip-10-0-137-20.ec2.internal\" DevicePath \"\""
Sep 17 17:36:20.510007 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:36:20.509873    2565 reconciler_common.go:299] "Volume detached for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/a416f0c6-706f-48ae-a806-48c7fc5b0171-oauth-serving-cert\") on node \"ip-10-0-137-20.ec2.internal\" DevicePath \"\""
Sep 17 17:36:20.714218 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:36:20.714134    2565 generic.go:356] "Generic (PLEG): container finished" podID="a416f0c6-706f-48ae-a806-48c7fc5b0171" containerID="58a3238da0d06cda597df7483c2b2a388acb957633e3d857a8fb38eecea390b0" exitCode=0
Sep 17 17:36:20.714218 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:36:20.714184    2565 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-6d8f854898-t959r"
Sep 17 17:36:20.714439 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:36:20.714223    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-6d8f854898-t959r" event={"ID":"a416f0c6-706f-48ae-a806-48c7fc5b0171","Type":"ContainerDied","Data":"58a3238da0d06cda597df7483c2b2a388acb957633e3d857a8fb38eecea390b0"}
Sep 17 17:36:20.714439 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:36:20.714283    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-6d8f854898-t959r" event={"ID":"a416f0c6-706f-48ae-a806-48c7fc5b0171","Type":"ContainerDied","Data":"ef05f077f87c7fd92fdd0c046a9bd14b0a0fde2452ad47e8012c221abc1dbb9c"}
Sep 17 17:36:20.714439 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:36:20.714312    2565 scope.go:117] "RemoveContainer" containerID="58a3238da0d06cda597df7483c2b2a388acb957633e3d857a8fb38eecea390b0"
Sep 17 17:36:20.722542 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:36:20.722525    2565 scope.go:117] "RemoveContainer" containerID="58a3238da0d06cda597df7483c2b2a388acb957633e3d857a8fb38eecea390b0"
Sep 17 17:36:20.722791 ip-10-0-137-20 kubenswrapper[2565]: E0917 17:36:20.722775    2565 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"58a3238da0d06cda597df7483c2b2a388acb957633e3d857a8fb38eecea390b0\": container with ID starting with 58a3238da0d06cda597df7483c2b2a388acb957633e3d857a8fb38eecea390b0 not found: ID does not exist" containerID="58a3238da0d06cda597df7483c2b2a388acb957633e3d857a8fb38eecea390b0"
Sep 17 17:36:20.722832 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:36:20.722800    2565 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"58a3238da0d06cda597df7483c2b2a388acb957633e3d857a8fb38eecea390b0"} err="failed to get container status \"58a3238da0d06cda597df7483c2b2a388acb957633e3d857a8fb38eecea390b0\": rpc error: code = NotFound desc = could not find container \"58a3238da0d06cda597df7483c2b2a388acb957633e3d857a8fb38eecea390b0\": container with ID starting with 58a3238da0d06cda597df7483c2b2a388acb957633e3d857a8fb38eecea390b0 not found: ID does not exist"
Sep 17 17:36:20.741919 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:36:20.741895    2565 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-6d8f854898-t959r"]
Sep 17 17:36:20.745793 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:36:20.745774    2565 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-console/console-6d8f854898-t959r"]
Sep 17 17:36:22.155961 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:36:22.155923    2565 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="a416f0c6-706f-48ae-a806-48c7fc5b0171" path="/var/lib/kubelet/pods/a416f0c6-706f-48ae-a806-48c7fc5b0171/volumes"
Sep 17 17:36:30.743174 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:36:30.743127    2565 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-console/console-87747d4df-tk8xw" podUID="cedae061-1174-4bc8-b7b9-0d034929ddad" containerName="console" containerID="cri-o://3789759b52cab6aafcf0ce91d69c2d69ccd11246729fd93665b9a16d715537b0" gracePeriod=15
Sep 17 17:36:30.980876 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:36:30.980854    2565 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-87747d4df-tk8xw"
Sep 17 17:36:31.078723 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:36:31.078628    2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/cedae061-1174-4bc8-b7b9-0d034929ddad-console-serving-cert\") pod \"cedae061-1174-4bc8-b7b9-0d034929ddad\" (UID: \"cedae061-1174-4bc8-b7b9-0d034929ddad\") "
Sep 17 17:36:31.078723 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:36:31.078692    2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/cedae061-1174-4bc8-b7b9-0d034929ddad-oauth-serving-cert\") pod \"cedae061-1174-4bc8-b7b9-0d034929ddad\" (UID: \"cedae061-1174-4bc8-b7b9-0d034929ddad\") "
Sep 17 17:36:31.078723 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:36:31.078723    2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/cedae061-1174-4bc8-b7b9-0d034929ddad-service-ca\") pod \"cedae061-1174-4bc8-b7b9-0d034929ddad\" (UID: \"cedae061-1174-4bc8-b7b9-0d034929ddad\") "
Sep 17 17:36:31.078973 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:36:31.078745    2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/cedae061-1174-4bc8-b7b9-0d034929ddad-console-config\") pod \"cedae061-1174-4bc8-b7b9-0d034929ddad\" (UID: \"cedae061-1174-4bc8-b7b9-0d034929ddad\") "
Sep 17 17:36:31.078973 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:36:31.078762    2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-4pb2j\" (UniqueName: \"kubernetes.io/projected/cedae061-1174-4bc8-b7b9-0d034929ddad-kube-api-access-4pb2j\") pod \"cedae061-1174-4bc8-b7b9-0d034929ddad\" (UID: \"cedae061-1174-4bc8-b7b9-0d034929ddad\") "
Sep 17 17:36:31.078973 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:36:31.078780    2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/cedae061-1174-4bc8-b7b9-0d034929ddad-console-oauth-config\") pod \"cedae061-1174-4bc8-b7b9-0d034929ddad\" (UID: \"cedae061-1174-4bc8-b7b9-0d034929ddad\") "
Sep 17 17:36:31.079196 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:36:31.079154    2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/cedae061-1174-4bc8-b7b9-0d034929ddad-service-ca" (OuterVolumeSpecName: "service-ca") pod "cedae061-1174-4bc8-b7b9-0d034929ddad" (UID: "cedae061-1174-4bc8-b7b9-0d034929ddad"). InnerVolumeSpecName "service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 17 17:36:31.079196 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:36:31.079156    2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/cedae061-1174-4bc8-b7b9-0d034929ddad-console-config" (OuterVolumeSpecName: "console-config") pod "cedae061-1174-4bc8-b7b9-0d034929ddad" (UID: "cedae061-1174-4bc8-b7b9-0d034929ddad"). InnerVolumeSpecName "console-config". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 17 17:36:31.079316 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:36:31.079225    2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/cedae061-1174-4bc8-b7b9-0d034929ddad-oauth-serving-cert" (OuterVolumeSpecName: "oauth-serving-cert") pod "cedae061-1174-4bc8-b7b9-0d034929ddad" (UID: "cedae061-1174-4bc8-b7b9-0d034929ddad"). InnerVolumeSpecName "oauth-serving-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 17 17:36:31.080813 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:36:31.080783    2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/cedae061-1174-4bc8-b7b9-0d034929ddad-console-serving-cert" (OuterVolumeSpecName: "console-serving-cert") pod "cedae061-1174-4bc8-b7b9-0d034929ddad" (UID: "cedae061-1174-4bc8-b7b9-0d034929ddad"). InnerVolumeSpecName "console-serving-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 17 17:36:31.080920 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:36:31.080846    2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/cedae061-1174-4bc8-b7b9-0d034929ddad-console-oauth-config" (OuterVolumeSpecName: "console-oauth-config") pod "cedae061-1174-4bc8-b7b9-0d034929ddad" (UID: "cedae061-1174-4bc8-b7b9-0d034929ddad"). InnerVolumeSpecName "console-oauth-config". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 17 17:36:31.080920 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:36:31.080870    2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/cedae061-1174-4bc8-b7b9-0d034929ddad-kube-api-access-4pb2j" (OuterVolumeSpecName: "kube-api-access-4pb2j") pod "cedae061-1174-4bc8-b7b9-0d034929ddad" (UID: "cedae061-1174-4bc8-b7b9-0d034929ddad"). InnerVolumeSpecName "kube-api-access-4pb2j". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 17 17:36:31.180153 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:36:31.180125    2565 reconciler_common.go:299] "Volume detached for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/cedae061-1174-4bc8-b7b9-0d034929ddad-oauth-serving-cert\") on node \"ip-10-0-137-20.ec2.internal\" DevicePath \"\""
Sep 17 17:36:31.180153 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:36:31.180149    2565 reconciler_common.go:299] "Volume detached for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/cedae061-1174-4bc8-b7b9-0d034929ddad-service-ca\") on node \"ip-10-0-137-20.ec2.internal\" DevicePath \"\""
Sep 17 17:36:31.180153 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:36:31.180158    2565 reconciler_common.go:299] "Volume detached for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/cedae061-1174-4bc8-b7b9-0d034929ddad-console-config\") on node \"ip-10-0-137-20.ec2.internal\" DevicePath \"\""
Sep 17 17:36:31.180337 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:36:31.180166    2565 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-4pb2j\" (UniqueName: \"kubernetes.io/projected/cedae061-1174-4bc8-b7b9-0d034929ddad-kube-api-access-4pb2j\") on node \"ip-10-0-137-20.ec2.internal\" DevicePath \"\""
Sep 17 17:36:31.180337 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:36:31.180175    2565 reconciler_common.go:299] "Volume detached for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/cedae061-1174-4bc8-b7b9-0d034929ddad-console-oauth-config\") on node \"ip-10-0-137-20.ec2.internal\" DevicePath \"\""
Sep 17 17:36:31.180337 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:36:31.180183    2565 reconciler_common.go:299] "Volume detached for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/cedae061-1174-4bc8-b7b9-0d034929ddad-console-serving-cert\") on node \"ip-10-0-137-20.ec2.internal\" DevicePath \"\""
Sep 17 17:36:31.754917 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:36:31.754876    2565 generic.go:356] "Generic (PLEG): container finished" podID="cedae061-1174-4bc8-b7b9-0d034929ddad" containerID="3789759b52cab6aafcf0ce91d69c2d69ccd11246729fd93665b9a16d715537b0" exitCode=0
Sep 17 17:36:31.755340 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:36:31.754927    2565 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-87747d4df-tk8xw"
Sep 17 17:36:31.755340 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:36:31.754954    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-87747d4df-tk8xw" event={"ID":"cedae061-1174-4bc8-b7b9-0d034929ddad","Type":"ContainerDied","Data":"3789759b52cab6aafcf0ce91d69c2d69ccd11246729fd93665b9a16d715537b0"}
Sep 17 17:36:31.755340 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:36:31.755001    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-87747d4df-tk8xw" event={"ID":"cedae061-1174-4bc8-b7b9-0d034929ddad","Type":"ContainerDied","Data":"8d6fcc892e090b6d00361162e09aca3454f6a166ea0781b71e5c38b4d917eeaa"}
Sep 17 17:36:31.755340 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:36:31.755048    2565 scope.go:117] "RemoveContainer" containerID="3789759b52cab6aafcf0ce91d69c2d69ccd11246729fd93665b9a16d715537b0"
Sep 17 17:36:31.763240 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:36:31.763224    2565 scope.go:117] "RemoveContainer" containerID="3789759b52cab6aafcf0ce91d69c2d69ccd11246729fd93665b9a16d715537b0"
Sep 17 17:36:31.763495 ip-10-0-137-20 kubenswrapper[2565]: E0917 17:36:31.763468    2565 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"3789759b52cab6aafcf0ce91d69c2d69ccd11246729fd93665b9a16d715537b0\": container with ID starting with 3789759b52cab6aafcf0ce91d69c2d69ccd11246729fd93665b9a16d715537b0 not found: ID does not exist" containerID="3789759b52cab6aafcf0ce91d69c2d69ccd11246729fd93665b9a16d715537b0"
Sep 17 17:36:31.763534 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:36:31.763503    2565 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"3789759b52cab6aafcf0ce91d69c2d69ccd11246729fd93665b9a16d715537b0"} err="failed to get container status \"3789759b52cab6aafcf0ce91d69c2d69ccd11246729fd93665b9a16d715537b0\": rpc error: code = NotFound desc = could not find container \"3789759b52cab6aafcf0ce91d69c2d69ccd11246729fd93665b9a16d715537b0\": container with ID starting with 3789759b52cab6aafcf0ce91d69c2d69ccd11246729fd93665b9a16d715537b0 not found: ID does not exist"
Sep 17 17:36:31.776783 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:36:31.776752    2565 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-87747d4df-tk8xw"]
Sep 17 17:36:31.782222 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:36:31.782202    2565 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-console/console-87747d4df-tk8xw"]
Sep 17 17:36:32.154754 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:36:32.154723    2565 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="cedae061-1174-4bc8-b7b9-0d034929ddad" path="/var/lib/kubelet/pods/cedae061-1174-4bc8-b7b9-0d034929ddad/volumes"
Sep 17 17:37:03.419296 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:37:03.419218    2565 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-monitoring/alertmanager-main-0"]
Sep 17 17:37:03.419768 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:37:03.419509    2565 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="16a894e3-8971-45a7-9f5e-c1da4d5b932f" containerName="alertmanager" containerID="cri-o://664bc8312cf37b1f15381bb1251f7da8cec4ed651408d41becdbef57f97230bd" gracePeriod=120
Sep 17 17:37:03.419768 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:37:03.419559    2565 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="16a894e3-8971-45a7-9f5e-c1da4d5b932f" containerName="kube-rbac-proxy-metric" containerID="cri-o://a5e3d87b9a19c1adb40dbef70d3e60284afaf4a6b69bf01cac006949ec723835" gracePeriod=120
Sep 17 17:37:03.419768 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:37:03.419611    2565 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="16a894e3-8971-45a7-9f5e-c1da4d5b932f" containerName="prom-label-proxy" containerID="cri-o://071a64b8ea3daeaea63a644fa08c4629e8aed942101e35e2a19a5125ea9264e0" gracePeriod=120
Sep 17 17:37:03.419768 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:37:03.419616    2565 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="16a894e3-8971-45a7-9f5e-c1da4d5b932f" containerName="kube-rbac-proxy-web" containerID="cri-o://b5e5e1a34891738d9b7dd1ad20f72ed0aa99dfeca3ac4b8c31c59db926925785" gracePeriod=120
Sep 17 17:37:03.419768 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:37:03.419672    2565 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="16a894e3-8971-45a7-9f5e-c1da4d5b932f" containerName="kube-rbac-proxy" containerID="cri-o://134ec427f4263789d20f4763dcb06daf17337a7ca9916131c5f088039ad48699" gracePeriod=120
Sep 17 17:37:03.419768 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:37:03.419659    2565 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="16a894e3-8971-45a7-9f5e-c1da4d5b932f" containerName="config-reloader" containerID="cri-o://a30feba4b2557cb4bec23b611e8fa094a26c018639d581f94fbb2d866ee19089" gracePeriod=120
Sep 17 17:37:03.875586 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:37:03.875560    2565 generic.go:356] "Generic (PLEG): container finished" podID="16a894e3-8971-45a7-9f5e-c1da4d5b932f" containerID="071a64b8ea3daeaea63a644fa08c4629e8aed942101e35e2a19a5125ea9264e0" exitCode=0
Sep 17 17:37:03.875586 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:37:03.875582    2565 generic.go:356] "Generic (PLEG): container finished" podID="16a894e3-8971-45a7-9f5e-c1da4d5b932f" containerID="a5e3d87b9a19c1adb40dbef70d3e60284afaf4a6b69bf01cac006949ec723835" exitCode=0
Sep 17 17:37:03.875586 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:37:03.875589    2565 generic.go:356] "Generic (PLEG): container finished" podID="16a894e3-8971-45a7-9f5e-c1da4d5b932f" containerID="134ec427f4263789d20f4763dcb06daf17337a7ca9916131c5f088039ad48699" exitCode=0
Sep 17 17:37:03.875778 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:37:03.875595    2565 generic.go:356] "Generic (PLEG): container finished" podID="16a894e3-8971-45a7-9f5e-c1da4d5b932f" containerID="a30feba4b2557cb4bec23b611e8fa094a26c018639d581f94fbb2d866ee19089" exitCode=0
Sep 17 17:37:03.875778 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:37:03.875600    2565 generic.go:356] "Generic (PLEG): container finished" podID="16a894e3-8971-45a7-9f5e-c1da4d5b932f" containerID="664bc8312cf37b1f15381bb1251f7da8cec4ed651408d41becdbef57f97230bd" exitCode=0
Sep 17 17:37:03.875778 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:37:03.875630    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"16a894e3-8971-45a7-9f5e-c1da4d5b932f","Type":"ContainerDied","Data":"071a64b8ea3daeaea63a644fa08c4629e8aed942101e35e2a19a5125ea9264e0"}
Sep 17 17:37:03.875778 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:37:03.875691    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"16a894e3-8971-45a7-9f5e-c1da4d5b932f","Type":"ContainerDied","Data":"a5e3d87b9a19c1adb40dbef70d3e60284afaf4a6b69bf01cac006949ec723835"}
Sep 17 17:37:03.875778 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:37:03.875718    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"16a894e3-8971-45a7-9f5e-c1da4d5b932f","Type":"ContainerDied","Data":"134ec427f4263789d20f4763dcb06daf17337a7ca9916131c5f088039ad48699"}
Sep 17 17:37:03.875778 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:37:03.875742    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"16a894e3-8971-45a7-9f5e-c1da4d5b932f","Type":"ContainerDied","Data":"a30feba4b2557cb4bec23b611e8fa094a26c018639d581f94fbb2d866ee19089"}
Sep 17 17:37:03.875778 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:37:03.875764    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"16a894e3-8971-45a7-9f5e-c1da4d5b932f","Type":"ContainerDied","Data":"664bc8312cf37b1f15381bb1251f7da8cec4ed651408d41becdbef57f97230bd"}
Sep 17 17:37:04.659726 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:37:04.659703    2565 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0"
Sep 17 17:37:04.716652 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:37:04.716623    2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/16a894e3-8971-45a7-9f5e-c1da4d5b932f-secret-alertmanager-kube-rbac-proxy-web\") pod \"16a894e3-8971-45a7-9f5e-c1da4d5b932f\" (UID: \"16a894e3-8971-45a7-9f5e-c1da4d5b932f\") "
Sep 17 17:37:04.716799 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:37:04.716666    2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/16a894e3-8971-45a7-9f5e-c1da4d5b932f-metrics-client-ca\") pod \"16a894e3-8971-45a7-9f5e-c1da4d5b932f\" (UID: \"16a894e3-8971-45a7-9f5e-c1da4d5b932f\") "
Sep 17 17:37:04.716799 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:37:04.716697    2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-5m6k9\" (UniqueName: \"kubernetes.io/projected/16a894e3-8971-45a7-9f5e-c1da4d5b932f-kube-api-access-5m6k9\") pod \"16a894e3-8971-45a7-9f5e-c1da4d5b932f\" (UID: \"16a894e3-8971-45a7-9f5e-c1da4d5b932f\") "
Sep 17 17:37:04.716908 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:37:04.716873    2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/16a894e3-8971-45a7-9f5e-c1da4d5b932f-cluster-tls-config\") pod \"16a894e3-8971-45a7-9f5e-c1da4d5b932f\" (UID: \"16a894e3-8971-45a7-9f5e-c1da4d5b932f\") "
Sep 17 17:37:04.716968 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:37:04.716936    2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/16a894e3-8971-45a7-9f5e-c1da4d5b932f-secret-alertmanager-kube-rbac-proxy\") pod \"16a894e3-8971-45a7-9f5e-c1da4d5b932f\" (UID: \"16a894e3-8971-45a7-9f5e-c1da4d5b932f\") "
Sep 17 17:37:04.716968 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:37:04.716968    2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/16a894e3-8971-45a7-9f5e-c1da4d5b932f-alertmanager-trusted-ca-bundle\") pod \"16a894e3-8971-45a7-9f5e-c1da4d5b932f\" (UID: \"16a894e3-8971-45a7-9f5e-c1da4d5b932f\") "
Sep 17 17:37:04.717104 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:37:04.717012    2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/16a894e3-8971-45a7-9f5e-c1da4d5b932f-tls-assets\") pod \"16a894e3-8971-45a7-9f5e-c1da4d5b932f\" (UID: \"16a894e3-8971-45a7-9f5e-c1da4d5b932f\") "
Sep 17 17:37:04.717104 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:37:04.717057    2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/16a894e3-8971-45a7-9f5e-c1da4d5b932f-secret-alertmanager-kube-rbac-proxy-metric\") pod \"16a894e3-8971-45a7-9f5e-c1da4d5b932f\" (UID: \"16a894e3-8971-45a7-9f5e-c1da4d5b932f\") "
Sep 17 17:37:04.717104 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:37:04.717070    2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/16a894e3-8971-45a7-9f5e-c1da4d5b932f-metrics-client-ca" (OuterVolumeSpecName: "metrics-client-ca") pod "16a894e3-8971-45a7-9f5e-c1da4d5b932f" (UID: "16a894e3-8971-45a7-9f5e-c1da4d5b932f"). InnerVolumeSpecName "metrics-client-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 17 17:37:04.717249 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:37:04.717084    2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/16a894e3-8971-45a7-9f5e-c1da4d5b932f-secret-alertmanager-main-tls\") pod \"16a894e3-8971-45a7-9f5e-c1da4d5b932f\" (UID: \"16a894e3-8971-45a7-9f5e-c1da4d5b932f\") "
Sep 17 17:37:04.717249 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:37:04.717154    2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/16a894e3-8971-45a7-9f5e-c1da4d5b932f-config-volume\") pod \"16a894e3-8971-45a7-9f5e-c1da4d5b932f\" (UID: \"16a894e3-8971-45a7-9f5e-c1da4d5b932f\") "
Sep 17 17:37:04.717249 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:37:04.717184    2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/16a894e3-8971-45a7-9f5e-c1da4d5b932f-alertmanager-main-db\") pod \"16a894e3-8971-45a7-9f5e-c1da4d5b932f\" (UID: \"16a894e3-8971-45a7-9f5e-c1da4d5b932f\") "
Sep 17 17:37:04.717249 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:37:04.717241    2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/16a894e3-8971-45a7-9f5e-c1da4d5b932f-config-out\") pod \"16a894e3-8971-45a7-9f5e-c1da4d5b932f\" (UID: \"16a894e3-8971-45a7-9f5e-c1da4d5b932f\") "
Sep 17 17:37:04.717446 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:37:04.717293    2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/16a894e3-8971-45a7-9f5e-c1da4d5b932f-web-config\") pod \"16a894e3-8971-45a7-9f5e-c1da4d5b932f\" (UID: \"16a894e3-8971-45a7-9f5e-c1da4d5b932f\") "
Sep 17 17:37:04.717608 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:37:04.717587    2565 reconciler_common.go:299] "Volume detached for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/16a894e3-8971-45a7-9f5e-c1da4d5b932f-metrics-client-ca\") on node \"ip-10-0-137-20.ec2.internal\" DevicePath \"\""
Sep 17 17:37:04.717751 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:37:04.717588    2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/16a894e3-8971-45a7-9f5e-c1da4d5b932f-alertmanager-main-db" (OuterVolumeSpecName: "alertmanager-main-db") pod "16a894e3-8971-45a7-9f5e-c1da4d5b932f" (UID: "16a894e3-8971-45a7-9f5e-c1da4d5b932f"). InnerVolumeSpecName "alertmanager-main-db". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 17 17:37:04.718039 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:37:04.717993    2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/16a894e3-8971-45a7-9f5e-c1da4d5b932f-alertmanager-trusted-ca-bundle" (OuterVolumeSpecName: "alertmanager-trusted-ca-bundle") pod "16a894e3-8971-45a7-9f5e-c1da4d5b932f" (UID: "16a894e3-8971-45a7-9f5e-c1da4d5b932f"). InnerVolumeSpecName "alertmanager-trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 17 17:37:04.719874 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:37:04.719836    2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/16a894e3-8971-45a7-9f5e-c1da4d5b932f-secret-alertmanager-kube-rbac-proxy-web" (OuterVolumeSpecName: "secret-alertmanager-kube-rbac-proxy-web") pod "16a894e3-8971-45a7-9f5e-c1da4d5b932f" (UID: "16a894e3-8971-45a7-9f5e-c1da4d5b932f"). InnerVolumeSpecName "secret-alertmanager-kube-rbac-proxy-web". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 17 17:37:04.719970 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:37:04.719951    2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/16a894e3-8971-45a7-9f5e-c1da4d5b932f-kube-api-access-5m6k9" (OuterVolumeSpecName: "kube-api-access-5m6k9") pod "16a894e3-8971-45a7-9f5e-c1da4d5b932f" (UID: "16a894e3-8971-45a7-9f5e-c1da4d5b932f"). InnerVolumeSpecName "kube-api-access-5m6k9". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 17 17:37:04.720394 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:37:04.720368    2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/16a894e3-8971-45a7-9f5e-c1da4d5b932f-secret-alertmanager-main-tls" (OuterVolumeSpecName: "secret-alertmanager-main-tls") pod "16a894e3-8971-45a7-9f5e-c1da4d5b932f" (UID: "16a894e3-8971-45a7-9f5e-c1da4d5b932f"). InnerVolumeSpecName "secret-alertmanager-main-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 17 17:37:04.720394 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:37:04.720381    2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/16a894e3-8971-45a7-9f5e-c1da4d5b932f-tls-assets" (OuterVolumeSpecName: "tls-assets") pod "16a894e3-8971-45a7-9f5e-c1da4d5b932f" (UID: "16a894e3-8971-45a7-9f5e-c1da4d5b932f"). InnerVolumeSpecName "tls-assets". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 17 17:37:04.720833 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:37:04.720785    2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/16a894e3-8971-45a7-9f5e-c1da4d5b932f-config-out" (OuterVolumeSpecName: "config-out") pod "16a894e3-8971-45a7-9f5e-c1da4d5b932f" (UID: "16a894e3-8971-45a7-9f5e-c1da4d5b932f"). InnerVolumeSpecName "config-out". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 17 17:37:04.721264 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:37:04.721240    2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/16a894e3-8971-45a7-9f5e-c1da4d5b932f-secret-alertmanager-kube-rbac-proxy" (OuterVolumeSpecName: "secret-alertmanager-kube-rbac-proxy") pod "16a894e3-8971-45a7-9f5e-c1da4d5b932f" (UID: "16a894e3-8971-45a7-9f5e-c1da4d5b932f"). InnerVolumeSpecName "secret-alertmanager-kube-rbac-proxy". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 17 17:37:04.721671 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:37:04.721640    2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/16a894e3-8971-45a7-9f5e-c1da4d5b932f-secret-alertmanager-kube-rbac-proxy-metric" (OuterVolumeSpecName: "secret-alertmanager-kube-rbac-proxy-metric") pod "16a894e3-8971-45a7-9f5e-c1da4d5b932f" (UID: "16a894e3-8971-45a7-9f5e-c1da4d5b932f"). InnerVolumeSpecName "secret-alertmanager-kube-rbac-proxy-metric". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 17 17:37:04.721772 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:37:04.721682    2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/16a894e3-8971-45a7-9f5e-c1da4d5b932f-config-volume" (OuterVolumeSpecName: "config-volume") pod "16a894e3-8971-45a7-9f5e-c1da4d5b932f" (UID: "16a894e3-8971-45a7-9f5e-c1da4d5b932f"). InnerVolumeSpecName "config-volume". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 17 17:37:04.726294 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:37:04.726078    2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/16a894e3-8971-45a7-9f5e-c1da4d5b932f-cluster-tls-config" (OuterVolumeSpecName: "cluster-tls-config") pod "16a894e3-8971-45a7-9f5e-c1da4d5b932f" (UID: "16a894e3-8971-45a7-9f5e-c1da4d5b932f"). InnerVolumeSpecName "cluster-tls-config". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 17 17:37:04.733357 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:37:04.733334    2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/16a894e3-8971-45a7-9f5e-c1da4d5b932f-web-config" (OuterVolumeSpecName: "web-config") pod "16a894e3-8971-45a7-9f5e-c1da4d5b932f" (UID: "16a894e3-8971-45a7-9f5e-c1da4d5b932f"). InnerVolumeSpecName "web-config". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 17 17:37:04.818989 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:37:04.818914    2565 reconciler_common.go:299] "Volume detached for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/16a894e3-8971-45a7-9f5e-c1da4d5b932f-cluster-tls-config\") on node \"ip-10-0-137-20.ec2.internal\" DevicePath \"\""
Sep 17 17:37:04.818989 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:37:04.818943    2565 reconciler_common.go:299] "Volume detached for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/16a894e3-8971-45a7-9f5e-c1da4d5b932f-secret-alertmanager-kube-rbac-proxy\") on node \"ip-10-0-137-20.ec2.internal\" DevicePath \"\""
Sep 17 17:37:04.818989 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:37:04.818973    2565 reconciler_common.go:299] "Volume detached for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/16a894e3-8971-45a7-9f5e-c1da4d5b932f-alertmanager-trusted-ca-bundle\") on node \"ip-10-0-137-20.ec2.internal\" DevicePath \"\""
Sep 17 17:37:04.818989 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:37:04.818984    2565 reconciler_common.go:299] "Volume detached for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/16a894e3-8971-45a7-9f5e-c1da4d5b932f-tls-assets\") on node \"ip-10-0-137-20.ec2.internal\" DevicePath \"\""
Sep 17 17:37:04.818989 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:37:04.818992    2565 reconciler_common.go:299] "Volume detached for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/16a894e3-8971-45a7-9f5e-c1da4d5b932f-secret-alertmanager-kube-rbac-proxy-metric\") on node \"ip-10-0-137-20.ec2.internal\" DevicePath \"\""
Sep 17 17:37:04.819257 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:37:04.819001    2565 reconciler_common.go:299] "Volume detached for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/16a894e3-8971-45a7-9f5e-c1da4d5b932f-secret-alertmanager-main-tls\") on node \"ip-10-0-137-20.ec2.internal\" DevicePath \"\""
Sep 17 17:37:04.819257 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:37:04.819009    2565 reconciler_common.go:299] "Volume detached for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/16a894e3-8971-45a7-9f5e-c1da4d5b932f-config-volume\") on node \"ip-10-0-137-20.ec2.internal\" DevicePath \"\""
Sep 17 17:37:04.819257 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:37:04.819016    2565 reconciler_common.go:299] "Volume detached for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/16a894e3-8971-45a7-9f5e-c1da4d5b932f-alertmanager-main-db\") on node \"ip-10-0-137-20.ec2.internal\" DevicePath \"\""
Sep 17 17:37:04.819257 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:37:04.819036    2565 reconciler_common.go:299] "Volume detached for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/16a894e3-8971-45a7-9f5e-c1da4d5b932f-config-out\") on node \"ip-10-0-137-20.ec2.internal\" DevicePath \"\""
Sep 17 17:37:04.819257 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:37:04.819044    2565 reconciler_common.go:299] "Volume detached for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/16a894e3-8971-45a7-9f5e-c1da4d5b932f-web-config\") on node \"ip-10-0-137-20.ec2.internal\" DevicePath \"\""
Sep 17 17:37:04.819257 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:37:04.819051    2565 reconciler_common.go:299] "Volume detached for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/16a894e3-8971-45a7-9f5e-c1da4d5b932f-secret-alertmanager-kube-rbac-proxy-web\") on node \"ip-10-0-137-20.ec2.internal\" DevicePath \"\""
Sep 17 17:37:04.819257 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:37:04.819059    2565 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-5m6k9\" (UniqueName: \"kubernetes.io/projected/16a894e3-8971-45a7-9f5e-c1da4d5b932f-kube-api-access-5m6k9\") on node \"ip-10-0-137-20.ec2.internal\" DevicePath \"\""
Sep 17 17:37:04.883081 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:37:04.883050    2565 generic.go:356] "Generic (PLEG): container finished" podID="16a894e3-8971-45a7-9f5e-c1da4d5b932f" containerID="b5e5e1a34891738d9b7dd1ad20f72ed0aa99dfeca3ac4b8c31c59db926925785" exitCode=0
Sep 17 17:37:04.883220 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:37:04.883129    2565 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0"
Sep 17 17:37:04.883220 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:37:04.883128    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"16a894e3-8971-45a7-9f5e-c1da4d5b932f","Type":"ContainerDied","Data":"b5e5e1a34891738d9b7dd1ad20f72ed0aa99dfeca3ac4b8c31c59db926925785"}
Sep 17 17:37:04.883220 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:37:04.883216    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"16a894e3-8971-45a7-9f5e-c1da4d5b932f","Type":"ContainerDied","Data":"2383f71041ca396917229686da03a57e468e9a524324ee334cf1452d4f9098d7"}
Sep 17 17:37:04.883316 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:37:04.883266    2565 scope.go:117] "RemoveContainer" containerID="071a64b8ea3daeaea63a644fa08c4629e8aed942101e35e2a19a5125ea9264e0"
Sep 17 17:37:04.894038 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:37:04.894002    2565 scope.go:117] "RemoveContainer" containerID="a5e3d87b9a19c1adb40dbef70d3e60284afaf4a6b69bf01cac006949ec723835"
Sep 17 17:37:04.900757 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:37:04.900739    2565 scope.go:117] "RemoveContainer" containerID="134ec427f4263789d20f4763dcb06daf17337a7ca9916131c5f088039ad48699"
Sep 17 17:37:04.907545 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:37:04.907522    2565 scope.go:117] "RemoveContainer" containerID="b5e5e1a34891738d9b7dd1ad20f72ed0aa99dfeca3ac4b8c31c59db926925785"
Sep 17 17:37:04.913347 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:37:04.913328    2565 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-monitoring/alertmanager-main-0"]
Sep 17 17:37:04.914723 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:37:04.914710    2565 scope.go:117] "RemoveContainer" containerID="a30feba4b2557cb4bec23b611e8fa094a26c018639d581f94fbb2d866ee19089"
Sep 17 17:37:04.921079 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:37:04.921065    2565 scope.go:117] "RemoveContainer" containerID="664bc8312cf37b1f15381bb1251f7da8cec4ed651408d41becdbef57f97230bd"
Sep 17 17:37:04.925506 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:37:04.925486    2565 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-monitoring/alertmanager-main-0"]
Sep 17 17:37:04.927930 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:37:04.927917    2565 scope.go:117] "RemoveContainer" containerID="39869e0624da33959e6dd5262faea8c3f150bfd6fa4b8e932781017c7e556cf3"
Sep 17 17:37:04.933958 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:37:04.933940    2565 scope.go:117] "RemoveContainer" containerID="071a64b8ea3daeaea63a644fa08c4629e8aed942101e35e2a19a5125ea9264e0"
Sep 17 17:37:04.934229 ip-10-0-137-20 kubenswrapper[2565]: E0917 17:37:04.934212    2565 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"071a64b8ea3daeaea63a644fa08c4629e8aed942101e35e2a19a5125ea9264e0\": container with ID starting with 071a64b8ea3daeaea63a644fa08c4629e8aed942101e35e2a19a5125ea9264e0 not found: ID does not exist" containerID="071a64b8ea3daeaea63a644fa08c4629e8aed942101e35e2a19a5125ea9264e0"
Sep 17 17:37:04.934296 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:37:04.934236    2565 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"071a64b8ea3daeaea63a644fa08c4629e8aed942101e35e2a19a5125ea9264e0"} err="failed to get container status \"071a64b8ea3daeaea63a644fa08c4629e8aed942101e35e2a19a5125ea9264e0\": rpc error: code = NotFound desc = could not find container \"071a64b8ea3daeaea63a644fa08c4629e8aed942101e35e2a19a5125ea9264e0\": container with ID starting with 071a64b8ea3daeaea63a644fa08c4629e8aed942101e35e2a19a5125ea9264e0 not found: ID does not exist"
Sep 17 17:37:04.934296 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:37:04.934251    2565 scope.go:117] "RemoveContainer" containerID="a5e3d87b9a19c1adb40dbef70d3e60284afaf4a6b69bf01cac006949ec723835"
Sep 17 17:37:04.934503 ip-10-0-137-20 kubenswrapper[2565]: E0917 17:37:04.934485    2565 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"a5e3d87b9a19c1adb40dbef70d3e60284afaf4a6b69bf01cac006949ec723835\": container with ID starting with a5e3d87b9a19c1adb40dbef70d3e60284afaf4a6b69bf01cac006949ec723835 not found: ID does not exist" containerID="a5e3d87b9a19c1adb40dbef70d3e60284afaf4a6b69bf01cac006949ec723835"
Sep 17 17:37:04.934549 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:37:04.934512    2565 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"a5e3d87b9a19c1adb40dbef70d3e60284afaf4a6b69bf01cac006949ec723835"} err="failed to get container status \"a5e3d87b9a19c1adb40dbef70d3e60284afaf4a6b69bf01cac006949ec723835\": rpc error: code = NotFound desc = could not find container \"a5e3d87b9a19c1adb40dbef70d3e60284afaf4a6b69bf01cac006949ec723835\": container with ID starting with a5e3d87b9a19c1adb40dbef70d3e60284afaf4a6b69bf01cac006949ec723835 not found: ID does not exist"
Sep 17 17:37:04.934549 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:37:04.934527    2565 scope.go:117] "RemoveContainer" containerID="134ec427f4263789d20f4763dcb06daf17337a7ca9916131c5f088039ad48699"
Sep 17 17:37:04.934758 ip-10-0-137-20 kubenswrapper[2565]: E0917 17:37:04.934740    2565 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"134ec427f4263789d20f4763dcb06daf17337a7ca9916131c5f088039ad48699\": container with ID starting with 134ec427f4263789d20f4763dcb06daf17337a7ca9916131c5f088039ad48699 not found: ID does not exist" containerID="134ec427f4263789d20f4763dcb06daf17337a7ca9916131c5f088039ad48699"
Sep 17 17:37:04.934797 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:37:04.934762    2565 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"134ec427f4263789d20f4763dcb06daf17337a7ca9916131c5f088039ad48699"} err="failed to get container status \"134ec427f4263789d20f4763dcb06daf17337a7ca9916131c5f088039ad48699\": rpc error: code = NotFound desc = could not find container \"134ec427f4263789d20f4763dcb06daf17337a7ca9916131c5f088039ad48699\": container with ID starting with 134ec427f4263789d20f4763dcb06daf17337a7ca9916131c5f088039ad48699 not found: ID does not exist"
Sep 17 17:37:04.934797 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:37:04.934776    2565 scope.go:117] "RemoveContainer" containerID="b5e5e1a34891738d9b7dd1ad20f72ed0aa99dfeca3ac4b8c31c59db926925785"
Sep 17 17:37:04.935011 ip-10-0-137-20 kubenswrapper[2565]: E0917 17:37:04.934995    2565 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"b5e5e1a34891738d9b7dd1ad20f72ed0aa99dfeca3ac4b8c31c59db926925785\": container with ID starting with b5e5e1a34891738d9b7dd1ad20f72ed0aa99dfeca3ac4b8c31c59db926925785 not found: ID does not exist" containerID="b5e5e1a34891738d9b7dd1ad20f72ed0aa99dfeca3ac4b8c31c59db926925785"
Sep 17 17:37:04.935089 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:37:04.935012    2565 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"b5e5e1a34891738d9b7dd1ad20f72ed0aa99dfeca3ac4b8c31c59db926925785"} err="failed to get container status \"b5e5e1a34891738d9b7dd1ad20f72ed0aa99dfeca3ac4b8c31c59db926925785\": rpc error: code = NotFound desc = could not find container \"b5e5e1a34891738d9b7dd1ad20f72ed0aa99dfeca3ac4b8c31c59db926925785\": container with ID starting with b5e5e1a34891738d9b7dd1ad20f72ed0aa99dfeca3ac4b8c31c59db926925785 not found: ID does not exist"
Sep 17 17:37:04.935089 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:37:04.935044    2565 scope.go:117] "RemoveContainer" containerID="a30feba4b2557cb4bec23b611e8fa094a26c018639d581f94fbb2d866ee19089"
Sep 17 17:37:04.935276 ip-10-0-137-20 kubenswrapper[2565]: E0917 17:37:04.935257    2565 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"a30feba4b2557cb4bec23b611e8fa094a26c018639d581f94fbb2d866ee19089\": container with ID starting with a30feba4b2557cb4bec23b611e8fa094a26c018639d581f94fbb2d866ee19089 not found: ID does not exist" containerID="a30feba4b2557cb4bec23b611e8fa094a26c018639d581f94fbb2d866ee19089"
Sep 17 17:37:04.935317 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:37:04.935280    2565 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"a30feba4b2557cb4bec23b611e8fa094a26c018639d581f94fbb2d866ee19089"} err="failed to get container status \"a30feba4b2557cb4bec23b611e8fa094a26c018639d581f94fbb2d866ee19089\": rpc error: code = NotFound desc = could not find container \"a30feba4b2557cb4bec23b611e8fa094a26c018639d581f94fbb2d866ee19089\": container with ID starting with a30feba4b2557cb4bec23b611e8fa094a26c018639d581f94fbb2d866ee19089 not found: ID does not exist"
Sep 17 17:37:04.935317 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:37:04.935293    2565 scope.go:117] "RemoveContainer" containerID="664bc8312cf37b1f15381bb1251f7da8cec4ed651408d41becdbef57f97230bd"
Sep 17 17:37:04.935515 ip-10-0-137-20 kubenswrapper[2565]: E0917 17:37:04.935492    2565 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"664bc8312cf37b1f15381bb1251f7da8cec4ed651408d41becdbef57f97230bd\": container with ID starting with 664bc8312cf37b1f15381bb1251f7da8cec4ed651408d41becdbef57f97230bd not found: ID does not exist" containerID="664bc8312cf37b1f15381bb1251f7da8cec4ed651408d41becdbef57f97230bd"
Sep 17 17:37:04.935562 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:37:04.935516    2565 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"664bc8312cf37b1f15381bb1251f7da8cec4ed651408d41becdbef57f97230bd"} err="failed to get container status \"664bc8312cf37b1f15381bb1251f7da8cec4ed651408d41becdbef57f97230bd\": rpc error: code = NotFound desc = could not find container \"664bc8312cf37b1f15381bb1251f7da8cec4ed651408d41becdbef57f97230bd\": container with ID starting with 664bc8312cf37b1f15381bb1251f7da8cec4ed651408d41becdbef57f97230bd not found: ID does not exist"
Sep 17 17:37:04.935562 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:37:04.935528    2565 scope.go:117] "RemoveContainer" containerID="39869e0624da33959e6dd5262faea8c3f150bfd6fa4b8e932781017c7e556cf3"
Sep 17 17:37:04.935762 ip-10-0-137-20 kubenswrapper[2565]: E0917 17:37:04.935743    2565 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"39869e0624da33959e6dd5262faea8c3f150bfd6fa4b8e932781017c7e556cf3\": container with ID starting with 39869e0624da33959e6dd5262faea8c3f150bfd6fa4b8e932781017c7e556cf3 not found: ID does not exist" containerID="39869e0624da33959e6dd5262faea8c3f150bfd6fa4b8e932781017c7e556cf3"
Sep 17 17:37:04.935860 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:37:04.935770    2565 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"39869e0624da33959e6dd5262faea8c3f150bfd6fa4b8e932781017c7e556cf3"} err="failed to get container status \"39869e0624da33959e6dd5262faea8c3f150bfd6fa4b8e932781017c7e556cf3\": rpc error: code = NotFound desc = could not find container \"39869e0624da33959e6dd5262faea8c3f150bfd6fa4b8e932781017c7e556cf3\": container with ID starting with 39869e0624da33959e6dd5262faea8c3f150bfd6fa4b8e932781017c7e556cf3 not found: ID does not exist"
Sep 17 17:37:04.961412 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:37:04.961393    2565 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/alertmanager-main-0"]
Sep 17 17:37:04.961614 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:37:04.961602    2565 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="9ab56083-4127-4bbb-af70-4ef55006fe47" containerName="registry"
Sep 17 17:37:04.961659 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:37:04.961615    2565 state_mem.go:107] "Deleted CPUSet assignment" podUID="9ab56083-4127-4bbb-af70-4ef55006fe47" containerName="registry"
Sep 17 17:37:04.961659 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:37:04.961622    2565 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="16a894e3-8971-45a7-9f5e-c1da4d5b932f" containerName="kube-rbac-proxy-metric"
Sep 17 17:37:04.961659 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:37:04.961627    2565 state_mem.go:107] "Deleted CPUSet assignment" podUID="16a894e3-8971-45a7-9f5e-c1da4d5b932f" containerName="kube-rbac-proxy-metric"
Sep 17 17:37:04.961659 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:37:04.961635    2565 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="16a894e3-8971-45a7-9f5e-c1da4d5b932f" containerName="kube-rbac-proxy-web"
Sep 17 17:37:04.961659 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:37:04.961640    2565 state_mem.go:107] "Deleted CPUSet assignment" podUID="16a894e3-8971-45a7-9f5e-c1da4d5b932f" containerName="kube-rbac-proxy-web"
Sep 17 17:37:04.961659 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:37:04.961648    2565 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="16a894e3-8971-45a7-9f5e-c1da4d5b932f" containerName="init-config-reloader"
Sep 17 17:37:04.961659 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:37:04.961653    2565 state_mem.go:107] "Deleted CPUSet assignment" podUID="16a894e3-8971-45a7-9f5e-c1da4d5b932f" containerName="init-config-reloader"
Sep 17 17:37:04.961659 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:37:04.961661    2565 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="16a894e3-8971-45a7-9f5e-c1da4d5b932f" containerName="prom-label-proxy"
Sep 17 17:37:04.961914 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:37:04.961666    2565 state_mem.go:107] "Deleted CPUSet assignment" podUID="16a894e3-8971-45a7-9f5e-c1da4d5b932f" containerName="prom-label-proxy"
Sep 17 17:37:04.961914 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:37:04.961673    2565 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="16a894e3-8971-45a7-9f5e-c1da4d5b932f" containerName="alertmanager"
Sep 17 17:37:04.961914 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:37:04.961677    2565 state_mem.go:107] "Deleted CPUSet assignment" podUID="16a894e3-8971-45a7-9f5e-c1da4d5b932f" containerName="alertmanager"
Sep 17 17:37:04.961914 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:37:04.961685    2565 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="16a894e3-8971-45a7-9f5e-c1da4d5b932f" containerName="kube-rbac-proxy"
Sep 17 17:37:04.961914 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:37:04.961690    2565 state_mem.go:107] "Deleted CPUSet assignment" podUID="16a894e3-8971-45a7-9f5e-c1da4d5b932f" containerName="kube-rbac-proxy"
Sep 17 17:37:04.961914 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:37:04.961698    2565 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="16a894e3-8971-45a7-9f5e-c1da4d5b932f" containerName="config-reloader"
Sep 17 17:37:04.961914 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:37:04.961703    2565 state_mem.go:107] "Deleted CPUSet assignment" podUID="16a894e3-8971-45a7-9f5e-c1da4d5b932f" containerName="config-reloader"
Sep 17 17:37:04.961914 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:37:04.961711    2565 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a416f0c6-706f-48ae-a806-48c7fc5b0171" containerName="console"
Sep 17 17:37:04.961914 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:37:04.961716    2565 state_mem.go:107] "Deleted CPUSet assignment" podUID="a416f0c6-706f-48ae-a806-48c7fc5b0171" containerName="console"
Sep 17 17:37:04.961914 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:37:04.961723    2565 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="cedae061-1174-4bc8-b7b9-0d034929ddad" containerName="console"
Sep 17 17:37:04.961914 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:37:04.961728    2565 state_mem.go:107] "Deleted CPUSet assignment" podUID="cedae061-1174-4bc8-b7b9-0d034929ddad" containerName="console"
Sep 17 17:37:04.961914 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:37:04.961763    2565 memory_manager.go:368] "RemoveStaleState removing state" podUID="cedae061-1174-4bc8-b7b9-0d034929ddad" containerName="console"
Sep 17 17:37:04.961914 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:37:04.961770    2565 memory_manager.go:368] "RemoveStaleState removing state" podUID="9ab56083-4127-4bbb-af70-4ef55006fe47" containerName="registry"
Sep 17 17:37:04.961914 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:37:04.961777    2565 memory_manager.go:368] "RemoveStaleState removing state" podUID="a416f0c6-706f-48ae-a806-48c7fc5b0171" containerName="console"
Sep 17 17:37:04.961914 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:37:04.961782    2565 memory_manager.go:368] "RemoveStaleState removing state" podUID="16a894e3-8971-45a7-9f5e-c1da4d5b932f" containerName="config-reloader"
Sep 17 17:37:04.961914 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:37:04.961787    2565 memory_manager.go:368] "RemoveStaleState removing state" podUID="16a894e3-8971-45a7-9f5e-c1da4d5b932f" containerName="kube-rbac-proxy"
Sep 17 17:37:04.961914 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:37:04.961791    2565 memory_manager.go:368] "RemoveStaleState removing state" podUID="16a894e3-8971-45a7-9f5e-c1da4d5b932f" containerName="prom-label-proxy"
Sep 17 17:37:04.961914 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:37:04.961796    2565 memory_manager.go:368] "RemoveStaleState removing state" podUID="16a894e3-8971-45a7-9f5e-c1da4d5b932f" containerName="alertmanager"
Sep 17 17:37:04.961914 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:37:04.961800    2565 memory_manager.go:368] "RemoveStaleState removing state" podUID="16a894e3-8971-45a7-9f5e-c1da4d5b932f" containerName="kube-rbac-proxy-web"
Sep 17 17:37:04.961914 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:37:04.961805    2565 memory_manager.go:368] "RemoveStaleState removing state" podUID="16a894e3-8971-45a7-9f5e-c1da4d5b932f" containerName="kube-rbac-proxy-metric"
Sep 17 17:37:04.966583 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:37:04.966568    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0"
Sep 17 17:37:04.970641 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:37:04.970624    2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-web-config\""
Sep 17 17:37:04.970891 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:37:04.970875    2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-cluster-tls-config\""
Sep 17 17:37:04.971210 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:37:04.971193    2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy\""
Sep 17 17:37:04.971297 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:37:04.971197    2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-dockercfg-q65bv\""
Sep 17 17:37:04.971297 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:37:04.971197    2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-tls\""
Sep 17 17:37:04.971476 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:37:04.971449    2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy-web\""
Sep 17 17:37:04.971476 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:37:04.971464    2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy-metric\""
Sep 17 17:37:04.971594 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:37:04.971510    2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-generated\""
Sep 17 17:37:04.971594 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:37:04.971455    2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-tls-assets-0\""
Sep 17 17:37:04.977710 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:37:04.977687    2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"alertmanager-trusted-ca-bundle\""
Sep 17 17:37:04.989774 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:37:04.989752    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/alertmanager-main-0"]
Sep 17 17:37:05.020168 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:37:05.020143    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/43e20c01-6600-4445-aee3-1a90c7aac2cd-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"43e20c01-6600-4445-aee3-1a90c7aac2cd\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 17 17:37:05.020259 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:37:05.020175    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/43e20c01-6600-4445-aee3-1a90c7aac2cd-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"43e20c01-6600-4445-aee3-1a90c7aac2cd\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 17 17:37:05.020259 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:37:05.020198    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/43e20c01-6600-4445-aee3-1a90c7aac2cd-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"43e20c01-6600-4445-aee3-1a90c7aac2cd\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 17 17:37:05.020341 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:37:05.020283    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/43e20c01-6600-4445-aee3-1a90c7aac2cd-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"43e20c01-6600-4445-aee3-1a90c7aac2cd\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 17 17:37:05.020341 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:37:05.020315    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/43e20c01-6600-4445-aee3-1a90c7aac2cd-config-volume\") pod \"alertmanager-main-0\" (UID: \"43e20c01-6600-4445-aee3-1a90c7aac2cd\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 17 17:37:05.020438 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:37:05.020348    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/43e20c01-6600-4445-aee3-1a90c7aac2cd-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"43e20c01-6600-4445-aee3-1a90c7aac2cd\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 17 17:37:05.020438 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:37:05.020365    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-lwbsr\" (UniqueName: \"kubernetes.io/projected/43e20c01-6600-4445-aee3-1a90c7aac2cd-kube-api-access-lwbsr\") pod \"alertmanager-main-0\" (UID: \"43e20c01-6600-4445-aee3-1a90c7aac2cd\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 17 17:37:05.020538 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:37:05.020428    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/43e20c01-6600-4445-aee3-1a90c7aac2cd-web-config\") pod \"alertmanager-main-0\" (UID: \"43e20c01-6600-4445-aee3-1a90c7aac2cd\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 17 17:37:05.020538 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:37:05.020471    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/43e20c01-6600-4445-aee3-1a90c7aac2cd-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"43e20c01-6600-4445-aee3-1a90c7aac2cd\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 17 17:37:05.020538 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:37:05.020519    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/43e20c01-6600-4445-aee3-1a90c7aac2cd-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"43e20c01-6600-4445-aee3-1a90c7aac2cd\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 17 17:37:05.020644 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:37:05.020545    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/43e20c01-6600-4445-aee3-1a90c7aac2cd-config-out\") pod \"alertmanager-main-0\" (UID: \"43e20c01-6600-4445-aee3-1a90c7aac2cd\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 17 17:37:05.020644 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:37:05.020578    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/43e20c01-6600-4445-aee3-1a90c7aac2cd-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"43e20c01-6600-4445-aee3-1a90c7aac2cd\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 17 17:37:05.020644 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:37:05.020610    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/43e20c01-6600-4445-aee3-1a90c7aac2cd-tls-assets\") pod \"alertmanager-main-0\" (UID: \"43e20c01-6600-4445-aee3-1a90c7aac2cd\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 17 17:37:05.121357 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:37:05.121331    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/43e20c01-6600-4445-aee3-1a90c7aac2cd-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"43e20c01-6600-4445-aee3-1a90c7aac2cd\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 17 17:37:05.121357 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:37:05.121358    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lwbsr\" (UniqueName: \"kubernetes.io/projected/43e20c01-6600-4445-aee3-1a90c7aac2cd-kube-api-access-lwbsr\") pod \"alertmanager-main-0\" (UID: \"43e20c01-6600-4445-aee3-1a90c7aac2cd\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 17 17:37:05.121536 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:37:05.121376    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/43e20c01-6600-4445-aee3-1a90c7aac2cd-web-config\") pod \"alertmanager-main-0\" (UID: \"43e20c01-6600-4445-aee3-1a90c7aac2cd\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 17 17:37:05.121536 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:37:05.121404    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/43e20c01-6600-4445-aee3-1a90c7aac2cd-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"43e20c01-6600-4445-aee3-1a90c7aac2cd\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 17 17:37:05.121536 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:37:05.121431    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/43e20c01-6600-4445-aee3-1a90c7aac2cd-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"43e20c01-6600-4445-aee3-1a90c7aac2cd\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 17 17:37:05.121536 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:37:05.121517    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/43e20c01-6600-4445-aee3-1a90c7aac2cd-config-out\") pod \"alertmanager-main-0\" (UID: \"43e20c01-6600-4445-aee3-1a90c7aac2cd\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 17 17:37:05.121722 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:37:05.121558    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/43e20c01-6600-4445-aee3-1a90c7aac2cd-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"43e20c01-6600-4445-aee3-1a90c7aac2cd\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 17 17:37:05.121722 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:37:05.121595    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/43e20c01-6600-4445-aee3-1a90c7aac2cd-tls-assets\") pod \"alertmanager-main-0\" (UID: \"43e20c01-6600-4445-aee3-1a90c7aac2cd\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 17 17:37:05.121722 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:37:05.121623    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/43e20c01-6600-4445-aee3-1a90c7aac2cd-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"43e20c01-6600-4445-aee3-1a90c7aac2cd\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 17 17:37:05.121722 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:37:05.121650    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/43e20c01-6600-4445-aee3-1a90c7aac2cd-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"43e20c01-6600-4445-aee3-1a90c7aac2cd\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 17 17:37:05.121722 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:37:05.121675    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/43e20c01-6600-4445-aee3-1a90c7aac2cd-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"43e20c01-6600-4445-aee3-1a90c7aac2cd\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 17 17:37:05.121966 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:37:05.121730    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/43e20c01-6600-4445-aee3-1a90c7aac2cd-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"43e20c01-6600-4445-aee3-1a90c7aac2cd\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 17 17:37:05.121966 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:37:05.121765    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/43e20c01-6600-4445-aee3-1a90c7aac2cd-config-volume\") pod \"alertmanager-main-0\" (UID: \"43e20c01-6600-4445-aee3-1a90c7aac2cd\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 17 17:37:05.121966 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:37:05.121771    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/43e20c01-6600-4445-aee3-1a90c7aac2cd-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"43e20c01-6600-4445-aee3-1a90c7aac2cd\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 17 17:37:05.122192 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:37:05.122170    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/43e20c01-6600-4445-aee3-1a90c7aac2cd-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"43e20c01-6600-4445-aee3-1a90c7aac2cd\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 17 17:37:05.122844 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:37:05.122821    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/43e20c01-6600-4445-aee3-1a90c7aac2cd-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"43e20c01-6600-4445-aee3-1a90c7aac2cd\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 17 17:37:05.124466 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:37:05.124411    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/43e20c01-6600-4445-aee3-1a90c7aac2cd-config-out\") pod \"alertmanager-main-0\" (UID: \"43e20c01-6600-4445-aee3-1a90c7aac2cd\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 17 17:37:05.124466 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:37:05.124455    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/43e20c01-6600-4445-aee3-1a90c7aac2cd-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"43e20c01-6600-4445-aee3-1a90c7aac2cd\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 17 17:37:05.124621 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:37:05.124513    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/43e20c01-6600-4445-aee3-1a90c7aac2cd-web-config\") pod \"alertmanager-main-0\" (UID: \"43e20c01-6600-4445-aee3-1a90c7aac2cd\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 17 17:37:05.124753 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:37:05.124728    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/43e20c01-6600-4445-aee3-1a90c7aac2cd-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"43e20c01-6600-4445-aee3-1a90c7aac2cd\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 17 17:37:05.124863 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:37:05.124847    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/43e20c01-6600-4445-aee3-1a90c7aac2cd-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"43e20c01-6600-4445-aee3-1a90c7aac2cd\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 17 17:37:05.124946 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:37:05.124929    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/43e20c01-6600-4445-aee3-1a90c7aac2cd-tls-assets\") pod \"alertmanager-main-0\" (UID: \"43e20c01-6600-4445-aee3-1a90c7aac2cd\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 17 17:37:05.125059 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:37:05.125043    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/43e20c01-6600-4445-aee3-1a90c7aac2cd-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"43e20c01-6600-4445-aee3-1a90c7aac2cd\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 17 17:37:05.125172 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:37:05.125158    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/43e20c01-6600-4445-aee3-1a90c7aac2cd-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"43e20c01-6600-4445-aee3-1a90c7aac2cd\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 17 17:37:05.125985 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:37:05.125962    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/43e20c01-6600-4445-aee3-1a90c7aac2cd-config-volume\") pod \"alertmanager-main-0\" (UID: \"43e20c01-6600-4445-aee3-1a90c7aac2cd\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 17 17:37:05.133356 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:37:05.133340    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-lwbsr\" (UniqueName: \"kubernetes.io/projected/43e20c01-6600-4445-aee3-1a90c7aac2cd-kube-api-access-lwbsr\") pod \"alertmanager-main-0\" (UID: \"43e20c01-6600-4445-aee3-1a90c7aac2cd\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 17 17:37:05.276288 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:37:05.276261    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0"
Sep 17 17:37:05.561863 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:37:05.561840    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/alertmanager-main-0"]
Sep 17 17:37:05.563150 ip-10-0-137-20 kubenswrapper[2565]: W0917 17:37:05.563127    2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod43e20c01_6600_4445_aee3_1a90c7aac2cd.slice/crio-2f1ff584c5d6904f96d62ec6b9957ec6bcd7f9129433cb5083f6cf3af769ccc1 WatchSource:0}: Error finding container 2f1ff584c5d6904f96d62ec6b9957ec6bcd7f9129433cb5083f6cf3af769ccc1: Status 404 returned error can't find the container with id 2f1ff584c5d6904f96d62ec6b9957ec6bcd7f9129433cb5083f6cf3af769ccc1
Sep 17 17:37:05.888165 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:37:05.888140    2565 generic.go:356] "Generic (PLEG): container finished" podID="43e20c01-6600-4445-aee3-1a90c7aac2cd" containerID="7453eded3dc454a84d47a514002675ad9ee32c15994d85c5373685b6b90fb621" exitCode=0
Sep 17 17:37:05.888486 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:37:05.888228    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"43e20c01-6600-4445-aee3-1a90c7aac2cd","Type":"ContainerDied","Data":"7453eded3dc454a84d47a514002675ad9ee32c15994d85c5373685b6b90fb621"}
Sep 17 17:37:05.888486 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:37:05.888291    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"43e20c01-6600-4445-aee3-1a90c7aac2cd","Type":"ContainerStarted","Data":"2f1ff584c5d6904f96d62ec6b9957ec6bcd7f9129433cb5083f6cf3af769ccc1"}
Sep 17 17:37:06.155733 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:37:06.155707    2565 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="16a894e3-8971-45a7-9f5e-c1da4d5b932f" path="/var/lib/kubelet/pods/16a894e3-8971-45a7-9f5e-c1da4d5b932f/volumes"
Sep 17 17:37:06.896599 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:37:06.896571    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"43e20c01-6600-4445-aee3-1a90c7aac2cd","Type":"ContainerStarted","Data":"951b711c481292714516ae8eba22fb8278faf8d3b069d1acef8297c972bb72bd"}
Sep 17 17:37:06.896925 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:37:06.896640    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"43e20c01-6600-4445-aee3-1a90c7aac2cd","Type":"ContainerStarted","Data":"bfc9e9a1c8bae505b0422db4b37a5a27727972f47eb208e8dd7bd11ca60f977c"}
Sep 17 17:37:06.896925 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:37:06.896673    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"43e20c01-6600-4445-aee3-1a90c7aac2cd","Type":"ContainerStarted","Data":"b817e9ed4b692374257d7864e6b08619f902df08a83088bcaf793831d228ae48"}
Sep 17 17:37:06.896925 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:37:06.896697    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"43e20c01-6600-4445-aee3-1a90c7aac2cd","Type":"ContainerStarted","Data":"df2b0a9b37c4702ad83dbf47a4f6659934ee688957c873000c828d43329cf618"}
Sep 17 17:37:06.896925 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:37:06.896729    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"43e20c01-6600-4445-aee3-1a90c7aac2cd","Type":"ContainerStarted","Data":"51278a4448e6372b11ce101b288c9a76e304442a481ddff393e455a85929fe49"}
Sep 17 17:37:06.896925 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:37:06.896756    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"43e20c01-6600-4445-aee3-1a90c7aac2cd","Type":"ContainerStarted","Data":"84dee9246747fc88216d805ab0b06cd083e237256fdac525b7ea1704cf7e1aee"}
Sep 17 17:37:06.935652 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:37:06.935042    2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/alertmanager-main-0" podStartSLOduration=2.918708498 podStartE2EDuration="2.935005682s" totalImagesPullingTime="0s" totalInitContainerRuntime="16.297184ms" isStatefulPod=false podCreationTimestamp="2026-09-17 17:37:04 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-17 17:37:06.93326778 +0000 UTC m=+176.810552195" watchObservedRunningTime="2026-09-17 17:37:06.935005682 +0000 UTC m=+176.812290098"
Sep 17 17:37:07.464820 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:37:07.463987    2565 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/telemeter-client-8485894d99-zw8mr"]
Sep 17 17:37:07.467840 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:37:07.467822    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/telemeter-client-8485894d99-zw8mr"
Sep 17 17:37:07.478974 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:37:07.478954    2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"telemeter-client-tls\""
Sep 17 17:37:07.479183 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:37:07.479164    2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"telemeter-client-serving-certs-ca-bundle\""
Sep 17 17:37:07.479256 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:37:07.479209    2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"telemeter-client-dockercfg-v7dgt\""
Sep 17 17:37:07.479338 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:37:07.479264    2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"federate-client-certs\""
Sep 17 17:37:07.479338 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:37:07.479295    2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"telemeter-client-kube-rbac-proxy-config\""
Sep 17 17:37:07.479427 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:37:07.479334    2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"telemeter-client\""
Sep 17 17:37:07.484609 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:37:07.484590    2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"telemeter-trusted-ca-bundle-8i12ta5c71j38\""
Sep 17 17:37:07.489131 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:37:07.489110    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/telemeter-client-8485894d99-zw8mr"]
Sep 17 17:37:07.537518 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:37:07.537483    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/aadb9010-ba56-493a-bb4e-8cbb1bf77555-metrics-client-ca\") pod \"telemeter-client-8485894d99-zw8mr\" (UID: \"aadb9010-ba56-493a-bb4e-8cbb1bf77555\") " pod="openshift-monitoring/telemeter-client-8485894d99-zw8mr"
Sep 17 17:37:07.537518 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:37:07.537519    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"telemeter-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/aadb9010-ba56-493a-bb4e-8cbb1bf77555-telemeter-trusted-ca-bundle\") pod \"telemeter-client-8485894d99-zw8mr\" (UID: \"aadb9010-ba56-493a-bb4e-8cbb1bf77555\") " pod="openshift-monitoring/telemeter-client-8485894d99-zw8mr"
Sep 17 17:37:07.537717 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:37:07.537540    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dtp2z\" (UniqueName: \"kubernetes.io/projected/aadb9010-ba56-493a-bb4e-8cbb1bf77555-kube-api-access-dtp2z\") pod \"telemeter-client-8485894d99-zw8mr\" (UID: \"aadb9010-ba56-493a-bb4e-8cbb1bf77555\") " pod="openshift-monitoring/telemeter-client-8485894d99-zw8mr"
Sep 17 17:37:07.537717 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:37:07.537563    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/aadb9010-ba56-493a-bb4e-8cbb1bf77555-serving-certs-ca-bundle\") pod \"telemeter-client-8485894d99-zw8mr\" (UID: \"aadb9010-ba56-493a-bb4e-8cbb1bf77555\") " pod="openshift-monitoring/telemeter-client-8485894d99-zw8mr"
Sep 17 17:37:07.537717 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:37:07.537640    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"telemeter-client-tls\" (UniqueName: \"kubernetes.io/secret/aadb9010-ba56-493a-bb4e-8cbb1bf77555-telemeter-client-tls\") pod \"telemeter-client-8485894d99-zw8mr\" (UID: \"aadb9010-ba56-493a-bb4e-8cbb1bf77555\") " pod="openshift-monitoring/telemeter-client-8485894d99-zw8mr"
Sep 17 17:37:07.537717 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:37:07.537673    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"federate-client-tls\" (UniqueName: \"kubernetes.io/secret/aadb9010-ba56-493a-bb4e-8cbb1bf77555-federate-client-tls\") pod \"telemeter-client-8485894d99-zw8mr\" (UID: \"aadb9010-ba56-493a-bb4e-8cbb1bf77555\") " pod="openshift-monitoring/telemeter-client-8485894d99-zw8mr"
Sep 17 17:37:07.537717 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:37:07.537693    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-telemeter-client\" (UniqueName: \"kubernetes.io/secret/aadb9010-ba56-493a-bb4e-8cbb1bf77555-secret-telemeter-client\") pod \"telemeter-client-8485894d99-zw8mr\" (UID: \"aadb9010-ba56-493a-bb4e-8cbb1bf77555\") " pod="openshift-monitoring/telemeter-client-8485894d99-zw8mr"
Sep 17 17:37:07.537879 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:37:07.537719    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-telemeter-client-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/aadb9010-ba56-493a-bb4e-8cbb1bf77555-secret-telemeter-client-kube-rbac-proxy-config\") pod \"telemeter-client-8485894d99-zw8mr\" (UID: \"aadb9010-ba56-493a-bb4e-8cbb1bf77555\") " pod="openshift-monitoring/telemeter-client-8485894d99-zw8mr"
Sep 17 17:37:07.638403 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:37:07.638363    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/aadb9010-ba56-493a-bb4e-8cbb1bf77555-metrics-client-ca\") pod \"telemeter-client-8485894d99-zw8mr\" (UID: \"aadb9010-ba56-493a-bb4e-8cbb1bf77555\") " pod="openshift-monitoring/telemeter-client-8485894d99-zw8mr"
Sep 17 17:37:07.638403 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:37:07.638402    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"telemeter-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/aadb9010-ba56-493a-bb4e-8cbb1bf77555-telemeter-trusted-ca-bundle\") pod \"telemeter-client-8485894d99-zw8mr\" (UID: \"aadb9010-ba56-493a-bb4e-8cbb1bf77555\") " pod="openshift-monitoring/telemeter-client-8485894d99-zw8mr"
Sep 17 17:37:07.638625 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:37:07.638422    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dtp2z\" (UniqueName: \"kubernetes.io/projected/aadb9010-ba56-493a-bb4e-8cbb1bf77555-kube-api-access-dtp2z\") pod \"telemeter-client-8485894d99-zw8mr\" (UID: \"aadb9010-ba56-493a-bb4e-8cbb1bf77555\") " pod="openshift-monitoring/telemeter-client-8485894d99-zw8mr"
Sep 17 17:37:07.638625 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:37:07.638558    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/aadb9010-ba56-493a-bb4e-8cbb1bf77555-serving-certs-ca-bundle\") pod \"telemeter-client-8485894d99-zw8mr\" (UID: \"aadb9010-ba56-493a-bb4e-8cbb1bf77555\") " pod="openshift-monitoring/telemeter-client-8485894d99-zw8mr"
Sep 17 17:37:07.638720 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:37:07.638634    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"telemeter-client-tls\" (UniqueName: \"kubernetes.io/secret/aadb9010-ba56-493a-bb4e-8cbb1bf77555-telemeter-client-tls\") pod \"telemeter-client-8485894d99-zw8mr\" (UID: \"aadb9010-ba56-493a-bb4e-8cbb1bf77555\") " pod="openshift-monitoring/telemeter-client-8485894d99-zw8mr"
Sep 17 17:37:07.638720 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:37:07.638669    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"federate-client-tls\" (UniqueName: \"kubernetes.io/secret/aadb9010-ba56-493a-bb4e-8cbb1bf77555-federate-client-tls\") pod \"telemeter-client-8485894d99-zw8mr\" (UID: \"aadb9010-ba56-493a-bb4e-8cbb1bf77555\") " pod="openshift-monitoring/telemeter-client-8485894d99-zw8mr"
Sep 17 17:37:07.638720 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:37:07.638696    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-telemeter-client\" (UniqueName: \"kubernetes.io/secret/aadb9010-ba56-493a-bb4e-8cbb1bf77555-secret-telemeter-client\") pod \"telemeter-client-8485894d99-zw8mr\" (UID: \"aadb9010-ba56-493a-bb4e-8cbb1bf77555\") " pod="openshift-monitoring/telemeter-client-8485894d99-zw8mr"
Sep 17 17:37:07.638879 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:37:07.638740    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-telemeter-client-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/aadb9010-ba56-493a-bb4e-8cbb1bf77555-secret-telemeter-client-kube-rbac-proxy-config\") pod \"telemeter-client-8485894d99-zw8mr\" (UID: \"aadb9010-ba56-493a-bb4e-8cbb1bf77555\") " pod="openshift-monitoring/telemeter-client-8485894d99-zw8mr"
Sep 17 17:37:07.639227 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:37:07.639198    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/aadb9010-ba56-493a-bb4e-8cbb1bf77555-metrics-client-ca\") pod \"telemeter-client-8485894d99-zw8mr\" (UID: \"aadb9010-ba56-493a-bb4e-8cbb1bf77555\") " pod="openshift-monitoring/telemeter-client-8485894d99-zw8mr"
Sep 17 17:37:07.639335 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:37:07.639272    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/aadb9010-ba56-493a-bb4e-8cbb1bf77555-serving-certs-ca-bundle\") pod \"telemeter-client-8485894d99-zw8mr\" (UID: \"aadb9010-ba56-493a-bb4e-8cbb1bf77555\") " pod="openshift-monitoring/telemeter-client-8485894d99-zw8mr"
Sep 17 17:37:07.639501 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:37:07.639468    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"telemeter-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/aadb9010-ba56-493a-bb4e-8cbb1bf77555-telemeter-trusted-ca-bundle\") pod \"telemeter-client-8485894d99-zw8mr\" (UID: \"aadb9010-ba56-493a-bb4e-8cbb1bf77555\") " pod="openshift-monitoring/telemeter-client-8485894d99-zw8mr"
Sep 17 17:37:07.641697 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:37:07.641655    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"federate-client-tls\" (UniqueName: \"kubernetes.io/secret/aadb9010-ba56-493a-bb4e-8cbb1bf77555-federate-client-tls\") pod \"telemeter-client-8485894d99-zw8mr\" (UID: \"aadb9010-ba56-493a-bb4e-8cbb1bf77555\") " pod="openshift-monitoring/telemeter-client-8485894d99-zw8mr"
Sep 17 17:37:07.641800 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:37:07.641712    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-telemeter-client-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/aadb9010-ba56-493a-bb4e-8cbb1bf77555-secret-telemeter-client-kube-rbac-proxy-config\") pod \"telemeter-client-8485894d99-zw8mr\" (UID: \"aadb9010-ba56-493a-bb4e-8cbb1bf77555\") " pod="openshift-monitoring/telemeter-client-8485894d99-zw8mr"
Sep 17 17:37:07.642197 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:37:07.642136    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"telemeter-client-tls\" (UniqueName: \"kubernetes.io/secret/aadb9010-ba56-493a-bb4e-8cbb1bf77555-telemeter-client-tls\") pod \"telemeter-client-8485894d99-zw8mr\" (UID: \"aadb9010-ba56-493a-bb4e-8cbb1bf77555\") " pod="openshift-monitoring/telemeter-client-8485894d99-zw8mr"
Sep 17 17:37:07.642713 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:37:07.642693    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-telemeter-client\" (UniqueName: \"kubernetes.io/secret/aadb9010-ba56-493a-bb4e-8cbb1bf77555-secret-telemeter-client\") pod \"telemeter-client-8485894d99-zw8mr\" (UID: \"aadb9010-ba56-493a-bb4e-8cbb1bf77555\") " pod="openshift-monitoring/telemeter-client-8485894d99-zw8mr"
Sep 17 17:37:07.652475 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:37:07.652450    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-dtp2z\" (UniqueName: \"kubernetes.io/projected/aadb9010-ba56-493a-bb4e-8cbb1bf77555-kube-api-access-dtp2z\") pod \"telemeter-client-8485894d99-zw8mr\" (UID: \"aadb9010-ba56-493a-bb4e-8cbb1bf77555\") " pod="openshift-monitoring/telemeter-client-8485894d99-zw8mr"
Sep 17 17:37:07.777705 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:37:07.777623    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/telemeter-client-8485894d99-zw8mr"
Sep 17 17:37:08.071895 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:37:08.071862    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/telemeter-client-8485894d99-zw8mr"]
Sep 17 17:37:08.072372 ip-10-0-137-20 kubenswrapper[2565]: W0917 17:37:08.072337    2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podaadb9010_ba56_493a_bb4e_8cbb1bf77555.slice/crio-932a6d36ffb0e29410ba6bf369e6ab1fed9cee958b24f40c624178125eeb24e2 WatchSource:0}: Error finding container 932a6d36ffb0e29410ba6bf369e6ab1fed9cee958b24f40c624178125eeb24e2: Status 404 returned error can't find the container with id 932a6d36ffb0e29410ba6bf369e6ab1fed9cee958b24f40c624178125eeb24e2
Sep 17 17:37:08.906837 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:37:08.906803    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/telemeter-client-8485894d99-zw8mr" event={"ID":"aadb9010-ba56-493a-bb4e-8cbb1bf77555","Type":"ContainerStarted","Data":"932a6d36ffb0e29410ba6bf369e6ab1fed9cee958b24f40c624178125eeb24e2"}
Sep 17 17:37:09.911983 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:37:09.911957    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/telemeter-client-8485894d99-zw8mr" event={"ID":"aadb9010-ba56-493a-bb4e-8cbb1bf77555","Type":"ContainerStarted","Data":"491697a185941d8c249c78e52f8901a17f86c4d8785b369afb264424260170c9"}
Sep 17 17:37:09.912291 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:37:09.912071    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/telemeter-client-8485894d99-zw8mr" event={"ID":"aadb9010-ba56-493a-bb4e-8cbb1bf77555","Type":"ContainerStarted","Data":"edc7fe441ba5e836d2087622717c3b1c5e03861df9dbed4c16101978c2eaad2e"}
Sep 17 17:37:10.917319 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:37:10.917283    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/telemeter-client-8485894d99-zw8mr" event={"ID":"aadb9010-ba56-493a-bb4e-8cbb1bf77555","Type":"ContainerStarted","Data":"be6252208a7d3ef9679b48a194f81c56e021ed60b6756fb5b6141af2fa524955"}
Sep 17 17:37:10.955432 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:37:10.955383    2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/telemeter-client-8485894d99-zw8mr" podStartSLOduration=2.286719899 podStartE2EDuration="3.955369648s" totalImagesPullingTime="1.668649749s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-17 17:37:07 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-17 17:37:10.953191586 +0000 UTC m=+180.830476004" watchObservedRunningTime="2026-09-17 17:37:10.955369648 +0000 UTC m=+180.832654063"
Sep 17 17:37:11.075066 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:37:11.075045    2565 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-654c5b799-l26tj"]
Sep 17 17:37:11.078192 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:37:11.078177    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-654c5b799-l26tj"
Sep 17 17:37:11.097538 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:37:11.097517    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-654c5b799-l26tj"]
Sep 17 17:37:11.167131 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:37:11.167105    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/20a613bd-5bb4-4a02-bf07-c1d15356894c-trusted-ca-bundle\") pod \"console-654c5b799-l26tj\" (UID: \"20a613bd-5bb4-4a02-bf07-c1d15356894c\") " pod="openshift-console/console-654c5b799-l26tj"
Sep 17 17:37:11.167131 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:37:11.167132    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/20a613bd-5bb4-4a02-bf07-c1d15356894c-console-oauth-config\") pod \"console-654c5b799-l26tj\" (UID: \"20a613bd-5bb4-4a02-bf07-c1d15356894c\") " pod="openshift-console/console-654c5b799-l26tj"
Sep 17 17:37:11.167280 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:37:11.167156    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/20a613bd-5bb4-4a02-bf07-c1d15356894c-oauth-serving-cert\") pod \"console-654c5b799-l26tj\" (UID: \"20a613bd-5bb4-4a02-bf07-c1d15356894c\") " pod="openshift-console/console-654c5b799-l26tj"
Sep 17 17:37:11.167280 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:37:11.167185    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/20a613bd-5bb4-4a02-bf07-c1d15356894c-console-config\") pod \"console-654c5b799-l26tj\" (UID: \"20a613bd-5bb4-4a02-bf07-c1d15356894c\") " pod="openshift-console/console-654c5b799-l26tj"
Sep 17 17:37:11.167280 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:37:11.167219    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/20a613bd-5bb4-4a02-bf07-c1d15356894c-console-serving-cert\") pod \"console-654c5b799-l26tj\" (UID: \"20a613bd-5bb4-4a02-bf07-c1d15356894c\") " pod="openshift-console/console-654c5b799-l26tj"
Sep 17 17:37:11.167280 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:37:11.167234    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8wrcx\" (UniqueName: \"kubernetes.io/projected/20a613bd-5bb4-4a02-bf07-c1d15356894c-kube-api-access-8wrcx\") pod \"console-654c5b799-l26tj\" (UID: \"20a613bd-5bb4-4a02-bf07-c1d15356894c\") " pod="openshift-console/console-654c5b799-l26tj"
Sep 17 17:37:11.167425 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:37:11.167290    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/20a613bd-5bb4-4a02-bf07-c1d15356894c-service-ca\") pod \"console-654c5b799-l26tj\" (UID: \"20a613bd-5bb4-4a02-bf07-c1d15356894c\") " pod="openshift-console/console-654c5b799-l26tj"
Sep 17 17:37:11.267557 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:37:11.267533    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/20a613bd-5bb4-4a02-bf07-c1d15356894c-oauth-serving-cert\") pod \"console-654c5b799-l26tj\" (UID: \"20a613bd-5bb4-4a02-bf07-c1d15356894c\") " pod="openshift-console/console-654c5b799-l26tj"
Sep 17 17:37:11.267670 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:37:11.267564    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/20a613bd-5bb4-4a02-bf07-c1d15356894c-console-config\") pod \"console-654c5b799-l26tj\" (UID: \"20a613bd-5bb4-4a02-bf07-c1d15356894c\") " pod="openshift-console/console-654c5b799-l26tj"
Sep 17 17:37:11.267670 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:37:11.267584    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/20a613bd-5bb4-4a02-bf07-c1d15356894c-console-serving-cert\") pod \"console-654c5b799-l26tj\" (UID: \"20a613bd-5bb4-4a02-bf07-c1d15356894c\") " pod="openshift-console/console-654c5b799-l26tj"
Sep 17 17:37:11.267670 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:37:11.267599    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8wrcx\" (UniqueName: \"kubernetes.io/projected/20a613bd-5bb4-4a02-bf07-c1d15356894c-kube-api-access-8wrcx\") pod \"console-654c5b799-l26tj\" (UID: \"20a613bd-5bb4-4a02-bf07-c1d15356894c\") " pod="openshift-console/console-654c5b799-l26tj"
Sep 17 17:37:11.267670 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:37:11.267631    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/20a613bd-5bb4-4a02-bf07-c1d15356894c-service-ca\") pod \"console-654c5b799-l26tj\" (UID: \"20a613bd-5bb4-4a02-bf07-c1d15356894c\") " pod="openshift-console/console-654c5b799-l26tj"
Sep 17 17:37:11.267879 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:37:11.267670    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/20a613bd-5bb4-4a02-bf07-c1d15356894c-trusted-ca-bundle\") pod \"console-654c5b799-l26tj\" (UID: \"20a613bd-5bb4-4a02-bf07-c1d15356894c\") " pod="openshift-console/console-654c5b799-l26tj"
Sep 17 17:37:11.267879 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:37:11.267694    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/20a613bd-5bb4-4a02-bf07-c1d15356894c-console-oauth-config\") pod \"console-654c5b799-l26tj\" (UID: \"20a613bd-5bb4-4a02-bf07-c1d15356894c\") " pod="openshift-console/console-654c5b799-l26tj"
Sep 17 17:37:11.268365 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:37:11.268338    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/20a613bd-5bb4-4a02-bf07-c1d15356894c-oauth-serving-cert\") pod \"console-654c5b799-l26tj\" (UID: \"20a613bd-5bb4-4a02-bf07-c1d15356894c\") " pod="openshift-console/console-654c5b799-l26tj"
Sep 17 17:37:11.268466 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:37:11.268367    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/20a613bd-5bb4-4a02-bf07-c1d15356894c-service-ca\") pod \"console-654c5b799-l26tj\" (UID: \"20a613bd-5bb4-4a02-bf07-c1d15356894c\") " pod="openshift-console/console-654c5b799-l26tj"
Sep 17 17:37:11.268466 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:37:11.268369    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/20a613bd-5bb4-4a02-bf07-c1d15356894c-console-config\") pod \"console-654c5b799-l26tj\" (UID: \"20a613bd-5bb4-4a02-bf07-c1d15356894c\") " pod="openshift-console/console-654c5b799-l26tj"
Sep 17 17:37:11.268541 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:37:11.268482    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/20a613bd-5bb4-4a02-bf07-c1d15356894c-trusted-ca-bundle\") pod \"console-654c5b799-l26tj\" (UID: \"20a613bd-5bb4-4a02-bf07-c1d15356894c\") " pod="openshift-console/console-654c5b799-l26tj"
Sep 17 17:37:11.269890 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:37:11.269874    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/20a613bd-5bb4-4a02-bf07-c1d15356894c-console-oauth-config\") pod \"console-654c5b799-l26tj\" (UID: \"20a613bd-5bb4-4a02-bf07-c1d15356894c\") " pod="openshift-console/console-654c5b799-l26tj"
Sep 17 17:37:11.270014 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:37:11.269997    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/20a613bd-5bb4-4a02-bf07-c1d15356894c-console-serving-cert\") pod \"console-654c5b799-l26tj\" (UID: \"20a613bd-5bb4-4a02-bf07-c1d15356894c\") " pod="openshift-console/console-654c5b799-l26tj"
Sep 17 17:37:11.277334 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:37:11.277317    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-8wrcx\" (UniqueName: \"kubernetes.io/projected/20a613bd-5bb4-4a02-bf07-c1d15356894c-kube-api-access-8wrcx\") pod \"console-654c5b799-l26tj\" (UID: \"20a613bd-5bb4-4a02-bf07-c1d15356894c\") " pod="openshift-console/console-654c5b799-l26tj"
Sep 17 17:37:11.387486 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:37:11.387463    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-654c5b799-l26tj"
Sep 17 17:37:11.670418 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:37:11.670391    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-654c5b799-l26tj"]
Sep 17 17:37:11.676009 ip-10-0-137-20 kubenswrapper[2565]: W0917 17:37:11.675981    2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod20a613bd_5bb4_4a02_bf07_c1d15356894c.slice/crio-5bf28a678d74ce62e52d0f0fc5d1d75d928afe3066b4cca27eac0fd1392c53b2 WatchSource:0}: Error finding container 5bf28a678d74ce62e52d0f0fc5d1d75d928afe3066b4cca27eac0fd1392c53b2: Status 404 returned error can't find the container with id 5bf28a678d74ce62e52d0f0fc5d1d75d928afe3066b4cca27eac0fd1392c53b2
Sep 17 17:37:11.923693 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:37:11.923653    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-654c5b799-l26tj" event={"ID":"20a613bd-5bb4-4a02-bf07-c1d15356894c","Type":"ContainerStarted","Data":"6bdb5859e85a51364ef6d11f12ba7d4519c95feef0dfbab000eaa10c0e40d466"}
Sep 17 17:37:11.924169 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:37:11.923713    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-654c5b799-l26tj" event={"ID":"20a613bd-5bb4-4a02-bf07-c1d15356894c","Type":"ContainerStarted","Data":"5bf28a678d74ce62e52d0f0fc5d1d75d928afe3066b4cca27eac0fd1392c53b2"}
Sep 17 17:37:11.947042 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:37:11.946935    2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-654c5b799-l26tj" podStartSLOduration=0.946921812 podStartE2EDuration="946.921812ms" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-17 17:37:11 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-17 17:37:11.945016056 +0000 UTC m=+181.822300497" watchObservedRunningTime="2026-09-17 17:37:11.946921812 +0000 UTC m=+181.824206466"
Sep 17 17:37:21.387711 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:37:21.387683    2565 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-654c5b799-l26tj"
Sep 17 17:37:21.388105 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:37:21.387733    2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-654c5b799-l26tj"
Sep 17 17:37:21.392317 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:37:21.392294    2565 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-654c5b799-l26tj"
Sep 17 17:37:21.963375 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:37:21.963353    2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-654c5b799-l26tj"
Sep 17 17:37:22.014838 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:37:22.014801    2565 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-7b8db48b-rv5v5"]
Sep 17 17:37:47.033566 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:37:47.033520    2565 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-console/console-7b8db48b-rv5v5" podUID="7e66f8c1-3440-4ab4-8cef-1c5f2fe893c4" containerName="console" containerID="cri-o://a5f3cbc93af022831ef9950368b40b5296cbca3b5287d6672a89425cb84331f5" gracePeriod=15
Sep 17 17:37:47.276114 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:37:47.276090    2565 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-7b8db48b-rv5v5"
Sep 17 17:37:47.413436 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:37:47.413414    2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/7e66f8c1-3440-4ab4-8cef-1c5f2fe893c4-oauth-serving-cert\") pod \"7e66f8c1-3440-4ab4-8cef-1c5f2fe893c4\" (UID: \"7e66f8c1-3440-4ab4-8cef-1c5f2fe893c4\") "
Sep 17 17:37:47.413552 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:37:47.413441    2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/7e66f8c1-3440-4ab4-8cef-1c5f2fe893c4-service-ca\") pod \"7e66f8c1-3440-4ab4-8cef-1c5f2fe893c4\" (UID: \"7e66f8c1-3440-4ab4-8cef-1c5f2fe893c4\") "
Sep 17 17:37:47.413552 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:37:47.413493    2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/7e66f8c1-3440-4ab4-8cef-1c5f2fe893c4-console-serving-cert\") pod \"7e66f8c1-3440-4ab4-8cef-1c5f2fe893c4\" (UID: \"7e66f8c1-3440-4ab4-8cef-1c5f2fe893c4\") "
Sep 17 17:37:47.413552 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:37:47.413511    2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-77h6g\" (UniqueName: \"kubernetes.io/projected/7e66f8c1-3440-4ab4-8cef-1c5f2fe893c4-kube-api-access-77h6g\") pod \"7e66f8c1-3440-4ab4-8cef-1c5f2fe893c4\" (UID: \"7e66f8c1-3440-4ab4-8cef-1c5f2fe893c4\") "
Sep 17 17:37:47.413552 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:37:47.413535    2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/7e66f8c1-3440-4ab4-8cef-1c5f2fe893c4-console-config\") pod \"7e66f8c1-3440-4ab4-8cef-1c5f2fe893c4\" (UID: \"7e66f8c1-3440-4ab4-8cef-1c5f2fe893c4\") "
Sep 17 17:37:47.413552 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:37:47.413551    2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/7e66f8c1-3440-4ab4-8cef-1c5f2fe893c4-trusted-ca-bundle\") pod \"7e66f8c1-3440-4ab4-8cef-1c5f2fe893c4\" (UID: \"7e66f8c1-3440-4ab4-8cef-1c5f2fe893c4\") "
Sep 17 17:37:47.413817 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:37:47.413589    2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/7e66f8c1-3440-4ab4-8cef-1c5f2fe893c4-console-oauth-config\") pod \"7e66f8c1-3440-4ab4-8cef-1c5f2fe893c4\" (UID: \"7e66f8c1-3440-4ab4-8cef-1c5f2fe893c4\") "
Sep 17 17:37:47.413924 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:37:47.413808    2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/7e66f8c1-3440-4ab4-8cef-1c5f2fe893c4-oauth-serving-cert" (OuterVolumeSpecName: "oauth-serving-cert") pod "7e66f8c1-3440-4ab4-8cef-1c5f2fe893c4" (UID: "7e66f8c1-3440-4ab4-8cef-1c5f2fe893c4"). InnerVolumeSpecName "oauth-serving-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 17 17:37:47.414066 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:37:47.413981    2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/7e66f8c1-3440-4ab4-8cef-1c5f2fe893c4-console-config" (OuterVolumeSpecName: "console-config") pod "7e66f8c1-3440-4ab4-8cef-1c5f2fe893c4" (UID: "7e66f8c1-3440-4ab4-8cef-1c5f2fe893c4"). InnerVolumeSpecName "console-config". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 17 17:37:47.414066 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:37:47.414000    2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/7e66f8c1-3440-4ab4-8cef-1c5f2fe893c4-service-ca" (OuterVolumeSpecName: "service-ca") pod "7e66f8c1-3440-4ab4-8cef-1c5f2fe893c4" (UID: "7e66f8c1-3440-4ab4-8cef-1c5f2fe893c4"). InnerVolumeSpecName "service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 17 17:37:47.414163 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:37:47.414098    2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/7e66f8c1-3440-4ab4-8cef-1c5f2fe893c4-trusted-ca-bundle" (OuterVolumeSpecName: "trusted-ca-bundle") pod "7e66f8c1-3440-4ab4-8cef-1c5f2fe893c4" (UID: "7e66f8c1-3440-4ab4-8cef-1c5f2fe893c4"). InnerVolumeSpecName "trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 17 17:37:47.415565 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:37:47.415538    2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/7e66f8c1-3440-4ab4-8cef-1c5f2fe893c4-console-serving-cert" (OuterVolumeSpecName: "console-serving-cert") pod "7e66f8c1-3440-4ab4-8cef-1c5f2fe893c4" (UID: "7e66f8c1-3440-4ab4-8cef-1c5f2fe893c4"). InnerVolumeSpecName "console-serving-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 17 17:37:47.415650 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:37:47.415626    2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/7e66f8c1-3440-4ab4-8cef-1c5f2fe893c4-console-oauth-config" (OuterVolumeSpecName: "console-oauth-config") pod "7e66f8c1-3440-4ab4-8cef-1c5f2fe893c4" (UID: "7e66f8c1-3440-4ab4-8cef-1c5f2fe893c4"). InnerVolumeSpecName "console-oauth-config". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 17 17:37:47.415697 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:37:47.415667    2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/7e66f8c1-3440-4ab4-8cef-1c5f2fe893c4-kube-api-access-77h6g" (OuterVolumeSpecName: "kube-api-access-77h6g") pod "7e66f8c1-3440-4ab4-8cef-1c5f2fe893c4" (UID: "7e66f8c1-3440-4ab4-8cef-1c5f2fe893c4"). InnerVolumeSpecName "kube-api-access-77h6g". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 17 17:37:47.514258 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:37:47.514227    2565 reconciler_common.go:299] "Volume detached for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/7e66f8c1-3440-4ab4-8cef-1c5f2fe893c4-console-serving-cert\") on node \"ip-10-0-137-20.ec2.internal\" DevicePath \"\""
Sep 17 17:37:47.514258 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:37:47.514255    2565 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-77h6g\" (UniqueName: \"kubernetes.io/projected/7e66f8c1-3440-4ab4-8cef-1c5f2fe893c4-kube-api-access-77h6g\") on node \"ip-10-0-137-20.ec2.internal\" DevicePath \"\""
Sep 17 17:37:47.514258 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:37:47.514264    2565 reconciler_common.go:299] "Volume detached for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/7e66f8c1-3440-4ab4-8cef-1c5f2fe893c4-console-config\") on node \"ip-10-0-137-20.ec2.internal\" DevicePath \"\""
Sep 17 17:37:47.514412 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:37:47.514272    2565 reconciler_common.go:299] "Volume detached for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/7e66f8c1-3440-4ab4-8cef-1c5f2fe893c4-trusted-ca-bundle\") on node \"ip-10-0-137-20.ec2.internal\" DevicePath \"\""
Sep 17 17:37:47.514412 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:37:47.514280    2565 reconciler_common.go:299] "Volume detached for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/7e66f8c1-3440-4ab4-8cef-1c5f2fe893c4-console-oauth-config\") on node \"ip-10-0-137-20.ec2.internal\" DevicePath \"\""
Sep 17 17:37:47.514412 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:37:47.514288    2565 reconciler_common.go:299] "Volume detached for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/7e66f8c1-3440-4ab4-8cef-1c5f2fe893c4-oauth-serving-cert\") on node \"ip-10-0-137-20.ec2.internal\" DevicePath \"\""
Sep 17 17:37:47.514412 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:37:47.514296    2565 reconciler_common.go:299] "Volume detached for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/7e66f8c1-3440-4ab4-8cef-1c5f2fe893c4-service-ca\") on node \"ip-10-0-137-20.ec2.internal\" DevicePath \"\""
Sep 17 17:37:48.053108 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:37:48.053076    2565 generic.go:356] "Generic (PLEG): container finished" podID="7e66f8c1-3440-4ab4-8cef-1c5f2fe893c4" containerID="a5f3cbc93af022831ef9950368b40b5296cbca3b5287d6672a89425cb84331f5" exitCode=0
Sep 17 17:37:48.053483 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:37:48.053118    2565 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-7b8db48b-rv5v5"
Sep 17 17:37:48.053483 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:37:48.053148    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-7b8db48b-rv5v5" event={"ID":"7e66f8c1-3440-4ab4-8cef-1c5f2fe893c4","Type":"ContainerDied","Data":"a5f3cbc93af022831ef9950368b40b5296cbca3b5287d6672a89425cb84331f5"}
Sep 17 17:37:48.053483 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:37:48.053189    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-7b8db48b-rv5v5" event={"ID":"7e66f8c1-3440-4ab4-8cef-1c5f2fe893c4","Type":"ContainerDied","Data":"9d6ebde33baacfa32cb81bd556604001fb843cd576044d849086d4a992431bfd"}
Sep 17 17:37:48.053483 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:37:48.053208    2565 scope.go:117] "RemoveContainer" containerID="a5f3cbc93af022831ef9950368b40b5296cbca3b5287d6672a89425cb84331f5"
Sep 17 17:37:48.061649 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:37:48.061633    2565 scope.go:117] "RemoveContainer" containerID="a5f3cbc93af022831ef9950368b40b5296cbca3b5287d6672a89425cb84331f5"
Sep 17 17:37:48.061927 ip-10-0-137-20 kubenswrapper[2565]: E0917 17:37:48.061908    2565 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"a5f3cbc93af022831ef9950368b40b5296cbca3b5287d6672a89425cb84331f5\": container with ID starting with a5f3cbc93af022831ef9950368b40b5296cbca3b5287d6672a89425cb84331f5 not found: ID does not exist" containerID="a5f3cbc93af022831ef9950368b40b5296cbca3b5287d6672a89425cb84331f5"
Sep 17 17:37:48.062003 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:37:48.061936    2565 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"a5f3cbc93af022831ef9950368b40b5296cbca3b5287d6672a89425cb84331f5"} err="failed to get container status \"a5f3cbc93af022831ef9950368b40b5296cbca3b5287d6672a89425cb84331f5\": rpc error: code = NotFound desc = could not find container \"a5f3cbc93af022831ef9950368b40b5296cbca3b5287d6672a89425cb84331f5\": container with ID starting with a5f3cbc93af022831ef9950368b40b5296cbca3b5287d6672a89425cb84331f5 not found: ID does not exist"
Sep 17 17:37:48.079561 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:37:48.079542    2565 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-7b8db48b-rv5v5"]
Sep 17 17:37:48.087160 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:37:48.087138    2565 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-console/console-7b8db48b-rv5v5"]
Sep 17 17:37:48.155091 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:37:48.155069    2565 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="7e66f8c1-3440-4ab4-8cef-1c5f2fe893c4" path="/var/lib/kubelet/pods/7e66f8c1-3440-4ab4-8cef-1c5f2fe893c4/volumes"
Sep 17 17:38:24.197577 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:38:24.197546    2565 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-559db6f977-s2k6s"]
Sep 17 17:38:24.198007 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:38:24.197794    2565 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="7e66f8c1-3440-4ab4-8cef-1c5f2fe893c4" containerName="console"
Sep 17 17:38:24.198007 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:38:24.197804    2565 state_mem.go:107] "Deleted CPUSet assignment" podUID="7e66f8c1-3440-4ab4-8cef-1c5f2fe893c4" containerName="console"
Sep 17 17:38:24.198007 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:38:24.197857    2565 memory_manager.go:368] "RemoveStaleState removing state" podUID="7e66f8c1-3440-4ab4-8cef-1c5f2fe893c4" containerName="console"
Sep 17 17:38:24.201731 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:38:24.201716    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-559db6f977-s2k6s"
Sep 17 17:38:24.222461 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:38:24.222440    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-559db6f977-s2k6s"]
Sep 17 17:38:24.251685 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:38:24.251662    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/b4162b2e-9aef-42bb-8463-b693b7b07ef5-console-oauth-config\") pod \"console-559db6f977-s2k6s\" (UID: \"b4162b2e-9aef-42bb-8463-b693b7b07ef5\") " pod="openshift-console/console-559db6f977-s2k6s"
Sep 17 17:38:24.251792 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:38:24.251687    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/b4162b2e-9aef-42bb-8463-b693b7b07ef5-trusted-ca-bundle\") pod \"console-559db6f977-s2k6s\" (UID: \"b4162b2e-9aef-42bb-8463-b693b7b07ef5\") " pod="openshift-console/console-559db6f977-s2k6s"
Sep 17 17:38:24.251792 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:38:24.251784    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/b4162b2e-9aef-42bb-8463-b693b7b07ef5-service-ca\") pod \"console-559db6f977-s2k6s\" (UID: \"b4162b2e-9aef-42bb-8463-b693b7b07ef5\") " pod="openshift-console/console-559db6f977-s2k6s"
Sep 17 17:38:24.251878 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:38:24.251821    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/b4162b2e-9aef-42bb-8463-b693b7b07ef5-console-config\") pod \"console-559db6f977-s2k6s\" (UID: \"b4162b2e-9aef-42bb-8463-b693b7b07ef5\") " pod="openshift-console/console-559db6f977-s2k6s"
Sep 17 17:38:24.251878 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:38:24.251845    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bpxpc\" (UniqueName: \"kubernetes.io/projected/b4162b2e-9aef-42bb-8463-b693b7b07ef5-kube-api-access-bpxpc\") pod \"console-559db6f977-s2k6s\" (UID: \"b4162b2e-9aef-42bb-8463-b693b7b07ef5\") " pod="openshift-console/console-559db6f977-s2k6s"
Sep 17 17:38:24.251947 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:38:24.251876    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/b4162b2e-9aef-42bb-8463-b693b7b07ef5-console-serving-cert\") pod \"console-559db6f977-s2k6s\" (UID: \"b4162b2e-9aef-42bb-8463-b693b7b07ef5\") " pod="openshift-console/console-559db6f977-s2k6s"
Sep 17 17:38:24.251947 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:38:24.251911    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/b4162b2e-9aef-42bb-8463-b693b7b07ef5-oauth-serving-cert\") pod \"console-559db6f977-s2k6s\" (UID: \"b4162b2e-9aef-42bb-8463-b693b7b07ef5\") " pod="openshift-console/console-559db6f977-s2k6s"
Sep 17 17:38:24.352171 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:38:24.352153    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/b4162b2e-9aef-42bb-8463-b693b7b07ef5-service-ca\") pod \"console-559db6f977-s2k6s\" (UID: \"b4162b2e-9aef-42bb-8463-b693b7b07ef5\") " pod="openshift-console/console-559db6f977-s2k6s"
Sep 17 17:38:24.352296 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:38:24.352182    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/b4162b2e-9aef-42bb-8463-b693b7b07ef5-console-config\") pod \"console-559db6f977-s2k6s\" (UID: \"b4162b2e-9aef-42bb-8463-b693b7b07ef5\") " pod="openshift-console/console-559db6f977-s2k6s"
Sep 17 17:38:24.352296 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:38:24.352196    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bpxpc\" (UniqueName: \"kubernetes.io/projected/b4162b2e-9aef-42bb-8463-b693b7b07ef5-kube-api-access-bpxpc\") pod \"console-559db6f977-s2k6s\" (UID: \"b4162b2e-9aef-42bb-8463-b693b7b07ef5\") " pod="openshift-console/console-559db6f977-s2k6s"
Sep 17 17:38:24.352296 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:38:24.352213    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/b4162b2e-9aef-42bb-8463-b693b7b07ef5-console-serving-cert\") pod \"console-559db6f977-s2k6s\" (UID: \"b4162b2e-9aef-42bb-8463-b693b7b07ef5\") " pod="openshift-console/console-559db6f977-s2k6s"
Sep 17 17:38:24.352296 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:38:24.352227    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/b4162b2e-9aef-42bb-8463-b693b7b07ef5-oauth-serving-cert\") pod \"console-559db6f977-s2k6s\" (UID: \"b4162b2e-9aef-42bb-8463-b693b7b07ef5\") " pod="openshift-console/console-559db6f977-s2k6s"
Sep 17 17:38:24.352296 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:38:24.352253    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/b4162b2e-9aef-42bb-8463-b693b7b07ef5-console-oauth-config\") pod \"console-559db6f977-s2k6s\" (UID: \"b4162b2e-9aef-42bb-8463-b693b7b07ef5\") " pod="openshift-console/console-559db6f977-s2k6s"
Sep 17 17:38:24.352296 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:38:24.352279    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/b4162b2e-9aef-42bb-8463-b693b7b07ef5-trusted-ca-bundle\") pod \"console-559db6f977-s2k6s\" (UID: \"b4162b2e-9aef-42bb-8463-b693b7b07ef5\") " pod="openshift-console/console-559db6f977-s2k6s"
Sep 17 17:38:24.352848 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:38:24.352829    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/b4162b2e-9aef-42bb-8463-b693b7b07ef5-console-config\") pod \"console-559db6f977-s2k6s\" (UID: \"b4162b2e-9aef-42bb-8463-b693b7b07ef5\") " pod="openshift-console/console-559db6f977-s2k6s"
Sep 17 17:38:24.352956 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:38:24.352919    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/b4162b2e-9aef-42bb-8463-b693b7b07ef5-service-ca\") pod \"console-559db6f977-s2k6s\" (UID: \"b4162b2e-9aef-42bb-8463-b693b7b07ef5\") " pod="openshift-console/console-559db6f977-s2k6s"
Sep 17 17:38:24.353013 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:38:24.352990    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/b4162b2e-9aef-42bb-8463-b693b7b07ef5-oauth-serving-cert\") pod \"console-559db6f977-s2k6s\" (UID: \"b4162b2e-9aef-42bb-8463-b693b7b07ef5\") " pod="openshift-console/console-559db6f977-s2k6s"
Sep 17 17:38:24.353208 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:38:24.353194    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/b4162b2e-9aef-42bb-8463-b693b7b07ef5-trusted-ca-bundle\") pod \"console-559db6f977-s2k6s\" (UID: \"b4162b2e-9aef-42bb-8463-b693b7b07ef5\") " pod="openshift-console/console-559db6f977-s2k6s"
Sep 17 17:38:24.354579 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:38:24.354559    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/b4162b2e-9aef-42bb-8463-b693b7b07ef5-console-oauth-config\") pod \"console-559db6f977-s2k6s\" (UID: \"b4162b2e-9aef-42bb-8463-b693b7b07ef5\") " pod="openshift-console/console-559db6f977-s2k6s"
Sep 17 17:38:24.354695 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:38:24.354678    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/b4162b2e-9aef-42bb-8463-b693b7b07ef5-console-serving-cert\") pod \"console-559db6f977-s2k6s\" (UID: \"b4162b2e-9aef-42bb-8463-b693b7b07ef5\") " pod="openshift-console/console-559db6f977-s2k6s"
Sep 17 17:38:24.361862 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:38:24.361838    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-bpxpc\" (UniqueName: \"kubernetes.io/projected/b4162b2e-9aef-42bb-8463-b693b7b07ef5-kube-api-access-bpxpc\") pod \"console-559db6f977-s2k6s\" (UID: \"b4162b2e-9aef-42bb-8463-b693b7b07ef5\") " pod="openshift-console/console-559db6f977-s2k6s"
Sep 17 17:38:24.510224 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:38:24.510171    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-559db6f977-s2k6s"
Sep 17 17:38:24.780258 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:38:24.780233    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-559db6f977-s2k6s"]
Sep 17 17:38:24.782056 ip-10-0-137-20 kubenswrapper[2565]: W0917 17:38:24.782009    2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb4162b2e_9aef_42bb_8463_b693b7b07ef5.slice/crio-b5da8266f30cbda332e63bb8983e450fe122727818116c2e7ad43f0d0d9f22fd WatchSource:0}: Error finding container b5da8266f30cbda332e63bb8983e450fe122727818116c2e7ad43f0d0d9f22fd: Status 404 returned error can't find the container with id b5da8266f30cbda332e63bb8983e450fe122727818116c2e7ad43f0d0d9f22fd
Sep 17 17:38:25.180197 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:38:25.180164    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-559db6f977-s2k6s" event={"ID":"b4162b2e-9aef-42bb-8463-b693b7b07ef5","Type":"ContainerStarted","Data":"cca984d354944884efece9efed4d979c56b89e824202c25a3911c9ca5d8bcc32"}
Sep 17 17:38:25.180372 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:38:25.180205    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-559db6f977-s2k6s" event={"ID":"b4162b2e-9aef-42bb-8463-b693b7b07ef5","Type":"ContainerStarted","Data":"b5da8266f30cbda332e63bb8983e450fe122727818116c2e7ad43f0d0d9f22fd"}
Sep 17 17:38:25.205168 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:38:25.205128    2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-559db6f977-s2k6s" podStartSLOduration=1.20511392 podStartE2EDuration="1.20511392s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-17 17:38:24 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-17 17:38:25.203762783 +0000 UTC m=+255.081047222" watchObservedRunningTime="2026-09-17 17:38:25.20511392 +0000 UTC m=+255.082398701"
Sep 17 17:38:34.511290 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:38:34.511257    2565 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-559db6f977-s2k6s"
Sep 17 17:38:34.511681 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:38:34.511313    2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-559db6f977-s2k6s"
Sep 17 17:38:34.515694 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:38:34.515669    2565 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-559db6f977-s2k6s"
Sep 17 17:38:35.218432 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:38:35.218402    2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-559db6f977-s2k6s"
Sep 17 17:38:35.264320 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:38:35.264282    2565 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-654c5b799-l26tj"]
Sep 17 17:39:00.283197 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:39:00.283141    2565 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-console/console-654c5b799-l26tj" podUID="20a613bd-5bb4-4a02-bf07-c1d15356894c" containerName="console" containerID="cri-o://6bdb5859e85a51364ef6d11f12ba7d4519c95feef0dfbab000eaa10c0e40d466" gracePeriod=15
Sep 17 17:39:00.517172 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:39:00.517150    2565 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-654c5b799-l26tj"
Sep 17 17:39:00.603715 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:39:00.603687    2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/20a613bd-5bb4-4a02-bf07-c1d15356894c-service-ca\") pod \"20a613bd-5bb4-4a02-bf07-c1d15356894c\" (UID: \"20a613bd-5bb4-4a02-bf07-c1d15356894c\") "
Sep 17 17:39:00.603839 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:39:00.603746    2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/20a613bd-5bb4-4a02-bf07-c1d15356894c-console-config\") pod \"20a613bd-5bb4-4a02-bf07-c1d15356894c\" (UID: \"20a613bd-5bb4-4a02-bf07-c1d15356894c\") "
Sep 17 17:39:00.603839 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:39:00.603773    2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/20a613bd-5bb4-4a02-bf07-c1d15356894c-trusted-ca-bundle\") pod \"20a613bd-5bb4-4a02-bf07-c1d15356894c\" (UID: \"20a613bd-5bb4-4a02-bf07-c1d15356894c\") "
Sep 17 17:39:00.603839 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:39:00.603793    2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/20a613bd-5bb4-4a02-bf07-c1d15356894c-oauth-serving-cert\") pod \"20a613bd-5bb4-4a02-bf07-c1d15356894c\" (UID: \"20a613bd-5bb4-4a02-bf07-c1d15356894c\") "
Sep 17 17:39:00.603839 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:39:00.603815    2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/20a613bd-5bb4-4a02-bf07-c1d15356894c-console-serving-cert\") pod \"20a613bd-5bb4-4a02-bf07-c1d15356894c\" (UID: \"20a613bd-5bb4-4a02-bf07-c1d15356894c\") "
Sep 17 17:39:00.604064 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:39:00.603852    2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-8wrcx\" (UniqueName: \"kubernetes.io/projected/20a613bd-5bb4-4a02-bf07-c1d15356894c-kube-api-access-8wrcx\") pod \"20a613bd-5bb4-4a02-bf07-c1d15356894c\" (UID: \"20a613bd-5bb4-4a02-bf07-c1d15356894c\") "
Sep 17 17:39:00.604064 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:39:00.603930    2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/20a613bd-5bb4-4a02-bf07-c1d15356894c-console-oauth-config\") pod \"20a613bd-5bb4-4a02-bf07-c1d15356894c\" (UID: \"20a613bd-5bb4-4a02-bf07-c1d15356894c\") "
Sep 17 17:39:00.604215 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:39:00.604090    2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/20a613bd-5bb4-4a02-bf07-c1d15356894c-service-ca" (OuterVolumeSpecName: "service-ca") pod "20a613bd-5bb4-4a02-bf07-c1d15356894c" (UID: "20a613bd-5bb4-4a02-bf07-c1d15356894c"). InnerVolumeSpecName "service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 17 17:39:00.604273 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:39:00.604236    2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/20a613bd-5bb4-4a02-bf07-c1d15356894c-oauth-serving-cert" (OuterVolumeSpecName: "oauth-serving-cert") pod "20a613bd-5bb4-4a02-bf07-c1d15356894c" (UID: "20a613bd-5bb4-4a02-bf07-c1d15356894c"). InnerVolumeSpecName "oauth-serving-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 17 17:39:00.604273 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:39:00.604246    2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/20a613bd-5bb4-4a02-bf07-c1d15356894c-trusted-ca-bundle" (OuterVolumeSpecName: "trusted-ca-bundle") pod "20a613bd-5bb4-4a02-bf07-c1d15356894c" (UID: "20a613bd-5bb4-4a02-bf07-c1d15356894c"). InnerVolumeSpecName "trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 17 17:39:00.604384 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:39:00.604264    2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/20a613bd-5bb4-4a02-bf07-c1d15356894c-console-config" (OuterVolumeSpecName: "console-config") pod "20a613bd-5bb4-4a02-bf07-c1d15356894c" (UID: "20a613bd-5bb4-4a02-bf07-c1d15356894c"). InnerVolumeSpecName "console-config". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 17 17:39:00.605904 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:39:00.605880    2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/20a613bd-5bb4-4a02-bf07-c1d15356894c-kube-api-access-8wrcx" (OuterVolumeSpecName: "kube-api-access-8wrcx") pod "20a613bd-5bb4-4a02-bf07-c1d15356894c" (UID: "20a613bd-5bb4-4a02-bf07-c1d15356894c"). InnerVolumeSpecName "kube-api-access-8wrcx". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 17 17:39:00.605995 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:39:00.605904    2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/20a613bd-5bb4-4a02-bf07-c1d15356894c-console-serving-cert" (OuterVolumeSpecName: "console-serving-cert") pod "20a613bd-5bb4-4a02-bf07-c1d15356894c" (UID: "20a613bd-5bb4-4a02-bf07-c1d15356894c"). InnerVolumeSpecName "console-serving-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 17 17:39:00.606092 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:39:00.605991    2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/20a613bd-5bb4-4a02-bf07-c1d15356894c-console-oauth-config" (OuterVolumeSpecName: "console-oauth-config") pod "20a613bd-5bb4-4a02-bf07-c1d15356894c" (UID: "20a613bd-5bb4-4a02-bf07-c1d15356894c"). InnerVolumeSpecName "console-oauth-config". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 17 17:39:00.705506 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:39:00.705471    2565 reconciler_common.go:299] "Volume detached for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/20a613bd-5bb4-4a02-bf07-c1d15356894c-console-config\") on node \"ip-10-0-137-20.ec2.internal\" DevicePath \"\""
Sep 17 17:39:00.705506 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:39:00.705503    2565 reconciler_common.go:299] "Volume detached for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/20a613bd-5bb4-4a02-bf07-c1d15356894c-trusted-ca-bundle\") on node \"ip-10-0-137-20.ec2.internal\" DevicePath \"\""
Sep 17 17:39:00.705506 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:39:00.705514    2565 reconciler_common.go:299] "Volume detached for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/20a613bd-5bb4-4a02-bf07-c1d15356894c-oauth-serving-cert\") on node \"ip-10-0-137-20.ec2.internal\" DevicePath \"\""
Sep 17 17:39:00.705693 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:39:00.705523    2565 reconciler_common.go:299] "Volume detached for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/20a613bd-5bb4-4a02-bf07-c1d15356894c-console-serving-cert\") on node \"ip-10-0-137-20.ec2.internal\" DevicePath \"\""
Sep 17 17:39:00.705693 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:39:00.705531    2565 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-8wrcx\" (UniqueName: \"kubernetes.io/projected/20a613bd-5bb4-4a02-bf07-c1d15356894c-kube-api-access-8wrcx\") on node \"ip-10-0-137-20.ec2.internal\" DevicePath \"\""
Sep 17 17:39:00.705693 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:39:00.705539    2565 reconciler_common.go:299] "Volume detached for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/20a613bd-5bb4-4a02-bf07-c1d15356894c-console-oauth-config\") on node \"ip-10-0-137-20.ec2.internal\" DevicePath \"\""
Sep 17 17:39:00.705693 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:39:00.705547    2565 reconciler_common.go:299] "Volume detached for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/20a613bd-5bb4-4a02-bf07-c1d15356894c-service-ca\") on node \"ip-10-0-137-20.ec2.internal\" DevicePath \"\""
Sep 17 17:39:01.304886 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:39:01.304853    2565 generic.go:356] "Generic (PLEG): container finished" podID="20a613bd-5bb4-4a02-bf07-c1d15356894c" containerID="6bdb5859e85a51364ef6d11f12ba7d4519c95feef0dfbab000eaa10c0e40d466" exitCode=0
Sep 17 17:39:01.305280 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:39:01.304896    2565 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-654c5b799-l26tj"
Sep 17 17:39:01.305280 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:39:01.304908    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-654c5b799-l26tj" event={"ID":"20a613bd-5bb4-4a02-bf07-c1d15356894c","Type":"ContainerDied","Data":"6bdb5859e85a51364ef6d11f12ba7d4519c95feef0dfbab000eaa10c0e40d466"}
Sep 17 17:39:01.305280 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:39:01.304946    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-654c5b799-l26tj" event={"ID":"20a613bd-5bb4-4a02-bf07-c1d15356894c","Type":"ContainerDied","Data":"5bf28a678d74ce62e52d0f0fc5d1d75d928afe3066b4cca27eac0fd1392c53b2"}
Sep 17 17:39:01.305280 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:39:01.304969    2565 scope.go:117] "RemoveContainer" containerID="6bdb5859e85a51364ef6d11f12ba7d4519c95feef0dfbab000eaa10c0e40d466"
Sep 17 17:39:01.313426 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:39:01.313408    2565 scope.go:117] "RemoveContainer" containerID="6bdb5859e85a51364ef6d11f12ba7d4519c95feef0dfbab000eaa10c0e40d466"
Sep 17 17:39:01.313687 ip-10-0-137-20 kubenswrapper[2565]: E0917 17:39:01.313669    2565 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"6bdb5859e85a51364ef6d11f12ba7d4519c95feef0dfbab000eaa10c0e40d466\": container with ID starting with 6bdb5859e85a51364ef6d11f12ba7d4519c95feef0dfbab000eaa10c0e40d466 not found: ID does not exist" containerID="6bdb5859e85a51364ef6d11f12ba7d4519c95feef0dfbab000eaa10c0e40d466"
Sep 17 17:39:01.313750 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:39:01.313699    2565 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"6bdb5859e85a51364ef6d11f12ba7d4519c95feef0dfbab000eaa10c0e40d466"} err="failed to get container status \"6bdb5859e85a51364ef6d11f12ba7d4519c95feef0dfbab000eaa10c0e40d466\": rpc error: code = NotFound desc = could not find container \"6bdb5859e85a51364ef6d11f12ba7d4519c95feef0dfbab000eaa10c0e40d466\": container with ID starting with 6bdb5859e85a51364ef6d11f12ba7d4519c95feef0dfbab000eaa10c0e40d466 not found: ID does not exist"
Sep 17 17:39:01.327217 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:39:01.327190    2565 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-654c5b799-l26tj"]
Sep 17 17:39:01.331363 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:39:01.331344    2565 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-console/console-654c5b799-l26tj"]
Sep 17 17:39:02.155730 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:39:02.155700    2565 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="20a613bd-5bb4-4a02-bf07-c1d15356894c" path="/var/lib/kubelet/pods/20a613bd-5bb4-4a02-bf07-c1d15356894c/volumes"
Sep 17 17:39:12.063792 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:39:12.063761    2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-d9927_30837de7-1447-4de2-ac62-c809813b3338/ovn-acl-logging/0.log"
Sep 17 17:39:12.071622 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:39:12.071597    2565 kubelet.go:1648] "Image garbage collection succeeded"
Sep 17 17:39:12.073891 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:39:12.073873    2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-d9927_30837de7-1447-4de2-ac62-c809813b3338/ovn-acl-logging/0.log"
Sep 17 17:39:25.642690 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:39:25.642653    2565 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-marketplace/1acd376c2f780381bc434f5f1985f786a96c8309f7a005b52ed03f0c1amvf9v"]
Sep 17 17:39:25.645054 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:39:25.642923    2565 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="20a613bd-5bb4-4a02-bf07-c1d15356894c" containerName="console"
Sep 17 17:39:25.645054 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:39:25.642933    2565 state_mem.go:107] "Deleted CPUSet assignment" podUID="20a613bd-5bb4-4a02-bf07-c1d15356894c" containerName="console"
Sep 17 17:39:25.645054 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:39:25.642973    2565 memory_manager.go:368] "RemoveStaleState removing state" podUID="20a613bd-5bb4-4a02-bf07-c1d15356894c" containerName="console"
Sep 17 17:39:25.645941 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:39:25.645926    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/1acd376c2f780381bc434f5f1985f786a96c8309f7a005b52ed03f0c1amvf9v"
Sep 17 17:39:25.650315 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:39:25.650292    2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"kube-root-ca.crt\""
Sep 17 17:39:25.650410 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:39:25.650316    2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-marketplace\"/\"default-dockercfg-dsz9c\""
Sep 17 17:39:25.651632 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:39:25.651611    2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"openshift-service-ca.crt\""
Sep 17 17:39:25.657050 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:39:25.657008    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/1acd376c2f780381bc434f5f1985f786a96c8309f7a005b52ed03f0c1amvf9v"]
Sep 17 17:39:25.798812 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:39:25.798777    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/45cab434-48ff-480d-a6cd-00f17341c55a-util\") pod \"1acd376c2f780381bc434f5f1985f786a96c8309f7a005b52ed03f0c1amvf9v\" (UID: \"45cab434-48ff-480d-a6cd-00f17341c55a\") " pod="openshift-marketplace/1acd376c2f780381bc434f5f1985f786a96c8309f7a005b52ed03f0c1amvf9v"
Sep 17 17:39:25.798812 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:39:25.798821    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6pvtg\" (UniqueName: \"kubernetes.io/projected/45cab434-48ff-480d-a6cd-00f17341c55a-kube-api-access-6pvtg\") pod \"1acd376c2f780381bc434f5f1985f786a96c8309f7a005b52ed03f0c1amvf9v\" (UID: \"45cab434-48ff-480d-a6cd-00f17341c55a\") " pod="openshift-marketplace/1acd376c2f780381bc434f5f1985f786a96c8309f7a005b52ed03f0c1amvf9v"
Sep 17 17:39:25.799102 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:39:25.798921    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/45cab434-48ff-480d-a6cd-00f17341c55a-bundle\") pod \"1acd376c2f780381bc434f5f1985f786a96c8309f7a005b52ed03f0c1amvf9v\" (UID: \"45cab434-48ff-480d-a6cd-00f17341c55a\") " pod="openshift-marketplace/1acd376c2f780381bc434f5f1985f786a96c8309f7a005b52ed03f0c1amvf9v"
Sep 17 17:39:25.899702 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:39:25.899609    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/45cab434-48ff-480d-a6cd-00f17341c55a-util\") pod \"1acd376c2f780381bc434f5f1985f786a96c8309f7a005b52ed03f0c1amvf9v\" (UID: \"45cab434-48ff-480d-a6cd-00f17341c55a\") " pod="openshift-marketplace/1acd376c2f780381bc434f5f1985f786a96c8309f7a005b52ed03f0c1amvf9v"
Sep 17 17:39:25.899702 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:39:25.899658    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6pvtg\" (UniqueName: \"kubernetes.io/projected/45cab434-48ff-480d-a6cd-00f17341c55a-kube-api-access-6pvtg\") pod \"1acd376c2f780381bc434f5f1985f786a96c8309f7a005b52ed03f0c1amvf9v\" (UID: \"45cab434-48ff-480d-a6cd-00f17341c55a\") " pod="openshift-marketplace/1acd376c2f780381bc434f5f1985f786a96c8309f7a005b52ed03f0c1amvf9v"
Sep 17 17:39:25.899702 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:39:25.899698    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/45cab434-48ff-480d-a6cd-00f17341c55a-bundle\") pod \"1acd376c2f780381bc434f5f1985f786a96c8309f7a005b52ed03f0c1amvf9v\" (UID: \"45cab434-48ff-480d-a6cd-00f17341c55a\") " pod="openshift-marketplace/1acd376c2f780381bc434f5f1985f786a96c8309f7a005b52ed03f0c1amvf9v"
Sep 17 17:39:25.900001 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:39:25.899984    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/45cab434-48ff-480d-a6cd-00f17341c55a-util\") pod \"1acd376c2f780381bc434f5f1985f786a96c8309f7a005b52ed03f0c1amvf9v\" (UID: \"45cab434-48ff-480d-a6cd-00f17341c55a\") " pod="openshift-marketplace/1acd376c2f780381bc434f5f1985f786a96c8309f7a005b52ed03f0c1amvf9v"
Sep 17 17:39:25.900073 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:39:25.900014    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/45cab434-48ff-480d-a6cd-00f17341c55a-bundle\") pod \"1acd376c2f780381bc434f5f1985f786a96c8309f7a005b52ed03f0c1amvf9v\" (UID: \"45cab434-48ff-480d-a6cd-00f17341c55a\") " pod="openshift-marketplace/1acd376c2f780381bc434f5f1985f786a96c8309f7a005b52ed03f0c1amvf9v"
Sep 17 17:39:25.909375 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:39:25.909355    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-6pvtg\" (UniqueName: \"kubernetes.io/projected/45cab434-48ff-480d-a6cd-00f17341c55a-kube-api-access-6pvtg\") pod \"1acd376c2f780381bc434f5f1985f786a96c8309f7a005b52ed03f0c1amvf9v\" (UID: \"45cab434-48ff-480d-a6cd-00f17341c55a\") " pod="openshift-marketplace/1acd376c2f780381bc434f5f1985f786a96c8309f7a005b52ed03f0c1amvf9v"
Sep 17 17:39:25.955398 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:39:25.955372    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/1acd376c2f780381bc434f5f1985f786a96c8309f7a005b52ed03f0c1amvf9v"
Sep 17 17:39:26.227481 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:39:26.227457    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/1acd376c2f780381bc434f5f1985f786a96c8309f7a005b52ed03f0c1amvf9v"]
Sep 17 17:39:26.229794 ip-10-0-137-20 kubenswrapper[2565]: W0917 17:39:26.229728    2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod45cab434_48ff_480d_a6cd_00f17341c55a.slice/crio-0f0b361ef765192e0f5a7d90615971c9a904461bb42070ee343cf0d19ba1abd5 WatchSource:0}: Error finding container 0f0b361ef765192e0f5a7d90615971c9a904461bb42070ee343cf0d19ba1abd5: Status 404 returned error can't find the container with id 0f0b361ef765192e0f5a7d90615971c9a904461bb42070ee343cf0d19ba1abd5
Sep 17 17:39:26.231816 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:39:26.231798    2565 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Sep 17 17:39:26.394671 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:39:26.394638    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/1acd376c2f780381bc434f5f1985f786a96c8309f7a005b52ed03f0c1amvf9v" event={"ID":"45cab434-48ff-480d-a6cd-00f17341c55a","Type":"ContainerStarted","Data":"0f0b361ef765192e0f5a7d90615971c9a904461bb42070ee343cf0d19ba1abd5"}
Sep 17 17:39:31.416224 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:39:31.416195    2565 generic.go:356] "Generic (PLEG): container finished" podID="45cab434-48ff-480d-a6cd-00f17341c55a" containerID="9e4b0db312d9198f06e76ad3176de733fd4d137d4198ad38c98658383bf06256" exitCode=0
Sep 17 17:39:31.416572 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:39:31.416254    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/1acd376c2f780381bc434f5f1985f786a96c8309f7a005b52ed03f0c1amvf9v" event={"ID":"45cab434-48ff-480d-a6cd-00f17341c55a","Type":"ContainerDied","Data":"9e4b0db312d9198f06e76ad3176de733fd4d137d4198ad38c98658383bf06256"}
Sep 17 17:39:34.431278 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:39:34.431244    2565 generic.go:356] "Generic (PLEG): container finished" podID="45cab434-48ff-480d-a6cd-00f17341c55a" containerID="d7ccf9b03c0978de8187355d6eeaffb49e7d4eeefb12a4819a3f40c825e2ce70" exitCode=0
Sep 17 17:39:34.431717 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:39:34.431310    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/1acd376c2f780381bc434f5f1985f786a96c8309f7a005b52ed03f0c1amvf9v" event={"ID":"45cab434-48ff-480d-a6cd-00f17341c55a","Type":"ContainerDied","Data":"d7ccf9b03c0978de8187355d6eeaffb49e7d4eeefb12a4819a3f40c825e2ce70"}
Sep 17 17:39:40.457794 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:39:40.457715    2565 generic.go:356] "Generic (PLEG): container finished" podID="45cab434-48ff-480d-a6cd-00f17341c55a" containerID="5a1f8997e4a879ff4ccfbd022c8bbf703d18ed6d769224b294f2dae970d9797b" exitCode=0
Sep 17 17:39:40.458161 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:39:40.457797    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/1acd376c2f780381bc434f5f1985f786a96c8309f7a005b52ed03f0c1amvf9v" event={"ID":"45cab434-48ff-480d-a6cd-00f17341c55a","Type":"ContainerDied","Data":"5a1f8997e4a879ff4ccfbd022c8bbf703d18ed6d769224b294f2dae970d9797b"}
Sep 17 17:39:41.589976 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:39:41.589948    2565 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/1acd376c2f780381bc434f5f1985f786a96c8309f7a005b52ed03f0c1amvf9v"
Sep 17 17:39:41.623689 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:39:41.623661    2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/45cab434-48ff-480d-a6cd-00f17341c55a-util\") pod \"45cab434-48ff-480d-a6cd-00f17341c55a\" (UID: \"45cab434-48ff-480d-a6cd-00f17341c55a\") "
Sep 17 17:39:41.623823 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:39:41.623743    2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/45cab434-48ff-480d-a6cd-00f17341c55a-bundle\") pod \"45cab434-48ff-480d-a6cd-00f17341c55a\" (UID: \"45cab434-48ff-480d-a6cd-00f17341c55a\") "
Sep 17 17:39:41.623823 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:39:41.623787    2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-6pvtg\" (UniqueName: \"kubernetes.io/projected/45cab434-48ff-480d-a6cd-00f17341c55a-kube-api-access-6pvtg\") pod \"45cab434-48ff-480d-a6cd-00f17341c55a\" (UID: \"45cab434-48ff-480d-a6cd-00f17341c55a\") "
Sep 17 17:39:41.624323 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:39:41.624300    2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/45cab434-48ff-480d-a6cd-00f17341c55a-bundle" (OuterVolumeSpecName: "bundle") pod "45cab434-48ff-480d-a6cd-00f17341c55a" (UID: "45cab434-48ff-480d-a6cd-00f17341c55a"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 17 17:39:41.625850 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:39:41.625827    2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/45cab434-48ff-480d-a6cd-00f17341c55a-kube-api-access-6pvtg" (OuterVolumeSpecName: "kube-api-access-6pvtg") pod "45cab434-48ff-480d-a6cd-00f17341c55a" (UID: "45cab434-48ff-480d-a6cd-00f17341c55a"). InnerVolumeSpecName "kube-api-access-6pvtg". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 17 17:39:41.627886 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:39:41.627865    2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/45cab434-48ff-480d-a6cd-00f17341c55a-util" (OuterVolumeSpecName: "util") pod "45cab434-48ff-480d-a6cd-00f17341c55a" (UID: "45cab434-48ff-480d-a6cd-00f17341c55a"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 17 17:39:41.724307 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:39:41.724251    2565 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/45cab434-48ff-480d-a6cd-00f17341c55a-bundle\") on node \"ip-10-0-137-20.ec2.internal\" DevicePath \"\""
Sep 17 17:39:41.724307 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:39:41.724271    2565 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-6pvtg\" (UniqueName: \"kubernetes.io/projected/45cab434-48ff-480d-a6cd-00f17341c55a-kube-api-access-6pvtg\") on node \"ip-10-0-137-20.ec2.internal\" DevicePath \"\""
Sep 17 17:39:41.724307 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:39:41.724281    2565 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/45cab434-48ff-480d-a6cd-00f17341c55a-util\") on node \"ip-10-0-137-20.ec2.internal\" DevicePath \"\""
Sep 17 17:39:42.467232 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:39:42.467155    2565 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/1acd376c2f780381bc434f5f1985f786a96c8309f7a005b52ed03f0c1amvf9v"
Sep 17 17:39:42.467232 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:39:42.467161    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/1acd376c2f780381bc434f5f1985f786a96c8309f7a005b52ed03f0c1amvf9v" event={"ID":"45cab434-48ff-480d-a6cd-00f17341c55a","Type":"ContainerDied","Data":"0f0b361ef765192e0f5a7d90615971c9a904461bb42070ee343cf0d19ba1abd5"}
Sep 17 17:39:42.467232 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:39:42.467210    2565 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="0f0b361ef765192e0f5a7d90615971c9a904461bb42070ee343cf0d19ba1abd5"
Sep 17 17:39:48.575481 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:39:48.575449    2565 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-keda/custom-metrics-autoscaler-operator-566c9cd69c-x2bfq"]
Sep 17 17:39:48.575865 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:39:48.575727    2565 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="45cab434-48ff-480d-a6cd-00f17341c55a" containerName="pull"
Sep 17 17:39:48.575865 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:39:48.575737    2565 state_mem.go:107] "Deleted CPUSet assignment" podUID="45cab434-48ff-480d-a6cd-00f17341c55a" containerName="pull"
Sep 17 17:39:48.575865 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:39:48.575747    2565 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="45cab434-48ff-480d-a6cd-00f17341c55a" containerName="extract"
Sep 17 17:39:48.575865 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:39:48.575752    2565 state_mem.go:107] "Deleted CPUSet assignment" podUID="45cab434-48ff-480d-a6cd-00f17341c55a" containerName="extract"
Sep 17 17:39:48.575865 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:39:48.575762    2565 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="45cab434-48ff-480d-a6cd-00f17341c55a" containerName="util"
Sep 17 17:39:48.575865 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:39:48.575767    2565 state_mem.go:107] "Deleted CPUSet assignment" podUID="45cab434-48ff-480d-a6cd-00f17341c55a" containerName="util"
Sep 17 17:39:48.575865 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:39:48.575811    2565 memory_manager.go:368] "RemoveStaleState removing state" podUID="45cab434-48ff-480d-a6cd-00f17341c55a" containerName="extract"
Sep 17 17:39:48.617352 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:39:48.617325    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-keda/custom-metrics-autoscaler-operator-566c9cd69c-x2bfq"]
Sep 17 17:39:48.617498 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:39:48.617412    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-keda/custom-metrics-autoscaler-operator-566c9cd69c-x2bfq"
Sep 17 17:39:48.620462 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:39:48.620442    2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-keda\"/\"kedaorg-certs\""
Sep 17 17:39:48.620984 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:39:48.620963    2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-keda\"/\"custom-metrics-autoscaler-operator-dockercfg-r276z\""
Sep 17 17:39:48.621100 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:39:48.620991    2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-keda\"/\"kube-root-ca.crt\""
Sep 17 17:39:48.621210 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:39:48.621192    2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-keda\"/\"openshift-service-ca.crt\""
Sep 17 17:39:48.674166 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:39:48.674145    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"certificates\" (UniqueName: \"kubernetes.io/secret/216fddd4-38e5-41c4-897d-738889b4143d-certificates\") pod \"custom-metrics-autoscaler-operator-566c9cd69c-x2bfq\" (UID: \"216fddd4-38e5-41c4-897d-738889b4143d\") " pod="openshift-keda/custom-metrics-autoscaler-operator-566c9cd69c-x2bfq"
Sep 17 17:39:48.674293 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:39:48.674196    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6xqc7\" (UniqueName: \"kubernetes.io/projected/216fddd4-38e5-41c4-897d-738889b4143d-kube-api-access-6xqc7\") pod \"custom-metrics-autoscaler-operator-566c9cd69c-x2bfq\" (UID: \"216fddd4-38e5-41c4-897d-738889b4143d\") " pod="openshift-keda/custom-metrics-autoscaler-operator-566c9cd69c-x2bfq"
Sep 17 17:39:48.775543 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:39:48.775500    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6xqc7\" (UniqueName: \"kubernetes.io/projected/216fddd4-38e5-41c4-897d-738889b4143d-kube-api-access-6xqc7\") pod \"custom-metrics-autoscaler-operator-566c9cd69c-x2bfq\" (UID: \"216fddd4-38e5-41c4-897d-738889b4143d\") " pod="openshift-keda/custom-metrics-autoscaler-operator-566c9cd69c-x2bfq"
Sep 17 17:39:48.775700 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:39:48.775563    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"certificates\" (UniqueName: \"kubernetes.io/secret/216fddd4-38e5-41c4-897d-738889b4143d-certificates\") pod \"custom-metrics-autoscaler-operator-566c9cd69c-x2bfq\" (UID: \"216fddd4-38e5-41c4-897d-738889b4143d\") " pod="openshift-keda/custom-metrics-autoscaler-operator-566c9cd69c-x2bfq"
Sep 17 17:39:48.777778 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:39:48.777761    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"certificates\" (UniqueName: \"kubernetes.io/secret/216fddd4-38e5-41c4-897d-738889b4143d-certificates\") pod \"custom-metrics-autoscaler-operator-566c9cd69c-x2bfq\" (UID: \"216fddd4-38e5-41c4-897d-738889b4143d\") " pod="openshift-keda/custom-metrics-autoscaler-operator-566c9cd69c-x2bfq"
Sep 17 17:39:48.785790 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:39:48.785754    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-6xqc7\" (UniqueName: \"kubernetes.io/projected/216fddd4-38e5-41c4-897d-738889b4143d-kube-api-access-6xqc7\") pod \"custom-metrics-autoscaler-operator-566c9cd69c-x2bfq\" (UID: \"216fddd4-38e5-41c4-897d-738889b4143d\") " pod="openshift-keda/custom-metrics-autoscaler-operator-566c9cd69c-x2bfq"
Sep 17 17:39:48.927183 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:39:48.927157    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-keda/custom-metrics-autoscaler-operator-566c9cd69c-x2bfq"
Sep 17 17:39:49.232743 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:39:49.232655    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-keda/custom-metrics-autoscaler-operator-566c9cd69c-x2bfq"]
Sep 17 17:39:49.236674 ip-10-0-137-20 kubenswrapper[2565]: W0917 17:39:49.236642    2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod216fddd4_38e5_41c4_897d_738889b4143d.slice/crio-cdf6e3f1ed408d6a0c2edb1eb8802895bc25f5da3d971054b4d35c9e0c229c8a WatchSource:0}: Error finding container cdf6e3f1ed408d6a0c2edb1eb8802895bc25f5da3d971054b4d35c9e0c229c8a: Status 404 returned error can't find the container with id cdf6e3f1ed408d6a0c2edb1eb8802895bc25f5da3d971054b4d35c9e0c229c8a
Sep 17 17:39:49.494059 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:39:49.493946    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-keda/custom-metrics-autoscaler-operator-566c9cd69c-x2bfq" event={"ID":"216fddd4-38e5-41c4-897d-738889b4143d","Type":"ContainerStarted","Data":"cdf6e3f1ed408d6a0c2edb1eb8802895bc25f5da3d971054b4d35c9e0c229c8a"}
Sep 17 17:39:53.429214 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:39:53.429177    2565 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-keda/keda-operator-579c8958f7-2st78"]
Sep 17 17:39:53.452041 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:39:53.451997    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-keda/keda-operator-579c8958f7-2st78"]
Sep 17 17:39:53.452202 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:39:53.452139    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-keda/keda-operator-579c8958f7-2st78"
Sep 17 17:39:53.455090 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:39:53.455062    2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-keda\"/\"keda-operator-certs\""
Sep 17 17:39:53.455223 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:39:53.455102    2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-keda\"/\"keda-operator-dockercfg-xxw2d\""
Sep 17 17:39:53.455223 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:39:53.455145    2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-keda\"/\"keda-ocp-cabundle\""
Sep 17 17:39:53.511580 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:39:53.511543    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-keda/custom-metrics-autoscaler-operator-566c9cd69c-x2bfq" event={"ID":"216fddd4-38e5-41c4-897d-738889b4143d","Type":"ContainerStarted","Data":"4e5edae86238c5fc9286c555109a9a31a6ba2d861a9b0d4404a8a9a4ceb84233"}
Sep 17 17:39:53.511766 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:39:53.511713    2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-keda/custom-metrics-autoscaler-operator-566c9cd69c-x2bfq"
Sep 17 17:39:53.515978 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:39:53.515954    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cabundle0\" (UniqueName: \"kubernetes.io/configmap/3dadf1fc-acf6-4589-8ddd-65961b6ef4bb-cabundle0\") pod \"keda-operator-579c8958f7-2st78\" (UID: \"3dadf1fc-acf6-4589-8ddd-65961b6ef4bb\") " pod="openshift-keda/keda-operator-579c8958f7-2st78"
Sep 17 17:39:53.516105 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:39:53.516014    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6sz75\" (UniqueName: \"kubernetes.io/projected/3dadf1fc-acf6-4589-8ddd-65961b6ef4bb-kube-api-access-6sz75\") pod \"keda-operator-579c8958f7-2st78\" (UID: \"3dadf1fc-acf6-4589-8ddd-65961b6ef4bb\") " pod="openshift-keda/keda-operator-579c8958f7-2st78"
Sep 17 17:39:53.516166 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:39:53.516117    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"certificates\" (UniqueName: \"kubernetes.io/projected/3dadf1fc-acf6-4589-8ddd-65961b6ef4bb-certificates\") pod \"keda-operator-579c8958f7-2st78\" (UID: \"3dadf1fc-acf6-4589-8ddd-65961b6ef4bb\") " pod="openshift-keda/keda-operator-579c8958f7-2st78"
Sep 17 17:39:53.538042 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:39:53.537978    2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-keda/custom-metrics-autoscaler-operator-566c9cd69c-x2bfq" podStartSLOduration=2.128206384 podStartE2EDuration="5.537963136s" totalImagesPullingTime="3.409756752s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-17 17:39:48 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-17 17:39:53.537245313 +0000 UTC m=+343.414529728" watchObservedRunningTime="2026-09-17 17:39:53.537963136 +0000 UTC m=+343.415247550"
Sep 17 17:39:53.617586 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:39:53.617538    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cabundle0\" (UniqueName: \"kubernetes.io/configmap/3dadf1fc-acf6-4589-8ddd-65961b6ef4bb-cabundle0\") pod \"keda-operator-579c8958f7-2st78\" (UID: \"3dadf1fc-acf6-4589-8ddd-65961b6ef4bb\") " pod="openshift-keda/keda-operator-579c8958f7-2st78"
Sep 17 17:39:53.617806 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:39:53.617654    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6sz75\" (UniqueName: \"kubernetes.io/projected/3dadf1fc-acf6-4589-8ddd-65961b6ef4bb-kube-api-access-6sz75\") pod \"keda-operator-579c8958f7-2st78\" (UID: \"3dadf1fc-acf6-4589-8ddd-65961b6ef4bb\") " pod="openshift-keda/keda-operator-579c8958f7-2st78"
Sep 17 17:39:53.617806 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:39:53.617704    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"certificates\" (UniqueName: \"kubernetes.io/projected/3dadf1fc-acf6-4589-8ddd-65961b6ef4bb-certificates\") pod \"keda-operator-579c8958f7-2st78\" (UID: \"3dadf1fc-acf6-4589-8ddd-65961b6ef4bb\") " pod="openshift-keda/keda-operator-579c8958f7-2st78"
Sep 17 17:39:53.617929 ip-10-0-137-20 kubenswrapper[2565]: E0917 17:39:53.617834    2565 projected.go:266] Couldn't get secret openshift-keda/keda-operator-certs: secret "keda-operator-certs" not found
Sep 17 17:39:53.617929 ip-10-0-137-20 kubenswrapper[2565]: E0917 17:39:53.617851    2565 secret.go:281] references non-existent secret key: ca.crt
Sep 17 17:39:53.617929 ip-10-0-137-20 kubenswrapper[2565]: E0917 17:39:53.617863    2565 projected.go:279] Couldn't get secret payload openshift-keda/kedaorg-certs: references non-existent secret key: ca.crt
Sep 17 17:39:53.617929 ip-10-0-137-20 kubenswrapper[2565]: E0917 17:39:53.617879    2565 projected.go:196] Error preparing data for projected volume certificates for pod openshift-keda/keda-operator-579c8958f7-2st78: [secret "keda-operator-certs" not found, references non-existent secret key: ca.crt]
Sep 17 17:39:53.618174 ip-10-0-137-20 kubenswrapper[2565]: E0917 17:39:53.617936    2565 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/3dadf1fc-acf6-4589-8ddd-65961b6ef4bb-certificates podName:3dadf1fc-acf6-4589-8ddd-65961b6ef4bb nodeName:}" failed. No retries permitted until 2026-09-17 17:39:54.117915719 +0000 UTC m=+343.995200112 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "certificates" (UniqueName: "kubernetes.io/projected/3dadf1fc-acf6-4589-8ddd-65961b6ef4bb-certificates") pod "keda-operator-579c8958f7-2st78" (UID: "3dadf1fc-acf6-4589-8ddd-65961b6ef4bb") : [secret "keda-operator-certs" not found, references non-existent secret key: ca.crt]
Sep 17 17:39:53.618375 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:39:53.618346    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cabundle0\" (UniqueName: \"kubernetes.io/configmap/3dadf1fc-acf6-4589-8ddd-65961b6ef4bb-cabundle0\") pod \"keda-operator-579c8958f7-2st78\" (UID: \"3dadf1fc-acf6-4589-8ddd-65961b6ef4bb\") " pod="openshift-keda/keda-operator-579c8958f7-2st78"
Sep 17 17:39:53.641721 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:39:53.641697    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-6sz75\" (UniqueName: \"kubernetes.io/projected/3dadf1fc-acf6-4589-8ddd-65961b6ef4bb-kube-api-access-6sz75\") pod \"keda-operator-579c8958f7-2st78\" (UID: \"3dadf1fc-acf6-4589-8ddd-65961b6ef4bb\") " pod="openshift-keda/keda-operator-579c8958f7-2st78"
Sep 17 17:39:53.652106 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:39:53.652078    2565 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-keda/keda-metrics-apiserver-77857b7b98-m2q7m"]
Sep 17 17:39:53.675846 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:39:53.675826    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-keda/keda-metrics-apiserver-77857b7b98-m2q7m"]
Sep 17 17:39:53.675971 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:39:53.675913    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-keda/keda-metrics-apiserver-77857b7b98-m2q7m"
Sep 17 17:39:53.683894 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:39:53.683874    2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-keda\"/\"keda-metrics-apiserver-certs\""
Sep 17 17:39:53.718875 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:39:53.718845    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"certificates\" (UniqueName: \"kubernetes.io/projected/c1e630ed-db58-4f31-9b6a-a4ad189f6d09-certificates\") pod \"keda-metrics-apiserver-77857b7b98-m2q7m\" (UID: \"c1e630ed-db58-4f31-9b6a-a4ad189f6d09\") " pod="openshift-keda/keda-metrics-apiserver-77857b7b98-m2q7m"
Sep 17 17:39:53.719011 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:39:53.718886    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-t4k8j\" (UniqueName: \"kubernetes.io/projected/c1e630ed-db58-4f31-9b6a-a4ad189f6d09-kube-api-access-t4k8j\") pod \"keda-metrics-apiserver-77857b7b98-m2q7m\" (UID: \"c1e630ed-db58-4f31-9b6a-a4ad189f6d09\") " pod="openshift-keda/keda-metrics-apiserver-77857b7b98-m2q7m"
Sep 17 17:39:53.719011 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:39:53.718981    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"temp-vol\" (UniqueName: \"kubernetes.io/empty-dir/c1e630ed-db58-4f31-9b6a-a4ad189f6d09-temp-vol\") pod \"keda-metrics-apiserver-77857b7b98-m2q7m\" (UID: \"c1e630ed-db58-4f31-9b6a-a4ad189f6d09\") " pod="openshift-keda/keda-metrics-apiserver-77857b7b98-m2q7m"
Sep 17 17:39:53.819629 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:39:53.819593    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"certificates\" (UniqueName: \"kubernetes.io/projected/c1e630ed-db58-4f31-9b6a-a4ad189f6d09-certificates\") pod \"keda-metrics-apiserver-77857b7b98-m2q7m\" (UID: \"c1e630ed-db58-4f31-9b6a-a4ad189f6d09\") " pod="openshift-keda/keda-metrics-apiserver-77857b7b98-m2q7m"
Sep 17 17:39:53.819789 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:39:53.819641    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-t4k8j\" (UniqueName: \"kubernetes.io/projected/c1e630ed-db58-4f31-9b6a-a4ad189f6d09-kube-api-access-t4k8j\") pod \"keda-metrics-apiserver-77857b7b98-m2q7m\" (UID: \"c1e630ed-db58-4f31-9b6a-a4ad189f6d09\") " pod="openshift-keda/keda-metrics-apiserver-77857b7b98-m2q7m"
Sep 17 17:39:53.819789 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:39:53.819671    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"temp-vol\" (UniqueName: \"kubernetes.io/empty-dir/c1e630ed-db58-4f31-9b6a-a4ad189f6d09-temp-vol\") pod \"keda-metrics-apiserver-77857b7b98-m2q7m\" (UID: \"c1e630ed-db58-4f31-9b6a-a4ad189f6d09\") " pod="openshift-keda/keda-metrics-apiserver-77857b7b98-m2q7m"
Sep 17 17:39:53.819789 ip-10-0-137-20 kubenswrapper[2565]: E0917 17:39:53.819740    2565 secret.go:281] references non-existent secret key: tls.crt
Sep 17 17:39:53.819789 ip-10-0-137-20 kubenswrapper[2565]: E0917 17:39:53.819764    2565 projected.go:279] Couldn't get secret payload openshift-keda/kedaorg-certs: references non-existent secret key: tls.crt
Sep 17 17:39:53.819789 ip-10-0-137-20 kubenswrapper[2565]: E0917 17:39:53.819785    2565 projected.go:196] Error preparing data for projected volume certificates for pod openshift-keda/keda-metrics-apiserver-77857b7b98-m2q7m: references non-existent secret key: tls.crt
Sep 17 17:39:53.820045 ip-10-0-137-20 kubenswrapper[2565]: E0917 17:39:53.819844    2565 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/c1e630ed-db58-4f31-9b6a-a4ad189f6d09-certificates podName:c1e630ed-db58-4f31-9b6a-a4ad189f6d09 nodeName:}" failed. No retries permitted until 2026-09-17 17:39:54.319826625 +0000 UTC m=+344.197111019 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "certificates" (UniqueName: "kubernetes.io/projected/c1e630ed-db58-4f31-9b6a-a4ad189f6d09-certificates") pod "keda-metrics-apiserver-77857b7b98-m2q7m" (UID: "c1e630ed-db58-4f31-9b6a-a4ad189f6d09") : references non-existent secret key: tls.crt
Sep 17 17:39:53.820045 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:39:53.819989    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"temp-vol\" (UniqueName: \"kubernetes.io/empty-dir/c1e630ed-db58-4f31-9b6a-a4ad189f6d09-temp-vol\") pod \"keda-metrics-apiserver-77857b7b98-m2q7m\" (UID: \"c1e630ed-db58-4f31-9b6a-a4ad189f6d09\") " pod="openshift-keda/keda-metrics-apiserver-77857b7b98-m2q7m"
Sep 17 17:39:53.831193 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:39:53.831171    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-t4k8j\" (UniqueName: \"kubernetes.io/projected/c1e630ed-db58-4f31-9b6a-a4ad189f6d09-kube-api-access-t4k8j\") pod \"keda-metrics-apiserver-77857b7b98-m2q7m\" (UID: \"c1e630ed-db58-4f31-9b6a-a4ad189f6d09\") " pod="openshift-keda/keda-metrics-apiserver-77857b7b98-m2q7m"
Sep 17 17:39:54.122510 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:39:54.122483    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"certificates\" (UniqueName: \"kubernetes.io/projected/3dadf1fc-acf6-4589-8ddd-65961b6ef4bb-certificates\") pod \"keda-operator-579c8958f7-2st78\" (UID: \"3dadf1fc-acf6-4589-8ddd-65961b6ef4bb\") " pod="openshift-keda/keda-operator-579c8958f7-2st78"
Sep 17 17:39:54.122673 ip-10-0-137-20 kubenswrapper[2565]: E0917 17:39:54.122638    2565 secret.go:281] references non-existent secret key: ca.crt
Sep 17 17:39:54.122673 ip-10-0-137-20 kubenswrapper[2565]: E0917 17:39:54.122658    2565 projected.go:279] Couldn't get secret payload openshift-keda/kedaorg-certs: references non-existent secret key: ca.crt
Sep 17 17:39:54.122673 ip-10-0-137-20 kubenswrapper[2565]: E0917 17:39:54.122666    2565 projected.go:196] Error preparing data for projected volume certificates for pod openshift-keda/keda-operator-579c8958f7-2st78: references non-existent secret key: ca.crt
Sep 17 17:39:54.122779 ip-10-0-137-20 kubenswrapper[2565]: E0917 17:39:54.122718    2565 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/3dadf1fc-acf6-4589-8ddd-65961b6ef4bb-certificates podName:3dadf1fc-acf6-4589-8ddd-65961b6ef4bb nodeName:}" failed. No retries permitted until 2026-09-17 17:39:55.122703935 +0000 UTC m=+344.999988328 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "certificates" (UniqueName: "kubernetes.io/projected/3dadf1fc-acf6-4589-8ddd-65961b6ef4bb-certificates") pod "keda-operator-579c8958f7-2st78" (UID: "3dadf1fc-acf6-4589-8ddd-65961b6ef4bb") : references non-existent secret key: ca.crt
Sep 17 17:39:54.322061 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:39:54.322015    2565 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-keda/keda-admission-6c4d864b86-cw7s4"]
Sep 17 17:39:54.323885 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:39:54.323852    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"certificates\" (UniqueName: \"kubernetes.io/projected/c1e630ed-db58-4f31-9b6a-a4ad189f6d09-certificates\") pod \"keda-metrics-apiserver-77857b7b98-m2q7m\" (UID: \"c1e630ed-db58-4f31-9b6a-a4ad189f6d09\") " pod="openshift-keda/keda-metrics-apiserver-77857b7b98-m2q7m"
Sep 17 17:39:54.326611 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:39:54.326585    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"certificates\" (UniqueName: \"kubernetes.io/projected/c1e630ed-db58-4f31-9b6a-a4ad189f6d09-certificates\") pod \"keda-metrics-apiserver-77857b7b98-m2q7m\" (UID: \"c1e630ed-db58-4f31-9b6a-a4ad189f6d09\") " pod="openshift-keda/keda-metrics-apiserver-77857b7b98-m2q7m"
Sep 17 17:39:54.341283 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:39:54.341262    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-keda/keda-admission-6c4d864b86-cw7s4"]
Sep 17 17:39:54.341374 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:39:54.341347    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-keda/keda-admission-6c4d864b86-cw7s4"
Sep 17 17:39:54.344486 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:39:54.344464    2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-keda\"/\"keda-admission-webhooks-certs\""
Sep 17 17:39:54.424913 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:39:54.424827    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9dkvw\" (UniqueName: \"kubernetes.io/projected/e637875a-cfc9-4652-8558-e2fa0097ce4f-kube-api-access-9dkvw\") pod \"keda-admission-6c4d864b86-cw7s4\" (UID: \"e637875a-cfc9-4652-8558-e2fa0097ce4f\") " pod="openshift-keda/keda-admission-6c4d864b86-cw7s4"
Sep 17 17:39:54.424913 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:39:54.424882    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"certificates\" (UniqueName: \"kubernetes.io/projected/e637875a-cfc9-4652-8558-e2fa0097ce4f-certificates\") pod \"keda-admission-6c4d864b86-cw7s4\" (UID: \"e637875a-cfc9-4652-8558-e2fa0097ce4f\") " pod="openshift-keda/keda-admission-6c4d864b86-cw7s4"
Sep 17 17:39:54.525771 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:39:54.525740    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9dkvw\" (UniqueName: \"kubernetes.io/projected/e637875a-cfc9-4652-8558-e2fa0097ce4f-kube-api-access-9dkvw\") pod \"keda-admission-6c4d864b86-cw7s4\" (UID: \"e637875a-cfc9-4652-8558-e2fa0097ce4f\") " pod="openshift-keda/keda-admission-6c4d864b86-cw7s4"
Sep 17 17:39:54.526314 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:39:54.525784    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"certificates\" (UniqueName: \"kubernetes.io/projected/e637875a-cfc9-4652-8558-e2fa0097ce4f-certificates\") pod \"keda-admission-6c4d864b86-cw7s4\" (UID: \"e637875a-cfc9-4652-8558-e2fa0097ce4f\") " pod="openshift-keda/keda-admission-6c4d864b86-cw7s4"
Sep 17 17:39:54.528977 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:39:54.528946    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"certificates\" (UniqueName: \"kubernetes.io/projected/e637875a-cfc9-4652-8558-e2fa0097ce4f-certificates\") pod \"keda-admission-6c4d864b86-cw7s4\" (UID: \"e637875a-cfc9-4652-8558-e2fa0097ce4f\") " pod="openshift-keda/keda-admission-6c4d864b86-cw7s4"
Sep 17 17:39:54.535395 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:39:54.535375    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-9dkvw\" (UniqueName: \"kubernetes.io/projected/e637875a-cfc9-4652-8558-e2fa0097ce4f-kube-api-access-9dkvw\") pod \"keda-admission-6c4d864b86-cw7s4\" (UID: \"e637875a-cfc9-4652-8558-e2fa0097ce4f\") " pod="openshift-keda/keda-admission-6c4d864b86-cw7s4"
Sep 17 17:39:54.586423 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:39:54.586383    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-keda/keda-metrics-apiserver-77857b7b98-m2q7m"
Sep 17 17:39:54.653830 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:39:54.652503    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-keda/keda-admission-6c4d864b86-cw7s4"
Sep 17 17:39:54.868056 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:39:54.868012    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-keda/keda-metrics-apiserver-77857b7b98-m2q7m"]
Sep 17 17:39:54.935881 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:39:54.935845    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-keda/keda-admission-6c4d864b86-cw7s4"]
Sep 17 17:39:54.937953 ip-10-0-137-20 kubenswrapper[2565]: W0917 17:39:54.937924    2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode637875a_cfc9_4652_8558_e2fa0097ce4f.slice/crio-84c269c1ba0bba2c0f24f667bd1d7a682a8313578510922ada56b97de3aeb31c WatchSource:0}: Error finding container 84c269c1ba0bba2c0f24f667bd1d7a682a8313578510922ada56b97de3aeb31c: Status 404 returned error can't find the container with id 84c269c1ba0bba2c0f24f667bd1d7a682a8313578510922ada56b97de3aeb31c
Sep 17 17:39:55.130763 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:39:55.130734    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"certificates\" (UniqueName: \"kubernetes.io/projected/3dadf1fc-acf6-4589-8ddd-65961b6ef4bb-certificates\") pod \"keda-operator-579c8958f7-2st78\" (UID: \"3dadf1fc-acf6-4589-8ddd-65961b6ef4bb\") " pod="openshift-keda/keda-operator-579c8958f7-2st78"
Sep 17 17:39:55.133499 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:39:55.133473    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"certificates\" (UniqueName: \"kubernetes.io/projected/3dadf1fc-acf6-4589-8ddd-65961b6ef4bb-certificates\") pod \"keda-operator-579c8958f7-2st78\" (UID: \"3dadf1fc-acf6-4589-8ddd-65961b6ef4bb\") " pod="openshift-keda/keda-operator-579c8958f7-2st78"
Sep 17 17:39:55.262232 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:39:55.262198    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-keda/keda-operator-579c8958f7-2st78"
Sep 17 17:39:55.521835 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:39:55.521743    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-keda/keda-metrics-apiserver-77857b7b98-m2q7m" event={"ID":"c1e630ed-db58-4f31-9b6a-a4ad189f6d09","Type":"ContainerStarted","Data":"1e35ccc276db7f6d9f3b4c910d7bb386046b8e4b814fc33505732a09d78fe689"}
Sep 17 17:39:55.523413 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:39:55.523380    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-keda/keda-admission-6c4d864b86-cw7s4" event={"ID":"e637875a-cfc9-4652-8558-e2fa0097ce4f","Type":"ContainerStarted","Data":"84c269c1ba0bba2c0f24f667bd1d7a682a8313578510922ada56b97de3aeb31c"}
Sep 17 17:39:55.541537 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:39:55.541512    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-keda/keda-operator-579c8958f7-2st78"]
Sep 17 17:39:55.544247 ip-10-0-137-20 kubenswrapper[2565]: W0917 17:39:55.544219    2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3dadf1fc_acf6_4589_8ddd_65961b6ef4bb.slice/crio-eedb373610f361f2af88b44d218c346c19c9a4fa331c58152fad016e81a5880a WatchSource:0}: Error finding container eedb373610f361f2af88b44d218c346c19c9a4fa331c58152fad016e81a5880a: Status 404 returned error can't find the container with id eedb373610f361f2af88b44d218c346c19c9a4fa331c58152fad016e81a5880a
Sep 17 17:39:56.530627 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:39:56.530594    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-keda/keda-operator-579c8958f7-2st78" event={"ID":"3dadf1fc-acf6-4589-8ddd-65961b6ef4bb","Type":"ContainerStarted","Data":"eedb373610f361f2af88b44d218c346c19c9a4fa331c58152fad016e81a5880a"}
Sep 17 17:39:56.532482 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:39:56.532454    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-keda/keda-admission-6c4d864b86-cw7s4" event={"ID":"e637875a-cfc9-4652-8558-e2fa0097ce4f","Type":"ContainerStarted","Data":"c395c64b038e636f670818dc81fbcac8d63e1c940b1d711e975942b59f87b709"}
Sep 17 17:39:56.532688 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:39:56.532664    2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-keda/keda-admission-6c4d864b86-cw7s4"
Sep 17 17:39:56.557215 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:39:56.557126    2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-keda/keda-admission-6c4d864b86-cw7s4" podStartSLOduration=1.205526405 podStartE2EDuration="2.557109566s" totalImagesPullingTime="1.351583161s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-17 17:39:54 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-17 17:39:56.554567222 +0000 UTC m=+346.431851637" watchObservedRunningTime="2026-09-17 17:39:56.557109566 +0000 UTC m=+346.434393971"
Sep 17 17:39:59.556141 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:39:59.556104    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-keda/keda-operator-579c8958f7-2st78" event={"ID":"3dadf1fc-acf6-4589-8ddd-65961b6ef4bb","Type":"ContainerStarted","Data":"5303858775e8e918cbf062f45ad6983bb8a60c313fc542f1df878c446cbd9d6e"}
Sep 17 17:39:59.556593 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:39:59.556210    2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-keda/keda-operator-579c8958f7-2st78"
Sep 17 17:39:59.557893 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:39:59.557870    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-keda/keda-metrics-apiserver-77857b7b98-m2q7m" event={"ID":"c1e630ed-db58-4f31-9b6a-a4ad189f6d09","Type":"ContainerStarted","Data":"ac09e0fd500ebc0f027570136cf60385953f4dc73cb39090417f4b82d4d173f8"}
Sep 17 17:39:59.558053 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:39:59.558016    2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-keda/keda-metrics-apiserver-77857b7b98-m2q7m"
Sep 17 17:39:59.575606 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:39:59.575566    2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-keda/keda-operator-579c8958f7-2st78" podStartSLOduration=3.107254392 podStartE2EDuration="6.575553946s" totalImagesPullingTime="3.468299554s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-17 17:39:53 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-17 17:39:59.574264518 +0000 UTC m=+349.451548935" watchObservedRunningTime="2026-09-17 17:39:59.575553946 +0000 UTC m=+349.452838361"
Sep 17 17:39:59.591720 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:39:59.591682    2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-keda/keda-metrics-apiserver-77857b7b98-m2q7m" podStartSLOduration=3.554464158 podStartE2EDuration="6.591666428s" totalImagesPullingTime="3.03720227s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-17 17:39:53 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-17 17:39:59.591647601 +0000 UTC m=+349.468932019" watchObservedRunningTime="2026-09-17 17:39:59.591666428 +0000 UTC m=+349.468950844"
Sep 17 17:40:10.567210 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:40:10.567178    2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-keda/keda-metrics-apiserver-77857b7b98-m2q7m"
Sep 17 17:40:14.517756 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:40:14.517730    2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-keda/custom-metrics-autoscaler-operator-566c9cd69c-x2bfq"
Sep 17 17:40:17.548361 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:40:17.548328    2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-keda/keda-admission-6c4d864b86-cw7s4"
Sep 17 17:40:20.565234 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:40:20.565203    2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-keda/keda-operator-579c8958f7-2st78"
Sep 17 17:40:44.079914 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:40:44.079882    2565 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-6c6b6c5656-hljcq"]
Sep 17 17:40:44.088334 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:40:44.088313    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-6c6b6c5656-hljcq"
Sep 17 17:40:44.116205 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:40:44.116174    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-6c6b6c5656-hljcq"]
Sep 17 17:40:44.133038 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:40:44.133002    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/bed50f13-88fb-41c6-9894-0b88a4eea04d-console-oauth-config\") pod \"console-6c6b6c5656-hljcq\" (UID: \"bed50f13-88fb-41c6-9894-0b88a4eea04d\") " pod="openshift-console/console-6c6b6c5656-hljcq"
Sep 17 17:40:44.133205 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:40:44.133054    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/bed50f13-88fb-41c6-9894-0b88a4eea04d-oauth-serving-cert\") pod \"console-6c6b6c5656-hljcq\" (UID: \"bed50f13-88fb-41c6-9894-0b88a4eea04d\") " pod="openshift-console/console-6c6b6c5656-hljcq"
Sep 17 17:40:44.133205 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:40:44.133142    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/bed50f13-88fb-41c6-9894-0b88a4eea04d-service-ca\") pod \"console-6c6b6c5656-hljcq\" (UID: \"bed50f13-88fb-41c6-9894-0b88a4eea04d\") " pod="openshift-console/console-6c6b6c5656-hljcq"
Sep 17 17:40:44.133205 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:40:44.133184    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/bed50f13-88fb-41c6-9894-0b88a4eea04d-console-config\") pod \"console-6c6b6c5656-hljcq\" (UID: \"bed50f13-88fb-41c6-9894-0b88a4eea04d\") " pod="openshift-console/console-6c6b6c5656-hljcq"
Sep 17 17:40:44.133319 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:40:44.133208    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/bed50f13-88fb-41c6-9894-0b88a4eea04d-trusted-ca-bundle\") pod \"console-6c6b6c5656-hljcq\" (UID: \"bed50f13-88fb-41c6-9894-0b88a4eea04d\") " pod="openshift-console/console-6c6b6c5656-hljcq"
Sep 17 17:40:44.133319 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:40:44.133235    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/bed50f13-88fb-41c6-9894-0b88a4eea04d-console-serving-cert\") pod \"console-6c6b6c5656-hljcq\" (UID: \"bed50f13-88fb-41c6-9894-0b88a4eea04d\") " pod="openshift-console/console-6c6b6c5656-hljcq"
Sep 17 17:40:44.133319 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:40:44.133280    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9mrwt\" (UniqueName: \"kubernetes.io/projected/bed50f13-88fb-41c6-9894-0b88a4eea04d-kube-api-access-9mrwt\") pod \"console-6c6b6c5656-hljcq\" (UID: \"bed50f13-88fb-41c6-9894-0b88a4eea04d\") " pod="openshift-console/console-6c6b6c5656-hljcq"
Sep 17 17:40:44.234174 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:40:44.234135    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/bed50f13-88fb-41c6-9894-0b88a4eea04d-console-oauth-config\") pod \"console-6c6b6c5656-hljcq\" (UID: \"bed50f13-88fb-41c6-9894-0b88a4eea04d\") " pod="openshift-console/console-6c6b6c5656-hljcq"
Sep 17 17:40:44.234174 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:40:44.234180    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/bed50f13-88fb-41c6-9894-0b88a4eea04d-oauth-serving-cert\") pod \"console-6c6b6c5656-hljcq\" (UID: \"bed50f13-88fb-41c6-9894-0b88a4eea04d\") " pod="openshift-console/console-6c6b6c5656-hljcq"
Sep 17 17:40:44.234402 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:40:44.234204    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/bed50f13-88fb-41c6-9894-0b88a4eea04d-service-ca\") pod \"console-6c6b6c5656-hljcq\" (UID: \"bed50f13-88fb-41c6-9894-0b88a4eea04d\") " pod="openshift-console/console-6c6b6c5656-hljcq"
Sep 17 17:40:44.234402 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:40:44.234240    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/bed50f13-88fb-41c6-9894-0b88a4eea04d-console-config\") pod \"console-6c6b6c5656-hljcq\" (UID: \"bed50f13-88fb-41c6-9894-0b88a4eea04d\") " pod="openshift-console/console-6c6b6c5656-hljcq"
Sep 17 17:40:44.234402 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:40:44.234271    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/bed50f13-88fb-41c6-9894-0b88a4eea04d-trusted-ca-bundle\") pod \"console-6c6b6c5656-hljcq\" (UID: \"bed50f13-88fb-41c6-9894-0b88a4eea04d\") " pod="openshift-console/console-6c6b6c5656-hljcq"
Sep 17 17:40:44.234402 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:40:44.234304    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/bed50f13-88fb-41c6-9894-0b88a4eea04d-console-serving-cert\") pod \"console-6c6b6c5656-hljcq\" (UID: \"bed50f13-88fb-41c6-9894-0b88a4eea04d\") " pod="openshift-console/console-6c6b6c5656-hljcq"
Sep 17 17:40:44.234402 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:40:44.234333    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9mrwt\" (UniqueName: \"kubernetes.io/projected/bed50f13-88fb-41c6-9894-0b88a4eea04d-kube-api-access-9mrwt\") pod \"console-6c6b6c5656-hljcq\" (UID: \"bed50f13-88fb-41c6-9894-0b88a4eea04d\") " pod="openshift-console/console-6c6b6c5656-hljcq"
Sep 17 17:40:44.235000 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:40:44.234966    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/bed50f13-88fb-41c6-9894-0b88a4eea04d-oauth-serving-cert\") pod \"console-6c6b6c5656-hljcq\" (UID: \"bed50f13-88fb-41c6-9894-0b88a4eea04d\") " pod="openshift-console/console-6c6b6c5656-hljcq"
Sep 17 17:40:44.235158 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:40:44.235055    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/bed50f13-88fb-41c6-9894-0b88a4eea04d-console-config\") pod \"console-6c6b6c5656-hljcq\" (UID: \"bed50f13-88fb-41c6-9894-0b88a4eea04d\") " pod="openshift-console/console-6c6b6c5656-hljcq"
Sep 17 17:40:44.235158 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:40:44.235129    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/bed50f13-88fb-41c6-9894-0b88a4eea04d-service-ca\") pod \"console-6c6b6c5656-hljcq\" (UID: \"bed50f13-88fb-41c6-9894-0b88a4eea04d\") " pod="openshift-console/console-6c6b6c5656-hljcq"
Sep 17 17:40:44.235272 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:40:44.235189    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/bed50f13-88fb-41c6-9894-0b88a4eea04d-trusted-ca-bundle\") pod \"console-6c6b6c5656-hljcq\" (UID: \"bed50f13-88fb-41c6-9894-0b88a4eea04d\") " pod="openshift-console/console-6c6b6c5656-hljcq"
Sep 17 17:40:44.236851 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:40:44.236818    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/bed50f13-88fb-41c6-9894-0b88a4eea04d-console-serving-cert\") pod \"console-6c6b6c5656-hljcq\" (UID: \"bed50f13-88fb-41c6-9894-0b88a4eea04d\") " pod="openshift-console/console-6c6b6c5656-hljcq"
Sep 17 17:40:44.236851 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:40:44.236843    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/bed50f13-88fb-41c6-9894-0b88a4eea04d-console-oauth-config\") pod \"console-6c6b6c5656-hljcq\" (UID: \"bed50f13-88fb-41c6-9894-0b88a4eea04d\") " pod="openshift-console/console-6c6b6c5656-hljcq"
Sep 17 17:40:44.245231 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:40:44.245203    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-9mrwt\" (UniqueName: \"kubernetes.io/projected/bed50f13-88fb-41c6-9894-0b88a4eea04d-kube-api-access-9mrwt\") pod \"console-6c6b6c5656-hljcq\" (UID: \"bed50f13-88fb-41c6-9894-0b88a4eea04d\") " pod="openshift-console/console-6c6b6c5656-hljcq"
Sep 17 17:40:44.397383 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:40:44.397331    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-6c6b6c5656-hljcq"
Sep 17 17:40:44.678466 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:40:44.678387    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-6c6b6c5656-hljcq"]
Sep 17 17:40:44.681730 ip-10-0-137-20 kubenswrapper[2565]: W0917 17:40:44.681705    2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podbed50f13_88fb_41c6_9894_0b88a4eea04d.slice/crio-a983c04389e58e7d95651d0e6343070e460061fedbd1f63afd6c9485c69404b4 WatchSource:0}: Error finding container a983c04389e58e7d95651d0e6343070e460061fedbd1f63afd6c9485c69404b4: Status 404 returned error can't find the container with id a983c04389e58e7d95651d0e6343070e460061fedbd1f63afd6c9485c69404b4
Sep 17 17:40:44.748566 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:40:44.748530    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-6c6b6c5656-hljcq" event={"ID":"bed50f13-88fb-41c6-9894-0b88a4eea04d","Type":"ContainerStarted","Data":"99a0af43330adc5a8ad779015533155df62d14fb27920fd967e3975986ebc81f"}
Sep 17 17:40:44.748705 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:40:44.748582    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-6c6b6c5656-hljcq" event={"ID":"bed50f13-88fb-41c6-9894-0b88a4eea04d","Type":"ContainerStarted","Data":"a983c04389e58e7d95651d0e6343070e460061fedbd1f63afd6c9485c69404b4"}
Sep 17 17:40:44.772603 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:40:44.772549    2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-6c6b6c5656-hljcq" podStartSLOduration=0.772532249 podStartE2EDuration="772.532249ms" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-17 17:40:44 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-17 17:40:44.771747697 +0000 UTC m=+394.649032117" watchObservedRunningTime="2026-09-17 17:40:44.772532249 +0000 UTC m=+394.649816664"
Sep 17 17:40:54.398084 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:40:54.398043    2565 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-6c6b6c5656-hljcq"
Sep 17 17:40:54.398514 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:40:54.398109    2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-6c6b6c5656-hljcq"
Sep 17 17:40:54.403133 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:40:54.403109    2565 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-6c6b6c5656-hljcq"
Sep 17 17:40:54.792687 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:40:54.792616    2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-6c6b6c5656-hljcq"
Sep 17 17:40:54.862007 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:40:54.861976    2565 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-559db6f977-s2k6s"]
Sep 17 17:41:14.917811 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:41:14.917776    2565 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kserve/kserve-controller-manager-9b487cb79-qp7jz"]
Sep 17 17:41:14.921370 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:41:14.921349    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve/kserve-controller-manager-9b487cb79-qp7jz"
Sep 17 17:41:14.924210 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:41:14.924188    2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kserve\"/\"kserve-webhook-server-cert\""
Sep 17 17:41:14.924484 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:41:14.924300    2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kserve\"/\"kserve-controller-manager-dockercfg-xtzfl\""
Sep 17 17:41:14.924561 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:41:14.924353    2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kserve\"/\"openshift-service-ca.crt\""
Sep 17 17:41:14.924866 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:41:14.924375    2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kserve\"/\"kube-root-ca.crt\""
Sep 17 17:41:14.926766 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:41:14.926741    2565 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kserve/llmisvc-controller-manager-645f9c8fd6-h9x48"]
Sep 17 17:41:14.935148 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:41:14.935124    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve/llmisvc-controller-manager-645f9c8fd6-h9x48"
Sep 17 17:41:14.936072 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:41:14.936043    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve/kserve-controller-manager-9b487cb79-qp7jz"]
Sep 17 17:41:14.938213 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:41:14.938165    2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kserve\"/\"llmisvc-controller-manager-dockercfg-mhrsv\""
Sep 17 17:41:14.938424 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:41:14.938408    2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kserve\"/\"llmisvc-webhook-server-cert\""
Sep 17 17:41:14.949319 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:41:14.949281    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve/llmisvc-controller-manager-645f9c8fd6-h9x48"]
Sep 17 17:41:14.995669 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:41:14.995633    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qhjjx\" (UniqueName: \"kubernetes.io/projected/018b2f50-1a29-436a-997a-97c6c3ac724e-kube-api-access-qhjjx\") pod \"kserve-controller-manager-9b487cb79-qp7jz\" (UID: \"018b2f50-1a29-436a-997a-97c6c3ac724e\") " pod="kserve/kserve-controller-manager-9b487cb79-qp7jz"
Sep 17 17:41:14.995669 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:41:14.995667    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/018b2f50-1a29-436a-997a-97c6c3ac724e-cert\") pod \"kserve-controller-manager-9b487cb79-qp7jz\" (UID: \"018b2f50-1a29-436a-997a-97c6c3ac724e\") " pod="kserve/kserve-controller-manager-9b487cb79-qp7jz"
Sep 17 17:41:15.096350 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:41:15.096307    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qhjjx\" (UniqueName: \"kubernetes.io/projected/018b2f50-1a29-436a-997a-97c6c3ac724e-kube-api-access-qhjjx\") pod \"kserve-controller-manager-9b487cb79-qp7jz\" (UID: \"018b2f50-1a29-436a-997a-97c6c3ac724e\") " pod="kserve/kserve-controller-manager-9b487cb79-qp7jz"
Sep 17 17:41:15.096350 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:41:15.096355    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/018b2f50-1a29-436a-997a-97c6c3ac724e-cert\") pod \"kserve-controller-manager-9b487cb79-qp7jz\" (UID: \"018b2f50-1a29-436a-997a-97c6c3ac724e\") " pod="kserve/kserve-controller-manager-9b487cb79-qp7jz"
Sep 17 17:41:15.096587 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:41:15.096390    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/6e815937-ab94-4f42-a671-e2fc88bb7df3-cert\") pod \"llmisvc-controller-manager-645f9c8fd6-h9x48\" (UID: \"6e815937-ab94-4f42-a671-e2fc88bb7df3\") " pod="kserve/llmisvc-controller-manager-645f9c8fd6-h9x48"
Sep 17 17:41:15.096587 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:41:15.096465    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mmwft\" (UniqueName: \"kubernetes.io/projected/6e815937-ab94-4f42-a671-e2fc88bb7df3-kube-api-access-mmwft\") pod \"llmisvc-controller-manager-645f9c8fd6-h9x48\" (UID: \"6e815937-ab94-4f42-a671-e2fc88bb7df3\") " pod="kserve/llmisvc-controller-manager-645f9c8fd6-h9x48"
Sep 17 17:41:15.096587 ip-10-0-137-20 kubenswrapper[2565]: E0917 17:41:15.096543    2565 secret.go:189] Couldn't get secret kserve/kserve-webhook-server-cert: secret "kserve-webhook-server-cert" not found
Sep 17 17:41:15.096707 ip-10-0-137-20 kubenswrapper[2565]: E0917 17:41:15.096617    2565 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/018b2f50-1a29-436a-997a-97c6c3ac724e-cert podName:018b2f50-1a29-436a-997a-97c6c3ac724e nodeName:}" failed. No retries permitted until 2026-09-17 17:41:15.596595937 +0000 UTC m=+425.473880352 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "cert" (UniqueName: "kubernetes.io/secret/018b2f50-1a29-436a-997a-97c6c3ac724e-cert") pod "kserve-controller-manager-9b487cb79-qp7jz" (UID: "018b2f50-1a29-436a-997a-97c6c3ac724e") : secret "kserve-webhook-server-cert" not found
Sep 17 17:41:15.104922 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:41:15.104894    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-qhjjx\" (UniqueName: \"kubernetes.io/projected/018b2f50-1a29-436a-997a-97c6c3ac724e-kube-api-access-qhjjx\") pod \"kserve-controller-manager-9b487cb79-qp7jz\" (UID: \"018b2f50-1a29-436a-997a-97c6c3ac724e\") " pod="kserve/kserve-controller-manager-9b487cb79-qp7jz"
Sep 17 17:41:15.197756 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:41:15.197668    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/6e815937-ab94-4f42-a671-e2fc88bb7df3-cert\") pod \"llmisvc-controller-manager-645f9c8fd6-h9x48\" (UID: \"6e815937-ab94-4f42-a671-e2fc88bb7df3\") " pod="kserve/llmisvc-controller-manager-645f9c8fd6-h9x48"
Sep 17 17:41:15.197756 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:41:15.197722    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mmwft\" (UniqueName: \"kubernetes.io/projected/6e815937-ab94-4f42-a671-e2fc88bb7df3-kube-api-access-mmwft\") pod \"llmisvc-controller-manager-645f9c8fd6-h9x48\" (UID: \"6e815937-ab94-4f42-a671-e2fc88bb7df3\") " pod="kserve/llmisvc-controller-manager-645f9c8fd6-h9x48"
Sep 17 17:41:15.200086 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:41:15.200060    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/6e815937-ab94-4f42-a671-e2fc88bb7df3-cert\") pod \"llmisvc-controller-manager-645f9c8fd6-h9x48\" (UID: \"6e815937-ab94-4f42-a671-e2fc88bb7df3\") " pod="kserve/llmisvc-controller-manager-645f9c8fd6-h9x48"
Sep 17 17:41:15.208361 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:41:15.208336    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-mmwft\" (UniqueName: \"kubernetes.io/projected/6e815937-ab94-4f42-a671-e2fc88bb7df3-kube-api-access-mmwft\") pod \"llmisvc-controller-manager-645f9c8fd6-h9x48\" (UID: \"6e815937-ab94-4f42-a671-e2fc88bb7df3\") " pod="kserve/llmisvc-controller-manager-645f9c8fd6-h9x48"
Sep 17 17:41:15.248319 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:41:15.248298    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve/llmisvc-controller-manager-645f9c8fd6-h9x48"
Sep 17 17:41:15.517280 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:41:15.517257    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve/llmisvc-controller-manager-645f9c8fd6-h9x48"]
Sep 17 17:41:15.519615 ip-10-0-137-20 kubenswrapper[2565]: W0917 17:41:15.519590    2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-pod6e815937_ab94_4f42_a671_e2fc88bb7df3.slice/crio-7f323465b1d56cd1f877543b9dd7362d187d7c188bc4bddd1b22cea5c0d1aea8 WatchSource:0}: Error finding container 7f323465b1d56cd1f877543b9dd7362d187d7c188bc4bddd1b22cea5c0d1aea8: Status 404 returned error can't find the container with id 7f323465b1d56cd1f877543b9dd7362d187d7c188bc4bddd1b22cea5c0d1aea8
Sep 17 17:41:15.601593 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:41:15.601545    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/018b2f50-1a29-436a-997a-97c6c3ac724e-cert\") pod \"kserve-controller-manager-9b487cb79-qp7jz\" (UID: \"018b2f50-1a29-436a-997a-97c6c3ac724e\") " pod="kserve/kserve-controller-manager-9b487cb79-qp7jz"
Sep 17 17:41:15.603915 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:41:15.603886    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/018b2f50-1a29-436a-997a-97c6c3ac724e-cert\") pod \"kserve-controller-manager-9b487cb79-qp7jz\" (UID: \"018b2f50-1a29-436a-997a-97c6c3ac724e\") " pod="kserve/kserve-controller-manager-9b487cb79-qp7jz"
Sep 17 17:41:15.836254 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:41:15.836169    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve/kserve-controller-manager-9b487cb79-qp7jz"
Sep 17 17:41:15.878909 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:41:15.878877    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve/llmisvc-controller-manager-645f9c8fd6-h9x48" event={"ID":"6e815937-ab94-4f42-a671-e2fc88bb7df3","Type":"ContainerStarted","Data":"7f323465b1d56cd1f877543b9dd7362d187d7c188bc4bddd1b22cea5c0d1aea8"}
Sep 17 17:41:16.112838 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:41:16.112804    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve/kserve-controller-manager-9b487cb79-qp7jz"]
Sep 17 17:41:16.116987 ip-10-0-137-20 kubenswrapper[2565]: W0917 17:41:16.116958    2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod018b2f50_1a29_436a_997a_97c6c3ac724e.slice/crio-ed3daa4c3eae01d1b142aab63a25de8b352c7c2e1e7674dcd980db289eccf787 WatchSource:0}: Error finding container ed3daa4c3eae01d1b142aab63a25de8b352c7c2e1e7674dcd980db289eccf787: Status 404 returned error can't find the container with id ed3daa4c3eae01d1b142aab63a25de8b352c7c2e1e7674dcd980db289eccf787
Sep 17 17:41:16.888347 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:41:16.888242    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve/kserve-controller-manager-9b487cb79-qp7jz" event={"ID":"018b2f50-1a29-436a-997a-97c6c3ac724e","Type":"ContainerStarted","Data":"ed3daa4c3eae01d1b142aab63a25de8b352c7c2e1e7674dcd980db289eccf787"}
Sep 17 17:41:18.901770 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:41:18.901734    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve/llmisvc-controller-manager-645f9c8fd6-h9x48" event={"ID":"6e815937-ab94-4f42-a671-e2fc88bb7df3","Type":"ContainerStarted","Data":"ba67136bd1d695392a49d6b8ece15c81db994964949057da9e18708591062d05"}
Sep 17 17:41:18.902190 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:41:18.901902    2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve/llmisvc-controller-manager-645f9c8fd6-h9x48"
Sep 17 17:41:18.903288 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:41:18.903268    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve/kserve-controller-manager-9b487cb79-qp7jz" event={"ID":"018b2f50-1a29-436a-997a-97c6c3ac724e","Type":"ContainerStarted","Data":"0d5ea0408ac0a2dc14214d3f163d21f6d569d0d9368647ded087f392a1b804de"}
Sep 17 17:41:18.903373 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:41:18.903309    2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve/kserve-controller-manager-9b487cb79-qp7jz"
Sep 17 17:41:18.921420 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:41:18.921377    2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kserve/llmisvc-controller-manager-645f9c8fd6-h9x48" podStartSLOduration=1.734650775 podStartE2EDuration="4.921363457s" totalImagesPullingTime="3.186712682s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-17 17:41:14 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-17 17:41:18.919596083 +0000 UTC m=+428.796880499" watchObservedRunningTime="2026-09-17 17:41:18.921363457 +0000 UTC m=+428.798647865"
Sep 17 17:41:18.937144 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:41:18.937103    2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kserve/kserve-controller-manager-9b487cb79-qp7jz" podStartSLOduration=2.345639698 podStartE2EDuration="4.937090699s" totalImagesPullingTime="2.591451001s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-17 17:41:14 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-17 17:41:18.936409995 +0000 UTC m=+428.813694410" watchObservedRunningTime="2026-09-17 17:41:18.937090699 +0000 UTC m=+428.814375114"
Sep 17 17:41:19.880607 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:41:19.880565    2565 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-console/console-559db6f977-s2k6s" podUID="b4162b2e-9aef-42bb-8463-b693b7b07ef5" containerName="console" containerID="cri-o://cca984d354944884efece9efed4d979c56b89e824202c25a3911c9ca5d8bcc32" gracePeriod=15
Sep 17 17:41:20.119882 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:41:20.119855    2565 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-559db6f977-s2k6s"
Sep 17 17:41:20.246343 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:41:20.246252    2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/b4162b2e-9aef-42bb-8463-b693b7b07ef5-console-oauth-config\") pod \"b4162b2e-9aef-42bb-8463-b693b7b07ef5\" (UID: \"b4162b2e-9aef-42bb-8463-b693b7b07ef5\") "
Sep 17 17:41:20.246343 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:41:20.246289    2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/b4162b2e-9aef-42bb-8463-b693b7b07ef5-console-config\") pod \"b4162b2e-9aef-42bb-8463-b693b7b07ef5\" (UID: \"b4162b2e-9aef-42bb-8463-b693b7b07ef5\") "
Sep 17 17:41:20.246606 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:41:20.246356    2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/b4162b2e-9aef-42bb-8463-b693b7b07ef5-trusted-ca-bundle\") pod \"b4162b2e-9aef-42bb-8463-b693b7b07ef5\" (UID: \"b4162b2e-9aef-42bb-8463-b693b7b07ef5\") "
Sep 17 17:41:20.246606 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:41:20.246459    2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-bpxpc\" (UniqueName: \"kubernetes.io/projected/b4162b2e-9aef-42bb-8463-b693b7b07ef5-kube-api-access-bpxpc\") pod \"b4162b2e-9aef-42bb-8463-b693b7b07ef5\" (UID: \"b4162b2e-9aef-42bb-8463-b693b7b07ef5\") "
Sep 17 17:41:20.246606 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:41:20.246510    2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/b4162b2e-9aef-42bb-8463-b693b7b07ef5-service-ca\") pod \"b4162b2e-9aef-42bb-8463-b693b7b07ef5\" (UID: \"b4162b2e-9aef-42bb-8463-b693b7b07ef5\") "
Sep 17 17:41:20.246606 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:41:20.246540    2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/b4162b2e-9aef-42bb-8463-b693b7b07ef5-console-serving-cert\") pod \"b4162b2e-9aef-42bb-8463-b693b7b07ef5\" (UID: \"b4162b2e-9aef-42bb-8463-b693b7b07ef5\") "
Sep 17 17:41:20.246606 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:41:20.246576    2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/b4162b2e-9aef-42bb-8463-b693b7b07ef5-oauth-serving-cert\") pod \"b4162b2e-9aef-42bb-8463-b693b7b07ef5\" (UID: \"b4162b2e-9aef-42bb-8463-b693b7b07ef5\") "
Sep 17 17:41:20.247307 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:41:20.246979    2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/b4162b2e-9aef-42bb-8463-b693b7b07ef5-trusted-ca-bundle" (OuterVolumeSpecName: "trusted-ca-bundle") pod "b4162b2e-9aef-42bb-8463-b693b7b07ef5" (UID: "b4162b2e-9aef-42bb-8463-b693b7b07ef5"). InnerVolumeSpecName "trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 17 17:41:20.247456 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:41:20.247271    2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/b4162b2e-9aef-42bb-8463-b693b7b07ef5-service-ca" (OuterVolumeSpecName: "service-ca") pod "b4162b2e-9aef-42bb-8463-b693b7b07ef5" (UID: "b4162b2e-9aef-42bb-8463-b693b7b07ef5"). InnerVolumeSpecName "service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 17 17:41:20.247456 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:41:20.247281    2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/b4162b2e-9aef-42bb-8463-b693b7b07ef5-oauth-serving-cert" (OuterVolumeSpecName: "oauth-serving-cert") pod "b4162b2e-9aef-42bb-8463-b693b7b07ef5" (UID: "b4162b2e-9aef-42bb-8463-b693b7b07ef5"). InnerVolumeSpecName "oauth-serving-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 17 17:41:20.247456 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:41:20.247362    2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/b4162b2e-9aef-42bb-8463-b693b7b07ef5-console-config" (OuterVolumeSpecName: "console-config") pod "b4162b2e-9aef-42bb-8463-b693b7b07ef5" (UID: "b4162b2e-9aef-42bb-8463-b693b7b07ef5"). InnerVolumeSpecName "console-config". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 17 17:41:20.248846 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:41:20.248795    2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/b4162b2e-9aef-42bb-8463-b693b7b07ef5-console-serving-cert" (OuterVolumeSpecName: "console-serving-cert") pod "b4162b2e-9aef-42bb-8463-b693b7b07ef5" (UID: "b4162b2e-9aef-42bb-8463-b693b7b07ef5"). InnerVolumeSpecName "console-serving-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 17 17:41:20.248953 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:41:20.248889    2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/b4162b2e-9aef-42bb-8463-b693b7b07ef5-kube-api-access-bpxpc" (OuterVolumeSpecName: "kube-api-access-bpxpc") pod "b4162b2e-9aef-42bb-8463-b693b7b07ef5" (UID: "b4162b2e-9aef-42bb-8463-b693b7b07ef5"). InnerVolumeSpecName "kube-api-access-bpxpc". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 17 17:41:20.248953 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:41:20.248944    2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/b4162b2e-9aef-42bb-8463-b693b7b07ef5-console-oauth-config" (OuterVolumeSpecName: "console-oauth-config") pod "b4162b2e-9aef-42bb-8463-b693b7b07ef5" (UID: "b4162b2e-9aef-42bb-8463-b693b7b07ef5"). InnerVolumeSpecName "console-oauth-config". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 17 17:41:20.347565 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:41:20.347527    2565 reconciler_common.go:299] "Volume detached for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/b4162b2e-9aef-42bb-8463-b693b7b07ef5-trusted-ca-bundle\") on node \"ip-10-0-137-20.ec2.internal\" DevicePath \"\""
Sep 17 17:41:20.347565 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:41:20.347561    2565 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-bpxpc\" (UniqueName: \"kubernetes.io/projected/b4162b2e-9aef-42bb-8463-b693b7b07ef5-kube-api-access-bpxpc\") on node \"ip-10-0-137-20.ec2.internal\" DevicePath \"\""
Sep 17 17:41:20.347565 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:41:20.347572    2565 reconciler_common.go:299] "Volume detached for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/b4162b2e-9aef-42bb-8463-b693b7b07ef5-service-ca\") on node \"ip-10-0-137-20.ec2.internal\" DevicePath \"\""
Sep 17 17:41:20.347803 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:41:20.347582    2565 reconciler_common.go:299] "Volume detached for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/b4162b2e-9aef-42bb-8463-b693b7b07ef5-console-serving-cert\") on node \"ip-10-0-137-20.ec2.internal\" DevicePath \"\""
Sep 17 17:41:20.347803 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:41:20.347591    2565 reconciler_common.go:299] "Volume detached for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/b4162b2e-9aef-42bb-8463-b693b7b07ef5-oauth-serving-cert\") on node \"ip-10-0-137-20.ec2.internal\" DevicePath \"\""
Sep 17 17:41:20.347803 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:41:20.347599    2565 reconciler_common.go:299] "Volume detached for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/b4162b2e-9aef-42bb-8463-b693b7b07ef5-console-oauth-config\") on node \"ip-10-0-137-20.ec2.internal\" DevicePath \"\""
Sep 17 17:41:20.347803 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:41:20.347608    2565 reconciler_common.go:299] "Volume detached for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/b4162b2e-9aef-42bb-8463-b693b7b07ef5-console-config\") on node \"ip-10-0-137-20.ec2.internal\" DevicePath \"\""
Sep 17 17:41:20.913484 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:41:20.913447    2565 generic.go:356] "Generic (PLEG): container finished" podID="b4162b2e-9aef-42bb-8463-b693b7b07ef5" containerID="cca984d354944884efece9efed4d979c56b89e824202c25a3911c9ca5d8bcc32" exitCode=0
Sep 17 17:41:20.913658 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:41:20.913497    2565 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-559db6f977-s2k6s"
Sep 17 17:41:20.913658 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:41:20.913539    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-559db6f977-s2k6s" event={"ID":"b4162b2e-9aef-42bb-8463-b693b7b07ef5","Type":"ContainerDied","Data":"cca984d354944884efece9efed4d979c56b89e824202c25a3911c9ca5d8bcc32"}
Sep 17 17:41:20.913658 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:41:20.913595    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-559db6f977-s2k6s" event={"ID":"b4162b2e-9aef-42bb-8463-b693b7b07ef5","Type":"ContainerDied","Data":"b5da8266f30cbda332e63bb8983e450fe122727818116c2e7ad43f0d0d9f22fd"}
Sep 17 17:41:20.913658 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:41:20.913618    2565 scope.go:117] "RemoveContainer" containerID="cca984d354944884efece9efed4d979c56b89e824202c25a3911c9ca5d8bcc32"
Sep 17 17:41:20.922323 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:41:20.922305    2565 scope.go:117] "RemoveContainer" containerID="cca984d354944884efece9efed4d979c56b89e824202c25a3911c9ca5d8bcc32"
Sep 17 17:41:20.922576 ip-10-0-137-20 kubenswrapper[2565]: E0917 17:41:20.922559    2565 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"cca984d354944884efece9efed4d979c56b89e824202c25a3911c9ca5d8bcc32\": container with ID starting with cca984d354944884efece9efed4d979c56b89e824202c25a3911c9ca5d8bcc32 not found: ID does not exist" containerID="cca984d354944884efece9efed4d979c56b89e824202c25a3911c9ca5d8bcc32"
Sep 17 17:41:20.922621 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:41:20.922584    2565 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"cca984d354944884efece9efed4d979c56b89e824202c25a3911c9ca5d8bcc32"} err="failed to get container status \"cca984d354944884efece9efed4d979c56b89e824202c25a3911c9ca5d8bcc32\": rpc error: code = NotFound desc = could not find container \"cca984d354944884efece9efed4d979c56b89e824202c25a3911c9ca5d8bcc32\": container with ID starting with cca984d354944884efece9efed4d979c56b89e824202c25a3911c9ca5d8bcc32 not found: ID does not exist"
Sep 17 17:41:20.937942 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:41:20.937912    2565 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-559db6f977-s2k6s"]
Sep 17 17:41:20.944523 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:41:20.944494    2565 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-console/console-559db6f977-s2k6s"]
Sep 17 17:41:22.155991 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:41:22.155958    2565 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="b4162b2e-9aef-42bb-8463-b693b7b07ef5" path="/var/lib/kubelet/pods/b4162b2e-9aef-42bb-8463-b693b7b07ef5/volumes"
Sep 17 17:41:49.909716 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:41:49.909684    2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kserve/llmisvc-controller-manager-645f9c8fd6-h9x48"
Sep 17 17:41:49.912666 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:41:49.912646    2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kserve/kserve-controller-manager-9b487cb79-qp7jz"
Sep 17 17:41:58.140805 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:41:58.140771    2565 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kserve/kserve-controller-manager-9b487cb79-qp7jz"]
Sep 17 17:41:58.141276 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:41:58.140987    2565 kuberuntime_container.go:873] "Killing container with a grace period" pod="kserve/kserve-controller-manager-9b487cb79-qp7jz" podUID="018b2f50-1a29-436a-997a-97c6c3ac724e" containerName="manager" containerID="cri-o://0d5ea0408ac0a2dc14214d3f163d21f6d569d0d9368647ded087f392a1b804de" gracePeriod=10
Sep 17 17:41:58.164065 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:41:58.163998    2565 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kserve/kserve-controller-manager-9b487cb79-78k8l"]
Sep 17 17:41:58.164356 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:41:58.164342    2565 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b4162b2e-9aef-42bb-8463-b693b7b07ef5" containerName="console"
Sep 17 17:41:58.164398 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:41:58.164358    2565 state_mem.go:107] "Deleted CPUSet assignment" podUID="b4162b2e-9aef-42bb-8463-b693b7b07ef5" containerName="console"
Sep 17 17:41:58.164432 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:41:58.164427    2565 memory_manager.go:368] "RemoveStaleState removing state" podUID="b4162b2e-9aef-42bb-8463-b693b7b07ef5" containerName="console"
Sep 17 17:41:58.167270 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:41:58.167253    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve/kserve-controller-manager-9b487cb79-78k8l"
Sep 17 17:41:58.179064 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:41:58.179043    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve/kserve-controller-manager-9b487cb79-78k8l"]
Sep 17 17:41:58.236965 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:41:58.236941    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rt229\" (UniqueName: \"kubernetes.io/projected/4950e910-c87f-4318-9cbd-25eac814a99f-kube-api-access-rt229\") pod \"kserve-controller-manager-9b487cb79-78k8l\" (UID: \"4950e910-c87f-4318-9cbd-25eac814a99f\") " pod="kserve/kserve-controller-manager-9b487cb79-78k8l"
Sep 17 17:41:58.237066 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:41:58.236993    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/4950e910-c87f-4318-9cbd-25eac814a99f-cert\") pod \"kserve-controller-manager-9b487cb79-78k8l\" (UID: \"4950e910-c87f-4318-9cbd-25eac814a99f\") " pod="kserve/kserve-controller-manager-9b487cb79-78k8l"
Sep 17 17:41:58.337719 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:41:58.337687    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/4950e910-c87f-4318-9cbd-25eac814a99f-cert\") pod \"kserve-controller-manager-9b487cb79-78k8l\" (UID: \"4950e910-c87f-4318-9cbd-25eac814a99f\") " pod="kserve/kserve-controller-manager-9b487cb79-78k8l"
Sep 17 17:41:58.337867 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:41:58.337781    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rt229\" (UniqueName: \"kubernetes.io/projected/4950e910-c87f-4318-9cbd-25eac814a99f-kube-api-access-rt229\") pod \"kserve-controller-manager-9b487cb79-78k8l\" (UID: \"4950e910-c87f-4318-9cbd-25eac814a99f\") " pod="kserve/kserve-controller-manager-9b487cb79-78k8l"
Sep 17 17:41:58.339981 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:41:58.339954    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/4950e910-c87f-4318-9cbd-25eac814a99f-cert\") pod \"kserve-controller-manager-9b487cb79-78k8l\" (UID: \"4950e910-c87f-4318-9cbd-25eac814a99f\") " pod="kserve/kserve-controller-manager-9b487cb79-78k8l"
Sep 17 17:41:58.347286 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:41:58.347263    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-rt229\" (UniqueName: \"kubernetes.io/projected/4950e910-c87f-4318-9cbd-25eac814a99f-kube-api-access-rt229\") pod \"kserve-controller-manager-9b487cb79-78k8l\" (UID: \"4950e910-c87f-4318-9cbd-25eac814a99f\") " pod="kserve/kserve-controller-manager-9b487cb79-78k8l"
Sep 17 17:41:58.380936 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:41:58.380914    2565 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve/kserve-controller-manager-9b487cb79-qp7jz"
Sep 17 17:41:58.438304 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:41:58.438239    2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-qhjjx\" (UniqueName: \"kubernetes.io/projected/018b2f50-1a29-436a-997a-97c6c3ac724e-kube-api-access-qhjjx\") pod \"018b2f50-1a29-436a-997a-97c6c3ac724e\" (UID: \"018b2f50-1a29-436a-997a-97c6c3ac724e\") "
Sep 17 17:41:58.438437 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:41:58.438326    2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/018b2f50-1a29-436a-997a-97c6c3ac724e-cert\") pod \"018b2f50-1a29-436a-997a-97c6c3ac724e\" (UID: \"018b2f50-1a29-436a-997a-97c6c3ac724e\") "
Sep 17 17:41:58.440310 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:41:58.440285    2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/018b2f50-1a29-436a-997a-97c6c3ac724e-cert" (OuterVolumeSpecName: "cert") pod "018b2f50-1a29-436a-997a-97c6c3ac724e" (UID: "018b2f50-1a29-436a-997a-97c6c3ac724e"). InnerVolumeSpecName "cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 17 17:41:58.440417 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:41:58.440364    2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/018b2f50-1a29-436a-997a-97c6c3ac724e-kube-api-access-qhjjx" (OuterVolumeSpecName: "kube-api-access-qhjjx") pod "018b2f50-1a29-436a-997a-97c6c3ac724e" (UID: "018b2f50-1a29-436a-997a-97c6c3ac724e"). InnerVolumeSpecName "kube-api-access-qhjjx". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 17 17:41:58.536015 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:41:58.535987    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve/kserve-controller-manager-9b487cb79-78k8l"
Sep 17 17:41:58.538910 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:41:58.538847    2565 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-qhjjx\" (UniqueName: \"kubernetes.io/projected/018b2f50-1a29-436a-997a-97c6c3ac724e-kube-api-access-qhjjx\") on node \"ip-10-0-137-20.ec2.internal\" DevicePath \"\""
Sep 17 17:41:58.538910 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:41:58.538865    2565 reconciler_common.go:299] "Volume detached for volume \"cert\" (UniqueName: \"kubernetes.io/secret/018b2f50-1a29-436a-997a-97c6c3ac724e-cert\") on node \"ip-10-0-137-20.ec2.internal\" DevicePath \"\""
Sep 17 17:41:58.804337 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:41:58.804311    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve/kserve-controller-manager-9b487cb79-78k8l"]
Sep 17 17:41:58.806130 ip-10-0-137-20 kubenswrapper[2565]: W0917 17:41:58.806105    2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4950e910_c87f_4318_9cbd_25eac814a99f.slice/crio-e1643c89db5a4ec7bd509d3f941bfd085a0cf51963dc32e763e31b7a86708ac9 WatchSource:0}: Error finding container e1643c89db5a4ec7bd509d3f941bfd085a0cf51963dc32e763e31b7a86708ac9: Status 404 returned error can't find the container with id e1643c89db5a4ec7bd509d3f941bfd085a0cf51963dc32e763e31b7a86708ac9
Sep 17 17:41:59.082436 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:41:59.082345    2565 generic.go:356] "Generic (PLEG): container finished" podID="018b2f50-1a29-436a-997a-97c6c3ac724e" containerID="0d5ea0408ac0a2dc14214d3f163d21f6d569d0d9368647ded087f392a1b804de" exitCode=0
Sep 17 17:41:59.082436 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:41:59.082384    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve/kserve-controller-manager-9b487cb79-qp7jz" event={"ID":"018b2f50-1a29-436a-997a-97c6c3ac724e","Type":"ContainerDied","Data":"0d5ea0408ac0a2dc14214d3f163d21f6d569d0d9368647ded087f392a1b804de"}
Sep 17 17:41:59.082436 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:41:59.082437    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve/kserve-controller-manager-9b487cb79-qp7jz" event={"ID":"018b2f50-1a29-436a-997a-97c6c3ac724e","Type":"ContainerDied","Data":"ed3daa4c3eae01d1b142aab63a25de8b352c7c2e1e7674dcd980db289eccf787"}
Sep 17 17:41:59.082720 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:41:59.082396    2565 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve/kserve-controller-manager-9b487cb79-qp7jz"
Sep 17 17:41:59.082720 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:41:59.082468    2565 scope.go:117] "RemoveContainer" containerID="0d5ea0408ac0a2dc14214d3f163d21f6d569d0d9368647ded087f392a1b804de"
Sep 17 17:41:59.083784 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:41:59.083764    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve/kserve-controller-manager-9b487cb79-78k8l" event={"ID":"4950e910-c87f-4318-9cbd-25eac814a99f","Type":"ContainerStarted","Data":"e1643c89db5a4ec7bd509d3f941bfd085a0cf51963dc32e763e31b7a86708ac9"}
Sep 17 17:41:59.091312 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:41:59.091292    2565 scope.go:117] "RemoveContainer" containerID="0d5ea0408ac0a2dc14214d3f163d21f6d569d0d9368647ded087f392a1b804de"
Sep 17 17:41:59.091537 ip-10-0-137-20 kubenswrapper[2565]: E0917 17:41:59.091522    2565 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"0d5ea0408ac0a2dc14214d3f163d21f6d569d0d9368647ded087f392a1b804de\": container with ID starting with 0d5ea0408ac0a2dc14214d3f163d21f6d569d0d9368647ded087f392a1b804de not found: ID does not exist" containerID="0d5ea0408ac0a2dc14214d3f163d21f6d569d0d9368647ded087f392a1b804de"
Sep 17 17:41:59.091599 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:41:59.091550    2565 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"0d5ea0408ac0a2dc14214d3f163d21f6d569d0d9368647ded087f392a1b804de"} err="failed to get container status \"0d5ea0408ac0a2dc14214d3f163d21f6d569d0d9368647ded087f392a1b804de\": rpc error: code = NotFound desc = could not find container \"0d5ea0408ac0a2dc14214d3f163d21f6d569d0d9368647ded087f392a1b804de\": container with ID starting with 0d5ea0408ac0a2dc14214d3f163d21f6d569d0d9368647ded087f392a1b804de not found: ID does not exist"
Sep 17 17:41:59.111555 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:41:59.109078    2565 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kserve/kserve-controller-manager-9b487cb79-qp7jz"]
Sep 17 17:41:59.113412 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:41:59.113391    2565 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kserve/kserve-controller-manager-9b487cb79-qp7jz"]
Sep 17 17:42:00.090950 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:42:00.090918    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve/kserve-controller-manager-9b487cb79-78k8l" event={"ID":"4950e910-c87f-4318-9cbd-25eac814a99f","Type":"ContainerStarted","Data":"ca9b89b46822f616b68bb4957e7d659e9a08f84a389f622acafe93b8ecb359d5"}
Sep 17 17:42:00.091394 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:42:00.091101    2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve/kserve-controller-manager-9b487cb79-78k8l"
Sep 17 17:42:00.112954 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:42:00.112903    2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kserve/kserve-controller-manager-9b487cb79-78k8l" podStartSLOduration=1.702848521 podStartE2EDuration="2.112888592s" totalImagesPullingTime="410.040071ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-17 17:41:58 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-17 17:42:00.111238674 +0000 UTC m=+469.988523090" watchObservedRunningTime="2026-09-17 17:42:00.112888592 +0000 UTC m=+469.990173007"
Sep 17 17:42:00.155990 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:42:00.155959    2565 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="018b2f50-1a29-436a-997a-97c6c3ac724e" path="/var/lib/kubelet/pods/018b2f50-1a29-436a-997a-97c6c3ac724e/volumes"
Sep 17 17:42:31.100893 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:42:31.100860    2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kserve/kserve-controller-manager-9b487cb79-78k8l"
Sep 17 17:42:32.502349 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:42:32.502319    2565 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kserve/model-serving-api-7b954ffc95-sgr9b"]
Sep 17 17:42:32.502701 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:42:32.502653    2565 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="018b2f50-1a29-436a-997a-97c6c3ac724e" containerName="manager"
Sep 17 17:42:32.502701 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:42:32.502665    2565 state_mem.go:107] "Deleted CPUSet assignment" podUID="018b2f50-1a29-436a-997a-97c6c3ac724e" containerName="manager"
Sep 17 17:42:32.502780 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:42:32.502722    2565 memory_manager.go:368] "RemoveStaleState removing state" podUID="018b2f50-1a29-436a-997a-97c6c3ac724e" containerName="manager"
Sep 17 17:42:32.505564 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:42:32.505546    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve/model-serving-api-7b954ffc95-sgr9b"
Sep 17 17:42:32.513089 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:42:32.513070    2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kserve\"/\"model-serving-api-dockercfg-dvdm9\""
Sep 17 17:42:32.514150 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:42:32.514133    2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kserve\"/\"model-serving-api-tls\""
Sep 17 17:42:32.519971 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:42:32.519950    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve/model-serving-api-7b954ffc95-sgr9b"]
Sep 17 17:42:32.526330 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:42:32.526310    2565 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kserve/odh-model-controller-7b966d7984-pjq5z"]
Sep 17 17:42:32.529384 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:42:32.529367    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve/odh-model-controller-7b966d7984-pjq5z"
Sep 17 17:42:32.532765 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:42:32.532734    2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kserve\"/\"odh-model-controller-dockercfg-hmzml\""
Sep 17 17:42:32.533036 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:42:32.533006    2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kserve\"/\"odh-model-controller-metrics-tls\""
Sep 17 17:42:32.533173 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:42:32.533157    2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kserve\"/\"odh-model-controller-webhook-cert\""
Sep 17 17:42:32.541370 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:42:32.541350    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve/odh-model-controller-7b966d7984-pjq5z"]
Sep 17 17:42:32.592603 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:42:32.592573    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-lq2hl\" (UniqueName: \"kubernetes.io/projected/86cff6b1-9d41-4174-8e3d-75b8728e0739-kube-api-access-lq2hl\") pod \"model-serving-api-7b954ffc95-sgr9b\" (UID: \"86cff6b1-9d41-4174-8e3d-75b8728e0739\") " pod="kserve/model-serving-api-7b954ffc95-sgr9b"
Sep 17 17:42:32.592727 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:42:32.592630    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/86cff6b1-9d41-4174-8e3d-75b8728e0739-tls-certs\") pod \"model-serving-api-7b954ffc95-sgr9b\" (UID: \"86cff6b1-9d41-4174-8e3d-75b8728e0739\") " pod="kserve/model-serving-api-7b954ffc95-sgr9b"
Sep 17 17:42:32.693844 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:42:32.693821    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/b54f7b04-81e8-4a44-bf33-49b39b9051c3-metrics-certs\") pod \"odh-model-controller-7b966d7984-pjq5z\" (UID: \"b54f7b04-81e8-4a44-bf33-49b39b9051c3\") " pod="kserve/odh-model-controller-7b966d7984-pjq5z"
Sep 17 17:42:32.693992 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:42:32.693870    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lq2hl\" (UniqueName: \"kubernetes.io/projected/86cff6b1-9d41-4174-8e3d-75b8728e0739-kube-api-access-lq2hl\") pod \"model-serving-api-7b954ffc95-sgr9b\" (UID: \"86cff6b1-9d41-4174-8e3d-75b8728e0739\") " pod="kserve/model-serving-api-7b954ffc95-sgr9b"
Sep 17 17:42:32.693992 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:42:32.693928    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/86cff6b1-9d41-4174-8e3d-75b8728e0739-tls-certs\") pod \"model-serving-api-7b954ffc95-sgr9b\" (UID: \"86cff6b1-9d41-4174-8e3d-75b8728e0739\") " pod="kserve/model-serving-api-7b954ffc95-sgr9b"
Sep 17 17:42:32.693992 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:42:32.693961    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ls98p\" (UniqueName: \"kubernetes.io/projected/b54f7b04-81e8-4a44-bf33-49b39b9051c3-kube-api-access-ls98p\") pod \"odh-model-controller-7b966d7984-pjq5z\" (UID: \"b54f7b04-81e8-4a44-bf33-49b39b9051c3\") " pod="kserve/odh-model-controller-7b966d7984-pjq5z"
Sep 17 17:42:32.694181 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:42:32.693993    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/b54f7b04-81e8-4a44-bf33-49b39b9051c3-cert\") pod \"odh-model-controller-7b966d7984-pjq5z\" (UID: \"b54f7b04-81e8-4a44-bf33-49b39b9051c3\") " pod="kserve/odh-model-controller-7b966d7984-pjq5z"
Sep 17 17:42:32.696410 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:42:32.696385    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/86cff6b1-9d41-4174-8e3d-75b8728e0739-tls-certs\") pod \"model-serving-api-7b954ffc95-sgr9b\" (UID: \"86cff6b1-9d41-4174-8e3d-75b8728e0739\") " pod="kserve/model-serving-api-7b954ffc95-sgr9b"
Sep 17 17:42:32.703893 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:42:32.703873    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-lq2hl\" (UniqueName: \"kubernetes.io/projected/86cff6b1-9d41-4174-8e3d-75b8728e0739-kube-api-access-lq2hl\") pod \"model-serving-api-7b954ffc95-sgr9b\" (UID: \"86cff6b1-9d41-4174-8e3d-75b8728e0739\") " pod="kserve/model-serving-api-7b954ffc95-sgr9b"
Sep 17 17:42:32.794915 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:42:32.794852    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ls98p\" (UniqueName: \"kubernetes.io/projected/b54f7b04-81e8-4a44-bf33-49b39b9051c3-kube-api-access-ls98p\") pod \"odh-model-controller-7b966d7984-pjq5z\" (UID: \"b54f7b04-81e8-4a44-bf33-49b39b9051c3\") " pod="kserve/odh-model-controller-7b966d7984-pjq5z"
Sep 17 17:42:32.794915 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:42:32.794883    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/b54f7b04-81e8-4a44-bf33-49b39b9051c3-cert\") pod \"odh-model-controller-7b966d7984-pjq5z\" (UID: \"b54f7b04-81e8-4a44-bf33-49b39b9051c3\") " pod="kserve/odh-model-controller-7b966d7984-pjq5z"
Sep 17 17:42:32.794915 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:42:32.794905    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/b54f7b04-81e8-4a44-bf33-49b39b9051c3-metrics-certs\") pod \"odh-model-controller-7b966d7984-pjq5z\" (UID: \"b54f7b04-81e8-4a44-bf33-49b39b9051c3\") " pod="kserve/odh-model-controller-7b966d7984-pjq5z"
Sep 17 17:42:32.795183 ip-10-0-137-20 kubenswrapper[2565]: E0917 17:42:32.795064    2565 secret.go:189] Couldn't get secret kserve/odh-model-controller-metrics-tls: secret "odh-model-controller-metrics-tls" not found
Sep 17 17:42:32.795183 ip-10-0-137-20 kubenswrapper[2565]: E0917 17:42:32.795147    2565 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b54f7b04-81e8-4a44-bf33-49b39b9051c3-metrics-certs podName:b54f7b04-81e8-4a44-bf33-49b39b9051c3 nodeName:}" failed. No retries permitted until 2026-09-17 17:42:33.295125458 +0000 UTC m=+503.172409854 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/b54f7b04-81e8-4a44-bf33-49b39b9051c3-metrics-certs") pod "odh-model-controller-7b966d7984-pjq5z" (UID: "b54f7b04-81e8-4a44-bf33-49b39b9051c3") : secret "odh-model-controller-metrics-tls" not found
Sep 17 17:42:32.797336 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:42:32.797315    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/b54f7b04-81e8-4a44-bf33-49b39b9051c3-cert\") pod \"odh-model-controller-7b966d7984-pjq5z\" (UID: \"b54f7b04-81e8-4a44-bf33-49b39b9051c3\") " pod="kserve/odh-model-controller-7b966d7984-pjq5z"
Sep 17 17:42:32.809604 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:42:32.809580    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-ls98p\" (UniqueName: \"kubernetes.io/projected/b54f7b04-81e8-4a44-bf33-49b39b9051c3-kube-api-access-ls98p\") pod \"odh-model-controller-7b966d7984-pjq5z\" (UID: \"b54f7b04-81e8-4a44-bf33-49b39b9051c3\") " pod="kserve/odh-model-controller-7b966d7984-pjq5z"
Sep 17 17:42:32.815460 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:42:32.815440    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve/model-serving-api-7b954ffc95-sgr9b"
Sep 17 17:42:33.086153 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:42:33.086126    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve/model-serving-api-7b954ffc95-sgr9b"]
Sep 17 17:42:33.088372 ip-10-0-137-20 kubenswrapper[2565]: W0917 17:42:33.088344    2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod86cff6b1_9d41_4174_8e3d_75b8728e0739.slice/crio-c2bd4e1c3454fff7ec2d123eb81b58aecce8b65f1b5d2ce01f93590461f05ba5 WatchSource:0}: Error finding container c2bd4e1c3454fff7ec2d123eb81b58aecce8b65f1b5d2ce01f93590461f05ba5: Status 404 returned error can't find the container with id c2bd4e1c3454fff7ec2d123eb81b58aecce8b65f1b5d2ce01f93590461f05ba5
Sep 17 17:42:33.229631 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:42:33.229599    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve/model-serving-api-7b954ffc95-sgr9b" event={"ID":"86cff6b1-9d41-4174-8e3d-75b8728e0739","Type":"ContainerStarted","Data":"c2bd4e1c3454fff7ec2d123eb81b58aecce8b65f1b5d2ce01f93590461f05ba5"}
Sep 17 17:42:33.297813 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:42:33.297782    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/b54f7b04-81e8-4a44-bf33-49b39b9051c3-metrics-certs\") pod \"odh-model-controller-7b966d7984-pjq5z\" (UID: \"b54f7b04-81e8-4a44-bf33-49b39b9051c3\") " pod="kserve/odh-model-controller-7b966d7984-pjq5z"
Sep 17 17:42:33.300182 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:42:33.300163    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/b54f7b04-81e8-4a44-bf33-49b39b9051c3-metrics-certs\") pod \"odh-model-controller-7b966d7984-pjq5z\" (UID: \"b54f7b04-81e8-4a44-bf33-49b39b9051c3\") " pod="kserve/odh-model-controller-7b966d7984-pjq5z"
Sep 17 17:42:33.438811 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:42:33.438780    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve/odh-model-controller-7b966d7984-pjq5z"
Sep 17 17:42:33.755395 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:42:33.755368    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve/odh-model-controller-7b966d7984-pjq5z"]
Sep 17 17:42:33.758080 ip-10-0-137-20 kubenswrapper[2565]: W0917 17:42:33.758048    2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb54f7b04_81e8_4a44_bf33_49b39b9051c3.slice/crio-1246884be08a74ff32a91e56c0e1d30817856073f751d9e839a27193dc872ac1 WatchSource:0}: Error finding container 1246884be08a74ff32a91e56c0e1d30817856073f751d9e839a27193dc872ac1: Status 404 returned error can't find the container with id 1246884be08a74ff32a91e56c0e1d30817856073f751d9e839a27193dc872ac1
Sep 17 17:42:34.241203 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:42:34.241158    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve/odh-model-controller-7b966d7984-pjq5z" event={"ID":"b54f7b04-81e8-4a44-bf33-49b39b9051c3","Type":"ContainerStarted","Data":"1246884be08a74ff32a91e56c0e1d30817856073f751d9e839a27193dc872ac1"}
Sep 17 17:42:37.259905 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:42:37.259866    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve/model-serving-api-7b954ffc95-sgr9b" event={"ID":"86cff6b1-9d41-4174-8e3d-75b8728e0739","Type":"ContainerStarted","Data":"1d2692bc2f94f229d464231bf32a47090a0d488a617bcde88e0a00920a1e29b5"}
Sep 17 17:42:37.260362 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:42:37.260040    2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve/model-serving-api-7b954ffc95-sgr9b"
Sep 17 17:42:37.261432 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:42:37.261406    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve/odh-model-controller-7b966d7984-pjq5z" event={"ID":"b54f7b04-81e8-4a44-bf33-49b39b9051c3","Type":"ContainerStarted","Data":"699543feff3f50bf494d87a4517431deee0c6b2c3dacdf34fd487b1c9fcf9cfb"}
Sep 17 17:42:37.261595 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:42:37.261503    2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve/odh-model-controller-7b966d7984-pjq5z"
Sep 17 17:42:37.285177 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:42:37.285137    2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kserve/model-serving-api-7b954ffc95-sgr9b" podStartSLOduration=1.816308628 podStartE2EDuration="5.285123658s" totalImagesPullingTime="3.46881503s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-17 17:42:32 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-17 17:42:37.283989226 +0000 UTC m=+507.161273645" watchObservedRunningTime="2026-09-17 17:42:37.285123658 +0000 UTC m=+507.162408072"
Sep 17 17:42:37.308436 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:42:37.308397    2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kserve/odh-model-controller-7b966d7984-pjq5z" podStartSLOduration=2.463550063 podStartE2EDuration="5.308385269s" totalImagesPullingTime="2.844835206s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-17 17:42:32 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-17 17:42:37.307104308 +0000 UTC m=+507.184388723" watchObservedRunningTime="2026-09-17 17:42:37.308385269 +0000 UTC m=+507.185669684"
Sep 17 17:42:48.268681 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:42:48.268653    2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kserve/odh-model-controller-7b966d7984-pjq5z"
Sep 17 17:42:48.270440 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:42:48.270419    2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kserve/model-serving-api-7b954ffc95-sgr9b"
Sep 17 17:44:12.094955 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:44:12.094928    2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-d9927_30837de7-1447-4de2-ac62-c809813b3338/ovn-acl-logging/0.log"
Sep 17 17:44:12.099911 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:44:12.099891    2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-d9927_30837de7-1447-4de2-ac62-c809813b3338/ovn-acl-logging/0.log"
Sep 17 17:44:19.492624 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:44:19.492591    2565 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kserve-ci-e2e-test/success-200-isvc-5b8c1-predictor-c5f74cf7b-l8qrz"]
Sep 17 17:44:19.495961 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:44:19.495940    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/success-200-isvc-5b8c1-predictor-c5f74cf7b-l8qrz"
Sep 17 17:44:19.498843 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:44:19.498822    2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kserve-ci-e2e-test\"/\"kube-root-ca.crt\""
Sep 17 17:44:19.498966 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:44:19.498834    2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kserve-ci-e2e-test\"/\"success-200-isvc-5b8c1-predictor-serving-cert\""
Sep 17 17:44:19.500482 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:44:19.500464    2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kserve-ci-e2e-test\"/\"success-200-isvc-5b8c1-kube-rbac-proxy-sar-config\""
Sep 17 17:44:19.500606 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:44:19.500501    2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kserve-ci-e2e-test\"/\"default-dockercfg-28pvs\""
Sep 17 17:44:19.500675 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:44:19.500502    2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kserve-ci-e2e-test\"/\"openshift-service-ca.crt\""
Sep 17 17:44:19.506898 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:44:19.506877    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/success-200-isvc-5b8c1-predictor-c5f74cf7b-l8qrz"]
Sep 17 17:44:19.601447 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:44:19.601416    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-75q2d\" (UniqueName: \"kubernetes.io/projected/b696b601-e784-4c61-963d-fd05aceb4ac6-kube-api-access-75q2d\") pod \"success-200-isvc-5b8c1-predictor-c5f74cf7b-l8qrz\" (UID: \"b696b601-e784-4c61-963d-fd05aceb4ac6\") " pod="kserve-ci-e2e-test/success-200-isvc-5b8c1-predictor-c5f74cf7b-l8qrz"
Sep 17 17:44:19.601580 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:44:19.601462    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/b696b601-e784-4c61-963d-fd05aceb4ac6-kube-rbac-proxy-sar-config\") pod \"success-200-isvc-5b8c1-predictor-c5f74cf7b-l8qrz\" (UID: \"b696b601-e784-4c61-963d-fd05aceb4ac6\") " pod="kserve-ci-e2e-test/success-200-isvc-5b8c1-predictor-c5f74cf7b-l8qrz"
Sep 17 17:44:19.601580 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:44:19.601515    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/b696b601-e784-4c61-963d-fd05aceb4ac6-proxy-tls\") pod \"success-200-isvc-5b8c1-predictor-c5f74cf7b-l8qrz\" (UID: \"b696b601-e784-4c61-963d-fd05aceb4ac6\") " pod="kserve-ci-e2e-test/success-200-isvc-5b8c1-predictor-c5f74cf7b-l8qrz"
Sep 17 17:44:19.702409 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:44:19.702375    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-75q2d\" (UniqueName: \"kubernetes.io/projected/b696b601-e784-4c61-963d-fd05aceb4ac6-kube-api-access-75q2d\") pod \"success-200-isvc-5b8c1-predictor-c5f74cf7b-l8qrz\" (UID: \"b696b601-e784-4c61-963d-fd05aceb4ac6\") " pod="kserve-ci-e2e-test/success-200-isvc-5b8c1-predictor-c5f74cf7b-l8qrz"
Sep 17 17:44:19.702577 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:44:19.702437    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/b696b601-e784-4c61-963d-fd05aceb4ac6-kube-rbac-proxy-sar-config\") pod \"success-200-isvc-5b8c1-predictor-c5f74cf7b-l8qrz\" (UID: \"b696b601-e784-4c61-963d-fd05aceb4ac6\") " pod="kserve-ci-e2e-test/success-200-isvc-5b8c1-predictor-c5f74cf7b-l8qrz"
Sep 17 17:44:19.702577 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:44:19.702480    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/b696b601-e784-4c61-963d-fd05aceb4ac6-proxy-tls\") pod \"success-200-isvc-5b8c1-predictor-c5f74cf7b-l8qrz\" (UID: \"b696b601-e784-4c61-963d-fd05aceb4ac6\") " pod="kserve-ci-e2e-test/success-200-isvc-5b8c1-predictor-c5f74cf7b-l8qrz"
Sep 17 17:44:19.702681 ip-10-0-137-20 kubenswrapper[2565]: E0917 17:44:19.702623    2565 secret.go:189] Couldn't get secret kserve-ci-e2e-test/success-200-isvc-5b8c1-predictor-serving-cert: secret "success-200-isvc-5b8c1-predictor-serving-cert" not found
Sep 17 17:44:19.702736 ip-10-0-137-20 kubenswrapper[2565]: E0917 17:44:19.702680    2565 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b696b601-e784-4c61-963d-fd05aceb4ac6-proxy-tls podName:b696b601-e784-4c61-963d-fd05aceb4ac6 nodeName:}" failed. No retries permitted until 2026-09-17 17:44:20.202661872 +0000 UTC m=+610.079946286 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "proxy-tls" (UniqueName: "kubernetes.io/secret/b696b601-e784-4c61-963d-fd05aceb4ac6-proxy-tls") pod "success-200-isvc-5b8c1-predictor-c5f74cf7b-l8qrz" (UID: "b696b601-e784-4c61-963d-fd05aceb4ac6") : secret "success-200-isvc-5b8c1-predictor-serving-cert" not found
Sep 17 17:44:19.703854 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:44:19.703831    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/b696b601-e784-4c61-963d-fd05aceb4ac6-kube-rbac-proxy-sar-config\") pod \"success-200-isvc-5b8c1-predictor-c5f74cf7b-l8qrz\" (UID: \"b696b601-e784-4c61-963d-fd05aceb4ac6\") " pod="kserve-ci-e2e-test/success-200-isvc-5b8c1-predictor-c5f74cf7b-l8qrz"
Sep 17 17:44:19.715328 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:44:19.715304    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-75q2d\" (UniqueName: \"kubernetes.io/projected/b696b601-e784-4c61-963d-fd05aceb4ac6-kube-api-access-75q2d\") pod \"success-200-isvc-5b8c1-predictor-c5f74cf7b-l8qrz\" (UID: \"b696b601-e784-4c61-963d-fd05aceb4ac6\") " pod="kserve-ci-e2e-test/success-200-isvc-5b8c1-predictor-c5f74cf7b-l8qrz"
Sep 17 17:44:19.781758 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:44:19.781697    2565 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kserve-ci-e2e-test/error-404-isvc-5b8c1-predictor-6d6d86965-4rplj"]
Sep 17 17:44:19.791558 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:44:19.791507    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/error-404-isvc-5b8c1-predictor-6d6d86965-4rplj"
Sep 17 17:44:19.795255 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:44:19.795234    2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kserve-ci-e2e-test\"/\"error-404-isvc-5b8c1-predictor-serving-cert\""
Sep 17 17:44:19.795376 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:44:19.795286    2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kserve-ci-e2e-test\"/\"error-404-isvc-5b8c1-kube-rbac-proxy-sar-config\""
Sep 17 17:44:19.798198 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:44:19.798173    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/error-404-isvc-5b8c1-predictor-6d6d86965-4rplj"]
Sep 17 17:44:19.903432 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:44:19.903406    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/0d0e8cef-9085-4468-9430-9585789557b9-kube-rbac-proxy-sar-config\") pod \"error-404-isvc-5b8c1-predictor-6d6d86965-4rplj\" (UID: \"0d0e8cef-9085-4468-9430-9585789557b9\") " pod="kserve-ci-e2e-test/error-404-isvc-5b8c1-predictor-6d6d86965-4rplj"
Sep 17 17:44:19.903626 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:44:19.903469    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9c7gc\" (UniqueName: \"kubernetes.io/projected/0d0e8cef-9085-4468-9430-9585789557b9-kube-api-access-9c7gc\") pod \"error-404-isvc-5b8c1-predictor-6d6d86965-4rplj\" (UID: \"0d0e8cef-9085-4468-9430-9585789557b9\") " pod="kserve-ci-e2e-test/error-404-isvc-5b8c1-predictor-6d6d86965-4rplj"
Sep 17 17:44:19.903626 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:44:19.903536    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/0d0e8cef-9085-4468-9430-9585789557b9-proxy-tls\") pod \"error-404-isvc-5b8c1-predictor-6d6d86965-4rplj\" (UID: \"0d0e8cef-9085-4468-9430-9585789557b9\") " pod="kserve-ci-e2e-test/error-404-isvc-5b8c1-predictor-6d6d86965-4rplj"
Sep 17 17:44:20.004835 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:44:20.004788    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/0d0e8cef-9085-4468-9430-9585789557b9-kube-rbac-proxy-sar-config\") pod \"error-404-isvc-5b8c1-predictor-6d6d86965-4rplj\" (UID: \"0d0e8cef-9085-4468-9430-9585789557b9\") " pod="kserve-ci-e2e-test/error-404-isvc-5b8c1-predictor-6d6d86965-4rplj"
Sep 17 17:44:20.004993 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:44:20.004872    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9c7gc\" (UniqueName: \"kubernetes.io/projected/0d0e8cef-9085-4468-9430-9585789557b9-kube-api-access-9c7gc\") pod \"error-404-isvc-5b8c1-predictor-6d6d86965-4rplj\" (UID: \"0d0e8cef-9085-4468-9430-9585789557b9\") " pod="kserve-ci-e2e-test/error-404-isvc-5b8c1-predictor-6d6d86965-4rplj"
Sep 17 17:44:20.004993 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:44:20.004904    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/0d0e8cef-9085-4468-9430-9585789557b9-proxy-tls\") pod \"error-404-isvc-5b8c1-predictor-6d6d86965-4rplj\" (UID: \"0d0e8cef-9085-4468-9430-9585789557b9\") " pod="kserve-ci-e2e-test/error-404-isvc-5b8c1-predictor-6d6d86965-4rplj"
Sep 17 17:44:20.005183 ip-10-0-137-20 kubenswrapper[2565]: E0917 17:44:20.005082    2565 secret.go:189] Couldn't get secret kserve-ci-e2e-test/error-404-isvc-5b8c1-predictor-serving-cert: secret "error-404-isvc-5b8c1-predictor-serving-cert" not found
Sep 17 17:44:20.005183 ip-10-0-137-20 kubenswrapper[2565]: E0917 17:44:20.005154    2565 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/0d0e8cef-9085-4468-9430-9585789557b9-proxy-tls podName:0d0e8cef-9085-4468-9430-9585789557b9 nodeName:}" failed. No retries permitted until 2026-09-17 17:44:20.505136558 +0000 UTC m=+610.382420958 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "proxy-tls" (UniqueName: "kubernetes.io/secret/0d0e8cef-9085-4468-9430-9585789557b9-proxy-tls") pod "error-404-isvc-5b8c1-predictor-6d6d86965-4rplj" (UID: "0d0e8cef-9085-4468-9430-9585789557b9") : secret "error-404-isvc-5b8c1-predictor-serving-cert" not found
Sep 17 17:44:20.005565 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:44:20.005542    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/0d0e8cef-9085-4468-9430-9585789557b9-kube-rbac-proxy-sar-config\") pod \"error-404-isvc-5b8c1-predictor-6d6d86965-4rplj\" (UID: \"0d0e8cef-9085-4468-9430-9585789557b9\") " pod="kserve-ci-e2e-test/error-404-isvc-5b8c1-predictor-6d6d86965-4rplj"
Sep 17 17:44:20.015440 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:44:20.015415    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-9c7gc\" (UniqueName: \"kubernetes.io/projected/0d0e8cef-9085-4468-9430-9585789557b9-kube-api-access-9c7gc\") pod \"error-404-isvc-5b8c1-predictor-6d6d86965-4rplj\" (UID: \"0d0e8cef-9085-4468-9430-9585789557b9\") " pod="kserve-ci-e2e-test/error-404-isvc-5b8c1-predictor-6d6d86965-4rplj"
Sep 17 17:44:20.206965 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:44:20.206927    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/b696b601-e784-4c61-963d-fd05aceb4ac6-proxy-tls\") pod \"success-200-isvc-5b8c1-predictor-c5f74cf7b-l8qrz\" (UID: \"b696b601-e784-4c61-963d-fd05aceb4ac6\") " pod="kserve-ci-e2e-test/success-200-isvc-5b8c1-predictor-c5f74cf7b-l8qrz"
Sep 17 17:44:20.209290 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:44:20.209270    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/b696b601-e784-4c61-963d-fd05aceb4ac6-proxy-tls\") pod \"success-200-isvc-5b8c1-predictor-c5f74cf7b-l8qrz\" (UID: \"b696b601-e784-4c61-963d-fd05aceb4ac6\") " pod="kserve-ci-e2e-test/success-200-isvc-5b8c1-predictor-c5f74cf7b-l8qrz"
Sep 17 17:44:20.407461 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:44:20.407416    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/success-200-isvc-5b8c1-predictor-c5f74cf7b-l8qrz"
Sep 17 17:44:20.508813 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:44:20.508766    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/0d0e8cef-9085-4468-9430-9585789557b9-proxy-tls\") pod \"error-404-isvc-5b8c1-predictor-6d6d86965-4rplj\" (UID: \"0d0e8cef-9085-4468-9430-9585789557b9\") " pod="kserve-ci-e2e-test/error-404-isvc-5b8c1-predictor-6d6d86965-4rplj"
Sep 17 17:44:20.509242 ip-10-0-137-20 kubenswrapper[2565]: E0917 17:44:20.508932    2565 secret.go:189] Couldn't get secret kserve-ci-e2e-test/error-404-isvc-5b8c1-predictor-serving-cert: secret "error-404-isvc-5b8c1-predictor-serving-cert" not found
Sep 17 17:44:20.509242 ip-10-0-137-20 kubenswrapper[2565]: E0917 17:44:20.509047    2565 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/0d0e8cef-9085-4468-9430-9585789557b9-proxy-tls podName:0d0e8cef-9085-4468-9430-9585789557b9 nodeName:}" failed. No retries permitted until 2026-09-17 17:44:21.509004298 +0000 UTC m=+611.386288693 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "proxy-tls" (UniqueName: "kubernetes.io/secret/0d0e8cef-9085-4468-9430-9585789557b9-proxy-tls") pod "error-404-isvc-5b8c1-predictor-6d6d86965-4rplj" (UID: "0d0e8cef-9085-4468-9430-9585789557b9") : secret "error-404-isvc-5b8c1-predictor-serving-cert" not found
Sep 17 17:44:20.684202 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:44:20.684172    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/success-200-isvc-5b8c1-predictor-c5f74cf7b-l8qrz"]
Sep 17 17:44:20.685909 ip-10-0-137-20 kubenswrapper[2565]: W0917 17:44:20.685884    2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb696b601_e784_4c61_963d_fd05aceb4ac6.slice/crio-7c16dae88e8051ea3ca533ea5325c225561aee5f867900b71cd1a026eecb9e9a WatchSource:0}: Error finding container 7c16dae88e8051ea3ca533ea5325c225561aee5f867900b71cd1a026eecb9e9a: Status 404 returned error can't find the container with id 7c16dae88e8051ea3ca533ea5325c225561aee5f867900b71cd1a026eecb9e9a
Sep 17 17:44:20.698737 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:44:20.698714    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/success-200-isvc-5b8c1-predictor-c5f74cf7b-l8qrz" event={"ID":"b696b601-e784-4c61-963d-fd05aceb4ac6","Type":"ContainerStarted","Data":"7c16dae88e8051ea3ca533ea5325c225561aee5f867900b71cd1a026eecb9e9a"}
Sep 17 17:44:21.517325 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:44:21.516685    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/0d0e8cef-9085-4468-9430-9585789557b9-proxy-tls\") pod \"error-404-isvc-5b8c1-predictor-6d6d86965-4rplj\" (UID: \"0d0e8cef-9085-4468-9430-9585789557b9\") " pod="kserve-ci-e2e-test/error-404-isvc-5b8c1-predictor-6d6d86965-4rplj"
Sep 17 17:44:21.526796 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:44:21.526746    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/0d0e8cef-9085-4468-9430-9585789557b9-proxy-tls\") pod \"error-404-isvc-5b8c1-predictor-6d6d86965-4rplj\" (UID: \"0d0e8cef-9085-4468-9430-9585789557b9\") " pod="kserve-ci-e2e-test/error-404-isvc-5b8c1-predictor-6d6d86965-4rplj"
Sep 17 17:44:21.606205 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:44:21.605859    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/error-404-isvc-5b8c1-predictor-6d6d86965-4rplj"
Sep 17 17:44:21.913980 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:44:21.913953    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/error-404-isvc-5b8c1-predictor-6d6d86965-4rplj"]
Sep 17 17:44:21.918509 ip-10-0-137-20 kubenswrapper[2565]: W0917 17:44:21.918478    2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0d0e8cef_9085_4468_9430_9585789557b9.slice/crio-025c40fe52983c057f73aa51eade2072bab75453da72bd0b611c32b6b3a26eec WatchSource:0}: Error finding container 025c40fe52983c057f73aa51eade2072bab75453da72bd0b611c32b6b3a26eec: Status 404 returned error can't find the container with id 025c40fe52983c057f73aa51eade2072bab75453da72bd0b611c32b6b3a26eec
Sep 17 17:44:22.724707 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:44:22.724667    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/error-404-isvc-5b8c1-predictor-6d6d86965-4rplj" event={"ID":"0d0e8cef-9085-4468-9430-9585789557b9","Type":"ContainerStarted","Data":"025c40fe52983c057f73aa51eade2072bab75453da72bd0b611c32b6b3a26eec"}
Sep 17 17:44:37.653017 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:44:37.652999    2565 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Sep 17 17:44:37.822702 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:44:37.822668    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/success-200-isvc-5b8c1-predictor-c5f74cf7b-l8qrz" event={"ID":"b696b601-e784-4c61-963d-fd05aceb4ac6","Type":"ContainerStarted","Data":"771d34b9b701a5ab1f8e944403c3245d4e1dd00016a61e91b94e5585811eb4d2"}
Sep 17 17:44:37.824164 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:44:37.824140    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/error-404-isvc-5b8c1-predictor-6d6d86965-4rplj" event={"ID":"0d0e8cef-9085-4468-9430-9585789557b9","Type":"ContainerStarted","Data":"4a9af9c7433f079ca990c8d71571c7b91e9bff5b52360700f9d0cedc62ff579f"}
Sep 17 17:44:40.846206 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:44:40.845879    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/error-404-isvc-5b8c1-predictor-6d6d86965-4rplj" event={"ID":"0d0e8cef-9085-4468-9430-9585789557b9","Type":"ContainerStarted","Data":"d0d790bb8633e1eb29a2d6a0f06ae91e6f8ab868701705617e082ae2b2e6af0d"}
Sep 17 17:44:40.846206 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:44:40.845957    2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve-ci-e2e-test/error-404-isvc-5b8c1-predictor-6d6d86965-4rplj"
Sep 17 17:44:40.846206 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:44:40.845994    2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve-ci-e2e-test/error-404-isvc-5b8c1-predictor-6d6d86965-4rplj"
Sep 17 17:44:40.847984 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:44:40.847944    2565 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/error-404-isvc-5b8c1-predictor-6d6d86965-4rplj" podUID="0d0e8cef-9085-4468-9430-9585789557b9" containerName="kserve-container" probeResult="failure" output="dial tcp 10.133.0.33:8080: connect: connection refused"
Sep 17 17:44:40.848358 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:44:40.848329    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/success-200-isvc-5b8c1-predictor-c5f74cf7b-l8qrz" event={"ID":"b696b601-e784-4c61-963d-fd05aceb4ac6","Type":"ContainerStarted","Data":"41f4ff8d69532697010d821e94d180aa58a984721c2d22bb1c26950a0a463fca"}
Sep 17 17:44:40.848505 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:44:40.848484    2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve-ci-e2e-test/success-200-isvc-5b8c1-predictor-c5f74cf7b-l8qrz"
Sep 17 17:44:40.866589 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:44:40.866542    2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kserve-ci-e2e-test/error-404-isvc-5b8c1-predictor-6d6d86965-4rplj" podStartSLOduration=3.431003939 podStartE2EDuration="21.866527365s" totalImagesPullingTime="18.435523426s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-17 17:44:19 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-17 17:44:40.864832141 +0000 UTC m=+630.742116659" watchObservedRunningTime="2026-09-17 17:44:40.866527365 +0000 UTC m=+630.743811778"
Sep 17 17:44:40.884829 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:44:40.884790    2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kserve-ci-e2e-test/success-200-isvc-5b8c1-predictor-c5f74cf7b-l8qrz" podStartSLOduration=2.292739923 podStartE2EDuration="21.884773342s" totalImagesPullingTime="19.592033419s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-17 17:44:19 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-17 17:44:40.884173853 +0000 UTC m=+630.761458281" watchObservedRunningTime="2026-09-17 17:44:40.884773342 +0000 UTC m=+630.762057759"
Sep 17 17:44:41.854318 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:44:41.854280    2565 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/error-404-isvc-5b8c1-predictor-6d6d86965-4rplj" podUID="0d0e8cef-9085-4468-9430-9585789557b9" containerName="kserve-container" probeResult="failure" output="dial tcp 10.133.0.33:8080: connect: connection refused"
Sep 17 17:44:41.854761 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:44:41.854394    2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve-ci-e2e-test/success-200-isvc-5b8c1-predictor-c5f74cf7b-l8qrz"
Sep 17 17:44:41.855655 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:44:41.855630    2565 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/success-200-isvc-5b8c1-predictor-c5f74cf7b-l8qrz" podUID="b696b601-e784-4c61-963d-fd05aceb4ac6" containerName="kserve-container" probeResult="failure" output="dial tcp 10.133.0.32:8080: connect: connection refused"
Sep 17 17:44:42.858310 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:44:42.858269    2565 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/success-200-isvc-5b8c1-predictor-c5f74cf7b-l8qrz" podUID="b696b601-e784-4c61-963d-fd05aceb4ac6" containerName="kserve-container" probeResult="failure" output="dial tcp 10.133.0.32:8080: connect: connection refused"
Sep 17 17:44:46.858448 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:44:46.858417    2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kserve-ci-e2e-test/error-404-isvc-5b8c1-predictor-6d6d86965-4rplj"
Sep 17 17:44:46.859046 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:44:46.858999    2565 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/error-404-isvc-5b8c1-predictor-6d6d86965-4rplj" podUID="0d0e8cef-9085-4468-9430-9585789557b9" containerName="kserve-container" probeResult="failure" output="dial tcp 10.133.0.33:8080: connect: connection refused"
Sep 17 17:44:47.863203 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:44:47.863175    2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kserve-ci-e2e-test/success-200-isvc-5b8c1-predictor-c5f74cf7b-l8qrz"
Sep 17 17:44:47.863672 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:44:47.863648    2565 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/success-200-isvc-5b8c1-predictor-c5f74cf7b-l8qrz" podUID="b696b601-e784-4c61-963d-fd05aceb4ac6" containerName="kserve-container" probeResult="failure" output="dial tcp 10.133.0.32:8080: connect: connection refused"
Sep 17 17:44:56.859577 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:44:56.859541    2565 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/error-404-isvc-5b8c1-predictor-6d6d86965-4rplj" podUID="0d0e8cef-9085-4468-9430-9585789557b9" containerName="kserve-container" probeResult="failure" output="dial tcp 10.133.0.33:8080: connect: connection refused"
Sep 17 17:44:57.863927 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:44:57.863889    2565 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/success-200-isvc-5b8c1-predictor-c5f74cf7b-l8qrz" podUID="b696b601-e784-4c61-963d-fd05aceb4ac6" containerName="kserve-container" probeResult="failure" output="dial tcp 10.133.0.32:8080: connect: connection refused"
Sep 17 17:45:06.859695 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:45:06.859654    2565 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/error-404-isvc-5b8c1-predictor-6d6d86965-4rplj" podUID="0d0e8cef-9085-4468-9430-9585789557b9" containerName="kserve-container" probeResult="failure" output="dial tcp 10.133.0.33:8080: connect: connection refused"
Sep 17 17:45:07.864304 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:45:07.864261    2565 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/success-200-isvc-5b8c1-predictor-c5f74cf7b-l8qrz" podUID="b696b601-e784-4c61-963d-fd05aceb4ac6" containerName="kserve-container" probeResult="failure" output="dial tcp 10.133.0.32:8080: connect: connection refused"
Sep 17 17:45:16.859659 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:45:16.859623    2565 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/error-404-isvc-5b8c1-predictor-6d6d86965-4rplj" podUID="0d0e8cef-9085-4468-9430-9585789557b9" containerName="kserve-container" probeResult="failure" output="dial tcp 10.133.0.33:8080: connect: connection refused"
Sep 17 17:45:17.863727 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:45:17.863691    2565 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/success-200-isvc-5b8c1-predictor-c5f74cf7b-l8qrz" podUID="b696b601-e784-4c61-963d-fd05aceb4ac6" containerName="kserve-container" probeResult="failure" output="dial tcp 10.133.0.32:8080: connect: connection refused"
Sep 17 17:45:26.860012 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:45:26.859979    2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kserve-ci-e2e-test/error-404-isvc-5b8c1-predictor-6d6d86965-4rplj"
Sep 17 17:45:27.864838 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:45:27.864810    2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kserve-ci-e2e-test/success-200-isvc-5b8c1-predictor-c5f74cf7b-l8qrz"
Sep 17 17:45:49.549538 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:45:49.549458    2565 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kserve-ci-e2e-test/success-200-isvc-5b8c1-predictor-c5f74cf7b-l8qrz"]
Sep 17 17:45:49.550014 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:45:49.549723    2565 kuberuntime_container.go:873] "Killing container with a grace period" pod="kserve-ci-e2e-test/success-200-isvc-5b8c1-predictor-c5f74cf7b-l8qrz" podUID="b696b601-e784-4c61-963d-fd05aceb4ac6" containerName="kserve-container" containerID="cri-o://771d34b9b701a5ab1f8e944403c3245d4e1dd00016a61e91b94e5585811eb4d2" gracePeriod=30
Sep 17 17:45:49.550014 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:45:49.549804    2565 kuberuntime_container.go:873] "Killing container with a grace period" pod="kserve-ci-e2e-test/success-200-isvc-5b8c1-predictor-c5f74cf7b-l8qrz" podUID="b696b601-e784-4c61-963d-fd05aceb4ac6" containerName="kube-rbac-proxy" containerID="cri-o://41f4ff8d69532697010d821e94d180aa58a984721c2d22bb1c26950a0a463fca" gracePeriod=30
Sep 17 17:45:49.597551 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:45:49.597526    2565 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kserve-ci-e2e-test/error-404-isvc-5b8c1-predictor-6d6d86965-4rplj"]
Sep 17 17:45:49.597786 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:45:49.597749    2565 kuberuntime_container.go:873] "Killing container with a grace period" pod="kserve-ci-e2e-test/error-404-isvc-5b8c1-predictor-6d6d86965-4rplj" podUID="0d0e8cef-9085-4468-9430-9585789557b9" containerName="kserve-container" containerID="cri-o://4a9af9c7433f079ca990c8d71571c7b91e9bff5b52360700f9d0cedc62ff579f" gracePeriod=30
Sep 17 17:45:49.597888 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:45:49.597778    2565 kuberuntime_container.go:873] "Killing container with a grace period" pod="kserve-ci-e2e-test/error-404-isvc-5b8c1-predictor-6d6d86965-4rplj" podUID="0d0e8cef-9085-4468-9430-9585789557b9" containerName="kube-rbac-proxy" containerID="cri-o://d0d790bb8633e1eb29a2d6a0f06ae91e6f8ab868701705617e082ae2b2e6af0d" gracePeriod=30
Sep 17 17:45:49.706673 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:45:49.706639    2565 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kserve-ci-e2e-test/success-200-isvc-2d039-predictor-58dcb9dff4-h64gn"]
Sep 17 17:45:49.713284 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:45:49.713263    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/success-200-isvc-2d039-predictor-58dcb9dff4-h64gn"
Sep 17 17:45:49.716993 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:45:49.716976    2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kserve-ci-e2e-test\"/\"success-200-isvc-2d039-kube-rbac-proxy-sar-config\""
Sep 17 17:45:49.717101 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:45:49.717053    2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kserve-ci-e2e-test\"/\"success-200-isvc-2d039-predictor-serving-cert\""
Sep 17 17:45:49.726995 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:45:49.726976    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/success-200-isvc-2d039-predictor-58dcb9dff4-h64gn"]
Sep 17 17:45:49.843924 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:45:49.843859    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/a9696747-1e5f-41ed-9a1a-d6b0d76c6a16-proxy-tls\") pod \"success-200-isvc-2d039-predictor-58dcb9dff4-h64gn\" (UID: \"a9696747-1e5f-41ed-9a1a-d6b0d76c6a16\") " pod="kserve-ci-e2e-test/success-200-isvc-2d039-predictor-58dcb9dff4-h64gn"
Sep 17 17:45:49.843924 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:45:49.843891    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/a9696747-1e5f-41ed-9a1a-d6b0d76c6a16-kube-rbac-proxy-sar-config\") pod \"success-200-isvc-2d039-predictor-58dcb9dff4-h64gn\" (UID: \"a9696747-1e5f-41ed-9a1a-d6b0d76c6a16\") " pod="kserve-ci-e2e-test/success-200-isvc-2d039-predictor-58dcb9dff4-h64gn"
Sep 17 17:45:49.844130 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:45:49.843935    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-krkwp\" (UniqueName: \"kubernetes.io/projected/a9696747-1e5f-41ed-9a1a-d6b0d76c6a16-kube-api-access-krkwp\") pod \"success-200-isvc-2d039-predictor-58dcb9dff4-h64gn\" (UID: \"a9696747-1e5f-41ed-9a1a-d6b0d76c6a16\") " pod="kserve-ci-e2e-test/success-200-isvc-2d039-predictor-58dcb9dff4-h64gn"
Sep 17 17:45:49.907876 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:45:49.907845    2565 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kserve-ci-e2e-test/error-404-isvc-2d039-predictor-c5df97d75-lhdfb"]
Sep 17 17:45:49.911477 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:45:49.911452    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/error-404-isvc-2d039-predictor-c5df97d75-lhdfb"
Sep 17 17:45:49.915371 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:45:49.915352    2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kserve-ci-e2e-test\"/\"error-404-isvc-2d039-predictor-serving-cert\""
Sep 17 17:45:49.915550 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:45:49.915382    2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kserve-ci-e2e-test\"/\"error-404-isvc-2d039-kube-rbac-proxy-sar-config\""
Sep 17 17:45:49.924637 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:45:49.924616    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/error-404-isvc-2d039-predictor-c5df97d75-lhdfb"]
Sep 17 17:45:49.945096 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:45:49.945046    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/a9696747-1e5f-41ed-9a1a-d6b0d76c6a16-proxy-tls\") pod \"success-200-isvc-2d039-predictor-58dcb9dff4-h64gn\" (UID: \"a9696747-1e5f-41ed-9a1a-d6b0d76c6a16\") " pod="kserve-ci-e2e-test/success-200-isvc-2d039-predictor-58dcb9dff4-h64gn"
Sep 17 17:45:49.945216 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:45:49.945108    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/a9696747-1e5f-41ed-9a1a-d6b0d76c6a16-kube-rbac-proxy-sar-config\") pod \"success-200-isvc-2d039-predictor-58dcb9dff4-h64gn\" (UID: \"a9696747-1e5f-41ed-9a1a-d6b0d76c6a16\") " pod="kserve-ci-e2e-test/success-200-isvc-2d039-predictor-58dcb9dff4-h64gn"
Sep 17 17:45:49.945216 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:45:49.945160    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-krkwp\" (UniqueName: \"kubernetes.io/projected/a9696747-1e5f-41ed-9a1a-d6b0d76c6a16-kube-api-access-krkwp\") pod \"success-200-isvc-2d039-predictor-58dcb9dff4-h64gn\" (UID: \"a9696747-1e5f-41ed-9a1a-d6b0d76c6a16\") " pod="kserve-ci-e2e-test/success-200-isvc-2d039-predictor-58dcb9dff4-h64gn"
Sep 17 17:45:49.945870 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:45:49.945850    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/a9696747-1e5f-41ed-9a1a-d6b0d76c6a16-kube-rbac-proxy-sar-config\") pod \"success-200-isvc-2d039-predictor-58dcb9dff4-h64gn\" (UID: \"a9696747-1e5f-41ed-9a1a-d6b0d76c6a16\") " pod="kserve-ci-e2e-test/success-200-isvc-2d039-predictor-58dcb9dff4-h64gn"
Sep 17 17:45:49.948444 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:45:49.948336    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/a9696747-1e5f-41ed-9a1a-d6b0d76c6a16-proxy-tls\") pod \"success-200-isvc-2d039-predictor-58dcb9dff4-h64gn\" (UID: \"a9696747-1e5f-41ed-9a1a-d6b0d76c6a16\") " pod="kserve-ci-e2e-test/success-200-isvc-2d039-predictor-58dcb9dff4-h64gn"
Sep 17 17:45:49.955295 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:45:49.955271    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-krkwp\" (UniqueName: \"kubernetes.io/projected/a9696747-1e5f-41ed-9a1a-d6b0d76c6a16-kube-api-access-krkwp\") pod \"success-200-isvc-2d039-predictor-58dcb9dff4-h64gn\" (UID: \"a9696747-1e5f-41ed-9a1a-d6b0d76c6a16\") " pod="kserve-ci-e2e-test/success-200-isvc-2d039-predictor-58dcb9dff4-h64gn"
Sep 17 17:45:50.023675 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:45:50.023653    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/success-200-isvc-2d039-predictor-58dcb9dff4-h64gn"
Sep 17 17:45:50.045925 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:45:50.045899    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-48k7q\" (UniqueName: \"kubernetes.io/projected/2b144984-69c6-4be8-abfb-d9533de0a0e5-kube-api-access-48k7q\") pod \"error-404-isvc-2d039-predictor-c5df97d75-lhdfb\" (UID: \"2b144984-69c6-4be8-abfb-d9533de0a0e5\") " pod="kserve-ci-e2e-test/error-404-isvc-2d039-predictor-c5df97d75-lhdfb"
Sep 17 17:45:50.046071 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:45:50.045937    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/2b144984-69c6-4be8-abfb-d9533de0a0e5-kube-rbac-proxy-sar-config\") pod \"error-404-isvc-2d039-predictor-c5df97d75-lhdfb\" (UID: \"2b144984-69c6-4be8-abfb-d9533de0a0e5\") " pod="kserve-ci-e2e-test/error-404-isvc-2d039-predictor-c5df97d75-lhdfb"
Sep 17 17:45:50.046071 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:45:50.045982    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/2b144984-69c6-4be8-abfb-d9533de0a0e5-proxy-tls\") pod \"error-404-isvc-2d039-predictor-c5df97d75-lhdfb\" (UID: \"2b144984-69c6-4be8-abfb-d9533de0a0e5\") " pod="kserve-ci-e2e-test/error-404-isvc-2d039-predictor-c5df97d75-lhdfb"
Sep 17 17:45:50.143968 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:45:50.143942    2565 generic.go:356] "Generic (PLEG): container finished" podID="b696b601-e784-4c61-963d-fd05aceb4ac6" containerID="41f4ff8d69532697010d821e94d180aa58a984721c2d22bb1c26950a0a463fca" exitCode=2
Sep 17 17:45:50.144112 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:45:50.144011    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/success-200-isvc-5b8c1-predictor-c5f74cf7b-l8qrz" event={"ID":"b696b601-e784-4c61-963d-fd05aceb4ac6","Type":"ContainerDied","Data":"41f4ff8d69532697010d821e94d180aa58a984721c2d22bb1c26950a0a463fca"}
Sep 17 17:45:50.145907 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:45:50.145880    2565 generic.go:356] "Generic (PLEG): container finished" podID="0d0e8cef-9085-4468-9430-9585789557b9" containerID="d0d790bb8633e1eb29a2d6a0f06ae91e6f8ab868701705617e082ae2b2e6af0d" exitCode=2
Sep 17 17:45:50.146052 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:45:50.145953    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/error-404-isvc-5b8c1-predictor-6d6d86965-4rplj" event={"ID":"0d0e8cef-9085-4468-9430-9585789557b9","Type":"ContainerDied","Data":"d0d790bb8633e1eb29a2d6a0f06ae91e6f8ab868701705617e082ae2b2e6af0d"}
Sep 17 17:45:50.152632 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:45:50.147847    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-48k7q\" (UniqueName: \"kubernetes.io/projected/2b144984-69c6-4be8-abfb-d9533de0a0e5-kube-api-access-48k7q\") pod \"error-404-isvc-2d039-predictor-c5df97d75-lhdfb\" (UID: \"2b144984-69c6-4be8-abfb-d9533de0a0e5\") " pod="kserve-ci-e2e-test/error-404-isvc-2d039-predictor-c5df97d75-lhdfb"
Sep 17 17:45:50.152632 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:45:50.147898    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/2b144984-69c6-4be8-abfb-d9533de0a0e5-kube-rbac-proxy-sar-config\") pod \"error-404-isvc-2d039-predictor-c5df97d75-lhdfb\" (UID: \"2b144984-69c6-4be8-abfb-d9533de0a0e5\") " pod="kserve-ci-e2e-test/error-404-isvc-2d039-predictor-c5df97d75-lhdfb"
Sep 17 17:45:50.152632 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:45:50.147958    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/2b144984-69c6-4be8-abfb-d9533de0a0e5-proxy-tls\") pod \"error-404-isvc-2d039-predictor-c5df97d75-lhdfb\" (UID: \"2b144984-69c6-4be8-abfb-d9533de0a0e5\") " pod="kserve-ci-e2e-test/error-404-isvc-2d039-predictor-c5df97d75-lhdfb"
Sep 17 17:45:50.152632 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:45:50.149546    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/2b144984-69c6-4be8-abfb-d9533de0a0e5-kube-rbac-proxy-sar-config\") pod \"error-404-isvc-2d039-predictor-c5df97d75-lhdfb\" (UID: \"2b144984-69c6-4be8-abfb-d9533de0a0e5\") " pod="kserve-ci-e2e-test/error-404-isvc-2d039-predictor-c5df97d75-lhdfb"
Sep 17 17:45:50.157031 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:45:50.157000    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/2b144984-69c6-4be8-abfb-d9533de0a0e5-proxy-tls\") pod \"error-404-isvc-2d039-predictor-c5df97d75-lhdfb\" (UID: \"2b144984-69c6-4be8-abfb-d9533de0a0e5\") " pod="kserve-ci-e2e-test/error-404-isvc-2d039-predictor-c5df97d75-lhdfb"
Sep 17 17:45:50.161330 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:45:50.161312    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-48k7q\" (UniqueName: \"kubernetes.io/projected/2b144984-69c6-4be8-abfb-d9533de0a0e5-kube-api-access-48k7q\") pod \"error-404-isvc-2d039-predictor-c5df97d75-lhdfb\" (UID: \"2b144984-69c6-4be8-abfb-d9533de0a0e5\") " pod="kserve-ci-e2e-test/error-404-isvc-2d039-predictor-c5df97d75-lhdfb"
Sep 17 17:45:50.221717 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:45:50.221696    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/error-404-isvc-2d039-predictor-c5df97d75-lhdfb"
Sep 17 17:45:50.300497 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:45:50.300468    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/success-200-isvc-2d039-predictor-58dcb9dff4-h64gn"]
Sep 17 17:45:50.303931 ip-10-0-137-20 kubenswrapper[2565]: W0917 17:45:50.303903    2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda9696747_1e5f_41ed_9a1a_d6b0d76c6a16.slice/crio-c14688ddb4985cc1dae4b5699d0a8f345284f274ab4c7e17ac118fdbf517aaa8 WatchSource:0}: Error finding container c14688ddb4985cc1dae4b5699d0a8f345284f274ab4c7e17ac118fdbf517aaa8: Status 404 returned error can't find the container with id c14688ddb4985cc1dae4b5699d0a8f345284f274ab4c7e17ac118fdbf517aaa8
Sep 17 17:45:50.499940 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:45:50.499911    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/error-404-isvc-2d039-predictor-c5df97d75-lhdfb"]
Sep 17 17:45:50.501694 ip-10-0-137-20 kubenswrapper[2565]: W0917 17:45:50.501670    2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2b144984_69c6_4be8_abfb_d9533de0a0e5.slice/crio-f37404a79d79279208f0a3736d600a842de47a40e786168847609ee0bfc821f1 WatchSource:0}: Error finding container f37404a79d79279208f0a3736d600a842de47a40e786168847609ee0bfc821f1: Status 404 returned error can't find the container with id f37404a79d79279208f0a3736d600a842de47a40e786168847609ee0bfc821f1
Sep 17 17:45:51.153827 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:45:51.153787    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/success-200-isvc-2d039-predictor-58dcb9dff4-h64gn" event={"ID":"a9696747-1e5f-41ed-9a1a-d6b0d76c6a16","Type":"ContainerStarted","Data":"f18c05bd8554869508f2e518620243aca718c82a6e1773b54b2bb23c65d0b9b5"}
Sep 17 17:45:51.154312 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:45:51.153867    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/success-200-isvc-2d039-predictor-58dcb9dff4-h64gn" event={"ID":"a9696747-1e5f-41ed-9a1a-d6b0d76c6a16","Type":"ContainerStarted","Data":"d709c4909ec8efbf8c92ff5600bbdd9e7264c6aa854b2eedec0be9db942e87b2"}
Sep 17 17:45:51.154312 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:45:51.154062    2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve-ci-e2e-test/success-200-isvc-2d039-predictor-58dcb9dff4-h64gn"
Sep 17 17:45:51.154312 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:45:51.154118    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/success-200-isvc-2d039-predictor-58dcb9dff4-h64gn" event={"ID":"a9696747-1e5f-41ed-9a1a-d6b0d76c6a16","Type":"ContainerStarted","Data":"c14688ddb4985cc1dae4b5699d0a8f345284f274ab4c7e17ac118fdbf517aaa8"}
Sep 17 17:45:51.156004 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:45:51.155980    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/error-404-isvc-2d039-predictor-c5df97d75-lhdfb" event={"ID":"2b144984-69c6-4be8-abfb-d9533de0a0e5","Type":"ContainerStarted","Data":"623b5b2fab281a4a759646774307275c977afcc60baedeeb3847341aea90e6bf"}
Sep 17 17:45:51.156155 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:45:51.156047    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/error-404-isvc-2d039-predictor-c5df97d75-lhdfb" event={"ID":"2b144984-69c6-4be8-abfb-d9533de0a0e5","Type":"ContainerStarted","Data":"1fdd1327e2a20ddd23067b9eff553251e89e52b2a956253c59bbd0361f2af9eb"}
Sep 17 17:45:51.156155 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:45:51.156079    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/error-404-isvc-2d039-predictor-c5df97d75-lhdfb" event={"ID":"2b144984-69c6-4be8-abfb-d9533de0a0e5","Type":"ContainerStarted","Data":"f37404a79d79279208f0a3736d600a842de47a40e786168847609ee0bfc821f1"}
Sep 17 17:45:51.156155 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:45:51.156111    2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve-ci-e2e-test/error-404-isvc-2d039-predictor-c5df97d75-lhdfb"
Sep 17 17:45:51.178054 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:45:51.178001    2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kserve-ci-e2e-test/success-200-isvc-2d039-predictor-58dcb9dff4-h64gn" podStartSLOduration=2.1779860380000002 podStartE2EDuration="2.177986038s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-17 17:45:49 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-17 17:45:51.177348764 +0000 UTC m=+701.054633178" watchObservedRunningTime="2026-09-17 17:45:51.177986038 +0000 UTC m=+701.055270451"
Sep 17 17:45:51.199750 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:45:51.199708    2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kserve-ci-e2e-test/error-404-isvc-2d039-predictor-c5df97d75-lhdfb" podStartSLOduration=2.199695405 podStartE2EDuration="2.199695405s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-17 17:45:49 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-17 17:45:51.198078674 +0000 UTC m=+701.075363088" watchObservedRunningTime="2026-09-17 17:45:51.199695405 +0000 UTC m=+701.076979820"
Sep 17 17:45:51.854757 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:45:51.854718    2565 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/error-404-isvc-5b8c1-predictor-6d6d86965-4rplj" podUID="0d0e8cef-9085-4468-9430-9585789557b9" containerName="kube-rbac-proxy" probeResult="failure" output="Get \"https://10.133.0.33:8643/healthz\": dial tcp 10.133.0.33:8643: connect: connection refused"
Sep 17 17:45:52.160312 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:45:52.160218    2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve-ci-e2e-test/error-404-isvc-2d039-predictor-c5df97d75-lhdfb"
Sep 17 17:45:52.160741 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:45:52.160461    2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve-ci-e2e-test/success-200-isvc-2d039-predictor-58dcb9dff4-h64gn"
Sep 17 17:45:52.161570 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:45:52.161544    2565 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/error-404-isvc-2d039-predictor-c5df97d75-lhdfb" podUID="2b144984-69c6-4be8-abfb-d9533de0a0e5" containerName="kserve-container" probeResult="failure" output="dial tcp 10.133.0.35:8080: connect: connection refused"
Sep 17 17:45:52.161649 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:45:52.161544    2565 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/success-200-isvc-2d039-predictor-58dcb9dff4-h64gn" podUID="a9696747-1e5f-41ed-9a1a-d6b0d76c6a16" containerName="kserve-container" probeResult="failure" output="dial tcp 10.133.0.34:8080: connect: connection refused"
Sep 17 17:45:52.585087 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:45:52.585065    2565 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/success-200-isvc-5b8c1-predictor-c5f74cf7b-l8qrz"
Sep 17 17:45:52.768335 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:45:52.768258    2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/b696b601-e784-4c61-963d-fd05aceb4ac6-proxy-tls\") pod \"b696b601-e784-4c61-963d-fd05aceb4ac6\" (UID: \"b696b601-e784-4c61-963d-fd05aceb4ac6\") "
Sep 17 17:45:52.768335 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:45:52.768298    2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/b696b601-e784-4c61-963d-fd05aceb4ac6-kube-rbac-proxy-sar-config\") pod \"b696b601-e784-4c61-963d-fd05aceb4ac6\" (UID: \"b696b601-e784-4c61-963d-fd05aceb4ac6\") "
Sep 17 17:45:52.768556 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:45:52.768380    2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-75q2d\" (UniqueName: \"kubernetes.io/projected/b696b601-e784-4c61-963d-fd05aceb4ac6-kube-api-access-75q2d\") pod \"b696b601-e784-4c61-963d-fd05aceb4ac6\" (UID: \"b696b601-e784-4c61-963d-fd05aceb4ac6\") "
Sep 17 17:45:52.768707 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:45:52.768683    2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/b696b601-e784-4c61-963d-fd05aceb4ac6-kube-rbac-proxy-sar-config" (OuterVolumeSpecName: "kube-rbac-proxy-sar-config") pod "b696b601-e784-4c61-963d-fd05aceb4ac6" (UID: "b696b601-e784-4c61-963d-fd05aceb4ac6"). InnerVolumeSpecName "kube-rbac-proxy-sar-config". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 17 17:45:52.770494 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:45:52.770472    2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/b696b601-e784-4c61-963d-fd05aceb4ac6-kube-api-access-75q2d" (OuterVolumeSpecName: "kube-api-access-75q2d") pod "b696b601-e784-4c61-963d-fd05aceb4ac6" (UID: "b696b601-e784-4c61-963d-fd05aceb4ac6"). InnerVolumeSpecName "kube-api-access-75q2d". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 17 17:45:52.770494 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:45:52.770480    2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/b696b601-e784-4c61-963d-fd05aceb4ac6-proxy-tls" (OuterVolumeSpecName: "proxy-tls") pod "b696b601-e784-4c61-963d-fd05aceb4ac6" (UID: "b696b601-e784-4c61-963d-fd05aceb4ac6"). InnerVolumeSpecName "proxy-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 17 17:45:52.869778 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:45:52.869746    2565 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-75q2d\" (UniqueName: \"kubernetes.io/projected/b696b601-e784-4c61-963d-fd05aceb4ac6-kube-api-access-75q2d\") on node \"ip-10-0-137-20.ec2.internal\" DevicePath \"\""
Sep 17 17:45:52.869778 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:45:52.869772    2565 reconciler_common.go:299] "Volume detached for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/b696b601-e784-4c61-963d-fd05aceb4ac6-proxy-tls\") on node \"ip-10-0-137-20.ec2.internal\" DevicePath \"\""
Sep 17 17:45:52.869981 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:45:52.869786    2565 reconciler_common.go:299] "Volume detached for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/b696b601-e784-4c61-963d-fd05aceb4ac6-kube-rbac-proxy-sar-config\") on node \"ip-10-0-137-20.ec2.internal\" DevicePath \"\""
Sep 17 17:45:53.131353 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:45:53.131329    2565 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/error-404-isvc-5b8c1-predictor-6d6d86965-4rplj"
Sep 17 17:45:53.171915 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:45:53.171878    2565 generic.go:356] "Generic (PLEG): container finished" podID="b696b601-e784-4c61-963d-fd05aceb4ac6" containerID="771d34b9b701a5ab1f8e944403c3245d4e1dd00016a61e91b94e5585811eb4d2" exitCode=0
Sep 17 17:45:53.172316 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:45:53.171931    2565 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/success-200-isvc-5b8c1-predictor-c5f74cf7b-l8qrz"
Sep 17 17:45:53.172316 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:45:53.171961    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/success-200-isvc-5b8c1-predictor-c5f74cf7b-l8qrz" event={"ID":"b696b601-e784-4c61-963d-fd05aceb4ac6","Type":"ContainerDied","Data":"771d34b9b701a5ab1f8e944403c3245d4e1dd00016a61e91b94e5585811eb4d2"}
Sep 17 17:45:53.172316 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:45:53.172073    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/success-200-isvc-5b8c1-predictor-c5f74cf7b-l8qrz" event={"ID":"b696b601-e784-4c61-963d-fd05aceb4ac6","Type":"ContainerDied","Data":"7c16dae88e8051ea3ca533ea5325c225561aee5f867900b71cd1a026eecb9e9a"}
Sep 17 17:45:53.172316 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:45:53.172096    2565 scope.go:117] "RemoveContainer" containerID="41f4ff8d69532697010d821e94d180aa58a984721c2d22bb1c26950a0a463fca"
Sep 17 17:45:53.174038 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:45:53.173831    2565 generic.go:356] "Generic (PLEG): container finished" podID="0d0e8cef-9085-4468-9430-9585789557b9" containerID="4a9af9c7433f079ca990c8d71571c7b91e9bff5b52360700f9d0cedc62ff579f" exitCode=0
Sep 17 17:45:53.174038 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:45:53.173907    2565 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/error-404-isvc-5b8c1-predictor-6d6d86965-4rplj"
Sep 17 17:45:53.174038 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:45:53.173958    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/error-404-isvc-5b8c1-predictor-6d6d86965-4rplj" event={"ID":"0d0e8cef-9085-4468-9430-9585789557b9","Type":"ContainerDied","Data":"4a9af9c7433f079ca990c8d71571c7b91e9bff5b52360700f9d0cedc62ff579f"}
Sep 17 17:45:53.174038 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:45:53.174010    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/error-404-isvc-5b8c1-predictor-6d6d86965-4rplj" event={"ID":"0d0e8cef-9085-4468-9430-9585789557b9","Type":"ContainerDied","Data":"025c40fe52983c057f73aa51eade2072bab75453da72bd0b611c32b6b3a26eec"}
Sep 17 17:45:53.174494 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:45:53.174468    2565 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/error-404-isvc-2d039-predictor-c5df97d75-lhdfb" podUID="2b144984-69c6-4be8-abfb-d9533de0a0e5" containerName="kserve-container" probeResult="failure" output="dial tcp 10.133.0.35:8080: connect: connection refused"
Sep 17 17:45:53.174652 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:45:53.174625    2565 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/success-200-isvc-2d039-predictor-58dcb9dff4-h64gn" podUID="a9696747-1e5f-41ed-9a1a-d6b0d76c6a16" containerName="kserve-container" probeResult="failure" output="dial tcp 10.133.0.34:8080: connect: connection refused"
Sep 17 17:45:53.182268 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:45:53.182251    2565 scope.go:117] "RemoveContainer" containerID="771d34b9b701a5ab1f8e944403c3245d4e1dd00016a61e91b94e5585811eb4d2"
Sep 17 17:45:53.189560 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:45:53.189545    2565 scope.go:117] "RemoveContainer" containerID="41f4ff8d69532697010d821e94d180aa58a984721c2d22bb1c26950a0a463fca"
Sep 17 17:45:53.189791 ip-10-0-137-20 kubenswrapper[2565]: E0917 17:45:53.189776    2565 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"41f4ff8d69532697010d821e94d180aa58a984721c2d22bb1c26950a0a463fca\": container with ID starting with 41f4ff8d69532697010d821e94d180aa58a984721c2d22bb1c26950a0a463fca not found: ID does not exist" containerID="41f4ff8d69532697010d821e94d180aa58a984721c2d22bb1c26950a0a463fca"
Sep 17 17:45:53.189834 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:45:53.189797    2565 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"41f4ff8d69532697010d821e94d180aa58a984721c2d22bb1c26950a0a463fca"} err="failed to get container status \"41f4ff8d69532697010d821e94d180aa58a984721c2d22bb1c26950a0a463fca\": rpc error: code = NotFound desc = could not find container \"41f4ff8d69532697010d821e94d180aa58a984721c2d22bb1c26950a0a463fca\": container with ID starting with 41f4ff8d69532697010d821e94d180aa58a984721c2d22bb1c26950a0a463fca not found: ID does not exist"
Sep 17 17:45:53.189834 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:45:53.189810    2565 scope.go:117] "RemoveContainer" containerID="771d34b9b701a5ab1f8e944403c3245d4e1dd00016a61e91b94e5585811eb4d2"
Sep 17 17:45:53.190010 ip-10-0-137-20 kubenswrapper[2565]: E0917 17:45:53.189996    2565 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"771d34b9b701a5ab1f8e944403c3245d4e1dd00016a61e91b94e5585811eb4d2\": container with ID starting with 771d34b9b701a5ab1f8e944403c3245d4e1dd00016a61e91b94e5585811eb4d2 not found: ID does not exist" containerID="771d34b9b701a5ab1f8e944403c3245d4e1dd00016a61e91b94e5585811eb4d2"
Sep 17 17:45:53.190064 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:45:53.190013    2565 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"771d34b9b701a5ab1f8e944403c3245d4e1dd00016a61e91b94e5585811eb4d2"} err="failed to get container status \"771d34b9b701a5ab1f8e944403c3245d4e1dd00016a61e91b94e5585811eb4d2\": rpc error: code = NotFound desc = could not find container \"771d34b9b701a5ab1f8e944403c3245d4e1dd00016a61e91b94e5585811eb4d2\": container with ID starting with 771d34b9b701a5ab1f8e944403c3245d4e1dd00016a61e91b94e5585811eb4d2 not found: ID does not exist"
Sep 17 17:45:53.190064 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:45:53.190060    2565 scope.go:117] "RemoveContainer" containerID="d0d790bb8633e1eb29a2d6a0f06ae91e6f8ab868701705617e082ae2b2e6af0d"
Sep 17 17:45:53.196751 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:45:53.196731    2565 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kserve-ci-e2e-test/success-200-isvc-5b8c1-predictor-c5f74cf7b-l8qrz"]
Sep 17 17:45:53.199767 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:45:53.198727    2565 scope.go:117] "RemoveContainer" containerID="4a9af9c7433f079ca990c8d71571c7b91e9bff5b52360700f9d0cedc62ff579f"
Sep 17 17:45:53.201502 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:45:53.201480    2565 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kserve-ci-e2e-test/success-200-isvc-5b8c1-predictor-c5f74cf7b-l8qrz"]
Sep 17 17:45:53.206305 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:45:53.206290    2565 scope.go:117] "RemoveContainer" containerID="d0d790bb8633e1eb29a2d6a0f06ae91e6f8ab868701705617e082ae2b2e6af0d"
Sep 17 17:45:53.206536 ip-10-0-137-20 kubenswrapper[2565]: E0917 17:45:53.206520    2565 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"d0d790bb8633e1eb29a2d6a0f06ae91e6f8ab868701705617e082ae2b2e6af0d\": container with ID starting with d0d790bb8633e1eb29a2d6a0f06ae91e6f8ab868701705617e082ae2b2e6af0d not found: ID does not exist" containerID="d0d790bb8633e1eb29a2d6a0f06ae91e6f8ab868701705617e082ae2b2e6af0d"
Sep 17 17:45:53.206581 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:45:53.206541    2565 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d0d790bb8633e1eb29a2d6a0f06ae91e6f8ab868701705617e082ae2b2e6af0d"} err="failed to get container status \"d0d790bb8633e1eb29a2d6a0f06ae91e6f8ab868701705617e082ae2b2e6af0d\": rpc error: code = NotFound desc = could not find container \"d0d790bb8633e1eb29a2d6a0f06ae91e6f8ab868701705617e082ae2b2e6af0d\": container with ID starting with d0d790bb8633e1eb29a2d6a0f06ae91e6f8ab868701705617e082ae2b2e6af0d not found: ID does not exist"
Sep 17 17:45:53.206581 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:45:53.206558    2565 scope.go:117] "RemoveContainer" containerID="4a9af9c7433f079ca990c8d71571c7b91e9bff5b52360700f9d0cedc62ff579f"
Sep 17 17:45:53.206762 ip-10-0-137-20 kubenswrapper[2565]: E0917 17:45:53.206750    2565 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"4a9af9c7433f079ca990c8d71571c7b91e9bff5b52360700f9d0cedc62ff579f\": container with ID starting with 4a9af9c7433f079ca990c8d71571c7b91e9bff5b52360700f9d0cedc62ff579f not found: ID does not exist" containerID="4a9af9c7433f079ca990c8d71571c7b91e9bff5b52360700f9d0cedc62ff579f"
Sep 17 17:45:53.206797 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:45:53.206764    2565 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"4a9af9c7433f079ca990c8d71571c7b91e9bff5b52360700f9d0cedc62ff579f"} err="failed to get container status \"4a9af9c7433f079ca990c8d71571c7b91e9bff5b52360700f9d0cedc62ff579f\": rpc error: code = NotFound desc = could not find container \"4a9af9c7433f079ca990c8d71571c7b91e9bff5b52360700f9d0cedc62ff579f\": container with ID starting with 4a9af9c7433f079ca990c8d71571c7b91e9bff5b52360700f9d0cedc62ff579f not found: ID does not exist"
Sep 17 17:45:53.272475 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:45:53.272453    2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/0d0e8cef-9085-4468-9430-9585789557b9-proxy-tls\") pod \"0d0e8cef-9085-4468-9430-9585789557b9\" (UID: \"0d0e8cef-9085-4468-9430-9585789557b9\") "
Sep 17 17:45:53.272565 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:45:53.272508    2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/0d0e8cef-9085-4468-9430-9585789557b9-kube-rbac-proxy-sar-config\") pod \"0d0e8cef-9085-4468-9430-9585789557b9\" (UID: \"0d0e8cef-9085-4468-9430-9585789557b9\") "
Sep 17 17:45:53.272565 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:45:53.272539    2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-9c7gc\" (UniqueName: \"kubernetes.io/projected/0d0e8cef-9085-4468-9430-9585789557b9-kube-api-access-9c7gc\") pod \"0d0e8cef-9085-4468-9430-9585789557b9\" (UID: \"0d0e8cef-9085-4468-9430-9585789557b9\") "
Sep 17 17:45:53.273074 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:45:53.272894    2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/0d0e8cef-9085-4468-9430-9585789557b9-kube-rbac-proxy-sar-config" (OuterVolumeSpecName: "kube-rbac-proxy-sar-config") pod "0d0e8cef-9085-4468-9430-9585789557b9" (UID: "0d0e8cef-9085-4468-9430-9585789557b9"). InnerVolumeSpecName "kube-rbac-proxy-sar-config". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 17 17:45:53.274490 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:45:53.274464    2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/0d0e8cef-9085-4468-9430-9585789557b9-kube-api-access-9c7gc" (OuterVolumeSpecName: "kube-api-access-9c7gc") pod "0d0e8cef-9085-4468-9430-9585789557b9" (UID: "0d0e8cef-9085-4468-9430-9585789557b9"). InnerVolumeSpecName "kube-api-access-9c7gc". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 17 17:45:53.274564 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:45:53.274536    2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/0d0e8cef-9085-4468-9430-9585789557b9-proxy-tls" (OuterVolumeSpecName: "proxy-tls") pod "0d0e8cef-9085-4468-9430-9585789557b9" (UID: "0d0e8cef-9085-4468-9430-9585789557b9"). InnerVolumeSpecName "proxy-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 17 17:45:53.373558 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:45:53.373533    2565 reconciler_common.go:299] "Volume detached for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/0d0e8cef-9085-4468-9430-9585789557b9-kube-rbac-proxy-sar-config\") on node \"ip-10-0-137-20.ec2.internal\" DevicePath \"\""
Sep 17 17:45:53.373558 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:45:53.373556    2565 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-9c7gc\" (UniqueName: \"kubernetes.io/projected/0d0e8cef-9085-4468-9430-9585789557b9-kube-api-access-9c7gc\") on node \"ip-10-0-137-20.ec2.internal\" DevicePath \"\""
Sep 17 17:45:53.373686 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:45:53.373565    2565 reconciler_common.go:299] "Volume detached for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/0d0e8cef-9085-4468-9430-9585789557b9-proxy-tls\") on node \"ip-10-0-137-20.ec2.internal\" DevicePath \"\""
Sep 17 17:45:53.497485 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:45:53.497457    2565 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kserve-ci-e2e-test/error-404-isvc-5b8c1-predictor-6d6d86965-4rplj"]
Sep 17 17:45:53.501783 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:45:53.501761    2565 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kserve-ci-e2e-test/error-404-isvc-5b8c1-predictor-6d6d86965-4rplj"]
Sep 17 17:45:54.157407 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:45:54.157375    2565 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="0d0e8cef-9085-4468-9430-9585789557b9" path="/var/lib/kubelet/pods/0d0e8cef-9085-4468-9430-9585789557b9/volumes"
Sep 17 17:45:54.157784 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:45:54.157771    2565 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="b696b601-e784-4c61-963d-fd05aceb4ac6" path="/var/lib/kubelet/pods/b696b601-e784-4c61-963d-fd05aceb4ac6/volumes"
Sep 17 17:45:58.180164 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:45:58.180079    2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kserve-ci-e2e-test/error-404-isvc-2d039-predictor-c5df97d75-lhdfb"
Sep 17 17:45:58.180620 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:45:58.180597    2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kserve-ci-e2e-test/success-200-isvc-2d039-predictor-58dcb9dff4-h64gn"
Sep 17 17:45:58.180705 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:45:58.180677    2565 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/error-404-isvc-2d039-predictor-c5df97d75-lhdfb" podUID="2b144984-69c6-4be8-abfb-d9533de0a0e5" containerName="kserve-container" probeResult="failure" output="dial tcp 10.133.0.35:8080: connect: connection refused"
Sep 17 17:45:58.181152 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:45:58.181117    2565 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/success-200-isvc-2d039-predictor-58dcb9dff4-h64gn" podUID="a9696747-1e5f-41ed-9a1a-d6b0d76c6a16" containerName="kserve-container" probeResult="failure" output="dial tcp 10.133.0.34:8080: connect: connection refused"
Sep 17 17:46:08.180895 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:46:08.180860    2565 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/error-404-isvc-2d039-predictor-c5df97d75-lhdfb" podUID="2b144984-69c6-4be8-abfb-d9533de0a0e5" containerName="kserve-container" probeResult="failure" output="dial tcp 10.133.0.35:8080: connect: connection refused"
Sep 17 17:46:08.181279 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:46:08.181170    2565 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/success-200-isvc-2d039-predictor-58dcb9dff4-h64gn" podUID="a9696747-1e5f-41ed-9a1a-d6b0d76c6a16" containerName="kserve-container" probeResult="failure" output="dial tcp 10.133.0.34:8080: connect: connection refused"
Sep 17 17:46:18.180647 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:46:18.180612    2565 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/error-404-isvc-2d039-predictor-c5df97d75-lhdfb" podUID="2b144984-69c6-4be8-abfb-d9533de0a0e5" containerName="kserve-container" probeResult="failure" output="dial tcp 10.133.0.35:8080: connect: connection refused"
Sep 17 17:46:18.181060 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:46:18.180999    2565 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/success-200-isvc-2d039-predictor-58dcb9dff4-h64gn" podUID="a9696747-1e5f-41ed-9a1a-d6b0d76c6a16" containerName="kserve-container" probeResult="failure" output="dial tcp 10.133.0.34:8080: connect: connection refused"
Sep 17 17:46:28.181071 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:46:28.181030    2565 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/success-200-isvc-2d039-predictor-58dcb9dff4-h64gn" podUID="a9696747-1e5f-41ed-9a1a-d6b0d76c6a16" containerName="kserve-container" probeResult="failure" output="dial tcp 10.133.0.34:8080: connect: connection refused"
Sep 17 17:46:28.181528 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:46:28.181070    2565 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/error-404-isvc-2d039-predictor-c5df97d75-lhdfb" podUID="2b144984-69c6-4be8-abfb-d9533de0a0e5" containerName="kserve-container" probeResult="failure" output="dial tcp 10.133.0.35:8080: connect: connection refused"
Sep 17 17:46:38.182096 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:46:38.182069    2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kserve-ci-e2e-test/success-200-isvc-2d039-predictor-58dcb9dff4-h64gn"
Sep 17 17:46:38.182515 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:46:38.182160    2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kserve-ci-e2e-test/error-404-isvc-2d039-predictor-c5df97d75-lhdfb"
Sep 17 17:46:50.158055 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:46:50.158005    2565 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kserve-ci-e2e-test/success-200-isvc-390a8-predictor-6749bbb468-v5k5t"]
Sep 17 17:46:50.158539 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:46:50.158521    2565 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="0d0e8cef-9085-4468-9430-9585789557b9" containerName="kube-rbac-proxy"
Sep 17 17:46:50.158636 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:46:50.158542    2565 state_mem.go:107] "Deleted CPUSet assignment" podUID="0d0e8cef-9085-4468-9430-9585789557b9" containerName="kube-rbac-proxy"
Sep 17 17:46:50.158636 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:46:50.158564    2565 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b696b601-e784-4c61-963d-fd05aceb4ac6" containerName="kube-rbac-proxy"
Sep 17 17:46:50.158636 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:46:50.158573    2565 state_mem.go:107] "Deleted CPUSet assignment" podUID="b696b601-e784-4c61-963d-fd05aceb4ac6" containerName="kube-rbac-proxy"
Sep 17 17:46:50.158636 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:46:50.158585    2565 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b696b601-e784-4c61-963d-fd05aceb4ac6" containerName="kserve-container"
Sep 17 17:46:50.158636 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:46:50.158593    2565 state_mem.go:107] "Deleted CPUSet assignment" podUID="b696b601-e784-4c61-963d-fd05aceb4ac6" containerName="kserve-container"
Sep 17 17:46:50.158636 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:46:50.158621    2565 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="0d0e8cef-9085-4468-9430-9585789557b9" containerName="kserve-container"
Sep 17 17:46:50.158636 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:46:50.158630    2565 state_mem.go:107] "Deleted CPUSet assignment" podUID="0d0e8cef-9085-4468-9430-9585789557b9" containerName="kserve-container"
Sep 17 17:46:50.158939 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:46:50.158720    2565 memory_manager.go:368] "RemoveStaleState removing state" podUID="b696b601-e784-4c61-963d-fd05aceb4ac6" containerName="kube-rbac-proxy"
Sep 17 17:46:50.158939 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:46:50.158735    2565 memory_manager.go:368] "RemoveStaleState removing state" podUID="0d0e8cef-9085-4468-9430-9585789557b9" containerName="kube-rbac-proxy"
Sep 17 17:46:50.158939 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:46:50.158745    2565 memory_manager.go:368] "RemoveStaleState removing state" podUID="0d0e8cef-9085-4468-9430-9585789557b9" containerName="kserve-container"
Sep 17 17:46:50.158939 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:46:50.158756    2565 memory_manager.go:368] "RemoveStaleState removing state" podUID="b696b601-e784-4c61-963d-fd05aceb4ac6" containerName="kserve-container"
Sep 17 17:46:50.161976 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:46:50.161962    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/success-200-isvc-390a8-predictor-6749bbb468-v5k5t"
Sep 17 17:46:50.164881 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:46:50.164858    2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kserve-ci-e2e-test\"/\"success-200-isvc-390a8-kube-rbac-proxy-sar-config\""
Sep 17 17:46:50.164991 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:46:50.164948    2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kserve-ci-e2e-test\"/\"success-200-isvc-390a8-predictor-serving-cert\""
Sep 17 17:46:50.172504 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:46:50.172485    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/success-200-isvc-390a8-predictor-6749bbb468-v5k5t"]
Sep 17 17:46:50.265116 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:46:50.265084    2565 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kserve-ci-e2e-test/error-404-isvc-390a8-predictor-56ccd8877-qp6qj"]
Sep 17 17:46:50.269139 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:46:50.269122    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/error-404-isvc-390a8-predictor-56ccd8877-qp6qj"
Sep 17 17:46:50.272271 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:46:50.272244    2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kserve-ci-e2e-test\"/\"error-404-isvc-390a8-predictor-serving-cert\""
Sep 17 17:46:50.272563 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:46:50.272546    2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kserve-ci-e2e-test\"/\"error-404-isvc-390a8-kube-rbac-proxy-sar-config\""
Sep 17 17:46:50.278427 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:46:50.278408    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/error-404-isvc-390a8-predictor-56ccd8877-qp6qj"]
Sep 17 17:46:50.326724 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:46:50.326696    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-h4rgb\" (UniqueName: \"kubernetes.io/projected/e2fe5e12-9e83-432a-b26b-4cfd217203a7-kube-api-access-h4rgb\") pod \"success-200-isvc-390a8-predictor-6749bbb468-v5k5t\" (UID: \"e2fe5e12-9e83-432a-b26b-4cfd217203a7\") " pod="kserve-ci-e2e-test/success-200-isvc-390a8-predictor-6749bbb468-v5k5t"
Sep 17 17:46:50.326866 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:46:50.326740    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/e2fe5e12-9e83-432a-b26b-4cfd217203a7-kube-rbac-proxy-sar-config\") pod \"success-200-isvc-390a8-predictor-6749bbb468-v5k5t\" (UID: \"e2fe5e12-9e83-432a-b26b-4cfd217203a7\") " pod="kserve-ci-e2e-test/success-200-isvc-390a8-predictor-6749bbb468-v5k5t"
Sep 17 17:46:50.326866 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:46:50.326760    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/e2fe5e12-9e83-432a-b26b-4cfd217203a7-proxy-tls\") pod \"success-200-isvc-390a8-predictor-6749bbb468-v5k5t\" (UID: \"e2fe5e12-9e83-432a-b26b-4cfd217203a7\") " pod="kserve-ci-e2e-test/success-200-isvc-390a8-predictor-6749bbb468-v5k5t"
Sep 17 17:46:50.427274 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:46:50.427194    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-h4rgb\" (UniqueName: \"kubernetes.io/projected/e2fe5e12-9e83-432a-b26b-4cfd217203a7-kube-api-access-h4rgb\") pod \"success-200-isvc-390a8-predictor-6749bbb468-v5k5t\" (UID: \"e2fe5e12-9e83-432a-b26b-4cfd217203a7\") " pod="kserve-ci-e2e-test/success-200-isvc-390a8-predictor-6749bbb468-v5k5t"
Sep 17 17:46:50.427274 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:46:50.427244    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mncv6\" (UniqueName: \"kubernetes.io/projected/f6fd4091-5a8d-4971-b38f-7659800208d0-kube-api-access-mncv6\") pod \"error-404-isvc-390a8-predictor-56ccd8877-qp6qj\" (UID: \"f6fd4091-5a8d-4971-b38f-7659800208d0\") " pod="kserve-ci-e2e-test/error-404-isvc-390a8-predictor-56ccd8877-qp6qj"
Sep 17 17:46:50.427274 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:46:50.427271    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/e2fe5e12-9e83-432a-b26b-4cfd217203a7-kube-rbac-proxy-sar-config\") pod \"success-200-isvc-390a8-predictor-6749bbb468-v5k5t\" (UID: \"e2fe5e12-9e83-432a-b26b-4cfd217203a7\") " pod="kserve-ci-e2e-test/success-200-isvc-390a8-predictor-6749bbb468-v5k5t"
Sep 17 17:46:50.427534 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:46:50.427298    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/e2fe5e12-9e83-432a-b26b-4cfd217203a7-proxy-tls\") pod \"success-200-isvc-390a8-predictor-6749bbb468-v5k5t\" (UID: \"e2fe5e12-9e83-432a-b26b-4cfd217203a7\") " pod="kserve-ci-e2e-test/success-200-isvc-390a8-predictor-6749bbb468-v5k5t"
Sep 17 17:46:50.427534 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:46:50.427346    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/f6fd4091-5a8d-4971-b38f-7659800208d0-kube-rbac-proxy-sar-config\") pod \"error-404-isvc-390a8-predictor-56ccd8877-qp6qj\" (UID: \"f6fd4091-5a8d-4971-b38f-7659800208d0\") " pod="kserve-ci-e2e-test/error-404-isvc-390a8-predictor-56ccd8877-qp6qj"
Sep 17 17:46:50.427534 ip-10-0-137-20 kubenswrapper[2565]: E0917 17:46:50.427371    2565 secret.go:189] Couldn't get secret kserve-ci-e2e-test/success-200-isvc-390a8-predictor-serving-cert: secret "success-200-isvc-390a8-predictor-serving-cert" not found
Sep 17 17:46:50.427534 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:46:50.427411    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/f6fd4091-5a8d-4971-b38f-7659800208d0-proxy-tls\") pod \"error-404-isvc-390a8-predictor-56ccd8877-qp6qj\" (UID: \"f6fd4091-5a8d-4971-b38f-7659800208d0\") " pod="kserve-ci-e2e-test/error-404-isvc-390a8-predictor-56ccd8877-qp6qj"
Sep 17 17:46:50.427534 ip-10-0-137-20 kubenswrapper[2565]: E0917 17:46:50.427426    2565 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/e2fe5e12-9e83-432a-b26b-4cfd217203a7-proxy-tls podName:e2fe5e12-9e83-432a-b26b-4cfd217203a7 nodeName:}" failed. No retries permitted until 2026-09-17 17:46:50.927408392 +0000 UTC m=+760.804692786 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "proxy-tls" (UniqueName: "kubernetes.io/secret/e2fe5e12-9e83-432a-b26b-4cfd217203a7-proxy-tls") pod "success-200-isvc-390a8-predictor-6749bbb468-v5k5t" (UID: "e2fe5e12-9e83-432a-b26b-4cfd217203a7") : secret "success-200-isvc-390a8-predictor-serving-cert" not found
Sep 17 17:46:50.427907 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:46:50.427889    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/e2fe5e12-9e83-432a-b26b-4cfd217203a7-kube-rbac-proxy-sar-config\") pod \"success-200-isvc-390a8-predictor-6749bbb468-v5k5t\" (UID: \"e2fe5e12-9e83-432a-b26b-4cfd217203a7\") " pod="kserve-ci-e2e-test/success-200-isvc-390a8-predictor-6749bbb468-v5k5t"
Sep 17 17:46:50.438899 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:46:50.438880    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-h4rgb\" (UniqueName: \"kubernetes.io/projected/e2fe5e12-9e83-432a-b26b-4cfd217203a7-kube-api-access-h4rgb\") pod \"success-200-isvc-390a8-predictor-6749bbb468-v5k5t\" (UID: \"e2fe5e12-9e83-432a-b26b-4cfd217203a7\") " pod="kserve-ci-e2e-test/success-200-isvc-390a8-predictor-6749bbb468-v5k5t"
Sep 17 17:46:50.528854 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:46:50.528820    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mncv6\" (UniqueName: \"kubernetes.io/projected/f6fd4091-5a8d-4971-b38f-7659800208d0-kube-api-access-mncv6\") pod \"error-404-isvc-390a8-predictor-56ccd8877-qp6qj\" (UID: \"f6fd4091-5a8d-4971-b38f-7659800208d0\") " pod="kserve-ci-e2e-test/error-404-isvc-390a8-predictor-56ccd8877-qp6qj"
Sep 17 17:46:50.529005 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:46:50.528880    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/f6fd4091-5a8d-4971-b38f-7659800208d0-kube-rbac-proxy-sar-config\") pod \"error-404-isvc-390a8-predictor-56ccd8877-qp6qj\" (UID: \"f6fd4091-5a8d-4971-b38f-7659800208d0\") " pod="kserve-ci-e2e-test/error-404-isvc-390a8-predictor-56ccd8877-qp6qj"
Sep 17 17:46:50.529005 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:46:50.528926    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/f6fd4091-5a8d-4971-b38f-7659800208d0-proxy-tls\") pod \"error-404-isvc-390a8-predictor-56ccd8877-qp6qj\" (UID: \"f6fd4091-5a8d-4971-b38f-7659800208d0\") " pod="kserve-ci-e2e-test/error-404-isvc-390a8-predictor-56ccd8877-qp6qj"
Sep 17 17:46:50.529149 ip-10-0-137-20 kubenswrapper[2565]: E0917 17:46:50.529108    2565 secret.go:189] Couldn't get secret kserve-ci-e2e-test/error-404-isvc-390a8-predictor-serving-cert: secret "error-404-isvc-390a8-predictor-serving-cert" not found
Sep 17 17:46:50.529201 ip-10-0-137-20 kubenswrapper[2565]: E0917 17:46:50.529168    2565 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/f6fd4091-5a8d-4971-b38f-7659800208d0-proxy-tls podName:f6fd4091-5a8d-4971-b38f-7659800208d0 nodeName:}" failed. No retries permitted until 2026-09-17 17:46:51.029150742 +0000 UTC m=+760.906435142 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "proxy-tls" (UniqueName: "kubernetes.io/secret/f6fd4091-5a8d-4971-b38f-7659800208d0-proxy-tls") pod "error-404-isvc-390a8-predictor-56ccd8877-qp6qj" (UID: "f6fd4091-5a8d-4971-b38f-7659800208d0") : secret "error-404-isvc-390a8-predictor-serving-cert" not found
Sep 17 17:46:50.529485 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:46:50.529466    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/f6fd4091-5a8d-4971-b38f-7659800208d0-kube-rbac-proxy-sar-config\") pod \"error-404-isvc-390a8-predictor-56ccd8877-qp6qj\" (UID: \"f6fd4091-5a8d-4971-b38f-7659800208d0\") " pod="kserve-ci-e2e-test/error-404-isvc-390a8-predictor-56ccd8877-qp6qj"
Sep 17 17:46:50.541899 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:46:50.541881    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-mncv6\" (UniqueName: \"kubernetes.io/projected/f6fd4091-5a8d-4971-b38f-7659800208d0-kube-api-access-mncv6\") pod \"error-404-isvc-390a8-predictor-56ccd8877-qp6qj\" (UID: \"f6fd4091-5a8d-4971-b38f-7659800208d0\") " pod="kserve-ci-e2e-test/error-404-isvc-390a8-predictor-56ccd8877-qp6qj"
Sep 17 17:46:50.932925 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:46:50.932888    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/e2fe5e12-9e83-432a-b26b-4cfd217203a7-proxy-tls\") pod \"success-200-isvc-390a8-predictor-6749bbb468-v5k5t\" (UID: \"e2fe5e12-9e83-432a-b26b-4cfd217203a7\") " pod="kserve-ci-e2e-test/success-200-isvc-390a8-predictor-6749bbb468-v5k5t"
Sep 17 17:46:50.935289 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:46:50.935270    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/e2fe5e12-9e83-432a-b26b-4cfd217203a7-proxy-tls\") pod \"success-200-isvc-390a8-predictor-6749bbb468-v5k5t\" (UID: \"e2fe5e12-9e83-432a-b26b-4cfd217203a7\") " pod="kserve-ci-e2e-test/success-200-isvc-390a8-predictor-6749bbb468-v5k5t"
Sep 17 17:46:51.033924 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:46:51.033891    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/f6fd4091-5a8d-4971-b38f-7659800208d0-proxy-tls\") pod \"error-404-isvc-390a8-predictor-56ccd8877-qp6qj\" (UID: \"f6fd4091-5a8d-4971-b38f-7659800208d0\") " pod="kserve-ci-e2e-test/error-404-isvc-390a8-predictor-56ccd8877-qp6qj"
Sep 17 17:46:51.034081 ip-10-0-137-20 kubenswrapper[2565]: E0917 17:46:51.034049    2565 secret.go:189] Couldn't get secret kserve-ci-e2e-test/error-404-isvc-390a8-predictor-serving-cert: secret "error-404-isvc-390a8-predictor-serving-cert" not found
Sep 17 17:46:51.034128 ip-10-0-137-20 kubenswrapper[2565]: E0917 17:46:51.034122    2565 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/f6fd4091-5a8d-4971-b38f-7659800208d0-proxy-tls podName:f6fd4091-5a8d-4971-b38f-7659800208d0 nodeName:}" failed. No retries permitted until 2026-09-17 17:46:52.034104169 +0000 UTC m=+761.911388571 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "proxy-tls" (UniqueName: "kubernetes.io/secret/f6fd4091-5a8d-4971-b38f-7659800208d0-proxy-tls") pod "error-404-isvc-390a8-predictor-56ccd8877-qp6qj" (UID: "f6fd4091-5a8d-4971-b38f-7659800208d0") : secret "error-404-isvc-390a8-predictor-serving-cert" not found
Sep 17 17:46:51.072478 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:46:51.072450    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/success-200-isvc-390a8-predictor-6749bbb468-v5k5t"
Sep 17 17:46:51.347301 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:46:51.347276    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/success-200-isvc-390a8-predictor-6749bbb468-v5k5t"]
Sep 17 17:46:51.348916 ip-10-0-137-20 kubenswrapper[2565]: W0917 17:46:51.348889    2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode2fe5e12_9e83_432a_b26b_4cfd217203a7.slice/crio-b7afd2b507c441329580a36022596aff4c23b50b06056aba6c87280e2cb66310 WatchSource:0}: Error finding container b7afd2b507c441329580a36022596aff4c23b50b06056aba6c87280e2cb66310: Status 404 returned error can't find the container with id b7afd2b507c441329580a36022596aff4c23b50b06056aba6c87280e2cb66310
Sep 17 17:46:51.431406 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:46:51.431376    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/success-200-isvc-390a8-predictor-6749bbb468-v5k5t" event={"ID":"e2fe5e12-9e83-432a-b26b-4cfd217203a7","Type":"ContainerStarted","Data":"b9fc73a651f270a9aaebe1b5b29dd9028f2627573e985d3b7198cf83adbe8d1f"}
Sep 17 17:46:51.431504 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:46:51.431433    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/success-200-isvc-390a8-predictor-6749bbb468-v5k5t" event={"ID":"e2fe5e12-9e83-432a-b26b-4cfd217203a7","Type":"ContainerStarted","Data":"b7afd2b507c441329580a36022596aff4c23b50b06056aba6c87280e2cb66310"}
Sep 17 17:46:52.042177 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:46:52.042148    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/f6fd4091-5a8d-4971-b38f-7659800208d0-proxy-tls\") pod \"error-404-isvc-390a8-predictor-56ccd8877-qp6qj\" (UID: \"f6fd4091-5a8d-4971-b38f-7659800208d0\") " pod="kserve-ci-e2e-test/error-404-isvc-390a8-predictor-56ccd8877-qp6qj"
Sep 17 17:46:52.044446 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:46:52.044426    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/f6fd4091-5a8d-4971-b38f-7659800208d0-proxy-tls\") pod \"error-404-isvc-390a8-predictor-56ccd8877-qp6qj\" (UID: \"f6fd4091-5a8d-4971-b38f-7659800208d0\") " pod="kserve-ci-e2e-test/error-404-isvc-390a8-predictor-56ccd8877-qp6qj"
Sep 17 17:46:52.081954 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:46:52.081926    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/error-404-isvc-390a8-predictor-56ccd8877-qp6qj"
Sep 17 17:46:52.352667 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:46:52.352626    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/error-404-isvc-390a8-predictor-56ccd8877-qp6qj"]
Sep 17 17:46:52.354660 ip-10-0-137-20 kubenswrapper[2565]: W0917 17:46:52.354636    2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf6fd4091_5a8d_4971_b38f_7659800208d0.slice/crio-14526a04b2e4bf45696eeb3c50c5c9ab41a3a2a7b8672329cdea916bc2671337 WatchSource:0}: Error finding container 14526a04b2e4bf45696eeb3c50c5c9ab41a3a2a7b8672329cdea916bc2671337: Status 404 returned error can't find the container with id 14526a04b2e4bf45696eeb3c50c5c9ab41a3a2a7b8672329cdea916bc2671337
Sep 17 17:46:52.436965 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:46:52.436943    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/error-404-isvc-390a8-predictor-56ccd8877-qp6qj" event={"ID":"f6fd4091-5a8d-4971-b38f-7659800208d0","Type":"ContainerStarted","Data":"9afe5bb756b942124d852ced2da9f35d09ec18d470c4e656fb5f0cbc343291db"}
Sep 17 17:46:52.437078 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:46:52.436992    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/error-404-isvc-390a8-predictor-56ccd8877-qp6qj" event={"ID":"f6fd4091-5a8d-4971-b38f-7659800208d0","Type":"ContainerStarted","Data":"14526a04b2e4bf45696eeb3c50c5c9ab41a3a2a7b8672329cdea916bc2671337"}
Sep 17 17:46:52.438713 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:46:52.438689    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/success-200-isvc-390a8-predictor-6749bbb468-v5k5t" event={"ID":"e2fe5e12-9e83-432a-b26b-4cfd217203a7","Type":"ContainerStarted","Data":"1d0c74b0a648827ead186c29bf3a3699ec5df5b95be7bd78ea9947e2fb8ed1d0"}
Sep 17 17:46:52.438870 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:46:52.438857    2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve-ci-e2e-test/success-200-isvc-390a8-predictor-6749bbb468-v5k5t"
Sep 17 17:46:52.439050 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:46:52.438990    2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve-ci-e2e-test/success-200-isvc-390a8-predictor-6749bbb468-v5k5t"
Sep 17 17:46:52.440336 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:46:52.440312    2565 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/success-200-isvc-390a8-predictor-6749bbb468-v5k5t" podUID="e2fe5e12-9e83-432a-b26b-4cfd217203a7" containerName="kserve-container" probeResult="failure" output="dial tcp 10.133.0.36:8080: connect: connection refused"
Sep 17 17:46:52.461929 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:46:52.461888    2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kserve-ci-e2e-test/success-200-isvc-390a8-predictor-6749bbb468-v5k5t" podStartSLOduration=2.461874916 podStartE2EDuration="2.461874916s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-17 17:46:50 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-17 17:46:52.460855546 +0000 UTC m=+762.338139962" watchObservedRunningTime="2026-09-17 17:46:52.461874916 +0000 UTC m=+762.339159333"
Sep 17 17:46:53.444551 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:46:53.444509    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/error-404-isvc-390a8-predictor-56ccd8877-qp6qj" event={"ID":"f6fd4091-5a8d-4971-b38f-7659800208d0","Type":"ContainerStarted","Data":"60b0ab5a73d9814d6fa3289a7980b989ced8f24822b576787a7aa7fc21739254"}
Sep 17 17:46:53.445039 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:46:53.444610    2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve-ci-e2e-test/error-404-isvc-390a8-predictor-56ccd8877-qp6qj"
Sep 17 17:46:53.445039 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:46:53.444662    2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve-ci-e2e-test/error-404-isvc-390a8-predictor-56ccd8877-qp6qj"
Sep 17 17:46:53.445039 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:46:53.444790    2565 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/success-200-isvc-390a8-predictor-6749bbb468-v5k5t" podUID="e2fe5e12-9e83-432a-b26b-4cfd217203a7" containerName="kserve-container" probeResult="failure" output="dial tcp 10.133.0.36:8080: connect: connection refused"
Sep 17 17:46:53.445804 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:46:53.445785    2565 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/error-404-isvc-390a8-predictor-56ccd8877-qp6qj" podUID="f6fd4091-5a8d-4971-b38f-7659800208d0" containerName="kserve-container" probeResult="failure" output="dial tcp 10.133.0.37:8080: connect: connection refused"
Sep 17 17:46:53.491705 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:46:53.491662    2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kserve-ci-e2e-test/error-404-isvc-390a8-predictor-56ccd8877-qp6qj" podStartSLOduration=3.491647846 podStartE2EDuration="3.491647846s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-17 17:46:50 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-17 17:46:53.491050686 +0000 UTC m=+763.368335095" watchObservedRunningTime="2026-09-17 17:46:53.491647846 +0000 UTC m=+763.368932260"
Sep 17 17:46:54.449853 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:46:54.449814    2565 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/error-404-isvc-390a8-predictor-56ccd8877-qp6qj" podUID="f6fd4091-5a8d-4971-b38f-7659800208d0" containerName="kserve-container" probeResult="failure" output="dial tcp 10.133.0.37:8080: connect: connection refused"
Sep 17 17:46:58.449454 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:46:58.449424    2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kserve-ci-e2e-test/success-200-isvc-390a8-predictor-6749bbb468-v5k5t"
Sep 17 17:46:58.449929 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:46:58.449902    2565 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/success-200-isvc-390a8-predictor-6749bbb468-v5k5t" podUID="e2fe5e12-9e83-432a-b26b-4cfd217203a7" containerName="kserve-container" probeResult="failure" output="dial tcp 10.133.0.36:8080: connect: connection refused"
Sep 17 17:46:59.456314 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:46:59.456284    2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kserve-ci-e2e-test/error-404-isvc-390a8-predictor-56ccd8877-qp6qj"
Sep 17 17:46:59.456795 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:46:59.456730    2565 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/error-404-isvc-390a8-predictor-56ccd8877-qp6qj" podUID="f6fd4091-5a8d-4971-b38f-7659800208d0" containerName="kserve-container" probeResult="failure" output="dial tcp 10.133.0.37:8080: connect: connection refused"
Sep 17 17:47:08.449994 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:47:08.449953    2565 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/success-200-isvc-390a8-predictor-6749bbb468-v5k5t" podUID="e2fe5e12-9e83-432a-b26b-4cfd217203a7" containerName="kserve-container" probeResult="failure" output="dial tcp 10.133.0.36:8080: connect: connection refused"
Sep 17 17:47:09.457410 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:47:09.457374    2565 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/error-404-isvc-390a8-predictor-56ccd8877-qp6qj" podUID="f6fd4091-5a8d-4971-b38f-7659800208d0" containerName="kserve-container" probeResult="failure" output="dial tcp 10.133.0.37:8080: connect: connection refused"
Sep 17 17:47:18.449948 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:47:18.449902    2565 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/success-200-isvc-390a8-predictor-6749bbb468-v5k5t" podUID="e2fe5e12-9e83-432a-b26b-4cfd217203a7" containerName="kserve-container" probeResult="failure" output="dial tcp 10.133.0.36:8080: connect: connection refused"
Sep 17 17:47:19.456742 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:47:19.456698    2565 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/error-404-isvc-390a8-predictor-56ccd8877-qp6qj" podUID="f6fd4091-5a8d-4971-b38f-7659800208d0" containerName="kserve-container" probeResult="failure" output="dial tcp 10.133.0.37:8080: connect: connection refused"
Sep 17 17:47:28.450910 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:47:28.450808    2565 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/success-200-isvc-390a8-predictor-6749bbb468-v5k5t" podUID="e2fe5e12-9e83-432a-b26b-4cfd217203a7" containerName="kserve-container" probeResult="failure" output="dial tcp 10.133.0.36:8080: connect: connection refused"
Sep 17 17:47:29.456879 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:47:29.456840    2565 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/error-404-isvc-390a8-predictor-56ccd8877-qp6qj" podUID="f6fd4091-5a8d-4971-b38f-7659800208d0" containerName="kserve-container" probeResult="failure" output="dial tcp 10.133.0.37:8080: connect: connection refused"
Sep 17 17:47:38.450616 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:47:38.450588    2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kserve-ci-e2e-test/success-200-isvc-390a8-predictor-6749bbb468-v5k5t"
Sep 17 17:47:39.457776 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:47:39.457746    2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kserve-ci-e2e-test/error-404-isvc-390a8-predictor-56ccd8877-qp6qj"
Sep 17 17:49:12.124642 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:49:12.124572    2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-d9927_30837de7-1447-4de2-ac62-c809813b3338/ovn-acl-logging/0.log"
Sep 17 17:49:12.127058 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:49:12.127035    2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-d9927_30837de7-1447-4de2-ac62-c809813b3338/ovn-acl-logging/0.log"
Sep 17 17:54:12.162863 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:54:12.162831    2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-d9927_30837de7-1447-4de2-ac62-c809813b3338/ovn-acl-logging/0.log"
Sep 17 17:54:12.163413 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:54:12.163062    2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-d9927_30837de7-1447-4de2-ac62-c809813b3338/ovn-acl-logging/0.log"
Sep 17 17:55:04.267469 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:55:04.267387    2565 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kserve-ci-e2e-test/success-200-isvc-2d039-predictor-58dcb9dff4-h64gn"]
Sep 17 17:55:04.268062 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:55:04.267695    2565 kuberuntime_container.go:873] "Killing container with a grace period" pod="kserve-ci-e2e-test/success-200-isvc-2d039-predictor-58dcb9dff4-h64gn" podUID="a9696747-1e5f-41ed-9a1a-d6b0d76c6a16" containerName="kserve-container" containerID="cri-o://d709c4909ec8efbf8c92ff5600bbdd9e7264c6aa854b2eedec0be9db942e87b2" gracePeriod=30
Sep 17 17:55:04.268062 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:55:04.267708    2565 kuberuntime_container.go:873] "Killing container with a grace period" pod="kserve-ci-e2e-test/success-200-isvc-2d039-predictor-58dcb9dff4-h64gn" podUID="a9696747-1e5f-41ed-9a1a-d6b0d76c6a16" containerName="kube-rbac-proxy" containerID="cri-o://f18c05bd8554869508f2e518620243aca718c82a6e1773b54b2bb23c65d0b9b5" gracePeriod=30
Sep 17 17:55:04.345365 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:55:04.345329    2565 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kserve-ci-e2e-test/error-404-isvc-2d039-predictor-c5df97d75-lhdfb"]
Sep 17 17:55:04.345642 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:55:04.345618    2565 kuberuntime_container.go:873] "Killing container with a grace period" pod="kserve-ci-e2e-test/error-404-isvc-2d039-predictor-c5df97d75-lhdfb" podUID="2b144984-69c6-4be8-abfb-d9533de0a0e5" containerName="kserve-container" containerID="cri-o://1fdd1327e2a20ddd23067b9eff553251e89e52b2a956253c59bbd0361f2af9eb" gracePeriod=30
Sep 17 17:55:04.345720 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:55:04.345689    2565 kuberuntime_container.go:873] "Killing container with a grace period" pod="kserve-ci-e2e-test/error-404-isvc-2d039-predictor-c5df97d75-lhdfb" podUID="2b144984-69c6-4be8-abfb-d9533de0a0e5" containerName="kube-rbac-proxy" containerID="cri-o://623b5b2fab281a4a759646774307275c977afcc60baedeeb3847341aea90e6bf" gracePeriod=30
Sep 17 17:55:04.367413 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:55:04.367386    2565 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kserve-ci-e2e-test/success-200-isvc-2b6c1-predictor-65f5775b6-6s9hp"]
Sep 17 17:55:04.371175 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:55:04.371152    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/success-200-isvc-2b6c1-predictor-65f5775b6-6s9hp"
Sep 17 17:55:04.374828 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:55:04.374803    2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kserve-ci-e2e-test\"/\"success-200-isvc-2b6c1-kube-rbac-proxy-sar-config\""
Sep 17 17:55:04.374942 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:55:04.374839    2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kserve-ci-e2e-test\"/\"success-200-isvc-2b6c1-predictor-serving-cert\""
Sep 17 17:55:04.386437 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:55:04.386401    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/success-200-isvc-2b6c1-predictor-65f5775b6-6s9hp"]
Sep 17 17:55:04.468475 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:55:04.468448    2565 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kserve-ci-e2e-test/error-404-isvc-2b6c1-predictor-88cbb7679-pf5xf"]
Sep 17 17:55:04.471892 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:55:04.471875    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/error-404-isvc-2b6c1-predictor-88cbb7679-pf5xf"
Sep 17 17:55:04.474779 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:55:04.474762    2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kserve-ci-e2e-test\"/\"error-404-isvc-2b6c1-predictor-serving-cert\""
Sep 17 17:55:04.474850 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:55:04.474794    2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kserve-ci-e2e-test\"/\"error-404-isvc-2b6c1-kube-rbac-proxy-sar-config\""
Sep 17 17:55:04.481833 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:55:04.481804    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/error-404-isvc-2b6c1-predictor-88cbb7679-pf5xf"]
Sep 17 17:55:04.565758 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:55:04.565690    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4vfw5\" (UniqueName: \"kubernetes.io/projected/64036856-d29a-4195-ba9d-dcd53de9cd4a-kube-api-access-4vfw5\") pod \"success-200-isvc-2b6c1-predictor-65f5775b6-6s9hp\" (UID: \"64036856-d29a-4195-ba9d-dcd53de9cd4a\") " pod="kserve-ci-e2e-test/success-200-isvc-2b6c1-predictor-65f5775b6-6s9hp"
Sep 17 17:55:04.565758 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:55:04.565742    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/64036856-d29a-4195-ba9d-dcd53de9cd4a-kube-rbac-proxy-sar-config\") pod \"success-200-isvc-2b6c1-predictor-65f5775b6-6s9hp\" (UID: \"64036856-d29a-4195-ba9d-dcd53de9cd4a\") " pod="kserve-ci-e2e-test/success-200-isvc-2b6c1-predictor-65f5775b6-6s9hp"
Sep 17 17:55:04.565940 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:55:04.565808    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/64036856-d29a-4195-ba9d-dcd53de9cd4a-proxy-tls\") pod \"success-200-isvc-2b6c1-predictor-65f5775b6-6s9hp\" (UID: \"64036856-d29a-4195-ba9d-dcd53de9cd4a\") " pod="kserve-ci-e2e-test/success-200-isvc-2b6c1-predictor-65f5775b6-6s9hp"
Sep 17 17:55:04.666783 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:55:04.666744    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/a598baa1-44e2-4711-9d08-a245a368b3eb-kube-rbac-proxy-sar-config\") pod \"error-404-isvc-2b6c1-predictor-88cbb7679-pf5xf\" (UID: \"a598baa1-44e2-4711-9d08-a245a368b3eb\") " pod="kserve-ci-e2e-test/error-404-isvc-2b6c1-predictor-88cbb7679-pf5xf"
Sep 17 17:55:04.666956 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:55:04.666795    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4vfw5\" (UniqueName: \"kubernetes.io/projected/64036856-d29a-4195-ba9d-dcd53de9cd4a-kube-api-access-4vfw5\") pod \"success-200-isvc-2b6c1-predictor-65f5775b6-6s9hp\" (UID: \"64036856-d29a-4195-ba9d-dcd53de9cd4a\") " pod="kserve-ci-e2e-test/success-200-isvc-2b6c1-predictor-65f5775b6-6s9hp"
Sep 17 17:55:04.666956 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:55:04.666868    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/a598baa1-44e2-4711-9d08-a245a368b3eb-proxy-tls\") pod \"error-404-isvc-2b6c1-predictor-88cbb7679-pf5xf\" (UID: \"a598baa1-44e2-4711-9d08-a245a368b3eb\") " pod="kserve-ci-e2e-test/error-404-isvc-2b6c1-predictor-88cbb7679-pf5xf"
Sep 17 17:55:04.666956 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:55:04.666903    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/64036856-d29a-4195-ba9d-dcd53de9cd4a-kube-rbac-proxy-sar-config\") pod \"success-200-isvc-2b6c1-predictor-65f5775b6-6s9hp\" (UID: \"64036856-d29a-4195-ba9d-dcd53de9cd4a\") " pod="kserve-ci-e2e-test/success-200-isvc-2b6c1-predictor-65f5775b6-6s9hp"
Sep 17 17:55:04.667174 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:55:04.666994    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dmvbg\" (UniqueName: \"kubernetes.io/projected/a598baa1-44e2-4711-9d08-a245a368b3eb-kube-api-access-dmvbg\") pod \"error-404-isvc-2b6c1-predictor-88cbb7679-pf5xf\" (UID: \"a598baa1-44e2-4711-9d08-a245a368b3eb\") " pod="kserve-ci-e2e-test/error-404-isvc-2b6c1-predictor-88cbb7679-pf5xf"
Sep 17 17:55:04.667174 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:55:04.667050    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/64036856-d29a-4195-ba9d-dcd53de9cd4a-proxy-tls\") pod \"success-200-isvc-2b6c1-predictor-65f5775b6-6s9hp\" (UID: \"64036856-d29a-4195-ba9d-dcd53de9cd4a\") " pod="kserve-ci-e2e-test/success-200-isvc-2b6c1-predictor-65f5775b6-6s9hp"
Sep 17 17:55:04.667763 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:55:04.667740    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/64036856-d29a-4195-ba9d-dcd53de9cd4a-kube-rbac-proxy-sar-config\") pod \"success-200-isvc-2b6c1-predictor-65f5775b6-6s9hp\" (UID: \"64036856-d29a-4195-ba9d-dcd53de9cd4a\") " pod="kserve-ci-e2e-test/success-200-isvc-2b6c1-predictor-65f5775b6-6s9hp"
Sep 17 17:55:04.668663 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:55:04.668637    2565 generic.go:356] "Generic (PLEG): container finished" podID="2b144984-69c6-4be8-abfb-d9533de0a0e5" containerID="623b5b2fab281a4a759646774307275c977afcc60baedeeb3847341aea90e6bf" exitCode=2
Sep 17 17:55:04.668764 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:55:04.668681    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/error-404-isvc-2d039-predictor-c5df97d75-lhdfb" event={"ID":"2b144984-69c6-4be8-abfb-d9533de0a0e5","Type":"ContainerDied","Data":"623b5b2fab281a4a759646774307275c977afcc60baedeeb3847341aea90e6bf"}
Sep 17 17:55:04.669876 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:55:04.669853    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/64036856-d29a-4195-ba9d-dcd53de9cd4a-proxy-tls\") pod \"success-200-isvc-2b6c1-predictor-65f5775b6-6s9hp\" (UID: \"64036856-d29a-4195-ba9d-dcd53de9cd4a\") " pod="kserve-ci-e2e-test/success-200-isvc-2b6c1-predictor-65f5775b6-6s9hp"
Sep 17 17:55:04.670818 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:55:04.670797    2565 generic.go:356] "Generic (PLEG): container finished" podID="a9696747-1e5f-41ed-9a1a-d6b0d76c6a16" containerID="f18c05bd8554869508f2e518620243aca718c82a6e1773b54b2bb23c65d0b9b5" exitCode=2
Sep 17 17:55:04.670910 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:55:04.670830    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/success-200-isvc-2d039-predictor-58dcb9dff4-h64gn" event={"ID":"a9696747-1e5f-41ed-9a1a-d6b0d76c6a16","Type":"ContainerDied","Data":"f18c05bd8554869508f2e518620243aca718c82a6e1773b54b2bb23c65d0b9b5"}
Sep 17 17:55:04.675984 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:55:04.675963    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-4vfw5\" (UniqueName: \"kubernetes.io/projected/64036856-d29a-4195-ba9d-dcd53de9cd4a-kube-api-access-4vfw5\") pod \"success-200-isvc-2b6c1-predictor-65f5775b6-6s9hp\" (UID: \"64036856-d29a-4195-ba9d-dcd53de9cd4a\") " pod="kserve-ci-e2e-test/success-200-isvc-2b6c1-predictor-65f5775b6-6s9hp"
Sep 17 17:55:04.683663 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:55:04.683628    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/success-200-isvc-2b6c1-predictor-65f5775b6-6s9hp"
Sep 17 17:55:04.767713 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:55:04.767682    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dmvbg\" (UniqueName: \"kubernetes.io/projected/a598baa1-44e2-4711-9d08-a245a368b3eb-kube-api-access-dmvbg\") pod \"error-404-isvc-2b6c1-predictor-88cbb7679-pf5xf\" (UID: \"a598baa1-44e2-4711-9d08-a245a368b3eb\") " pod="kserve-ci-e2e-test/error-404-isvc-2b6c1-predictor-88cbb7679-pf5xf"
Sep 17 17:55:04.767875 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:55:04.767740    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/a598baa1-44e2-4711-9d08-a245a368b3eb-kube-rbac-proxy-sar-config\") pod \"error-404-isvc-2b6c1-predictor-88cbb7679-pf5xf\" (UID: \"a598baa1-44e2-4711-9d08-a245a368b3eb\") " pod="kserve-ci-e2e-test/error-404-isvc-2b6c1-predictor-88cbb7679-pf5xf"
Sep 17 17:55:04.767875 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:55:04.767779    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/a598baa1-44e2-4711-9d08-a245a368b3eb-proxy-tls\") pod \"error-404-isvc-2b6c1-predictor-88cbb7679-pf5xf\" (UID: \"a598baa1-44e2-4711-9d08-a245a368b3eb\") " pod="kserve-ci-e2e-test/error-404-isvc-2b6c1-predictor-88cbb7679-pf5xf"
Sep 17 17:55:04.767875 ip-10-0-137-20 kubenswrapper[2565]: E0917 17:55:04.767864    2565 secret.go:189] Couldn't get secret kserve-ci-e2e-test/error-404-isvc-2b6c1-predictor-serving-cert: secret "error-404-isvc-2b6c1-predictor-serving-cert" not found
Sep 17 17:55:04.768070 ip-10-0-137-20 kubenswrapper[2565]: E0917 17:55:04.767912    2565 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a598baa1-44e2-4711-9d08-a245a368b3eb-proxy-tls podName:a598baa1-44e2-4711-9d08-a245a368b3eb nodeName:}" failed. No retries permitted until 2026-09-17 17:55:05.267896601 +0000 UTC m=+1255.145180994 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "proxy-tls" (UniqueName: "kubernetes.io/secret/a598baa1-44e2-4711-9d08-a245a368b3eb-proxy-tls") pod "error-404-isvc-2b6c1-predictor-88cbb7679-pf5xf" (UID: "a598baa1-44e2-4711-9d08-a245a368b3eb") : secret "error-404-isvc-2b6c1-predictor-serving-cert" not found
Sep 17 17:55:04.768461 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:55:04.768435    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/a598baa1-44e2-4711-9d08-a245a368b3eb-kube-rbac-proxy-sar-config\") pod \"error-404-isvc-2b6c1-predictor-88cbb7679-pf5xf\" (UID: \"a598baa1-44e2-4711-9d08-a245a368b3eb\") " pod="kserve-ci-e2e-test/error-404-isvc-2b6c1-predictor-88cbb7679-pf5xf"
Sep 17 17:55:04.777217 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:55:04.777196    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-dmvbg\" (UniqueName: \"kubernetes.io/projected/a598baa1-44e2-4711-9d08-a245a368b3eb-kube-api-access-dmvbg\") pod \"error-404-isvc-2b6c1-predictor-88cbb7679-pf5xf\" (UID: \"a598baa1-44e2-4711-9d08-a245a368b3eb\") " pod="kserve-ci-e2e-test/error-404-isvc-2b6c1-predictor-88cbb7679-pf5xf"
Sep 17 17:55:04.965284 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:55:04.965261    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/success-200-isvc-2b6c1-predictor-65f5775b6-6s9hp"]
Sep 17 17:55:04.966868 ip-10-0-137-20 kubenswrapper[2565]: W0917 17:55:04.966845    2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod64036856_d29a_4195_ba9d_dcd53de9cd4a.slice/crio-cfc4420711f3846f26f747cb1c6cb91b9aa7573a979d6af3f4d556e0a4ac94fc WatchSource:0}: Error finding container cfc4420711f3846f26f747cb1c6cb91b9aa7573a979d6af3f4d556e0a4ac94fc: Status 404 returned error can't find the container with id cfc4420711f3846f26f747cb1c6cb91b9aa7573a979d6af3f4d556e0a4ac94fc
Sep 17 17:55:05.271689 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:55:05.271600    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/a598baa1-44e2-4711-9d08-a245a368b3eb-proxy-tls\") pod \"error-404-isvc-2b6c1-predictor-88cbb7679-pf5xf\" (UID: \"a598baa1-44e2-4711-9d08-a245a368b3eb\") " pod="kserve-ci-e2e-test/error-404-isvc-2b6c1-predictor-88cbb7679-pf5xf"
Sep 17 17:55:05.274041 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:55:05.274005    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/a598baa1-44e2-4711-9d08-a245a368b3eb-proxy-tls\") pod \"error-404-isvc-2b6c1-predictor-88cbb7679-pf5xf\" (UID: \"a598baa1-44e2-4711-9d08-a245a368b3eb\") " pod="kserve-ci-e2e-test/error-404-isvc-2b6c1-predictor-88cbb7679-pf5xf"
Sep 17 17:55:05.383311 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:55:05.383273    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/error-404-isvc-2b6c1-predictor-88cbb7679-pf5xf"
Sep 17 17:55:05.655251 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:55:05.655223    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/error-404-isvc-2b6c1-predictor-88cbb7679-pf5xf"]
Sep 17 17:55:05.657834 ip-10-0-137-20 kubenswrapper[2565]: W0917 17:55:05.657804    2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda598baa1_44e2_4711_9d08_a245a368b3eb.slice/crio-785aca6be3b9afa8864b3923b1c67d4fcfe5d4e63c669317a6d734c56ad340bd WatchSource:0}: Error finding container 785aca6be3b9afa8864b3923b1c67d4fcfe5d4e63c669317a6d734c56ad340bd: Status 404 returned error can't find the container with id 785aca6be3b9afa8864b3923b1c67d4fcfe5d4e63c669317a6d734c56ad340bd
Sep 17 17:55:05.676692 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:55:05.676661    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/error-404-isvc-2b6c1-predictor-88cbb7679-pf5xf" event={"ID":"a598baa1-44e2-4711-9d08-a245a368b3eb","Type":"ContainerStarted","Data":"785aca6be3b9afa8864b3923b1c67d4fcfe5d4e63c669317a6d734c56ad340bd"}
Sep 17 17:55:05.678522 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:55:05.678501    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/success-200-isvc-2b6c1-predictor-65f5775b6-6s9hp" event={"ID":"64036856-d29a-4195-ba9d-dcd53de9cd4a","Type":"ContainerStarted","Data":"7616e9015f3cd8c2f00d6de37759062440da7cc828d3157c789ac9155c143173"}
Sep 17 17:55:05.678647 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:55:05.678546    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/success-200-isvc-2b6c1-predictor-65f5775b6-6s9hp" event={"ID":"64036856-d29a-4195-ba9d-dcd53de9cd4a","Type":"ContainerStarted","Data":"08487acbe04f08fd45b8dc9734ca92228792e4ad5d4e3ebbc3ccf59056354a2f"}
Sep 17 17:55:05.678647 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:55:05.678570    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/success-200-isvc-2b6c1-predictor-65f5775b6-6s9hp" event={"ID":"64036856-d29a-4195-ba9d-dcd53de9cd4a","Type":"ContainerStarted","Data":"cfc4420711f3846f26f747cb1c6cb91b9aa7573a979d6af3f4d556e0a4ac94fc"}
Sep 17 17:55:05.678647 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:55:05.678603    2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve-ci-e2e-test/success-200-isvc-2b6c1-predictor-65f5775b6-6s9hp"
Sep 17 17:55:05.678811 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:55:05.678651    2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve-ci-e2e-test/success-200-isvc-2b6c1-predictor-65f5775b6-6s9hp"
Sep 17 17:55:05.679966 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:55:05.679940    2565 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/success-200-isvc-2b6c1-predictor-65f5775b6-6s9hp" podUID="64036856-d29a-4195-ba9d-dcd53de9cd4a" containerName="kserve-container" probeResult="failure" output="dial tcp 10.133.0.38:8080: connect: connection refused"
Sep 17 17:55:05.698508 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:55:05.698459    2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kserve-ci-e2e-test/success-200-isvc-2b6c1-predictor-65f5775b6-6s9hp" podStartSLOduration=1.698442231 podStartE2EDuration="1.698442231s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-17 17:55:04 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-17 17:55:05.697295973 +0000 UTC m=+1255.574580389" watchObservedRunningTime="2026-09-17 17:55:05.698442231 +0000 UTC m=+1255.575726645"
Sep 17 17:55:06.685947 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:55:06.685912    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/error-404-isvc-2b6c1-predictor-88cbb7679-pf5xf" event={"ID":"a598baa1-44e2-4711-9d08-a245a368b3eb","Type":"ContainerStarted","Data":"695d7e57268e60374c4a1f4931a25a215c3286d7a43da3f545bcf46568b4e08b"}
Sep 17 17:55:06.686449 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:55:06.685977    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/error-404-isvc-2b6c1-predictor-88cbb7679-pf5xf" event={"ID":"a598baa1-44e2-4711-9d08-a245a368b3eb","Type":"ContainerStarted","Data":"9e828ce2f1eb8fef7df6b6ec9eb82df972671b751db9989028a47c20916c1523"}
Sep 17 17:55:06.686449 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:55:06.686236    2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve-ci-e2e-test/error-404-isvc-2b6c1-predictor-88cbb7679-pf5xf"
Sep 17 17:55:06.686449 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:55:06.686365    2565 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/success-200-isvc-2b6c1-predictor-65f5775b6-6s9hp" podUID="64036856-d29a-4195-ba9d-dcd53de9cd4a" containerName="kserve-container" probeResult="failure" output="dial tcp 10.133.0.38:8080: connect: connection refused"
Sep 17 17:55:06.707747 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:55:06.707710    2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kserve-ci-e2e-test/error-404-isvc-2b6c1-predictor-88cbb7679-pf5xf" podStartSLOduration=2.7076981140000003 podStartE2EDuration="2.707698114s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-17 17:55:04 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-17 17:55:06.707264694 +0000 UTC m=+1256.584549111" watchObservedRunningTime="2026-09-17 17:55:06.707698114 +0000 UTC m=+1256.584982529"
Sep 17 17:55:07.694485 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:55:07.694455    2565 generic.go:356] "Generic (PLEG): container finished" podID="a9696747-1e5f-41ed-9a1a-d6b0d76c6a16" containerID="d709c4909ec8efbf8c92ff5600bbdd9e7264c6aa854b2eedec0be9db942e87b2" exitCode=0
Sep 17 17:55:07.694861 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:55:07.694510    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/success-200-isvc-2d039-predictor-58dcb9dff4-h64gn" event={"ID":"a9696747-1e5f-41ed-9a1a-d6b0d76c6a16","Type":"ContainerDied","Data":"d709c4909ec8efbf8c92ff5600bbdd9e7264c6aa854b2eedec0be9db942e87b2"}
Sep 17 17:55:07.696788 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:55:07.696765    2565 generic.go:356] "Generic (PLEG): container finished" podID="2b144984-69c6-4be8-abfb-d9533de0a0e5" containerID="1fdd1327e2a20ddd23067b9eff553251e89e52b2a956253c59bbd0361f2af9eb" exitCode=0
Sep 17 17:55:07.696911 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:55:07.696811    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/error-404-isvc-2d039-predictor-c5df97d75-lhdfb" event={"ID":"2b144984-69c6-4be8-abfb-d9533de0a0e5","Type":"ContainerDied","Data":"1fdd1327e2a20ddd23067b9eff553251e89e52b2a956253c59bbd0361f2af9eb"}
Sep 17 17:55:07.697122 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:55:07.697108    2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve-ci-e2e-test/error-404-isvc-2b6c1-predictor-88cbb7679-pf5xf"
Sep 17 17:55:07.698298 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:55:07.698274    2565 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/error-404-isvc-2b6c1-predictor-88cbb7679-pf5xf" podUID="a598baa1-44e2-4711-9d08-a245a368b3eb" containerName="kserve-container" probeResult="failure" output="dial tcp 10.133.0.39:8080: connect: connection refused"
Sep 17 17:55:07.811735 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:55:07.811713    2565 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/success-200-isvc-2d039-predictor-58dcb9dff4-h64gn"
Sep 17 17:55:07.814465 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:55:07.814447    2565 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/error-404-isvc-2d039-predictor-c5df97d75-lhdfb"
Sep 17 17:55:07.893310 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:55:07.893283    2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/2b144984-69c6-4be8-abfb-d9533de0a0e5-proxy-tls\") pod \"2b144984-69c6-4be8-abfb-d9533de0a0e5\" (UID: \"2b144984-69c6-4be8-abfb-d9533de0a0e5\") "
Sep 17 17:55:07.893484 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:55:07.893324    2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-48k7q\" (UniqueName: \"kubernetes.io/projected/2b144984-69c6-4be8-abfb-d9533de0a0e5-kube-api-access-48k7q\") pod \"2b144984-69c6-4be8-abfb-d9533de0a0e5\" (UID: \"2b144984-69c6-4be8-abfb-d9533de0a0e5\") "
Sep 17 17:55:07.893484 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:55:07.893349    2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/2b144984-69c6-4be8-abfb-d9533de0a0e5-kube-rbac-proxy-sar-config\") pod \"2b144984-69c6-4be8-abfb-d9533de0a0e5\" (UID: \"2b144984-69c6-4be8-abfb-d9533de0a0e5\") "
Sep 17 17:55:07.893484 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:55:07.893369    2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/a9696747-1e5f-41ed-9a1a-d6b0d76c6a16-kube-rbac-proxy-sar-config\") pod \"a9696747-1e5f-41ed-9a1a-d6b0d76c6a16\" (UID: \"a9696747-1e5f-41ed-9a1a-d6b0d76c6a16\") "
Sep 17 17:55:07.893656 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:55:07.893481    2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/a9696747-1e5f-41ed-9a1a-d6b0d76c6a16-proxy-tls\") pod \"a9696747-1e5f-41ed-9a1a-d6b0d76c6a16\" (UID: \"a9696747-1e5f-41ed-9a1a-d6b0d76c6a16\") "
Sep 17 17:55:07.893656 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:55:07.893531    2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-krkwp\" (UniqueName: \"kubernetes.io/projected/a9696747-1e5f-41ed-9a1a-d6b0d76c6a16-kube-api-access-krkwp\") pod \"a9696747-1e5f-41ed-9a1a-d6b0d76c6a16\" (UID: \"a9696747-1e5f-41ed-9a1a-d6b0d76c6a16\") "
Sep 17 17:55:07.893768 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:55:07.893711    2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/a9696747-1e5f-41ed-9a1a-d6b0d76c6a16-kube-rbac-proxy-sar-config" (OuterVolumeSpecName: "kube-rbac-proxy-sar-config") pod "a9696747-1e5f-41ed-9a1a-d6b0d76c6a16" (UID: "a9696747-1e5f-41ed-9a1a-d6b0d76c6a16"). InnerVolumeSpecName "kube-rbac-proxy-sar-config". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 17 17:55:07.893768 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:55:07.893714    2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/2b144984-69c6-4be8-abfb-d9533de0a0e5-kube-rbac-proxy-sar-config" (OuterVolumeSpecName: "kube-rbac-proxy-sar-config") pod "2b144984-69c6-4be8-abfb-d9533de0a0e5" (UID: "2b144984-69c6-4be8-abfb-d9533de0a0e5"). InnerVolumeSpecName "kube-rbac-proxy-sar-config". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 17 17:55:07.893866 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:55:07.893845    2565 reconciler_common.go:299] "Volume detached for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/2b144984-69c6-4be8-abfb-d9533de0a0e5-kube-rbac-proxy-sar-config\") on node \"ip-10-0-137-20.ec2.internal\" DevicePath \"\""
Sep 17 17:55:07.893924 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:55:07.893864    2565 reconciler_common.go:299] "Volume detached for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/a9696747-1e5f-41ed-9a1a-d6b0d76c6a16-kube-rbac-proxy-sar-config\") on node \"ip-10-0-137-20.ec2.internal\" DevicePath \"\""
Sep 17 17:55:07.895590 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:55:07.895557    2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/2b144984-69c6-4be8-abfb-d9533de0a0e5-proxy-tls" (OuterVolumeSpecName: "proxy-tls") pod "2b144984-69c6-4be8-abfb-d9533de0a0e5" (UID: "2b144984-69c6-4be8-abfb-d9533de0a0e5"). InnerVolumeSpecName "proxy-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 17 17:55:07.895590 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:55:07.895579    2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a9696747-1e5f-41ed-9a1a-d6b0d76c6a16-kube-api-access-krkwp" (OuterVolumeSpecName: "kube-api-access-krkwp") pod "a9696747-1e5f-41ed-9a1a-d6b0d76c6a16" (UID: "a9696747-1e5f-41ed-9a1a-d6b0d76c6a16"). InnerVolumeSpecName "kube-api-access-krkwp". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 17 17:55:07.895768 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:55:07.895602    2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/2b144984-69c6-4be8-abfb-d9533de0a0e5-kube-api-access-48k7q" (OuterVolumeSpecName: "kube-api-access-48k7q") pod "2b144984-69c6-4be8-abfb-d9533de0a0e5" (UID: "2b144984-69c6-4be8-abfb-d9533de0a0e5"). InnerVolumeSpecName "kube-api-access-48k7q". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 17 17:55:07.895768 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:55:07.895601    2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/a9696747-1e5f-41ed-9a1a-d6b0d76c6a16-proxy-tls" (OuterVolumeSpecName: "proxy-tls") pod "a9696747-1e5f-41ed-9a1a-d6b0d76c6a16" (UID: "a9696747-1e5f-41ed-9a1a-d6b0d76c6a16"). InnerVolumeSpecName "proxy-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 17 17:55:07.994187 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:55:07.994165    2565 reconciler_common.go:299] "Volume detached for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/a9696747-1e5f-41ed-9a1a-d6b0d76c6a16-proxy-tls\") on node \"ip-10-0-137-20.ec2.internal\" DevicePath \"\""
Sep 17 17:55:07.994187 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:55:07.994184    2565 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-krkwp\" (UniqueName: \"kubernetes.io/projected/a9696747-1e5f-41ed-9a1a-d6b0d76c6a16-kube-api-access-krkwp\") on node \"ip-10-0-137-20.ec2.internal\" DevicePath \"\""
Sep 17 17:55:07.994331 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:55:07.994193    2565 reconciler_common.go:299] "Volume detached for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/2b144984-69c6-4be8-abfb-d9533de0a0e5-proxy-tls\") on node \"ip-10-0-137-20.ec2.internal\" DevicePath \"\""
Sep 17 17:55:07.994331 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:55:07.994202    2565 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-48k7q\" (UniqueName: \"kubernetes.io/projected/2b144984-69c6-4be8-abfb-d9533de0a0e5-kube-api-access-48k7q\") on node \"ip-10-0-137-20.ec2.internal\" DevicePath \"\""
Sep 17 17:55:08.704497 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:55:08.704466    2565 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/error-404-isvc-2d039-predictor-c5df97d75-lhdfb"
Sep 17 17:55:08.704970 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:55:08.704471    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/error-404-isvc-2d039-predictor-c5df97d75-lhdfb" event={"ID":"2b144984-69c6-4be8-abfb-d9533de0a0e5","Type":"ContainerDied","Data":"f37404a79d79279208f0a3736d600a842de47a40e786168847609ee0bfc821f1"}
Sep 17 17:55:08.704970 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:55:08.704640    2565 scope.go:117] "RemoveContainer" containerID="623b5b2fab281a4a759646774307275c977afcc60baedeeb3847341aea90e6bf"
Sep 17 17:55:08.706216 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:55:08.706196    2565 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/success-200-isvc-2d039-predictor-58dcb9dff4-h64gn"
Sep 17 17:55:08.706216 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:55:08.706213    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/success-200-isvc-2d039-predictor-58dcb9dff4-h64gn" event={"ID":"a9696747-1e5f-41ed-9a1a-d6b0d76c6a16","Type":"ContainerDied","Data":"c14688ddb4985cc1dae4b5699d0a8f345284f274ab4c7e17ac118fdbf517aaa8"}
Sep 17 17:55:08.706699 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:55:08.706670    2565 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/error-404-isvc-2b6c1-predictor-88cbb7679-pf5xf" podUID="a598baa1-44e2-4711-9d08-a245a368b3eb" containerName="kserve-container" probeResult="failure" output="dial tcp 10.133.0.39:8080: connect: connection refused"
Sep 17 17:55:08.712985 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:55:08.712966    2565 scope.go:117] "RemoveContainer" containerID="1fdd1327e2a20ddd23067b9eff553251e89e52b2a956253c59bbd0361f2af9eb"
Sep 17 17:55:08.720157 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:55:08.720141    2565 scope.go:117] "RemoveContainer" containerID="f18c05bd8554869508f2e518620243aca718c82a6e1773b54b2bb23c65d0b9b5"
Sep 17 17:55:08.724345 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:55:08.724325    2565 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kserve-ci-e2e-test/success-200-isvc-2d039-predictor-58dcb9dff4-h64gn"]
Sep 17 17:55:08.727426 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:55:08.727410    2565 scope.go:117] "RemoveContainer" containerID="d709c4909ec8efbf8c92ff5600bbdd9e7264c6aa854b2eedec0be9db942e87b2"
Sep 17 17:55:08.730944 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:55:08.730919    2565 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kserve-ci-e2e-test/success-200-isvc-2d039-predictor-58dcb9dff4-h64gn"]
Sep 17 17:55:08.741178 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:55:08.741158    2565 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kserve-ci-e2e-test/error-404-isvc-2d039-predictor-c5df97d75-lhdfb"]
Sep 17 17:55:08.746087 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:55:08.746061    2565 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kserve-ci-e2e-test/error-404-isvc-2d039-predictor-c5df97d75-lhdfb"]
Sep 17 17:55:10.157251 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:55:10.157221    2565 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="2b144984-69c6-4be8-abfb-d9533de0a0e5" path="/var/lib/kubelet/pods/2b144984-69c6-4be8-abfb-d9533de0a0e5/volumes"
Sep 17 17:55:10.157651 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:55:10.157637    2565 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="a9696747-1e5f-41ed-9a1a-d6b0d76c6a16" path="/var/lib/kubelet/pods/a9696747-1e5f-41ed-9a1a-d6b0d76c6a16/volumes"
Sep 17 17:55:11.690498 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:55:11.690470    2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kserve-ci-e2e-test/success-200-isvc-2b6c1-predictor-65f5775b6-6s9hp"
Sep 17 17:55:11.691105 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:55:11.691077    2565 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/success-200-isvc-2b6c1-predictor-65f5775b6-6s9hp" podUID="64036856-d29a-4195-ba9d-dcd53de9cd4a" containerName="kserve-container" probeResult="failure" output="dial tcp 10.133.0.38:8080: connect: connection refused"
Sep 17 17:55:13.711653 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:55:13.711627    2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kserve-ci-e2e-test/error-404-isvc-2b6c1-predictor-88cbb7679-pf5xf"
Sep 17 17:55:13.712064 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:55:13.712015    2565 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/error-404-isvc-2b6c1-predictor-88cbb7679-pf5xf" podUID="a598baa1-44e2-4711-9d08-a245a368b3eb" containerName="kserve-container" probeResult="failure" output="dial tcp 10.133.0.39:8080: connect: connection refused"
Sep 17 17:55:21.692035 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:55:21.691980    2565 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/success-200-isvc-2b6c1-predictor-65f5775b6-6s9hp" podUID="64036856-d29a-4195-ba9d-dcd53de9cd4a" containerName="kserve-container" probeResult="failure" output="dial tcp 10.133.0.38:8080: connect: connection refused"
Sep 17 17:55:23.712694 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:55:23.712658    2565 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/error-404-isvc-2b6c1-predictor-88cbb7679-pf5xf" podUID="a598baa1-44e2-4711-9d08-a245a368b3eb" containerName="kserve-container" probeResult="failure" output="dial tcp 10.133.0.39:8080: connect: connection refused"
Sep 17 17:55:31.691811 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:55:31.691768    2565 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/success-200-isvc-2b6c1-predictor-65f5775b6-6s9hp" podUID="64036856-d29a-4195-ba9d-dcd53de9cd4a" containerName="kserve-container" probeResult="failure" output="dial tcp 10.133.0.38:8080: connect: connection refused"
Sep 17 17:55:33.712851 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:55:33.712809    2565 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/error-404-isvc-2b6c1-predictor-88cbb7679-pf5xf" podUID="a598baa1-44e2-4711-9d08-a245a368b3eb" containerName="kserve-container" probeResult="failure" output="dial tcp 10.133.0.39:8080: connect: connection refused"
Sep 17 17:55:41.692006 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:55:41.691967    2565 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/success-200-isvc-2b6c1-predictor-65f5775b6-6s9hp" podUID="64036856-d29a-4195-ba9d-dcd53de9cd4a" containerName="kserve-container" probeResult="failure" output="dial tcp 10.133.0.38:8080: connect: connection refused"
Sep 17 17:55:43.712058 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:55:43.712002    2565 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/error-404-isvc-2b6c1-predictor-88cbb7679-pf5xf" podUID="a598baa1-44e2-4711-9d08-a245a368b3eb" containerName="kserve-container" probeResult="failure" output="dial tcp 10.133.0.39:8080: connect: connection refused"
Sep 17 17:55:51.692502 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:55:51.692474    2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kserve-ci-e2e-test/success-200-isvc-2b6c1-predictor-65f5775b6-6s9hp"
Sep 17 17:55:53.712521 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:55:53.712491    2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kserve-ci-e2e-test/error-404-isvc-2b6c1-predictor-88cbb7679-pf5xf"
Sep 17 17:56:04.876905 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:56:04.876869    2565 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kserve-ci-e2e-test/success-200-isvc-390a8-predictor-6749bbb468-v5k5t"]
Sep 17 17:56:04.877374 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:56:04.877202    2565 kuberuntime_container.go:873] "Killing container with a grace period" pod="kserve-ci-e2e-test/success-200-isvc-390a8-predictor-6749bbb468-v5k5t" podUID="e2fe5e12-9e83-432a-b26b-4cfd217203a7" containerName="kserve-container" containerID="cri-o://b9fc73a651f270a9aaebe1b5b29dd9028f2627573e985d3b7198cf83adbe8d1f" gracePeriod=30
Sep 17 17:56:04.877374 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:56:04.877254    2565 kuberuntime_container.go:873] "Killing container with a grace period" pod="kserve-ci-e2e-test/success-200-isvc-390a8-predictor-6749bbb468-v5k5t" podUID="e2fe5e12-9e83-432a-b26b-4cfd217203a7" containerName="kube-rbac-proxy" containerID="cri-o://1d0c74b0a648827ead186c29bf3a3699ec5df5b95be7bd78ea9947e2fb8ed1d0" gracePeriod=30
Sep 17 17:56:04.908785 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:56:04.908754    2565 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kserve-ci-e2e-test/error-404-isvc-390a8-predictor-56ccd8877-qp6qj"]
Sep 17 17:56:04.909147 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:56:04.909071    2565 kuberuntime_container.go:873] "Killing container with a grace period" pod="kserve-ci-e2e-test/error-404-isvc-390a8-predictor-56ccd8877-qp6qj" podUID="f6fd4091-5a8d-4971-b38f-7659800208d0" containerName="kserve-container" containerID="cri-o://9afe5bb756b942124d852ced2da9f35d09ec18d470c4e656fb5f0cbc343291db" gracePeriod=30
Sep 17 17:56:04.909147 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:56:04.909098    2565 kuberuntime_container.go:873] "Killing container with a grace period" pod="kserve-ci-e2e-test/error-404-isvc-390a8-predictor-56ccd8877-qp6qj" podUID="f6fd4091-5a8d-4971-b38f-7659800208d0" containerName="kube-rbac-proxy" containerID="cri-o://60b0ab5a73d9814d6fa3289a7980b989ced8f24822b576787a7aa7fc21739254" gracePeriod=30
Sep 17 17:56:04.969345 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:56:04.969313    2565 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kserve-ci-e2e-test/success-200-isvc-6b6c1-predictor-d7d64c7db-ntzbm"]
Sep 17 17:56:04.969688 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:56:04.969676    2565 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="2b144984-69c6-4be8-abfb-d9533de0a0e5" containerName="kserve-container"
Sep 17 17:56:04.969733 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:56:04.969690    2565 state_mem.go:107] "Deleted CPUSet assignment" podUID="2b144984-69c6-4be8-abfb-d9533de0a0e5" containerName="kserve-container"
Sep 17 17:56:04.969733 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:56:04.969708    2565 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a9696747-1e5f-41ed-9a1a-d6b0d76c6a16" containerName="kube-rbac-proxy"
Sep 17 17:56:04.969733 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:56:04.969714    2565 state_mem.go:107] "Deleted CPUSet assignment" podUID="a9696747-1e5f-41ed-9a1a-d6b0d76c6a16" containerName="kube-rbac-proxy"
Sep 17 17:56:04.969733 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:56:04.969726    2565 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="2b144984-69c6-4be8-abfb-d9533de0a0e5" containerName="kube-rbac-proxy"
Sep 17 17:56:04.969733 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:56:04.969731    2565 state_mem.go:107] "Deleted CPUSet assignment" podUID="2b144984-69c6-4be8-abfb-d9533de0a0e5" containerName="kube-rbac-proxy"
Sep 17 17:56:04.969892 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:56:04.969747    2565 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a9696747-1e5f-41ed-9a1a-d6b0d76c6a16" containerName="kserve-container"
Sep 17 17:56:04.969892 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:56:04.969752    2565 state_mem.go:107] "Deleted CPUSet assignment" podUID="a9696747-1e5f-41ed-9a1a-d6b0d76c6a16" containerName="kserve-container"
Sep 17 17:56:04.969892 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:56:04.969801    2565 memory_manager.go:368] "RemoveStaleState removing state" podUID="a9696747-1e5f-41ed-9a1a-d6b0d76c6a16" containerName="kserve-container"
Sep 17 17:56:04.969892 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:56:04.969809    2565 memory_manager.go:368] "RemoveStaleState removing state" podUID="a9696747-1e5f-41ed-9a1a-d6b0d76c6a16" containerName="kube-rbac-proxy"
Sep 17 17:56:04.969892 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:56:04.969818    2565 memory_manager.go:368] "RemoveStaleState removing state" podUID="2b144984-69c6-4be8-abfb-d9533de0a0e5" containerName="kserve-container"
Sep 17 17:56:04.969892 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:56:04.969824    2565 memory_manager.go:368] "RemoveStaleState removing state" podUID="2b144984-69c6-4be8-abfb-d9533de0a0e5" containerName="kube-rbac-proxy"
Sep 17 17:56:04.972922 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:56:04.972906    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/success-200-isvc-6b6c1-predictor-d7d64c7db-ntzbm"
Sep 17 17:56:04.975620 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:56:04.975598    2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kserve-ci-e2e-test\"/\"success-200-isvc-6b6c1-predictor-serving-cert\""
Sep 17 17:56:04.975727 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:56:04.975604    2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kserve-ci-e2e-test\"/\"success-200-isvc-6b6c1-kube-rbac-proxy-sar-config\""
Sep 17 17:56:04.982747 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:56:04.982725    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/success-200-isvc-6b6c1-predictor-d7d64c7db-ntzbm"]
Sep 17 17:56:05.057735 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:56:05.057702    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-lt5tp\" (UniqueName: \"kubernetes.io/projected/2b989a47-1798-48a2-9568-9cb971a3b4a3-kube-api-access-lt5tp\") pod \"success-200-isvc-6b6c1-predictor-d7d64c7db-ntzbm\" (UID: \"2b989a47-1798-48a2-9568-9cb971a3b4a3\") " pod="kserve-ci-e2e-test/success-200-isvc-6b6c1-predictor-d7d64c7db-ntzbm"
Sep 17 17:56:05.057902 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:56:05.057779    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/2b989a47-1798-48a2-9568-9cb971a3b4a3-proxy-tls\") pod \"success-200-isvc-6b6c1-predictor-d7d64c7db-ntzbm\" (UID: \"2b989a47-1798-48a2-9568-9cb971a3b4a3\") " pod="kserve-ci-e2e-test/success-200-isvc-6b6c1-predictor-d7d64c7db-ntzbm"
Sep 17 17:56:05.057902 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:56:05.057860    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/2b989a47-1798-48a2-9568-9cb971a3b4a3-kube-rbac-proxy-sar-config\") pod \"success-200-isvc-6b6c1-predictor-d7d64c7db-ntzbm\" (UID: \"2b989a47-1798-48a2-9568-9cb971a3b4a3\") " pod="kserve-ci-e2e-test/success-200-isvc-6b6c1-predictor-d7d64c7db-ntzbm"
Sep 17 17:56:05.109104 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:56:05.109076    2565 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kserve-ci-e2e-test/error-404-isvc-6b6c1-predictor-66c99c6495-hh86l"]
Sep 17 17:56:05.112604 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:56:05.112584    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/error-404-isvc-6b6c1-predictor-66c99c6495-hh86l"
Sep 17 17:56:05.115293 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:56:05.115272    2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kserve-ci-e2e-test\"/\"error-404-isvc-6b6c1-predictor-serving-cert\""
Sep 17 17:56:05.115394 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:56:05.115318    2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kserve-ci-e2e-test\"/\"error-404-isvc-6b6c1-kube-rbac-proxy-sar-config\""
Sep 17 17:56:05.123769 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:56:05.123745    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/error-404-isvc-6b6c1-predictor-66c99c6495-hh86l"]
Sep 17 17:56:05.159136 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:56:05.159063    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/bd8084fe-ed00-4a05-a0a3-0ab206ba7278-kube-rbac-proxy-sar-config\") pod \"error-404-isvc-6b6c1-predictor-66c99c6495-hh86l\" (UID: \"bd8084fe-ed00-4a05-a0a3-0ab206ba7278\") " pod="kserve-ci-e2e-test/error-404-isvc-6b6c1-predictor-66c99c6495-hh86l"
Sep 17 17:56:05.159136 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:56:05.159098    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lt5tp\" (UniqueName: \"kubernetes.io/projected/2b989a47-1798-48a2-9568-9cb971a3b4a3-kube-api-access-lt5tp\") pod \"success-200-isvc-6b6c1-predictor-d7d64c7db-ntzbm\" (UID: \"2b989a47-1798-48a2-9568-9cb971a3b4a3\") " pod="kserve-ci-e2e-test/success-200-isvc-6b6c1-predictor-d7d64c7db-ntzbm"
Sep 17 17:56:05.159136 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:56:05.159123    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/bd8084fe-ed00-4a05-a0a3-0ab206ba7278-proxy-tls\") pod \"error-404-isvc-6b6c1-predictor-66c99c6495-hh86l\" (UID: \"bd8084fe-ed00-4a05-a0a3-0ab206ba7278\") " pod="kserve-ci-e2e-test/error-404-isvc-6b6c1-predictor-66c99c6495-hh86l"
Sep 17 17:56:05.159452 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:56:05.159175    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/2b989a47-1798-48a2-9568-9cb971a3b4a3-proxy-tls\") pod \"success-200-isvc-6b6c1-predictor-d7d64c7db-ntzbm\" (UID: \"2b989a47-1798-48a2-9568-9cb971a3b4a3\") " pod="kserve-ci-e2e-test/success-200-isvc-6b6c1-predictor-d7d64c7db-ntzbm"
Sep 17 17:56:05.159452 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:56:05.159220    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-fl6zx\" (UniqueName: \"kubernetes.io/projected/bd8084fe-ed00-4a05-a0a3-0ab206ba7278-kube-api-access-fl6zx\") pod \"error-404-isvc-6b6c1-predictor-66c99c6495-hh86l\" (UID: \"bd8084fe-ed00-4a05-a0a3-0ab206ba7278\") " pod="kserve-ci-e2e-test/error-404-isvc-6b6c1-predictor-66c99c6495-hh86l"
Sep 17 17:56:05.159452 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:56:05.159266    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/2b989a47-1798-48a2-9568-9cb971a3b4a3-kube-rbac-proxy-sar-config\") pod \"success-200-isvc-6b6c1-predictor-d7d64c7db-ntzbm\" (UID: \"2b989a47-1798-48a2-9568-9cb971a3b4a3\") " pod="kserve-ci-e2e-test/success-200-isvc-6b6c1-predictor-d7d64c7db-ntzbm"
Sep 17 17:56:05.159452 ip-10-0-137-20 kubenswrapper[2565]: E0917 17:56:05.159290    2565 secret.go:189] Couldn't get secret kserve-ci-e2e-test/success-200-isvc-6b6c1-predictor-serving-cert: secret "success-200-isvc-6b6c1-predictor-serving-cert" not found
Sep 17 17:56:05.159452 ip-10-0-137-20 kubenswrapper[2565]: E0917 17:56:05.159367    2565 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/2b989a47-1798-48a2-9568-9cb971a3b4a3-proxy-tls podName:2b989a47-1798-48a2-9568-9cb971a3b4a3 nodeName:}" failed. No retries permitted until 2026-09-17 17:56:05.659350069 +0000 UTC m=+1315.536634462 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "proxy-tls" (UniqueName: "kubernetes.io/secret/2b989a47-1798-48a2-9568-9cb971a3b4a3-proxy-tls") pod "success-200-isvc-6b6c1-predictor-d7d64c7db-ntzbm" (UID: "2b989a47-1798-48a2-9568-9cb971a3b4a3") : secret "success-200-isvc-6b6c1-predictor-serving-cert" not found
Sep 17 17:56:05.159923 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:56:05.159904    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/2b989a47-1798-48a2-9568-9cb971a3b4a3-kube-rbac-proxy-sar-config\") pod \"success-200-isvc-6b6c1-predictor-d7d64c7db-ntzbm\" (UID: \"2b989a47-1798-48a2-9568-9cb971a3b4a3\") " pod="kserve-ci-e2e-test/success-200-isvc-6b6c1-predictor-d7d64c7db-ntzbm"
Sep 17 17:56:05.168516 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:56:05.168492    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-lt5tp\" (UniqueName: \"kubernetes.io/projected/2b989a47-1798-48a2-9568-9cb971a3b4a3-kube-api-access-lt5tp\") pod \"success-200-isvc-6b6c1-predictor-d7d64c7db-ntzbm\" (UID: \"2b989a47-1798-48a2-9568-9cb971a3b4a3\") " pod="kserve-ci-e2e-test/success-200-isvc-6b6c1-predictor-d7d64c7db-ntzbm"
Sep 17 17:56:05.260406 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:56:05.260373    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/bd8084fe-ed00-4a05-a0a3-0ab206ba7278-kube-rbac-proxy-sar-config\") pod \"error-404-isvc-6b6c1-predictor-66c99c6495-hh86l\" (UID: \"bd8084fe-ed00-4a05-a0a3-0ab206ba7278\") " pod="kserve-ci-e2e-test/error-404-isvc-6b6c1-predictor-66c99c6495-hh86l"
Sep 17 17:56:05.260406 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:56:05.260409    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/bd8084fe-ed00-4a05-a0a3-0ab206ba7278-proxy-tls\") pod \"error-404-isvc-6b6c1-predictor-66c99c6495-hh86l\" (UID: \"bd8084fe-ed00-4a05-a0a3-0ab206ba7278\") " pod="kserve-ci-e2e-test/error-404-isvc-6b6c1-predictor-66c99c6495-hh86l"
Sep 17 17:56:05.260629 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:56:05.260467    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fl6zx\" (UniqueName: \"kubernetes.io/projected/bd8084fe-ed00-4a05-a0a3-0ab206ba7278-kube-api-access-fl6zx\") pod \"error-404-isvc-6b6c1-predictor-66c99c6495-hh86l\" (UID: \"bd8084fe-ed00-4a05-a0a3-0ab206ba7278\") " pod="kserve-ci-e2e-test/error-404-isvc-6b6c1-predictor-66c99c6495-hh86l"
Sep 17 17:56:05.260629 ip-10-0-137-20 kubenswrapper[2565]: E0917 17:56:05.260587    2565 secret.go:189] Couldn't get secret kserve-ci-e2e-test/error-404-isvc-6b6c1-predictor-serving-cert: secret "error-404-isvc-6b6c1-predictor-serving-cert" not found
Sep 17 17:56:05.260708 ip-10-0-137-20 kubenswrapper[2565]: E0917 17:56:05.260657    2565 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/bd8084fe-ed00-4a05-a0a3-0ab206ba7278-proxy-tls podName:bd8084fe-ed00-4a05-a0a3-0ab206ba7278 nodeName:}" failed. No retries permitted until 2026-09-17 17:56:05.7606419 +0000 UTC m=+1315.637926292 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "proxy-tls" (UniqueName: "kubernetes.io/secret/bd8084fe-ed00-4a05-a0a3-0ab206ba7278-proxy-tls") pod "error-404-isvc-6b6c1-predictor-66c99c6495-hh86l" (UID: "bd8084fe-ed00-4a05-a0a3-0ab206ba7278") : secret "error-404-isvc-6b6c1-predictor-serving-cert" not found
Sep 17 17:56:05.261041 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:56:05.261005    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/bd8084fe-ed00-4a05-a0a3-0ab206ba7278-kube-rbac-proxy-sar-config\") pod \"error-404-isvc-6b6c1-predictor-66c99c6495-hh86l\" (UID: \"bd8084fe-ed00-4a05-a0a3-0ab206ba7278\") " pod="kserve-ci-e2e-test/error-404-isvc-6b6c1-predictor-66c99c6495-hh86l"
Sep 17 17:56:05.269772 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:56:05.269752    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-fl6zx\" (UniqueName: \"kubernetes.io/projected/bd8084fe-ed00-4a05-a0a3-0ab206ba7278-kube-api-access-fl6zx\") pod \"error-404-isvc-6b6c1-predictor-66c99c6495-hh86l\" (UID: \"bd8084fe-ed00-4a05-a0a3-0ab206ba7278\") " pod="kserve-ci-e2e-test/error-404-isvc-6b6c1-predictor-66c99c6495-hh86l"
Sep 17 17:56:05.663717 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:56:05.663670    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/2b989a47-1798-48a2-9568-9cb971a3b4a3-proxy-tls\") pod \"success-200-isvc-6b6c1-predictor-d7d64c7db-ntzbm\" (UID: \"2b989a47-1798-48a2-9568-9cb971a3b4a3\") " pod="kserve-ci-e2e-test/success-200-isvc-6b6c1-predictor-d7d64c7db-ntzbm"
Sep 17 17:56:05.666189 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:56:05.666166    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/2b989a47-1798-48a2-9568-9cb971a3b4a3-proxy-tls\") pod \"success-200-isvc-6b6c1-predictor-d7d64c7db-ntzbm\" (UID: \"2b989a47-1798-48a2-9568-9cb971a3b4a3\") " pod="kserve-ci-e2e-test/success-200-isvc-6b6c1-predictor-d7d64c7db-ntzbm"
Sep 17 17:56:05.764882 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:56:05.764848    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/bd8084fe-ed00-4a05-a0a3-0ab206ba7278-proxy-tls\") pod \"error-404-isvc-6b6c1-predictor-66c99c6495-hh86l\" (UID: \"bd8084fe-ed00-4a05-a0a3-0ab206ba7278\") " pod="kserve-ci-e2e-test/error-404-isvc-6b6c1-predictor-66c99c6495-hh86l"
Sep 17 17:56:05.767234 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:56:05.767211    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/bd8084fe-ed00-4a05-a0a3-0ab206ba7278-proxy-tls\") pod \"error-404-isvc-6b6c1-predictor-66c99c6495-hh86l\" (UID: \"bd8084fe-ed00-4a05-a0a3-0ab206ba7278\") " pod="kserve-ci-e2e-test/error-404-isvc-6b6c1-predictor-66c99c6495-hh86l"
Sep 17 17:56:05.885250 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:56:05.885211    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/success-200-isvc-6b6c1-predictor-d7d64c7db-ntzbm"
Sep 17 17:56:05.968458 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:56:05.968420    2565 generic.go:356] "Generic (PLEG): container finished" podID="f6fd4091-5a8d-4971-b38f-7659800208d0" containerID="60b0ab5a73d9814d6fa3289a7980b989ced8f24822b576787a7aa7fc21739254" exitCode=2
Sep 17 17:56:05.968617 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:56:05.968493    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/error-404-isvc-390a8-predictor-56ccd8877-qp6qj" event={"ID":"f6fd4091-5a8d-4971-b38f-7659800208d0","Type":"ContainerDied","Data":"60b0ab5a73d9814d6fa3289a7980b989ced8f24822b576787a7aa7fc21739254"}
Sep 17 17:56:05.970406 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:56:05.970384    2565 generic.go:356] "Generic (PLEG): container finished" podID="e2fe5e12-9e83-432a-b26b-4cfd217203a7" containerID="1d0c74b0a648827ead186c29bf3a3699ec5df5b95be7bd78ea9947e2fb8ed1d0" exitCode=2
Sep 17 17:56:05.970526 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:56:05.970434    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/success-200-isvc-390a8-predictor-6749bbb468-v5k5t" event={"ID":"e2fe5e12-9e83-432a-b26b-4cfd217203a7","Type":"ContainerDied","Data":"1d0c74b0a648827ead186c29bf3a3699ec5df5b95be7bd78ea9947e2fb8ed1d0"}
Sep 17 17:56:06.023547 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:56:06.023524    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/error-404-isvc-6b6c1-predictor-66c99c6495-hh86l"
Sep 17 17:56:06.158139 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:56:06.158108    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/success-200-isvc-6b6c1-predictor-d7d64c7db-ntzbm"]
Sep 17 17:56:06.161534 ip-10-0-137-20 kubenswrapper[2565]: W0917 17:56:06.161508    2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2b989a47_1798_48a2_9568_9cb971a3b4a3.slice/crio-b8275188efb76ddbffae7bc51f5337b68f92a4ea416199e2b29dcc5a6edb76a4 WatchSource:0}: Error finding container b8275188efb76ddbffae7bc51f5337b68f92a4ea416199e2b29dcc5a6edb76a4: Status 404 returned error can't find the container with id b8275188efb76ddbffae7bc51f5337b68f92a4ea416199e2b29dcc5a6edb76a4
Sep 17 17:56:06.294493 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:56:06.294465    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/error-404-isvc-6b6c1-predictor-66c99c6495-hh86l"]
Sep 17 17:56:06.297464 ip-10-0-137-20 kubenswrapper[2565]: W0917 17:56:06.297437    2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podbd8084fe_ed00_4a05_a0a3_0ab206ba7278.slice/crio-18ba11dfa6abd743e5364784e1f32caec2e145150bc9048fd575daceb3bbf953 WatchSource:0}: Error finding container 18ba11dfa6abd743e5364784e1f32caec2e145150bc9048fd575daceb3bbf953: Status 404 returned error can't find the container with id 18ba11dfa6abd743e5364784e1f32caec2e145150bc9048fd575daceb3bbf953
Sep 17 17:56:06.977698 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:56:06.977660    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/error-404-isvc-6b6c1-predictor-66c99c6495-hh86l" event={"ID":"bd8084fe-ed00-4a05-a0a3-0ab206ba7278","Type":"ContainerStarted","Data":"f3b9639ae256937b3ef622512dccac7171527c15ded42b18ff9c03b15d919b72"}
Sep 17 17:56:06.978189 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:56:06.977733    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/error-404-isvc-6b6c1-predictor-66c99c6495-hh86l" event={"ID":"bd8084fe-ed00-4a05-a0a3-0ab206ba7278","Type":"ContainerStarted","Data":"607c5c0f936659f69727d94518d033bcc2a01ae9bda86e43eb5a699a99a0380f"}
Sep 17 17:56:06.978189 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:56:06.977766    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/error-404-isvc-6b6c1-predictor-66c99c6495-hh86l" event={"ID":"bd8084fe-ed00-4a05-a0a3-0ab206ba7278","Type":"ContainerStarted","Data":"18ba11dfa6abd743e5364784e1f32caec2e145150bc9048fd575daceb3bbf953"}
Sep 17 17:56:06.978189 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:56:06.977805    2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve-ci-e2e-test/error-404-isvc-6b6c1-predictor-66c99c6495-hh86l"
Sep 17 17:56:06.978189 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:56:06.977840    2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve-ci-e2e-test/error-404-isvc-6b6c1-predictor-66c99c6495-hh86l"
Sep 17 17:56:06.979174 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:56:06.979145    2565 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/error-404-isvc-6b6c1-predictor-66c99c6495-hh86l" podUID="bd8084fe-ed00-4a05-a0a3-0ab206ba7278" containerName="kserve-container" probeResult="failure" output="dial tcp 10.133.0.41:8080: connect: connection refused"
Sep 17 17:56:06.979534 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:56:06.979512    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/success-200-isvc-6b6c1-predictor-d7d64c7db-ntzbm" event={"ID":"2b989a47-1798-48a2-9568-9cb971a3b4a3","Type":"ContainerStarted","Data":"007da526c833243dfa6b5f7fd58341ead2cdee320756e467eb759e7370437600"}
Sep 17 17:56:06.979613 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:56:06.979556    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/success-200-isvc-6b6c1-predictor-d7d64c7db-ntzbm" event={"ID":"2b989a47-1798-48a2-9568-9cb971a3b4a3","Type":"ContainerStarted","Data":"abecdf9e34ea83e8a7cef2a4e678f3f82980a47421f6ec394545a95b1b4f94ee"}
Sep 17 17:56:06.979613 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:56:06.979576    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/success-200-isvc-6b6c1-predictor-d7d64c7db-ntzbm" event={"ID":"2b989a47-1798-48a2-9568-9cb971a3b4a3","Type":"ContainerStarted","Data":"b8275188efb76ddbffae7bc51f5337b68f92a4ea416199e2b29dcc5a6edb76a4"}
Sep 17 17:56:06.979807 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:56:06.979785    2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve-ci-e2e-test/success-200-isvc-6b6c1-predictor-d7d64c7db-ntzbm"
Sep 17 17:56:06.979871 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:56:06.979833    2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve-ci-e2e-test/success-200-isvc-6b6c1-predictor-d7d64c7db-ntzbm"
Sep 17 17:56:06.980825 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:56:06.980802    2565 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/success-200-isvc-6b6c1-predictor-d7d64c7db-ntzbm" podUID="2b989a47-1798-48a2-9568-9cb971a3b4a3" containerName="kserve-container" probeResult="failure" output="dial tcp 10.133.0.40:8080: connect: connection refused"
Sep 17 17:56:06.997211 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:56:06.997168    2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kserve-ci-e2e-test/error-404-isvc-6b6c1-predictor-66c99c6495-hh86l" podStartSLOduration=1.997154773 podStartE2EDuration="1.997154773s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-17 17:56:05 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-17 17:56:06.996130301 +0000 UTC m=+1316.873414714" watchObservedRunningTime="2026-09-17 17:56:06.997154773 +0000 UTC m=+1316.874439187"
Sep 17 17:56:07.015553 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:56:07.015514    2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kserve-ci-e2e-test/success-200-isvc-6b6c1-predictor-d7d64c7db-ntzbm" podStartSLOduration=3.01550037 podStartE2EDuration="3.01550037s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-17 17:56:04 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-17 17:56:07.014548758 +0000 UTC m=+1316.891833250" watchObservedRunningTime="2026-09-17 17:56:07.01550037 +0000 UTC m=+1316.892784854"
Sep 17 17:56:07.984796 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:56:07.984750    2565 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/error-404-isvc-6b6c1-predictor-66c99c6495-hh86l" podUID="bd8084fe-ed00-4a05-a0a3-0ab206ba7278" containerName="kserve-container" probeResult="failure" output="dial tcp 10.133.0.41:8080: connect: connection refused"
Sep 17 17:56:07.985141 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:56:07.984809    2565 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/success-200-isvc-6b6c1-predictor-d7d64c7db-ntzbm" podUID="2b989a47-1798-48a2-9568-9cb971a3b4a3" containerName="kserve-container" probeResult="failure" output="dial tcp 10.133.0.40:8080: connect: connection refused"
Sep 17 17:56:08.121999 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:56:08.121980    2565 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/success-200-isvc-390a8-predictor-6749bbb468-v5k5t"
Sep 17 17:56:08.184862 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:56:08.184837    2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-h4rgb\" (UniqueName: \"kubernetes.io/projected/e2fe5e12-9e83-432a-b26b-4cfd217203a7-kube-api-access-h4rgb\") pod \"e2fe5e12-9e83-432a-b26b-4cfd217203a7\" (UID: \"e2fe5e12-9e83-432a-b26b-4cfd217203a7\") "
Sep 17 17:56:08.184996 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:56:08.184878    2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/e2fe5e12-9e83-432a-b26b-4cfd217203a7-proxy-tls\") pod \"e2fe5e12-9e83-432a-b26b-4cfd217203a7\" (UID: \"e2fe5e12-9e83-432a-b26b-4cfd217203a7\") "
Sep 17 17:56:08.184996 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:56:08.184940    2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/e2fe5e12-9e83-432a-b26b-4cfd217203a7-kube-rbac-proxy-sar-config\") pod \"e2fe5e12-9e83-432a-b26b-4cfd217203a7\" (UID: \"e2fe5e12-9e83-432a-b26b-4cfd217203a7\") "
Sep 17 17:56:08.185330 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:56:08.185304    2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/e2fe5e12-9e83-432a-b26b-4cfd217203a7-kube-rbac-proxy-sar-config" (OuterVolumeSpecName: "kube-rbac-proxy-sar-config") pod "e2fe5e12-9e83-432a-b26b-4cfd217203a7" (UID: "e2fe5e12-9e83-432a-b26b-4cfd217203a7"). InnerVolumeSpecName "kube-rbac-proxy-sar-config". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 17 17:56:08.186940 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:56:08.186919    2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/e2fe5e12-9e83-432a-b26b-4cfd217203a7-proxy-tls" (OuterVolumeSpecName: "proxy-tls") pod "e2fe5e12-9e83-432a-b26b-4cfd217203a7" (UID: "e2fe5e12-9e83-432a-b26b-4cfd217203a7"). InnerVolumeSpecName "proxy-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 17 17:56:08.187007 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:56:08.186966    2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e2fe5e12-9e83-432a-b26b-4cfd217203a7-kube-api-access-h4rgb" (OuterVolumeSpecName: "kube-api-access-h4rgb") pod "e2fe5e12-9e83-432a-b26b-4cfd217203a7" (UID: "e2fe5e12-9e83-432a-b26b-4cfd217203a7"). InnerVolumeSpecName "kube-api-access-h4rgb". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 17 17:56:08.286526 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:56:08.286497    2565 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-h4rgb\" (UniqueName: \"kubernetes.io/projected/e2fe5e12-9e83-432a-b26b-4cfd217203a7-kube-api-access-h4rgb\") on node \"ip-10-0-137-20.ec2.internal\" DevicePath \"\""
Sep 17 17:56:08.286526 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:56:08.286526    2565 reconciler_common.go:299] "Volume detached for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/e2fe5e12-9e83-432a-b26b-4cfd217203a7-proxy-tls\") on node \"ip-10-0-137-20.ec2.internal\" DevicePath \"\""
Sep 17 17:56:08.286672 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:56:08.286540    2565 reconciler_common.go:299] "Volume detached for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/e2fe5e12-9e83-432a-b26b-4cfd217203a7-kube-rbac-proxy-sar-config\") on node \"ip-10-0-137-20.ec2.internal\" DevicePath \"\""
Sep 17 17:56:08.349446 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:56:08.349409    2565 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/error-404-isvc-390a8-predictor-56ccd8877-qp6qj"
Sep 17 17:56:08.387782 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:56:08.387755    2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-mncv6\" (UniqueName: \"kubernetes.io/projected/f6fd4091-5a8d-4971-b38f-7659800208d0-kube-api-access-mncv6\") pod \"f6fd4091-5a8d-4971-b38f-7659800208d0\" (UID: \"f6fd4091-5a8d-4971-b38f-7659800208d0\") "
Sep 17 17:56:08.387930 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:56:08.387809    2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/f6fd4091-5a8d-4971-b38f-7659800208d0-proxy-tls\") pod \"f6fd4091-5a8d-4971-b38f-7659800208d0\" (UID: \"f6fd4091-5a8d-4971-b38f-7659800208d0\") "
Sep 17 17:56:08.387930 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:56:08.387888    2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/f6fd4091-5a8d-4971-b38f-7659800208d0-kube-rbac-proxy-sar-config\") pod \"f6fd4091-5a8d-4971-b38f-7659800208d0\" (UID: \"f6fd4091-5a8d-4971-b38f-7659800208d0\") "
Sep 17 17:56:08.388271 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:56:08.388249    2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/f6fd4091-5a8d-4971-b38f-7659800208d0-kube-rbac-proxy-sar-config" (OuterVolumeSpecName: "kube-rbac-proxy-sar-config") pod "f6fd4091-5a8d-4971-b38f-7659800208d0" (UID: "f6fd4091-5a8d-4971-b38f-7659800208d0"). InnerVolumeSpecName "kube-rbac-proxy-sar-config". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 17 17:56:08.389710 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:56:08.389689    2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f6fd4091-5a8d-4971-b38f-7659800208d0-kube-api-access-mncv6" (OuterVolumeSpecName: "kube-api-access-mncv6") pod "f6fd4091-5a8d-4971-b38f-7659800208d0" (UID: "f6fd4091-5a8d-4971-b38f-7659800208d0"). InnerVolumeSpecName "kube-api-access-mncv6". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 17 17:56:08.389817 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:56:08.389746    2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/f6fd4091-5a8d-4971-b38f-7659800208d0-proxy-tls" (OuterVolumeSpecName: "proxy-tls") pod "f6fd4091-5a8d-4971-b38f-7659800208d0" (UID: "f6fd4091-5a8d-4971-b38f-7659800208d0"). InnerVolumeSpecName "proxy-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 17 17:56:08.489435 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:56:08.489376    2565 reconciler_common.go:299] "Volume detached for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/f6fd4091-5a8d-4971-b38f-7659800208d0-proxy-tls\") on node \"ip-10-0-137-20.ec2.internal\" DevicePath \"\""
Sep 17 17:56:08.489435 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:56:08.489398    2565 reconciler_common.go:299] "Volume detached for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/f6fd4091-5a8d-4971-b38f-7659800208d0-kube-rbac-proxy-sar-config\") on node \"ip-10-0-137-20.ec2.internal\" DevicePath \"\""
Sep 17 17:56:08.489435 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:56:08.489409    2565 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-mncv6\" (UniqueName: \"kubernetes.io/projected/f6fd4091-5a8d-4971-b38f-7659800208d0-kube-api-access-mncv6\") on node \"ip-10-0-137-20.ec2.internal\" DevicePath \"\""
Sep 17 17:56:08.991057 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:56:08.991006    2565 generic.go:356] "Generic (PLEG): container finished" podID="e2fe5e12-9e83-432a-b26b-4cfd217203a7" containerID="b9fc73a651f270a9aaebe1b5b29dd9028f2627573e985d3b7198cf83adbe8d1f" exitCode=0
Sep 17 17:56:08.991057 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:56:08.991049    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/success-200-isvc-390a8-predictor-6749bbb468-v5k5t" event={"ID":"e2fe5e12-9e83-432a-b26b-4cfd217203a7","Type":"ContainerDied","Data":"b9fc73a651f270a9aaebe1b5b29dd9028f2627573e985d3b7198cf83adbe8d1f"}
Sep 17 17:56:08.991794 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:56:08.991081    2565 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/success-200-isvc-390a8-predictor-6749bbb468-v5k5t"
Sep 17 17:56:08.991794 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:56:08.991105    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/success-200-isvc-390a8-predictor-6749bbb468-v5k5t" event={"ID":"e2fe5e12-9e83-432a-b26b-4cfd217203a7","Type":"ContainerDied","Data":"b7afd2b507c441329580a36022596aff4c23b50b06056aba6c87280e2cb66310"}
Sep 17 17:56:08.991794 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:56:08.991131    2565 scope.go:117] "RemoveContainer" containerID="1d0c74b0a648827ead186c29bf3a3699ec5df5b95be7bd78ea9947e2fb8ed1d0"
Sep 17 17:56:08.993093 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:56:08.993067    2565 generic.go:356] "Generic (PLEG): container finished" podID="f6fd4091-5a8d-4971-b38f-7659800208d0" containerID="9afe5bb756b942124d852ced2da9f35d09ec18d470c4e656fb5f0cbc343291db" exitCode=0
Sep 17 17:56:08.993225 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:56:08.993108    2565 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/error-404-isvc-390a8-predictor-56ccd8877-qp6qj"
Sep 17 17:56:08.993225 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:56:08.993119    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/error-404-isvc-390a8-predictor-56ccd8877-qp6qj" event={"ID":"f6fd4091-5a8d-4971-b38f-7659800208d0","Type":"ContainerDied","Data":"9afe5bb756b942124d852ced2da9f35d09ec18d470c4e656fb5f0cbc343291db"}
Sep 17 17:56:08.993225 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:56:08.993159    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/error-404-isvc-390a8-predictor-56ccd8877-qp6qj" event={"ID":"f6fd4091-5a8d-4971-b38f-7659800208d0","Type":"ContainerDied","Data":"14526a04b2e4bf45696eeb3c50c5c9ab41a3a2a7b8672329cdea916bc2671337"}
Sep 17 17:56:09.000690 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:56:09.000537    2565 scope.go:117] "RemoveContainer" containerID="b9fc73a651f270a9aaebe1b5b29dd9028f2627573e985d3b7198cf83adbe8d1f"
Sep 17 17:56:09.008340 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:56:09.008324    2565 scope.go:117] "RemoveContainer" containerID="1d0c74b0a648827ead186c29bf3a3699ec5df5b95be7bd78ea9947e2fb8ed1d0"
Sep 17 17:56:09.008614 ip-10-0-137-20 kubenswrapper[2565]: E0917 17:56:09.008597    2565 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"1d0c74b0a648827ead186c29bf3a3699ec5df5b95be7bd78ea9947e2fb8ed1d0\": container with ID starting with 1d0c74b0a648827ead186c29bf3a3699ec5df5b95be7bd78ea9947e2fb8ed1d0 not found: ID does not exist" containerID="1d0c74b0a648827ead186c29bf3a3699ec5df5b95be7bd78ea9947e2fb8ed1d0"
Sep 17 17:56:09.008655 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:56:09.008622    2565 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"1d0c74b0a648827ead186c29bf3a3699ec5df5b95be7bd78ea9947e2fb8ed1d0"} err="failed to get container status \"1d0c74b0a648827ead186c29bf3a3699ec5df5b95be7bd78ea9947e2fb8ed1d0\": rpc error: code = NotFound desc = could not find container \"1d0c74b0a648827ead186c29bf3a3699ec5df5b95be7bd78ea9947e2fb8ed1d0\": container with ID starting with 1d0c74b0a648827ead186c29bf3a3699ec5df5b95be7bd78ea9947e2fb8ed1d0 not found: ID does not exist"
Sep 17 17:56:09.008655 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:56:09.008637    2565 scope.go:117] "RemoveContainer" containerID="b9fc73a651f270a9aaebe1b5b29dd9028f2627573e985d3b7198cf83adbe8d1f"
Sep 17 17:56:09.008868 ip-10-0-137-20 kubenswrapper[2565]: E0917 17:56:09.008840    2565 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"b9fc73a651f270a9aaebe1b5b29dd9028f2627573e985d3b7198cf83adbe8d1f\": container with ID starting with b9fc73a651f270a9aaebe1b5b29dd9028f2627573e985d3b7198cf83adbe8d1f not found: ID does not exist" containerID="b9fc73a651f270a9aaebe1b5b29dd9028f2627573e985d3b7198cf83adbe8d1f"
Sep 17 17:56:09.008901 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:56:09.008872    2565 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"b9fc73a651f270a9aaebe1b5b29dd9028f2627573e985d3b7198cf83adbe8d1f"} err="failed to get container status \"b9fc73a651f270a9aaebe1b5b29dd9028f2627573e985d3b7198cf83adbe8d1f\": rpc error: code = NotFound desc = could not find container \"b9fc73a651f270a9aaebe1b5b29dd9028f2627573e985d3b7198cf83adbe8d1f\": container with ID starting with b9fc73a651f270a9aaebe1b5b29dd9028f2627573e985d3b7198cf83adbe8d1f not found: ID does not exist"
Sep 17 17:56:09.008901 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:56:09.008885    2565 scope.go:117] "RemoveContainer" containerID="60b0ab5a73d9814d6fa3289a7980b989ced8f24822b576787a7aa7fc21739254"
Sep 17 17:56:09.014308 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:56:09.014281    2565 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kserve-ci-e2e-test/success-200-isvc-390a8-predictor-6749bbb468-v5k5t"]
Sep 17 17:56:09.018994 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:56:09.018975    2565 scope.go:117] "RemoveContainer" containerID="9afe5bb756b942124d852ced2da9f35d09ec18d470c4e656fb5f0cbc343291db"
Sep 17 17:56:09.020625 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:56:09.020456    2565 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kserve-ci-e2e-test/success-200-isvc-390a8-predictor-6749bbb468-v5k5t"]
Sep 17 17:56:09.030749 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:56:09.030731    2565 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kserve-ci-e2e-test/error-404-isvc-390a8-predictor-56ccd8877-qp6qj"]
Sep 17 17:56:09.034008 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:56:09.033991    2565 scope.go:117] "RemoveContainer" containerID="60b0ab5a73d9814d6fa3289a7980b989ced8f24822b576787a7aa7fc21739254"
Sep 17 17:56:09.034399 ip-10-0-137-20 kubenswrapper[2565]: E0917 17:56:09.034381    2565 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"60b0ab5a73d9814d6fa3289a7980b989ced8f24822b576787a7aa7fc21739254\": container with ID starting with 60b0ab5a73d9814d6fa3289a7980b989ced8f24822b576787a7aa7fc21739254 not found: ID does not exist" containerID="60b0ab5a73d9814d6fa3289a7980b989ced8f24822b576787a7aa7fc21739254"
Sep 17 17:56:09.034463 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:56:09.034410    2565 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"60b0ab5a73d9814d6fa3289a7980b989ced8f24822b576787a7aa7fc21739254"} err="failed to get container status \"60b0ab5a73d9814d6fa3289a7980b989ced8f24822b576787a7aa7fc21739254\": rpc error: code = NotFound desc = could not find container \"60b0ab5a73d9814d6fa3289a7980b989ced8f24822b576787a7aa7fc21739254\": container with ID starting with 60b0ab5a73d9814d6fa3289a7980b989ced8f24822b576787a7aa7fc21739254 not found: ID does not exist"
Sep 17 17:56:09.034463 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:56:09.034427    2565 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kserve-ci-e2e-test/error-404-isvc-390a8-predictor-56ccd8877-qp6qj"]
Sep 17 17:56:09.034463 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:56:09.034435    2565 scope.go:117] "RemoveContainer" containerID="9afe5bb756b942124d852ced2da9f35d09ec18d470c4e656fb5f0cbc343291db"
Sep 17 17:56:09.034686 ip-10-0-137-20 kubenswrapper[2565]: E0917 17:56:09.034667    2565 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"9afe5bb756b942124d852ced2da9f35d09ec18d470c4e656fb5f0cbc343291db\": container with ID starting with 9afe5bb756b942124d852ced2da9f35d09ec18d470c4e656fb5f0cbc343291db not found: ID does not exist" containerID="9afe5bb756b942124d852ced2da9f35d09ec18d470c4e656fb5f0cbc343291db"
Sep 17 17:56:09.034726 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:56:09.034691    2565 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"9afe5bb756b942124d852ced2da9f35d09ec18d470c4e656fb5f0cbc343291db"} err="failed to get container status \"9afe5bb756b942124d852ced2da9f35d09ec18d470c4e656fb5f0cbc343291db\": rpc error: code = NotFound desc = could not find container \"9afe5bb756b942124d852ced2da9f35d09ec18d470c4e656fb5f0cbc343291db\": container with ID starting with 9afe5bb756b942124d852ced2da9f35d09ec18d470c4e656fb5f0cbc343291db not found: ID does not exist"
Sep 17 17:56:10.158511 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:56:10.158478    2565 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="e2fe5e12-9e83-432a-b26b-4cfd217203a7" path="/var/lib/kubelet/pods/e2fe5e12-9e83-432a-b26b-4cfd217203a7/volumes"
Sep 17 17:56:10.158894 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:56:10.158880    2565 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="f6fd4091-5a8d-4971-b38f-7659800208d0" path="/var/lib/kubelet/pods/f6fd4091-5a8d-4971-b38f-7659800208d0/volumes"
Sep 17 17:56:12.988991 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:56:12.988964    2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kserve-ci-e2e-test/success-200-isvc-6b6c1-predictor-d7d64c7db-ntzbm"
Sep 17 17:56:12.989525 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:56:12.989288    2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kserve-ci-e2e-test/error-404-isvc-6b6c1-predictor-66c99c6495-hh86l"
Sep 17 17:56:12.989525 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:56:12.989424    2565 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/success-200-isvc-6b6c1-predictor-d7d64c7db-ntzbm" podUID="2b989a47-1798-48a2-9568-9cb971a3b4a3" containerName="kserve-container" probeResult="failure" output="dial tcp 10.133.0.40:8080: connect: connection refused"
Sep 17 17:56:12.989798 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:56:12.989774    2565 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/error-404-isvc-6b6c1-predictor-66c99c6495-hh86l" podUID="bd8084fe-ed00-4a05-a0a3-0ab206ba7278" containerName="kserve-container" probeResult="failure" output="dial tcp 10.133.0.41:8080: connect: connection refused"
Sep 17 17:56:18.581489 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:56:18.581454    2565 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kserve-ci-e2e-test/success-200-isvc-2b6c1-predictor-65f5775b6-6s9hp"]
Sep 17 17:56:18.581974 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:56:18.581683    2565 kuberuntime_container.go:873] "Killing container with a grace period" pod="kserve-ci-e2e-test/success-200-isvc-2b6c1-predictor-65f5775b6-6s9hp" podUID="64036856-d29a-4195-ba9d-dcd53de9cd4a" containerName="kserve-container" containerID="cri-o://08487acbe04f08fd45b8dc9734ca92228792e4ad5d4e3ebbc3ccf59056354a2f" gracePeriod=30
Sep 17 17:56:18.581974 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:56:18.581719    2565 kuberuntime_container.go:873] "Killing container with a grace period" pod="kserve-ci-e2e-test/success-200-isvc-2b6c1-predictor-65f5775b6-6s9hp" podUID="64036856-d29a-4195-ba9d-dcd53de9cd4a" containerName="kube-rbac-proxy" containerID="cri-o://7616e9015f3cd8c2f00d6de37759062440da7cc828d3157c789ac9155c143173" gracePeriod=30
Sep 17 17:56:18.628625 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:56:18.628592    2565 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kserve-ci-e2e-test/error-404-isvc-2b6c1-predictor-88cbb7679-pf5xf"]
Sep 17 17:56:18.628841 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:56:18.628800    2565 kuberuntime_container.go:873] "Killing container with a grace period" pod="kserve-ci-e2e-test/error-404-isvc-2b6c1-predictor-88cbb7679-pf5xf" podUID="a598baa1-44e2-4711-9d08-a245a368b3eb" containerName="kserve-container" containerID="cri-o://9e828ce2f1eb8fef7df6b6ec9eb82df972671b751db9989028a47c20916c1523" gracePeriod=30
Sep 17 17:56:18.628944 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:56:18.628846    2565 kuberuntime_container.go:873] "Killing container with a grace period" pod="kserve-ci-e2e-test/error-404-isvc-2b6c1-predictor-88cbb7679-pf5xf" podUID="a598baa1-44e2-4711-9d08-a245a368b3eb" containerName="kube-rbac-proxy" containerID="cri-o://695d7e57268e60374c4a1f4931a25a215c3286d7a43da3f545bcf46568b4e08b" gracePeriod=30
Sep 17 17:56:18.707589 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:56:18.707548    2565 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/error-404-isvc-2b6c1-predictor-88cbb7679-pf5xf" podUID="a598baa1-44e2-4711-9d08-a245a368b3eb" containerName="kube-rbac-proxy" probeResult="failure" output="Get \"https://10.133.0.39:8643/healthz\": dial tcp 10.133.0.39:8643: connect: connection refused"
Sep 17 17:56:18.714772 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:56:18.714744    2565 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kserve-ci-e2e-test/success-200-isvc-c5c33-predictor-7d76dcc468-tldgv"]
Sep 17 17:56:18.715135 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:56:18.715121    2565 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e2fe5e12-9e83-432a-b26b-4cfd217203a7" containerName="kube-rbac-proxy"
Sep 17 17:56:18.715205 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:56:18.715137    2565 state_mem.go:107] "Deleted CPUSet assignment" podUID="e2fe5e12-9e83-432a-b26b-4cfd217203a7" containerName="kube-rbac-proxy"
Sep 17 17:56:18.715205 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:56:18.715159    2565 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e2fe5e12-9e83-432a-b26b-4cfd217203a7" containerName="kserve-container"
Sep 17 17:56:18.715205 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:56:18.715165    2565 state_mem.go:107] "Deleted CPUSet assignment" podUID="e2fe5e12-9e83-432a-b26b-4cfd217203a7" containerName="kserve-container"
Sep 17 17:56:18.715205 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:56:18.715171    2565 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f6fd4091-5a8d-4971-b38f-7659800208d0" containerName="kube-rbac-proxy"
Sep 17 17:56:18.715205 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:56:18.715177    2565 state_mem.go:107] "Deleted CPUSet assignment" podUID="f6fd4091-5a8d-4971-b38f-7659800208d0" containerName="kube-rbac-proxy"
Sep 17 17:56:18.715205 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:56:18.715186    2565 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f6fd4091-5a8d-4971-b38f-7659800208d0" containerName="kserve-container"
Sep 17 17:56:18.715205 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:56:18.715192    2565 state_mem.go:107] "Deleted CPUSet assignment" podUID="f6fd4091-5a8d-4971-b38f-7659800208d0" containerName="kserve-container"
Sep 17 17:56:18.715561 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:56:18.715245    2565 memory_manager.go:368] "RemoveStaleState removing state" podUID="e2fe5e12-9e83-432a-b26b-4cfd217203a7" containerName="kube-rbac-proxy"
Sep 17 17:56:18.715561 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:56:18.715251    2565 memory_manager.go:368] "RemoveStaleState removing state" podUID="e2fe5e12-9e83-432a-b26b-4cfd217203a7" containerName="kserve-container"
Sep 17 17:56:18.715561 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:56:18.715259    2565 memory_manager.go:368] "RemoveStaleState removing state" podUID="f6fd4091-5a8d-4971-b38f-7659800208d0" containerName="kube-rbac-proxy"
Sep 17 17:56:18.715561 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:56:18.715266    2565 memory_manager.go:368] "RemoveStaleState removing state" podUID="f6fd4091-5a8d-4971-b38f-7659800208d0" containerName="kserve-container"
Sep 17 17:56:18.718475 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:56:18.718459    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/success-200-isvc-c5c33-predictor-7d76dcc468-tldgv"
Sep 17 17:56:18.721246 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:56:18.721223    2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kserve-ci-e2e-test\"/\"success-200-isvc-c5c33-kube-rbac-proxy-sar-config\""
Sep 17 17:56:18.721519 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:56:18.721504    2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kserve-ci-e2e-test\"/\"success-200-isvc-c5c33-predictor-serving-cert\""
Sep 17 17:56:18.728766 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:56:18.728744    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/success-200-isvc-c5c33-predictor-7d76dcc468-tldgv"]
Sep 17 17:56:18.780424 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:56:18.780392    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/2e69c7c5-5af7-477f-92db-cea0c52943b1-kube-rbac-proxy-sar-config\") pod \"success-200-isvc-c5c33-predictor-7d76dcc468-tldgv\" (UID: \"2e69c7c5-5af7-477f-92db-cea0c52943b1\") " pod="kserve-ci-e2e-test/success-200-isvc-c5c33-predictor-7d76dcc468-tldgv"
Sep 17 17:56:18.780592 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:56:18.780435    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-w9b29\" (UniqueName: \"kubernetes.io/projected/2e69c7c5-5af7-477f-92db-cea0c52943b1-kube-api-access-w9b29\") pod \"success-200-isvc-c5c33-predictor-7d76dcc468-tldgv\" (UID: \"2e69c7c5-5af7-477f-92db-cea0c52943b1\") " pod="kserve-ci-e2e-test/success-200-isvc-c5c33-predictor-7d76dcc468-tldgv"
Sep 17 17:56:18.780592 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:56:18.780464    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/2e69c7c5-5af7-477f-92db-cea0c52943b1-proxy-tls\") pod \"success-200-isvc-c5c33-predictor-7d76dcc468-tldgv\" (UID: \"2e69c7c5-5af7-477f-92db-cea0c52943b1\") " pod="kserve-ci-e2e-test/success-200-isvc-c5c33-predictor-7d76dcc468-tldgv"
Sep 17 17:56:18.813184 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:56:18.813153    2565 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kserve-ci-e2e-test/error-404-isvc-c5c33-predictor-6cd4d7bc49-lmkh5"]
Sep 17 17:56:18.824648 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:56:18.824613    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/error-404-isvc-c5c33-predictor-6cd4d7bc49-lmkh5"
Sep 17 17:56:18.826259 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:56:18.826234    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/error-404-isvc-c5c33-predictor-6cd4d7bc49-lmkh5"]
Sep 17 17:56:18.827240 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:56:18.827220    2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kserve-ci-e2e-test\"/\"error-404-isvc-c5c33-predictor-serving-cert\""
Sep 17 17:56:18.827240 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:56:18.827234    2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kserve-ci-e2e-test\"/\"error-404-isvc-c5c33-kube-rbac-proxy-sar-config\""
Sep 17 17:56:18.881916 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:56:18.881890    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-p5rcj\" (UniqueName: \"kubernetes.io/projected/b4205277-b668-41c9-974a-730d76d07f5a-kube-api-access-p5rcj\") pod \"error-404-isvc-c5c33-predictor-6cd4d7bc49-lmkh5\" (UID: \"b4205277-b668-41c9-974a-730d76d07f5a\") " pod="kserve-ci-e2e-test/error-404-isvc-c5c33-predictor-6cd4d7bc49-lmkh5"
Sep 17 17:56:18.882111 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:56:18.881937    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/b4205277-b668-41c9-974a-730d76d07f5a-proxy-tls\") pod \"error-404-isvc-c5c33-predictor-6cd4d7bc49-lmkh5\" (UID: \"b4205277-b668-41c9-974a-730d76d07f5a\") " pod="kserve-ci-e2e-test/error-404-isvc-c5c33-predictor-6cd4d7bc49-lmkh5"
Sep 17 17:56:18.882111 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:56:18.881957    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/b4205277-b668-41c9-974a-730d76d07f5a-kube-rbac-proxy-sar-config\") pod \"error-404-isvc-c5c33-predictor-6cd4d7bc49-lmkh5\" (UID: \"b4205277-b668-41c9-974a-730d76d07f5a\") " pod="kserve-ci-e2e-test/error-404-isvc-c5c33-predictor-6cd4d7bc49-lmkh5"
Sep 17 17:56:18.882111 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:56:18.882088    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/2e69c7c5-5af7-477f-92db-cea0c52943b1-kube-rbac-proxy-sar-config\") pod \"success-200-isvc-c5c33-predictor-7d76dcc468-tldgv\" (UID: \"2e69c7c5-5af7-477f-92db-cea0c52943b1\") " pod="kserve-ci-e2e-test/success-200-isvc-c5c33-predictor-7d76dcc468-tldgv"
Sep 17 17:56:18.882290 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:56:18.882127    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-w9b29\" (UniqueName: \"kubernetes.io/projected/2e69c7c5-5af7-477f-92db-cea0c52943b1-kube-api-access-w9b29\") pod \"success-200-isvc-c5c33-predictor-7d76dcc468-tldgv\" (UID: \"2e69c7c5-5af7-477f-92db-cea0c52943b1\") " pod="kserve-ci-e2e-test/success-200-isvc-c5c33-predictor-7d76dcc468-tldgv"
Sep 17 17:56:18.882290 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:56:18.882155    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/2e69c7c5-5af7-477f-92db-cea0c52943b1-proxy-tls\") pod \"success-200-isvc-c5c33-predictor-7d76dcc468-tldgv\" (UID: \"2e69c7c5-5af7-477f-92db-cea0c52943b1\") " pod="kserve-ci-e2e-test/success-200-isvc-c5c33-predictor-7d76dcc468-tldgv"
Sep 17 17:56:18.882290 ip-10-0-137-20 kubenswrapper[2565]: E0917 17:56:18.882249    2565 secret.go:189] Couldn't get secret kserve-ci-e2e-test/success-200-isvc-c5c33-predictor-serving-cert: secret "success-200-isvc-c5c33-predictor-serving-cert" not found
Sep 17 17:56:18.882407 ip-10-0-137-20 kubenswrapper[2565]: E0917 17:56:18.882334    2565 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/2e69c7c5-5af7-477f-92db-cea0c52943b1-proxy-tls podName:2e69c7c5-5af7-477f-92db-cea0c52943b1 nodeName:}" failed. No retries permitted until 2026-09-17 17:56:19.382313476 +0000 UTC m=+1329.259597874 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "proxy-tls" (UniqueName: "kubernetes.io/secret/2e69c7c5-5af7-477f-92db-cea0c52943b1-proxy-tls") pod "success-200-isvc-c5c33-predictor-7d76dcc468-tldgv" (UID: "2e69c7c5-5af7-477f-92db-cea0c52943b1") : secret "success-200-isvc-c5c33-predictor-serving-cert" not found
Sep 17 17:56:18.882742 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:56:18.882726    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/2e69c7c5-5af7-477f-92db-cea0c52943b1-kube-rbac-proxy-sar-config\") pod \"success-200-isvc-c5c33-predictor-7d76dcc468-tldgv\" (UID: \"2e69c7c5-5af7-477f-92db-cea0c52943b1\") " pod="kserve-ci-e2e-test/success-200-isvc-c5c33-predictor-7d76dcc468-tldgv"
Sep 17 17:56:18.890562 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:56:18.890543    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-w9b29\" (UniqueName: \"kubernetes.io/projected/2e69c7c5-5af7-477f-92db-cea0c52943b1-kube-api-access-w9b29\") pod \"success-200-isvc-c5c33-predictor-7d76dcc468-tldgv\" (UID: \"2e69c7c5-5af7-477f-92db-cea0c52943b1\") " pod="kserve-ci-e2e-test/success-200-isvc-c5c33-predictor-7d76dcc468-tldgv"
Sep 17 17:56:18.983001 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:56:18.982968    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-p5rcj\" (UniqueName: \"kubernetes.io/projected/b4205277-b668-41c9-974a-730d76d07f5a-kube-api-access-p5rcj\") pod \"error-404-isvc-c5c33-predictor-6cd4d7bc49-lmkh5\" (UID: \"b4205277-b668-41c9-974a-730d76d07f5a\") " pod="kserve-ci-e2e-test/error-404-isvc-c5c33-predictor-6cd4d7bc49-lmkh5"
Sep 17 17:56:18.983182 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:56:18.983041    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/b4205277-b668-41c9-974a-730d76d07f5a-proxy-tls\") pod \"error-404-isvc-c5c33-predictor-6cd4d7bc49-lmkh5\" (UID: \"b4205277-b668-41c9-974a-730d76d07f5a\") " pod="kserve-ci-e2e-test/error-404-isvc-c5c33-predictor-6cd4d7bc49-lmkh5"
Sep 17 17:56:18.983182 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:56:18.983064    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/b4205277-b668-41c9-974a-730d76d07f5a-kube-rbac-proxy-sar-config\") pod \"error-404-isvc-c5c33-predictor-6cd4d7bc49-lmkh5\" (UID: \"b4205277-b668-41c9-974a-730d76d07f5a\") " pod="kserve-ci-e2e-test/error-404-isvc-c5c33-predictor-6cd4d7bc49-lmkh5"
Sep 17 17:56:18.983891 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:56:18.983864    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/b4205277-b668-41c9-974a-730d76d07f5a-kube-rbac-proxy-sar-config\") pod \"error-404-isvc-c5c33-predictor-6cd4d7bc49-lmkh5\" (UID: \"b4205277-b668-41c9-974a-730d76d07f5a\") " pod="kserve-ci-e2e-test/error-404-isvc-c5c33-predictor-6cd4d7bc49-lmkh5"
Sep 17 17:56:18.985591 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:56:18.985565    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/b4205277-b668-41c9-974a-730d76d07f5a-proxy-tls\") pod \"error-404-isvc-c5c33-predictor-6cd4d7bc49-lmkh5\" (UID: \"b4205277-b668-41c9-974a-730d76d07f5a\") " pod="kserve-ci-e2e-test/error-404-isvc-c5c33-predictor-6cd4d7bc49-lmkh5"
Sep 17 17:56:18.991259 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:56:18.991236    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-p5rcj\" (UniqueName: \"kubernetes.io/projected/b4205277-b668-41c9-974a-730d76d07f5a-kube-api-access-p5rcj\") pod \"error-404-isvc-c5c33-predictor-6cd4d7bc49-lmkh5\" (UID: \"b4205277-b668-41c9-974a-730d76d07f5a\") " pod="kserve-ci-e2e-test/error-404-isvc-c5c33-predictor-6cd4d7bc49-lmkh5"
Sep 17 17:56:19.042069 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:56:19.042016    2565 generic.go:356] "Generic (PLEG): container finished" podID="a598baa1-44e2-4711-9d08-a245a368b3eb" containerID="695d7e57268e60374c4a1f4931a25a215c3286d7a43da3f545bcf46568b4e08b" exitCode=2
Sep 17 17:56:19.042210 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:56:19.042090    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/error-404-isvc-2b6c1-predictor-88cbb7679-pf5xf" event={"ID":"a598baa1-44e2-4711-9d08-a245a368b3eb","Type":"ContainerDied","Data":"695d7e57268e60374c4a1f4931a25a215c3286d7a43da3f545bcf46568b4e08b"}
Sep 17 17:56:19.043753 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:56:19.043731    2565 generic.go:356] "Generic (PLEG): container finished" podID="64036856-d29a-4195-ba9d-dcd53de9cd4a" containerID="7616e9015f3cd8c2f00d6de37759062440da7cc828d3157c789ac9155c143173" exitCode=2
Sep 17 17:56:19.043857 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:56:19.043778    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/success-200-isvc-2b6c1-predictor-65f5775b6-6s9hp" event={"ID":"64036856-d29a-4195-ba9d-dcd53de9cd4a","Type":"ContainerDied","Data":"7616e9015f3cd8c2f00d6de37759062440da7cc828d3157c789ac9155c143173"}
Sep 17 17:56:19.137812 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:56:19.137780    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/error-404-isvc-c5c33-predictor-6cd4d7bc49-lmkh5"
Sep 17 17:56:19.387422 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:56:19.387391    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/2e69c7c5-5af7-477f-92db-cea0c52943b1-proxy-tls\") pod \"success-200-isvc-c5c33-predictor-7d76dcc468-tldgv\" (UID: \"2e69c7c5-5af7-477f-92db-cea0c52943b1\") " pod="kserve-ci-e2e-test/success-200-isvc-c5c33-predictor-7d76dcc468-tldgv"
Sep 17 17:56:19.389929 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:56:19.389858    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/2e69c7c5-5af7-477f-92db-cea0c52943b1-proxy-tls\") pod \"success-200-isvc-c5c33-predictor-7d76dcc468-tldgv\" (UID: \"2e69c7c5-5af7-477f-92db-cea0c52943b1\") " pod="kserve-ci-e2e-test/success-200-isvc-c5c33-predictor-7d76dcc468-tldgv"
Sep 17 17:56:19.412993 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:56:19.412971    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/error-404-isvc-c5c33-predictor-6cd4d7bc49-lmkh5"]
Sep 17 17:56:19.414849 ip-10-0-137-20 kubenswrapper[2565]: W0917 17:56:19.414821    2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb4205277_b668_41c9_974a_730d76d07f5a.slice/crio-a2f6f67275ec8530b2769b21a192a7db5f631aa841c3f39c3ca29dfb1f71381f WatchSource:0}: Error finding container a2f6f67275ec8530b2769b21a192a7db5f631aa841c3f39c3ca29dfb1f71381f: Status 404 returned error can't find the container with id a2f6f67275ec8530b2769b21a192a7db5f631aa841c3f39c3ca29dfb1f71381f
Sep 17 17:56:19.631601 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:56:19.631566    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/success-200-isvc-c5c33-predictor-7d76dcc468-tldgv"
Sep 17 17:56:19.906719 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:56:19.906694    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/success-200-isvc-c5c33-predictor-7d76dcc468-tldgv"]
Sep 17 17:56:19.908752 ip-10-0-137-20 kubenswrapper[2565]: W0917 17:56:19.908723    2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2e69c7c5_5af7_477f_92db_cea0c52943b1.slice/crio-af157a82cd97ac678dbabed9a6d7b93a4d446de3eed179a59b70edd75a2d1b2e WatchSource:0}: Error finding container af157a82cd97ac678dbabed9a6d7b93a4d446de3eed179a59b70edd75a2d1b2e: Status 404 returned error can't find the container with id af157a82cd97ac678dbabed9a6d7b93a4d446de3eed179a59b70edd75a2d1b2e
Sep 17 17:56:20.054056 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:56:20.053072    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/success-200-isvc-c5c33-predictor-7d76dcc468-tldgv" event={"ID":"2e69c7c5-5af7-477f-92db-cea0c52943b1","Type":"ContainerStarted","Data":"25a53fff0067927b5179739646e2f28de41773c61250b612e5296ab44f6464bf"}
Sep 17 17:56:20.054056 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:56:20.053147    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/success-200-isvc-c5c33-predictor-7d76dcc468-tldgv" event={"ID":"2e69c7c5-5af7-477f-92db-cea0c52943b1","Type":"ContainerStarted","Data":"90986357adc7fe3324f4fb73a2ffcad8942cf922d287b0895eb91c66ee7aa5dd"}
Sep 17 17:56:20.054056 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:56:20.053179    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/success-200-isvc-c5c33-predictor-7d76dcc468-tldgv" event={"ID":"2e69c7c5-5af7-477f-92db-cea0c52943b1","Type":"ContainerStarted","Data":"af157a82cd97ac678dbabed9a6d7b93a4d446de3eed179a59b70edd75a2d1b2e"}
Sep 17 17:56:20.054319 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:56:20.054100    2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve-ci-e2e-test/success-200-isvc-c5c33-predictor-7d76dcc468-tldgv"
Sep 17 17:56:20.054319 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:56:20.054147    2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve-ci-e2e-test/success-200-isvc-c5c33-predictor-7d76dcc468-tldgv"
Sep 17 17:56:20.055886 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:56:20.055842    2565 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/success-200-isvc-c5c33-predictor-7d76dcc468-tldgv" podUID="2e69c7c5-5af7-477f-92db-cea0c52943b1" containerName="kserve-container" probeResult="failure" output="dial tcp 10.133.0.42:8080: connect: connection refused"
Sep 17 17:56:20.056293 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:56:20.056269    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/error-404-isvc-c5c33-predictor-6cd4d7bc49-lmkh5" event={"ID":"b4205277-b668-41c9-974a-730d76d07f5a","Type":"ContainerStarted","Data":"1cf60e373868613cee2d199b3e1680b473bd7c227b09b1b826936b93661e1e82"}
Sep 17 17:56:20.056399 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:56:20.056323    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/error-404-isvc-c5c33-predictor-6cd4d7bc49-lmkh5" event={"ID":"b4205277-b668-41c9-974a-730d76d07f5a","Type":"ContainerStarted","Data":"a6eadae9e1c78c3ae81f71db5427b20ff0d56acff9802cf551a84d2cf3271b0b"}
Sep 17 17:56:20.056399 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:56:20.056357    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/error-404-isvc-c5c33-predictor-6cd4d7bc49-lmkh5" event={"ID":"b4205277-b668-41c9-974a-730d76d07f5a","Type":"ContainerStarted","Data":"a2f6f67275ec8530b2769b21a192a7db5f631aa841c3f39c3ca29dfb1f71381f"}
Sep 17 17:56:20.056399 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:56:20.056392    2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve-ci-e2e-test/error-404-isvc-c5c33-predictor-6cd4d7bc49-lmkh5"
Sep 17 17:56:20.072016 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:56:20.071974    2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kserve-ci-e2e-test/success-200-isvc-c5c33-predictor-7d76dcc468-tldgv" podStartSLOduration=2.071961668 podStartE2EDuration="2.071961668s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-17 17:56:18 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-17 17:56:20.071192075 +0000 UTC m=+1329.948476492" watchObservedRunningTime="2026-09-17 17:56:20.071961668 +0000 UTC m=+1329.949246083"
Sep 17 17:56:20.090862 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:56:20.090826    2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kserve-ci-e2e-test/error-404-isvc-c5c33-predictor-6cd4d7bc49-lmkh5" podStartSLOduration=2.090813216 podStartE2EDuration="2.090813216s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-17 17:56:18 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-17 17:56:20.089758995 +0000 UTC m=+1329.967043410" watchObservedRunningTime="2026-09-17 17:56:20.090813216 +0000 UTC m=+1329.968097631"
Sep 17 17:56:21.061331 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:56:21.061296    2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve-ci-e2e-test/error-404-isvc-c5c33-predictor-6cd4d7bc49-lmkh5"
Sep 17 17:56:21.061331 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:56:21.061307    2565 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/success-200-isvc-c5c33-predictor-7d76dcc468-tldgv" podUID="2e69c7c5-5af7-477f-92db-cea0c52943b1" containerName="kserve-container" probeResult="failure" output="dial tcp 10.133.0.42:8080: connect: connection refused"
Sep 17 17:56:21.062518 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:56:21.062495    2565 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/error-404-isvc-c5c33-predictor-6cd4d7bc49-lmkh5" podUID="b4205277-b668-41c9-974a-730d76d07f5a" containerName="kserve-container" probeResult="failure" output="dial tcp 10.133.0.43:8080: connect: connection refused"
Sep 17 17:56:21.686482 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:56:21.686444    2565 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/success-200-isvc-2b6c1-predictor-65f5775b6-6s9hp" podUID="64036856-d29a-4195-ba9d-dcd53de9cd4a" containerName="kube-rbac-proxy" probeResult="failure" output="Get \"https://10.133.0.38:8643/healthz\": dial tcp 10.133.0.38:8643: connect: connection refused"
Sep 17 17:56:21.691784 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:56:21.691760    2565 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/success-200-isvc-2b6c1-predictor-65f5775b6-6s9hp" podUID="64036856-d29a-4195-ba9d-dcd53de9cd4a" containerName="kserve-container" probeResult="failure" output="dial tcp 10.133.0.38:8080: connect: connection refused"
Sep 17 17:56:21.937751 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:56:21.937694    2565 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/success-200-isvc-2b6c1-predictor-65f5775b6-6s9hp"
Sep 17 17:56:22.011700 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:56:22.011665    2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/64036856-d29a-4195-ba9d-dcd53de9cd4a-kube-rbac-proxy-sar-config\") pod \"64036856-d29a-4195-ba9d-dcd53de9cd4a\" (UID: \"64036856-d29a-4195-ba9d-dcd53de9cd4a\") "
Sep 17 17:56:22.011867 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:56:22.011708    2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-4vfw5\" (UniqueName: \"kubernetes.io/projected/64036856-d29a-4195-ba9d-dcd53de9cd4a-kube-api-access-4vfw5\") pod \"64036856-d29a-4195-ba9d-dcd53de9cd4a\" (UID: \"64036856-d29a-4195-ba9d-dcd53de9cd4a\") "
Sep 17 17:56:22.011949 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:56:22.011892    2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/64036856-d29a-4195-ba9d-dcd53de9cd4a-proxy-tls\") pod \"64036856-d29a-4195-ba9d-dcd53de9cd4a\" (UID: \"64036856-d29a-4195-ba9d-dcd53de9cd4a\") "
Sep 17 17:56:22.012059 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:56:22.012014    2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/64036856-d29a-4195-ba9d-dcd53de9cd4a-kube-rbac-proxy-sar-config" (OuterVolumeSpecName: "kube-rbac-proxy-sar-config") pod "64036856-d29a-4195-ba9d-dcd53de9cd4a" (UID: "64036856-d29a-4195-ba9d-dcd53de9cd4a"). InnerVolumeSpecName "kube-rbac-proxy-sar-config". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 17 17:56:22.012172 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:56:22.012159    2565 reconciler_common.go:299] "Volume detached for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/64036856-d29a-4195-ba9d-dcd53de9cd4a-kube-rbac-proxy-sar-config\") on node \"ip-10-0-137-20.ec2.internal\" DevicePath \"\""
Sep 17 17:56:22.013788 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:56:22.013764    2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/64036856-d29a-4195-ba9d-dcd53de9cd4a-proxy-tls" (OuterVolumeSpecName: "proxy-tls") pod "64036856-d29a-4195-ba9d-dcd53de9cd4a" (UID: "64036856-d29a-4195-ba9d-dcd53de9cd4a"). InnerVolumeSpecName "proxy-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 17 17:56:22.013874 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:56:22.013809    2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/64036856-d29a-4195-ba9d-dcd53de9cd4a-kube-api-access-4vfw5" (OuterVolumeSpecName: "kube-api-access-4vfw5") pod "64036856-d29a-4195-ba9d-dcd53de9cd4a" (UID: "64036856-d29a-4195-ba9d-dcd53de9cd4a"). InnerVolumeSpecName "kube-api-access-4vfw5". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 17 17:56:22.067003 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:56:22.066972    2565 generic.go:356] "Generic (PLEG): container finished" podID="64036856-d29a-4195-ba9d-dcd53de9cd4a" containerID="08487acbe04f08fd45b8dc9734ca92228792e4ad5d4e3ebbc3ccf59056354a2f" exitCode=0
Sep 17 17:56:22.067426 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:56:22.067045    2565 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/success-200-isvc-2b6c1-predictor-65f5775b6-6s9hp"
Sep 17 17:56:22.067426 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:56:22.067078    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/success-200-isvc-2b6c1-predictor-65f5775b6-6s9hp" event={"ID":"64036856-d29a-4195-ba9d-dcd53de9cd4a","Type":"ContainerDied","Data":"08487acbe04f08fd45b8dc9734ca92228792e4ad5d4e3ebbc3ccf59056354a2f"}
Sep 17 17:56:22.067426 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:56:22.067155    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/success-200-isvc-2b6c1-predictor-65f5775b6-6s9hp" event={"ID":"64036856-d29a-4195-ba9d-dcd53de9cd4a","Type":"ContainerDied","Data":"cfc4420711f3846f26f747cb1c6cb91b9aa7573a979d6af3f4d556e0a4ac94fc"}
Sep 17 17:56:22.067426 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:56:22.067193    2565 scope.go:117] "RemoveContainer" containerID="7616e9015f3cd8c2f00d6de37759062440da7cc828d3157c789ac9155c143173"
Sep 17 17:56:22.067726 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:56:22.067701    2565 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/error-404-isvc-c5c33-predictor-6cd4d7bc49-lmkh5" podUID="b4205277-b668-41c9-974a-730d76d07f5a" containerName="kserve-container" probeResult="failure" output="dial tcp 10.133.0.43:8080: connect: connection refused"
Sep 17 17:56:22.068001 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:56:22.067979    2565 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/success-200-isvc-c5c33-predictor-7d76dcc468-tldgv" podUID="2e69c7c5-5af7-477f-92db-cea0c52943b1" containerName="kserve-container" probeResult="failure" output="dial tcp 10.133.0.42:8080: connect: connection refused"
Sep 17 17:56:22.076068 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:56:22.076049    2565 scope.go:117] "RemoveContainer" containerID="08487acbe04f08fd45b8dc9734ca92228792e4ad5d4e3ebbc3ccf59056354a2f"
Sep 17 17:56:22.083382 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:56:22.083365    2565 scope.go:117] "RemoveContainer" containerID="7616e9015f3cd8c2f00d6de37759062440da7cc828d3157c789ac9155c143173"
Sep 17 17:56:22.083640 ip-10-0-137-20 kubenswrapper[2565]: E0917 17:56:22.083623    2565 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"7616e9015f3cd8c2f00d6de37759062440da7cc828d3157c789ac9155c143173\": container with ID starting with 7616e9015f3cd8c2f00d6de37759062440da7cc828d3157c789ac9155c143173 not found: ID does not exist" containerID="7616e9015f3cd8c2f00d6de37759062440da7cc828d3157c789ac9155c143173"
Sep 17 17:56:22.083698 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:56:22.083647    2565 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"7616e9015f3cd8c2f00d6de37759062440da7cc828d3157c789ac9155c143173"} err="failed to get container status \"7616e9015f3cd8c2f00d6de37759062440da7cc828d3157c789ac9155c143173\": rpc error: code = NotFound desc = could not find container \"7616e9015f3cd8c2f00d6de37759062440da7cc828d3157c789ac9155c143173\": container with ID starting with 7616e9015f3cd8c2f00d6de37759062440da7cc828d3157c789ac9155c143173 not found: ID does not exist"
Sep 17 17:56:22.083698 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:56:22.083663    2565 scope.go:117] "RemoveContainer" containerID="08487acbe04f08fd45b8dc9734ca92228792e4ad5d4e3ebbc3ccf59056354a2f"
Sep 17 17:56:22.083886 ip-10-0-137-20 kubenswrapper[2565]: E0917 17:56:22.083869    2565 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"08487acbe04f08fd45b8dc9734ca92228792e4ad5d4e3ebbc3ccf59056354a2f\": container with ID starting with 08487acbe04f08fd45b8dc9734ca92228792e4ad5d4e3ebbc3ccf59056354a2f not found: ID does not exist" containerID="08487acbe04f08fd45b8dc9734ca92228792e4ad5d4e3ebbc3ccf59056354a2f"
Sep 17 17:56:22.083935 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:56:22.083896    2565 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"08487acbe04f08fd45b8dc9734ca92228792e4ad5d4e3ebbc3ccf59056354a2f"} err="failed to get container status \"08487acbe04f08fd45b8dc9734ca92228792e4ad5d4e3ebbc3ccf59056354a2f\": rpc error: code = NotFound desc = could not find container \"08487acbe04f08fd45b8dc9734ca92228792e4ad5d4e3ebbc3ccf59056354a2f\": container with ID starting with 08487acbe04f08fd45b8dc9734ca92228792e4ad5d4e3ebbc3ccf59056354a2f not found: ID does not exist"
Sep 17 17:56:22.089156 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:56:22.089131    2565 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kserve-ci-e2e-test/success-200-isvc-2b6c1-predictor-65f5775b6-6s9hp"]
Sep 17 17:56:22.091565 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:56:22.091545    2565 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kserve-ci-e2e-test/success-200-isvc-2b6c1-predictor-65f5775b6-6s9hp"]
Sep 17 17:56:22.112861 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:56:22.112839    2565 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-4vfw5\" (UniqueName: \"kubernetes.io/projected/64036856-d29a-4195-ba9d-dcd53de9cd4a-kube-api-access-4vfw5\") on node \"ip-10-0-137-20.ec2.internal\" DevicePath \"\""
Sep 17 17:56:22.112861 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:56:22.112860    2565 reconciler_common.go:299] "Volume detached for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/64036856-d29a-4195-ba9d-dcd53de9cd4a-proxy-tls\") on node \"ip-10-0-137-20.ec2.internal\" DevicePath \"\""
Sep 17 17:56:22.156930 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:56:22.156902    2565 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="64036856-d29a-4195-ba9d-dcd53de9cd4a" path="/var/lib/kubelet/pods/64036856-d29a-4195-ba9d-dcd53de9cd4a/volumes"
Sep 17 17:56:22.574982 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:56:22.574957    2565 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/error-404-isvc-2b6c1-predictor-88cbb7679-pf5xf"
Sep 17 17:56:22.717725 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:56:22.717693    2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-dmvbg\" (UniqueName: \"kubernetes.io/projected/a598baa1-44e2-4711-9d08-a245a368b3eb-kube-api-access-dmvbg\") pod \"a598baa1-44e2-4711-9d08-a245a368b3eb\" (UID: \"a598baa1-44e2-4711-9d08-a245a368b3eb\") "
Sep 17 17:56:22.717916 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:56:22.717748    2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/a598baa1-44e2-4711-9d08-a245a368b3eb-kube-rbac-proxy-sar-config\") pod \"a598baa1-44e2-4711-9d08-a245a368b3eb\" (UID: \"a598baa1-44e2-4711-9d08-a245a368b3eb\") "
Sep 17 17:56:22.717916 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:56:22.717809    2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/a598baa1-44e2-4711-9d08-a245a368b3eb-proxy-tls\") pod \"a598baa1-44e2-4711-9d08-a245a368b3eb\" (UID: \"a598baa1-44e2-4711-9d08-a245a368b3eb\") "
Sep 17 17:56:22.718150 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:56:22.718127    2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/a598baa1-44e2-4711-9d08-a245a368b3eb-kube-rbac-proxy-sar-config" (OuterVolumeSpecName: "kube-rbac-proxy-sar-config") pod "a598baa1-44e2-4711-9d08-a245a368b3eb" (UID: "a598baa1-44e2-4711-9d08-a245a368b3eb"). InnerVolumeSpecName "kube-rbac-proxy-sar-config". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 17 17:56:22.719701 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:56:22.719680    2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a598baa1-44e2-4711-9d08-a245a368b3eb-kube-api-access-dmvbg" (OuterVolumeSpecName: "kube-api-access-dmvbg") pod "a598baa1-44e2-4711-9d08-a245a368b3eb" (UID: "a598baa1-44e2-4711-9d08-a245a368b3eb"). InnerVolumeSpecName "kube-api-access-dmvbg". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 17 17:56:22.719746 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:56:22.719707    2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/a598baa1-44e2-4711-9d08-a245a368b3eb-proxy-tls" (OuterVolumeSpecName: "proxy-tls") pod "a598baa1-44e2-4711-9d08-a245a368b3eb" (UID: "a598baa1-44e2-4711-9d08-a245a368b3eb"). InnerVolumeSpecName "proxy-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 17 17:56:22.819406 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:56:22.819367    2565 reconciler_common.go:299] "Volume detached for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/a598baa1-44e2-4711-9d08-a245a368b3eb-kube-rbac-proxy-sar-config\") on node \"ip-10-0-137-20.ec2.internal\" DevicePath \"\""
Sep 17 17:56:22.819406 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:56:22.819404    2565 reconciler_common.go:299] "Volume detached for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/a598baa1-44e2-4711-9d08-a245a368b3eb-proxy-tls\") on node \"ip-10-0-137-20.ec2.internal\" DevicePath \"\""
Sep 17 17:56:22.819630 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:56:22.819420    2565 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-dmvbg\" (UniqueName: \"kubernetes.io/projected/a598baa1-44e2-4711-9d08-a245a368b3eb-kube-api-access-dmvbg\") on node \"ip-10-0-137-20.ec2.internal\" DevicePath \"\""
Sep 17 17:56:22.990065 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:56:22.989942    2565 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/error-404-isvc-6b6c1-predictor-66c99c6495-hh86l" podUID="bd8084fe-ed00-4a05-a0a3-0ab206ba7278" containerName="kserve-container" probeResult="failure" output="dial tcp 10.133.0.41:8080: connect: connection refused"
Sep 17 17:56:22.990065 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:56:22.989942    2565 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/success-200-isvc-6b6c1-predictor-d7d64c7db-ntzbm" podUID="2b989a47-1798-48a2-9568-9cb971a3b4a3" containerName="kserve-container" probeResult="failure" output="dial tcp 10.133.0.40:8080: connect: connection refused"
Sep 17 17:56:23.073829 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:56:23.073791    2565 generic.go:356] "Generic (PLEG): container finished" podID="a598baa1-44e2-4711-9d08-a245a368b3eb" containerID="9e828ce2f1eb8fef7df6b6ec9eb82df972671b751db9989028a47c20916c1523" exitCode=0
Sep 17 17:56:23.074343 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:56:23.073851    2565 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/error-404-isvc-2b6c1-predictor-88cbb7679-pf5xf"
Sep 17 17:56:23.074343 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:56:23.073872    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/error-404-isvc-2b6c1-predictor-88cbb7679-pf5xf" event={"ID":"a598baa1-44e2-4711-9d08-a245a368b3eb","Type":"ContainerDied","Data":"9e828ce2f1eb8fef7df6b6ec9eb82df972671b751db9989028a47c20916c1523"}
Sep 17 17:56:23.074343 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:56:23.073968    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/error-404-isvc-2b6c1-predictor-88cbb7679-pf5xf" event={"ID":"a598baa1-44e2-4711-9d08-a245a368b3eb","Type":"ContainerDied","Data":"785aca6be3b9afa8864b3923b1c67d4fcfe5d4e63c669317a6d734c56ad340bd"}
Sep 17 17:56:23.074343 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:56:23.074001    2565 scope.go:117] "RemoveContainer" containerID="695d7e57268e60374c4a1f4931a25a215c3286d7a43da3f545bcf46568b4e08b"
Sep 17 17:56:23.082733 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:56:23.082717    2565 scope.go:117] "RemoveContainer" containerID="9e828ce2f1eb8fef7df6b6ec9eb82df972671b751db9989028a47c20916c1523"
Sep 17 17:56:23.090677 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:56:23.090658    2565 scope.go:117] "RemoveContainer" containerID="695d7e57268e60374c4a1f4931a25a215c3286d7a43da3f545bcf46568b4e08b"
Sep 17 17:56:23.090965 ip-10-0-137-20 kubenswrapper[2565]: E0917 17:56:23.090947    2565 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"695d7e57268e60374c4a1f4931a25a215c3286d7a43da3f545bcf46568b4e08b\": container with ID starting with 695d7e57268e60374c4a1f4931a25a215c3286d7a43da3f545bcf46568b4e08b not found: ID does not exist" containerID="695d7e57268e60374c4a1f4931a25a215c3286d7a43da3f545bcf46568b4e08b"
Sep 17 17:56:23.091050 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:56:23.090981    2565 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"695d7e57268e60374c4a1f4931a25a215c3286d7a43da3f545bcf46568b4e08b"} err="failed to get container status \"695d7e57268e60374c4a1f4931a25a215c3286d7a43da3f545bcf46568b4e08b\": rpc error: code = NotFound desc = could not find container \"695d7e57268e60374c4a1f4931a25a215c3286d7a43da3f545bcf46568b4e08b\": container with ID starting with 695d7e57268e60374c4a1f4931a25a215c3286d7a43da3f545bcf46568b4e08b not found: ID does not exist"
Sep 17 17:56:23.091050 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:56:23.091006    2565 scope.go:117] "RemoveContainer" containerID="9e828ce2f1eb8fef7df6b6ec9eb82df972671b751db9989028a47c20916c1523"
Sep 17 17:56:23.091303 ip-10-0-137-20 kubenswrapper[2565]: E0917 17:56:23.091286    2565 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"9e828ce2f1eb8fef7df6b6ec9eb82df972671b751db9989028a47c20916c1523\": container with ID starting with 9e828ce2f1eb8fef7df6b6ec9eb82df972671b751db9989028a47c20916c1523 not found: ID does not exist" containerID="9e828ce2f1eb8fef7df6b6ec9eb82df972671b751db9989028a47c20916c1523"
Sep 17 17:56:23.091343 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:56:23.091310    2565 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"9e828ce2f1eb8fef7df6b6ec9eb82df972671b751db9989028a47c20916c1523"} err="failed to get container status \"9e828ce2f1eb8fef7df6b6ec9eb82df972671b751db9989028a47c20916c1523\": rpc error: code = NotFound desc = could not find container \"9e828ce2f1eb8fef7df6b6ec9eb82df972671b751db9989028a47c20916c1523\": container with ID starting with 9e828ce2f1eb8fef7df6b6ec9eb82df972671b751db9989028a47c20916c1523 not found: ID does not exist"
Sep 17 17:56:23.098684 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:56:23.098653    2565 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kserve-ci-e2e-test/error-404-isvc-2b6c1-predictor-88cbb7679-pf5xf"]
Sep 17 17:56:23.102778 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:56:23.102753    2565 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kserve-ci-e2e-test/error-404-isvc-2b6c1-predictor-88cbb7679-pf5xf"]
Sep 17 17:56:24.157861 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:56:24.157828    2565 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="a598baa1-44e2-4711-9d08-a245a368b3eb" path="/var/lib/kubelet/pods/a598baa1-44e2-4711-9d08-a245a368b3eb/volumes"
Sep 17 17:56:27.072215 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:56:27.072138    2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kserve-ci-e2e-test/error-404-isvc-c5c33-predictor-6cd4d7bc49-lmkh5"
Sep 17 17:56:27.072591 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:56:27.072234    2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kserve-ci-e2e-test/success-200-isvc-c5c33-predictor-7d76dcc468-tldgv"
Sep 17 17:56:27.072646 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:56:27.072585    2565 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/success-200-isvc-c5c33-predictor-7d76dcc468-tldgv" podUID="2e69c7c5-5af7-477f-92db-cea0c52943b1" containerName="kserve-container" probeResult="failure" output="dial tcp 10.133.0.42:8080: connect: connection refused"
Sep 17 17:56:27.072705 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:56:27.072647    2565 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/error-404-isvc-c5c33-predictor-6cd4d7bc49-lmkh5" podUID="b4205277-b668-41c9-974a-730d76d07f5a" containerName="kserve-container" probeResult="failure" output="dial tcp 10.133.0.43:8080: connect: connection refused"
Sep 17 17:56:32.990146 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:56:32.990108    2565 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/error-404-isvc-6b6c1-predictor-66c99c6495-hh86l" podUID="bd8084fe-ed00-4a05-a0a3-0ab206ba7278" containerName="kserve-container" probeResult="failure" output="dial tcp 10.133.0.41:8080: connect: connection refused"
Sep 17 17:56:32.990545 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:56:32.990109    2565 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/success-200-isvc-6b6c1-predictor-d7d64c7db-ntzbm" podUID="2b989a47-1798-48a2-9568-9cb971a3b4a3" containerName="kserve-container" probeResult="failure" output="dial tcp 10.133.0.40:8080: connect: connection refused"
Sep 17 17:56:37.072717 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:56:37.072671    2565 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/error-404-isvc-c5c33-predictor-6cd4d7bc49-lmkh5" podUID="b4205277-b668-41c9-974a-730d76d07f5a" containerName="kserve-container" probeResult="failure" output="dial tcp 10.133.0.43:8080: connect: connection refused"
Sep 17 17:56:37.072717 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:56:37.072681    2565 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/success-200-isvc-c5c33-predictor-7d76dcc468-tldgv" podUID="2e69c7c5-5af7-477f-92db-cea0c52943b1" containerName="kserve-container" probeResult="failure" output="dial tcp 10.133.0.42:8080: connect: connection refused"
Sep 17 17:56:42.990113 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:56:42.990074    2565 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/success-200-isvc-6b6c1-predictor-d7d64c7db-ntzbm" podUID="2b989a47-1798-48a2-9568-9cb971a3b4a3" containerName="kserve-container" probeResult="failure" output="dial tcp 10.133.0.40:8080: connect: connection refused"
Sep 17 17:56:42.990528 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:56:42.990081    2565 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/error-404-isvc-6b6c1-predictor-66c99c6495-hh86l" podUID="bd8084fe-ed00-4a05-a0a3-0ab206ba7278" containerName="kserve-container" probeResult="failure" output="dial tcp 10.133.0.41:8080: connect: connection refused"
Sep 17 17:56:47.073188 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:56:47.073154    2565 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/error-404-isvc-c5c33-predictor-6cd4d7bc49-lmkh5" podUID="b4205277-b668-41c9-974a-730d76d07f5a" containerName="kserve-container" probeResult="failure" output="dial tcp 10.133.0.43:8080: connect: connection refused"
Sep 17 17:56:47.073674 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:56:47.073153    2565 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/success-200-isvc-c5c33-predictor-7d76dcc468-tldgv" podUID="2e69c7c5-5af7-477f-92db-cea0c52943b1" containerName="kserve-container" probeResult="failure" output="dial tcp 10.133.0.42:8080: connect: connection refused"
Sep 17 17:56:52.989779 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:56:52.989741    2565 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/success-200-isvc-6b6c1-predictor-d7d64c7db-ntzbm" podUID="2b989a47-1798-48a2-9568-9cb971a3b4a3" containerName="kserve-container" probeResult="failure" output="dial tcp 10.133.0.40:8080: connect: connection refused"
Sep 17 17:56:52.990694 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:56:52.990676    2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kserve-ci-e2e-test/error-404-isvc-6b6c1-predictor-66c99c6495-hh86l"
Sep 17 17:56:57.073611 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:56:57.073575    2565 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/error-404-isvc-c5c33-predictor-6cd4d7bc49-lmkh5" podUID="b4205277-b668-41c9-974a-730d76d07f5a" containerName="kserve-container" probeResult="failure" output="dial tcp 10.133.0.43:8080: connect: connection refused"
Sep 17 17:56:57.073998 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:56:57.073582    2565 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/success-200-isvc-c5c33-predictor-7d76dcc468-tldgv" podUID="2e69c7c5-5af7-477f-92db-cea0c52943b1" containerName="kserve-container" probeResult="failure" output="dial tcp 10.133.0.42:8080: connect: connection refused"
Sep 17 17:57:02.990627 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:57:02.990594    2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kserve-ci-e2e-test/success-200-isvc-6b6c1-predictor-d7d64c7db-ntzbm"
Sep 17 17:57:07.073531 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:57:07.073503    2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kserve-ci-e2e-test/success-200-isvc-c5c33-predictor-7d76dcc468-tldgv"
Sep 17 17:57:07.073967 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:57:07.073581    2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kserve-ci-e2e-test/error-404-isvc-c5c33-predictor-6cd4d7bc49-lmkh5"
Sep 17 17:57:29.212006 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:57:29.211974    2565 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kserve-ci-e2e-test/success-200-isvc-6b6c1-predictor-d7d64c7db-ntzbm"]
Sep 17 17:57:29.212468 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:57:29.212229    2565 kuberuntime_container.go:873] "Killing container with a grace period" pod="kserve-ci-e2e-test/success-200-isvc-6b6c1-predictor-d7d64c7db-ntzbm" podUID="2b989a47-1798-48a2-9568-9cb971a3b4a3" containerName="kserve-container" containerID="cri-o://abecdf9e34ea83e8a7cef2a4e678f3f82980a47421f6ec394545a95b1b4f94ee" gracePeriod=30
Sep 17 17:57:29.212468 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:57:29.212334    2565 kuberuntime_container.go:873] "Killing container with a grace period" pod="kserve-ci-e2e-test/success-200-isvc-6b6c1-predictor-d7d64c7db-ntzbm" podUID="2b989a47-1798-48a2-9568-9cb971a3b4a3" containerName="kube-rbac-proxy" containerID="cri-o://007da526c833243dfa6b5f7fd58341ead2cdee320756e467eb759e7370437600" gracePeriod=30
Sep 17 17:57:29.243956 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:57:29.243927    2565 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kserve-ci-e2e-test/error-404-isvc-6b6c1-predictor-66c99c6495-hh86l"]
Sep 17 17:57:29.244183 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:57:29.244161    2565 kuberuntime_container.go:873] "Killing container with a grace period" pod="kserve-ci-e2e-test/error-404-isvc-6b6c1-predictor-66c99c6495-hh86l" podUID="bd8084fe-ed00-4a05-a0a3-0ab206ba7278" containerName="kserve-container" containerID="cri-o://607c5c0f936659f69727d94518d033bcc2a01ae9bda86e43eb5a699a99a0380f" gracePeriod=30
Sep 17 17:57:29.244275 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:57:29.244253    2565 kuberuntime_container.go:873] "Killing container with a grace period" pod="kserve-ci-e2e-test/error-404-isvc-6b6c1-predictor-66c99c6495-hh86l" podUID="bd8084fe-ed00-4a05-a0a3-0ab206ba7278" containerName="kube-rbac-proxy" containerID="cri-o://f3b9639ae256937b3ef622512dccac7171527c15ded42b18ff9c03b15d919b72" gracePeriod=30
Sep 17 17:57:29.349983 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:57:29.349946    2565 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kserve-ci-e2e-test/success-200-isvc-f13ae-predictor-b68795d4b-lvvqr"]
Sep 17 17:57:29.350411 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:57:29.350389    2565 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a598baa1-44e2-4711-9d08-a245a368b3eb" containerName="kserve-container"
Sep 17 17:57:29.350411 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:57:29.350412    2565 state_mem.go:107] "Deleted CPUSet assignment" podUID="a598baa1-44e2-4711-9d08-a245a368b3eb" containerName="kserve-container"
Sep 17 17:57:29.350578 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:57:29.350436    2565 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a598baa1-44e2-4711-9d08-a245a368b3eb" containerName="kube-rbac-proxy"
Sep 17 17:57:29.350578 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:57:29.350445    2565 state_mem.go:107] "Deleted CPUSet assignment" podUID="a598baa1-44e2-4711-9d08-a245a368b3eb" containerName="kube-rbac-proxy"
Sep 17 17:57:29.350578 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:57:29.350459    2565 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="64036856-d29a-4195-ba9d-dcd53de9cd4a" containerName="kserve-container"
Sep 17 17:57:29.350578 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:57:29.350468    2565 state_mem.go:107] "Deleted CPUSet assignment" podUID="64036856-d29a-4195-ba9d-dcd53de9cd4a" containerName="kserve-container"
Sep 17 17:57:29.350578 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:57:29.350487    2565 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="64036856-d29a-4195-ba9d-dcd53de9cd4a" containerName="kube-rbac-proxy"
Sep 17 17:57:29.350578 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:57:29.350492    2565 state_mem.go:107] "Deleted CPUSet assignment" podUID="64036856-d29a-4195-ba9d-dcd53de9cd4a" containerName="kube-rbac-proxy"
Sep 17 17:57:29.350578 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:57:29.350555    2565 memory_manager.go:368] "RemoveStaleState removing state" podUID="64036856-d29a-4195-ba9d-dcd53de9cd4a" containerName="kube-rbac-proxy"
Sep 17 17:57:29.350578 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:57:29.350567    2565 memory_manager.go:368] "RemoveStaleState removing state" podUID="a598baa1-44e2-4711-9d08-a245a368b3eb" containerName="kube-rbac-proxy"
Sep 17 17:57:29.350578 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:57:29.350576    2565 memory_manager.go:368] "RemoveStaleState removing state" podUID="64036856-d29a-4195-ba9d-dcd53de9cd4a" containerName="kserve-container"
Sep 17 17:57:29.350578 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:57:29.350582    2565 memory_manager.go:368] "RemoveStaleState removing state" podUID="a598baa1-44e2-4711-9d08-a245a368b3eb" containerName="kserve-container"
Sep 17 17:57:29.353563 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:57:29.353542    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/success-200-isvc-f13ae-predictor-b68795d4b-lvvqr"
Sep 17 17:57:29.356916 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:57:29.356894    2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kserve-ci-e2e-test\"/\"success-200-isvc-f13ae-predictor-serving-cert\""
Sep 17 17:57:29.357075 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:57:29.357056    2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kserve-ci-e2e-test\"/\"success-200-isvc-f13ae-kube-rbac-proxy-sar-config\""
Sep 17 17:57:29.366540 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:57:29.366519    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/success-200-isvc-f13ae-predictor-b68795d4b-lvvqr"]
Sep 17 17:57:29.382480 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:57:29.382450    2565 generic.go:356] "Generic (PLEG): container finished" podID="2b989a47-1798-48a2-9568-9cb971a3b4a3" containerID="007da526c833243dfa6b5f7fd58341ead2cdee320756e467eb759e7370437600" exitCode=2
Sep 17 17:57:29.382616 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:57:29.382489    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/success-200-isvc-6b6c1-predictor-d7d64c7db-ntzbm" event={"ID":"2b989a47-1798-48a2-9568-9cb971a3b4a3","Type":"ContainerDied","Data":"007da526c833243dfa6b5f7fd58341ead2cdee320756e467eb759e7370437600"}
Sep 17 17:57:29.387952 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:57:29.387918    2565 generic.go:356] "Generic (PLEG): container finished" podID="bd8084fe-ed00-4a05-a0a3-0ab206ba7278" containerID="f3b9639ae256937b3ef622512dccac7171527c15ded42b18ff9c03b15d919b72" exitCode=2
Sep 17 17:57:29.388097 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:57:29.387968    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/error-404-isvc-6b6c1-predictor-66c99c6495-hh86l" event={"ID":"bd8084fe-ed00-4a05-a0a3-0ab206ba7278","Type":"ContainerDied","Data":"f3b9639ae256937b3ef622512dccac7171527c15ded42b18ff9c03b15d919b72"}
Sep 17 17:57:29.464288 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:57:29.464206    2565 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kserve-ci-e2e-test/error-404-isvc-f13ae-predictor-776cc9696c-7jk49"]
Sep 17 17:57:29.468765 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:57:29.468737    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/error-404-isvc-f13ae-predictor-776cc9696c-7jk49"
Sep 17 17:57:29.469514 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:57:29.469150    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/error-404-isvc-f13ae-predictor-776cc9696c-7jk49"]
Sep 17 17:57:29.472063 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:57:29.471985    2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kserve-ci-e2e-test\"/\"error-404-isvc-f13ae-predictor-serving-cert\""
Sep 17 17:57:29.472063 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:57:29.472001    2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kserve-ci-e2e-test\"/\"error-404-isvc-f13ae-kube-rbac-proxy-sar-config\""
Sep 17 17:57:29.475538 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:57:29.475515    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/198a6626-b57d-460d-a29b-a7932f83006a-proxy-tls\") pod \"success-200-isvc-f13ae-predictor-b68795d4b-lvvqr\" (UID: \"198a6626-b57d-460d-a29b-a7932f83006a\") " pod="kserve-ci-e2e-test/success-200-isvc-f13ae-predictor-b68795d4b-lvvqr"
Sep 17 17:57:29.475643 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:57:29.475584    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-d5zhn\" (UniqueName: \"kubernetes.io/projected/198a6626-b57d-460d-a29b-a7932f83006a-kube-api-access-d5zhn\") pod \"success-200-isvc-f13ae-predictor-b68795d4b-lvvqr\" (UID: \"198a6626-b57d-460d-a29b-a7932f83006a\") " pod="kserve-ci-e2e-test/success-200-isvc-f13ae-predictor-b68795d4b-lvvqr"
Sep 17 17:57:29.475733 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:57:29.475672    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/198a6626-b57d-460d-a29b-a7932f83006a-kube-rbac-proxy-sar-config\") pod \"success-200-isvc-f13ae-predictor-b68795d4b-lvvqr\" (UID: \"198a6626-b57d-460d-a29b-a7932f83006a\") " pod="kserve-ci-e2e-test/success-200-isvc-f13ae-predictor-b68795d4b-lvvqr"
Sep 17 17:57:29.576596 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:57:29.576566    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/198a6626-b57d-460d-a29b-a7932f83006a-kube-rbac-proxy-sar-config\") pod \"success-200-isvc-f13ae-predictor-b68795d4b-lvvqr\" (UID: \"198a6626-b57d-460d-a29b-a7932f83006a\") " pod="kserve-ci-e2e-test/success-200-isvc-f13ae-predictor-b68795d4b-lvvqr"
Sep 17 17:57:29.576796 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:57:29.576612    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/198a6626-b57d-460d-a29b-a7932f83006a-proxy-tls\") pod \"success-200-isvc-f13ae-predictor-b68795d4b-lvvqr\" (UID: \"198a6626-b57d-460d-a29b-a7932f83006a\") " pod="kserve-ci-e2e-test/success-200-isvc-f13ae-predictor-b68795d4b-lvvqr"
Sep 17 17:57:29.576796 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:57:29.576650    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dbfg9\" (UniqueName: \"kubernetes.io/projected/8dfad140-1f0d-494d-825c-dd4e3391d778-kube-api-access-dbfg9\") pod \"error-404-isvc-f13ae-predictor-776cc9696c-7jk49\" (UID: \"8dfad140-1f0d-494d-825c-dd4e3391d778\") " pod="kserve-ci-e2e-test/error-404-isvc-f13ae-predictor-776cc9696c-7jk49"
Sep 17 17:57:29.576796 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:57:29.576673    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/8dfad140-1f0d-494d-825c-dd4e3391d778-kube-rbac-proxy-sar-config\") pod \"error-404-isvc-f13ae-predictor-776cc9696c-7jk49\" (UID: \"8dfad140-1f0d-494d-825c-dd4e3391d778\") " pod="kserve-ci-e2e-test/error-404-isvc-f13ae-predictor-776cc9696c-7jk49"
Sep 17 17:57:29.576796 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:57:29.576701    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/8dfad140-1f0d-494d-825c-dd4e3391d778-proxy-tls\") pod \"error-404-isvc-f13ae-predictor-776cc9696c-7jk49\" (UID: \"8dfad140-1f0d-494d-825c-dd4e3391d778\") " pod="kserve-ci-e2e-test/error-404-isvc-f13ae-predictor-776cc9696c-7jk49"
Sep 17 17:57:29.576796 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:57:29.576778    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-d5zhn\" (UniqueName: \"kubernetes.io/projected/198a6626-b57d-460d-a29b-a7932f83006a-kube-api-access-d5zhn\") pod \"success-200-isvc-f13ae-predictor-b68795d4b-lvvqr\" (UID: \"198a6626-b57d-460d-a29b-a7932f83006a\") " pod="kserve-ci-e2e-test/success-200-isvc-f13ae-predictor-b68795d4b-lvvqr"
Sep 17 17:57:29.577148 ip-10-0-137-20 kubenswrapper[2565]: E0917 17:57:29.577129    2565 secret.go:189] Couldn't get secret kserve-ci-e2e-test/success-200-isvc-f13ae-predictor-serving-cert: secret "success-200-isvc-f13ae-predictor-serving-cert" not found
Sep 17 17:57:29.577222 ip-10-0-137-20 kubenswrapper[2565]: E0917 17:57:29.577212    2565 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/198a6626-b57d-460d-a29b-a7932f83006a-proxy-tls podName:198a6626-b57d-460d-a29b-a7932f83006a nodeName:}" failed. No retries permitted until 2026-09-17 17:57:30.077190906 +0000 UTC m=+1399.954475308 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "proxy-tls" (UniqueName: "kubernetes.io/secret/198a6626-b57d-460d-a29b-a7932f83006a-proxy-tls") pod "success-200-isvc-f13ae-predictor-b68795d4b-lvvqr" (UID: "198a6626-b57d-460d-a29b-a7932f83006a") : secret "success-200-isvc-f13ae-predictor-serving-cert" not found
Sep 17 17:57:29.577370 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:57:29.577350    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/198a6626-b57d-460d-a29b-a7932f83006a-kube-rbac-proxy-sar-config\") pod \"success-200-isvc-f13ae-predictor-b68795d4b-lvvqr\" (UID: \"198a6626-b57d-460d-a29b-a7932f83006a\") " pod="kserve-ci-e2e-test/success-200-isvc-f13ae-predictor-b68795d4b-lvvqr"
Sep 17 17:57:29.589186 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:57:29.589155    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-d5zhn\" (UniqueName: \"kubernetes.io/projected/198a6626-b57d-460d-a29b-a7932f83006a-kube-api-access-d5zhn\") pod \"success-200-isvc-f13ae-predictor-b68795d4b-lvvqr\" (UID: \"198a6626-b57d-460d-a29b-a7932f83006a\") " pod="kserve-ci-e2e-test/success-200-isvc-f13ae-predictor-b68795d4b-lvvqr"
Sep 17 17:57:29.678115 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:57:29.678079    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dbfg9\" (UniqueName: \"kubernetes.io/projected/8dfad140-1f0d-494d-825c-dd4e3391d778-kube-api-access-dbfg9\") pod \"error-404-isvc-f13ae-predictor-776cc9696c-7jk49\" (UID: \"8dfad140-1f0d-494d-825c-dd4e3391d778\") " pod="kserve-ci-e2e-test/error-404-isvc-f13ae-predictor-776cc9696c-7jk49"
Sep 17 17:57:29.678115 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:57:29.678116    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/8dfad140-1f0d-494d-825c-dd4e3391d778-kube-rbac-proxy-sar-config\") pod \"error-404-isvc-f13ae-predictor-776cc9696c-7jk49\" (UID: \"8dfad140-1f0d-494d-825c-dd4e3391d778\") " pod="kserve-ci-e2e-test/error-404-isvc-f13ae-predictor-776cc9696c-7jk49"
Sep 17 17:57:29.678339 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:57:29.678142    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/8dfad140-1f0d-494d-825c-dd4e3391d778-proxy-tls\") pod \"error-404-isvc-f13ae-predictor-776cc9696c-7jk49\" (UID: \"8dfad140-1f0d-494d-825c-dd4e3391d778\") " pod="kserve-ci-e2e-test/error-404-isvc-f13ae-predictor-776cc9696c-7jk49"
Sep 17 17:57:29.678830 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:57:29.678810    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/8dfad140-1f0d-494d-825c-dd4e3391d778-kube-rbac-proxy-sar-config\") pod \"error-404-isvc-f13ae-predictor-776cc9696c-7jk49\" (UID: \"8dfad140-1f0d-494d-825c-dd4e3391d778\") " pod="kserve-ci-e2e-test/error-404-isvc-f13ae-predictor-776cc9696c-7jk49"
Sep 17 17:57:29.680799 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:57:29.680780    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/8dfad140-1f0d-494d-825c-dd4e3391d778-proxy-tls\") pod \"error-404-isvc-f13ae-predictor-776cc9696c-7jk49\" (UID: \"8dfad140-1f0d-494d-825c-dd4e3391d778\") " pod="kserve-ci-e2e-test/error-404-isvc-f13ae-predictor-776cc9696c-7jk49"
Sep 17 17:57:29.687202 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:57:29.687177    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-dbfg9\" (UniqueName: \"kubernetes.io/projected/8dfad140-1f0d-494d-825c-dd4e3391d778-kube-api-access-dbfg9\") pod \"error-404-isvc-f13ae-predictor-776cc9696c-7jk49\" (UID: \"8dfad140-1f0d-494d-825c-dd4e3391d778\") " pod="kserve-ci-e2e-test/error-404-isvc-f13ae-predictor-776cc9696c-7jk49"
Sep 17 17:57:29.781874 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:57:29.781784    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/error-404-isvc-f13ae-predictor-776cc9696c-7jk49"
Sep 17 17:57:30.058452 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:57:30.058425    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/error-404-isvc-f13ae-predictor-776cc9696c-7jk49"]
Sep 17 17:57:30.060254 ip-10-0-137-20 kubenswrapper[2565]: W0917 17:57:30.060229    2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8dfad140_1f0d_494d_825c_dd4e3391d778.slice/crio-579957a3a34c3ba72fd767e78feaee59d6d4ba081d6d8efa2e6ca70e9bf55f85 WatchSource:0}: Error finding container 579957a3a34c3ba72fd767e78feaee59d6d4ba081d6d8efa2e6ca70e9bf55f85: Status 404 returned error can't find the container with id 579957a3a34c3ba72fd767e78feaee59d6d4ba081d6d8efa2e6ca70e9bf55f85
Sep 17 17:57:30.081678 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:57:30.081656    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/198a6626-b57d-460d-a29b-a7932f83006a-proxy-tls\") pod \"success-200-isvc-f13ae-predictor-b68795d4b-lvvqr\" (UID: \"198a6626-b57d-460d-a29b-a7932f83006a\") " pod="kserve-ci-e2e-test/success-200-isvc-f13ae-predictor-b68795d4b-lvvqr"
Sep 17 17:57:30.083736 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:57:30.083718    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/198a6626-b57d-460d-a29b-a7932f83006a-proxy-tls\") pod \"success-200-isvc-f13ae-predictor-b68795d4b-lvvqr\" (UID: \"198a6626-b57d-460d-a29b-a7932f83006a\") " pod="kserve-ci-e2e-test/success-200-isvc-f13ae-predictor-b68795d4b-lvvqr"
Sep 17 17:57:30.265568 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:57:30.265535    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/success-200-isvc-f13ae-predictor-b68795d4b-lvvqr"
Sep 17 17:57:30.394796 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:57:30.394763    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/error-404-isvc-f13ae-predictor-776cc9696c-7jk49" event={"ID":"8dfad140-1f0d-494d-825c-dd4e3391d778","Type":"ContainerStarted","Data":"21a517bfe9d6a35c7331286c82b71f23ccc9772e109a45024c011864b337631d"}
Sep 17 17:57:30.394959 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:57:30.394827    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/error-404-isvc-f13ae-predictor-776cc9696c-7jk49" event={"ID":"8dfad140-1f0d-494d-825c-dd4e3391d778","Type":"ContainerStarted","Data":"fd1078748cae37103cff662c1e895e7b66c57e6ad873fa18b6abfec202fde992"}
Sep 17 17:57:30.394959 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:57:30.394855    2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve-ci-e2e-test/error-404-isvc-f13ae-predictor-776cc9696c-7jk49"
Sep 17 17:57:30.394959 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:57:30.394921    2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve-ci-e2e-test/error-404-isvc-f13ae-predictor-776cc9696c-7jk49"
Sep 17 17:57:30.395143 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:57:30.394983    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/error-404-isvc-f13ae-predictor-776cc9696c-7jk49" event={"ID":"8dfad140-1f0d-494d-825c-dd4e3391d778","Type":"ContainerStarted","Data":"579957a3a34c3ba72fd767e78feaee59d6d4ba081d6d8efa2e6ca70e9bf55f85"}
Sep 17 17:57:30.396280 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:57:30.396250    2565 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/error-404-isvc-f13ae-predictor-776cc9696c-7jk49" podUID="8dfad140-1f0d-494d-825c-dd4e3391d778" containerName="kserve-container" probeResult="failure" output="dial tcp 10.133.0.45:8080: connect: connection refused"
Sep 17 17:57:30.413962 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:57:30.413918    2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kserve-ci-e2e-test/error-404-isvc-f13ae-predictor-776cc9696c-7jk49" podStartSLOduration=1.413906608 podStartE2EDuration="1.413906608s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-17 17:57:29 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-17 17:57:30.412150351 +0000 UTC m=+1400.289434765" watchObservedRunningTime="2026-09-17 17:57:30.413906608 +0000 UTC m=+1400.291191097"
Sep 17 17:57:30.534943 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:57:30.534914    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/success-200-isvc-f13ae-predictor-b68795d4b-lvvqr"]
Sep 17 17:57:30.536628 ip-10-0-137-20 kubenswrapper[2565]: W0917 17:57:30.536606    2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod198a6626_b57d_460d_a29b_a7932f83006a.slice/crio-dafc1ac60778121edbcaa05f8d5c4020c9564f26545c91d3883d2824c65588ce WatchSource:0}: Error finding container dafc1ac60778121edbcaa05f8d5c4020c9564f26545c91d3883d2824c65588ce: Status 404 returned error can't find the container with id dafc1ac60778121edbcaa05f8d5c4020c9564f26545c91d3883d2824c65588ce
Sep 17 17:57:31.401791 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:57:31.401755    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/success-200-isvc-f13ae-predictor-b68795d4b-lvvqr" event={"ID":"198a6626-b57d-460d-a29b-a7932f83006a","Type":"ContainerStarted","Data":"9911f5896661c77a764c5fff86aab24ac273ebc407937917617519b5ab269988"}
Sep 17 17:57:31.402199 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:57:31.401822    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/success-200-isvc-f13ae-predictor-b68795d4b-lvvqr" event={"ID":"198a6626-b57d-460d-a29b-a7932f83006a","Type":"ContainerStarted","Data":"9538ba633a50e514bfacd8afbf87d81788890b1abc59d73e61331d880270ef14"}
Sep 17 17:57:31.402199 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:57:31.401861    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/success-200-isvc-f13ae-predictor-b68795d4b-lvvqr" event={"ID":"198a6626-b57d-460d-a29b-a7932f83006a","Type":"ContainerStarted","Data":"dafc1ac60778121edbcaa05f8d5c4020c9564f26545c91d3883d2824c65588ce"}
Sep 17 17:57:31.402199 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:57:31.402043    2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve-ci-e2e-test/success-200-isvc-f13ae-predictor-b68795d4b-lvvqr"
Sep 17 17:57:31.402199 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:57:31.402097    2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve-ci-e2e-test/success-200-isvc-f13ae-predictor-b68795d4b-lvvqr"
Sep 17 17:57:31.402414 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:57:31.402194    2565 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/error-404-isvc-f13ae-predictor-776cc9696c-7jk49" podUID="8dfad140-1f0d-494d-825c-dd4e3391d778" containerName="kserve-container" probeResult="failure" output="dial tcp 10.133.0.45:8080: connect: connection refused"
Sep 17 17:57:31.403430 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:57:31.403403    2565 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/success-200-isvc-f13ae-predictor-b68795d4b-lvvqr" podUID="198a6626-b57d-460d-a29b-a7932f83006a" containerName="kserve-container" probeResult="failure" output="dial tcp 10.133.0.44:8080: connect: connection refused"
Sep 17 17:57:31.420057 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:57:31.420001    2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kserve-ci-e2e-test/success-200-isvc-f13ae-predictor-b68795d4b-lvvqr" podStartSLOduration=2.419988479 podStartE2EDuration="2.419988479s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-17 17:57:29 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-17 17:57:31.419235297 +0000 UTC m=+1401.296519726" watchObservedRunningTime="2026-09-17 17:57:31.419988479 +0000 UTC m=+1401.297272893"
Sep 17 17:57:32.406283 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:57:32.406249    2565 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/success-200-isvc-f13ae-predictor-b68795d4b-lvvqr" podUID="198a6626-b57d-460d-a29b-a7932f83006a" containerName="kserve-container" probeResult="failure" output="dial tcp 10.133.0.44:8080: connect: connection refused"
Sep 17 17:57:32.592487 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:57:32.592462    2565 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/error-404-isvc-6b6c1-predictor-66c99c6495-hh86l"
Sep 17 17:57:32.705891 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:57:32.705867    2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/bd8084fe-ed00-4a05-a0a3-0ab206ba7278-proxy-tls\") pod \"bd8084fe-ed00-4a05-a0a3-0ab206ba7278\" (UID: \"bd8084fe-ed00-4a05-a0a3-0ab206ba7278\") "
Sep 17 17:57:32.706066 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:57:32.706049    2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-fl6zx\" (UniqueName: \"kubernetes.io/projected/bd8084fe-ed00-4a05-a0a3-0ab206ba7278-kube-api-access-fl6zx\") pod \"bd8084fe-ed00-4a05-a0a3-0ab206ba7278\" (UID: \"bd8084fe-ed00-4a05-a0a3-0ab206ba7278\") "
Sep 17 17:57:32.706121 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:57:32.706084    2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/bd8084fe-ed00-4a05-a0a3-0ab206ba7278-kube-rbac-proxy-sar-config\") pod \"bd8084fe-ed00-4a05-a0a3-0ab206ba7278\" (UID: \"bd8084fe-ed00-4a05-a0a3-0ab206ba7278\") "
Sep 17 17:57:32.706444 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:57:32.706419    2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/bd8084fe-ed00-4a05-a0a3-0ab206ba7278-kube-rbac-proxy-sar-config" (OuterVolumeSpecName: "kube-rbac-proxy-sar-config") pod "bd8084fe-ed00-4a05-a0a3-0ab206ba7278" (UID: "bd8084fe-ed00-4a05-a0a3-0ab206ba7278"). InnerVolumeSpecName "kube-rbac-proxy-sar-config". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 17 17:57:32.707794 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:57:32.707774    2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/bd8084fe-ed00-4a05-a0a3-0ab206ba7278-proxy-tls" (OuterVolumeSpecName: "proxy-tls") pod "bd8084fe-ed00-4a05-a0a3-0ab206ba7278" (UID: "bd8084fe-ed00-4a05-a0a3-0ab206ba7278"). InnerVolumeSpecName "proxy-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 17 17:57:32.707922 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:57:32.707904    2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/bd8084fe-ed00-4a05-a0a3-0ab206ba7278-kube-api-access-fl6zx" (OuterVolumeSpecName: "kube-api-access-fl6zx") pod "bd8084fe-ed00-4a05-a0a3-0ab206ba7278" (UID: "bd8084fe-ed00-4a05-a0a3-0ab206ba7278"). InnerVolumeSpecName "kube-api-access-fl6zx". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 17 17:57:32.806680 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:57:32.806653    2565 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-fl6zx\" (UniqueName: \"kubernetes.io/projected/bd8084fe-ed00-4a05-a0a3-0ab206ba7278-kube-api-access-fl6zx\") on node \"ip-10-0-137-20.ec2.internal\" DevicePath \"\""
Sep 17 17:57:32.806680 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:57:32.806679    2565 reconciler_common.go:299] "Volume detached for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/bd8084fe-ed00-4a05-a0a3-0ab206ba7278-kube-rbac-proxy-sar-config\") on node \"ip-10-0-137-20.ec2.internal\" DevicePath \"\""
Sep 17 17:57:32.806849 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:57:32.806690    2565 reconciler_common.go:299] "Volume detached for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/bd8084fe-ed00-4a05-a0a3-0ab206ba7278-proxy-tls\") on node \"ip-10-0-137-20.ec2.internal\" DevicePath \"\""
Sep 17 17:57:32.855895 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:57:32.855870    2565 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/success-200-isvc-6b6c1-predictor-d7d64c7db-ntzbm"
Sep 17 17:57:33.009048 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:57:33.008954    2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-lt5tp\" (UniqueName: \"kubernetes.io/projected/2b989a47-1798-48a2-9568-9cb971a3b4a3-kube-api-access-lt5tp\") pod \"2b989a47-1798-48a2-9568-9cb971a3b4a3\" (UID: \"2b989a47-1798-48a2-9568-9cb971a3b4a3\") "
Sep 17 17:57:33.009195 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:57:33.009097    2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/2b989a47-1798-48a2-9568-9cb971a3b4a3-proxy-tls\") pod \"2b989a47-1798-48a2-9568-9cb971a3b4a3\" (UID: \"2b989a47-1798-48a2-9568-9cb971a3b4a3\") "
Sep 17 17:57:33.009195 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:57:33.009128    2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/2b989a47-1798-48a2-9568-9cb971a3b4a3-kube-rbac-proxy-sar-config\") pod \"2b989a47-1798-48a2-9568-9cb971a3b4a3\" (UID: \"2b989a47-1798-48a2-9568-9cb971a3b4a3\") "
Sep 17 17:57:33.009525 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:57:33.009493    2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/2b989a47-1798-48a2-9568-9cb971a3b4a3-kube-rbac-proxy-sar-config" (OuterVolumeSpecName: "kube-rbac-proxy-sar-config") pod "2b989a47-1798-48a2-9568-9cb971a3b4a3" (UID: "2b989a47-1798-48a2-9568-9cb971a3b4a3"). InnerVolumeSpecName "kube-rbac-proxy-sar-config". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 17 17:57:33.011034 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:57:33.011005    2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/2b989a47-1798-48a2-9568-9cb971a3b4a3-proxy-tls" (OuterVolumeSpecName: "proxy-tls") pod "2b989a47-1798-48a2-9568-9cb971a3b4a3" (UID: "2b989a47-1798-48a2-9568-9cb971a3b4a3"). InnerVolumeSpecName "proxy-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 17 17:57:33.011120 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:57:33.011009    2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/2b989a47-1798-48a2-9568-9cb971a3b4a3-kube-api-access-lt5tp" (OuterVolumeSpecName: "kube-api-access-lt5tp") pod "2b989a47-1798-48a2-9568-9cb971a3b4a3" (UID: "2b989a47-1798-48a2-9568-9cb971a3b4a3"). InnerVolumeSpecName "kube-api-access-lt5tp". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 17 17:57:33.110353 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:57:33.110321    2565 reconciler_common.go:299] "Volume detached for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/2b989a47-1798-48a2-9568-9cb971a3b4a3-proxy-tls\") on node \"ip-10-0-137-20.ec2.internal\" DevicePath \"\""
Sep 17 17:57:33.110353 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:57:33.110346    2565 reconciler_common.go:299] "Volume detached for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/2b989a47-1798-48a2-9568-9cb971a3b4a3-kube-rbac-proxy-sar-config\") on node \"ip-10-0-137-20.ec2.internal\" DevicePath \"\""
Sep 17 17:57:33.110353 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:57:33.110355    2565 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-lt5tp\" (UniqueName: \"kubernetes.io/projected/2b989a47-1798-48a2-9568-9cb971a3b4a3-kube-api-access-lt5tp\") on node \"ip-10-0-137-20.ec2.internal\" DevicePath \"\""
Sep 17 17:57:33.411986 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:57:33.411953    2565 generic.go:356] "Generic (PLEG): container finished" podID="2b989a47-1798-48a2-9568-9cb971a3b4a3" containerID="abecdf9e34ea83e8a7cef2a4e678f3f82980a47421f6ec394545a95b1b4f94ee" exitCode=0
Sep 17 17:57:33.412394 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:57:33.411995    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/success-200-isvc-6b6c1-predictor-d7d64c7db-ntzbm" event={"ID":"2b989a47-1798-48a2-9568-9cb971a3b4a3","Type":"ContainerDied","Data":"abecdf9e34ea83e8a7cef2a4e678f3f82980a47421f6ec394545a95b1b4f94ee"}
Sep 17 17:57:33.412394 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:57:33.412038    2565 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/success-200-isvc-6b6c1-predictor-d7d64c7db-ntzbm"
Sep 17 17:57:33.412394 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:57:33.412113    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/success-200-isvc-6b6c1-predictor-d7d64c7db-ntzbm" event={"ID":"2b989a47-1798-48a2-9568-9cb971a3b4a3","Type":"ContainerDied","Data":"b8275188efb76ddbffae7bc51f5337b68f92a4ea416199e2b29dcc5a6edb76a4"}
Sep 17 17:57:33.412394 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:57:33.412158    2565 scope.go:117] "RemoveContainer" containerID="007da526c833243dfa6b5f7fd58341ead2cdee320756e467eb759e7370437600"
Sep 17 17:57:33.413713 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:57:33.413682    2565 generic.go:356] "Generic (PLEG): container finished" podID="bd8084fe-ed00-4a05-a0a3-0ab206ba7278" containerID="607c5c0f936659f69727d94518d033bcc2a01ae9bda86e43eb5a699a99a0380f" exitCode=0
Sep 17 17:57:33.413845 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:57:33.413733    2565 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/error-404-isvc-6b6c1-predictor-66c99c6495-hh86l"
Sep 17 17:57:33.413845 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:57:33.413769    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/error-404-isvc-6b6c1-predictor-66c99c6495-hh86l" event={"ID":"bd8084fe-ed00-4a05-a0a3-0ab206ba7278","Type":"ContainerDied","Data":"607c5c0f936659f69727d94518d033bcc2a01ae9bda86e43eb5a699a99a0380f"}
Sep 17 17:57:33.413845 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:57:33.413821    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/error-404-isvc-6b6c1-predictor-66c99c6495-hh86l" event={"ID":"bd8084fe-ed00-4a05-a0a3-0ab206ba7278","Type":"ContainerDied","Data":"18ba11dfa6abd743e5364784e1f32caec2e145150bc9048fd575daceb3bbf953"}
Sep 17 17:57:33.421119 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:57:33.421103    2565 scope.go:117] "RemoveContainer" containerID="abecdf9e34ea83e8a7cef2a4e678f3f82980a47421f6ec394545a95b1b4f94ee"
Sep 17 17:57:33.428842 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:57:33.428822    2565 scope.go:117] "RemoveContainer" containerID="007da526c833243dfa6b5f7fd58341ead2cdee320756e467eb759e7370437600"
Sep 17 17:57:33.429209 ip-10-0-137-20 kubenswrapper[2565]: E0917 17:57:33.429190    2565 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"007da526c833243dfa6b5f7fd58341ead2cdee320756e467eb759e7370437600\": container with ID starting with 007da526c833243dfa6b5f7fd58341ead2cdee320756e467eb759e7370437600 not found: ID does not exist" containerID="007da526c833243dfa6b5f7fd58341ead2cdee320756e467eb759e7370437600"
Sep 17 17:57:33.429268 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:57:33.429218    2565 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"007da526c833243dfa6b5f7fd58341ead2cdee320756e467eb759e7370437600"} err="failed to get container status \"007da526c833243dfa6b5f7fd58341ead2cdee320756e467eb759e7370437600\": rpc error: code = NotFound desc = could not find container \"007da526c833243dfa6b5f7fd58341ead2cdee320756e467eb759e7370437600\": container with ID starting with 007da526c833243dfa6b5f7fd58341ead2cdee320756e467eb759e7370437600 not found: ID does not exist"
Sep 17 17:57:33.429268 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:57:33.429233    2565 scope.go:117] "RemoveContainer" containerID="abecdf9e34ea83e8a7cef2a4e678f3f82980a47421f6ec394545a95b1b4f94ee"
Sep 17 17:57:33.429486 ip-10-0-137-20 kubenswrapper[2565]: E0917 17:57:33.429470    2565 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"abecdf9e34ea83e8a7cef2a4e678f3f82980a47421f6ec394545a95b1b4f94ee\": container with ID starting with abecdf9e34ea83e8a7cef2a4e678f3f82980a47421f6ec394545a95b1b4f94ee not found: ID does not exist" containerID="abecdf9e34ea83e8a7cef2a4e678f3f82980a47421f6ec394545a95b1b4f94ee"
Sep 17 17:57:33.429546 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:57:33.429496    2565 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"abecdf9e34ea83e8a7cef2a4e678f3f82980a47421f6ec394545a95b1b4f94ee"} err="failed to get container status \"abecdf9e34ea83e8a7cef2a4e678f3f82980a47421f6ec394545a95b1b4f94ee\": rpc error: code = NotFound desc = could not find container \"abecdf9e34ea83e8a7cef2a4e678f3f82980a47421f6ec394545a95b1b4f94ee\": container with ID starting with abecdf9e34ea83e8a7cef2a4e678f3f82980a47421f6ec394545a95b1b4f94ee not found: ID does not exist"
Sep 17 17:57:33.429546 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:57:33.429517    2565 scope.go:117] "RemoveContainer" containerID="f3b9639ae256937b3ef622512dccac7171527c15ded42b18ff9c03b15d919b72"
Sep 17 17:57:33.437085 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:57:33.437066    2565 scope.go:117] "RemoveContainer" containerID="607c5c0f936659f69727d94518d033bcc2a01ae9bda86e43eb5a699a99a0380f"
Sep 17 17:57:33.438264 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:57:33.438245    2565 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kserve-ci-e2e-test/error-404-isvc-6b6c1-predictor-66c99c6495-hh86l"]
Sep 17 17:57:33.442218 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:57:33.442197    2565 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kserve-ci-e2e-test/error-404-isvc-6b6c1-predictor-66c99c6495-hh86l"]
Sep 17 17:57:33.446082 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:57:33.446060    2565 scope.go:117] "RemoveContainer" containerID="f3b9639ae256937b3ef622512dccac7171527c15ded42b18ff9c03b15d919b72"
Sep 17 17:57:33.446396 ip-10-0-137-20 kubenswrapper[2565]: E0917 17:57:33.446373    2565 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"f3b9639ae256937b3ef622512dccac7171527c15ded42b18ff9c03b15d919b72\": container with ID starting with f3b9639ae256937b3ef622512dccac7171527c15ded42b18ff9c03b15d919b72 not found: ID does not exist" containerID="f3b9639ae256937b3ef622512dccac7171527c15ded42b18ff9c03b15d919b72"
Sep 17 17:57:33.446486 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:57:33.446401    2565 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"f3b9639ae256937b3ef622512dccac7171527c15ded42b18ff9c03b15d919b72"} err="failed to get container status \"f3b9639ae256937b3ef622512dccac7171527c15ded42b18ff9c03b15d919b72\": rpc error: code = NotFound desc = could not find container \"f3b9639ae256937b3ef622512dccac7171527c15ded42b18ff9c03b15d919b72\": container with ID starting with f3b9639ae256937b3ef622512dccac7171527c15ded42b18ff9c03b15d919b72 not found: ID does not exist"
Sep 17 17:57:33.446486 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:57:33.446418    2565 scope.go:117] "RemoveContainer" containerID="607c5c0f936659f69727d94518d033bcc2a01ae9bda86e43eb5a699a99a0380f"
Sep 17 17:57:33.446735 ip-10-0-137-20 kubenswrapper[2565]: E0917 17:57:33.446714    2565 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"607c5c0f936659f69727d94518d033bcc2a01ae9bda86e43eb5a699a99a0380f\": container with ID starting with 607c5c0f936659f69727d94518d033bcc2a01ae9bda86e43eb5a699a99a0380f not found: ID does not exist" containerID="607c5c0f936659f69727d94518d033bcc2a01ae9bda86e43eb5a699a99a0380f"
Sep 17 17:57:33.446779 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:57:33.446743    2565 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"607c5c0f936659f69727d94518d033bcc2a01ae9bda86e43eb5a699a99a0380f"} err="failed to get container status \"607c5c0f936659f69727d94518d033bcc2a01ae9bda86e43eb5a699a99a0380f\": rpc error: code = NotFound desc = could not find container \"607c5c0f936659f69727d94518d033bcc2a01ae9bda86e43eb5a699a99a0380f\": container with ID starting with 607c5c0f936659f69727d94518d033bcc2a01ae9bda86e43eb5a699a99a0380f not found: ID does not exist"
Sep 17 17:57:33.453925 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:57:33.453901    2565 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kserve-ci-e2e-test/success-200-isvc-6b6c1-predictor-d7d64c7db-ntzbm"]
Sep 17 17:57:33.456247 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:57:33.456227    2565 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kserve-ci-e2e-test/success-200-isvc-6b6c1-predictor-d7d64c7db-ntzbm"]
Sep 17 17:57:34.156762 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:57:34.156732    2565 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="2b989a47-1798-48a2-9568-9cb971a3b4a3" path="/var/lib/kubelet/pods/2b989a47-1798-48a2-9568-9cb971a3b4a3/volumes"
Sep 17 17:57:34.157175 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:57:34.157162    2565 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="bd8084fe-ed00-4a05-a0a3-0ab206ba7278" path="/var/lib/kubelet/pods/bd8084fe-ed00-4a05-a0a3-0ab206ba7278/volumes"
Sep 17 17:57:36.406530 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:57:36.406503    2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kserve-ci-e2e-test/error-404-isvc-f13ae-predictor-776cc9696c-7jk49"
Sep 17 17:57:36.407053 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:57:36.407006    2565 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/error-404-isvc-f13ae-predictor-776cc9696c-7jk49" podUID="8dfad140-1f0d-494d-825c-dd4e3391d778" containerName="kserve-container" probeResult="failure" output="dial tcp 10.133.0.45:8080: connect: connection refused"
Sep 17 17:57:37.410929 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:57:37.410900    2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kserve-ci-e2e-test/success-200-isvc-f13ae-predictor-b68795d4b-lvvqr"
Sep 17 17:57:37.411465 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:57:37.411438    2565 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/success-200-isvc-f13ae-predictor-b68795d4b-lvvqr" podUID="198a6626-b57d-460d-a29b-a7932f83006a" containerName="kserve-container" probeResult="failure" output="dial tcp 10.133.0.44:8080: connect: connection refused"
Sep 17 17:57:46.407478 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:57:46.407439    2565 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/error-404-isvc-f13ae-predictor-776cc9696c-7jk49" podUID="8dfad140-1f0d-494d-825c-dd4e3391d778" containerName="kserve-container" probeResult="failure" output="dial tcp 10.133.0.45:8080: connect: connection refused"
Sep 17 17:57:47.411559 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:57:47.411519    2565 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/success-200-isvc-f13ae-predictor-b68795d4b-lvvqr" podUID="198a6626-b57d-460d-a29b-a7932f83006a" containerName="kserve-container" probeResult="failure" output="dial tcp 10.133.0.44:8080: connect: connection refused"
Sep 17 17:57:56.407422 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:57:56.407336    2565 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/error-404-isvc-f13ae-predictor-776cc9696c-7jk49" podUID="8dfad140-1f0d-494d-825c-dd4e3391d778" containerName="kserve-container" probeResult="failure" output="dial tcp 10.133.0.45:8080: connect: connection refused"
Sep 17 17:57:57.411830 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:57:57.411788    2565 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/success-200-isvc-f13ae-predictor-b68795d4b-lvvqr" podUID="198a6626-b57d-460d-a29b-a7932f83006a" containerName="kserve-container" probeResult="failure" output="dial tcp 10.133.0.44:8080: connect: connection refused"
Sep 17 17:58:06.407296 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:58:06.407259    2565 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/error-404-isvc-f13ae-predictor-776cc9696c-7jk49" podUID="8dfad140-1f0d-494d-825c-dd4e3391d778" containerName="kserve-container" probeResult="failure" output="dial tcp 10.133.0.45:8080: connect: connection refused"
Sep 17 17:58:07.412174 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:58:07.412133    2565 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/success-200-isvc-f13ae-predictor-b68795d4b-lvvqr" podUID="198a6626-b57d-460d-a29b-a7932f83006a" containerName="kserve-container" probeResult="failure" output="dial tcp 10.133.0.44:8080: connect: connection refused"
Sep 17 17:58:16.408170 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:58:16.408141    2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kserve-ci-e2e-test/error-404-isvc-f13ae-predictor-776cc9696c-7jk49"
Sep 17 17:58:17.412620 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:58:17.412593    2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kserve-ci-e2e-test/success-200-isvc-f13ae-predictor-b68795d4b-lvvqr"
Sep 17 17:59:12.190393 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:59:12.190363    2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-d9927_30837de7-1447-4de2-ac62-c809813b3338/ovn-acl-logging/0.log"
Sep 17 17:59:12.192893 ip-10-0-137-20 kubenswrapper[2565]: I0917 17:59:12.192873    2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-d9927_30837de7-1447-4de2-ac62-c809813b3338/ovn-acl-logging/0.log"
Sep 17 18:04:12.218585 ip-10-0-137-20 kubenswrapper[2565]: I0917 18:04:12.218559    2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-d9927_30837de7-1447-4de2-ac62-c809813b3338/ovn-acl-logging/0.log"
Sep 17 18:04:12.221969 ip-10-0-137-20 kubenswrapper[2565]: I0917 18:04:12.221948    2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-d9927_30837de7-1447-4de2-ac62-c809813b3338/ovn-acl-logging/0.log"
Sep 17 18:05:33.446073 ip-10-0-137-20 kubenswrapper[2565]: I0917 18:05:33.445976    2565 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kserve-ci-e2e-test/success-200-isvc-c5c33-predictor-7d76dcc468-tldgv"]
Sep 17 18:05:33.446611 ip-10-0-137-20 kubenswrapper[2565]: I0917 18:05:33.446234    2565 kuberuntime_container.go:873] "Killing container with a grace period" pod="kserve-ci-e2e-test/success-200-isvc-c5c33-predictor-7d76dcc468-tldgv" podUID="2e69c7c5-5af7-477f-92db-cea0c52943b1" containerName="kserve-container" containerID="cri-o://90986357adc7fe3324f4fb73a2ffcad8942cf922d287b0895eb91c66ee7aa5dd" gracePeriod=30
Sep 17 18:05:33.446709 ip-10-0-137-20 kubenswrapper[2565]: I0917 18:05:33.446550    2565 kuberuntime_container.go:873] "Killing container with a grace period" pod="kserve-ci-e2e-test/success-200-isvc-c5c33-predictor-7d76dcc468-tldgv" podUID="2e69c7c5-5af7-477f-92db-cea0c52943b1" containerName="kube-rbac-proxy" containerID="cri-o://25a53fff0067927b5179739646e2f28de41773c61250b612e5296ab44f6464bf" gracePeriod=30
Sep 17 18:05:33.475149 ip-10-0-137-20 kubenswrapper[2565]: I0917 18:05:33.475118    2565 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kserve-ci-e2e-test/error-404-isvc-c5c33-predictor-6cd4d7bc49-lmkh5"]
Sep 17 18:05:33.475486 ip-10-0-137-20 kubenswrapper[2565]: I0917 18:05:33.475448    2565 kuberuntime_container.go:873] "Killing container with a grace period" pod="kserve-ci-e2e-test/error-404-isvc-c5c33-predictor-6cd4d7bc49-lmkh5" podUID="b4205277-b668-41c9-974a-730d76d07f5a" containerName="kserve-container" containerID="cri-o://a6eadae9e1c78c3ae81f71db5427b20ff0d56acff9802cf551a84d2cf3271b0b" gracePeriod=30
Sep 17 18:05:33.475618 ip-10-0-137-20 kubenswrapper[2565]: I0917 18:05:33.475458    2565 kuberuntime_container.go:873] "Killing container with a grace period" pod="kserve-ci-e2e-test/error-404-isvc-c5c33-predictor-6cd4d7bc49-lmkh5" podUID="b4205277-b668-41c9-974a-730d76d07f5a" containerName="kube-rbac-proxy" containerID="cri-o://1cf60e373868613cee2d199b3e1680b473bd7c227b09b1b826936b93661e1e82" gracePeriod=30
Sep 17 18:05:33.590240 ip-10-0-137-20 kubenswrapper[2565]: I0917 18:05:33.590213    2565 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kserve-ci-e2e-test/success-200-isvc-b5897-predictor-5dd89c9df6-pglvk"]
Sep 17 18:05:33.590637 ip-10-0-137-20 kubenswrapper[2565]: I0917 18:05:33.590624    2565 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="2b989a47-1798-48a2-9568-9cb971a3b4a3" containerName="kube-rbac-proxy"
Sep 17 18:05:33.590680 ip-10-0-137-20 kubenswrapper[2565]: I0917 18:05:33.590641    2565 state_mem.go:107] "Deleted CPUSet assignment" podUID="2b989a47-1798-48a2-9568-9cb971a3b4a3" containerName="kube-rbac-proxy"
Sep 17 18:05:33.590680 ip-10-0-137-20 kubenswrapper[2565]: I0917 18:05:33.590659    2565 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="2b989a47-1798-48a2-9568-9cb971a3b4a3" containerName="kserve-container"
Sep 17 18:05:33.590680 ip-10-0-137-20 kubenswrapper[2565]: I0917 18:05:33.590664    2565 state_mem.go:107] "Deleted CPUSet assignment" podUID="2b989a47-1798-48a2-9568-9cb971a3b4a3" containerName="kserve-container"
Sep 17 18:05:33.590680 ip-10-0-137-20 kubenswrapper[2565]: I0917 18:05:33.590672    2565 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="bd8084fe-ed00-4a05-a0a3-0ab206ba7278" containerName="kserve-container"
Sep 17 18:05:33.590680 ip-10-0-137-20 kubenswrapper[2565]: I0917 18:05:33.590677    2565 state_mem.go:107] "Deleted CPUSet assignment" podUID="bd8084fe-ed00-4a05-a0a3-0ab206ba7278" containerName="kserve-container"
Sep 17 18:05:33.590871 ip-10-0-137-20 kubenswrapper[2565]: I0917 18:05:33.590686    2565 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="bd8084fe-ed00-4a05-a0a3-0ab206ba7278" containerName="kube-rbac-proxy"
Sep 17 18:05:33.590871 ip-10-0-137-20 kubenswrapper[2565]: I0917 18:05:33.590692    2565 state_mem.go:107] "Deleted CPUSet assignment" podUID="bd8084fe-ed00-4a05-a0a3-0ab206ba7278" containerName="kube-rbac-proxy"
Sep 17 18:05:33.590871 ip-10-0-137-20 kubenswrapper[2565]: I0917 18:05:33.590762    2565 memory_manager.go:368] "RemoveStaleState removing state" podUID="bd8084fe-ed00-4a05-a0a3-0ab206ba7278" containerName="kserve-container"
Sep 17 18:05:33.590871 ip-10-0-137-20 kubenswrapper[2565]: I0917 18:05:33.590770    2565 memory_manager.go:368] "RemoveStaleState removing state" podUID="2b989a47-1798-48a2-9568-9cb971a3b4a3" containerName="kserve-container"
Sep 17 18:05:33.590871 ip-10-0-137-20 kubenswrapper[2565]: I0917 18:05:33.590777    2565 memory_manager.go:368] "RemoveStaleState removing state" podUID="2b989a47-1798-48a2-9568-9cb971a3b4a3" containerName="kube-rbac-proxy"
Sep 17 18:05:33.590871 ip-10-0-137-20 kubenswrapper[2565]: I0917 18:05:33.590783    2565 memory_manager.go:368] "RemoveStaleState removing state" podUID="bd8084fe-ed00-4a05-a0a3-0ab206ba7278" containerName="kube-rbac-proxy"
Sep 17 18:05:33.593805 ip-10-0-137-20 kubenswrapper[2565]: I0917 18:05:33.593789    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/success-200-isvc-b5897-predictor-5dd89c9df6-pglvk"
Sep 17 18:05:33.597307 ip-10-0-137-20 kubenswrapper[2565]: I0917 18:05:33.597287    2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kserve-ci-e2e-test\"/\"success-200-isvc-b5897-predictor-serving-cert\""
Sep 17 18:05:33.597438 ip-10-0-137-20 kubenswrapper[2565]: I0917 18:05:33.597415    2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kserve-ci-e2e-test\"/\"success-200-isvc-b5897-kube-rbac-proxy-sar-config\""
Sep 17 18:05:33.604951 ip-10-0-137-20 kubenswrapper[2565]: I0917 18:05:33.604918    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/success-200-isvc-b5897-predictor-5dd89c9df6-pglvk"]
Sep 17 18:05:33.606932 ip-10-0-137-20 kubenswrapper[2565]: I0917 18:05:33.606903    2565 generic.go:356] "Generic (PLEG): container finished" podID="b4205277-b668-41c9-974a-730d76d07f5a" containerID="1cf60e373868613cee2d199b3e1680b473bd7c227b09b1b826936b93661e1e82" exitCode=2
Sep 17 18:05:33.607056 ip-10-0-137-20 kubenswrapper[2565]: I0917 18:05:33.606937    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/error-404-isvc-c5c33-predictor-6cd4d7bc49-lmkh5" event={"ID":"b4205277-b668-41c9-974a-730d76d07f5a","Type":"ContainerDied","Data":"1cf60e373868613cee2d199b3e1680b473bd7c227b09b1b826936b93661e1e82"}
Sep 17 18:05:33.609914 ip-10-0-137-20 kubenswrapper[2565]: I0917 18:05:33.609866    2565 generic.go:356] "Generic (PLEG): container finished" podID="2e69c7c5-5af7-477f-92db-cea0c52943b1" containerID="25a53fff0067927b5179739646e2f28de41773c61250b612e5296ab44f6464bf" exitCode=2
Sep 17 18:05:33.610058 ip-10-0-137-20 kubenswrapper[2565]: I0917 18:05:33.609936    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/success-200-isvc-c5c33-predictor-7d76dcc468-tldgv" event={"ID":"2e69c7c5-5af7-477f-92db-cea0c52943b1","Type":"ContainerDied","Data":"25a53fff0067927b5179739646e2f28de41773c61250b612e5296ab44f6464bf"}
Sep 17 18:05:33.674194 ip-10-0-137-20 kubenswrapper[2565]: I0917 18:05:33.674148    2565 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kserve-ci-e2e-test/error-404-isvc-b5897-predictor-658695b98f-5vcmf"]
Sep 17 18:05:33.678408 ip-10-0-137-20 kubenswrapper[2565]: I0917 18:05:33.678390    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/error-404-isvc-b5897-predictor-658695b98f-5vcmf"
Sep 17 18:05:33.681364 ip-10-0-137-20 kubenswrapper[2565]: I0917 18:05:33.681341    2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kserve-ci-e2e-test\"/\"error-404-isvc-b5897-predictor-serving-cert\""
Sep 17 18:05:33.681481 ip-10-0-137-20 kubenswrapper[2565]: I0917 18:05:33.681343    2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kserve-ci-e2e-test\"/\"error-404-isvc-b5897-kube-rbac-proxy-sar-config\""
Sep 17 18:05:33.687378 ip-10-0-137-20 kubenswrapper[2565]: I0917 18:05:33.687360    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/error-404-isvc-b5897-predictor-658695b98f-5vcmf"]
Sep 17 18:05:33.730060 ip-10-0-137-20 kubenswrapper[2565]: I0917 18:05:33.729986    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/4a1f1981-8b92-46be-942d-e4657b35d2ce-kube-rbac-proxy-sar-config\") pod \"success-200-isvc-b5897-predictor-5dd89c9df6-pglvk\" (UID: \"4a1f1981-8b92-46be-942d-e4657b35d2ce\") " pod="kserve-ci-e2e-test/success-200-isvc-b5897-predictor-5dd89c9df6-pglvk"
Sep 17 18:05:33.730060 ip-10-0-137-20 kubenswrapper[2565]: I0917 18:05:33.730015    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/4a1f1981-8b92-46be-942d-e4657b35d2ce-proxy-tls\") pod \"success-200-isvc-b5897-predictor-5dd89c9df6-pglvk\" (UID: \"4a1f1981-8b92-46be-942d-e4657b35d2ce\") " pod="kserve-ci-e2e-test/success-200-isvc-b5897-predictor-5dd89c9df6-pglvk"
Sep 17 18:05:33.730240 ip-10-0-137-20 kubenswrapper[2565]: I0917 18:05:33.730178    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-frzmz\" (UniqueName: \"kubernetes.io/projected/4a1f1981-8b92-46be-942d-e4657b35d2ce-kube-api-access-frzmz\") pod \"success-200-isvc-b5897-predictor-5dd89c9df6-pglvk\" (UID: \"4a1f1981-8b92-46be-942d-e4657b35d2ce\") " pod="kserve-ci-e2e-test/success-200-isvc-b5897-predictor-5dd89c9df6-pglvk"
Sep 17 18:05:33.831578 ip-10-0-137-20 kubenswrapper[2565]: I0917 18:05:33.831541    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/772aca13-71ef-4f5a-b76b-82e7451ea20a-proxy-tls\") pod \"error-404-isvc-b5897-predictor-658695b98f-5vcmf\" (UID: \"772aca13-71ef-4f5a-b76b-82e7451ea20a\") " pod="kserve-ci-e2e-test/error-404-isvc-b5897-predictor-658695b98f-5vcmf"
Sep 17 18:05:33.831578 ip-10-0-137-20 kubenswrapper[2565]: I0917 18:05:33.831583    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-frzmz\" (UniqueName: \"kubernetes.io/projected/4a1f1981-8b92-46be-942d-e4657b35d2ce-kube-api-access-frzmz\") pod \"success-200-isvc-b5897-predictor-5dd89c9df6-pglvk\" (UID: \"4a1f1981-8b92-46be-942d-e4657b35d2ce\") " pod="kserve-ci-e2e-test/success-200-isvc-b5897-predictor-5dd89c9df6-pglvk"
Sep 17 18:05:33.831821 ip-10-0-137-20 kubenswrapper[2565]: I0917 18:05:33.831606    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7zqkx\" (UniqueName: \"kubernetes.io/projected/772aca13-71ef-4f5a-b76b-82e7451ea20a-kube-api-access-7zqkx\") pod \"error-404-isvc-b5897-predictor-658695b98f-5vcmf\" (UID: \"772aca13-71ef-4f5a-b76b-82e7451ea20a\") " pod="kserve-ci-e2e-test/error-404-isvc-b5897-predictor-658695b98f-5vcmf"
Sep 17 18:05:33.831821 ip-10-0-137-20 kubenswrapper[2565]: I0917 18:05:33.831659    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/772aca13-71ef-4f5a-b76b-82e7451ea20a-kube-rbac-proxy-sar-config\") pod \"error-404-isvc-b5897-predictor-658695b98f-5vcmf\" (UID: \"772aca13-71ef-4f5a-b76b-82e7451ea20a\") " pod="kserve-ci-e2e-test/error-404-isvc-b5897-predictor-658695b98f-5vcmf"
Sep 17 18:05:33.831821 ip-10-0-137-20 kubenswrapper[2565]: I0917 18:05:33.831760    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/4a1f1981-8b92-46be-942d-e4657b35d2ce-kube-rbac-proxy-sar-config\") pod \"success-200-isvc-b5897-predictor-5dd89c9df6-pglvk\" (UID: \"4a1f1981-8b92-46be-942d-e4657b35d2ce\") " pod="kserve-ci-e2e-test/success-200-isvc-b5897-predictor-5dd89c9df6-pglvk"
Sep 17 18:05:33.831821 ip-10-0-137-20 kubenswrapper[2565]: I0917 18:05:33.831797    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/4a1f1981-8b92-46be-942d-e4657b35d2ce-proxy-tls\") pod \"success-200-isvc-b5897-predictor-5dd89c9df6-pglvk\" (UID: \"4a1f1981-8b92-46be-942d-e4657b35d2ce\") " pod="kserve-ci-e2e-test/success-200-isvc-b5897-predictor-5dd89c9df6-pglvk"
Sep 17 18:05:33.832444 ip-10-0-137-20 kubenswrapper[2565]: I0917 18:05:33.832420    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/4a1f1981-8b92-46be-942d-e4657b35d2ce-kube-rbac-proxy-sar-config\") pod \"success-200-isvc-b5897-predictor-5dd89c9df6-pglvk\" (UID: \"4a1f1981-8b92-46be-942d-e4657b35d2ce\") " pod="kserve-ci-e2e-test/success-200-isvc-b5897-predictor-5dd89c9df6-pglvk"
Sep 17 18:05:33.834246 ip-10-0-137-20 kubenswrapper[2565]: I0917 18:05:33.834221    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/4a1f1981-8b92-46be-942d-e4657b35d2ce-proxy-tls\") pod \"success-200-isvc-b5897-predictor-5dd89c9df6-pglvk\" (UID: \"4a1f1981-8b92-46be-942d-e4657b35d2ce\") " pod="kserve-ci-e2e-test/success-200-isvc-b5897-predictor-5dd89c9df6-pglvk"
Sep 17 18:05:33.840393 ip-10-0-137-20 kubenswrapper[2565]: I0917 18:05:33.840374    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-frzmz\" (UniqueName: \"kubernetes.io/projected/4a1f1981-8b92-46be-942d-e4657b35d2ce-kube-api-access-frzmz\") pod \"success-200-isvc-b5897-predictor-5dd89c9df6-pglvk\" (UID: \"4a1f1981-8b92-46be-942d-e4657b35d2ce\") " pod="kserve-ci-e2e-test/success-200-isvc-b5897-predictor-5dd89c9df6-pglvk"
Sep 17 18:05:33.905322 ip-10-0-137-20 kubenswrapper[2565]: I0917 18:05:33.905295    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/success-200-isvc-b5897-predictor-5dd89c9df6-pglvk"
Sep 17 18:05:33.932641 ip-10-0-137-20 kubenswrapper[2565]: I0917 18:05:33.932609    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/772aca13-71ef-4f5a-b76b-82e7451ea20a-proxy-tls\") pod \"error-404-isvc-b5897-predictor-658695b98f-5vcmf\" (UID: \"772aca13-71ef-4f5a-b76b-82e7451ea20a\") " pod="kserve-ci-e2e-test/error-404-isvc-b5897-predictor-658695b98f-5vcmf"
Sep 17 18:05:33.932757 ip-10-0-137-20 kubenswrapper[2565]: I0917 18:05:33.932663    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7zqkx\" (UniqueName: \"kubernetes.io/projected/772aca13-71ef-4f5a-b76b-82e7451ea20a-kube-api-access-7zqkx\") pod \"error-404-isvc-b5897-predictor-658695b98f-5vcmf\" (UID: \"772aca13-71ef-4f5a-b76b-82e7451ea20a\") " pod="kserve-ci-e2e-test/error-404-isvc-b5897-predictor-658695b98f-5vcmf"
Sep 17 18:05:33.932757 ip-10-0-137-20 kubenswrapper[2565]: I0917 18:05:33.932727    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/772aca13-71ef-4f5a-b76b-82e7451ea20a-kube-rbac-proxy-sar-config\") pod \"error-404-isvc-b5897-predictor-658695b98f-5vcmf\" (UID: \"772aca13-71ef-4f5a-b76b-82e7451ea20a\") " pod="kserve-ci-e2e-test/error-404-isvc-b5897-predictor-658695b98f-5vcmf"
Sep 17 18:05:33.933526 ip-10-0-137-20 kubenswrapper[2565]: I0917 18:05:33.933503    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/772aca13-71ef-4f5a-b76b-82e7451ea20a-kube-rbac-proxy-sar-config\") pod \"error-404-isvc-b5897-predictor-658695b98f-5vcmf\" (UID: \"772aca13-71ef-4f5a-b76b-82e7451ea20a\") " pod="kserve-ci-e2e-test/error-404-isvc-b5897-predictor-658695b98f-5vcmf"
Sep 17 18:05:33.934881 ip-10-0-137-20 kubenswrapper[2565]: I0917 18:05:33.934864    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/772aca13-71ef-4f5a-b76b-82e7451ea20a-proxy-tls\") pod \"error-404-isvc-b5897-predictor-658695b98f-5vcmf\" (UID: \"772aca13-71ef-4f5a-b76b-82e7451ea20a\") " pod="kserve-ci-e2e-test/error-404-isvc-b5897-predictor-658695b98f-5vcmf"
Sep 17 18:05:33.941755 ip-10-0-137-20 kubenswrapper[2565]: I0917 18:05:33.941695    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-7zqkx\" (UniqueName: \"kubernetes.io/projected/772aca13-71ef-4f5a-b76b-82e7451ea20a-kube-api-access-7zqkx\") pod \"error-404-isvc-b5897-predictor-658695b98f-5vcmf\" (UID: \"772aca13-71ef-4f5a-b76b-82e7451ea20a\") " pod="kserve-ci-e2e-test/error-404-isvc-b5897-predictor-658695b98f-5vcmf"
Sep 17 18:05:33.990443 ip-10-0-137-20 kubenswrapper[2565]: I0917 18:05:33.990381    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/error-404-isvc-b5897-predictor-658695b98f-5vcmf"
Sep 17 18:05:34.179640 ip-10-0-137-20 kubenswrapper[2565]: I0917 18:05:34.179614    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/success-200-isvc-b5897-predictor-5dd89c9df6-pglvk"]
Sep 17 18:05:34.181091 ip-10-0-137-20 kubenswrapper[2565]: W0917 18:05:34.181063    2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4a1f1981_8b92_46be_942d_e4657b35d2ce.slice/crio-f0b0c64552c28e41ab4916c6e7f5bee51be687ca80ecf5859e6c9ac0f4386dc9 WatchSource:0}: Error finding container f0b0c64552c28e41ab4916c6e7f5bee51be687ca80ecf5859e6c9ac0f4386dc9: Status 404 returned error can't find the container with id f0b0c64552c28e41ab4916c6e7f5bee51be687ca80ecf5859e6c9ac0f4386dc9
Sep 17 18:05:34.261912 ip-10-0-137-20 kubenswrapper[2565]: I0917 18:05:34.261891    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/error-404-isvc-b5897-predictor-658695b98f-5vcmf"]
Sep 17 18:05:34.268988 ip-10-0-137-20 kubenswrapper[2565]: W0917 18:05:34.268960    2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod772aca13_71ef_4f5a_b76b_82e7451ea20a.slice/crio-9b43636c8103cb6ddd3cd006e35cff28c63b49339ff5c78b1afd6b869092b476 WatchSource:0}: Error finding container 9b43636c8103cb6ddd3cd006e35cff28c63b49339ff5c78b1afd6b869092b476: Status 404 returned error can't find the container with id 9b43636c8103cb6ddd3cd006e35cff28c63b49339ff5c78b1afd6b869092b476
Sep 17 18:05:34.616235 ip-10-0-137-20 kubenswrapper[2565]: I0917 18:05:34.616198    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/error-404-isvc-b5897-predictor-658695b98f-5vcmf" event={"ID":"772aca13-71ef-4f5a-b76b-82e7451ea20a","Type":"ContainerStarted","Data":"a465926da50d27237fccf625dfefe806871ddc631d1ee41b38bdbce24d251254"}
Sep 17 18:05:34.616669 ip-10-0-137-20 kubenswrapper[2565]: I0917 18:05:34.616274    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/error-404-isvc-b5897-predictor-658695b98f-5vcmf" event={"ID":"772aca13-71ef-4f5a-b76b-82e7451ea20a","Type":"ContainerStarted","Data":"caedce8b6d96f520b3522de57ba175185355a5d75e97ae32ba2039cf206f443d"}
Sep 17 18:05:34.616669 ip-10-0-137-20 kubenswrapper[2565]: I0917 18:05:34.616311    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/error-404-isvc-b5897-predictor-658695b98f-5vcmf" event={"ID":"772aca13-71ef-4f5a-b76b-82e7451ea20a","Type":"ContainerStarted","Data":"9b43636c8103cb6ddd3cd006e35cff28c63b49339ff5c78b1afd6b869092b476"}
Sep 17 18:05:34.616669 ip-10-0-137-20 kubenswrapper[2565]: I0917 18:05:34.616347    2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve-ci-e2e-test/error-404-isvc-b5897-predictor-658695b98f-5vcmf"
Sep 17 18:05:34.618050 ip-10-0-137-20 kubenswrapper[2565]: I0917 18:05:34.618010    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/success-200-isvc-b5897-predictor-5dd89c9df6-pglvk" event={"ID":"4a1f1981-8b92-46be-942d-e4657b35d2ce","Type":"ContainerStarted","Data":"0177156ca249c9b6850265215b16af7a519a4cd1cb2616d38f48dba76d6203d5"}
Sep 17 18:05:34.618170 ip-10-0-137-20 kubenswrapper[2565]: I0917 18:05:34.618068    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/success-200-isvc-b5897-predictor-5dd89c9df6-pglvk" event={"ID":"4a1f1981-8b92-46be-942d-e4657b35d2ce","Type":"ContainerStarted","Data":"8594cc9a84979afdb673a3a0f20cd6282289531167c6f2cd40603a279da75189"}
Sep 17 18:05:34.618170 ip-10-0-137-20 kubenswrapper[2565]: I0917 18:05:34.618097    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/success-200-isvc-b5897-predictor-5dd89c9df6-pglvk" event={"ID":"4a1f1981-8b92-46be-942d-e4657b35d2ce","Type":"ContainerStarted","Data":"f0b0c64552c28e41ab4916c6e7f5bee51be687ca80ecf5859e6c9ac0f4386dc9"}
Sep 17 18:05:34.618170 ip-10-0-137-20 kubenswrapper[2565]: I0917 18:05:34.618127    2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve-ci-e2e-test/success-200-isvc-b5897-predictor-5dd89c9df6-pglvk"
Sep 17 18:05:34.637312 ip-10-0-137-20 kubenswrapper[2565]: I0917 18:05:34.637268    2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kserve-ci-e2e-test/error-404-isvc-b5897-predictor-658695b98f-5vcmf" podStartSLOduration=1.637253874 podStartE2EDuration="1.637253874s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-17 18:05:33 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-17 18:05:34.636017699 +0000 UTC m=+1884.513302116" watchObservedRunningTime="2026-09-17 18:05:34.637253874 +0000 UTC m=+1884.514538288"
Sep 17 18:05:34.658718 ip-10-0-137-20 kubenswrapper[2565]: I0917 18:05:34.658672    2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kserve-ci-e2e-test/success-200-isvc-b5897-predictor-5dd89c9df6-pglvk" podStartSLOduration=1.658657783 podStartE2EDuration="1.658657783s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-17 18:05:33 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-17 18:05:34.658561529 +0000 UTC m=+1884.535845945" watchObservedRunningTime="2026-09-17 18:05:34.658657783 +0000 UTC m=+1884.535942198"
Sep 17 18:05:35.623321 ip-10-0-137-20 kubenswrapper[2565]: I0917 18:05:35.623283    2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve-ci-e2e-test/error-404-isvc-b5897-predictor-658695b98f-5vcmf"
Sep 17 18:05:35.623742 ip-10-0-137-20 kubenswrapper[2565]: I0917 18:05:35.623337    2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve-ci-e2e-test/success-200-isvc-b5897-predictor-5dd89c9df6-pglvk"
Sep 17 18:05:35.624327 ip-10-0-137-20 kubenswrapper[2565]: I0917 18:05:35.624295    2565 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/error-404-isvc-b5897-predictor-658695b98f-5vcmf" podUID="772aca13-71ef-4f5a-b76b-82e7451ea20a" containerName="kserve-container" probeResult="failure" output="dial tcp 10.133.0.47:8080: connect: connection refused"
Sep 17 18:05:35.624443 ip-10-0-137-20 kubenswrapper[2565]: I0917 18:05:35.624295    2565 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/success-200-isvc-b5897-predictor-5dd89c9df6-pglvk" podUID="4a1f1981-8b92-46be-942d-e4657b35d2ce" containerName="kserve-container" probeResult="failure" output="dial tcp 10.133.0.46:8080: connect: connection refused"
Sep 17 18:05:36.627842 ip-10-0-137-20 kubenswrapper[2565]: I0917 18:05:36.627783    2565 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/error-404-isvc-b5897-predictor-658695b98f-5vcmf" podUID="772aca13-71ef-4f5a-b76b-82e7451ea20a" containerName="kserve-container" probeResult="failure" output="dial tcp 10.133.0.47:8080: connect: connection refused"
Sep 17 18:05:36.627842 ip-10-0-137-20 kubenswrapper[2565]: I0917 18:05:36.627820    2565 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/success-200-isvc-b5897-predictor-5dd89c9df6-pglvk" podUID="4a1f1981-8b92-46be-942d-e4657b35d2ce" containerName="kserve-container" probeResult="failure" output="dial tcp 10.133.0.46:8080: connect: connection refused"
Sep 17 18:05:36.801162 ip-10-0-137-20 kubenswrapper[2565]: I0917 18:05:36.801140    2565 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/success-200-isvc-c5c33-predictor-7d76dcc468-tldgv"
Sep 17 18:05:36.831762 ip-10-0-137-20 kubenswrapper[2565]: I0917 18:05:36.831734    2565 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/error-404-isvc-c5c33-predictor-6cd4d7bc49-lmkh5"
Sep 17 18:05:36.962632 ip-10-0-137-20 kubenswrapper[2565]: I0917 18:05:36.962556    2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/b4205277-b668-41c9-974a-730d76d07f5a-kube-rbac-proxy-sar-config\") pod \"b4205277-b668-41c9-974a-730d76d07f5a\" (UID: \"b4205277-b668-41c9-974a-730d76d07f5a\") "
Sep 17 18:05:36.962632 ip-10-0-137-20 kubenswrapper[2565]: I0917 18:05:36.962589    2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/2e69c7c5-5af7-477f-92db-cea0c52943b1-kube-rbac-proxy-sar-config\") pod \"2e69c7c5-5af7-477f-92db-cea0c52943b1\" (UID: \"2e69c7c5-5af7-477f-92db-cea0c52943b1\") "
Sep 17 18:05:36.962632 ip-10-0-137-20 kubenswrapper[2565]: I0917 18:05:36.962616    2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/2e69c7c5-5af7-477f-92db-cea0c52943b1-proxy-tls\") pod \"2e69c7c5-5af7-477f-92db-cea0c52943b1\" (UID: \"2e69c7c5-5af7-477f-92db-cea0c52943b1\") "
Sep 17 18:05:36.962903 ip-10-0-137-20 kubenswrapper[2565]: I0917 18:05:36.962647    2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-p5rcj\" (UniqueName: \"kubernetes.io/projected/b4205277-b668-41c9-974a-730d76d07f5a-kube-api-access-p5rcj\") pod \"b4205277-b668-41c9-974a-730d76d07f5a\" (UID: \"b4205277-b668-41c9-974a-730d76d07f5a\") "
Sep 17 18:05:36.962903 ip-10-0-137-20 kubenswrapper[2565]: I0917 18:05:36.962750    2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-w9b29\" (UniqueName: \"kubernetes.io/projected/2e69c7c5-5af7-477f-92db-cea0c52943b1-kube-api-access-w9b29\") pod \"2e69c7c5-5af7-477f-92db-cea0c52943b1\" (UID: \"2e69c7c5-5af7-477f-92db-cea0c52943b1\") "
Sep 17 18:05:36.962903 ip-10-0-137-20 kubenswrapper[2565]: I0917 18:05:36.962789    2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/b4205277-b668-41c9-974a-730d76d07f5a-proxy-tls\") pod \"b4205277-b668-41c9-974a-730d76d07f5a\" (UID: \"b4205277-b668-41c9-974a-730d76d07f5a\") "
Sep 17 18:05:36.963086 ip-10-0-137-20 kubenswrapper[2565]: I0917 18:05:36.962988    2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/b4205277-b668-41c9-974a-730d76d07f5a-kube-rbac-proxy-sar-config" (OuterVolumeSpecName: "kube-rbac-proxy-sar-config") pod "b4205277-b668-41c9-974a-730d76d07f5a" (UID: "b4205277-b668-41c9-974a-730d76d07f5a"). InnerVolumeSpecName "kube-rbac-proxy-sar-config". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 17 18:05:36.963086 ip-10-0-137-20 kubenswrapper[2565]: I0917 18:05:36.962991    2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/2e69c7c5-5af7-477f-92db-cea0c52943b1-kube-rbac-proxy-sar-config" (OuterVolumeSpecName: "kube-rbac-proxy-sar-config") pod "2e69c7c5-5af7-477f-92db-cea0c52943b1" (UID: "2e69c7c5-5af7-477f-92db-cea0c52943b1"). InnerVolumeSpecName "kube-rbac-proxy-sar-config". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 17 18:05:36.963164 ip-10-0-137-20 kubenswrapper[2565]: I0917 18:05:36.963141    2565 reconciler_common.go:299] "Volume detached for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/b4205277-b668-41c9-974a-730d76d07f5a-kube-rbac-proxy-sar-config\") on node \"ip-10-0-137-20.ec2.internal\" DevicePath \"\""
Sep 17 18:05:36.963164 ip-10-0-137-20 kubenswrapper[2565]: I0917 18:05:36.963160    2565 reconciler_common.go:299] "Volume detached for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/2e69c7c5-5af7-477f-92db-cea0c52943b1-kube-rbac-proxy-sar-config\") on node \"ip-10-0-137-20.ec2.internal\" DevicePath \"\""
Sep 17 18:05:36.964736 ip-10-0-137-20 kubenswrapper[2565]: I0917 18:05:36.964711    2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/2e69c7c5-5af7-477f-92db-cea0c52943b1-proxy-tls" (OuterVolumeSpecName: "proxy-tls") pod "2e69c7c5-5af7-477f-92db-cea0c52943b1" (UID: "2e69c7c5-5af7-477f-92db-cea0c52943b1"). InnerVolumeSpecName "proxy-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 17 18:05:36.964842 ip-10-0-137-20 kubenswrapper[2565]: I0917 18:05:36.964823    2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/b4205277-b668-41c9-974a-730d76d07f5a-proxy-tls" (OuterVolumeSpecName: "proxy-tls") pod "b4205277-b668-41c9-974a-730d76d07f5a" (UID: "b4205277-b668-41c9-974a-730d76d07f5a"). InnerVolumeSpecName "proxy-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 17 18:05:36.965163 ip-10-0-137-20 kubenswrapper[2565]: I0917 18:05:36.965136    2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/2e69c7c5-5af7-477f-92db-cea0c52943b1-kube-api-access-w9b29" (OuterVolumeSpecName: "kube-api-access-w9b29") pod "2e69c7c5-5af7-477f-92db-cea0c52943b1" (UID: "2e69c7c5-5af7-477f-92db-cea0c52943b1"). InnerVolumeSpecName "kube-api-access-w9b29". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 17 18:05:36.965163 ip-10-0-137-20 kubenswrapper[2565]: I0917 18:05:36.965163    2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/b4205277-b668-41c9-974a-730d76d07f5a-kube-api-access-p5rcj" (OuterVolumeSpecName: "kube-api-access-p5rcj") pod "b4205277-b668-41c9-974a-730d76d07f5a" (UID: "b4205277-b668-41c9-974a-730d76d07f5a"). InnerVolumeSpecName "kube-api-access-p5rcj". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 17 18:05:37.064361 ip-10-0-137-20 kubenswrapper[2565]: I0917 18:05:37.064335    2565 reconciler_common.go:299] "Volume detached for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/b4205277-b668-41c9-974a-730d76d07f5a-proxy-tls\") on node \"ip-10-0-137-20.ec2.internal\" DevicePath \"\""
Sep 17 18:05:37.064361 ip-10-0-137-20 kubenswrapper[2565]: I0917 18:05:37.064356    2565 reconciler_common.go:299] "Volume detached for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/2e69c7c5-5af7-477f-92db-cea0c52943b1-proxy-tls\") on node \"ip-10-0-137-20.ec2.internal\" DevicePath \"\""
Sep 17 18:05:37.064361 ip-10-0-137-20 kubenswrapper[2565]: I0917 18:05:37.064367    2565 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-p5rcj\" (UniqueName: \"kubernetes.io/projected/b4205277-b668-41c9-974a-730d76d07f5a-kube-api-access-p5rcj\") on node \"ip-10-0-137-20.ec2.internal\" DevicePath \"\""
Sep 17 18:05:37.064542 ip-10-0-137-20 kubenswrapper[2565]: I0917 18:05:37.064376    2565 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-w9b29\" (UniqueName: \"kubernetes.io/projected/2e69c7c5-5af7-477f-92db-cea0c52943b1-kube-api-access-w9b29\") on node \"ip-10-0-137-20.ec2.internal\" DevicePath \"\""
Sep 17 18:05:37.633976 ip-10-0-137-20 kubenswrapper[2565]: I0917 18:05:37.633939    2565 generic.go:356] "Generic (PLEG): container finished" podID="2e69c7c5-5af7-477f-92db-cea0c52943b1" containerID="90986357adc7fe3324f4fb73a2ffcad8942cf922d287b0895eb91c66ee7aa5dd" exitCode=0
Sep 17 18:05:37.634437 ip-10-0-137-20 kubenswrapper[2565]: I0917 18:05:37.633983    2565 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/success-200-isvc-c5c33-predictor-7d76dcc468-tldgv"
Sep 17 18:05:37.634437 ip-10-0-137-20 kubenswrapper[2565]: I0917 18:05:37.634042    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/success-200-isvc-c5c33-predictor-7d76dcc468-tldgv" event={"ID":"2e69c7c5-5af7-477f-92db-cea0c52943b1","Type":"ContainerDied","Data":"90986357adc7fe3324f4fb73a2ffcad8942cf922d287b0895eb91c66ee7aa5dd"}
Sep 17 18:05:37.634437 ip-10-0-137-20 kubenswrapper[2565]: I0917 18:05:37.634113    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/success-200-isvc-c5c33-predictor-7d76dcc468-tldgv" event={"ID":"2e69c7c5-5af7-477f-92db-cea0c52943b1","Type":"ContainerDied","Data":"af157a82cd97ac678dbabed9a6d7b93a4d446de3eed179a59b70edd75a2d1b2e"}
Sep 17 18:05:37.634437 ip-10-0-137-20 kubenswrapper[2565]: I0917 18:05:37.634152    2565 scope.go:117] "RemoveContainer" containerID="25a53fff0067927b5179739646e2f28de41773c61250b612e5296ab44f6464bf"
Sep 17 18:05:37.635943 ip-10-0-137-20 kubenswrapper[2565]: I0917 18:05:37.635920    2565 generic.go:356] "Generic (PLEG): container finished" podID="b4205277-b668-41c9-974a-730d76d07f5a" containerID="a6eadae9e1c78c3ae81f71db5427b20ff0d56acff9802cf551a84d2cf3271b0b" exitCode=0
Sep 17 18:05:37.635943 ip-10-0-137-20 kubenswrapper[2565]: I0917 18:05:37.635940    2565 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/error-404-isvc-c5c33-predictor-6cd4d7bc49-lmkh5"
Sep 17 18:05:37.636150 ip-10-0-137-20 kubenswrapper[2565]: I0917 18:05:37.635965    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/error-404-isvc-c5c33-predictor-6cd4d7bc49-lmkh5" event={"ID":"b4205277-b668-41c9-974a-730d76d07f5a","Type":"ContainerDied","Data":"a6eadae9e1c78c3ae81f71db5427b20ff0d56acff9802cf551a84d2cf3271b0b"}
Sep 17 18:05:37.636150 ip-10-0-137-20 kubenswrapper[2565]: I0917 18:05:37.636033    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/error-404-isvc-c5c33-predictor-6cd4d7bc49-lmkh5" event={"ID":"b4205277-b668-41c9-974a-730d76d07f5a","Type":"ContainerDied","Data":"a2f6f67275ec8530b2769b21a192a7db5f631aa841c3f39c3ca29dfb1f71381f"}
Sep 17 18:05:37.642859 ip-10-0-137-20 kubenswrapper[2565]: I0917 18:05:37.642842    2565 scope.go:117] "RemoveContainer" containerID="90986357adc7fe3324f4fb73a2ffcad8942cf922d287b0895eb91c66ee7aa5dd"
Sep 17 18:05:37.652295 ip-10-0-137-20 kubenswrapper[2565]: I0917 18:05:37.652271    2565 scope.go:117] "RemoveContainer" containerID="25a53fff0067927b5179739646e2f28de41773c61250b612e5296ab44f6464bf"
Sep 17 18:05:37.652553 ip-10-0-137-20 kubenswrapper[2565]: E0917 18:05:37.652532    2565 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"25a53fff0067927b5179739646e2f28de41773c61250b612e5296ab44f6464bf\": container with ID starting with 25a53fff0067927b5179739646e2f28de41773c61250b612e5296ab44f6464bf not found: ID does not exist" containerID="25a53fff0067927b5179739646e2f28de41773c61250b612e5296ab44f6464bf"
Sep 17 18:05:37.652624 ip-10-0-137-20 kubenswrapper[2565]: I0917 18:05:37.652558    2565 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"25a53fff0067927b5179739646e2f28de41773c61250b612e5296ab44f6464bf"} err="failed to get container status \"25a53fff0067927b5179739646e2f28de41773c61250b612e5296ab44f6464bf\": rpc error: code = NotFound desc = could not find container \"25a53fff0067927b5179739646e2f28de41773c61250b612e5296ab44f6464bf\": container with ID starting with 25a53fff0067927b5179739646e2f28de41773c61250b612e5296ab44f6464bf not found: ID does not exist"
Sep 17 18:05:37.652624 ip-10-0-137-20 kubenswrapper[2565]: I0917 18:05:37.652583    2565 scope.go:117] "RemoveContainer" containerID="90986357adc7fe3324f4fb73a2ffcad8942cf922d287b0895eb91c66ee7aa5dd"
Sep 17 18:05:37.652890 ip-10-0-137-20 kubenswrapper[2565]: E0917 18:05:37.652872    2565 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"90986357adc7fe3324f4fb73a2ffcad8942cf922d287b0895eb91c66ee7aa5dd\": container with ID starting with 90986357adc7fe3324f4fb73a2ffcad8942cf922d287b0895eb91c66ee7aa5dd not found: ID does not exist" containerID="90986357adc7fe3324f4fb73a2ffcad8942cf922d287b0895eb91c66ee7aa5dd"
Sep 17 18:05:37.652955 ip-10-0-137-20 kubenswrapper[2565]: I0917 18:05:37.652891    2565 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"90986357adc7fe3324f4fb73a2ffcad8942cf922d287b0895eb91c66ee7aa5dd"} err="failed to get container status \"90986357adc7fe3324f4fb73a2ffcad8942cf922d287b0895eb91c66ee7aa5dd\": rpc error: code = NotFound desc = could not find container \"90986357adc7fe3324f4fb73a2ffcad8942cf922d287b0895eb91c66ee7aa5dd\": container with ID starting with 90986357adc7fe3324f4fb73a2ffcad8942cf922d287b0895eb91c66ee7aa5dd not found: ID does not exist"
Sep 17 18:05:37.652955 ip-10-0-137-20 kubenswrapper[2565]: I0917 18:05:37.652903    2565 scope.go:117] "RemoveContainer" containerID="1cf60e373868613cee2d199b3e1680b473bd7c227b09b1b826936b93661e1e82"
Sep 17 18:05:37.661111 ip-10-0-137-20 kubenswrapper[2565]: I0917 18:05:37.661094    2565 scope.go:117] "RemoveContainer" containerID="a6eadae9e1c78c3ae81f71db5427b20ff0d56acff9802cf551a84d2cf3271b0b"
Sep 17 18:05:37.662828 ip-10-0-137-20 kubenswrapper[2565]: I0917 18:05:37.662808    2565 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kserve-ci-e2e-test/success-200-isvc-c5c33-predictor-7d76dcc468-tldgv"]
Sep 17 18:05:37.666668 ip-10-0-137-20 kubenswrapper[2565]: I0917 18:05:37.666644    2565 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kserve-ci-e2e-test/success-200-isvc-c5c33-predictor-7d76dcc468-tldgv"]
Sep 17 18:05:37.670740 ip-10-0-137-20 kubenswrapper[2565]: I0917 18:05:37.670715    2565 scope.go:117] "RemoveContainer" containerID="1cf60e373868613cee2d199b3e1680b473bd7c227b09b1b826936b93661e1e82"
Sep 17 18:05:37.671191 ip-10-0-137-20 kubenswrapper[2565]: E0917 18:05:37.671167    2565 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"1cf60e373868613cee2d199b3e1680b473bd7c227b09b1b826936b93661e1e82\": container with ID starting with 1cf60e373868613cee2d199b3e1680b473bd7c227b09b1b826936b93661e1e82 not found: ID does not exist" containerID="1cf60e373868613cee2d199b3e1680b473bd7c227b09b1b826936b93661e1e82"
Sep 17 18:05:37.671278 ip-10-0-137-20 kubenswrapper[2565]: I0917 18:05:37.671195    2565 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"1cf60e373868613cee2d199b3e1680b473bd7c227b09b1b826936b93661e1e82"} err="failed to get container status \"1cf60e373868613cee2d199b3e1680b473bd7c227b09b1b826936b93661e1e82\": rpc error: code = NotFound desc = could not find container \"1cf60e373868613cee2d199b3e1680b473bd7c227b09b1b826936b93661e1e82\": container with ID starting with 1cf60e373868613cee2d199b3e1680b473bd7c227b09b1b826936b93661e1e82 not found: ID does not exist"
Sep 17 18:05:37.671278 ip-10-0-137-20 kubenswrapper[2565]: I0917 18:05:37.671214    2565 scope.go:117] "RemoveContainer" containerID="a6eadae9e1c78c3ae81f71db5427b20ff0d56acff9802cf551a84d2cf3271b0b"
Sep 17 18:05:37.671518 ip-10-0-137-20 kubenswrapper[2565]: E0917 18:05:37.671500    2565 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"a6eadae9e1c78c3ae81f71db5427b20ff0d56acff9802cf551a84d2cf3271b0b\": container with ID starting with a6eadae9e1c78c3ae81f71db5427b20ff0d56acff9802cf551a84d2cf3271b0b not found: ID does not exist" containerID="a6eadae9e1c78c3ae81f71db5427b20ff0d56acff9802cf551a84d2cf3271b0b"
Sep 17 18:05:37.671564 ip-10-0-137-20 kubenswrapper[2565]: I0917 18:05:37.671522    2565 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"a6eadae9e1c78c3ae81f71db5427b20ff0d56acff9802cf551a84d2cf3271b0b"} err="failed to get container status \"a6eadae9e1c78c3ae81f71db5427b20ff0d56acff9802cf551a84d2cf3271b0b\": rpc error: code = NotFound desc = could not find container \"a6eadae9e1c78c3ae81f71db5427b20ff0d56acff9802cf551a84d2cf3271b0b\": container with ID starting with a6eadae9e1c78c3ae81f71db5427b20ff0d56acff9802cf551a84d2cf3271b0b not found: ID does not exist"
Sep 17 18:05:37.677716 ip-10-0-137-20 kubenswrapper[2565]: I0917 18:05:37.677695    2565 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kserve-ci-e2e-test/error-404-isvc-c5c33-predictor-6cd4d7bc49-lmkh5"]
Sep 17 18:05:37.684439 ip-10-0-137-20 kubenswrapper[2565]: I0917 18:05:37.684420    2565 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kserve-ci-e2e-test/error-404-isvc-c5c33-predictor-6cd4d7bc49-lmkh5"]
Sep 17 18:05:38.157629 ip-10-0-137-20 kubenswrapper[2565]: I0917 18:05:38.157599    2565 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="2e69c7c5-5af7-477f-92db-cea0c52943b1" path="/var/lib/kubelet/pods/2e69c7c5-5af7-477f-92db-cea0c52943b1/volumes"
Sep 17 18:05:38.158045 ip-10-0-137-20 kubenswrapper[2565]: I0917 18:05:38.158015    2565 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="b4205277-b668-41c9-974a-730d76d07f5a" path="/var/lib/kubelet/pods/b4205277-b668-41c9-974a-730d76d07f5a/volumes"
Sep 17 18:05:41.632208 ip-10-0-137-20 kubenswrapper[2565]: I0917 18:05:41.632177    2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kserve-ci-e2e-test/error-404-isvc-b5897-predictor-658695b98f-5vcmf"
Sep 17 18:05:41.632683 ip-10-0-137-20 kubenswrapper[2565]: I0917 18:05:41.632260    2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kserve-ci-e2e-test/success-200-isvc-b5897-predictor-5dd89c9df6-pglvk"
Sep 17 18:05:41.632683 ip-10-0-137-20 kubenswrapper[2565]: I0917 18:05:41.632656    2565 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/success-200-isvc-b5897-predictor-5dd89c9df6-pglvk" podUID="4a1f1981-8b92-46be-942d-e4657b35d2ce" containerName="kserve-container" probeResult="failure" output="dial tcp 10.133.0.46:8080: connect: connection refused"
Sep 17 18:05:41.632798 ip-10-0-137-20 kubenswrapper[2565]: I0917 18:05:41.632656    2565 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/error-404-isvc-b5897-predictor-658695b98f-5vcmf" podUID="772aca13-71ef-4f5a-b76b-82e7451ea20a" containerName="kserve-container" probeResult="failure" output="dial tcp 10.133.0.47:8080: connect: connection refused"
Sep 17 18:05:51.632636 ip-10-0-137-20 kubenswrapper[2565]: I0917 18:05:51.632599    2565 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/success-200-isvc-b5897-predictor-5dd89c9df6-pglvk" podUID="4a1f1981-8b92-46be-942d-e4657b35d2ce" containerName="kserve-container" probeResult="failure" output="dial tcp 10.133.0.46:8080: connect: connection refused"
Sep 17 18:05:51.633047 ip-10-0-137-20 kubenswrapper[2565]: I0917 18:05:51.632599    2565 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/error-404-isvc-b5897-predictor-658695b98f-5vcmf" podUID="772aca13-71ef-4f5a-b76b-82e7451ea20a" containerName="kserve-container" probeResult="failure" output="dial tcp 10.133.0.47:8080: connect: connection refused"
Sep 17 18:06:01.632667 ip-10-0-137-20 kubenswrapper[2565]: I0917 18:06:01.632619    2565 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/success-200-isvc-b5897-predictor-5dd89c9df6-pglvk" podUID="4a1f1981-8b92-46be-942d-e4657b35d2ce" containerName="kserve-container" probeResult="failure" output="dial tcp 10.133.0.46:8080: connect: connection refused"
Sep 17 18:06:01.632667 ip-10-0-137-20 kubenswrapper[2565]: I0917 18:06:01.632628    2565 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/error-404-isvc-b5897-predictor-658695b98f-5vcmf" podUID="772aca13-71ef-4f5a-b76b-82e7451ea20a" containerName="kserve-container" probeResult="failure" output="dial tcp 10.133.0.47:8080: connect: connection refused"
Sep 17 18:06:11.633491 ip-10-0-137-20 kubenswrapper[2565]: I0917 18:06:11.633452    2565 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/error-404-isvc-b5897-predictor-658695b98f-5vcmf" podUID="772aca13-71ef-4f5a-b76b-82e7451ea20a" containerName="kserve-container" probeResult="failure" output="dial tcp 10.133.0.47:8080: connect: connection refused"
Sep 17 18:06:11.633968 ip-10-0-137-20 kubenswrapper[2565]: I0917 18:06:11.633452    2565 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/success-200-isvc-b5897-predictor-5dd89c9df6-pglvk" podUID="4a1f1981-8b92-46be-942d-e4657b35d2ce" containerName="kserve-container" probeResult="failure" output="dial tcp 10.133.0.46:8080: connect: connection refused"
Sep 17 18:06:21.633837 ip-10-0-137-20 kubenswrapper[2565]: I0917 18:06:21.633805    2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kserve-ci-e2e-test/error-404-isvc-b5897-predictor-658695b98f-5vcmf"
Sep 17 18:06:21.634362 ip-10-0-137-20 kubenswrapper[2565]: I0917 18:06:21.633896    2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kserve-ci-e2e-test/success-200-isvc-b5897-predictor-5dd89c9df6-pglvk"
Sep 17 18:06:43.776462 ip-10-0-137-20 kubenswrapper[2565]: I0917 18:06:43.776429    2565 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kserve-ci-e2e-test/success-200-isvc-b5897-predictor-5dd89c9df6-pglvk"]
Sep 17 18:06:43.776995 ip-10-0-137-20 kubenswrapper[2565]: I0917 18:06:43.776706    2565 kuberuntime_container.go:873] "Killing container with a grace period" pod="kserve-ci-e2e-test/success-200-isvc-b5897-predictor-5dd89c9df6-pglvk" podUID="4a1f1981-8b92-46be-942d-e4657b35d2ce" containerName="kserve-container" containerID="cri-o://8594cc9a84979afdb673a3a0f20cd6282289531167c6f2cd40603a279da75189" gracePeriod=30
Sep 17 18:06:43.776995 ip-10-0-137-20 kubenswrapper[2565]: I0917 18:06:43.776755    2565 kuberuntime_container.go:873] "Killing container with a grace period" pod="kserve-ci-e2e-test/success-200-isvc-b5897-predictor-5dd89c9df6-pglvk" podUID="4a1f1981-8b92-46be-942d-e4657b35d2ce" containerName="kube-rbac-proxy" containerID="cri-o://0177156ca249c9b6850265215b16af7a519a4cd1cb2616d38f48dba76d6203d5" gracePeriod=30
Sep 17 18:06:43.822972 ip-10-0-137-20 kubenswrapper[2565]: I0917 18:06:43.822883    2565 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kserve-ci-e2e-test/error-404-isvc-b5897-predictor-658695b98f-5vcmf"]
Sep 17 18:06:43.823296 ip-10-0-137-20 kubenswrapper[2565]: I0917 18:06:43.823201    2565 kuberuntime_container.go:873] "Killing container with a grace period" pod="kserve-ci-e2e-test/error-404-isvc-b5897-predictor-658695b98f-5vcmf" podUID="772aca13-71ef-4f5a-b76b-82e7451ea20a" containerName="kserve-container" containerID="cri-o://caedce8b6d96f520b3522de57ba175185355a5d75e97ae32ba2039cf206f443d" gracePeriod=30
Sep 17 18:06:43.823646 ip-10-0-137-20 kubenswrapper[2565]: I0917 18:06:43.823212    2565 kuberuntime_container.go:873] "Killing container with a grace period" pod="kserve-ci-e2e-test/error-404-isvc-b5897-predictor-658695b98f-5vcmf" podUID="772aca13-71ef-4f5a-b76b-82e7451ea20a" containerName="kube-rbac-proxy" containerID="cri-o://a465926da50d27237fccf625dfefe806871ddc631d1ee41b38bdbce24d251254" gracePeriod=30
Sep 17 18:06:43.862648 ip-10-0-137-20 kubenswrapper[2565]: I0917 18:06:43.862621    2565 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kserve-ci-e2e-test/success-200-isvc-3ef6b-predictor-77d7f67c46-4dshv"]
Sep 17 18:06:43.862991 ip-10-0-137-20 kubenswrapper[2565]: I0917 18:06:43.862980    2565 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b4205277-b668-41c9-974a-730d76d07f5a" containerName="kserve-container"
Sep 17 18:06:43.863052 ip-10-0-137-20 kubenswrapper[2565]: I0917 18:06:43.862993    2565 state_mem.go:107] "Deleted CPUSet assignment" podUID="b4205277-b668-41c9-974a-730d76d07f5a" containerName="kserve-container"
Sep 17 18:06:43.863052 ip-10-0-137-20 kubenswrapper[2565]: I0917 18:06:43.863002    2565 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b4205277-b668-41c9-974a-730d76d07f5a" containerName="kube-rbac-proxy"
Sep 17 18:06:43.863052 ip-10-0-137-20 kubenswrapper[2565]: I0917 18:06:43.863008    2565 state_mem.go:107] "Deleted CPUSet assignment" podUID="b4205277-b668-41c9-974a-730d76d07f5a" containerName="kube-rbac-proxy"
Sep 17 18:06:43.863052 ip-10-0-137-20 kubenswrapper[2565]: I0917 18:06:43.863032    2565 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="2e69c7c5-5af7-477f-92db-cea0c52943b1" containerName="kserve-container"
Sep 17 18:06:43.863052 ip-10-0-137-20 kubenswrapper[2565]: I0917 18:06:43.863041    2565 state_mem.go:107] "Deleted CPUSet assignment" podUID="2e69c7c5-5af7-477f-92db-cea0c52943b1" containerName="kserve-container"
Sep 17 18:06:43.863256 ip-10-0-137-20 kubenswrapper[2565]: I0917 18:06:43.863061    2565 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="2e69c7c5-5af7-477f-92db-cea0c52943b1" containerName="kube-rbac-proxy"
Sep 17 18:06:43.863256 ip-10-0-137-20 kubenswrapper[2565]: I0917 18:06:43.863068    2565 state_mem.go:107] "Deleted CPUSet assignment" podUID="2e69c7c5-5af7-477f-92db-cea0c52943b1" containerName="kube-rbac-proxy"
Sep 17 18:06:43.863256 ip-10-0-137-20 kubenswrapper[2565]: I0917 18:06:43.863120    2565 memory_manager.go:368] "RemoveStaleState removing state" podUID="2e69c7c5-5af7-477f-92db-cea0c52943b1" containerName="kube-rbac-proxy"
Sep 17 18:06:43.863256 ip-10-0-137-20 kubenswrapper[2565]: I0917 18:06:43.863130    2565 memory_manager.go:368] "RemoveStaleState removing state" podUID="b4205277-b668-41c9-974a-730d76d07f5a" containerName="kserve-container"
Sep 17 18:06:43.863256 ip-10-0-137-20 kubenswrapper[2565]: I0917 18:06:43.863136    2565 memory_manager.go:368] "RemoveStaleState removing state" podUID="2e69c7c5-5af7-477f-92db-cea0c52943b1" containerName="kserve-container"
Sep 17 18:06:43.863256 ip-10-0-137-20 kubenswrapper[2565]: I0917 18:06:43.863145    2565 memory_manager.go:368] "RemoveStaleState removing state" podUID="b4205277-b668-41c9-974a-730d76d07f5a" containerName="kube-rbac-proxy"
Sep 17 18:06:43.866339 ip-10-0-137-20 kubenswrapper[2565]: I0917 18:06:43.866322    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/success-200-isvc-3ef6b-predictor-77d7f67c46-4dshv"
Sep 17 18:06:43.870198 ip-10-0-137-20 kubenswrapper[2565]: I0917 18:06:43.870176    2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kserve-ci-e2e-test\"/\"success-200-isvc-3ef6b-kube-rbac-proxy-sar-config\""
Sep 17 18:06:43.870198 ip-10-0-137-20 kubenswrapper[2565]: I0917 18:06:43.870191    2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kserve-ci-e2e-test\"/\"success-200-isvc-3ef6b-predictor-serving-cert\""
Sep 17 18:06:43.880501 ip-10-0-137-20 kubenswrapper[2565]: I0917 18:06:43.880476    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/success-200-isvc-3ef6b-predictor-77d7f67c46-4dshv"]
Sep 17 18:06:43.906723 ip-10-0-137-20 kubenswrapper[2565]: I0917 18:06:43.906694    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-q5wpq\" (UniqueName: \"kubernetes.io/projected/7af4e9c8-bc7e-4cf3-a1b3-c28cfb1a0ff7-kube-api-access-q5wpq\") pod \"success-200-isvc-3ef6b-predictor-77d7f67c46-4dshv\" (UID: \"7af4e9c8-bc7e-4cf3-a1b3-c28cfb1a0ff7\") " pod="kserve-ci-e2e-test/success-200-isvc-3ef6b-predictor-77d7f67c46-4dshv"
Sep 17 18:06:43.906860 ip-10-0-137-20 kubenswrapper[2565]: I0917 18:06:43.906761    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/7af4e9c8-bc7e-4cf3-a1b3-c28cfb1a0ff7-kube-rbac-proxy-sar-config\") pod \"success-200-isvc-3ef6b-predictor-77d7f67c46-4dshv\" (UID: \"7af4e9c8-bc7e-4cf3-a1b3-c28cfb1a0ff7\") " pod="kserve-ci-e2e-test/success-200-isvc-3ef6b-predictor-77d7f67c46-4dshv"
Sep 17 18:06:43.906860 ip-10-0-137-20 kubenswrapper[2565]: I0917 18:06:43.906801    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/7af4e9c8-bc7e-4cf3-a1b3-c28cfb1a0ff7-proxy-tls\") pod \"success-200-isvc-3ef6b-predictor-77d7f67c46-4dshv\" (UID: \"7af4e9c8-bc7e-4cf3-a1b3-c28cfb1a0ff7\") " pod="kserve-ci-e2e-test/success-200-isvc-3ef6b-predictor-77d7f67c46-4dshv"
Sep 17 18:06:43.957748 ip-10-0-137-20 kubenswrapper[2565]: I0917 18:06:43.957712    2565 generic.go:356] "Generic (PLEG): container finished" podID="4a1f1981-8b92-46be-942d-e4657b35d2ce" containerID="0177156ca249c9b6850265215b16af7a519a4cd1cb2616d38f48dba76d6203d5" exitCode=2
Sep 17 18:06:43.957930 ip-10-0-137-20 kubenswrapper[2565]: I0917 18:06:43.957789    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/success-200-isvc-b5897-predictor-5dd89c9df6-pglvk" event={"ID":"4a1f1981-8b92-46be-942d-e4657b35d2ce","Type":"ContainerDied","Data":"0177156ca249c9b6850265215b16af7a519a4cd1cb2616d38f48dba76d6203d5"}
Sep 17 18:06:43.960668 ip-10-0-137-20 kubenswrapper[2565]: I0917 18:06:43.960639    2565 generic.go:356] "Generic (PLEG): container finished" podID="772aca13-71ef-4f5a-b76b-82e7451ea20a" containerID="a465926da50d27237fccf625dfefe806871ddc631d1ee41b38bdbce24d251254" exitCode=2
Sep 17 18:06:43.960790 ip-10-0-137-20 kubenswrapper[2565]: I0917 18:06:43.960686    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/error-404-isvc-b5897-predictor-658695b98f-5vcmf" event={"ID":"772aca13-71ef-4f5a-b76b-82e7451ea20a","Type":"ContainerDied","Data":"a465926da50d27237fccf625dfefe806871ddc631d1ee41b38bdbce24d251254"}
Sep 17 18:06:43.980471 ip-10-0-137-20 kubenswrapper[2565]: I0917 18:06:43.980444    2565 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kserve-ci-e2e-test/error-404-isvc-3ef6b-predictor-fcf5d54cf-rcnf9"]
Sep 17 18:06:43.983939 ip-10-0-137-20 kubenswrapper[2565]: I0917 18:06:43.983918    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/error-404-isvc-3ef6b-predictor-fcf5d54cf-rcnf9"
Sep 17 18:06:43.986544 ip-10-0-137-20 kubenswrapper[2565]: I0917 18:06:43.986524    2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kserve-ci-e2e-test\"/\"error-404-isvc-3ef6b-predictor-serving-cert\""
Sep 17 18:06:43.986647 ip-10-0-137-20 kubenswrapper[2565]: I0917 18:06:43.986557    2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kserve-ci-e2e-test\"/\"error-404-isvc-3ef6b-kube-rbac-proxy-sar-config\""
Sep 17 18:06:43.994102 ip-10-0-137-20 kubenswrapper[2565]: I0917 18:06:43.994076    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/error-404-isvc-3ef6b-predictor-fcf5d54cf-rcnf9"]
Sep 17 18:06:44.007925 ip-10-0-137-20 kubenswrapper[2565]: I0917 18:06:44.007903    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/aeb32779-c788-4434-8d88-5ec5d08e7144-kube-rbac-proxy-sar-config\") pod \"error-404-isvc-3ef6b-predictor-fcf5d54cf-rcnf9\" (UID: \"aeb32779-c788-4434-8d88-5ec5d08e7144\") " pod="kserve-ci-e2e-test/error-404-isvc-3ef6b-predictor-fcf5d54cf-rcnf9"
Sep 17 18:06:44.008068 ip-10-0-137-20 kubenswrapper[2565]: I0917 18:06:44.007957    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-q5wpq\" (UniqueName: \"kubernetes.io/projected/7af4e9c8-bc7e-4cf3-a1b3-c28cfb1a0ff7-kube-api-access-q5wpq\") pod \"success-200-isvc-3ef6b-predictor-77d7f67c46-4dshv\" (UID: \"7af4e9c8-bc7e-4cf3-a1b3-c28cfb1a0ff7\") " pod="kserve-ci-e2e-test/success-200-isvc-3ef6b-predictor-77d7f67c46-4dshv"
Sep 17 18:06:44.008148 ip-10-0-137-20 kubenswrapper[2565]: I0917 18:06:44.008118    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/aeb32779-c788-4434-8d88-5ec5d08e7144-proxy-tls\") pod \"error-404-isvc-3ef6b-predictor-fcf5d54cf-rcnf9\" (UID: \"aeb32779-c788-4434-8d88-5ec5d08e7144\") " pod="kserve-ci-e2e-test/error-404-isvc-3ef6b-predictor-fcf5d54cf-rcnf9"
Sep 17 18:06:44.008207 ip-10-0-137-20 kubenswrapper[2565]: I0917 18:06:44.008169    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/7af4e9c8-bc7e-4cf3-a1b3-c28cfb1a0ff7-kube-rbac-proxy-sar-config\") pod \"success-200-isvc-3ef6b-predictor-77d7f67c46-4dshv\" (UID: \"7af4e9c8-bc7e-4cf3-a1b3-c28cfb1a0ff7\") " pod="kserve-ci-e2e-test/success-200-isvc-3ef6b-predictor-77d7f67c46-4dshv"
Sep 17 18:06:44.008268 ip-10-0-137-20 kubenswrapper[2565]: I0917 18:06:44.008235    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-f6l72\" (UniqueName: \"kubernetes.io/projected/aeb32779-c788-4434-8d88-5ec5d08e7144-kube-api-access-f6l72\") pod \"error-404-isvc-3ef6b-predictor-fcf5d54cf-rcnf9\" (UID: \"aeb32779-c788-4434-8d88-5ec5d08e7144\") " pod="kserve-ci-e2e-test/error-404-isvc-3ef6b-predictor-fcf5d54cf-rcnf9"
Sep 17 18:06:44.008349 ip-10-0-137-20 kubenswrapper[2565]: I0917 18:06:44.008329    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/7af4e9c8-bc7e-4cf3-a1b3-c28cfb1a0ff7-proxy-tls\") pod \"success-200-isvc-3ef6b-predictor-77d7f67c46-4dshv\" (UID: \"7af4e9c8-bc7e-4cf3-a1b3-c28cfb1a0ff7\") " pod="kserve-ci-e2e-test/success-200-isvc-3ef6b-predictor-77d7f67c46-4dshv"
Sep 17 18:06:44.008919 ip-10-0-137-20 kubenswrapper[2565]: I0917 18:06:44.008896    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/7af4e9c8-bc7e-4cf3-a1b3-c28cfb1a0ff7-kube-rbac-proxy-sar-config\") pod \"success-200-isvc-3ef6b-predictor-77d7f67c46-4dshv\" (UID: \"7af4e9c8-bc7e-4cf3-a1b3-c28cfb1a0ff7\") " pod="kserve-ci-e2e-test/success-200-isvc-3ef6b-predictor-77d7f67c46-4dshv"
Sep 17 18:06:44.011195 ip-10-0-137-20 kubenswrapper[2565]: I0917 18:06:44.011171    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/7af4e9c8-bc7e-4cf3-a1b3-c28cfb1a0ff7-proxy-tls\") pod \"success-200-isvc-3ef6b-predictor-77d7f67c46-4dshv\" (UID: \"7af4e9c8-bc7e-4cf3-a1b3-c28cfb1a0ff7\") " pod="kserve-ci-e2e-test/success-200-isvc-3ef6b-predictor-77d7f67c46-4dshv"
Sep 17 18:06:44.018950 ip-10-0-137-20 kubenswrapper[2565]: I0917 18:06:44.018915    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-q5wpq\" (UniqueName: \"kubernetes.io/projected/7af4e9c8-bc7e-4cf3-a1b3-c28cfb1a0ff7-kube-api-access-q5wpq\") pod \"success-200-isvc-3ef6b-predictor-77d7f67c46-4dshv\" (UID: \"7af4e9c8-bc7e-4cf3-a1b3-c28cfb1a0ff7\") " pod="kserve-ci-e2e-test/success-200-isvc-3ef6b-predictor-77d7f67c46-4dshv"
Sep 17 18:06:44.109477 ip-10-0-137-20 kubenswrapper[2565]: I0917 18:06:44.109437    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/aeb32779-c788-4434-8d88-5ec5d08e7144-proxy-tls\") pod \"error-404-isvc-3ef6b-predictor-fcf5d54cf-rcnf9\" (UID: \"aeb32779-c788-4434-8d88-5ec5d08e7144\") " pod="kserve-ci-e2e-test/error-404-isvc-3ef6b-predictor-fcf5d54cf-rcnf9"
Sep 17 18:06:44.109649 ip-10-0-137-20 kubenswrapper[2565]: I0917 18:06:44.109495    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-f6l72\" (UniqueName: \"kubernetes.io/projected/aeb32779-c788-4434-8d88-5ec5d08e7144-kube-api-access-f6l72\") pod \"error-404-isvc-3ef6b-predictor-fcf5d54cf-rcnf9\" (UID: \"aeb32779-c788-4434-8d88-5ec5d08e7144\") " pod="kserve-ci-e2e-test/error-404-isvc-3ef6b-predictor-fcf5d54cf-rcnf9"
Sep 17 18:06:44.109649 ip-10-0-137-20 kubenswrapper[2565]: I0917 18:06:44.109567    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/aeb32779-c788-4434-8d88-5ec5d08e7144-kube-rbac-proxy-sar-config\") pod \"error-404-isvc-3ef6b-predictor-fcf5d54cf-rcnf9\" (UID: \"aeb32779-c788-4434-8d88-5ec5d08e7144\") " pod="kserve-ci-e2e-test/error-404-isvc-3ef6b-predictor-fcf5d54cf-rcnf9"
Sep 17 18:06:44.110383 ip-10-0-137-20 kubenswrapper[2565]: I0917 18:06:44.110334    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/aeb32779-c788-4434-8d88-5ec5d08e7144-kube-rbac-proxy-sar-config\") pod \"error-404-isvc-3ef6b-predictor-fcf5d54cf-rcnf9\" (UID: \"aeb32779-c788-4434-8d88-5ec5d08e7144\") " pod="kserve-ci-e2e-test/error-404-isvc-3ef6b-predictor-fcf5d54cf-rcnf9"
Sep 17 18:06:44.110383 ip-10-0-137-20 kubenswrapper[2565]: E0917 18:06:44.110374    2565 secret.go:189] Couldn't get secret kserve-ci-e2e-test/error-404-isvc-3ef6b-predictor-serving-cert: secret "error-404-isvc-3ef6b-predictor-serving-cert" not found
Sep 17 18:06:44.110557 ip-10-0-137-20 kubenswrapper[2565]: E0917 18:06:44.110448    2565 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/aeb32779-c788-4434-8d88-5ec5d08e7144-proxy-tls podName:aeb32779-c788-4434-8d88-5ec5d08e7144 nodeName:}" failed. No retries permitted until 2026-09-17 18:06:44.610432243 +0000 UTC m=+1954.487716649 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "proxy-tls" (UniqueName: "kubernetes.io/secret/aeb32779-c788-4434-8d88-5ec5d08e7144-proxy-tls") pod "error-404-isvc-3ef6b-predictor-fcf5d54cf-rcnf9" (UID: "aeb32779-c788-4434-8d88-5ec5d08e7144") : secret "error-404-isvc-3ef6b-predictor-serving-cert" not found
Sep 17 18:06:44.119578 ip-10-0-137-20 kubenswrapper[2565]: I0917 18:06:44.119552    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-f6l72\" (UniqueName: \"kubernetes.io/projected/aeb32779-c788-4434-8d88-5ec5d08e7144-kube-api-access-f6l72\") pod \"error-404-isvc-3ef6b-predictor-fcf5d54cf-rcnf9\" (UID: \"aeb32779-c788-4434-8d88-5ec5d08e7144\") " pod="kserve-ci-e2e-test/error-404-isvc-3ef6b-predictor-fcf5d54cf-rcnf9"
Sep 17 18:06:44.123674 ip-10-0-137-20 kubenswrapper[2565]: I0917 18:06:44.123648    2565 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kserve-ci-e2e-test/success-200-isvc-f13ae-predictor-b68795d4b-lvvqr"]
Sep 17 18:06:44.123939 ip-10-0-137-20 kubenswrapper[2565]: I0917 18:06:44.123913    2565 kuberuntime_container.go:873] "Killing container with a grace period" pod="kserve-ci-e2e-test/success-200-isvc-f13ae-predictor-b68795d4b-lvvqr" podUID="198a6626-b57d-460d-a29b-a7932f83006a" containerName="kserve-container" containerID="cri-o://9538ba633a50e514bfacd8afbf87d81788890b1abc59d73e61331d880270ef14" gracePeriod=30
Sep 17 18:06:44.124056 ip-10-0-137-20 kubenswrapper[2565]: I0917 18:06:44.123931    2565 kuberuntime_container.go:873] "Killing container with a grace period" pod="kserve-ci-e2e-test/success-200-isvc-f13ae-predictor-b68795d4b-lvvqr" podUID="198a6626-b57d-460d-a29b-a7932f83006a" containerName="kube-rbac-proxy" containerID="cri-o://9911f5896661c77a764c5fff86aab24ac273ebc407937917617519b5ab269988" gracePeriod=30
Sep 17 18:06:44.159051 ip-10-0-137-20 kubenswrapper[2565]: I0917 18:06:44.159005    2565 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kserve-ci-e2e-test/error-404-isvc-f13ae-predictor-776cc9696c-7jk49"]
Sep 17 18:06:44.159310 ip-10-0-137-20 kubenswrapper[2565]: I0917 18:06:44.159285    2565 kuberuntime_container.go:873] "Killing container with a grace period" pod="kserve-ci-e2e-test/error-404-isvc-f13ae-predictor-776cc9696c-7jk49" podUID="8dfad140-1f0d-494d-825c-dd4e3391d778" containerName="kserve-container" containerID="cri-o://fd1078748cae37103cff662c1e895e7b66c57e6ad873fa18b6abfec202fde992" gracePeriod=30
Sep 17 18:06:44.159392 ip-10-0-137-20 kubenswrapper[2565]: I0917 18:06:44.159319    2565 kuberuntime_container.go:873] "Killing container with a grace period" pod="kserve-ci-e2e-test/error-404-isvc-f13ae-predictor-776cc9696c-7jk49" podUID="8dfad140-1f0d-494d-825c-dd4e3391d778" containerName="kube-rbac-proxy" containerID="cri-o://21a517bfe9d6a35c7331286c82b71f23ccc9772e109a45024c011864b337631d" gracePeriod=30
Sep 17 18:06:44.176988 ip-10-0-137-20 kubenswrapper[2565]: I0917 18:06:44.176960    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/success-200-isvc-3ef6b-predictor-77d7f67c46-4dshv"
Sep 17 18:06:44.292340 ip-10-0-137-20 kubenswrapper[2565]: I0917 18:06:44.292307    2565 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kserve-ci-e2e-test/success-200-isvc-de145-predictor-7dbfbb5484-b92v8"]
Sep 17 18:06:44.296041 ip-10-0-137-20 kubenswrapper[2565]: I0917 18:06:44.296004    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/success-200-isvc-de145-predictor-7dbfbb5484-b92v8"
Sep 17 18:06:44.298627 ip-10-0-137-20 kubenswrapper[2565]: I0917 18:06:44.298600    2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kserve-ci-e2e-test\"/\"success-200-isvc-de145-predictor-serving-cert\""
Sep 17 18:06:44.298735 ip-10-0-137-20 kubenswrapper[2565]: I0917 18:06:44.298662    2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kserve-ci-e2e-test\"/\"success-200-isvc-de145-kube-rbac-proxy-sar-config\""
Sep 17 18:06:44.308034 ip-10-0-137-20 kubenswrapper[2565]: I0917 18:06:44.308000    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/success-200-isvc-de145-predictor-7dbfbb5484-b92v8"]
Sep 17 18:06:44.310914 ip-10-0-137-20 kubenswrapper[2565]: I0917 18:06:44.310889    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-589qc\" (UniqueName: \"kubernetes.io/projected/5a89f15e-8368-4d4e-8bfb-be3196e8a6ce-kube-api-access-589qc\") pod \"success-200-isvc-de145-predictor-7dbfbb5484-b92v8\" (UID: \"5a89f15e-8368-4d4e-8bfb-be3196e8a6ce\") " pod="kserve-ci-e2e-test/success-200-isvc-de145-predictor-7dbfbb5484-b92v8"
Sep 17 18:06:44.311042 ip-10-0-137-20 kubenswrapper[2565]: I0917 18:06:44.310942    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/5a89f15e-8368-4d4e-8bfb-be3196e8a6ce-proxy-tls\") pod \"success-200-isvc-de145-predictor-7dbfbb5484-b92v8\" (UID: \"5a89f15e-8368-4d4e-8bfb-be3196e8a6ce\") " pod="kserve-ci-e2e-test/success-200-isvc-de145-predictor-7dbfbb5484-b92v8"
Sep 17 18:06:44.311042 ip-10-0-137-20 kubenswrapper[2565]: I0917 18:06:44.310981    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/5a89f15e-8368-4d4e-8bfb-be3196e8a6ce-kube-rbac-proxy-sar-config\") pod \"success-200-isvc-de145-predictor-7dbfbb5484-b92v8\" (UID: \"5a89f15e-8368-4d4e-8bfb-be3196e8a6ce\") " pod="kserve-ci-e2e-test/success-200-isvc-de145-predictor-7dbfbb5484-b92v8"
Sep 17 18:06:44.409368 ip-10-0-137-20 kubenswrapper[2565]: I0917 18:06:44.409280    2565 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kserve-ci-e2e-test/error-404-isvc-de145-predictor-bfc475677-phjm7"]
Sep 17 18:06:44.411790 ip-10-0-137-20 kubenswrapper[2565]: I0917 18:06:44.411769    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/5a89f15e-8368-4d4e-8bfb-be3196e8a6ce-proxy-tls\") pod \"success-200-isvc-de145-predictor-7dbfbb5484-b92v8\" (UID: \"5a89f15e-8368-4d4e-8bfb-be3196e8a6ce\") " pod="kserve-ci-e2e-test/success-200-isvc-de145-predictor-7dbfbb5484-b92v8"
Sep 17 18:06:44.411921 ip-10-0-137-20 kubenswrapper[2565]: I0917 18:06:44.411800    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/5a89f15e-8368-4d4e-8bfb-be3196e8a6ce-kube-rbac-proxy-sar-config\") pod \"success-200-isvc-de145-predictor-7dbfbb5484-b92v8\" (UID: \"5a89f15e-8368-4d4e-8bfb-be3196e8a6ce\") " pod="kserve-ci-e2e-test/success-200-isvc-de145-predictor-7dbfbb5484-b92v8"
Sep 17 18:06:44.411921 ip-10-0-137-20 kubenswrapper[2565]: I0917 18:06:44.411892    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-589qc\" (UniqueName: \"kubernetes.io/projected/5a89f15e-8368-4d4e-8bfb-be3196e8a6ce-kube-api-access-589qc\") pod \"success-200-isvc-de145-predictor-7dbfbb5484-b92v8\" (UID: \"5a89f15e-8368-4d4e-8bfb-be3196e8a6ce\") " pod="kserve-ci-e2e-test/success-200-isvc-de145-predictor-7dbfbb5484-b92v8"
Sep 17 18:06:44.412056 ip-10-0-137-20 kubenswrapper[2565]: E0917 18:06:44.411945    2565 secret.go:189] Couldn't get secret kserve-ci-e2e-test/success-200-isvc-de145-predictor-serving-cert: secret "success-200-isvc-de145-predictor-serving-cert" not found
Sep 17 18:06:44.412056 ip-10-0-137-20 kubenswrapper[2565]: E0917 18:06:44.412012    2565 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/5a89f15e-8368-4d4e-8bfb-be3196e8a6ce-proxy-tls podName:5a89f15e-8368-4d4e-8bfb-be3196e8a6ce nodeName:}" failed. No retries permitted until 2026-09-17 18:06:44.911994143 +0000 UTC m=+1954.789278543 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "proxy-tls" (UniqueName: "kubernetes.io/secret/5a89f15e-8368-4d4e-8bfb-be3196e8a6ce-proxy-tls") pod "success-200-isvc-de145-predictor-7dbfbb5484-b92v8" (UID: "5a89f15e-8368-4d4e-8bfb-be3196e8a6ce") : secret "success-200-isvc-de145-predictor-serving-cert" not found
Sep 17 18:06:44.412531 ip-10-0-137-20 kubenswrapper[2565]: I0917 18:06:44.412511    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/5a89f15e-8368-4d4e-8bfb-be3196e8a6ce-kube-rbac-proxy-sar-config\") pod \"success-200-isvc-de145-predictor-7dbfbb5484-b92v8\" (UID: \"5a89f15e-8368-4d4e-8bfb-be3196e8a6ce\") " pod="kserve-ci-e2e-test/success-200-isvc-de145-predictor-7dbfbb5484-b92v8"
Sep 17 18:06:44.412890 ip-10-0-137-20 kubenswrapper[2565]: I0917 18:06:44.412875    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/error-404-isvc-de145-predictor-bfc475677-phjm7"
Sep 17 18:06:44.416036 ip-10-0-137-20 kubenswrapper[2565]: I0917 18:06:44.416003    2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kserve-ci-e2e-test\"/\"error-404-isvc-de145-predictor-serving-cert\""
Sep 17 18:06:44.416264 ip-10-0-137-20 kubenswrapper[2565]: I0917 18:06:44.416250    2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kserve-ci-e2e-test\"/\"error-404-isvc-de145-kube-rbac-proxy-sar-config\""
Sep 17 18:06:44.422440 ip-10-0-137-20 kubenswrapper[2565]: I0917 18:06:44.422395    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-589qc\" (UniqueName: \"kubernetes.io/projected/5a89f15e-8368-4d4e-8bfb-be3196e8a6ce-kube-api-access-589qc\") pod \"success-200-isvc-de145-predictor-7dbfbb5484-b92v8\" (UID: \"5a89f15e-8368-4d4e-8bfb-be3196e8a6ce\") " pod="kserve-ci-e2e-test/success-200-isvc-de145-predictor-7dbfbb5484-b92v8"
Sep 17 18:06:44.425261 ip-10-0-137-20 kubenswrapper[2565]: I0917 18:06:44.425238    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/error-404-isvc-de145-predictor-bfc475677-phjm7"]
Sep 17 18:06:44.458504 ip-10-0-137-20 kubenswrapper[2565]: I0917 18:06:44.458482    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/success-200-isvc-3ef6b-predictor-77d7f67c46-4dshv"]
Sep 17 18:06:44.459992 ip-10-0-137-20 kubenswrapper[2565]: W0917 18:06:44.459961    2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7af4e9c8_bc7e_4cf3_a1b3_c28cfb1a0ff7.slice/crio-84a87a978097ce61e61df4e6471bd14c775950eb41fda95a03a223450fc4c273 WatchSource:0}: Error finding container 84a87a978097ce61e61df4e6471bd14c775950eb41fda95a03a223450fc4c273: Status 404 returned error can't find the container with id 84a87a978097ce61e61df4e6471bd14c775950eb41fda95a03a223450fc4c273
Sep 17 18:06:44.516359 ip-10-0-137-20 kubenswrapper[2565]: I0917 18:06:44.516320    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/453eb767-5a5e-479c-8079-59038d055f71-proxy-tls\") pod \"error-404-isvc-de145-predictor-bfc475677-phjm7\" (UID: \"453eb767-5a5e-479c-8079-59038d055f71\") " pod="kserve-ci-e2e-test/error-404-isvc-de145-predictor-bfc475677-phjm7"
Sep 17 18:06:44.516503 ip-10-0-137-20 kubenswrapper[2565]: I0917 18:06:44.516430    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/453eb767-5a5e-479c-8079-59038d055f71-kube-rbac-proxy-sar-config\") pod \"error-404-isvc-de145-predictor-bfc475677-phjm7\" (UID: \"453eb767-5a5e-479c-8079-59038d055f71\") " pod="kserve-ci-e2e-test/error-404-isvc-de145-predictor-bfc475677-phjm7"
Sep 17 18:06:44.516503 ip-10-0-137-20 kubenswrapper[2565]: I0917 18:06:44.516475    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7t972\" (UniqueName: \"kubernetes.io/projected/453eb767-5a5e-479c-8079-59038d055f71-kube-api-access-7t972\") pod \"error-404-isvc-de145-predictor-bfc475677-phjm7\" (UID: \"453eb767-5a5e-479c-8079-59038d055f71\") " pod="kserve-ci-e2e-test/error-404-isvc-de145-predictor-bfc475677-phjm7"
Sep 17 18:06:44.617236 ip-10-0-137-20 kubenswrapper[2565]: I0917 18:06:44.617209    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/453eb767-5a5e-479c-8079-59038d055f71-kube-rbac-proxy-sar-config\") pod \"error-404-isvc-de145-predictor-bfc475677-phjm7\" (UID: \"453eb767-5a5e-479c-8079-59038d055f71\") " pod="kserve-ci-e2e-test/error-404-isvc-de145-predictor-bfc475677-phjm7"
Sep 17 18:06:44.617357 ip-10-0-137-20 kubenswrapper[2565]: I0917 18:06:44.617244    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7t972\" (UniqueName: \"kubernetes.io/projected/453eb767-5a5e-479c-8079-59038d055f71-kube-api-access-7t972\") pod \"error-404-isvc-de145-predictor-bfc475677-phjm7\" (UID: \"453eb767-5a5e-479c-8079-59038d055f71\") " pod="kserve-ci-e2e-test/error-404-isvc-de145-predictor-bfc475677-phjm7"
Sep 17 18:06:44.617357 ip-10-0-137-20 kubenswrapper[2565]: I0917 18:06:44.617316    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/aeb32779-c788-4434-8d88-5ec5d08e7144-proxy-tls\") pod \"error-404-isvc-3ef6b-predictor-fcf5d54cf-rcnf9\" (UID: \"aeb32779-c788-4434-8d88-5ec5d08e7144\") " pod="kserve-ci-e2e-test/error-404-isvc-3ef6b-predictor-fcf5d54cf-rcnf9"
Sep 17 18:06:44.617357 ip-10-0-137-20 kubenswrapper[2565]: I0917 18:06:44.617349    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/453eb767-5a5e-479c-8079-59038d055f71-proxy-tls\") pod \"error-404-isvc-de145-predictor-bfc475677-phjm7\" (UID: \"453eb767-5a5e-479c-8079-59038d055f71\") " pod="kserve-ci-e2e-test/error-404-isvc-de145-predictor-bfc475677-phjm7"
Sep 17 18:06:44.617512 ip-10-0-137-20 kubenswrapper[2565]: E0917 18:06:44.617447    2565 secret.go:189] Couldn't get secret kserve-ci-e2e-test/error-404-isvc-de145-predictor-serving-cert: secret "error-404-isvc-de145-predictor-serving-cert" not found
Sep 17 18:06:44.617512 ip-10-0-137-20 kubenswrapper[2565]: E0917 18:06:44.617505    2565 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/453eb767-5a5e-479c-8079-59038d055f71-proxy-tls podName:453eb767-5a5e-479c-8079-59038d055f71 nodeName:}" failed. No retries permitted until 2026-09-17 18:06:45.117486397 +0000 UTC m=+1954.994770806 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "proxy-tls" (UniqueName: "kubernetes.io/secret/453eb767-5a5e-479c-8079-59038d055f71-proxy-tls") pod "error-404-isvc-de145-predictor-bfc475677-phjm7" (UID: "453eb767-5a5e-479c-8079-59038d055f71") : secret "error-404-isvc-de145-predictor-serving-cert" not found
Sep 17 18:06:44.617898 ip-10-0-137-20 kubenswrapper[2565]: I0917 18:06:44.617871    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/453eb767-5a5e-479c-8079-59038d055f71-kube-rbac-proxy-sar-config\") pod \"error-404-isvc-de145-predictor-bfc475677-phjm7\" (UID: \"453eb767-5a5e-479c-8079-59038d055f71\") " pod="kserve-ci-e2e-test/error-404-isvc-de145-predictor-bfc475677-phjm7"
Sep 17 18:06:44.619672 ip-10-0-137-20 kubenswrapper[2565]: I0917 18:06:44.619648    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/aeb32779-c788-4434-8d88-5ec5d08e7144-proxy-tls\") pod \"error-404-isvc-3ef6b-predictor-fcf5d54cf-rcnf9\" (UID: \"aeb32779-c788-4434-8d88-5ec5d08e7144\") " pod="kserve-ci-e2e-test/error-404-isvc-3ef6b-predictor-fcf5d54cf-rcnf9"
Sep 17 18:06:44.627777 ip-10-0-137-20 kubenswrapper[2565]: I0917 18:06:44.627757    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-7t972\" (UniqueName: \"kubernetes.io/projected/453eb767-5a5e-479c-8079-59038d055f71-kube-api-access-7t972\") pod \"error-404-isvc-de145-predictor-bfc475677-phjm7\" (UID: \"453eb767-5a5e-479c-8079-59038d055f71\") " pod="kserve-ci-e2e-test/error-404-isvc-de145-predictor-bfc475677-phjm7"
Sep 17 18:06:44.894695 ip-10-0-137-20 kubenswrapper[2565]: I0917 18:06:44.894653    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/error-404-isvc-3ef6b-predictor-fcf5d54cf-rcnf9"
Sep 17 18:06:44.920903 ip-10-0-137-20 kubenswrapper[2565]: I0917 18:06:44.920869    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/5a89f15e-8368-4d4e-8bfb-be3196e8a6ce-proxy-tls\") pod \"success-200-isvc-de145-predictor-7dbfbb5484-b92v8\" (UID: \"5a89f15e-8368-4d4e-8bfb-be3196e8a6ce\") " pod="kserve-ci-e2e-test/success-200-isvc-de145-predictor-7dbfbb5484-b92v8"
Sep 17 18:06:44.923430 ip-10-0-137-20 kubenswrapper[2565]: I0917 18:06:44.923404    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/5a89f15e-8368-4d4e-8bfb-be3196e8a6ce-proxy-tls\") pod \"success-200-isvc-de145-predictor-7dbfbb5484-b92v8\" (UID: \"5a89f15e-8368-4d4e-8bfb-be3196e8a6ce\") " pod="kserve-ci-e2e-test/success-200-isvc-de145-predictor-7dbfbb5484-b92v8"
Sep 17 18:06:44.966902 ip-10-0-137-20 kubenswrapper[2565]: I0917 18:06:44.966868    2565 generic.go:356] "Generic (PLEG): container finished" podID="8dfad140-1f0d-494d-825c-dd4e3391d778" containerID="21a517bfe9d6a35c7331286c82b71f23ccc9772e109a45024c011864b337631d" exitCode=2
Sep 17 18:06:44.967084 ip-10-0-137-20 kubenswrapper[2565]: I0917 18:06:44.966936    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/error-404-isvc-f13ae-predictor-776cc9696c-7jk49" event={"ID":"8dfad140-1f0d-494d-825c-dd4e3391d778","Type":"ContainerDied","Data":"21a517bfe9d6a35c7331286c82b71f23ccc9772e109a45024c011864b337631d"}
Sep 17 18:06:44.969079 ip-10-0-137-20 kubenswrapper[2565]: I0917 18:06:44.969053    2565 generic.go:356] "Generic (PLEG): container finished" podID="198a6626-b57d-460d-a29b-a7932f83006a" containerID="9911f5896661c77a764c5fff86aab24ac273ebc407937917617519b5ab269988" exitCode=2
Sep 17 18:06:44.969186 ip-10-0-137-20 kubenswrapper[2565]: I0917 18:06:44.969083    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/success-200-isvc-f13ae-predictor-b68795d4b-lvvqr" event={"ID":"198a6626-b57d-460d-a29b-a7932f83006a","Type":"ContainerDied","Data":"9911f5896661c77a764c5fff86aab24ac273ebc407937917617519b5ab269988"}
Sep 17 18:06:44.971275 ip-10-0-137-20 kubenswrapper[2565]: I0917 18:06:44.971252    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/success-200-isvc-3ef6b-predictor-77d7f67c46-4dshv" event={"ID":"7af4e9c8-bc7e-4cf3-a1b3-c28cfb1a0ff7","Type":"ContainerStarted","Data":"7a54d77231bee2fa47e4c56a2378633aa5b65cf79328fe222844c998725d4b02"}
Sep 17 18:06:44.971370 ip-10-0-137-20 kubenswrapper[2565]: I0917 18:06:44.971306    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/success-200-isvc-3ef6b-predictor-77d7f67c46-4dshv" event={"ID":"7af4e9c8-bc7e-4cf3-a1b3-c28cfb1a0ff7","Type":"ContainerStarted","Data":"7c301ee7b3254a99c24233068e38445760d2ddebd47eb151f6c163539f809b0d"}
Sep 17 18:06:44.971370 ip-10-0-137-20 kubenswrapper[2565]: I0917 18:06:44.971336    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/success-200-isvc-3ef6b-predictor-77d7f67c46-4dshv" event={"ID":"7af4e9c8-bc7e-4cf3-a1b3-c28cfb1a0ff7","Type":"ContainerStarted","Data":"84a87a978097ce61e61df4e6471bd14c775950eb41fda95a03a223450fc4c273"}
Sep 17 18:06:44.971503 ip-10-0-137-20 kubenswrapper[2565]: I0917 18:06:44.971485    2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve-ci-e2e-test/success-200-isvc-3ef6b-predictor-77d7f67c46-4dshv"
Sep 17 18:06:44.991064 ip-10-0-137-20 kubenswrapper[2565]: I0917 18:06:44.991003    2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kserve-ci-e2e-test/success-200-isvc-3ef6b-predictor-77d7f67c46-4dshv" podStartSLOduration=1.990988313 podStartE2EDuration="1.990988313s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-17 18:06:43 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-17 18:06:44.989482005 +0000 UTC m=+1954.866766421" watchObservedRunningTime="2026-09-17 18:06:44.990988313 +0000 UTC m=+1954.868272729"
Sep 17 18:06:45.122966 ip-10-0-137-20 kubenswrapper[2565]: I0917 18:06:45.122919    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/453eb767-5a5e-479c-8079-59038d055f71-proxy-tls\") pod \"error-404-isvc-de145-predictor-bfc475677-phjm7\" (UID: \"453eb767-5a5e-479c-8079-59038d055f71\") " pod="kserve-ci-e2e-test/error-404-isvc-de145-predictor-bfc475677-phjm7"
Sep 17 18:06:45.125367 ip-10-0-137-20 kubenswrapper[2565]: I0917 18:06:45.125341    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/453eb767-5a5e-479c-8079-59038d055f71-proxy-tls\") pod \"error-404-isvc-de145-predictor-bfc475677-phjm7\" (UID: \"453eb767-5a5e-479c-8079-59038d055f71\") " pod="kserve-ci-e2e-test/error-404-isvc-de145-predictor-bfc475677-phjm7"
Sep 17 18:06:45.176926 ip-10-0-137-20 kubenswrapper[2565]: I0917 18:06:45.176903    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/error-404-isvc-3ef6b-predictor-fcf5d54cf-rcnf9"]
Sep 17 18:06:45.178690 ip-10-0-137-20 kubenswrapper[2565]: W0917 18:06:45.178659    2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podaeb32779_c788_4434_8d88_5ec5d08e7144.slice/crio-06de6e30f441cac7e0b024c4399250c542e1b014318a212332a8b6881edae784 WatchSource:0}: Error finding container 06de6e30f441cac7e0b024c4399250c542e1b014318a212332a8b6881edae784: Status 404 returned error can't find the container with id 06de6e30f441cac7e0b024c4399250c542e1b014318a212332a8b6881edae784
Sep 17 18:06:45.207506 ip-10-0-137-20 kubenswrapper[2565]: I0917 18:06:45.207485    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/success-200-isvc-de145-predictor-7dbfbb5484-b92v8"
Sep 17 18:06:45.325223 ip-10-0-137-20 kubenswrapper[2565]: I0917 18:06:45.325199    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/error-404-isvc-de145-predictor-bfc475677-phjm7"
Sep 17 18:06:45.487114 ip-10-0-137-20 kubenswrapper[2565]: I0917 18:06:45.487089    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/success-200-isvc-de145-predictor-7dbfbb5484-b92v8"]
Sep 17 18:06:45.488347 ip-10-0-137-20 kubenswrapper[2565]: W0917 18:06:45.488319    2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5a89f15e_8368_4d4e_8bfb_be3196e8a6ce.slice/crio-1bd266471eba4fe013deddffc7d77e4e3daecb1d3221877826aaa05ed020afc9 WatchSource:0}: Error finding container 1bd266471eba4fe013deddffc7d77e4e3daecb1d3221877826aaa05ed020afc9: Status 404 returned error can't find the container with id 1bd266471eba4fe013deddffc7d77e4e3daecb1d3221877826aaa05ed020afc9
Sep 17 18:06:45.603376 ip-10-0-137-20 kubenswrapper[2565]: I0917 18:06:45.603355    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/error-404-isvc-de145-predictor-bfc475677-phjm7"]
Sep 17 18:06:45.605111 ip-10-0-137-20 kubenswrapper[2565]: W0917 18:06:45.605086    2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod453eb767_5a5e_479c_8079_59038d055f71.slice/crio-6115fc9d47f96df452974aadd6686dd48f94c988d9382b7e38bd95df4883b778 WatchSource:0}: Error finding container 6115fc9d47f96df452974aadd6686dd48f94c988d9382b7e38bd95df4883b778: Status 404 returned error can't find the container with id 6115fc9d47f96df452974aadd6686dd48f94c988d9382b7e38bd95df4883b778
Sep 17 18:06:45.978397 ip-10-0-137-20 kubenswrapper[2565]: I0917 18:06:45.978300    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/error-404-isvc-de145-predictor-bfc475677-phjm7" event={"ID":"453eb767-5a5e-479c-8079-59038d055f71","Type":"ContainerStarted","Data":"c0b970644f36bf610b949d88fc56c56f5f7f1881458ae0ca73eee29b08278d65"}
Sep 17 18:06:45.978397 ip-10-0-137-20 kubenswrapper[2565]: I0917 18:06:45.978372    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/error-404-isvc-de145-predictor-bfc475677-phjm7" event={"ID":"453eb767-5a5e-479c-8079-59038d055f71","Type":"ContainerStarted","Data":"31ee0ab236f0d5019ecee7f43155507ecfe5ef1ff614423379e8039436c04202"}
Sep 17 18:06:45.978861 ip-10-0-137-20 kubenswrapper[2565]: I0917 18:06:45.978403    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/error-404-isvc-de145-predictor-bfc475677-phjm7" event={"ID":"453eb767-5a5e-479c-8079-59038d055f71","Type":"ContainerStarted","Data":"6115fc9d47f96df452974aadd6686dd48f94c988d9382b7e38bd95df4883b778"}
Sep 17 18:06:45.978861 ip-10-0-137-20 kubenswrapper[2565]: I0917 18:06:45.978435    2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve-ci-e2e-test/error-404-isvc-de145-predictor-bfc475677-phjm7"
Sep 17 18:06:45.980147 ip-10-0-137-20 kubenswrapper[2565]: I0917 18:06:45.980122    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/success-200-isvc-de145-predictor-7dbfbb5484-b92v8" event={"ID":"5a89f15e-8368-4d4e-8bfb-be3196e8a6ce","Type":"ContainerStarted","Data":"5eb717205ab4c89c4f30a282105989b511cb3981961d0886be067ade0cec8ba0"}
Sep 17 18:06:45.980262 ip-10-0-137-20 kubenswrapper[2565]: I0917 18:06:45.980181    2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve-ci-e2e-test/success-200-isvc-de145-predictor-7dbfbb5484-b92v8"
Sep 17 18:06:45.980262 ip-10-0-137-20 kubenswrapper[2565]: I0917 18:06:45.980213    2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve-ci-e2e-test/success-200-isvc-de145-predictor-7dbfbb5484-b92v8"
Sep 17 18:06:45.980262 ip-10-0-137-20 kubenswrapper[2565]: I0917 18:06:45.980242    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/success-200-isvc-de145-predictor-7dbfbb5484-b92v8" event={"ID":"5a89f15e-8368-4d4e-8bfb-be3196e8a6ce","Type":"ContainerStarted","Data":"03c259b85cd2b358754bd7d040bc4143ef52976e121c54ad2c4013dba806fed3"}
Sep 17 18:06:45.980413 ip-10-0-137-20 kubenswrapper[2565]: I0917 18:06:45.980274    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/success-200-isvc-de145-predictor-7dbfbb5484-b92v8" event={"ID":"5a89f15e-8368-4d4e-8bfb-be3196e8a6ce","Type":"ContainerStarted","Data":"1bd266471eba4fe013deddffc7d77e4e3daecb1d3221877826aaa05ed020afc9"}
Sep 17 18:06:45.981937 ip-10-0-137-20 kubenswrapper[2565]: I0917 18:06:45.981909    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/error-404-isvc-3ef6b-predictor-fcf5d54cf-rcnf9" event={"ID":"aeb32779-c788-4434-8d88-5ec5d08e7144","Type":"ContainerStarted","Data":"b421d963a5d2dd2ec83a608399355a8e9d71d1704fe37eab32a5a55646b27109"}
Sep 17 18:06:45.982062 ip-10-0-137-20 kubenswrapper[2565]: I0917 18:06:45.981952    2565 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/success-200-isvc-de145-predictor-7dbfbb5484-b92v8" podUID="5a89f15e-8368-4d4e-8bfb-be3196e8a6ce" containerName="kserve-container" probeResult="failure" output="dial tcp 10.133.0.50:8080: connect: connection refused"
Sep 17 18:06:45.982062 ip-10-0-137-20 kubenswrapper[2565]: I0917 18:06:45.981968    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/error-404-isvc-3ef6b-predictor-fcf5d54cf-rcnf9" event={"ID":"aeb32779-c788-4434-8d88-5ec5d08e7144","Type":"ContainerStarted","Data":"d168c3878154ccbbb58ad0848e89c5242d365ab29a6f232f8fd7de62f0e3e6c3"}
Sep 17 18:06:45.982169 ip-10-0-137-20 kubenswrapper[2565]: I0917 18:06:45.982123    2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve-ci-e2e-test/error-404-isvc-3ef6b-predictor-fcf5d54cf-rcnf9"
Sep 17 18:06:45.982225 ip-10-0-137-20 kubenswrapper[2565]: I0917 18:06:45.982196    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/error-404-isvc-3ef6b-predictor-fcf5d54cf-rcnf9" event={"ID":"aeb32779-c788-4434-8d88-5ec5d08e7144","Type":"ContainerStarted","Data":"06de6e30f441cac7e0b024c4399250c542e1b014318a212332a8b6881edae784"}
Sep 17 18:06:45.982288 ip-10-0-137-20 kubenswrapper[2565]: I0917 18:06:45.982275    2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve-ci-e2e-test/error-404-isvc-3ef6b-predictor-fcf5d54cf-rcnf9"
Sep 17 18:06:45.982342 ip-10-0-137-20 kubenswrapper[2565]: I0917 18:06:45.982310    2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve-ci-e2e-test/success-200-isvc-3ef6b-predictor-77d7f67c46-4dshv"
Sep 17 18:06:45.983047 ip-10-0-137-20 kubenswrapper[2565]: I0917 18:06:45.983002    2565 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/success-200-isvc-3ef6b-predictor-77d7f67c46-4dshv" podUID="7af4e9c8-bc7e-4cf3-a1b3-c28cfb1a0ff7" containerName="kserve-container" probeResult="failure" output="dial tcp 10.133.0.48:8080: connect: connection refused"
Sep 17 18:06:45.983230 ip-10-0-137-20 kubenswrapper[2565]: I0917 18:06:45.983208    2565 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/error-404-isvc-3ef6b-predictor-fcf5d54cf-rcnf9" podUID="aeb32779-c788-4434-8d88-5ec5d08e7144" containerName="kserve-container" probeResult="failure" output="dial tcp 10.133.0.49:8080: connect: connection refused"
Sep 17 18:06:46.002198 ip-10-0-137-20 kubenswrapper[2565]: I0917 18:06:46.002159    2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kserve-ci-e2e-test/error-404-isvc-de145-predictor-bfc475677-phjm7" podStartSLOduration=2.002147912 podStartE2EDuration="2.002147912s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-17 18:06:44 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-17 18:06:46.000235869 +0000 UTC m=+1955.877520328" watchObservedRunningTime="2026-09-17 18:06:46.002147912 +0000 UTC m=+1955.879432326"
Sep 17 18:06:46.022047 ip-10-0-137-20 kubenswrapper[2565]: I0917 18:06:46.021994    2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kserve-ci-e2e-test/error-404-isvc-3ef6b-predictor-fcf5d54cf-rcnf9" podStartSLOduration=3.021983024 podStartE2EDuration="3.021983024s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-17 18:06:43 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-17 18:06:46.020816113 +0000 UTC m=+1955.898100529" watchObservedRunningTime="2026-09-17 18:06:46.021983024 +0000 UTC m=+1955.899267439"
Sep 17 18:06:46.042202 ip-10-0-137-20 kubenswrapper[2565]: I0917 18:06:46.042165    2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kserve-ci-e2e-test/success-200-isvc-de145-predictor-7dbfbb5484-b92v8" podStartSLOduration=2.042153394 podStartE2EDuration="2.042153394s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-17 18:06:44 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-17 18:06:46.040845009 +0000 UTC m=+1955.918129423" watchObservedRunningTime="2026-09-17 18:06:46.042153394 +0000 UTC m=+1955.919437809"
Sep 17 18:06:46.402655 ip-10-0-137-20 kubenswrapper[2565]: I0917 18:06:46.402618    2565 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/error-404-isvc-f13ae-predictor-776cc9696c-7jk49" podUID="8dfad140-1f0d-494d-825c-dd4e3391d778" containerName="kube-rbac-proxy" probeResult="failure" output="Get \"https://10.133.0.45:8643/healthz\": dial tcp 10.133.0.45:8643: connect: connection refused"
Sep 17 18:06:46.407239 ip-10-0-137-20 kubenswrapper[2565]: I0917 18:06:46.407213    2565 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/error-404-isvc-f13ae-predictor-776cc9696c-7jk49" podUID="8dfad140-1f0d-494d-825c-dd4e3391d778" containerName="kserve-container" probeResult="failure" output="dial tcp 10.133.0.45:8080: connect: connection refused"
Sep 17 18:06:46.628959 ip-10-0-137-20 kubenswrapper[2565]: I0917 18:06:46.628926    2565 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/success-200-isvc-b5897-predictor-5dd89c9df6-pglvk" podUID="4a1f1981-8b92-46be-942d-e4657b35d2ce" containerName="kube-rbac-proxy" probeResult="failure" output="Get \"https://10.133.0.46:8643/healthz\": dial tcp 10.133.0.46:8643: connect: connection refused"
Sep 17 18:06:46.629150 ip-10-0-137-20 kubenswrapper[2565]: I0917 18:06:46.628921    2565 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/error-404-isvc-b5897-predictor-658695b98f-5vcmf" podUID="772aca13-71ef-4f5a-b76b-82e7451ea20a" containerName="kube-rbac-proxy" probeResult="failure" output="Get \"https://10.133.0.47:8643/healthz\": dial tcp 10.133.0.47:8643: connect: connection refused"
Sep 17 18:06:46.987604 ip-10-0-137-20 kubenswrapper[2565]: I0917 18:06:46.987569    2565 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/error-404-isvc-3ef6b-predictor-fcf5d54cf-rcnf9" podUID="aeb32779-c788-4434-8d88-5ec5d08e7144" containerName="kserve-container" probeResult="failure" output="dial tcp 10.133.0.49:8080: connect: connection refused"
Sep 17 18:06:46.988081 ip-10-0-137-20 kubenswrapper[2565]: I0917 18:06:46.987574    2565 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/success-200-isvc-3ef6b-predictor-77d7f67c46-4dshv" podUID="7af4e9c8-bc7e-4cf3-a1b3-c28cfb1a0ff7" containerName="kserve-container" probeResult="failure" output="dial tcp 10.133.0.48:8080: connect: connection refused"
Sep 17 18:06:46.988081 ip-10-0-137-20 kubenswrapper[2565]: I0917 18:06:46.987923    2565 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/success-200-isvc-de145-predictor-7dbfbb5484-b92v8" podUID="5a89f15e-8368-4d4e-8bfb-be3196e8a6ce" containerName="kserve-container" probeResult="failure" output="dial tcp 10.133.0.50:8080: connect: connection refused"
Sep 17 18:06:46.988081 ip-10-0-137-20 kubenswrapper[2565]: I0917 18:06:46.988039    2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve-ci-e2e-test/error-404-isvc-de145-predictor-bfc475677-phjm7"
Sep 17 18:06:46.988847 ip-10-0-137-20 kubenswrapper[2565]: I0917 18:06:46.988828    2565 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/error-404-isvc-de145-predictor-bfc475677-phjm7" podUID="453eb767-5a5e-479c-8079-59038d055f71" containerName="kserve-container" probeResult="failure" output="dial tcp 10.133.0.51:8080: connect: connection refused"
Sep 17 18:06:47.406526 ip-10-0-137-20 kubenswrapper[2565]: I0917 18:06:47.406488    2565 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/success-200-isvc-f13ae-predictor-b68795d4b-lvvqr" podUID="198a6626-b57d-460d-a29b-a7932f83006a" containerName="kube-rbac-proxy" probeResult="failure" output="Get \"https://10.133.0.44:8643/healthz\": dial tcp 10.133.0.44:8643: connect: connection refused"
Sep 17 18:06:47.411940 ip-10-0-137-20 kubenswrapper[2565]: I0917 18:06:47.411912    2565 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/success-200-isvc-f13ae-predictor-b68795d4b-lvvqr" podUID="198a6626-b57d-460d-a29b-a7932f83006a" containerName="kserve-container" probeResult="failure" output="dial tcp 10.133.0.44:8080: connect: connection refused"
Sep 17 18:06:47.619800 ip-10-0-137-20 kubenswrapper[2565]: I0917 18:06:47.619778    2565 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/success-200-isvc-b5897-predictor-5dd89c9df6-pglvk"
Sep 17 18:06:47.655327 ip-10-0-137-20 kubenswrapper[2565]: I0917 18:06:47.655297    2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/4a1f1981-8b92-46be-942d-e4657b35d2ce-kube-rbac-proxy-sar-config\") pod \"4a1f1981-8b92-46be-942d-e4657b35d2ce\" (UID: \"4a1f1981-8b92-46be-942d-e4657b35d2ce\") "
Sep 17 18:06:47.655500 ip-10-0-137-20 kubenswrapper[2565]: I0917 18:06:47.655333    2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/4a1f1981-8b92-46be-942d-e4657b35d2ce-proxy-tls\") pod \"4a1f1981-8b92-46be-942d-e4657b35d2ce\" (UID: \"4a1f1981-8b92-46be-942d-e4657b35d2ce\") "
Sep 17 18:06:47.655500 ip-10-0-137-20 kubenswrapper[2565]: I0917 18:06:47.655376    2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-frzmz\" (UniqueName: \"kubernetes.io/projected/4a1f1981-8b92-46be-942d-e4657b35d2ce-kube-api-access-frzmz\") pod \"4a1f1981-8b92-46be-942d-e4657b35d2ce\" (UID: \"4a1f1981-8b92-46be-942d-e4657b35d2ce\") "
Sep 17 18:06:47.655725 ip-10-0-137-20 kubenswrapper[2565]: I0917 18:06:47.655699    2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/4a1f1981-8b92-46be-942d-e4657b35d2ce-kube-rbac-proxy-sar-config" (OuterVolumeSpecName: "kube-rbac-proxy-sar-config") pod "4a1f1981-8b92-46be-942d-e4657b35d2ce" (UID: "4a1f1981-8b92-46be-942d-e4657b35d2ce"). InnerVolumeSpecName "kube-rbac-proxy-sar-config". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 17 18:06:47.657452 ip-10-0-137-20 kubenswrapper[2565]: I0917 18:06:47.657387    2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/4a1f1981-8b92-46be-942d-e4657b35d2ce-proxy-tls" (OuterVolumeSpecName: "proxy-tls") pod "4a1f1981-8b92-46be-942d-e4657b35d2ce" (UID: "4a1f1981-8b92-46be-942d-e4657b35d2ce"). InnerVolumeSpecName "proxy-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 17 18:06:47.657547 ip-10-0-137-20 kubenswrapper[2565]: I0917 18:06:47.657478    2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/4a1f1981-8b92-46be-942d-e4657b35d2ce-kube-api-access-frzmz" (OuterVolumeSpecName: "kube-api-access-frzmz") pod "4a1f1981-8b92-46be-942d-e4657b35d2ce" (UID: "4a1f1981-8b92-46be-942d-e4657b35d2ce"). InnerVolumeSpecName "kube-api-access-frzmz". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 17 18:06:47.756400 ip-10-0-137-20 kubenswrapper[2565]: I0917 18:06:47.756370    2565 reconciler_common.go:299] "Volume detached for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/4a1f1981-8b92-46be-942d-e4657b35d2ce-kube-rbac-proxy-sar-config\") on node \"ip-10-0-137-20.ec2.internal\" DevicePath \"\""
Sep 17 18:06:47.756400 ip-10-0-137-20 kubenswrapper[2565]: I0917 18:06:47.756398    2565 reconciler_common.go:299] "Volume detached for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/4a1f1981-8b92-46be-942d-e4657b35d2ce-proxy-tls\") on node \"ip-10-0-137-20.ec2.internal\" DevicePath \"\""
Sep 17 18:06:47.756577 ip-10-0-137-20 kubenswrapper[2565]: I0917 18:06:47.756415    2565 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-frzmz\" (UniqueName: \"kubernetes.io/projected/4a1f1981-8b92-46be-942d-e4657b35d2ce-kube-api-access-frzmz\") on node \"ip-10-0-137-20.ec2.internal\" DevicePath \"\""
Sep 17 18:06:47.995418 ip-10-0-137-20 kubenswrapper[2565]: I0917 18:06:47.995383    2565 generic.go:356] "Generic (PLEG): container finished" podID="772aca13-71ef-4f5a-b76b-82e7451ea20a" containerID="caedce8b6d96f520b3522de57ba175185355a5d75e97ae32ba2039cf206f443d" exitCode=0
Sep 17 18:06:47.995910 ip-10-0-137-20 kubenswrapper[2565]: I0917 18:06:47.995478    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/error-404-isvc-b5897-predictor-658695b98f-5vcmf" event={"ID":"772aca13-71ef-4f5a-b76b-82e7451ea20a","Type":"ContainerDied","Data":"caedce8b6d96f520b3522de57ba175185355a5d75e97ae32ba2039cf206f443d"}
Sep 17 18:06:47.997326 ip-10-0-137-20 kubenswrapper[2565]: I0917 18:06:47.997300    2565 generic.go:356] "Generic (PLEG): container finished" podID="4a1f1981-8b92-46be-942d-e4657b35d2ce" containerID="8594cc9a84979afdb673a3a0f20cd6282289531167c6f2cd40603a279da75189" exitCode=0
Sep 17 18:06:47.997428 ip-10-0-137-20 kubenswrapper[2565]: I0917 18:06:47.997364    2565 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/success-200-isvc-b5897-predictor-5dd89c9df6-pglvk"
Sep 17 18:06:47.997428 ip-10-0-137-20 kubenswrapper[2565]: I0917 18:06:47.997374    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/success-200-isvc-b5897-predictor-5dd89c9df6-pglvk" event={"ID":"4a1f1981-8b92-46be-942d-e4657b35d2ce","Type":"ContainerDied","Data":"8594cc9a84979afdb673a3a0f20cd6282289531167c6f2cd40603a279da75189"}
Sep 17 18:06:47.997552 ip-10-0-137-20 kubenswrapper[2565]: I0917 18:06:47.997439    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/success-200-isvc-b5897-predictor-5dd89c9df6-pglvk" event={"ID":"4a1f1981-8b92-46be-942d-e4657b35d2ce","Type":"ContainerDied","Data":"f0b0c64552c28e41ab4916c6e7f5bee51be687ca80ecf5859e6c9ac0f4386dc9"}
Sep 17 18:06:47.997552 ip-10-0-137-20 kubenswrapper[2565]: I0917 18:06:47.997478    2565 scope.go:117] "RemoveContainer" containerID="0177156ca249c9b6850265215b16af7a519a4cd1cb2616d38f48dba76d6203d5"
Sep 17 18:06:48.000376 ip-10-0-137-20 kubenswrapper[2565]: I0917 18:06:48.000354    2565 generic.go:356] "Generic (PLEG): container finished" podID="198a6626-b57d-460d-a29b-a7932f83006a" containerID="9538ba633a50e514bfacd8afbf87d81788890b1abc59d73e61331d880270ef14" exitCode=0
Sep 17 18:06:48.000471 ip-10-0-137-20 kubenswrapper[2565]: I0917 18:06:48.000430    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/success-200-isvc-f13ae-predictor-b68795d4b-lvvqr" event={"ID":"198a6626-b57d-460d-a29b-a7932f83006a","Type":"ContainerDied","Data":"9538ba633a50e514bfacd8afbf87d81788890b1abc59d73e61331d880270ef14"}
Sep 17 18:06:48.002014 ip-10-0-137-20 kubenswrapper[2565]: I0917 18:06:48.001927    2565 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/error-404-isvc-de145-predictor-bfc475677-phjm7" podUID="453eb767-5a5e-479c-8079-59038d055f71" containerName="kserve-container" probeResult="failure" output="dial tcp 10.133.0.51:8080: connect: connection refused"
Sep 17 18:06:48.015578 ip-10-0-137-20 kubenswrapper[2565]: I0917 18:06:48.015556    2565 scope.go:117] "RemoveContainer" containerID="8594cc9a84979afdb673a3a0f20cd6282289531167c6f2cd40603a279da75189"
Sep 17 18:06:48.024606 ip-10-0-137-20 kubenswrapper[2565]: I0917 18:06:48.024581    2565 scope.go:117] "RemoveContainer" containerID="0177156ca249c9b6850265215b16af7a519a4cd1cb2616d38f48dba76d6203d5"
Sep 17 18:06:48.024916 ip-10-0-137-20 kubenswrapper[2565]: E0917 18:06:48.024897    2565 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"0177156ca249c9b6850265215b16af7a519a4cd1cb2616d38f48dba76d6203d5\": container with ID starting with 0177156ca249c9b6850265215b16af7a519a4cd1cb2616d38f48dba76d6203d5 not found: ID does not exist" containerID="0177156ca249c9b6850265215b16af7a519a4cd1cb2616d38f48dba76d6203d5"
Sep 17 18:06:48.024998 ip-10-0-137-20 kubenswrapper[2565]: I0917 18:06:48.024928    2565 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"0177156ca249c9b6850265215b16af7a519a4cd1cb2616d38f48dba76d6203d5"} err="failed to get container status \"0177156ca249c9b6850265215b16af7a519a4cd1cb2616d38f48dba76d6203d5\": rpc error: code = NotFound desc = could not find container \"0177156ca249c9b6850265215b16af7a519a4cd1cb2616d38f48dba76d6203d5\": container with ID starting with 0177156ca249c9b6850265215b16af7a519a4cd1cb2616d38f48dba76d6203d5 not found: ID does not exist"
Sep 17 18:06:48.024998 ip-10-0-137-20 kubenswrapper[2565]: I0917 18:06:48.024952    2565 scope.go:117] "RemoveContainer" containerID="8594cc9a84979afdb673a3a0f20cd6282289531167c6f2cd40603a279da75189"
Sep 17 18:06:48.025656 ip-10-0-137-20 kubenswrapper[2565]: E0917 18:06:48.025634    2565 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"8594cc9a84979afdb673a3a0f20cd6282289531167c6f2cd40603a279da75189\": container with ID starting with 8594cc9a84979afdb673a3a0f20cd6282289531167c6f2cd40603a279da75189 not found: ID does not exist" containerID="8594cc9a84979afdb673a3a0f20cd6282289531167c6f2cd40603a279da75189"
Sep 17 18:06:48.025721 ip-10-0-137-20 kubenswrapper[2565]: I0917 18:06:48.025660    2565 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"8594cc9a84979afdb673a3a0f20cd6282289531167c6f2cd40603a279da75189"} err="failed to get container status \"8594cc9a84979afdb673a3a0f20cd6282289531167c6f2cd40603a279da75189\": rpc error: code = NotFound desc = could not find container \"8594cc9a84979afdb673a3a0f20cd6282289531167c6f2cd40603a279da75189\": container with ID starting with 8594cc9a84979afdb673a3a0f20cd6282289531167c6f2cd40603a279da75189 not found: ID does not exist"
Sep 17 18:06:48.031686 ip-10-0-137-20 kubenswrapper[2565]: I0917 18:06:48.031664    2565 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kserve-ci-e2e-test/success-200-isvc-b5897-predictor-5dd89c9df6-pglvk"]
Sep 17 18:06:48.034313 ip-10-0-137-20 kubenswrapper[2565]: I0917 18:06:48.034289    2565 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kserve-ci-e2e-test/success-200-isvc-b5897-predictor-5dd89c9df6-pglvk"]
Sep 17 18:06:48.112472 ip-10-0-137-20 kubenswrapper[2565]: I0917 18:06:48.112453    2565 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/success-200-isvc-f13ae-predictor-b68795d4b-lvvqr"
Sep 17 18:06:48.158824 ip-10-0-137-20 kubenswrapper[2565]: I0917 18:06:48.158793    2565 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="4a1f1981-8b92-46be-942d-e4657b35d2ce" path="/var/lib/kubelet/pods/4a1f1981-8b92-46be-942d-e4657b35d2ce/volumes"
Sep 17 18:06:48.159461 ip-10-0-137-20 kubenswrapper[2565]: I0917 18:06:48.159433    2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/198a6626-b57d-460d-a29b-a7932f83006a-proxy-tls\") pod \"198a6626-b57d-460d-a29b-a7932f83006a\" (UID: \"198a6626-b57d-460d-a29b-a7932f83006a\") "
Sep 17 18:06:48.159607 ip-10-0-137-20 kubenswrapper[2565]: I0917 18:06:48.159568    2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/198a6626-b57d-460d-a29b-a7932f83006a-kube-rbac-proxy-sar-config\") pod \"198a6626-b57d-460d-a29b-a7932f83006a\" (UID: \"198a6626-b57d-460d-a29b-a7932f83006a\") "
Sep 17 18:06:48.159669 ip-10-0-137-20 kubenswrapper[2565]: I0917 18:06:48.159613    2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-d5zhn\" (UniqueName: \"kubernetes.io/projected/198a6626-b57d-460d-a29b-a7932f83006a-kube-api-access-d5zhn\") pod \"198a6626-b57d-460d-a29b-a7932f83006a\" (UID: \"198a6626-b57d-460d-a29b-a7932f83006a\") "
Sep 17 18:06:48.160114 ip-10-0-137-20 kubenswrapper[2565]: I0917 18:06:48.160092    2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/198a6626-b57d-460d-a29b-a7932f83006a-kube-rbac-proxy-sar-config" (OuterVolumeSpecName: "kube-rbac-proxy-sar-config") pod "198a6626-b57d-460d-a29b-a7932f83006a" (UID: "198a6626-b57d-460d-a29b-a7932f83006a"). InnerVolumeSpecName "kube-rbac-proxy-sar-config". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 17 18:06:48.161773 ip-10-0-137-20 kubenswrapper[2565]: I0917 18:06:48.161736    2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/198a6626-b57d-460d-a29b-a7932f83006a-proxy-tls" (OuterVolumeSpecName: "proxy-tls") pod "198a6626-b57d-460d-a29b-a7932f83006a" (UID: "198a6626-b57d-460d-a29b-a7932f83006a"). InnerVolumeSpecName "proxy-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 17 18:06:48.162258 ip-10-0-137-20 kubenswrapper[2565]: I0917 18:06:48.162236    2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/198a6626-b57d-460d-a29b-a7932f83006a-kube-api-access-d5zhn" (OuterVolumeSpecName: "kube-api-access-d5zhn") pod "198a6626-b57d-460d-a29b-a7932f83006a" (UID: "198a6626-b57d-460d-a29b-a7932f83006a"). InnerVolumeSpecName "kube-api-access-d5zhn". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 17 18:06:48.260970 ip-10-0-137-20 kubenswrapper[2565]: I0917 18:06:48.260943    2565 reconciler_common.go:299] "Volume detached for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/198a6626-b57d-460d-a29b-a7932f83006a-kube-rbac-proxy-sar-config\") on node \"ip-10-0-137-20.ec2.internal\" DevicePath \"\""
Sep 17 18:06:48.260970 ip-10-0-137-20 kubenswrapper[2565]: I0917 18:06:48.260971    2565 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-d5zhn\" (UniqueName: \"kubernetes.io/projected/198a6626-b57d-460d-a29b-a7932f83006a-kube-api-access-d5zhn\") on node \"ip-10-0-137-20.ec2.internal\" DevicePath \"\""
Sep 17 18:06:48.261145 ip-10-0-137-20 kubenswrapper[2565]: I0917 18:06:48.260980    2565 reconciler_common.go:299] "Volume detached for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/198a6626-b57d-460d-a29b-a7932f83006a-proxy-tls\") on node \"ip-10-0-137-20.ec2.internal\" DevicePath \"\""
Sep 17 18:06:48.265832 ip-10-0-137-20 kubenswrapper[2565]: I0917 18:06:48.265811    2565 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/error-404-isvc-b5897-predictor-658695b98f-5vcmf"
Sep 17 18:06:48.361994 ip-10-0-137-20 kubenswrapper[2565]: I0917 18:06:48.361967    2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-7zqkx\" (UniqueName: \"kubernetes.io/projected/772aca13-71ef-4f5a-b76b-82e7451ea20a-kube-api-access-7zqkx\") pod \"772aca13-71ef-4f5a-b76b-82e7451ea20a\" (UID: \"772aca13-71ef-4f5a-b76b-82e7451ea20a\") "
Sep 17 18:06:48.362158 ip-10-0-137-20 kubenswrapper[2565]: I0917 18:06:48.362002    2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/772aca13-71ef-4f5a-b76b-82e7451ea20a-kube-rbac-proxy-sar-config\") pod \"772aca13-71ef-4f5a-b76b-82e7451ea20a\" (UID: \"772aca13-71ef-4f5a-b76b-82e7451ea20a\") "
Sep 17 18:06:48.362158 ip-10-0-137-20 kubenswrapper[2565]: I0917 18:06:48.362052    2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/772aca13-71ef-4f5a-b76b-82e7451ea20a-proxy-tls\") pod \"772aca13-71ef-4f5a-b76b-82e7451ea20a\" (UID: \"772aca13-71ef-4f5a-b76b-82e7451ea20a\") "
Sep 17 18:06:48.362342 ip-10-0-137-20 kubenswrapper[2565]: I0917 18:06:48.362321    2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/772aca13-71ef-4f5a-b76b-82e7451ea20a-kube-rbac-proxy-sar-config" (OuterVolumeSpecName: "kube-rbac-proxy-sar-config") pod "772aca13-71ef-4f5a-b76b-82e7451ea20a" (UID: "772aca13-71ef-4f5a-b76b-82e7451ea20a"). InnerVolumeSpecName "kube-rbac-proxy-sar-config". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 17 18:06:48.363925 ip-10-0-137-20 kubenswrapper[2565]: I0917 18:06:48.363903    2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/772aca13-71ef-4f5a-b76b-82e7451ea20a-proxy-tls" (OuterVolumeSpecName: "proxy-tls") pod "772aca13-71ef-4f5a-b76b-82e7451ea20a" (UID: "772aca13-71ef-4f5a-b76b-82e7451ea20a"). InnerVolumeSpecName "proxy-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 17 18:06:48.364007 ip-10-0-137-20 kubenswrapper[2565]: I0917 18:06:48.363946    2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/772aca13-71ef-4f5a-b76b-82e7451ea20a-kube-api-access-7zqkx" (OuterVolumeSpecName: "kube-api-access-7zqkx") pod "772aca13-71ef-4f5a-b76b-82e7451ea20a" (UID: "772aca13-71ef-4f5a-b76b-82e7451ea20a"). InnerVolumeSpecName "kube-api-access-7zqkx". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 17 18:06:48.462750 ip-10-0-137-20 kubenswrapper[2565]: I0917 18:06:48.462717    2565 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-7zqkx\" (UniqueName: \"kubernetes.io/projected/772aca13-71ef-4f5a-b76b-82e7451ea20a-kube-api-access-7zqkx\") on node \"ip-10-0-137-20.ec2.internal\" DevicePath \"\""
Sep 17 18:06:48.462848 ip-10-0-137-20 kubenswrapper[2565]: I0917 18:06:48.462753    2565 reconciler_common.go:299] "Volume detached for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/772aca13-71ef-4f5a-b76b-82e7451ea20a-kube-rbac-proxy-sar-config\") on node \"ip-10-0-137-20.ec2.internal\" DevicePath \"\""
Sep 17 18:06:48.462848 ip-10-0-137-20 kubenswrapper[2565]: I0917 18:06:48.462769    2565 reconciler_common.go:299] "Volume detached for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/772aca13-71ef-4f5a-b76b-82e7451ea20a-proxy-tls\") on node \"ip-10-0-137-20.ec2.internal\" DevicePath \"\""
Sep 17 18:06:48.573393 ip-10-0-137-20 kubenswrapper[2565]: I0917 18:06:48.573369    2565 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/error-404-isvc-f13ae-predictor-776cc9696c-7jk49"
Sep 17 18:06:48.664684 ip-10-0-137-20 kubenswrapper[2565]: I0917 18:06:48.664659    2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/8dfad140-1f0d-494d-825c-dd4e3391d778-proxy-tls\") pod \"8dfad140-1f0d-494d-825c-dd4e3391d778\" (UID: \"8dfad140-1f0d-494d-825c-dd4e3391d778\") "
Sep 17 18:06:48.664835 ip-10-0-137-20 kubenswrapper[2565]: I0917 18:06:48.664694    2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/8dfad140-1f0d-494d-825c-dd4e3391d778-kube-rbac-proxy-sar-config\") pod \"8dfad140-1f0d-494d-825c-dd4e3391d778\" (UID: \"8dfad140-1f0d-494d-825c-dd4e3391d778\") "
Sep 17 18:06:48.664835 ip-10-0-137-20 kubenswrapper[2565]: I0917 18:06:48.664720    2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-dbfg9\" (UniqueName: \"kubernetes.io/projected/8dfad140-1f0d-494d-825c-dd4e3391d778-kube-api-access-dbfg9\") pod \"8dfad140-1f0d-494d-825c-dd4e3391d778\" (UID: \"8dfad140-1f0d-494d-825c-dd4e3391d778\") "
Sep 17 18:06:48.665101 ip-10-0-137-20 kubenswrapper[2565]: I0917 18:06:48.665077    2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/8dfad140-1f0d-494d-825c-dd4e3391d778-kube-rbac-proxy-sar-config" (OuterVolumeSpecName: "kube-rbac-proxy-sar-config") pod "8dfad140-1f0d-494d-825c-dd4e3391d778" (UID: "8dfad140-1f0d-494d-825c-dd4e3391d778"). InnerVolumeSpecName "kube-rbac-proxy-sar-config". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 17 18:06:48.666764 ip-10-0-137-20 kubenswrapper[2565]: I0917 18:06:48.666742    2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/8dfad140-1f0d-494d-825c-dd4e3391d778-kube-api-access-dbfg9" (OuterVolumeSpecName: "kube-api-access-dbfg9") pod "8dfad140-1f0d-494d-825c-dd4e3391d778" (UID: "8dfad140-1f0d-494d-825c-dd4e3391d778"). InnerVolumeSpecName "kube-api-access-dbfg9". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 17 18:06:48.666864 ip-10-0-137-20 kubenswrapper[2565]: I0917 18:06:48.666772    2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/8dfad140-1f0d-494d-825c-dd4e3391d778-proxy-tls" (OuterVolumeSpecName: "proxy-tls") pod "8dfad140-1f0d-494d-825c-dd4e3391d778" (UID: "8dfad140-1f0d-494d-825c-dd4e3391d778"). InnerVolumeSpecName "proxy-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 17 18:06:48.765996 ip-10-0-137-20 kubenswrapper[2565]: I0917 18:06:48.765929    2565 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-dbfg9\" (UniqueName: \"kubernetes.io/projected/8dfad140-1f0d-494d-825c-dd4e3391d778-kube-api-access-dbfg9\") on node \"ip-10-0-137-20.ec2.internal\" DevicePath \"\""
Sep 17 18:06:48.765996 ip-10-0-137-20 kubenswrapper[2565]: I0917 18:06:48.765953    2565 reconciler_common.go:299] "Volume detached for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/8dfad140-1f0d-494d-825c-dd4e3391d778-proxy-tls\") on node \"ip-10-0-137-20.ec2.internal\" DevicePath \"\""
Sep 17 18:06:48.765996 ip-10-0-137-20 kubenswrapper[2565]: I0917 18:06:48.765963    2565 reconciler_common.go:299] "Volume detached for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/8dfad140-1f0d-494d-825c-dd4e3391d778-kube-rbac-proxy-sar-config\") on node \"ip-10-0-137-20.ec2.internal\" DevicePath \"\""
Sep 17 18:06:49.008430 ip-10-0-137-20 kubenswrapper[2565]: I0917 18:06:49.008402    2565 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/error-404-isvc-b5897-predictor-658695b98f-5vcmf"
Sep 17 18:06:49.008430 ip-10-0-137-20 kubenswrapper[2565]: I0917 18:06:49.008423    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/error-404-isvc-b5897-predictor-658695b98f-5vcmf" event={"ID":"772aca13-71ef-4f5a-b76b-82e7451ea20a","Type":"ContainerDied","Data":"9b43636c8103cb6ddd3cd006e35cff28c63b49339ff5c78b1afd6b869092b476"}
Sep 17 18:06:49.008899 ip-10-0-137-20 kubenswrapper[2565]: I0917 18:06:49.008493    2565 scope.go:117] "RemoveContainer" containerID="a465926da50d27237fccf625dfefe806871ddc631d1ee41b38bdbce24d251254"
Sep 17 18:06:49.011353 ip-10-0-137-20 kubenswrapper[2565]: I0917 18:06:49.011323    2565 generic.go:356] "Generic (PLEG): container finished" podID="8dfad140-1f0d-494d-825c-dd4e3391d778" containerID="fd1078748cae37103cff662c1e895e7b66c57e6ad873fa18b6abfec202fde992" exitCode=0
Sep 17 18:06:49.011489 ip-10-0-137-20 kubenswrapper[2565]: I0917 18:06:49.011375    2565 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/error-404-isvc-f13ae-predictor-776cc9696c-7jk49"
Sep 17 18:06:49.011489 ip-10-0-137-20 kubenswrapper[2565]: I0917 18:06:49.011373    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/error-404-isvc-f13ae-predictor-776cc9696c-7jk49" event={"ID":"8dfad140-1f0d-494d-825c-dd4e3391d778","Type":"ContainerDied","Data":"fd1078748cae37103cff662c1e895e7b66c57e6ad873fa18b6abfec202fde992"}
Sep 17 18:06:49.011602 ip-10-0-137-20 kubenswrapper[2565]: I0917 18:06:49.011529    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/error-404-isvc-f13ae-predictor-776cc9696c-7jk49" event={"ID":"8dfad140-1f0d-494d-825c-dd4e3391d778","Type":"ContainerDied","Data":"579957a3a34c3ba72fd767e78feaee59d6d4ba081d6d8efa2e6ca70e9bf55f85"}
Sep 17 18:06:49.013319 ip-10-0-137-20 kubenswrapper[2565]: I0917 18:06:49.013292    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/success-200-isvc-f13ae-predictor-b68795d4b-lvvqr" event={"ID":"198a6626-b57d-460d-a29b-a7932f83006a","Type":"ContainerDied","Data":"dafc1ac60778121edbcaa05f8d5c4020c9564f26545c91d3883d2824c65588ce"}
Sep 17 18:06:49.013451 ip-10-0-137-20 kubenswrapper[2565]: I0917 18:06:49.013342    2565 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/success-200-isvc-f13ae-predictor-b68795d4b-lvvqr"
Sep 17 18:06:49.018189 ip-10-0-137-20 kubenswrapper[2565]: I0917 18:06:49.018167    2565 scope.go:117] "RemoveContainer" containerID="caedce8b6d96f520b3522de57ba175185355a5d75e97ae32ba2039cf206f443d"
Sep 17 18:06:49.026730 ip-10-0-137-20 kubenswrapper[2565]: I0917 18:06:49.026716    2565 scope.go:117] "RemoveContainer" containerID="21a517bfe9d6a35c7331286c82b71f23ccc9772e109a45024c011864b337631d"
Sep 17 18:06:49.033730 ip-10-0-137-20 kubenswrapper[2565]: I0917 18:06:49.033714    2565 scope.go:117] "RemoveContainer" containerID="fd1078748cae37103cff662c1e895e7b66c57e6ad873fa18b6abfec202fde992"
Sep 17 18:06:49.036974 ip-10-0-137-20 kubenswrapper[2565]: I0917 18:06:49.036950    2565 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kserve-ci-e2e-test/error-404-isvc-b5897-predictor-658695b98f-5vcmf"]
Sep 17 18:06:49.041525 ip-10-0-137-20 kubenswrapper[2565]: I0917 18:06:49.041512    2565 scope.go:117] "RemoveContainer" containerID="21a517bfe9d6a35c7331286c82b71f23ccc9772e109a45024c011864b337631d"
Sep 17 18:06:49.041799 ip-10-0-137-20 kubenswrapper[2565]: E0917 18:06:49.041781    2565 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"21a517bfe9d6a35c7331286c82b71f23ccc9772e109a45024c011864b337631d\": container with ID starting with 21a517bfe9d6a35c7331286c82b71f23ccc9772e109a45024c011864b337631d not found: ID does not exist" containerID="21a517bfe9d6a35c7331286c82b71f23ccc9772e109a45024c011864b337631d"
Sep 17 18:06:49.041866 ip-10-0-137-20 kubenswrapper[2565]: I0917 18:06:49.041807    2565 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"21a517bfe9d6a35c7331286c82b71f23ccc9772e109a45024c011864b337631d"} err="failed to get container status \"21a517bfe9d6a35c7331286c82b71f23ccc9772e109a45024c011864b337631d\": rpc error: code = NotFound desc = could not find container \"21a517bfe9d6a35c7331286c82b71f23ccc9772e109a45024c011864b337631d\": container with ID starting with 21a517bfe9d6a35c7331286c82b71f23ccc9772e109a45024c011864b337631d not found: ID does not exist"
Sep 17 18:06:49.041866 ip-10-0-137-20 kubenswrapper[2565]: I0917 18:06:49.041824    2565 scope.go:117] "RemoveContainer" containerID="fd1078748cae37103cff662c1e895e7b66c57e6ad873fa18b6abfec202fde992"
Sep 17 18:06:49.042089 ip-10-0-137-20 kubenswrapper[2565]: E0917 18:06:49.042073    2565 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"fd1078748cae37103cff662c1e895e7b66c57e6ad873fa18b6abfec202fde992\": container with ID starting with fd1078748cae37103cff662c1e895e7b66c57e6ad873fa18b6abfec202fde992 not found: ID does not exist" containerID="fd1078748cae37103cff662c1e895e7b66c57e6ad873fa18b6abfec202fde992"
Sep 17 18:06:49.042136 ip-10-0-137-20 kubenswrapper[2565]: I0917 18:06:49.042098    2565 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"fd1078748cae37103cff662c1e895e7b66c57e6ad873fa18b6abfec202fde992"} err="failed to get container status \"fd1078748cae37103cff662c1e895e7b66c57e6ad873fa18b6abfec202fde992\": rpc error: code = NotFound desc = could not find container \"fd1078748cae37103cff662c1e895e7b66c57e6ad873fa18b6abfec202fde992\": container with ID starting with fd1078748cae37103cff662c1e895e7b66c57e6ad873fa18b6abfec202fde992 not found: ID does not exist"
Sep 17 18:06:49.042136 ip-10-0-137-20 kubenswrapper[2565]: I0917 18:06:49.042112    2565 scope.go:117] "RemoveContainer" containerID="9911f5896661c77a764c5fff86aab24ac273ebc407937917617519b5ab269988"
Sep 17 18:06:49.042645 ip-10-0-137-20 kubenswrapper[2565]: I0917 18:06:49.042626    2565 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kserve-ci-e2e-test/error-404-isvc-b5897-predictor-658695b98f-5vcmf"]
Sep 17 18:06:49.051464 ip-10-0-137-20 kubenswrapper[2565]: I0917 18:06:49.051444    2565 scope.go:117] "RemoveContainer" containerID="9538ba633a50e514bfacd8afbf87d81788890b1abc59d73e61331d880270ef14"
Sep 17 18:06:49.059701 ip-10-0-137-20 kubenswrapper[2565]: I0917 18:06:49.059679    2565 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kserve-ci-e2e-test/error-404-isvc-f13ae-predictor-776cc9696c-7jk49"]
Sep 17 18:06:49.066505 ip-10-0-137-20 kubenswrapper[2565]: I0917 18:06:49.066486    2565 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kserve-ci-e2e-test/error-404-isvc-f13ae-predictor-776cc9696c-7jk49"]
Sep 17 18:06:49.076716 ip-10-0-137-20 kubenswrapper[2565]: I0917 18:06:49.076699    2565 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kserve-ci-e2e-test/success-200-isvc-f13ae-predictor-b68795d4b-lvvqr"]
Sep 17 18:06:49.087178 ip-10-0-137-20 kubenswrapper[2565]: I0917 18:06:49.087154    2565 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kserve-ci-e2e-test/success-200-isvc-f13ae-predictor-b68795d4b-lvvqr"]
Sep 17 18:06:50.157737 ip-10-0-137-20 kubenswrapper[2565]: I0917 18:06:50.157707    2565 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="198a6626-b57d-460d-a29b-a7932f83006a" path="/var/lib/kubelet/pods/198a6626-b57d-460d-a29b-a7932f83006a/volumes"
Sep 17 18:06:50.158164 ip-10-0-137-20 kubenswrapper[2565]: I0917 18:06:50.158150    2565 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="772aca13-71ef-4f5a-b76b-82e7451ea20a" path="/var/lib/kubelet/pods/772aca13-71ef-4f5a-b76b-82e7451ea20a/volumes"
Sep 17 18:06:50.158541 ip-10-0-137-20 kubenswrapper[2565]: I0917 18:06:50.158530    2565 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="8dfad140-1f0d-494d-825c-dd4e3391d778" path="/var/lib/kubelet/pods/8dfad140-1f0d-494d-825c-dd4e3391d778/volumes"
Sep 17 18:06:51.992732 ip-10-0-137-20 kubenswrapper[2565]: I0917 18:06:51.992663    2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kserve-ci-e2e-test/success-200-isvc-de145-predictor-7dbfbb5484-b92v8"
Sep 17 18:06:51.993284 ip-10-0-137-20 kubenswrapper[2565]: I0917 18:06:51.993245    2565 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/success-200-isvc-de145-predictor-7dbfbb5484-b92v8" podUID="5a89f15e-8368-4d4e-8bfb-be3196e8a6ce" containerName="kserve-container" probeResult="failure" output="dial tcp 10.133.0.50:8080: connect: connection refused"
Sep 17 18:06:51.993408 ip-10-0-137-20 kubenswrapper[2565]: I0917 18:06:51.993392    2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kserve-ci-e2e-test/error-404-isvc-3ef6b-predictor-fcf5d54cf-rcnf9"
Sep 17 18:06:51.993497 ip-10-0-137-20 kubenswrapper[2565]: I0917 18:06:51.993485    2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kserve-ci-e2e-test/success-200-isvc-3ef6b-predictor-77d7f67c46-4dshv"
Sep 17 18:06:51.993888 ip-10-0-137-20 kubenswrapper[2565]: I0917 18:06:51.993854    2565 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/success-200-isvc-3ef6b-predictor-77d7f67c46-4dshv" podUID="7af4e9c8-bc7e-4cf3-a1b3-c28cfb1a0ff7" containerName="kserve-container" probeResult="failure" output="dial tcp 10.133.0.48:8080: connect: connection refused"
Sep 17 18:06:51.994168 ip-10-0-137-20 kubenswrapper[2565]: I0917 18:06:51.994141    2565 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/error-404-isvc-3ef6b-predictor-fcf5d54cf-rcnf9" podUID="aeb32779-c788-4434-8d88-5ec5d08e7144" containerName="kserve-container" probeResult="failure" output="dial tcp 10.133.0.49:8080: connect: connection refused"
Sep 17 18:06:53.004745 ip-10-0-137-20 kubenswrapper[2565]: I0917 18:06:53.004717    2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kserve-ci-e2e-test/error-404-isvc-de145-predictor-bfc475677-phjm7"
Sep 17 18:06:53.005208 ip-10-0-137-20 kubenswrapper[2565]: I0917 18:06:53.005179    2565 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/error-404-isvc-de145-predictor-bfc475677-phjm7" podUID="453eb767-5a5e-479c-8079-59038d055f71" containerName="kserve-container" probeResult="failure" output="dial tcp 10.133.0.51:8080: connect: connection refused"
Sep 17 18:07:01.993372 ip-10-0-137-20 kubenswrapper[2565]: I0917 18:07:01.993288    2565 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/success-200-isvc-de145-predictor-7dbfbb5484-b92v8" podUID="5a89f15e-8368-4d4e-8bfb-be3196e8a6ce" containerName="kserve-container" probeResult="failure" output="dial tcp 10.133.0.50:8080: connect: connection refused"
Sep 17 18:07:01.993816 ip-10-0-137-20 kubenswrapper[2565]: I0917 18:07:01.993790    2565 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/error-404-isvc-3ef6b-predictor-fcf5d54cf-rcnf9" podUID="aeb32779-c788-4434-8d88-5ec5d08e7144" containerName="kserve-container" probeResult="failure" output="dial tcp 10.133.0.49:8080: connect: connection refused"
Sep 17 18:07:01.993917 ip-10-0-137-20 kubenswrapper[2565]: I0917 18:07:01.993896    2565 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/success-200-isvc-3ef6b-predictor-77d7f67c46-4dshv" podUID="7af4e9c8-bc7e-4cf3-a1b3-c28cfb1a0ff7" containerName="kserve-container" probeResult="failure" output="dial tcp 10.133.0.48:8080: connect: connection refused"
Sep 17 18:07:03.005138 ip-10-0-137-20 kubenswrapper[2565]: I0917 18:07:03.005102    2565 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/error-404-isvc-de145-predictor-bfc475677-phjm7" podUID="453eb767-5a5e-479c-8079-59038d055f71" containerName="kserve-container" probeResult="failure" output="dial tcp 10.133.0.51:8080: connect: connection refused"
Sep 17 18:07:11.993524 ip-10-0-137-20 kubenswrapper[2565]: I0917 18:07:11.993481    2565 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/success-200-isvc-de145-predictor-7dbfbb5484-b92v8" podUID="5a89f15e-8368-4d4e-8bfb-be3196e8a6ce" containerName="kserve-container" probeResult="failure" output="dial tcp 10.133.0.50:8080: connect: connection refused"
Sep 17 18:07:11.993981 ip-10-0-137-20 kubenswrapper[2565]: I0917 18:07:11.993797    2565 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/error-404-isvc-3ef6b-predictor-fcf5d54cf-rcnf9" podUID="aeb32779-c788-4434-8d88-5ec5d08e7144" containerName="kserve-container" probeResult="failure" output="dial tcp 10.133.0.49:8080: connect: connection refused"
Sep 17 18:07:11.993981 ip-10-0-137-20 kubenswrapper[2565]: I0917 18:07:11.993923    2565 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/success-200-isvc-3ef6b-predictor-77d7f67c46-4dshv" podUID="7af4e9c8-bc7e-4cf3-a1b3-c28cfb1a0ff7" containerName="kserve-container" probeResult="failure" output="dial tcp 10.133.0.48:8080: connect: connection refused"
Sep 17 18:07:13.005712 ip-10-0-137-20 kubenswrapper[2565]: I0917 18:07:13.005670    2565 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/error-404-isvc-de145-predictor-bfc475677-phjm7" podUID="453eb767-5a5e-479c-8079-59038d055f71" containerName="kserve-container" probeResult="failure" output="dial tcp 10.133.0.51:8080: connect: connection refused"
Sep 17 18:07:21.993585 ip-10-0-137-20 kubenswrapper[2565]: I0917 18:07:21.993546    2565 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/success-200-isvc-de145-predictor-7dbfbb5484-b92v8" podUID="5a89f15e-8368-4d4e-8bfb-be3196e8a6ce" containerName="kserve-container" probeResult="failure" output="dial tcp 10.133.0.50:8080: connect: connection refused"
Sep 17 18:07:21.994088 ip-10-0-137-20 kubenswrapper[2565]: I0917 18:07:21.993779    2565 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/error-404-isvc-3ef6b-predictor-fcf5d54cf-rcnf9" podUID="aeb32779-c788-4434-8d88-5ec5d08e7144" containerName="kserve-container" probeResult="failure" output="dial tcp 10.133.0.49:8080: connect: connection refused"
Sep 17 18:07:21.994088 ip-10-0-137-20 kubenswrapper[2565]: I0917 18:07:21.993914    2565 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/success-200-isvc-3ef6b-predictor-77d7f67c46-4dshv" podUID="7af4e9c8-bc7e-4cf3-a1b3-c28cfb1a0ff7" containerName="kserve-container" probeResult="failure" output="dial tcp 10.133.0.48:8080: connect: connection refused"
Sep 17 18:07:23.005845 ip-10-0-137-20 kubenswrapper[2565]: I0917 18:07:23.005811    2565 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/error-404-isvc-de145-predictor-bfc475677-phjm7" podUID="453eb767-5a5e-479c-8079-59038d055f71" containerName="kserve-container" probeResult="failure" output="dial tcp 10.133.0.51:8080: connect: connection refused"
Sep 17 18:07:31.993285 ip-10-0-137-20 kubenswrapper[2565]: I0917 18:07:31.993249    2565 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/success-200-isvc-de145-predictor-7dbfbb5484-b92v8" podUID="5a89f15e-8368-4d4e-8bfb-be3196e8a6ce" containerName="kserve-container" probeResult="failure" output="dial tcp 10.133.0.50:8080: connect: connection refused"
Sep 17 18:07:31.995167 ip-10-0-137-20 kubenswrapper[2565]: I0917 18:07:31.995144    2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kserve-ci-e2e-test/success-200-isvc-3ef6b-predictor-77d7f67c46-4dshv"
Sep 17 18:07:31.995309 ip-10-0-137-20 kubenswrapper[2565]: I0917 18:07:31.995242    2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kserve-ci-e2e-test/error-404-isvc-3ef6b-predictor-fcf5d54cf-rcnf9"
Sep 17 18:07:33.006211 ip-10-0-137-20 kubenswrapper[2565]: I0917 18:07:33.006181    2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kserve-ci-e2e-test/error-404-isvc-de145-predictor-bfc475677-phjm7"
Sep 17 18:07:41.994190 ip-10-0-137-20 kubenswrapper[2565]: I0917 18:07:41.994156    2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kserve-ci-e2e-test/success-200-isvc-de145-predictor-7dbfbb5484-b92v8"
Sep 17 18:09:12.248168 ip-10-0-137-20 kubenswrapper[2565]: I0917 18:09:12.248139    2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-d9927_30837de7-1447-4de2-ac62-c809813b3338/ovn-acl-logging/0.log"
Sep 17 18:09:12.253528 ip-10-0-137-20 kubenswrapper[2565]: I0917 18:09:12.253506    2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-d9927_30837de7-1447-4de2-ac62-c809813b3338/ovn-acl-logging/0.log"
Sep 17 18:14:12.278458 ip-10-0-137-20 kubenswrapper[2565]: I0917 18:14:12.278431    2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-d9927_30837de7-1447-4de2-ac62-c809813b3338/ovn-acl-logging/0.log"
Sep 17 18:14:12.282794 ip-10-0-137-20 kubenswrapper[2565]: I0917 18:14:12.282773    2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-d9927_30837de7-1447-4de2-ac62-c809813b3338/ovn-acl-logging/0.log"
Sep 17 18:15:58.650218 ip-10-0-137-20 kubenswrapper[2565]: I0917 18:15:58.650134    2565 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kserve-ci-e2e-test/success-200-isvc-3ef6b-predictor-77d7f67c46-4dshv"]
Sep 17 18:15:58.650800 ip-10-0-137-20 kubenswrapper[2565]: I0917 18:15:58.650424    2565 kuberuntime_container.go:873] "Killing container with a grace period" pod="kserve-ci-e2e-test/success-200-isvc-3ef6b-predictor-77d7f67c46-4dshv" podUID="7af4e9c8-bc7e-4cf3-a1b3-c28cfb1a0ff7" containerName="kserve-container" containerID="cri-o://7c301ee7b3254a99c24233068e38445760d2ddebd47eb151f6c163539f809b0d" gracePeriod=30
Sep 17 18:15:58.650800 ip-10-0-137-20 kubenswrapper[2565]: I0917 18:15:58.650458    2565 kuberuntime_container.go:873] "Killing container with a grace period" pod="kserve-ci-e2e-test/success-200-isvc-3ef6b-predictor-77d7f67c46-4dshv" podUID="7af4e9c8-bc7e-4cf3-a1b3-c28cfb1a0ff7" containerName="kube-rbac-proxy" containerID="cri-o://7a54d77231bee2fa47e4c56a2378633aa5b65cf79328fe222844c998725d4b02" gracePeriod=30
Sep 17 18:15:58.681581 ip-10-0-137-20 kubenswrapper[2565]: I0917 18:15:58.681547    2565 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kserve-ci-e2e-test/error-404-isvc-3ef6b-predictor-fcf5d54cf-rcnf9"]
Sep 17 18:15:58.681851 ip-10-0-137-20 kubenswrapper[2565]: I0917 18:15:58.681813    2565 kuberuntime_container.go:873] "Killing container with a grace period" pod="kserve-ci-e2e-test/error-404-isvc-3ef6b-predictor-fcf5d54cf-rcnf9" podUID="aeb32779-c788-4434-8d88-5ec5d08e7144" containerName="kserve-container" containerID="cri-o://d168c3878154ccbbb58ad0848e89c5242d365ab29a6f232f8fd7de62f0e3e6c3" gracePeriod=30
Sep 17 18:15:58.681988 ip-10-0-137-20 kubenswrapper[2565]: I0917 18:15:58.681850    2565 kuberuntime_container.go:873] "Killing container with a grace period" pod="kserve-ci-e2e-test/error-404-isvc-3ef6b-predictor-fcf5d54cf-rcnf9" podUID="aeb32779-c788-4434-8d88-5ec5d08e7144" containerName="kube-rbac-proxy" containerID="cri-o://b421d963a5d2dd2ec83a608399355a8e9d71d1704fe37eab32a5a55646b27109" gracePeriod=30
Sep 17 18:15:59.488658 ip-10-0-137-20 kubenswrapper[2565]: I0917 18:15:59.488627    2565 generic.go:356] "Generic (PLEG): container finished" podID="7af4e9c8-bc7e-4cf3-a1b3-c28cfb1a0ff7" containerID="7a54d77231bee2fa47e4c56a2378633aa5b65cf79328fe222844c998725d4b02" exitCode=2
Sep 17 18:15:59.488846 ip-10-0-137-20 kubenswrapper[2565]: I0917 18:15:59.488698    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/success-200-isvc-3ef6b-predictor-77d7f67c46-4dshv" event={"ID":"7af4e9c8-bc7e-4cf3-a1b3-c28cfb1a0ff7","Type":"ContainerDied","Data":"7a54d77231bee2fa47e4c56a2378633aa5b65cf79328fe222844c998725d4b02"}
Sep 17 18:15:59.490560 ip-10-0-137-20 kubenswrapper[2565]: I0917 18:15:59.490539    2565 generic.go:356] "Generic (PLEG): container finished" podID="aeb32779-c788-4434-8d88-5ec5d08e7144" containerID="b421d963a5d2dd2ec83a608399355a8e9d71d1704fe37eab32a5a55646b27109" exitCode=2
Sep 17 18:15:59.490700 ip-10-0-137-20 kubenswrapper[2565]: I0917 18:15:59.490615    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/error-404-isvc-3ef6b-predictor-fcf5d54cf-rcnf9" event={"ID":"aeb32779-c788-4434-8d88-5ec5d08e7144","Type":"ContainerDied","Data":"b421d963a5d2dd2ec83a608399355a8e9d71d1704fe37eab32a5a55646b27109"}
Sep 17 18:16:01.726362 ip-10-0-137-20 kubenswrapper[2565]: I0917 18:16:01.726341    2565 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/error-404-isvc-3ef6b-predictor-fcf5d54cf-rcnf9"
Sep 17 18:16:01.783061 ip-10-0-137-20 kubenswrapper[2565]: I0917 18:16:01.783014    2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/aeb32779-c788-4434-8d88-5ec5d08e7144-proxy-tls\") pod \"aeb32779-c788-4434-8d88-5ec5d08e7144\" (UID: \"aeb32779-c788-4434-8d88-5ec5d08e7144\") "
Sep 17 18:16:01.783231 ip-10-0-137-20 kubenswrapper[2565]: I0917 18:16:01.783075    2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/aeb32779-c788-4434-8d88-5ec5d08e7144-kube-rbac-proxy-sar-config\") pod \"aeb32779-c788-4434-8d88-5ec5d08e7144\" (UID: \"aeb32779-c788-4434-8d88-5ec5d08e7144\") "
Sep 17 18:16:01.783231 ip-10-0-137-20 kubenswrapper[2565]: I0917 18:16:01.783097    2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-f6l72\" (UniqueName: \"kubernetes.io/projected/aeb32779-c788-4434-8d88-5ec5d08e7144-kube-api-access-f6l72\") pod \"aeb32779-c788-4434-8d88-5ec5d08e7144\" (UID: \"aeb32779-c788-4434-8d88-5ec5d08e7144\") "
Sep 17 18:16:01.783529 ip-10-0-137-20 kubenswrapper[2565]: I0917 18:16:01.783501    2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/aeb32779-c788-4434-8d88-5ec5d08e7144-kube-rbac-proxy-sar-config" (OuterVolumeSpecName: "kube-rbac-proxy-sar-config") pod "aeb32779-c788-4434-8d88-5ec5d08e7144" (UID: "aeb32779-c788-4434-8d88-5ec5d08e7144"). InnerVolumeSpecName "kube-rbac-proxy-sar-config". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 17 18:16:01.785226 ip-10-0-137-20 kubenswrapper[2565]: I0917 18:16:01.785201    2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/aeb32779-c788-4434-8d88-5ec5d08e7144-proxy-tls" (OuterVolumeSpecName: "proxy-tls") pod "aeb32779-c788-4434-8d88-5ec5d08e7144" (UID: "aeb32779-c788-4434-8d88-5ec5d08e7144"). InnerVolumeSpecName "proxy-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 17 18:16:01.785407 ip-10-0-137-20 kubenswrapper[2565]: I0917 18:16:01.785385    2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/aeb32779-c788-4434-8d88-5ec5d08e7144-kube-api-access-f6l72" (OuterVolumeSpecName: "kube-api-access-f6l72") pod "aeb32779-c788-4434-8d88-5ec5d08e7144" (UID: "aeb32779-c788-4434-8d88-5ec5d08e7144"). InnerVolumeSpecName "kube-api-access-f6l72". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 17 18:16:01.805954 ip-10-0-137-20 kubenswrapper[2565]: I0917 18:16:01.805900    2565 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/success-200-isvc-3ef6b-predictor-77d7f67c46-4dshv"
Sep 17 18:16:01.883958 ip-10-0-137-20 kubenswrapper[2565]: I0917 18:16:01.883928    2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-q5wpq\" (UniqueName: \"kubernetes.io/projected/7af4e9c8-bc7e-4cf3-a1b3-c28cfb1a0ff7-kube-api-access-q5wpq\") pod \"7af4e9c8-bc7e-4cf3-a1b3-c28cfb1a0ff7\" (UID: \"7af4e9c8-bc7e-4cf3-a1b3-c28cfb1a0ff7\") "
Sep 17 18:16:01.884119 ip-10-0-137-20 kubenswrapper[2565]: I0917 18:16:01.883991    2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/7af4e9c8-bc7e-4cf3-a1b3-c28cfb1a0ff7-proxy-tls\") pod \"7af4e9c8-bc7e-4cf3-a1b3-c28cfb1a0ff7\" (UID: \"7af4e9c8-bc7e-4cf3-a1b3-c28cfb1a0ff7\") "
Sep 17 18:16:01.884119 ip-10-0-137-20 kubenswrapper[2565]: I0917 18:16:01.884044    2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/7af4e9c8-bc7e-4cf3-a1b3-c28cfb1a0ff7-kube-rbac-proxy-sar-config\") pod \"7af4e9c8-bc7e-4cf3-a1b3-c28cfb1a0ff7\" (UID: \"7af4e9c8-bc7e-4cf3-a1b3-c28cfb1a0ff7\") "
Sep 17 18:16:01.884352 ip-10-0-137-20 kubenswrapper[2565]: I0917 18:16:01.884330    2565 reconciler_common.go:299] "Volume detached for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/aeb32779-c788-4434-8d88-5ec5d08e7144-proxy-tls\") on node \"ip-10-0-137-20.ec2.internal\" DevicePath \"\""
Sep 17 18:16:01.884419 ip-10-0-137-20 kubenswrapper[2565]: I0917 18:16:01.884356    2565 reconciler_common.go:299] "Volume detached for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/aeb32779-c788-4434-8d88-5ec5d08e7144-kube-rbac-proxy-sar-config\") on node \"ip-10-0-137-20.ec2.internal\" DevicePath \"\""
Sep 17 18:16:01.884419 ip-10-0-137-20 kubenswrapper[2565]: I0917 18:16:01.884369    2565 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-f6l72\" (UniqueName: \"kubernetes.io/projected/aeb32779-c788-4434-8d88-5ec5d08e7144-kube-api-access-f6l72\") on node \"ip-10-0-137-20.ec2.internal\" DevicePath \"\""
Sep 17 18:16:01.884419 ip-10-0-137-20 kubenswrapper[2565]: I0917 18:16:01.884397    2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/7af4e9c8-bc7e-4cf3-a1b3-c28cfb1a0ff7-kube-rbac-proxy-sar-config" (OuterVolumeSpecName: "kube-rbac-proxy-sar-config") pod "7af4e9c8-bc7e-4cf3-a1b3-c28cfb1a0ff7" (UID: "7af4e9c8-bc7e-4cf3-a1b3-c28cfb1a0ff7"). InnerVolumeSpecName "kube-rbac-proxy-sar-config". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 17 18:16:01.886085 ip-10-0-137-20 kubenswrapper[2565]: I0917 18:16:01.886061    2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/7af4e9c8-bc7e-4cf3-a1b3-c28cfb1a0ff7-proxy-tls" (OuterVolumeSpecName: "proxy-tls") pod "7af4e9c8-bc7e-4cf3-a1b3-c28cfb1a0ff7" (UID: "7af4e9c8-bc7e-4cf3-a1b3-c28cfb1a0ff7"). InnerVolumeSpecName "proxy-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 17 18:16:01.886214 ip-10-0-137-20 kubenswrapper[2565]: I0917 18:16:01.886118    2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/7af4e9c8-bc7e-4cf3-a1b3-c28cfb1a0ff7-kube-api-access-q5wpq" (OuterVolumeSpecName: "kube-api-access-q5wpq") pod "7af4e9c8-bc7e-4cf3-a1b3-c28cfb1a0ff7" (UID: "7af4e9c8-bc7e-4cf3-a1b3-c28cfb1a0ff7"). InnerVolumeSpecName "kube-api-access-q5wpq". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 17 18:16:01.985563 ip-10-0-137-20 kubenswrapper[2565]: I0917 18:16:01.985531    2565 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-q5wpq\" (UniqueName: \"kubernetes.io/projected/7af4e9c8-bc7e-4cf3-a1b3-c28cfb1a0ff7-kube-api-access-q5wpq\") on node \"ip-10-0-137-20.ec2.internal\" DevicePath \"\""
Sep 17 18:16:01.985563 ip-10-0-137-20 kubenswrapper[2565]: I0917 18:16:01.985557    2565 reconciler_common.go:299] "Volume detached for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/7af4e9c8-bc7e-4cf3-a1b3-c28cfb1a0ff7-proxy-tls\") on node \"ip-10-0-137-20.ec2.internal\" DevicePath \"\""
Sep 17 18:16:01.985563 ip-10-0-137-20 kubenswrapper[2565]: I0917 18:16:01.985568    2565 reconciler_common.go:299] "Volume detached for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/7af4e9c8-bc7e-4cf3-a1b3-c28cfb1a0ff7-kube-rbac-proxy-sar-config\") on node \"ip-10-0-137-20.ec2.internal\" DevicePath \"\""
Sep 17 18:16:02.506475 ip-10-0-137-20 kubenswrapper[2565]: I0917 18:16:02.506393    2565 generic.go:356] "Generic (PLEG): container finished" podID="7af4e9c8-bc7e-4cf3-a1b3-c28cfb1a0ff7" containerID="7c301ee7b3254a99c24233068e38445760d2ddebd47eb151f6c163539f809b0d" exitCode=0
Sep 17 18:16:02.506475 ip-10-0-137-20 kubenswrapper[2565]: I0917 18:16:02.506440    2565 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/success-200-isvc-3ef6b-predictor-77d7f67c46-4dshv"
Sep 17 18:16:02.506721 ip-10-0-137-20 kubenswrapper[2565]: I0917 18:16:02.506477    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/success-200-isvc-3ef6b-predictor-77d7f67c46-4dshv" event={"ID":"7af4e9c8-bc7e-4cf3-a1b3-c28cfb1a0ff7","Type":"ContainerDied","Data":"7c301ee7b3254a99c24233068e38445760d2ddebd47eb151f6c163539f809b0d"}
Sep 17 18:16:02.506721 ip-10-0-137-20 kubenswrapper[2565]: I0917 18:16:02.506537    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/success-200-isvc-3ef6b-predictor-77d7f67c46-4dshv" event={"ID":"7af4e9c8-bc7e-4cf3-a1b3-c28cfb1a0ff7","Type":"ContainerDied","Data":"84a87a978097ce61e61df4e6471bd14c775950eb41fda95a03a223450fc4c273"}
Sep 17 18:16:02.506721 ip-10-0-137-20 kubenswrapper[2565]: I0917 18:16:02.506563    2565 scope.go:117] "RemoveContainer" containerID="7a54d77231bee2fa47e4c56a2378633aa5b65cf79328fe222844c998725d4b02"
Sep 17 18:16:02.508400 ip-10-0-137-20 kubenswrapper[2565]: I0917 18:16:02.508376    2565 generic.go:356] "Generic (PLEG): container finished" podID="aeb32779-c788-4434-8d88-5ec5d08e7144" containerID="d168c3878154ccbbb58ad0848e89c5242d365ab29a6f232f8fd7de62f0e3e6c3" exitCode=0
Sep 17 18:16:02.508570 ip-10-0-137-20 kubenswrapper[2565]: I0917 18:16:02.508458    2565 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/error-404-isvc-3ef6b-predictor-fcf5d54cf-rcnf9"
Sep 17 18:16:02.508570 ip-10-0-137-20 kubenswrapper[2565]: I0917 18:16:02.508482    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/error-404-isvc-3ef6b-predictor-fcf5d54cf-rcnf9" event={"ID":"aeb32779-c788-4434-8d88-5ec5d08e7144","Type":"ContainerDied","Data":"d168c3878154ccbbb58ad0848e89c5242d365ab29a6f232f8fd7de62f0e3e6c3"}
Sep 17 18:16:02.508570 ip-10-0-137-20 kubenswrapper[2565]: I0917 18:16:02.508563    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/error-404-isvc-3ef6b-predictor-fcf5d54cf-rcnf9" event={"ID":"aeb32779-c788-4434-8d88-5ec5d08e7144","Type":"ContainerDied","Data":"06de6e30f441cac7e0b024c4399250c542e1b014318a212332a8b6881edae784"}
Sep 17 18:16:02.515689 ip-10-0-137-20 kubenswrapper[2565]: I0917 18:16:02.515672    2565 scope.go:117] "RemoveContainer" containerID="7c301ee7b3254a99c24233068e38445760d2ddebd47eb151f6c163539f809b0d"
Sep 17 18:16:02.522974 ip-10-0-137-20 kubenswrapper[2565]: I0917 18:16:02.522958    2565 scope.go:117] "RemoveContainer" containerID="7a54d77231bee2fa47e4c56a2378633aa5b65cf79328fe222844c998725d4b02"
Sep 17 18:16:02.523325 ip-10-0-137-20 kubenswrapper[2565]: E0917 18:16:02.523305    2565 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"7a54d77231bee2fa47e4c56a2378633aa5b65cf79328fe222844c998725d4b02\": container with ID starting with 7a54d77231bee2fa47e4c56a2378633aa5b65cf79328fe222844c998725d4b02 not found: ID does not exist" containerID="7a54d77231bee2fa47e4c56a2378633aa5b65cf79328fe222844c998725d4b02"
Sep 17 18:16:02.523413 ip-10-0-137-20 kubenswrapper[2565]: I0917 18:16:02.523332    2565 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"7a54d77231bee2fa47e4c56a2378633aa5b65cf79328fe222844c998725d4b02"} err="failed to get container status \"7a54d77231bee2fa47e4c56a2378633aa5b65cf79328fe222844c998725d4b02\": rpc error: code = NotFound desc = could not find container \"7a54d77231bee2fa47e4c56a2378633aa5b65cf79328fe222844c998725d4b02\": container with ID starting with 7a54d77231bee2fa47e4c56a2378633aa5b65cf79328fe222844c998725d4b02 not found: ID does not exist"
Sep 17 18:16:02.523413 ip-10-0-137-20 kubenswrapper[2565]: I0917 18:16:02.523348    2565 scope.go:117] "RemoveContainer" containerID="7c301ee7b3254a99c24233068e38445760d2ddebd47eb151f6c163539f809b0d"
Sep 17 18:16:02.523582 ip-10-0-137-20 kubenswrapper[2565]: E0917 18:16:02.523566    2565 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"7c301ee7b3254a99c24233068e38445760d2ddebd47eb151f6c163539f809b0d\": container with ID starting with 7c301ee7b3254a99c24233068e38445760d2ddebd47eb151f6c163539f809b0d not found: ID does not exist" containerID="7c301ee7b3254a99c24233068e38445760d2ddebd47eb151f6c163539f809b0d"
Sep 17 18:16:02.523626 ip-10-0-137-20 kubenswrapper[2565]: I0917 18:16:02.523588    2565 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"7c301ee7b3254a99c24233068e38445760d2ddebd47eb151f6c163539f809b0d"} err="failed to get container status \"7c301ee7b3254a99c24233068e38445760d2ddebd47eb151f6c163539f809b0d\": rpc error: code = NotFound desc = could not find container \"7c301ee7b3254a99c24233068e38445760d2ddebd47eb151f6c163539f809b0d\": container with ID starting with 7c301ee7b3254a99c24233068e38445760d2ddebd47eb151f6c163539f809b0d not found: ID does not exist"
Sep 17 18:16:02.523626 ip-10-0-137-20 kubenswrapper[2565]: I0917 18:16:02.523602    2565 scope.go:117] "RemoveContainer" containerID="b421d963a5d2dd2ec83a608399355a8e9d71d1704fe37eab32a5a55646b27109"
Sep 17 18:16:02.524732 ip-10-0-137-20 kubenswrapper[2565]: I0917 18:16:02.524693    2565 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kserve-ci-e2e-test/success-200-isvc-3ef6b-predictor-77d7f67c46-4dshv"]
Sep 17 18:16:02.526285 ip-10-0-137-20 kubenswrapper[2565]: I0917 18:16:02.526267    2565 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kserve-ci-e2e-test/success-200-isvc-3ef6b-predictor-77d7f67c46-4dshv"]
Sep 17 18:16:02.530975 ip-10-0-137-20 kubenswrapper[2565]: I0917 18:16:02.530959    2565 scope.go:117] "RemoveContainer" containerID="d168c3878154ccbbb58ad0848e89c5242d365ab29a6f232f8fd7de62f0e3e6c3"
Sep 17 18:16:02.538190 ip-10-0-137-20 kubenswrapper[2565]: I0917 18:16:02.538166    2565 scope.go:117] "RemoveContainer" containerID="b421d963a5d2dd2ec83a608399355a8e9d71d1704fe37eab32a5a55646b27109"
Sep 17 18:16:02.538459 ip-10-0-137-20 kubenswrapper[2565]: E0917 18:16:02.538442    2565 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"b421d963a5d2dd2ec83a608399355a8e9d71d1704fe37eab32a5a55646b27109\": container with ID starting with b421d963a5d2dd2ec83a608399355a8e9d71d1704fe37eab32a5a55646b27109 not found: ID does not exist" containerID="b421d963a5d2dd2ec83a608399355a8e9d71d1704fe37eab32a5a55646b27109"
Sep 17 18:16:02.538518 ip-10-0-137-20 kubenswrapper[2565]: I0917 18:16:02.538467    2565 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"b421d963a5d2dd2ec83a608399355a8e9d71d1704fe37eab32a5a55646b27109"} err="failed to get container status \"b421d963a5d2dd2ec83a608399355a8e9d71d1704fe37eab32a5a55646b27109\": rpc error: code = NotFound desc = could not find container \"b421d963a5d2dd2ec83a608399355a8e9d71d1704fe37eab32a5a55646b27109\": container with ID starting with b421d963a5d2dd2ec83a608399355a8e9d71d1704fe37eab32a5a55646b27109 not found: ID does not exist"
Sep 17 18:16:02.538518 ip-10-0-137-20 kubenswrapper[2565]: I0917 18:16:02.538483    2565 scope.go:117] "RemoveContainer" containerID="d168c3878154ccbbb58ad0848e89c5242d365ab29a6f232f8fd7de62f0e3e6c3"
Sep 17 18:16:02.538745 ip-10-0-137-20 kubenswrapper[2565]: E0917 18:16:02.538724    2565 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"d168c3878154ccbbb58ad0848e89c5242d365ab29a6f232f8fd7de62f0e3e6c3\": container with ID starting with d168c3878154ccbbb58ad0848e89c5242d365ab29a6f232f8fd7de62f0e3e6c3 not found: ID does not exist" containerID="d168c3878154ccbbb58ad0848e89c5242d365ab29a6f232f8fd7de62f0e3e6c3"
Sep 17 18:16:02.538799 ip-10-0-137-20 kubenswrapper[2565]: I0917 18:16:02.538755    2565 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d168c3878154ccbbb58ad0848e89c5242d365ab29a6f232f8fd7de62f0e3e6c3"} err="failed to get container status \"d168c3878154ccbbb58ad0848e89c5242d365ab29a6f232f8fd7de62f0e3e6c3\": rpc error: code = NotFound desc = could not find container \"d168c3878154ccbbb58ad0848e89c5242d365ab29a6f232f8fd7de62f0e3e6c3\": container with ID starting with d168c3878154ccbbb58ad0848e89c5242d365ab29a6f232f8fd7de62f0e3e6c3 not found: ID does not exist"
Sep 17 18:16:02.541444 ip-10-0-137-20 kubenswrapper[2565]: I0917 18:16:02.541425    2565 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kserve-ci-e2e-test/error-404-isvc-3ef6b-predictor-fcf5d54cf-rcnf9"]
Sep 17 18:16:02.545455 ip-10-0-137-20 kubenswrapper[2565]: I0917 18:16:02.545431    2565 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kserve-ci-e2e-test/error-404-isvc-3ef6b-predictor-fcf5d54cf-rcnf9"]
Sep 17 18:16:04.156576 ip-10-0-137-20 kubenswrapper[2565]: I0917 18:16:04.156547    2565 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="7af4e9c8-bc7e-4cf3-a1b3-c28cfb1a0ff7" path="/var/lib/kubelet/pods/7af4e9c8-bc7e-4cf3-a1b3-c28cfb1a0ff7/volumes"
Sep 17 18:16:04.156962 ip-10-0-137-20 kubenswrapper[2565]: I0917 18:16:04.156952    2565 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="aeb32779-c788-4434-8d88-5ec5d08e7144" path="/var/lib/kubelet/pods/aeb32779-c788-4434-8d88-5ec5d08e7144/volumes"
Sep 17 18:19:12.305348 ip-10-0-137-20 kubenswrapper[2565]: I0917 18:19:12.305238    2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-d9927_30837de7-1447-4de2-ac62-c809813b3338/ovn-acl-logging/0.log"
Sep 17 18:19:12.314153 ip-10-0-137-20 kubenswrapper[2565]: I0917 18:19:12.314130    2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-d9927_30837de7-1447-4de2-ac62-c809813b3338/ovn-acl-logging/0.log"
Sep 17 18:24:12.332243 ip-10-0-137-20 kubenswrapper[2565]: I0917 18:24:12.332141    2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-d9927_30837de7-1447-4de2-ac62-c809813b3338/ovn-acl-logging/0.log"
Sep 17 18:24:12.342660 ip-10-0-137-20 kubenswrapper[2565]: I0917 18:24:12.342642    2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-d9927_30837de7-1447-4de2-ac62-c809813b3338/ovn-acl-logging/0.log"
Sep 17 18:24:13.456034 ip-10-0-137-20 kubenswrapper[2565]: I0917 18:24:13.455984    2565 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kserve-ci-e2e-test/success-200-isvc-de145-predictor-7dbfbb5484-b92v8"]
Sep 17 18:24:13.456452 ip-10-0-137-20 kubenswrapper[2565]: I0917 18:24:13.456235    2565 kuberuntime_container.go:873] "Killing container with a grace period" pod="kserve-ci-e2e-test/success-200-isvc-de145-predictor-7dbfbb5484-b92v8" podUID="5a89f15e-8368-4d4e-8bfb-be3196e8a6ce" containerName="kserve-container" containerID="cri-o://03c259b85cd2b358754bd7d040bc4143ef52976e121c54ad2c4013dba806fed3" gracePeriod=30
Sep 17 18:24:13.456452 ip-10-0-137-20 kubenswrapper[2565]: I0917 18:24:13.456282    2565 kuberuntime_container.go:873] "Killing container with a grace period" pod="kserve-ci-e2e-test/success-200-isvc-de145-predictor-7dbfbb5484-b92v8" podUID="5a89f15e-8368-4d4e-8bfb-be3196e8a6ce" containerName="kube-rbac-proxy" containerID="cri-o://5eb717205ab4c89c4f30a282105989b511cb3981961d0886be067ade0cec8ba0" gracePeriod=30
Sep 17 18:24:13.485532 ip-10-0-137-20 kubenswrapper[2565]: I0917 18:24:13.485507    2565 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kserve-ci-e2e-test/error-404-isvc-de145-predictor-bfc475677-phjm7"]
Sep 17 18:24:13.485789 ip-10-0-137-20 kubenswrapper[2565]: I0917 18:24:13.485763    2565 kuberuntime_container.go:873] "Killing container with a grace period" pod="kserve-ci-e2e-test/error-404-isvc-de145-predictor-bfc475677-phjm7" podUID="453eb767-5a5e-479c-8079-59038d055f71" containerName="kube-rbac-proxy" containerID="cri-o://c0b970644f36bf610b949d88fc56c56f5f7f1881458ae0ca73eee29b08278d65" gracePeriod=30
Sep 17 18:24:13.485871 ip-10-0-137-20 kubenswrapper[2565]: I0917 18:24:13.485755    2565 kuberuntime_container.go:873] "Killing container with a grace period" pod="kserve-ci-e2e-test/error-404-isvc-de145-predictor-bfc475677-phjm7" podUID="453eb767-5a5e-479c-8079-59038d055f71" containerName="kserve-container" containerID="cri-o://31ee0ab236f0d5019ecee7f43155507ecfe5ef1ff614423379e8039436c04202" gracePeriod=30
Sep 17 18:24:13.671774 ip-10-0-137-20 kubenswrapper[2565]: I0917 18:24:13.671740    2565 generic.go:356] "Generic (PLEG): container finished" podID="453eb767-5a5e-479c-8079-59038d055f71" containerID="c0b970644f36bf610b949d88fc56c56f5f7f1881458ae0ca73eee29b08278d65" exitCode=2
Sep 17 18:24:13.671947 ip-10-0-137-20 kubenswrapper[2565]: I0917 18:24:13.671817    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/error-404-isvc-de145-predictor-bfc475677-phjm7" event={"ID":"453eb767-5a5e-479c-8079-59038d055f71","Type":"ContainerDied","Data":"c0b970644f36bf610b949d88fc56c56f5f7f1881458ae0ca73eee29b08278d65"}
Sep 17 18:24:13.673635 ip-10-0-137-20 kubenswrapper[2565]: I0917 18:24:13.673610    2565 generic.go:356] "Generic (PLEG): container finished" podID="5a89f15e-8368-4d4e-8bfb-be3196e8a6ce" containerID="5eb717205ab4c89c4f30a282105989b511cb3981961d0886be067ade0cec8ba0" exitCode=2
Sep 17 18:24:13.673744 ip-10-0-137-20 kubenswrapper[2565]: I0917 18:24:13.673680    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/success-200-isvc-de145-predictor-7dbfbb5484-b92v8" event={"ID":"5a89f15e-8368-4d4e-8bfb-be3196e8a6ce","Type":"ContainerDied","Data":"5eb717205ab4c89c4f30a282105989b511cb3981961d0886be067ade0cec8ba0"}
Sep 17 18:24:16.520892 ip-10-0-137-20 kubenswrapper[2565]: I0917 18:24:16.520865    2565 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/success-200-isvc-de145-predictor-7dbfbb5484-b92v8"
Sep 17 18:24:16.587473 ip-10-0-137-20 kubenswrapper[2565]: I0917 18:24:16.587441    2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/5a89f15e-8368-4d4e-8bfb-be3196e8a6ce-kube-rbac-proxy-sar-config\") pod \"5a89f15e-8368-4d4e-8bfb-be3196e8a6ce\" (UID: \"5a89f15e-8368-4d4e-8bfb-be3196e8a6ce\") "
Sep 17 18:24:16.587612 ip-10-0-137-20 kubenswrapper[2565]: I0917 18:24:16.587552    2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/5a89f15e-8368-4d4e-8bfb-be3196e8a6ce-proxy-tls\") pod \"5a89f15e-8368-4d4e-8bfb-be3196e8a6ce\" (UID: \"5a89f15e-8368-4d4e-8bfb-be3196e8a6ce\") "
Sep 17 18:24:16.587612 ip-10-0-137-20 kubenswrapper[2565]: I0917 18:24:16.587586    2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-589qc\" (UniqueName: \"kubernetes.io/projected/5a89f15e-8368-4d4e-8bfb-be3196e8a6ce-kube-api-access-589qc\") pod \"5a89f15e-8368-4d4e-8bfb-be3196e8a6ce\" (UID: \"5a89f15e-8368-4d4e-8bfb-be3196e8a6ce\") "
Sep 17 18:24:16.587837 ip-10-0-137-20 kubenswrapper[2565]: I0917 18:24:16.587819    2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/5a89f15e-8368-4d4e-8bfb-be3196e8a6ce-kube-rbac-proxy-sar-config" (OuterVolumeSpecName: "kube-rbac-proxy-sar-config") pod "5a89f15e-8368-4d4e-8bfb-be3196e8a6ce" (UID: "5a89f15e-8368-4d4e-8bfb-be3196e8a6ce"). InnerVolumeSpecName "kube-rbac-proxy-sar-config". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 17 18:24:16.589595 ip-10-0-137-20 kubenswrapper[2565]: I0917 18:24:16.589566    2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/5a89f15e-8368-4d4e-8bfb-be3196e8a6ce-proxy-tls" (OuterVolumeSpecName: "proxy-tls") pod "5a89f15e-8368-4d4e-8bfb-be3196e8a6ce" (UID: "5a89f15e-8368-4d4e-8bfb-be3196e8a6ce"). InnerVolumeSpecName "proxy-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 17 18:24:16.589741 ip-10-0-137-20 kubenswrapper[2565]: I0917 18:24:16.589708    2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/5a89f15e-8368-4d4e-8bfb-be3196e8a6ce-kube-api-access-589qc" (OuterVolumeSpecName: "kube-api-access-589qc") pod "5a89f15e-8368-4d4e-8bfb-be3196e8a6ce" (UID: "5a89f15e-8368-4d4e-8bfb-be3196e8a6ce"). InnerVolumeSpecName "kube-api-access-589qc". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 17 18:24:16.627837 ip-10-0-137-20 kubenswrapper[2565]: I0917 18:24:16.627813    2565 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/error-404-isvc-de145-predictor-bfc475677-phjm7"
Sep 17 18:24:16.688167 ip-10-0-137-20 kubenswrapper[2565]: I0917 18:24:16.688136    2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-7t972\" (UniqueName: \"kubernetes.io/projected/453eb767-5a5e-479c-8079-59038d055f71-kube-api-access-7t972\") pod \"453eb767-5a5e-479c-8079-59038d055f71\" (UID: \"453eb767-5a5e-479c-8079-59038d055f71\") "
Sep 17 18:24:16.688360 ip-10-0-137-20 kubenswrapper[2565]: I0917 18:24:16.688181    2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/453eb767-5a5e-479c-8079-59038d055f71-kube-rbac-proxy-sar-config\") pod \"453eb767-5a5e-479c-8079-59038d055f71\" (UID: \"453eb767-5a5e-479c-8079-59038d055f71\") "
Sep 17 18:24:16.688360 ip-10-0-137-20 kubenswrapper[2565]: I0917 18:24:16.688207    2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/453eb767-5a5e-479c-8079-59038d055f71-proxy-tls\") pod \"453eb767-5a5e-479c-8079-59038d055f71\" (UID: \"453eb767-5a5e-479c-8079-59038d055f71\") "
Sep 17 18:24:16.688507 ip-10-0-137-20 kubenswrapper[2565]: I0917 18:24:16.688424    2565 reconciler_common.go:299] "Volume detached for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/5a89f15e-8368-4d4e-8bfb-be3196e8a6ce-proxy-tls\") on node \"ip-10-0-137-20.ec2.internal\" DevicePath \"\""
Sep 17 18:24:16.688507 ip-10-0-137-20 kubenswrapper[2565]: I0917 18:24:16.688444    2565 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-589qc\" (UniqueName: \"kubernetes.io/projected/5a89f15e-8368-4d4e-8bfb-be3196e8a6ce-kube-api-access-589qc\") on node \"ip-10-0-137-20.ec2.internal\" DevicePath \"\""
Sep 17 18:24:16.688507 ip-10-0-137-20 kubenswrapper[2565]: I0917 18:24:16.688458    2565 reconciler_common.go:299] "Volume detached for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/5a89f15e-8368-4d4e-8bfb-be3196e8a6ce-kube-rbac-proxy-sar-config\") on node \"ip-10-0-137-20.ec2.internal\" DevicePath \"\""
Sep 17 18:24:16.688658 ip-10-0-137-20 kubenswrapper[2565]: I0917 18:24:16.688635    2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/453eb767-5a5e-479c-8079-59038d055f71-kube-rbac-proxy-sar-config" (OuterVolumeSpecName: "kube-rbac-proxy-sar-config") pod "453eb767-5a5e-479c-8079-59038d055f71" (UID: "453eb767-5a5e-479c-8079-59038d055f71"). InnerVolumeSpecName "kube-rbac-proxy-sar-config". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 17 18:24:16.688798 ip-10-0-137-20 kubenswrapper[2565]: I0917 18:24:16.688770    2565 generic.go:356] "Generic (PLEG): container finished" podID="453eb767-5a5e-479c-8079-59038d055f71" containerID="31ee0ab236f0d5019ecee7f43155507ecfe5ef1ff614423379e8039436c04202" exitCode=0
Sep 17 18:24:16.688889 ip-10-0-137-20 kubenswrapper[2565]: I0917 18:24:16.688844    2565 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/error-404-isvc-de145-predictor-bfc475677-phjm7"
Sep 17 18:24:16.688889 ip-10-0-137-20 kubenswrapper[2565]: I0917 18:24:16.688870    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/error-404-isvc-de145-predictor-bfc475677-phjm7" event={"ID":"453eb767-5a5e-479c-8079-59038d055f71","Type":"ContainerDied","Data":"31ee0ab236f0d5019ecee7f43155507ecfe5ef1ff614423379e8039436c04202"}
Sep 17 18:24:16.688989 ip-10-0-137-20 kubenswrapper[2565]: I0917 18:24:16.688943    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/error-404-isvc-de145-predictor-bfc475677-phjm7" event={"ID":"453eb767-5a5e-479c-8079-59038d055f71","Type":"ContainerDied","Data":"6115fc9d47f96df452974aadd6686dd48f94c988d9382b7e38bd95df4883b778"}
Sep 17 18:24:16.689061 ip-10-0-137-20 kubenswrapper[2565]: I0917 18:24:16.688988    2565 scope.go:117] "RemoveContainer" containerID="c0b970644f36bf610b949d88fc56c56f5f7f1881458ae0ca73eee29b08278d65"
Sep 17 18:24:16.690672 ip-10-0-137-20 kubenswrapper[2565]: I0917 18:24:16.690648    2565 generic.go:356] "Generic (PLEG): container finished" podID="5a89f15e-8368-4d4e-8bfb-be3196e8a6ce" containerID="03c259b85cd2b358754bd7d040bc4143ef52976e121c54ad2c4013dba806fed3" exitCode=0
Sep 17 18:24:16.690785 ip-10-0-137-20 kubenswrapper[2565]: I0917 18:24:16.690680    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/success-200-isvc-de145-predictor-7dbfbb5484-b92v8" event={"ID":"5a89f15e-8368-4d4e-8bfb-be3196e8a6ce","Type":"ContainerDied","Data":"03c259b85cd2b358754bd7d040bc4143ef52976e121c54ad2c4013dba806fed3"}
Sep 17 18:24:16.690785 ip-10-0-137-20 kubenswrapper[2565]: I0917 18:24:16.690702    2565 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/success-200-isvc-de145-predictor-7dbfbb5484-b92v8"
Sep 17 18:24:16.690785 ip-10-0-137-20 kubenswrapper[2565]: I0917 18:24:16.690733    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/success-200-isvc-de145-predictor-7dbfbb5484-b92v8" event={"ID":"5a89f15e-8368-4d4e-8bfb-be3196e8a6ce","Type":"ContainerDied","Data":"1bd266471eba4fe013deddffc7d77e4e3daecb1d3221877826aaa05ed020afc9"}
Sep 17 18:24:16.690785 ip-10-0-137-20 kubenswrapper[2565]: I0917 18:24:16.690655    2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/453eb767-5a5e-479c-8079-59038d055f71-kube-api-access-7t972" (OuterVolumeSpecName: "kube-api-access-7t972") pod "453eb767-5a5e-479c-8079-59038d055f71" (UID: "453eb767-5a5e-479c-8079-59038d055f71"). InnerVolumeSpecName "kube-api-access-7t972". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 17 18:24:16.690785 ip-10-0-137-20 kubenswrapper[2565]: I0917 18:24:16.690757    2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/453eb767-5a5e-479c-8079-59038d055f71-proxy-tls" (OuterVolumeSpecName: "proxy-tls") pod "453eb767-5a5e-479c-8079-59038d055f71" (UID: "453eb767-5a5e-479c-8079-59038d055f71"). InnerVolumeSpecName "proxy-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 17 18:24:16.698261 ip-10-0-137-20 kubenswrapper[2565]: I0917 18:24:16.698242    2565 scope.go:117] "RemoveContainer" containerID="31ee0ab236f0d5019ecee7f43155507ecfe5ef1ff614423379e8039436c04202"
Sep 17 18:24:16.705323 ip-10-0-137-20 kubenswrapper[2565]: I0917 18:24:16.705308    2565 scope.go:117] "RemoveContainer" containerID="c0b970644f36bf610b949d88fc56c56f5f7f1881458ae0ca73eee29b08278d65"
Sep 17 18:24:16.705562 ip-10-0-137-20 kubenswrapper[2565]: E0917 18:24:16.705543    2565 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"c0b970644f36bf610b949d88fc56c56f5f7f1881458ae0ca73eee29b08278d65\": container with ID starting with c0b970644f36bf610b949d88fc56c56f5f7f1881458ae0ca73eee29b08278d65 not found: ID does not exist" containerID="c0b970644f36bf610b949d88fc56c56f5f7f1881458ae0ca73eee29b08278d65"
Sep 17 18:24:16.705600 ip-10-0-137-20 kubenswrapper[2565]: I0917 18:24:16.705571    2565 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"c0b970644f36bf610b949d88fc56c56f5f7f1881458ae0ca73eee29b08278d65"} err="failed to get container status \"c0b970644f36bf610b949d88fc56c56f5f7f1881458ae0ca73eee29b08278d65\": rpc error: code = NotFound desc = could not find container \"c0b970644f36bf610b949d88fc56c56f5f7f1881458ae0ca73eee29b08278d65\": container with ID starting with c0b970644f36bf610b949d88fc56c56f5f7f1881458ae0ca73eee29b08278d65 not found: ID does not exist"
Sep 17 18:24:16.705600 ip-10-0-137-20 kubenswrapper[2565]: I0917 18:24:16.705587    2565 scope.go:117] "RemoveContainer" containerID="31ee0ab236f0d5019ecee7f43155507ecfe5ef1ff614423379e8039436c04202"
Sep 17 18:24:16.705820 ip-10-0-137-20 kubenswrapper[2565]: E0917 18:24:16.705805    2565 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"31ee0ab236f0d5019ecee7f43155507ecfe5ef1ff614423379e8039436c04202\": container with ID starting with 31ee0ab236f0d5019ecee7f43155507ecfe5ef1ff614423379e8039436c04202 not found: ID does not exist" containerID="31ee0ab236f0d5019ecee7f43155507ecfe5ef1ff614423379e8039436c04202"
Sep 17 18:24:16.705857 ip-10-0-137-20 kubenswrapper[2565]: I0917 18:24:16.705826    2565 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"31ee0ab236f0d5019ecee7f43155507ecfe5ef1ff614423379e8039436c04202"} err="failed to get container status \"31ee0ab236f0d5019ecee7f43155507ecfe5ef1ff614423379e8039436c04202\": rpc error: code = NotFound desc = could not find container \"31ee0ab236f0d5019ecee7f43155507ecfe5ef1ff614423379e8039436c04202\": container with ID starting with 31ee0ab236f0d5019ecee7f43155507ecfe5ef1ff614423379e8039436c04202 not found: ID does not exist"
Sep 17 18:24:16.705857 ip-10-0-137-20 kubenswrapper[2565]: I0917 18:24:16.705840    2565 scope.go:117] "RemoveContainer" containerID="5eb717205ab4c89c4f30a282105989b511cb3981961d0886be067ade0cec8ba0"
Sep 17 18:24:16.713297 ip-10-0-137-20 kubenswrapper[2565]: I0917 18:24:16.713278    2565 scope.go:117] "RemoveContainer" containerID="03c259b85cd2b358754bd7d040bc4143ef52976e121c54ad2c4013dba806fed3"
Sep 17 18:24:16.713368 ip-10-0-137-20 kubenswrapper[2565]: I0917 18:24:16.713312    2565 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kserve-ci-e2e-test/success-200-isvc-de145-predictor-7dbfbb5484-b92v8"]
Sep 17 18:24:16.718647 ip-10-0-137-20 kubenswrapper[2565]: I0917 18:24:16.718625    2565 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kserve-ci-e2e-test/success-200-isvc-de145-predictor-7dbfbb5484-b92v8"]
Sep 17 18:24:16.720949 ip-10-0-137-20 kubenswrapper[2565]: I0917 18:24:16.720926    2565 scope.go:117] "RemoveContainer" containerID="5eb717205ab4c89c4f30a282105989b511cb3981961d0886be067ade0cec8ba0"
Sep 17 18:24:16.721235 ip-10-0-137-20 kubenswrapper[2565]: E0917 18:24:16.721218    2565 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"5eb717205ab4c89c4f30a282105989b511cb3981961d0886be067ade0cec8ba0\": container with ID starting with 5eb717205ab4c89c4f30a282105989b511cb3981961d0886be067ade0cec8ba0 not found: ID does not exist" containerID="5eb717205ab4c89c4f30a282105989b511cb3981961d0886be067ade0cec8ba0"
Sep 17 18:24:16.721297 ip-10-0-137-20 kubenswrapper[2565]: I0917 18:24:16.721243    2565 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"5eb717205ab4c89c4f30a282105989b511cb3981961d0886be067ade0cec8ba0"} err="failed to get container status \"5eb717205ab4c89c4f30a282105989b511cb3981961d0886be067ade0cec8ba0\": rpc error: code = NotFound desc = could not find container \"5eb717205ab4c89c4f30a282105989b511cb3981961d0886be067ade0cec8ba0\": container with ID starting with 5eb717205ab4c89c4f30a282105989b511cb3981961d0886be067ade0cec8ba0 not found: ID does not exist"
Sep 17 18:24:16.721297 ip-10-0-137-20 kubenswrapper[2565]: I0917 18:24:16.721266    2565 scope.go:117] "RemoveContainer" containerID="03c259b85cd2b358754bd7d040bc4143ef52976e121c54ad2c4013dba806fed3"
Sep 17 18:24:16.721534 ip-10-0-137-20 kubenswrapper[2565]: E0917 18:24:16.721517    2565 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"03c259b85cd2b358754bd7d040bc4143ef52976e121c54ad2c4013dba806fed3\": container with ID starting with 03c259b85cd2b358754bd7d040bc4143ef52976e121c54ad2c4013dba806fed3 not found: ID does not exist" containerID="03c259b85cd2b358754bd7d040bc4143ef52976e121c54ad2c4013dba806fed3"
Sep 17 18:24:16.721576 ip-10-0-137-20 kubenswrapper[2565]: I0917 18:24:16.721540    2565 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"03c259b85cd2b358754bd7d040bc4143ef52976e121c54ad2c4013dba806fed3"} err="failed to get container status \"03c259b85cd2b358754bd7d040bc4143ef52976e121c54ad2c4013dba806fed3\": rpc error: code = NotFound desc = could not find container \"03c259b85cd2b358754bd7d040bc4143ef52976e121c54ad2c4013dba806fed3\": container with ID starting with 03c259b85cd2b358754bd7d040bc4143ef52976e121c54ad2c4013dba806fed3 not found: ID does not exist"
Sep 17 18:24:16.788777 ip-10-0-137-20 kubenswrapper[2565]: I0917 18:24:16.788747    2565 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-7t972\" (UniqueName: \"kubernetes.io/projected/453eb767-5a5e-479c-8079-59038d055f71-kube-api-access-7t972\") on node \"ip-10-0-137-20.ec2.internal\" DevicePath \"\""
Sep 17 18:24:16.788777 ip-10-0-137-20 kubenswrapper[2565]: I0917 18:24:16.788770    2565 reconciler_common.go:299] "Volume detached for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/453eb767-5a5e-479c-8079-59038d055f71-kube-rbac-proxy-sar-config\") on node \"ip-10-0-137-20.ec2.internal\" DevicePath \"\""
Sep 17 18:24:16.788777 ip-10-0-137-20 kubenswrapper[2565]: I0917 18:24:16.788781    2565 reconciler_common.go:299] "Volume detached for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/453eb767-5a5e-479c-8079-59038d055f71-proxy-tls\") on node \"ip-10-0-137-20.ec2.internal\" DevicePath \"\""
Sep 17 18:24:17.012736 ip-10-0-137-20 kubenswrapper[2565]: I0917 18:24:17.012706    2565 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kserve-ci-e2e-test/error-404-isvc-de145-predictor-bfc475677-phjm7"]
Sep 17 18:24:17.020485 ip-10-0-137-20 kubenswrapper[2565]: I0917 18:24:17.020454    2565 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kserve-ci-e2e-test/error-404-isvc-de145-predictor-bfc475677-phjm7"]
Sep 17 18:24:18.157505 ip-10-0-137-20 kubenswrapper[2565]: I0917 18:24:18.157476    2565 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="453eb767-5a5e-479c-8079-59038d055f71" path="/var/lib/kubelet/pods/453eb767-5a5e-479c-8079-59038d055f71/volumes"
Sep 17 18:24:18.157905 ip-10-0-137-20 kubenswrapper[2565]: I0917 18:24:18.157891    2565 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="5a89f15e-8368-4d4e-8bfb-be3196e8a6ce" path="/var/lib/kubelet/pods/5a89f15e-8368-4d4e-8bfb-be3196e8a6ce/volumes"
Sep 17 18:24:48.619656 ip-10-0-137-20 kubenswrapper[2565]: I0917 18:24:48.619625    2565 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_global-pull-secret-syncer-74xd8_3d1c780d-6713-495e-91fb-54a74668e6cc/global-pull-secret-syncer/0.log"
Sep 17 18:24:48.765929 ip-10-0-137-20 kubenswrapper[2565]: I0917 18:24:48.765898    2565 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_konnectivity-agent-p8jd8_2e847ec5-02d2-4210-a6ce-89d926b050f3/konnectivity-agent/0.log"
Sep 17 18:24:48.863805 ip-10-0-137-20 kubenswrapper[2565]: I0917 18:24:48.863783    2565 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_kube-apiserver-proxy-ip-10-0-137-20.ec2.internal_cda455bc0aed3ae7411ab9a4c6df01be/haproxy/0.log"
Sep 17 18:24:52.395586 ip-10-0-137-20 kubenswrapper[2565]: I0917 18:24:52.395558    2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_43e20c01-6600-4445-aee3-1a90c7aac2cd/alertmanager/0.log"
Sep 17 18:24:52.469862 ip-10-0-137-20 kubenswrapper[2565]: I0917 18:24:52.469835    2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_43e20c01-6600-4445-aee3-1a90c7aac2cd/config-reloader/0.log"
Sep 17 18:24:52.537525 ip-10-0-137-20 kubenswrapper[2565]: I0917 18:24:52.537500    2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_43e20c01-6600-4445-aee3-1a90c7aac2cd/kube-rbac-proxy-web/0.log"
Sep 17 18:24:52.564740 ip-10-0-137-20 kubenswrapper[2565]: I0917 18:24:52.564720    2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_43e20c01-6600-4445-aee3-1a90c7aac2cd/kube-rbac-proxy/0.log"
Sep 17 18:24:52.590852 ip-10-0-137-20 kubenswrapper[2565]: I0917 18:24:52.590831    2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_43e20c01-6600-4445-aee3-1a90c7aac2cd/kube-rbac-proxy-metric/0.log"
Sep 17 18:24:52.620144 ip-10-0-137-20 kubenswrapper[2565]: I0917 18:24:52.620124    2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_43e20c01-6600-4445-aee3-1a90c7aac2cd/prom-label-proxy/0.log"
Sep 17 18:24:52.662122 ip-10-0-137-20 kubenswrapper[2565]: I0917 18:24:52.662065    2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_43e20c01-6600-4445-aee3-1a90c7aac2cd/init-config-reloader/0.log"
Sep 17 18:24:53.267121 ip-10-0-137-20 kubenswrapper[2565]: I0917 18:24:53.267098    2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-rkh9d_2e8b2e4c-5079-4454-a69a-9a183b1c1014/node-exporter/0.log"
Sep 17 18:24:53.315473 ip-10-0-137-20 kubenswrapper[2565]: I0917 18:24:53.315445    2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-rkh9d_2e8b2e4c-5079-4454-a69a-9a183b1c1014/kube-rbac-proxy/0.log"
Sep 17 18:24:53.338160 ip-10-0-137-20 kubenswrapper[2565]: I0917 18:24:53.338141    2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-rkh9d_2e8b2e4c-5079-4454-a69a-9a183b1c1014/init-textfile/0.log"
Sep 17 18:24:53.370895 ip-10-0-137-20 kubenswrapper[2565]: I0917 18:24:53.370877    2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_openshift-state-metrics-86c5bc79d6-28pph_d4a0864d-cd66-4cd1-9e93-4f39b40bf3c3/kube-rbac-proxy-main/0.log"
Sep 17 18:24:53.397623 ip-10-0-137-20 kubenswrapper[2565]: I0917 18:24:53.397602    2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_openshift-state-metrics-86c5bc79d6-28pph_d4a0864d-cd66-4cd1-9e93-4f39b40bf3c3/kube-rbac-proxy-self/0.log"
Sep 17 18:24:53.419714 ip-10-0-137-20 kubenswrapper[2565]: I0917 18:24:53.419693    2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_openshift-state-metrics-86c5bc79d6-28pph_d4a0864d-cd66-4cd1-9e93-4f39b40bf3c3/openshift-state-metrics/0.log"
Sep 17 18:24:53.647783 ip-10-0-137-20 kubenswrapper[2565]: I0917 18:24:53.647754    2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-operator-7bd5d56d6-lfpcg_14c8fb7d-dbe8-4246-b56e-cbded2092d3a/prometheus-operator/0.log"
Sep 17 18:24:53.667260 ip-10-0-137-20 kubenswrapper[2565]: I0917 18:24:53.667233    2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-operator-7bd5d56d6-lfpcg_14c8fb7d-dbe8-4246-b56e-cbded2092d3a/kube-rbac-proxy/0.log"
Sep 17 18:24:53.720474 ip-10-0-137-20 kubenswrapper[2565]: I0917 18:24:53.720445    2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_telemeter-client-8485894d99-zw8mr_aadb9010-ba56-493a-bb4e-8cbb1bf77555/telemeter-client/0.log"
Sep 17 18:24:53.740900 ip-10-0-137-20 kubenswrapper[2565]: I0917 18:24:53.740877    2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_telemeter-client-8485894d99-zw8mr_aadb9010-ba56-493a-bb4e-8cbb1bf77555/reload/0.log"
Sep 17 18:24:53.767741 ip-10-0-137-20 kubenswrapper[2565]: I0917 18:24:53.767722    2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_telemeter-client-8485894d99-zw8mr_aadb9010-ba56-493a-bb4e-8cbb1bf77555/kube-rbac-proxy/0.log"
Sep 17 18:24:56.236369 ip-10-0-137-20 kubenswrapper[2565]: I0917 18:24:56.236344    2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console_console-6c6b6c5656-hljcq_bed50f13-88fb-41c6-9894-0b88a4eea04d/console/0.log"
Sep 17 18:24:56.920944 ip-10-0-137-20 kubenswrapper[2565]: I0917 18:24:56.920911    2565 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-must-gather-xfnjl/perf-node-gather-daemonset-sh8ck"]
Sep 17 18:24:56.921269 ip-10-0-137-20 kubenswrapper[2565]: I0917 18:24:56.921257    2565 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="772aca13-71ef-4f5a-b76b-82e7451ea20a" containerName="kserve-container"
Sep 17 18:24:56.921331 ip-10-0-137-20 kubenswrapper[2565]: I0917 18:24:56.921270    2565 state_mem.go:107] "Deleted CPUSet assignment" podUID="772aca13-71ef-4f5a-b76b-82e7451ea20a" containerName="kserve-container"
Sep 17 18:24:56.921331 ip-10-0-137-20 kubenswrapper[2565]: I0917 18:24:56.921282    2565 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="7af4e9c8-bc7e-4cf3-a1b3-c28cfb1a0ff7" containerName="kube-rbac-proxy"
Sep 17 18:24:56.921331 ip-10-0-137-20 kubenswrapper[2565]: I0917 18:24:56.921287    2565 state_mem.go:107] "Deleted CPUSet assignment" podUID="7af4e9c8-bc7e-4cf3-a1b3-c28cfb1a0ff7" containerName="kube-rbac-proxy"
Sep 17 18:24:56.921331 ip-10-0-137-20 kubenswrapper[2565]: I0917 18:24:56.921294    2565 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="453eb767-5a5e-479c-8079-59038d055f71" containerName="kserve-container"
Sep 17 18:24:56.921331 ip-10-0-137-20 kubenswrapper[2565]: I0917 18:24:56.921300    2565 state_mem.go:107] "Deleted CPUSet assignment" podUID="453eb767-5a5e-479c-8079-59038d055f71" containerName="kserve-container"
Sep 17 18:24:56.921331 ip-10-0-137-20 kubenswrapper[2565]: I0917 18:24:56.921307    2565 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="198a6626-b57d-460d-a29b-a7932f83006a" containerName="kube-rbac-proxy"
Sep 17 18:24:56.921331 ip-10-0-137-20 kubenswrapper[2565]: I0917 18:24:56.921312    2565 state_mem.go:107] "Deleted CPUSet assignment" podUID="198a6626-b57d-460d-a29b-a7932f83006a" containerName="kube-rbac-proxy"
Sep 17 18:24:56.921331 ip-10-0-137-20 kubenswrapper[2565]: I0917 18:24:56.921320    2565 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="4a1f1981-8b92-46be-942d-e4657b35d2ce" containerName="kserve-container"
Sep 17 18:24:56.921331 ip-10-0-137-20 kubenswrapper[2565]: I0917 18:24:56.921326    2565 state_mem.go:107] "Deleted CPUSet assignment" podUID="4a1f1981-8b92-46be-942d-e4657b35d2ce" containerName="kserve-container"
Sep 17 18:24:56.921331 ip-10-0-137-20 kubenswrapper[2565]: I0917 18:24:56.921331    2565 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="4a1f1981-8b92-46be-942d-e4657b35d2ce" containerName="kube-rbac-proxy"
Sep 17 18:24:56.921331 ip-10-0-137-20 kubenswrapper[2565]: I0917 18:24:56.921336    2565 state_mem.go:107] "Deleted CPUSet assignment" podUID="4a1f1981-8b92-46be-942d-e4657b35d2ce" containerName="kube-rbac-proxy"
Sep 17 18:24:56.921695 ip-10-0-137-20 kubenswrapper[2565]: I0917 18:24:56.921342    2565 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="453eb767-5a5e-479c-8079-59038d055f71" containerName="kube-rbac-proxy"
Sep 17 18:24:56.921695 ip-10-0-137-20 kubenswrapper[2565]: I0917 18:24:56.921347    2565 state_mem.go:107] "Deleted CPUSet assignment" podUID="453eb767-5a5e-479c-8079-59038d055f71" containerName="kube-rbac-proxy"
Sep 17 18:24:56.921695 ip-10-0-137-20 kubenswrapper[2565]: I0917 18:24:56.921353    2565 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="198a6626-b57d-460d-a29b-a7932f83006a" containerName="kserve-container"
Sep 17 18:24:56.921695 ip-10-0-137-20 kubenswrapper[2565]: I0917 18:24:56.921358    2565 state_mem.go:107] "Deleted CPUSet assignment" podUID="198a6626-b57d-460d-a29b-a7932f83006a" containerName="kserve-container"
Sep 17 18:24:56.921695 ip-10-0-137-20 kubenswrapper[2565]: I0917 18:24:56.921366    2565 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="8dfad140-1f0d-494d-825c-dd4e3391d778" containerName="kube-rbac-proxy"
Sep 17 18:24:56.921695 ip-10-0-137-20 kubenswrapper[2565]: I0917 18:24:56.921370    2565 state_mem.go:107] "Deleted CPUSet assignment" podUID="8dfad140-1f0d-494d-825c-dd4e3391d778" containerName="kube-rbac-proxy"
Sep 17 18:24:56.921695 ip-10-0-137-20 kubenswrapper[2565]: I0917 18:24:56.921379    2565 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="aeb32779-c788-4434-8d88-5ec5d08e7144" containerName="kserve-container"
Sep 17 18:24:56.921695 ip-10-0-137-20 kubenswrapper[2565]: I0917 18:24:56.921384    2565 state_mem.go:107] "Deleted CPUSet assignment" podUID="aeb32779-c788-4434-8d88-5ec5d08e7144" containerName="kserve-container"
Sep 17 18:24:56.921695 ip-10-0-137-20 kubenswrapper[2565]: I0917 18:24:56.921392    2565 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="5a89f15e-8368-4d4e-8bfb-be3196e8a6ce" containerName="kube-rbac-proxy"
Sep 17 18:24:56.921695 ip-10-0-137-20 kubenswrapper[2565]: I0917 18:24:56.921397    2565 state_mem.go:107] "Deleted CPUSet assignment" podUID="5a89f15e-8368-4d4e-8bfb-be3196e8a6ce" containerName="kube-rbac-proxy"
Sep 17 18:24:56.921695 ip-10-0-137-20 kubenswrapper[2565]: I0917 18:24:56.921403    2565 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="8dfad140-1f0d-494d-825c-dd4e3391d778" containerName="kserve-container"
Sep 17 18:24:56.921695 ip-10-0-137-20 kubenswrapper[2565]: I0917 18:24:56.921407    2565 state_mem.go:107] "Deleted CPUSet assignment" podUID="8dfad140-1f0d-494d-825c-dd4e3391d778" containerName="kserve-container"
Sep 17 18:24:56.921695 ip-10-0-137-20 kubenswrapper[2565]: I0917 18:24:56.921413    2565 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="7af4e9c8-bc7e-4cf3-a1b3-c28cfb1a0ff7" containerName="kserve-container"
Sep 17 18:24:56.921695 ip-10-0-137-20 kubenswrapper[2565]: I0917 18:24:56.921418    2565 state_mem.go:107] "Deleted CPUSet assignment" podUID="7af4e9c8-bc7e-4cf3-a1b3-c28cfb1a0ff7" containerName="kserve-container"
Sep 17 18:24:56.921695 ip-10-0-137-20 kubenswrapper[2565]: I0917 18:24:56.921423    2565 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="aeb32779-c788-4434-8d88-5ec5d08e7144" containerName="kube-rbac-proxy"
Sep 17 18:24:56.921695 ip-10-0-137-20 kubenswrapper[2565]: I0917 18:24:56.921428    2565 state_mem.go:107] "Deleted CPUSet assignment" podUID="aeb32779-c788-4434-8d88-5ec5d08e7144" containerName="kube-rbac-proxy"
Sep 17 18:24:56.921695 ip-10-0-137-20 kubenswrapper[2565]: I0917 18:24:56.921435    2565 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="772aca13-71ef-4f5a-b76b-82e7451ea20a" containerName="kube-rbac-proxy"
Sep 17 18:24:56.921695 ip-10-0-137-20 kubenswrapper[2565]: I0917 18:24:56.921439    2565 state_mem.go:107] "Deleted CPUSet assignment" podUID="772aca13-71ef-4f5a-b76b-82e7451ea20a" containerName="kube-rbac-proxy"
Sep 17 18:24:56.921695 ip-10-0-137-20 kubenswrapper[2565]: I0917 18:24:56.921445    2565 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="5a89f15e-8368-4d4e-8bfb-be3196e8a6ce" containerName="kserve-container"
Sep 17 18:24:56.921695 ip-10-0-137-20 kubenswrapper[2565]: I0917 18:24:56.921459    2565 state_mem.go:107] "Deleted CPUSet assignment" podUID="5a89f15e-8368-4d4e-8bfb-be3196e8a6ce" containerName="kserve-container"
Sep 17 18:24:56.921695 ip-10-0-137-20 kubenswrapper[2565]: I0917 18:24:56.921512    2565 memory_manager.go:368] "RemoveStaleState removing state" podUID="5a89f15e-8368-4d4e-8bfb-be3196e8a6ce" containerName="kserve-container"
Sep 17 18:24:56.921695 ip-10-0-137-20 kubenswrapper[2565]: I0917 18:24:56.921519    2565 memory_manager.go:368] "RemoveStaleState removing state" podUID="8dfad140-1f0d-494d-825c-dd4e3391d778" containerName="kserve-container"
Sep 17 18:24:56.921695 ip-10-0-137-20 kubenswrapper[2565]: I0917 18:24:56.921525    2565 memory_manager.go:368] "RemoveStaleState removing state" podUID="4a1f1981-8b92-46be-942d-e4657b35d2ce" containerName="kserve-container"
Sep 17 18:24:56.921695 ip-10-0-137-20 kubenswrapper[2565]: I0917 18:24:56.921530    2565 memory_manager.go:368] "RemoveStaleState removing state" podUID="198a6626-b57d-460d-a29b-a7932f83006a" containerName="kserve-container"
Sep 17 18:24:56.921695 ip-10-0-137-20 kubenswrapper[2565]: I0917 18:24:56.921534    2565 memory_manager.go:368] "RemoveStaleState removing state" podUID="453eb767-5a5e-479c-8079-59038d055f71" containerName="kube-rbac-proxy"
Sep 17 18:24:56.921695 ip-10-0-137-20 kubenswrapper[2565]: I0917 18:24:56.921540    2565 memory_manager.go:368] "RemoveStaleState removing state" podUID="198a6626-b57d-460d-a29b-a7932f83006a" containerName="kube-rbac-proxy"
Sep 17 18:24:56.921695 ip-10-0-137-20 kubenswrapper[2565]: I0917 18:24:56.921546    2565 memory_manager.go:368] "RemoveStaleState removing state" podUID="8dfad140-1f0d-494d-825c-dd4e3391d778" containerName="kube-rbac-proxy"
Sep 17 18:24:56.921695 ip-10-0-137-20 kubenswrapper[2565]: I0917 18:24:56.921550    2565 memory_manager.go:368] "RemoveStaleState removing state" podUID="aeb32779-c788-4434-8d88-5ec5d08e7144" containerName="kserve-container"
Sep 17 18:24:56.921695 ip-10-0-137-20 kubenswrapper[2565]: I0917 18:24:56.921555    2565 memory_manager.go:368] "RemoveStaleState removing state" podUID="453eb767-5a5e-479c-8079-59038d055f71" containerName="kserve-container"
Sep 17 18:24:56.921695 ip-10-0-137-20 kubenswrapper[2565]: I0917 18:24:56.921560    2565 memory_manager.go:368] "RemoveStaleState removing state" podUID="772aca13-71ef-4f5a-b76b-82e7451ea20a" containerName="kube-rbac-proxy"
Sep 17 18:24:56.921695 ip-10-0-137-20 kubenswrapper[2565]: I0917 18:24:56.921566    2565 memory_manager.go:368] "RemoveStaleState removing state" podUID="7af4e9c8-bc7e-4cf3-a1b3-c28cfb1a0ff7" containerName="kserve-container"
Sep 17 18:24:56.921695 ip-10-0-137-20 kubenswrapper[2565]: I0917 18:24:56.921571    2565 memory_manager.go:368] "RemoveStaleState removing state" podUID="7af4e9c8-bc7e-4cf3-a1b3-c28cfb1a0ff7" containerName="kube-rbac-proxy"
Sep 17 18:24:56.921695 ip-10-0-137-20 kubenswrapper[2565]: I0917 18:24:56.921576    2565 memory_manager.go:368] "RemoveStaleState removing state" podUID="4a1f1981-8b92-46be-942d-e4657b35d2ce" containerName="kube-rbac-proxy"
Sep 17 18:24:56.921695 ip-10-0-137-20 kubenswrapper[2565]: I0917 18:24:56.921581    2565 memory_manager.go:368] "RemoveStaleState removing state" podUID="772aca13-71ef-4f5a-b76b-82e7451ea20a" containerName="kserve-container"
Sep 17 18:24:56.921695 ip-10-0-137-20 kubenswrapper[2565]: I0917 18:24:56.921586    2565 memory_manager.go:368] "RemoveStaleState removing state" podUID="5a89f15e-8368-4d4e-8bfb-be3196e8a6ce" containerName="kube-rbac-proxy"
Sep 17 18:24:56.921695 ip-10-0-137-20 kubenswrapper[2565]: I0917 18:24:56.921591    2565 memory_manager.go:368] "RemoveStaleState removing state" podUID="aeb32779-c788-4434-8d88-5ec5d08e7144" containerName="kube-rbac-proxy"
Sep 17 18:24:56.924634 ip-10-0-137-20 kubenswrapper[2565]: I0917 18:24:56.924617    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-xfnjl/perf-node-gather-daemonset-sh8ck"
Sep 17 18:24:56.927469 ip-10-0-137-20 kubenswrapper[2565]: I0917 18:24:56.927448    2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-must-gather-xfnjl\"/\"default-dockercfg-mwcsm\""
Sep 17 18:24:56.928678 ip-10-0-137-20 kubenswrapper[2565]: I0917 18:24:56.928661    2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-xfnjl\"/\"kube-root-ca.crt\""
Sep 17 18:24:56.928751 ip-10-0-137-20 kubenswrapper[2565]: I0917 18:24:56.928721    2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-xfnjl\"/\"openshift-service-ca.crt\""
Sep 17 18:24:56.933009 ip-10-0-137-20 kubenswrapper[2565]: I0917 18:24:56.932990    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-xfnjl/perf-node-gather-daemonset-sh8ck"]
Sep 17 18:24:56.995269 ip-10-0-137-20 kubenswrapper[2565]: I0917 18:24:56.995241    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/d21ecd52-d974-456d-95a6-dc32c6d86422-lib-modules\") pod \"perf-node-gather-daemonset-sh8ck\" (UID: \"d21ecd52-d974-456d-95a6-dc32c6d86422\") " pod="openshift-must-gather-xfnjl/perf-node-gather-daemonset-sh8ck"
Sep 17 18:24:56.995413 ip-10-0-137-20 kubenswrapper[2565]: I0917 18:24:56.995300    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/d21ecd52-d974-456d-95a6-dc32c6d86422-proc\") pod \"perf-node-gather-daemonset-sh8ck\" (UID: \"d21ecd52-d974-456d-95a6-dc32c6d86422\") " pod="openshift-must-gather-xfnjl/perf-node-gather-daemonset-sh8ck"
Sep 17 18:24:56.995413 ip-10-0-137-20 kubenswrapper[2565]: I0917 18:24:56.995320    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-w4r8s\" (UniqueName: \"kubernetes.io/projected/d21ecd52-d974-456d-95a6-dc32c6d86422-kube-api-access-w4r8s\") pod \"perf-node-gather-daemonset-sh8ck\" (UID: \"d21ecd52-d974-456d-95a6-dc32c6d86422\") " pod="openshift-must-gather-xfnjl/perf-node-gather-daemonset-sh8ck"
Sep 17 18:24:56.995413 ip-10-0-137-20 kubenswrapper[2565]: I0917 18:24:56.995346    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/d21ecd52-d974-456d-95a6-dc32c6d86422-podres\") pod \"perf-node-gather-daemonset-sh8ck\" (UID: \"d21ecd52-d974-456d-95a6-dc32c6d86422\") " pod="openshift-must-gather-xfnjl/perf-node-gather-daemonset-sh8ck"
Sep 17 18:24:56.995413 ip-10-0-137-20 kubenswrapper[2565]: I0917 18:24:56.995365    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/d21ecd52-d974-456d-95a6-dc32c6d86422-sys\") pod \"perf-node-gather-daemonset-sh8ck\" (UID: \"d21ecd52-d974-456d-95a6-dc32c6d86422\") " pod="openshift-must-gather-xfnjl/perf-node-gather-daemonset-sh8ck"
Sep 17 18:24:57.096624 ip-10-0-137-20 kubenswrapper[2565]: I0917 18:24:57.096577    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-w4r8s\" (UniqueName: \"kubernetes.io/projected/d21ecd52-d974-456d-95a6-dc32c6d86422-kube-api-access-w4r8s\") pod \"perf-node-gather-daemonset-sh8ck\" (UID: \"d21ecd52-d974-456d-95a6-dc32c6d86422\") " pod="openshift-must-gather-xfnjl/perf-node-gather-daemonset-sh8ck"
Sep 17 18:24:57.096834 ip-10-0-137-20 kubenswrapper[2565]: I0917 18:24:57.096643    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/d21ecd52-d974-456d-95a6-dc32c6d86422-podres\") pod \"perf-node-gather-daemonset-sh8ck\" (UID: \"d21ecd52-d974-456d-95a6-dc32c6d86422\") " pod="openshift-must-gather-xfnjl/perf-node-gather-daemonset-sh8ck"
Sep 17 18:24:57.096834 ip-10-0-137-20 kubenswrapper[2565]: I0917 18:24:57.096668    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/d21ecd52-d974-456d-95a6-dc32c6d86422-sys\") pod \"perf-node-gather-daemonset-sh8ck\" (UID: \"d21ecd52-d974-456d-95a6-dc32c6d86422\") " pod="openshift-must-gather-xfnjl/perf-node-gather-daemonset-sh8ck"
Sep 17 18:24:57.096834 ip-10-0-137-20 kubenswrapper[2565]: I0917 18:24:57.096719    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/d21ecd52-d974-456d-95a6-dc32c6d86422-lib-modules\") pod \"perf-node-gather-daemonset-sh8ck\" (UID: \"d21ecd52-d974-456d-95a6-dc32c6d86422\") " pod="openshift-must-gather-xfnjl/perf-node-gather-daemonset-sh8ck"
Sep 17 18:24:57.096834 ip-10-0-137-20 kubenswrapper[2565]: I0917 18:24:57.096789    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/d21ecd52-d974-456d-95a6-dc32c6d86422-proc\") pod \"perf-node-gather-daemonset-sh8ck\" (UID: \"d21ecd52-d974-456d-95a6-dc32c6d86422\") " pod="openshift-must-gather-xfnjl/perf-node-gather-daemonset-sh8ck"
Sep 17 18:24:57.096834 ip-10-0-137-20 kubenswrapper[2565]: I0917 18:24:57.096811    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/d21ecd52-d974-456d-95a6-dc32c6d86422-podres\") pod \"perf-node-gather-daemonset-sh8ck\" (UID: \"d21ecd52-d974-456d-95a6-dc32c6d86422\") " pod="openshift-must-gather-xfnjl/perf-node-gather-daemonset-sh8ck"
Sep 17 18:24:57.097078 ip-10-0-137-20 kubenswrapper[2565]: I0917 18:24:57.096838    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/d21ecd52-d974-456d-95a6-dc32c6d86422-sys\") pod \"perf-node-gather-daemonset-sh8ck\" (UID: \"d21ecd52-d974-456d-95a6-dc32c6d86422\") " pod="openshift-must-gather-xfnjl/perf-node-gather-daemonset-sh8ck"
Sep 17 18:24:57.097078 ip-10-0-137-20 kubenswrapper[2565]: I0917 18:24:57.096861    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/d21ecd52-d974-456d-95a6-dc32c6d86422-proc\") pod \"perf-node-gather-daemonset-sh8ck\" (UID: \"d21ecd52-d974-456d-95a6-dc32c6d86422\") " pod="openshift-must-gather-xfnjl/perf-node-gather-daemonset-sh8ck"
Sep 17 18:24:57.097078 ip-10-0-137-20 kubenswrapper[2565]: I0917 18:24:57.096896    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/d21ecd52-d974-456d-95a6-dc32c6d86422-lib-modules\") pod \"perf-node-gather-daemonset-sh8ck\" (UID: \"d21ecd52-d974-456d-95a6-dc32c6d86422\") " pod="openshift-must-gather-xfnjl/perf-node-gather-daemonset-sh8ck"
Sep 17 18:24:57.106196 ip-10-0-137-20 kubenswrapper[2565]: I0917 18:24:57.106177    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-w4r8s\" (UniqueName: \"kubernetes.io/projected/d21ecd52-d974-456d-95a6-dc32c6d86422-kube-api-access-w4r8s\") pod \"perf-node-gather-daemonset-sh8ck\" (UID: \"d21ecd52-d974-456d-95a6-dc32c6d86422\") " pod="openshift-must-gather-xfnjl/perf-node-gather-daemonset-sh8ck"
Sep 17 18:24:57.236183 ip-10-0-137-20 kubenswrapper[2565]: I0917 18:24:57.236095    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-xfnjl/perf-node-gather-daemonset-sh8ck"
Sep 17 18:24:57.509729 ip-10-0-137-20 kubenswrapper[2565]: I0917 18:24:57.509705    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-xfnjl/perf-node-gather-daemonset-sh8ck"]
Sep 17 18:24:57.511503 ip-10-0-137-20 kubenswrapper[2565]: W0917 18:24:57.511478    2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-podd21ecd52_d974_456d_95a6_dc32c6d86422.slice/crio-f70caad01f7a5cae732689f08c920393104856da6be2b50cc8bf9f636a1049e1 WatchSource:0}: Error finding container f70caad01f7a5cae732689f08c920393104856da6be2b50cc8bf9f636a1049e1: Status 404 returned error can't find the container with id f70caad01f7a5cae732689f08c920393104856da6be2b50cc8bf9f636a1049e1
Sep 17 18:24:57.712231 ip-10-0-137-20 kubenswrapper[2565]: I0917 18:24:57.712201    2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-mj76m_054b367d-f6ce-4337-ab2b-cab945fc918e/dns/0.log"
Sep 17 18:24:57.731839 ip-10-0-137-20 kubenswrapper[2565]: I0917 18:24:57.731814    2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-mj76m_054b367d-f6ce-4337-ab2b-cab945fc918e/kube-rbac-proxy/0.log"
Sep 17 18:24:57.844776 ip-10-0-137-20 kubenswrapper[2565]: I0917 18:24:57.844705    2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-vs64q_caa60d4e-b46b-4fff-8a66-3a5b4f1a1602/dns-node-resolver/0.log"
Sep 17 18:24:57.867849 ip-10-0-137-20 kubenswrapper[2565]: I0917 18:24:57.867823    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-xfnjl/perf-node-gather-daemonset-sh8ck" event={"ID":"d21ecd52-d974-456d-95a6-dc32c6d86422","Type":"ContainerStarted","Data":"af042aedf097546f854cc2d94d3e1f6f5f183ec275c37756c963725cc7cdb692"}
Sep 17 18:24:57.868038 ip-10-0-137-20 kubenswrapper[2565]: I0917 18:24:57.867906    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-xfnjl/perf-node-gather-daemonset-sh8ck" event={"ID":"d21ecd52-d974-456d-95a6-dc32c6d86422","Type":"ContainerStarted","Data":"f70caad01f7a5cae732689f08c920393104856da6be2b50cc8bf9f636a1049e1"}
Sep 17 18:24:57.868038 ip-10-0-137-20 kubenswrapper[2565]: I0917 18:24:57.867946    2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-must-gather-xfnjl/perf-node-gather-daemonset-sh8ck"
Sep 17 18:24:57.886802 ip-10-0-137-20 kubenswrapper[2565]: I0917 18:24:57.886758    2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-must-gather-xfnjl/perf-node-gather-daemonset-sh8ck" podStartSLOduration=1.8867432910000002 podStartE2EDuration="1.886743291s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-17 18:24:56 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-17 18:24:57.885282866 +0000 UTC m=+3047.762567281" watchObservedRunningTime="2026-09-17 18:24:57.886743291 +0000 UTC m=+3047.764027706"
Sep 17 18:24:58.343308 ip-10-0-137-20 kubenswrapper[2565]: I0917 18:24:58.343279    2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-5blw5_27b114f9-fcc4-463b-897f-9e4561ad5540/node-ca/0.log"
Sep 17 18:24:59.517752 ip-10-0-137-20 kubenswrapper[2565]: I0917 18:24:59.517722    2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-cqzrc_f8f9debb-8e49-4e10-b950-bc0e3305c30e/serve-healthcheck-canary/0.log"
Sep 17 18:24:59.947017 ip-10-0-137-20 kubenswrapper[2565]: I0917 18:24:59.946990    2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-6mqqs_fd16e5c0-6ecb-473a-84c0-bbee28f44e68/kube-rbac-proxy/0.log"
Sep 17 18:24:59.967708 ip-10-0-137-20 kubenswrapper[2565]: I0917 18:24:59.967684    2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-6mqqs_fd16e5c0-6ecb-473a-84c0-bbee28f44e68/exporter/0.log"
Sep 17 18:24:59.987249 ip-10-0-137-20 kubenswrapper[2565]: I0917 18:24:59.987228    2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-6mqqs_fd16e5c0-6ecb-473a-84c0-bbee28f44e68/extractor/0.log"
Sep 17 18:25:02.386620 ip-10-0-137-20 kubenswrapper[2565]: I0917 18:25:02.386593    2565 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve_kserve-controller-manager-9b487cb79-78k8l_4950e910-c87f-4318-9cbd-25eac814a99f/manager/0.log"
Sep 17 18:25:02.414310 ip-10-0-137-20 kubenswrapper[2565]: I0917 18:25:02.414285    2565 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve_llmisvc-controller-manager-645f9c8fd6-h9x48_6e815937-ab94-4f42-a671-e2fc88bb7df3/manager/0.log"
Sep 17 18:25:02.434310 ip-10-0-137-20 kubenswrapper[2565]: I0917 18:25:02.434289    2565 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve_model-serving-api-7b954ffc95-sgr9b_86cff6b1-9d41-4174-8e3d-75b8728e0739/server/0.log"
Sep 17 18:25:02.524169 ip-10-0-137-20 kubenswrapper[2565]: I0917 18:25:02.524138    2565 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve_odh-model-controller-7b966d7984-pjq5z_b54f7b04-81e8-4a44-bf33-49b39b9051c3/manager/0.log"
Sep 17 18:25:03.882298 ip-10-0-137-20 kubenswrapper[2565]: I0917 18:25:03.882272    2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-must-gather-xfnjl/perf-node-gather-daemonset-sh8ck"
Sep 17 18:25:07.868356 ip-10-0-137-20 kubenswrapper[2565]: I0917 18:25:07.868330    2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-5l5k7_2feeabe5-1136-4f30-9a14-491423e838a3/kube-multus/0.log"
Sep 17 18:25:08.219859 ip-10-0-137-20 kubenswrapper[2565]: I0917 18:25:08.219835    2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-msv4b_048bee56-5e42-45f0-98a8-213ff62de69e/kube-multus-additional-cni-plugins/0.log"
Sep 17 18:25:08.243104 ip-10-0-137-20 kubenswrapper[2565]: I0917 18:25:08.243080    2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-msv4b_048bee56-5e42-45f0-98a8-213ff62de69e/egress-router-binary-copy/0.log"
Sep 17 18:25:08.264730 ip-10-0-137-20 kubenswrapper[2565]: I0917 18:25:08.264706    2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-msv4b_048bee56-5e42-45f0-98a8-213ff62de69e/cni-plugins/0.log"
Sep 17 18:25:08.285511 ip-10-0-137-20 kubenswrapper[2565]: I0917 18:25:08.285492    2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-msv4b_048bee56-5e42-45f0-98a8-213ff62de69e/bond-cni-plugin/0.log"
Sep 17 18:25:08.309088 ip-10-0-137-20 kubenswrapper[2565]: I0917 18:25:08.309065    2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-msv4b_048bee56-5e42-45f0-98a8-213ff62de69e/routeoverride-cni/0.log"
Sep 17 18:25:08.330124 ip-10-0-137-20 kubenswrapper[2565]: I0917 18:25:08.330104    2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-msv4b_048bee56-5e42-45f0-98a8-213ff62de69e/whereabouts-cni-bincopy/0.log"
Sep 17 18:25:08.352985 ip-10-0-137-20 kubenswrapper[2565]: I0917 18:25:08.352960    2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-msv4b_048bee56-5e42-45f0-98a8-213ff62de69e/whereabouts-cni/0.log"
Sep 17 18:25:08.531924 ip-10-0-137-20 kubenswrapper[2565]: I0917 18:25:08.531896    2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_network-metrics-daemon-rmlqx_e39c5e1e-113d-42f2-9a00-8a6d14cb6d67/network-metrics-daemon/0.log"
Sep 17 18:25:08.549603 ip-10-0-137-20 kubenswrapper[2565]: I0917 18:25:08.549582    2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_network-metrics-daemon-rmlqx_e39c5e1e-113d-42f2-9a00-8a6d14cb6d67/kube-rbac-proxy/0.log"
Sep 17 18:25:09.259441 ip-10-0-137-20 kubenswrapper[2565]: I0917 18:25:09.259407    2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-d9927_30837de7-1447-4de2-ac62-c809813b3338/ovn-controller/0.log"
Sep 17 18:25:09.277720 ip-10-0-137-20 kubenswrapper[2565]: I0917 18:25:09.277697    2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-d9927_30837de7-1447-4de2-ac62-c809813b3338/ovn-acl-logging/0.log"
Sep 17 18:25:09.277846 ip-10-0-137-20 kubenswrapper[2565]: I0917 18:25:09.277721    2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-d9927_30837de7-1447-4de2-ac62-c809813b3338/ovn-acl-logging/1.log"
Sep 17 18:25:09.295632 ip-10-0-137-20 kubenswrapper[2565]: I0917 18:25:09.295601    2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-d9927_30837de7-1447-4de2-ac62-c809813b3338/kube-rbac-proxy-node/0.log"
Sep 17 18:25:09.316395 ip-10-0-137-20 kubenswrapper[2565]: I0917 18:25:09.316367    2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-d9927_30837de7-1447-4de2-ac62-c809813b3338/kube-rbac-proxy-ovn-metrics/0.log"
Sep 17 18:25:09.335058 ip-10-0-137-20 kubenswrapper[2565]: I0917 18:25:09.335034    2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-d9927_30837de7-1447-4de2-ac62-c809813b3338/northd/0.log"
Sep 17 18:25:09.354878 ip-10-0-137-20 kubenswrapper[2565]: I0917 18:25:09.354859    2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-d9927_30837de7-1447-4de2-ac62-c809813b3338/nbdb/0.log"
Sep 17 18:25:09.374145 ip-10-0-137-20 kubenswrapper[2565]: I0917 18:25:09.374123    2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-d9927_30837de7-1447-4de2-ac62-c809813b3338/sbdb/0.log"
Sep 17 18:25:09.413522 ip-10-0-137-20 kubenswrapper[2565]: I0917 18:25:09.413501    2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-d9927_30837de7-1447-4de2-ac62-c809813b3338/ovnkube-controller/0.log"
Sep 17 18:25:10.827244 ip-10-0-137-20 kubenswrapper[2565]: I0917 18:25:10.827216    2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-network-diagnostics_network-check-target-jsc2p_574f2e00-d361-423b-946b-e1e9ea3a5db5/network-check-target-container/0.log"
Sep 17 18:25:11.726830 ip-10-0-137-20 kubenswrapper[2565]: I0917 18:25:11.726795    2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-network-operator_iptables-alerter-g8kpq_af1cc6fe-9695-4e2e-9244-6505022f6c50/iptables-alerter/0.log"
Sep 17 18:25:12.379384 ip-10-0-137-20 kubenswrapper[2565]: I0917 18:25:12.379358    2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-cluster-node-tuning-operator_tuned-mf8sz_d4013629-eb04-40d7-bdb1-11e1cafa9394/tuned/0.log"