Jul 01 16:58:49.405301 ip-10-0-137-244 systemd[1]: Starting Kubernetes Kubelet...
Jul 01 16:58:49.892394 ip-10-0-137-244 kubenswrapper[2565]: Flag --container-runtime-endpoint has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Jul 01 16:58:49.892394 ip-10-0-137-244 kubenswrapper[2565]: Flag --minimum-container-ttl-duration has been deprecated, Use --eviction-hard or --eviction-soft instead. Will be removed in a future version.
Jul 01 16:58:49.892394 ip-10-0-137-244 kubenswrapper[2565]: Flag --volume-plugin-dir has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Jul 01 16:58:49.892394 ip-10-0-137-244 kubenswrapper[2565]: Flag --pod-infra-container-image has been deprecated, will be removed in 1.35. Image garbage collector will get sandbox image information from CRI.
Jul 01 16:58:49.892394 ip-10-0-137-244 kubenswrapper[2565]: Flag --system-reserved has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Jul 01 16:58:49.892394 ip-10-0-137-244 kubenswrapper[2565]: I0701 16:58:49.840693    2565 server.go:213] "--pod-infra-container-image will not be pruned by the image garbage collector in kubelet and should also be set in the remote runtime"
Jul 01 16:58:49.892394 ip-10-0-137-244 kubenswrapper[2565]: W0701 16:58:49.847041    2565 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 01 16:58:49.892394 ip-10-0-137-244 kubenswrapper[2565]: W0701 16:58:49.847052    2565 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 01 16:58:49.892394 ip-10-0-137-244 kubenswrapper[2565]: W0701 16:58:49.847056    2565 feature_gate.go:328] unrecognized feature gate: Example
Jul 01 16:58:49.892394 ip-10-0-137-244 kubenswrapper[2565]: W0701 16:58:49.847062    2565 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 01 16:58:49.892394 ip-10-0-137-244 kubenswrapper[2565]: W0701 16:58:49.847066    2565 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 01 16:58:49.892394 ip-10-0-137-244 kubenswrapper[2565]: W0701 16:58:49.847069    2565 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 01 16:58:49.892394 ip-10-0-137-244 kubenswrapper[2565]: W0701 16:58:49.847072    2565 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 01 16:58:49.892394 ip-10-0-137-244 kubenswrapper[2565]: W0701 16:58:49.847075    2565 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 01 16:58:49.893928 ip-10-0-137-244 kubenswrapper[2565]: W0701 16:58:49.847078    2565 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 01 16:58:49.893928 ip-10-0-137-244 kubenswrapper[2565]: W0701 16:58:49.847082    2565 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 01 16:58:49.893928 ip-10-0-137-244 kubenswrapper[2565]: W0701 16:58:49.847085    2565 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 01 16:58:49.893928 ip-10-0-137-244 kubenswrapper[2565]: W0701 16:58:49.847088    2565 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 01 16:58:49.893928 ip-10-0-137-244 kubenswrapper[2565]: W0701 16:58:49.847091    2565 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 01 16:58:49.893928 ip-10-0-137-244 kubenswrapper[2565]: W0701 16:58:49.847094    2565 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 01 16:58:49.893928 ip-10-0-137-244 kubenswrapper[2565]: W0701 16:58:49.847097    2565 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 01 16:58:49.893928 ip-10-0-137-244 kubenswrapper[2565]: W0701 16:58:49.847108    2565 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 01 16:58:49.893928 ip-10-0-137-244 kubenswrapper[2565]: W0701 16:58:49.847111    2565 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 01 16:58:49.893928 ip-10-0-137-244 kubenswrapper[2565]: W0701 16:58:49.847114    2565 feature_gate.go:328] unrecognized feature gate: Example2
Jul 01 16:58:49.893928 ip-10-0-137-244 kubenswrapper[2565]: W0701 16:58:49.847117    2565 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 01 16:58:49.893928 ip-10-0-137-244 kubenswrapper[2565]: W0701 16:58:49.847120    2565 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 01 16:58:49.893928 ip-10-0-137-244 kubenswrapper[2565]: W0701 16:58:49.847123    2565 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 01 16:58:49.893928 ip-10-0-137-244 kubenswrapper[2565]: W0701 16:58:49.847125    2565 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 01 16:58:49.893928 ip-10-0-137-244 kubenswrapper[2565]: W0701 16:58:49.847128    2565 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 01 16:58:49.893928 ip-10-0-137-244 kubenswrapper[2565]: W0701 16:58:49.847130    2565 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 01 16:58:49.893928 ip-10-0-137-244 kubenswrapper[2565]: W0701 16:58:49.847133    2565 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 01 16:58:49.893928 ip-10-0-137-244 kubenswrapper[2565]: W0701 16:58:49.847136    2565 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 01 16:58:49.893928 ip-10-0-137-244 kubenswrapper[2565]: W0701 16:58:49.847138    2565 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 01 16:58:49.893928 ip-10-0-137-244 kubenswrapper[2565]: W0701 16:58:49.847141    2565 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 01 16:58:49.894792 ip-10-0-137-244 kubenswrapper[2565]: W0701 16:58:49.847144    2565 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 01 16:58:49.894792 ip-10-0-137-244 kubenswrapper[2565]: W0701 16:58:49.847146    2565 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 01 16:58:49.894792 ip-10-0-137-244 kubenswrapper[2565]: W0701 16:58:49.847149    2565 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 01 16:58:49.894792 ip-10-0-137-244 kubenswrapper[2565]: W0701 16:58:49.847152    2565 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 01 16:58:49.894792 ip-10-0-137-244 kubenswrapper[2565]: W0701 16:58:49.847154    2565 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 01 16:58:49.894792 ip-10-0-137-244 kubenswrapper[2565]: W0701 16:58:49.847157    2565 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 01 16:58:49.894792 ip-10-0-137-244 kubenswrapper[2565]: W0701 16:58:49.847159    2565 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 01 16:58:49.894792 ip-10-0-137-244 kubenswrapper[2565]: W0701 16:58:49.847163    2565 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 01 16:58:49.894792 ip-10-0-137-244 kubenswrapper[2565]: W0701 16:58:49.847166    2565 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 01 16:58:49.894792 ip-10-0-137-244 kubenswrapper[2565]: W0701 16:58:49.847169    2565 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 01 16:58:49.894792 ip-10-0-137-244 kubenswrapper[2565]: W0701 16:58:49.847172    2565 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 01 16:58:49.894792 ip-10-0-137-244 kubenswrapper[2565]: W0701 16:58:49.847174    2565 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 01 16:58:49.894792 ip-10-0-137-244 kubenswrapper[2565]: W0701 16:58:49.847177    2565 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 01 16:58:49.894792 ip-10-0-137-244 kubenswrapper[2565]: W0701 16:58:49.847180    2565 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 01 16:58:49.894792 ip-10-0-137-244 kubenswrapper[2565]: W0701 16:58:49.847182    2565 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 01 16:58:49.894792 ip-10-0-137-244 kubenswrapper[2565]: W0701 16:58:49.847185    2565 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 01 16:58:49.894792 ip-10-0-137-244 kubenswrapper[2565]: W0701 16:58:49.847187    2565 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 01 16:58:49.894792 ip-10-0-137-244 kubenswrapper[2565]: W0701 16:58:49.847190    2565 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 01 16:58:49.894792 ip-10-0-137-244 kubenswrapper[2565]: W0701 16:58:49.847192    2565 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 01 16:58:49.895665 ip-10-0-137-244 kubenswrapper[2565]: W0701 16:58:49.847195    2565 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 01 16:58:49.895665 ip-10-0-137-244 kubenswrapper[2565]: W0701 16:58:49.847198    2565 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 01 16:58:49.895665 ip-10-0-137-244 kubenswrapper[2565]: W0701 16:58:49.847200    2565 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 01 16:58:49.895665 ip-10-0-137-244 kubenswrapper[2565]: W0701 16:58:49.847203    2565 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 01 16:58:49.895665 ip-10-0-137-244 kubenswrapper[2565]: W0701 16:58:49.847205    2565 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 01 16:58:49.895665 ip-10-0-137-244 kubenswrapper[2565]: W0701 16:58:49.847209    2565 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 01 16:58:49.895665 ip-10-0-137-244 kubenswrapper[2565]: W0701 16:58:49.847211    2565 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 01 16:58:49.895665 ip-10-0-137-244 kubenswrapper[2565]: W0701 16:58:49.847214    2565 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 01 16:58:49.895665 ip-10-0-137-244 kubenswrapper[2565]: W0701 16:58:49.847216    2565 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 01 16:58:49.895665 ip-10-0-137-244 kubenswrapper[2565]: W0701 16:58:49.847219    2565 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 01 16:58:49.895665 ip-10-0-137-244 kubenswrapper[2565]: W0701 16:58:49.847221    2565 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 01 16:58:49.895665 ip-10-0-137-244 kubenswrapper[2565]: W0701 16:58:49.847224    2565 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 01 16:58:49.895665 ip-10-0-137-244 kubenswrapper[2565]: W0701 16:58:49.847226    2565 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 01 16:58:49.895665 ip-10-0-137-244 kubenswrapper[2565]: W0701 16:58:49.847229    2565 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 01 16:58:49.895665 ip-10-0-137-244 kubenswrapper[2565]: W0701 16:58:49.847232    2565 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 01 16:58:49.895665 ip-10-0-137-244 kubenswrapper[2565]: W0701 16:58:49.847234    2565 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 01 16:58:49.895665 ip-10-0-137-244 kubenswrapper[2565]: W0701 16:58:49.847237    2565 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 01 16:58:49.895665 ip-10-0-137-244 kubenswrapper[2565]: W0701 16:58:49.847239    2565 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 01 16:58:49.895665 ip-10-0-137-244 kubenswrapper[2565]: W0701 16:58:49.847242    2565 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 01 16:58:49.895665 ip-10-0-137-244 kubenswrapper[2565]: W0701 16:58:49.847244    2565 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 01 16:58:49.896529 ip-10-0-137-244 kubenswrapper[2565]: W0701 16:58:49.847247    2565 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 01 16:58:49.896529 ip-10-0-137-244 kubenswrapper[2565]: W0701 16:58:49.847249    2565 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 01 16:58:49.896529 ip-10-0-137-244 kubenswrapper[2565]: W0701 16:58:49.847252    2565 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 01 16:58:49.896529 ip-10-0-137-244 kubenswrapper[2565]: W0701 16:58:49.847254    2565 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 01 16:58:49.896529 ip-10-0-137-244 kubenswrapper[2565]: W0701 16:58:49.847257    2565 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 01 16:58:49.896529 ip-10-0-137-244 kubenswrapper[2565]: W0701 16:58:49.847260    2565 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 01 16:58:49.896529 ip-10-0-137-244 kubenswrapper[2565]: W0701 16:58:49.847262    2565 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 01 16:58:49.896529 ip-10-0-137-244 kubenswrapper[2565]: W0701 16:58:49.847265    2565 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 01 16:58:49.896529 ip-10-0-137-244 kubenswrapper[2565]: W0701 16:58:49.847267    2565 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 01 16:58:49.896529 ip-10-0-137-244 kubenswrapper[2565]: W0701 16:58:49.847270    2565 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 01 16:58:49.896529 ip-10-0-137-244 kubenswrapper[2565]: W0701 16:58:49.847272    2565 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 01 16:58:49.896529 ip-10-0-137-244 kubenswrapper[2565]: W0701 16:58:49.847275    2565 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 01 16:58:49.896529 ip-10-0-137-244 kubenswrapper[2565]: W0701 16:58:49.847277    2565 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 01 16:58:49.896529 ip-10-0-137-244 kubenswrapper[2565]: W0701 16:58:49.847280    2565 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 01 16:58:49.896529 ip-10-0-137-244 kubenswrapper[2565]: W0701 16:58:49.847282    2565 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 01 16:58:49.896529 ip-10-0-137-244 kubenswrapper[2565]: W0701 16:58:49.847285    2565 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 01 16:58:49.896529 ip-10-0-137-244 kubenswrapper[2565]: W0701 16:58:49.847287    2565 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 01 16:58:49.896529 ip-10-0-137-244 kubenswrapper[2565]: W0701 16:58:49.847290    2565 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 01 16:58:49.896529 ip-10-0-137-244 kubenswrapper[2565]: W0701 16:58:49.847292    2565 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 01 16:58:49.896529 ip-10-0-137-244 kubenswrapper[2565]: W0701 16:58:49.847295    2565 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 01 16:58:49.897416 ip-10-0-137-244 kubenswrapper[2565]: W0701 16:58:49.847297    2565 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 01 16:58:49.897416 ip-10-0-137-244 kubenswrapper[2565]: W0701 16:58:49.847301    2565 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 01 16:58:49.897416 ip-10-0-137-244 kubenswrapper[2565]: W0701 16:58:49.847304    2565 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 01 16:58:49.897416 ip-10-0-137-244 kubenswrapper[2565]: W0701 16:58:49.847307    2565 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 01 16:58:49.897416 ip-10-0-137-244 kubenswrapper[2565]: W0701 16:58:49.847310    2565 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 01 16:58:49.897416 ip-10-0-137-244 kubenswrapper[2565]: W0701 16:58:49.847313    2565 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 01 16:58:49.897416 ip-10-0-137-244 kubenswrapper[2565]: W0701 16:58:49.847316    2565 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 01 16:58:49.897416 ip-10-0-137-244 kubenswrapper[2565]: W0701 16:58:49.847318    2565 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 01 16:58:49.897416 ip-10-0-137-244 kubenswrapper[2565]: W0701 16:58:49.847321    2565 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 01 16:58:49.897416 ip-10-0-137-244 kubenswrapper[2565]: W0701 16:58:49.847324    2565 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 01 16:58:49.897416 ip-10-0-137-244 kubenswrapper[2565]: W0701 16:58:49.847327    2565 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 01 16:58:49.897416 ip-10-0-137-244 kubenswrapper[2565]: W0701 16:58:49.847330    2565 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 01 16:58:49.897416 ip-10-0-137-244 kubenswrapper[2565]: W0701 16:58:49.848967    2565 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 01 16:58:49.897416 ip-10-0-137-244 kubenswrapper[2565]: W0701 16:58:49.848978    2565 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 01 16:58:49.897416 ip-10-0-137-244 kubenswrapper[2565]: W0701 16:58:49.848982    2565 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 01 16:58:49.897416 ip-10-0-137-244 kubenswrapper[2565]: W0701 16:58:49.848985    2565 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 01 16:58:49.897416 ip-10-0-137-244 kubenswrapper[2565]: W0701 16:58:49.848988    2565 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 01 16:58:49.897416 ip-10-0-137-244 kubenswrapper[2565]: W0701 16:58:49.848990    2565 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 01 16:58:49.897416 ip-10-0-137-244 kubenswrapper[2565]: W0701 16:58:49.848993    2565 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 01 16:58:49.897416 ip-10-0-137-244 kubenswrapper[2565]: W0701 16:58:49.848996    2565 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 01 16:58:49.898966 ip-10-0-137-244 kubenswrapper[2565]: W0701 16:58:49.848999    2565 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 01 16:58:49.898966 ip-10-0-137-244 kubenswrapper[2565]: W0701 16:58:49.849001    2565 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 01 16:58:49.898966 ip-10-0-137-244 kubenswrapper[2565]: W0701 16:58:49.849004    2565 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 01 16:58:49.898966 ip-10-0-137-244 kubenswrapper[2565]: W0701 16:58:49.849006    2565 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 01 16:58:49.898966 ip-10-0-137-244 kubenswrapper[2565]: W0701 16:58:49.849011    2565 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 01 16:58:49.898966 ip-10-0-137-244 kubenswrapper[2565]: W0701 16:58:49.849016    2565 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 01 16:58:49.898966 ip-10-0-137-244 kubenswrapper[2565]: W0701 16:58:49.849041    2565 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 01 16:58:49.898966 ip-10-0-137-244 kubenswrapper[2565]: W0701 16:58:49.849045    2565 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 01 16:58:49.898966 ip-10-0-137-244 kubenswrapper[2565]: W0701 16:58:49.849049    2565 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 01 16:58:49.898966 ip-10-0-137-244 kubenswrapper[2565]: W0701 16:58:49.849053    2565 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 01 16:58:49.898966 ip-10-0-137-244 kubenswrapper[2565]: W0701 16:58:49.849058    2565 feature_gate.go:328] unrecognized feature gate: Example2
Jul 01 16:58:49.898966 ip-10-0-137-244 kubenswrapper[2565]: W0701 16:58:49.849062    2565 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 01 16:58:49.898966 ip-10-0-137-244 kubenswrapper[2565]: W0701 16:58:49.849066    2565 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 01 16:58:49.898966 ip-10-0-137-244 kubenswrapper[2565]: W0701 16:58:49.849070    2565 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 01 16:58:49.898966 ip-10-0-137-244 kubenswrapper[2565]: W0701 16:58:49.849072    2565 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 01 16:58:49.898966 ip-10-0-137-244 kubenswrapper[2565]: W0701 16:58:49.849075    2565 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 01 16:58:49.898966 ip-10-0-137-244 kubenswrapper[2565]: W0701 16:58:49.849078    2565 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 01 16:58:49.898966 ip-10-0-137-244 kubenswrapper[2565]: W0701 16:58:49.849081    2565 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 01 16:58:49.898966 ip-10-0-137-244 kubenswrapper[2565]: W0701 16:58:49.849084    2565 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 01 16:58:49.898966 ip-10-0-137-244 kubenswrapper[2565]: W0701 16:58:49.849087    2565 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 01 16:58:49.900279 ip-10-0-137-244 kubenswrapper[2565]: W0701 16:58:49.849090    2565 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 01 16:58:49.900279 ip-10-0-137-244 kubenswrapper[2565]: W0701 16:58:49.849093    2565 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 01 16:58:49.900279 ip-10-0-137-244 kubenswrapper[2565]: W0701 16:58:49.849095    2565 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 01 16:58:49.900279 ip-10-0-137-244 kubenswrapper[2565]: W0701 16:58:49.849098    2565 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 01 16:58:49.900279 ip-10-0-137-244 kubenswrapper[2565]: W0701 16:58:49.849100    2565 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 01 16:58:49.900279 ip-10-0-137-244 kubenswrapper[2565]: W0701 16:58:49.849103    2565 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 01 16:58:49.900279 ip-10-0-137-244 kubenswrapper[2565]: W0701 16:58:49.849106    2565 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 01 16:58:49.900279 ip-10-0-137-244 kubenswrapper[2565]: W0701 16:58:49.849108    2565 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 01 16:58:49.900279 ip-10-0-137-244 kubenswrapper[2565]: W0701 16:58:49.849111    2565 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 01 16:58:49.900279 ip-10-0-137-244 kubenswrapper[2565]: W0701 16:58:49.849114    2565 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 01 16:58:49.900279 ip-10-0-137-244 kubenswrapper[2565]: W0701 16:58:49.849117    2565 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 01 16:58:49.900279 ip-10-0-137-244 kubenswrapper[2565]: W0701 16:58:49.849119    2565 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 01 16:58:49.900279 ip-10-0-137-244 kubenswrapper[2565]: W0701 16:58:49.849122    2565 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 01 16:58:49.900279 ip-10-0-137-244 kubenswrapper[2565]: W0701 16:58:49.849125    2565 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 01 16:58:49.900279 ip-10-0-137-244 kubenswrapper[2565]: W0701 16:58:49.849128    2565 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 01 16:58:49.900279 ip-10-0-137-244 kubenswrapper[2565]: W0701 16:58:49.849131    2565 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 01 16:58:49.900279 ip-10-0-137-244 kubenswrapper[2565]: W0701 16:58:49.849135    2565 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 01 16:58:49.900279 ip-10-0-137-244 kubenswrapper[2565]: W0701 16:58:49.849137    2565 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 01 16:58:49.900279 ip-10-0-137-244 kubenswrapper[2565]: W0701 16:58:49.849140    2565 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 01 16:58:49.900279 ip-10-0-137-244 kubenswrapper[2565]: W0701 16:58:49.849143    2565 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 01 16:58:49.901400 ip-10-0-137-244 kubenswrapper[2565]: W0701 16:58:49.849146    2565 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 01 16:58:49.901400 ip-10-0-137-244 kubenswrapper[2565]: W0701 16:58:49.849148    2565 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 01 16:58:49.901400 ip-10-0-137-244 kubenswrapper[2565]: W0701 16:58:49.849151    2565 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 01 16:58:49.901400 ip-10-0-137-244 kubenswrapper[2565]: W0701 16:58:49.849154    2565 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 01 16:58:49.901400 ip-10-0-137-244 kubenswrapper[2565]: W0701 16:58:49.849156    2565 feature_gate.go:328] unrecognized feature gate: Example
Jul 01 16:58:49.901400 ip-10-0-137-244 kubenswrapper[2565]: W0701 16:58:49.849159    2565 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 01 16:58:49.901400 ip-10-0-137-244 kubenswrapper[2565]: W0701 16:58:49.849162    2565 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 01 16:58:49.901400 ip-10-0-137-244 kubenswrapper[2565]: W0701 16:58:49.849164    2565 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 01 16:58:49.901400 ip-10-0-137-244 kubenswrapper[2565]: W0701 16:58:49.849166    2565 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 01 16:58:49.901400 ip-10-0-137-244 kubenswrapper[2565]: W0701 16:58:49.849169    2565 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 01 16:58:49.901400 ip-10-0-137-244 kubenswrapper[2565]: W0701 16:58:49.849172    2565 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 01 16:58:49.901400 ip-10-0-137-244 kubenswrapper[2565]: W0701 16:58:49.849174    2565 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 01 16:58:49.901400 ip-10-0-137-244 kubenswrapper[2565]: W0701 16:58:49.849177    2565 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 01 16:58:49.901400 ip-10-0-137-244 kubenswrapper[2565]: W0701 16:58:49.849179    2565 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 01 16:58:49.901400 ip-10-0-137-244 kubenswrapper[2565]: W0701 16:58:49.849182    2565 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 01 16:58:49.901400 ip-10-0-137-244 kubenswrapper[2565]: W0701 16:58:49.849184    2565 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 01 16:58:49.901400 ip-10-0-137-244 kubenswrapper[2565]: W0701 16:58:49.849188    2565 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 01 16:58:49.901400 ip-10-0-137-244 kubenswrapper[2565]: W0701 16:58:49.849192    2565 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 01 16:58:49.901400 ip-10-0-137-244 kubenswrapper[2565]: W0701 16:58:49.849195    2565 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 01 16:58:49.902519 ip-10-0-137-244 kubenswrapper[2565]: W0701 16:58:49.849198    2565 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 01 16:58:49.902519 ip-10-0-137-244 kubenswrapper[2565]: W0701 16:58:49.849201    2565 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 01 16:58:49.902519 ip-10-0-137-244 kubenswrapper[2565]: W0701 16:58:49.849204    2565 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 01 16:58:49.902519 ip-10-0-137-244 kubenswrapper[2565]: W0701 16:58:49.849206    2565 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 01 16:58:49.902519 ip-10-0-137-244 kubenswrapper[2565]: W0701 16:58:49.849210    2565 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 01 16:58:49.902519 ip-10-0-137-244 kubenswrapper[2565]: W0701 16:58:49.849213    2565 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 01 16:58:49.902519 ip-10-0-137-244 kubenswrapper[2565]: W0701 16:58:49.849215    2565 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 01 16:58:49.902519 ip-10-0-137-244 kubenswrapper[2565]: W0701 16:58:49.849218    2565 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 01 16:58:49.902519 ip-10-0-137-244 kubenswrapper[2565]: W0701 16:58:49.849221    2565 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 01 16:58:49.902519 ip-10-0-137-244 kubenswrapper[2565]: W0701 16:58:49.849225    2565 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 01 16:58:49.902519 ip-10-0-137-244 kubenswrapper[2565]: W0701 16:58:49.849228    2565 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 01 16:58:49.902519 ip-10-0-137-244 kubenswrapper[2565]: W0701 16:58:49.849231    2565 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 01 16:58:49.902519 ip-10-0-137-244 kubenswrapper[2565]: W0701 16:58:49.849234    2565 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 01 16:58:49.902519 ip-10-0-137-244 kubenswrapper[2565]: W0701 16:58:49.849237    2565 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 01 16:58:49.902519 ip-10-0-137-244 kubenswrapper[2565]: W0701 16:58:49.849239    2565 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 01 16:58:49.902519 ip-10-0-137-244 kubenswrapper[2565]: W0701 16:58:49.849242    2565 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 01 16:58:49.902519 ip-10-0-137-244 kubenswrapper[2565]: W0701 16:58:49.849245    2565 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 01 16:58:49.902519 ip-10-0-137-244 kubenswrapper[2565]: W0701 16:58:49.849248    2565 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 01 16:58:49.902519 ip-10-0-137-244 kubenswrapper[2565]: W0701 16:58:49.849251    2565 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 01 16:58:49.903895 ip-10-0-137-244 kubenswrapper[2565]: W0701 16:58:49.849253    2565 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 01 16:58:49.903895 ip-10-0-137-244 kubenswrapper[2565]: W0701 16:58:49.849256    2565 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 01 16:58:49.903895 ip-10-0-137-244 kubenswrapper[2565]: W0701 16:58:49.849258    2565 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 01 16:58:49.903895 ip-10-0-137-244 kubenswrapper[2565]: W0701 16:58:49.849261    2565 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 01 16:58:49.903895 ip-10-0-137-244 kubenswrapper[2565]: W0701 16:58:49.849263    2565 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 01 16:58:49.903895 ip-10-0-137-244 kubenswrapper[2565]: W0701 16:58:49.849266    2565 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 01 16:58:49.903895 ip-10-0-137-244 kubenswrapper[2565]: W0701 16:58:49.849268    2565 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 01 16:58:49.903895 ip-10-0-137-244 kubenswrapper[2565]: W0701 16:58:49.849270    2565 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 01 16:58:49.903895 ip-10-0-137-244 kubenswrapper[2565]: W0701 16:58:49.849273    2565 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 01 16:58:49.903895 ip-10-0-137-244 kubenswrapper[2565]: W0701 16:58:49.849276    2565 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 01 16:58:49.903895 ip-10-0-137-244 kubenswrapper[2565]: W0701 16:58:49.849279    2565 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 01 16:58:49.903895 ip-10-0-137-244 kubenswrapper[2565]: W0701 16:58:49.849281    2565 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 01 16:58:49.903895 ip-10-0-137-244 kubenswrapper[2565]: W0701 16:58:49.849283    2565 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 01 16:58:49.903895 ip-10-0-137-244 kubenswrapper[2565]: W0701 16:58:49.849653    2565 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 01 16:58:49.903895 ip-10-0-137-244 kubenswrapper[2565]: W0701 16:58:49.849659    2565 feature_gate.go:328] unrecognized feature gate: Example
Jul 01 16:58:49.903895 ip-10-0-137-244 kubenswrapper[2565]: W0701 16:58:49.849662    2565 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 01 16:58:49.903895 ip-10-0-137-244 kubenswrapper[2565]: W0701 16:58:49.849665    2565 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 01 16:58:49.903895 ip-10-0-137-244 kubenswrapper[2565]: W0701 16:58:49.849668    2565 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 01 16:58:49.903895 ip-10-0-137-244 kubenswrapper[2565]: W0701 16:58:49.849671    2565 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 01 16:58:49.903895 ip-10-0-137-244 kubenswrapper[2565]: W0701 16:58:49.849674    2565 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 01 16:58:49.904786 ip-10-0-137-244 kubenswrapper[2565]: W0701 16:58:49.849677    2565 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 01 16:58:49.904786 ip-10-0-137-244 kubenswrapper[2565]: W0701 16:58:49.849680    2565 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 01 16:58:49.904786 ip-10-0-137-244 kubenswrapper[2565]: W0701 16:58:49.849683    2565 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 01 16:58:49.904786 ip-10-0-137-244 kubenswrapper[2565]: W0701 16:58:49.849685    2565 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 01 16:58:49.904786 ip-10-0-137-244 kubenswrapper[2565]: W0701 16:58:49.849688    2565 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 01 16:58:49.904786 ip-10-0-137-244 kubenswrapper[2565]: W0701 16:58:49.849692    2565 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 01 16:58:49.904786 ip-10-0-137-244 kubenswrapper[2565]: W0701 16:58:49.849695    2565 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 01 16:58:49.904786 ip-10-0-137-244 kubenswrapper[2565]: W0701 16:58:49.849697    2565 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 01 16:58:49.904786 ip-10-0-137-244 kubenswrapper[2565]: W0701 16:58:49.849700    2565 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 01 16:58:49.904786 ip-10-0-137-244 kubenswrapper[2565]: W0701 16:58:49.849703    2565 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 01 16:58:49.904786 ip-10-0-137-244 kubenswrapper[2565]: W0701 16:58:49.849707    2565 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 01 16:58:49.904786 ip-10-0-137-244 kubenswrapper[2565]: W0701 16:58:49.849711    2565 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 01 16:58:49.904786 ip-10-0-137-244 kubenswrapper[2565]: W0701 16:58:49.849713    2565 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 01 16:58:49.904786 ip-10-0-137-244 kubenswrapper[2565]: W0701 16:58:49.849716    2565 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 01 16:58:49.904786 ip-10-0-137-244 kubenswrapper[2565]: W0701 16:58:49.849718    2565 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 01 16:58:49.904786 ip-10-0-137-244 kubenswrapper[2565]: W0701 16:58:49.849721    2565 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 01 16:58:49.904786 ip-10-0-137-244 kubenswrapper[2565]: W0701 16:58:49.849724    2565 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 01 16:58:49.904786 ip-10-0-137-244 kubenswrapper[2565]: W0701 16:58:49.849729    2565 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 01 16:58:49.904786 ip-10-0-137-244 kubenswrapper[2565]: W0701 16:58:49.849732    2565 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 01 16:58:49.904786 ip-10-0-137-244 kubenswrapper[2565]: W0701 16:58:49.849735    2565 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 01 16:58:49.906336 ip-10-0-137-244 kubenswrapper[2565]: W0701 16:58:49.849738    2565 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 01 16:58:49.906336 ip-10-0-137-244 kubenswrapper[2565]: W0701 16:58:49.849740    2565 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 01 16:58:49.906336 ip-10-0-137-244 kubenswrapper[2565]: W0701 16:58:49.849743    2565 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 01 16:58:49.906336 ip-10-0-137-244 kubenswrapper[2565]: W0701 16:58:49.849745    2565 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 01 16:58:49.906336 ip-10-0-137-244 kubenswrapper[2565]: W0701 16:58:49.849748    2565 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 01 16:58:49.906336 ip-10-0-137-244 kubenswrapper[2565]: W0701 16:58:49.849750    2565 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 01 16:58:49.906336 ip-10-0-137-244 kubenswrapper[2565]: W0701 16:58:49.849753    2565 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 01 16:58:49.906336 ip-10-0-137-244 kubenswrapper[2565]: W0701 16:58:49.849756    2565 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 01 16:58:49.906336 ip-10-0-137-244 kubenswrapper[2565]: W0701 16:58:49.849758    2565 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 01 16:58:49.906336 ip-10-0-137-244 kubenswrapper[2565]: W0701 16:58:49.849760    2565 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 01 16:58:49.906336 ip-10-0-137-244 kubenswrapper[2565]: W0701 16:58:49.849764    2565 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 01 16:58:49.906336 ip-10-0-137-244 kubenswrapper[2565]: W0701 16:58:49.849766    2565 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 01 16:58:49.906336 ip-10-0-137-244 kubenswrapper[2565]: W0701 16:58:49.849769    2565 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 01 16:58:49.906336 ip-10-0-137-244 kubenswrapper[2565]: W0701 16:58:49.849771    2565 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 01 16:58:49.906336 ip-10-0-137-244 kubenswrapper[2565]: W0701 16:58:49.849774    2565 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 01 16:58:49.906336 ip-10-0-137-244 kubenswrapper[2565]: W0701 16:58:49.849776    2565 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 01 16:58:49.906336 ip-10-0-137-244 kubenswrapper[2565]: W0701 16:58:49.849779    2565 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 01 16:58:49.906336 ip-10-0-137-244 kubenswrapper[2565]: W0701 16:58:49.849781    2565 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 01 16:58:49.906336 ip-10-0-137-244 kubenswrapper[2565]: W0701 16:58:49.849784    2565 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 01 16:58:49.907227 ip-10-0-137-244 kubenswrapper[2565]: W0701 16:58:49.849786    2565 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 01 16:58:49.907227 ip-10-0-137-244 kubenswrapper[2565]: W0701 16:58:49.849789    2565 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 01 16:58:49.907227 ip-10-0-137-244 kubenswrapper[2565]: W0701 16:58:49.849791    2565 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 01 16:58:49.907227 ip-10-0-137-244 kubenswrapper[2565]: W0701 16:58:49.849794    2565 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 01 16:58:49.907227 ip-10-0-137-244 kubenswrapper[2565]: W0701 16:58:49.849797    2565 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 01 16:58:49.907227 ip-10-0-137-244 kubenswrapper[2565]: W0701 16:58:49.849799    2565 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 01 16:58:49.907227 ip-10-0-137-244 kubenswrapper[2565]: W0701 16:58:49.849802    2565 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 01 16:58:49.907227 ip-10-0-137-244 kubenswrapper[2565]: W0701 16:58:49.849805    2565 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 01 16:58:49.907227 ip-10-0-137-244 kubenswrapper[2565]: W0701 16:58:49.849808    2565 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 01 16:58:49.907227 ip-10-0-137-244 kubenswrapper[2565]: W0701 16:58:49.849810    2565 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 01 16:58:49.907227 ip-10-0-137-244 kubenswrapper[2565]: W0701 16:58:49.849813    2565 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 01 16:58:49.907227 ip-10-0-137-244 kubenswrapper[2565]: W0701 16:58:49.849816    2565 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 01 16:58:49.907227 ip-10-0-137-244 kubenswrapper[2565]: W0701 16:58:49.849818    2565 feature_gate.go:328] unrecognized feature gate: Example2
Jul 01 16:58:49.907227 ip-10-0-137-244 kubenswrapper[2565]: W0701 16:58:49.849821    2565 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 01 16:58:49.907227 ip-10-0-137-244 kubenswrapper[2565]: W0701 16:58:49.849823    2565 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 01 16:58:49.907227 ip-10-0-137-244 kubenswrapper[2565]: W0701 16:58:49.849825    2565 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 01 16:58:49.907227 ip-10-0-137-244 kubenswrapper[2565]: W0701 16:58:49.849828    2565 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 01 16:58:49.907227 ip-10-0-137-244 kubenswrapper[2565]: W0701 16:58:49.849831    2565 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 01 16:58:49.907227 ip-10-0-137-244 kubenswrapper[2565]: W0701 16:58:49.849833    2565 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 01 16:58:49.907227 ip-10-0-137-244 kubenswrapper[2565]: W0701 16:58:49.849836    2565 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 01 16:58:49.908768 ip-10-0-137-244 kubenswrapper[2565]: W0701 16:58:49.849841    2565 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 01 16:58:49.908768 ip-10-0-137-244 kubenswrapper[2565]: W0701 16:58:49.849844    2565 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 01 16:58:49.908768 ip-10-0-137-244 kubenswrapper[2565]: W0701 16:58:49.849846    2565 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 01 16:58:49.908768 ip-10-0-137-244 kubenswrapper[2565]: W0701 16:58:49.849849    2565 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 01 16:58:49.908768 ip-10-0-137-244 kubenswrapper[2565]: W0701 16:58:49.849852    2565 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 01 16:58:49.908768 ip-10-0-137-244 kubenswrapper[2565]: W0701 16:58:49.849854    2565 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 01 16:58:49.908768 ip-10-0-137-244 kubenswrapper[2565]: W0701 16:58:49.849857    2565 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 01 16:58:49.908768 ip-10-0-137-244 kubenswrapper[2565]: W0701 16:58:49.849859    2565 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 01 16:58:49.908768 ip-10-0-137-244 kubenswrapper[2565]: W0701 16:58:49.849862    2565 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 01 16:58:49.908768 ip-10-0-137-244 kubenswrapper[2565]: W0701 16:58:49.849864    2565 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 01 16:58:49.908768 ip-10-0-137-244 kubenswrapper[2565]: W0701 16:58:49.849867    2565 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 01 16:58:49.908768 ip-10-0-137-244 kubenswrapper[2565]: W0701 16:58:49.849869    2565 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 01 16:58:49.908768 ip-10-0-137-244 kubenswrapper[2565]: W0701 16:58:49.849872    2565 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 01 16:58:49.908768 ip-10-0-137-244 kubenswrapper[2565]: W0701 16:58:49.849874    2565 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 01 16:58:49.908768 ip-10-0-137-244 kubenswrapper[2565]: W0701 16:58:49.849877    2565 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 01 16:58:49.908768 ip-10-0-137-244 kubenswrapper[2565]: W0701 16:58:49.849879    2565 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 01 16:58:49.908768 ip-10-0-137-244 kubenswrapper[2565]: W0701 16:58:49.849882    2565 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 01 16:58:49.908768 ip-10-0-137-244 kubenswrapper[2565]: W0701 16:58:49.849885    2565 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 01 16:58:49.908768 ip-10-0-137-244 kubenswrapper[2565]: W0701 16:58:49.849887    2565 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 01 16:58:49.908768 ip-10-0-137-244 kubenswrapper[2565]: W0701 16:58:49.849890    2565 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 01 16:58:49.909649 ip-10-0-137-244 kubenswrapper[2565]: W0701 16:58:49.849893    2565 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 01 16:58:49.909649 ip-10-0-137-244 kubenswrapper[2565]: W0701 16:58:49.849896    2565 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 01 16:58:49.909649 ip-10-0-137-244 kubenswrapper[2565]: W0701 16:58:49.849898    2565 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 01 16:58:49.909649 ip-10-0-137-244 kubenswrapper[2565]: W0701 16:58:49.849901    2565 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 01 16:58:49.909649 ip-10-0-137-244 kubenswrapper[2565]: W0701 16:58:49.849903    2565 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 01 16:58:49.909649 ip-10-0-137-244 kubenswrapper[2565]: W0701 16:58:49.849906    2565 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 01 16:58:49.909649 ip-10-0-137-244 kubenswrapper[2565]: W0701 16:58:49.849909    2565 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 01 16:58:49.909649 ip-10-0-137-244 kubenswrapper[2565]: W0701 16:58:49.849911    2565 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 01 16:58:49.909649 ip-10-0-137-244 kubenswrapper[2565]: W0701 16:58:49.849914    2565 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 01 16:58:49.909649 ip-10-0-137-244 kubenswrapper[2565]: W0701 16:58:49.849916    2565 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 01 16:58:49.909649 ip-10-0-137-244 kubenswrapper[2565]: W0701 16:58:49.849919    2565 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 01 16:58:49.909649 ip-10-0-137-244 kubenswrapper[2565]: W0701 16:58:49.849923    2565 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 01 16:58:49.909649 ip-10-0-137-244 kubenswrapper[2565]: W0701 16:58:49.849927    2565 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 01 16:58:49.909649 ip-10-0-137-244 kubenswrapper[2565]: I0701 16:58:49.849994    2565 flags.go:64] FLAG: --address="0.0.0.0"
Jul 01 16:58:49.909649 ip-10-0-137-244 kubenswrapper[2565]: I0701 16:58:49.850001    2565 flags.go:64] FLAG: --allowed-unsafe-sysctls="[]"
Jul 01 16:58:49.909649 ip-10-0-137-244 kubenswrapper[2565]: I0701 16:58:49.850007    2565 flags.go:64] FLAG: --anonymous-auth="true"
Jul 01 16:58:49.909649 ip-10-0-137-244 kubenswrapper[2565]: I0701 16:58:49.850011    2565 flags.go:64] FLAG: --application-metrics-count-limit="100"
Jul 01 16:58:49.909649 ip-10-0-137-244 kubenswrapper[2565]: I0701 16:58:49.850015    2565 flags.go:64] FLAG: --authentication-token-webhook="false"
Jul 01 16:58:49.909649 ip-10-0-137-244 kubenswrapper[2565]: I0701 16:58:49.850039    2565 flags.go:64] FLAG: --authentication-token-webhook-cache-ttl="2m0s"
Jul 01 16:58:49.909649 ip-10-0-137-244 kubenswrapper[2565]: I0701 16:58:49.850044    2565 flags.go:64] FLAG: --authorization-mode="AlwaysAllow"
Jul 01 16:58:49.909649 ip-10-0-137-244 kubenswrapper[2565]: I0701 16:58:49.850050    2565 flags.go:64] FLAG: --authorization-webhook-cache-authorized-ttl="5m0s"
Jul 01 16:58:49.910946 ip-10-0-137-244 kubenswrapper[2565]: I0701 16:58:49.850053    2565 flags.go:64] FLAG: --authorization-webhook-cache-unauthorized-ttl="30s"
Jul 01 16:58:49.910946 ip-10-0-137-244 kubenswrapper[2565]: I0701 16:58:49.850055    2565 flags.go:64] FLAG: --boot-id-file="/proc/sys/kernel/random/boot_id"
Jul 01 16:58:49.910946 ip-10-0-137-244 kubenswrapper[2565]: I0701 16:58:49.850059    2565 flags.go:64] FLAG: --bootstrap-kubeconfig="/etc/kubernetes/kubeconfig"
Jul 01 16:58:49.910946 ip-10-0-137-244 kubenswrapper[2565]: I0701 16:58:49.850062    2565 flags.go:64] FLAG: --cert-dir="/var/lib/kubelet/pki"
Jul 01 16:58:49.910946 ip-10-0-137-244 kubenswrapper[2565]: I0701 16:58:49.850065    2565 flags.go:64] FLAG: --cgroup-driver="cgroupfs"
Jul 01 16:58:49.910946 ip-10-0-137-244 kubenswrapper[2565]: I0701 16:58:49.850068    2565 flags.go:64] FLAG: --cgroup-root=""
Jul 01 16:58:49.910946 ip-10-0-137-244 kubenswrapper[2565]: I0701 16:58:49.850071    2565 flags.go:64] FLAG: --cgroups-per-qos="true"
Jul 01 16:58:49.910946 ip-10-0-137-244 kubenswrapper[2565]: I0701 16:58:49.850074    2565 flags.go:64] FLAG: --client-ca-file=""
Jul 01 16:58:49.910946 ip-10-0-137-244 kubenswrapper[2565]: I0701 16:58:49.850077    2565 flags.go:64] FLAG: --cloud-provider="external"
Jul 01 16:58:49.910946 ip-10-0-137-244 kubenswrapper[2565]: I0701 16:58:49.850080    2565 flags.go:64] FLAG: --cluster-dns="[]"
Jul 01 16:58:49.910946 ip-10-0-137-244 kubenswrapper[2565]: I0701 16:58:49.850084    2565 flags.go:64] FLAG: --cluster-domain=""
Jul 01 16:58:49.910946 ip-10-0-137-244 kubenswrapper[2565]: I0701 16:58:49.850087    2565 flags.go:64] FLAG: --config="/etc/kubernetes/kubelet.conf"
Jul 01 16:58:49.910946 ip-10-0-137-244 kubenswrapper[2565]: I0701 16:58:49.850090    2565 flags.go:64] FLAG: --config-dir="/etc/openshift/kubelet.conf.d"
Jul 01 16:58:49.910946 ip-10-0-137-244 kubenswrapper[2565]: I0701 16:58:49.850093    2565 flags.go:64] FLAG: --container-hints="/etc/cadvisor/container_hints.json"
Jul 01 16:58:49.910946 ip-10-0-137-244 kubenswrapper[2565]: I0701 16:58:49.850096    2565 flags.go:64] FLAG: --container-log-max-files="5"
Jul 01 16:58:49.910946 ip-10-0-137-244 kubenswrapper[2565]: I0701 16:58:49.850100    2565 flags.go:64] FLAG: --container-log-max-size="10Mi"
Jul 01 16:58:49.910946 ip-10-0-137-244 kubenswrapper[2565]: I0701 16:58:49.850103    2565 flags.go:64] FLAG: --container-runtime-endpoint="/var/run/crio/crio.sock"
Jul 01 16:58:49.910946 ip-10-0-137-244 kubenswrapper[2565]: I0701 16:58:49.850106    2565 flags.go:64] FLAG: --containerd="/run/containerd/containerd.sock"
Jul 01 16:58:49.910946 ip-10-0-137-244 kubenswrapper[2565]: I0701 16:58:49.850109    2565 flags.go:64] FLAG: --containerd-namespace="k8s.io"
Jul 01 16:58:49.910946 ip-10-0-137-244 kubenswrapper[2565]: I0701 16:58:49.850112    2565 flags.go:64] FLAG: --contention-profiling="false"
Jul 01 16:58:49.910946 ip-10-0-137-244 kubenswrapper[2565]: I0701 16:58:49.850115    2565 flags.go:64] FLAG: --cpu-cfs-quota="true"
Jul 01 16:58:49.910946 ip-10-0-137-244 kubenswrapper[2565]: I0701 16:58:49.850118    2565 flags.go:64] FLAG: --cpu-cfs-quota-period="100ms"
Jul 01 16:58:49.910946 ip-10-0-137-244 kubenswrapper[2565]: I0701 16:58:49.850121    2565 flags.go:64] FLAG: --cpu-manager-policy="none"
Jul 01 16:58:49.910946 ip-10-0-137-244 kubenswrapper[2565]: I0701 16:58:49.850125    2565 flags.go:64] FLAG: --cpu-manager-policy-options=""
Jul 01 16:58:49.912065 ip-10-0-137-244 kubenswrapper[2565]: I0701 16:58:49.850128    2565 flags.go:64] FLAG: --cpu-manager-reconcile-period="10s"
Jul 01 16:58:49.912065 ip-10-0-137-244 kubenswrapper[2565]: I0701 16:58:49.850131    2565 flags.go:64] FLAG: --enable-controller-attach-detach="true"
Jul 01 16:58:49.912065 ip-10-0-137-244 kubenswrapper[2565]: I0701 16:58:49.850134    2565 flags.go:64] FLAG: --enable-debugging-handlers="true"
Jul 01 16:58:49.912065 ip-10-0-137-244 kubenswrapper[2565]: I0701 16:58:49.850137    2565 flags.go:64] FLAG: --enable-load-reader="false"
Jul 01 16:58:49.912065 ip-10-0-137-244 kubenswrapper[2565]: I0701 16:58:49.850140    2565 flags.go:64] FLAG: --enable-server="true"
Jul 01 16:58:49.912065 ip-10-0-137-244 kubenswrapper[2565]: I0701 16:58:49.850144    2565 flags.go:64] FLAG: --enforce-node-allocatable="[pods]"
Jul 01 16:58:49.912065 ip-10-0-137-244 kubenswrapper[2565]: I0701 16:58:49.850148    2565 flags.go:64] FLAG: --event-burst="100"
Jul 01 16:58:49.912065 ip-10-0-137-244 kubenswrapper[2565]: I0701 16:58:49.850151    2565 flags.go:64] FLAG: --event-qps="50"
Jul 01 16:58:49.912065 ip-10-0-137-244 kubenswrapper[2565]: I0701 16:58:49.850154    2565 flags.go:64] FLAG: --event-storage-age-limit="default=0"
Jul 01 16:58:49.912065 ip-10-0-137-244 kubenswrapper[2565]: I0701 16:58:49.850157    2565 flags.go:64] FLAG: --event-storage-event-limit="default=0"
Jul 01 16:58:49.912065 ip-10-0-137-244 kubenswrapper[2565]: I0701 16:58:49.850160    2565 flags.go:64] FLAG: --eviction-hard=""
Jul 01 16:58:49.912065 ip-10-0-137-244 kubenswrapper[2565]: I0701 16:58:49.850164    2565 flags.go:64] FLAG: --eviction-max-pod-grace-period="0"
Jul 01 16:58:49.912065 ip-10-0-137-244 kubenswrapper[2565]: I0701 16:58:49.850166    2565 flags.go:64] FLAG: --eviction-minimum-reclaim=""
Jul 01 16:58:49.912065 ip-10-0-137-244 kubenswrapper[2565]: I0701 16:58:49.850169    2565 flags.go:64] FLAG: --eviction-pressure-transition-period="5m0s"
Jul 01 16:58:49.912065 ip-10-0-137-244 kubenswrapper[2565]: I0701 16:58:49.850172    2565 flags.go:64] FLAG: --eviction-soft=""
Jul 01 16:58:49.912065 ip-10-0-137-244 kubenswrapper[2565]: I0701 16:58:49.850175    2565 flags.go:64] FLAG: --eviction-soft-grace-period=""
Jul 01 16:58:49.912065 ip-10-0-137-244 kubenswrapper[2565]: I0701 16:58:49.850178    2565 flags.go:64] FLAG: --exit-on-lock-contention="false"
Jul 01 16:58:49.912065 ip-10-0-137-244 kubenswrapper[2565]: I0701 16:58:49.850181    2565 flags.go:64] FLAG: --experimental-allocatable-ignore-eviction="false"
Jul 01 16:58:49.912065 ip-10-0-137-244 kubenswrapper[2565]: I0701 16:58:49.850184    2565 flags.go:64] FLAG: --experimental-mounter-path=""
Jul 01 16:58:49.912065 ip-10-0-137-244 kubenswrapper[2565]: I0701 16:58:49.850187    2565 flags.go:64] FLAG: --fail-cgroupv1="false"
Jul 01 16:58:49.912065 ip-10-0-137-244 kubenswrapper[2565]: I0701 16:58:49.850189    2565 flags.go:64] FLAG: --fail-swap-on="true"
Jul 01 16:58:49.912065 ip-10-0-137-244 kubenswrapper[2565]: I0701 16:58:49.850192    2565 flags.go:64] FLAG: --feature-gates=""
Jul 01 16:58:49.912065 ip-10-0-137-244 kubenswrapper[2565]: I0701 16:58:49.850196    2565 flags.go:64] FLAG: --file-check-frequency="20s"
Jul 01 16:58:49.912065 ip-10-0-137-244 kubenswrapper[2565]: I0701 16:58:49.850199    2565 flags.go:64] FLAG: --global-housekeeping-interval="1m0s"
Jul 01 16:58:49.912065 ip-10-0-137-244 kubenswrapper[2565]: I0701 16:58:49.850201    2565 flags.go:64] FLAG: --hairpin-mode="promiscuous-bridge"
Jul 01 16:58:49.913592 ip-10-0-137-244 kubenswrapper[2565]: I0701 16:58:49.850204    2565 flags.go:64] FLAG: --healthz-bind-address="127.0.0.1"
Jul 01 16:58:49.913592 ip-10-0-137-244 kubenswrapper[2565]: I0701 16:58:49.850207    2565 flags.go:64] FLAG: --healthz-port="10248"
Jul 01 16:58:49.913592 ip-10-0-137-244 kubenswrapper[2565]: I0701 16:58:49.850210    2565 flags.go:64] FLAG: --help="false"
Jul 01 16:58:49.913592 ip-10-0-137-244 kubenswrapper[2565]: I0701 16:58:49.850213    2565 flags.go:64] FLAG: --hostname-override="ip-10-0-137-244.ec2.internal"
Jul 01 16:58:49.913592 ip-10-0-137-244 kubenswrapper[2565]: I0701 16:58:49.850216    2565 flags.go:64] FLAG: --housekeeping-interval="10s"
Jul 01 16:58:49.913592 ip-10-0-137-244 kubenswrapper[2565]: I0701 16:58:49.850219    2565 flags.go:64] FLAG: --http-check-frequency="20s"
Jul 01 16:58:49.913592 ip-10-0-137-244 kubenswrapper[2565]: I0701 16:58:49.850224    2565 flags.go:64] FLAG: --image-credential-provider-bin-dir="/usr/libexec/kubelet-image-credential-provider-plugins"
Jul 01 16:58:49.913592 ip-10-0-137-244 kubenswrapper[2565]: I0701 16:58:49.850227    2565 flags.go:64] FLAG: --image-credential-provider-config="/etc/kubernetes/credential-providers/ecr-credential-provider.yaml"
Jul 01 16:58:49.913592 ip-10-0-137-244 kubenswrapper[2565]: I0701 16:58:49.850231    2565 flags.go:64] FLAG: --image-gc-high-threshold="85"
Jul 01 16:58:49.913592 ip-10-0-137-244 kubenswrapper[2565]: I0701 16:58:49.850233    2565 flags.go:64] FLAG: --image-gc-low-threshold="80"
Jul 01 16:58:49.913592 ip-10-0-137-244 kubenswrapper[2565]: I0701 16:58:49.850236    2565 flags.go:64] FLAG: --image-service-endpoint=""
Jul 01 16:58:49.913592 ip-10-0-137-244 kubenswrapper[2565]: I0701 16:58:49.850239    2565 flags.go:64] FLAG: --kernel-memcg-notification="false"
Jul 01 16:58:49.913592 ip-10-0-137-244 kubenswrapper[2565]: I0701 16:58:49.850243    2565 flags.go:64] FLAG: --kube-api-burst="100"
Jul 01 16:58:49.913592 ip-10-0-137-244 kubenswrapper[2565]: I0701 16:58:49.850246    2565 flags.go:64] FLAG: --kube-api-content-type="application/vnd.kubernetes.protobuf"
Jul 01 16:58:49.913592 ip-10-0-137-244 kubenswrapper[2565]: I0701 16:58:49.850249    2565 flags.go:64] FLAG: --kube-api-qps="50"
Jul 01 16:58:49.913592 ip-10-0-137-244 kubenswrapper[2565]: I0701 16:58:49.850252    2565 flags.go:64] FLAG: --kube-reserved=""
Jul 01 16:58:49.913592 ip-10-0-137-244 kubenswrapper[2565]: I0701 16:58:49.850255    2565 flags.go:64] FLAG: --kube-reserved-cgroup=""
Jul 01 16:58:49.913592 ip-10-0-137-244 kubenswrapper[2565]: I0701 16:58:49.850258    2565 flags.go:64] FLAG: --kubeconfig="/var/lib/kubelet/kubeconfig"
Jul 01 16:58:49.913592 ip-10-0-137-244 kubenswrapper[2565]: I0701 16:58:49.850261    2565 flags.go:64] FLAG: --kubelet-cgroups=""
Jul 01 16:58:49.913592 ip-10-0-137-244 kubenswrapper[2565]: I0701 16:58:49.850263    2565 flags.go:64] FLAG: --local-storage-capacity-isolation="true"
Jul 01 16:58:49.913592 ip-10-0-137-244 kubenswrapper[2565]: I0701 16:58:49.850266    2565 flags.go:64] FLAG: --lock-file=""
Jul 01 16:58:49.913592 ip-10-0-137-244 kubenswrapper[2565]: I0701 16:58:49.850269    2565 flags.go:64] FLAG: --log-cadvisor-usage="false"
Jul 01 16:58:49.913592 ip-10-0-137-244 kubenswrapper[2565]: I0701 16:58:49.850272    2565 flags.go:64] FLAG: --log-flush-frequency="5s"
Jul 01 16:58:49.913592 ip-10-0-137-244 kubenswrapper[2565]: I0701 16:58:49.850275    2565 flags.go:64] FLAG: --log-json-info-buffer-size="0"
Jul 01 16:58:49.914688 ip-10-0-137-244 kubenswrapper[2565]: I0701 16:58:49.850280    2565 flags.go:64] FLAG: --log-json-split-stream="false"
Jul 01 16:58:49.914688 ip-10-0-137-244 kubenswrapper[2565]: I0701 16:58:49.850284    2565 flags.go:64] FLAG: --log-text-info-buffer-size="0"
Jul 01 16:58:49.914688 ip-10-0-137-244 kubenswrapper[2565]: I0701 16:58:49.850286    2565 flags.go:64] FLAG: --log-text-split-stream="false"
Jul 01 16:58:49.914688 ip-10-0-137-244 kubenswrapper[2565]: I0701 16:58:49.850289    2565 flags.go:64] FLAG: --logging-format="text"
Jul 01 16:58:49.914688 ip-10-0-137-244 kubenswrapper[2565]: I0701 16:58:49.850292    2565 flags.go:64] FLAG: --machine-id-file="/etc/machine-id,/var/lib/dbus/machine-id"
Jul 01 16:58:49.914688 ip-10-0-137-244 kubenswrapper[2565]: I0701 16:58:49.850295    2565 flags.go:64] FLAG: --make-iptables-util-chains="true"
Jul 01 16:58:49.914688 ip-10-0-137-244 kubenswrapper[2565]: I0701 16:58:49.850298    2565 flags.go:64] FLAG: --manifest-url=""
Jul 01 16:58:49.914688 ip-10-0-137-244 kubenswrapper[2565]: I0701 16:58:49.850301    2565 flags.go:64] FLAG: --manifest-url-header=""
Jul 01 16:58:49.914688 ip-10-0-137-244 kubenswrapper[2565]: I0701 16:58:49.850309    2565 flags.go:64] FLAG: --max-housekeeping-interval="15s"
Jul 01 16:58:49.914688 ip-10-0-137-244 kubenswrapper[2565]: I0701 16:58:49.850312    2565 flags.go:64] FLAG: --max-open-files="1000000"
Jul 01 16:58:49.914688 ip-10-0-137-244 kubenswrapper[2565]: I0701 16:58:49.850316    2565 flags.go:64] FLAG: --max-pods="110"
Jul 01 16:58:49.914688 ip-10-0-137-244 kubenswrapper[2565]: I0701 16:58:49.850318    2565 flags.go:64] FLAG: --maximum-dead-containers="-1"
Jul 01 16:58:49.914688 ip-10-0-137-244 kubenswrapper[2565]: I0701 16:58:49.850321    2565 flags.go:64] FLAG: --maximum-dead-containers-per-container="1"
Jul 01 16:58:49.914688 ip-10-0-137-244 kubenswrapper[2565]: I0701 16:58:49.850324    2565 flags.go:64] FLAG: --memory-manager-policy="None"
Jul 01 16:58:49.914688 ip-10-0-137-244 kubenswrapper[2565]: I0701 16:58:49.850328    2565 flags.go:64] FLAG: --minimum-container-ttl-duration="6m0s"
Jul 01 16:58:49.914688 ip-10-0-137-244 kubenswrapper[2565]: I0701 16:58:49.850331    2565 flags.go:64] FLAG: --minimum-image-ttl-duration="2m0s"
Jul 01 16:58:49.914688 ip-10-0-137-244 kubenswrapper[2565]: I0701 16:58:49.850334    2565 flags.go:64] FLAG: --node-ip="0.0.0.0"
Jul 01 16:58:49.914688 ip-10-0-137-244 kubenswrapper[2565]: I0701 16:58:49.850337    2565 flags.go:64] FLAG: --node-labels="node-role.kubernetes.io/worker=,node.openshift.io/os_id=rhel"
Jul 01 16:58:49.914688 ip-10-0-137-244 kubenswrapper[2565]: I0701 16:58:49.850344    2565 flags.go:64] FLAG: --node-status-max-images="50"
Jul 01 16:58:49.914688 ip-10-0-137-244 kubenswrapper[2565]: I0701 16:58:49.850347    2565 flags.go:64] FLAG: --node-status-update-frequency="10s"
Jul 01 16:58:49.914688 ip-10-0-137-244 kubenswrapper[2565]: I0701 16:58:49.850352    2565 flags.go:64] FLAG: --oom-score-adj="-999"
Jul 01 16:58:49.914688 ip-10-0-137-244 kubenswrapper[2565]: I0701 16:58:49.850355    2565 flags.go:64] FLAG: --pod-cidr=""
Jul 01 16:58:49.914688 ip-10-0-137-244 kubenswrapper[2565]: I0701 16:58:49.850358    2565 flags.go:64] FLAG: --pod-infra-container-image="quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:8a2632cd31d78133be7b5fb6999469b2acf9b98d077c7f6ae5d0d94083d76b04"
Jul 01 16:58:49.916682 ip-10-0-137-244 kubenswrapper[2565]: I0701 16:58:49.850363    2565 flags.go:64] FLAG: --pod-manifest-path=""
Jul 01 16:58:49.916682 ip-10-0-137-244 kubenswrapper[2565]: I0701 16:58:49.850366    2565 flags.go:64] FLAG: --pod-max-pids="-1"
Jul 01 16:58:49.916682 ip-10-0-137-244 kubenswrapper[2565]: I0701 16:58:49.850369    2565 flags.go:64] FLAG: --pods-per-core="0"
Jul 01 16:58:49.916682 ip-10-0-137-244 kubenswrapper[2565]: I0701 16:58:49.850372    2565 flags.go:64] FLAG: --port="10250"
Jul 01 16:58:49.916682 ip-10-0-137-244 kubenswrapper[2565]: I0701 16:58:49.850375    2565 flags.go:64] FLAG: --protect-kernel-defaults="false"
Jul 01 16:58:49.916682 ip-10-0-137-244 kubenswrapper[2565]: I0701 16:58:49.850377    2565 flags.go:64] FLAG: --provider-id="aws:///us-east-1a/i-0a24a8fde90751bf1"
Jul 01 16:58:49.916682 ip-10-0-137-244 kubenswrapper[2565]: I0701 16:58:49.850381    2565 flags.go:64] FLAG: --qos-reserved=""
Jul 01 16:58:49.916682 ip-10-0-137-244 kubenswrapper[2565]: I0701 16:58:49.850383    2565 flags.go:64] FLAG: --read-only-port="10255"
Jul 01 16:58:49.916682 ip-10-0-137-244 kubenswrapper[2565]: I0701 16:58:49.850386    2565 flags.go:64] FLAG: --register-node="true"
Jul 01 16:58:49.916682 ip-10-0-137-244 kubenswrapper[2565]: I0701 16:58:49.850389    2565 flags.go:64] FLAG: --register-with-taints=""
Jul 01 16:58:49.916682 ip-10-0-137-244 kubenswrapper[2565]: I0701 16:58:49.850393    2565 flags.go:64] FLAG: --registry-burst="10"
Jul 01 16:58:49.916682 ip-10-0-137-244 kubenswrapper[2565]: I0701 16:58:49.850396    2565 flags.go:64] FLAG: --registry-qps="5"
Jul 01 16:58:49.916682 ip-10-0-137-244 kubenswrapper[2565]: I0701 16:58:49.850399    2565 flags.go:64] FLAG: --reserved-cpus=""
Jul 01 16:58:49.916682 ip-10-0-137-244 kubenswrapper[2565]: I0701 16:58:49.850401    2565 flags.go:64] FLAG: --reserved-memory=""
Jul 01 16:58:49.916682 ip-10-0-137-244 kubenswrapper[2565]: I0701 16:58:49.850405    2565 flags.go:64] FLAG: --resolv-conf="/etc/resolv.conf"
Jul 01 16:58:49.916682 ip-10-0-137-244 kubenswrapper[2565]: I0701 16:58:49.850408    2565 flags.go:64] FLAG: --root-dir="/var/lib/kubelet"
Jul 01 16:58:49.916682 ip-10-0-137-244 kubenswrapper[2565]: I0701 16:58:49.850411    2565 flags.go:64] FLAG: --rotate-certificates="false"
Jul 01 16:58:49.916682 ip-10-0-137-244 kubenswrapper[2565]: I0701 16:58:49.850413    2565 flags.go:64] FLAG: --rotate-server-certificates="false"
Jul 01 16:58:49.916682 ip-10-0-137-244 kubenswrapper[2565]: I0701 16:58:49.850416    2565 flags.go:64] FLAG: --runonce="false"
Jul 01 16:58:49.916682 ip-10-0-137-244 kubenswrapper[2565]: I0701 16:58:49.850419    2565 flags.go:64] FLAG: --runtime-cgroups="/system.slice/crio.service"
Jul 01 16:58:49.916682 ip-10-0-137-244 kubenswrapper[2565]: I0701 16:58:49.850422    2565 flags.go:64] FLAG: --runtime-request-timeout="2m0s"
Jul 01 16:58:49.916682 ip-10-0-137-244 kubenswrapper[2565]: I0701 16:58:49.850425    2565 flags.go:64] FLAG: --seccomp-default="false"
Jul 01 16:58:49.916682 ip-10-0-137-244 kubenswrapper[2565]: I0701 16:58:49.850428    2565 flags.go:64] FLAG: --serialize-image-pulls="true"
Jul 01 16:58:49.916682 ip-10-0-137-244 kubenswrapper[2565]: I0701 16:58:49.850432    2565 flags.go:64] FLAG: --storage-driver-buffer-duration="1m0s"
Jul 01 16:58:49.916682 ip-10-0-137-244 kubenswrapper[2565]: I0701 16:58:49.850435    2565 flags.go:64] FLAG: --storage-driver-db="cadvisor"
Jul 01 16:58:49.916682 ip-10-0-137-244 kubenswrapper[2565]: I0701 16:58:49.850438    2565 flags.go:64] FLAG: --storage-driver-host="localhost:8086"
Jul 01 16:58:49.917830 ip-10-0-137-244 kubenswrapper[2565]: I0701 16:58:49.850441    2565 flags.go:64] FLAG: --storage-driver-password="root"
Jul 01 16:58:49.917830 ip-10-0-137-244 kubenswrapper[2565]: I0701 16:58:49.850444    2565 flags.go:64] FLAG: --storage-driver-secure="false"
Jul 01 16:58:49.917830 ip-10-0-137-244 kubenswrapper[2565]: I0701 16:58:49.850447    2565 flags.go:64] FLAG: --storage-driver-table="stats"
Jul 01 16:58:49.917830 ip-10-0-137-244 kubenswrapper[2565]: I0701 16:58:49.850450    2565 flags.go:64] FLAG: --storage-driver-user="root"
Jul 01 16:58:49.917830 ip-10-0-137-244 kubenswrapper[2565]: I0701 16:58:49.850454    2565 flags.go:64] FLAG: --streaming-connection-idle-timeout="4h0m0s"
Jul 01 16:58:49.917830 ip-10-0-137-244 kubenswrapper[2565]: I0701 16:58:49.850457    2565 flags.go:64] FLAG: --sync-frequency="1m0s"
Jul 01 16:58:49.917830 ip-10-0-137-244 kubenswrapper[2565]: I0701 16:58:49.850460    2565 flags.go:64] FLAG: --system-cgroups=""
Jul 01 16:58:49.917830 ip-10-0-137-244 kubenswrapper[2565]: I0701 16:58:49.850462    2565 flags.go:64] FLAG: --system-reserved="cpu=500m,ephemeral-storage=1Gi,memory=1Gi"
Jul 01 16:58:49.917830 ip-10-0-137-244 kubenswrapper[2565]: I0701 16:58:49.850468    2565 flags.go:64] FLAG: --system-reserved-cgroup=""
Jul 01 16:58:49.917830 ip-10-0-137-244 kubenswrapper[2565]: I0701 16:58:49.850470    2565 flags.go:64] FLAG: --tls-cert-file=""
Jul 01 16:58:49.917830 ip-10-0-137-244 kubenswrapper[2565]: I0701 16:58:49.850473    2565 flags.go:64] FLAG: --tls-cipher-suites="[]"
Jul 01 16:58:49.917830 ip-10-0-137-244 kubenswrapper[2565]: I0701 16:58:49.850477    2565 flags.go:64] FLAG: --tls-min-version=""
Jul 01 16:58:49.917830 ip-10-0-137-244 kubenswrapper[2565]: I0701 16:58:49.850480    2565 flags.go:64] FLAG: --tls-private-key-file=""
Jul 01 16:58:49.917830 ip-10-0-137-244 kubenswrapper[2565]: I0701 16:58:49.850483    2565 flags.go:64] FLAG: --topology-manager-policy="none"
Jul 01 16:58:49.917830 ip-10-0-137-244 kubenswrapper[2565]: I0701 16:58:49.850486    2565 flags.go:64] FLAG: --topology-manager-policy-options=""
Jul 01 16:58:49.917830 ip-10-0-137-244 kubenswrapper[2565]: I0701 16:58:49.850488    2565 flags.go:64] FLAG: --topology-manager-scope="container"
Jul 01 16:58:49.917830 ip-10-0-137-244 kubenswrapper[2565]: I0701 16:58:49.850491    2565 flags.go:64] FLAG: --v="2"
Jul 01 16:58:49.917830 ip-10-0-137-244 kubenswrapper[2565]: I0701 16:58:49.850495    2565 flags.go:64] FLAG: --version="false"
Jul 01 16:58:49.917830 ip-10-0-137-244 kubenswrapper[2565]: I0701 16:58:49.850499    2565 flags.go:64] FLAG: --vmodule=""
Jul 01 16:58:49.917830 ip-10-0-137-244 kubenswrapper[2565]: I0701 16:58:49.850503    2565 flags.go:64] FLAG: --volume-plugin-dir="/etc/kubernetes/kubelet-plugins/volume/exec"
Jul 01 16:58:49.917830 ip-10-0-137-244 kubenswrapper[2565]: I0701 16:58:49.850506    2565 flags.go:64] FLAG: --volume-stats-agg-period="1m0s"
Jul 01 16:58:49.917830 ip-10-0-137-244 kubenswrapper[2565]: W0701 16:58:49.850621    2565 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 01 16:58:49.917830 ip-10-0-137-244 kubenswrapper[2565]: W0701 16:58:49.850624    2565 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 01 16:58:49.917830 ip-10-0-137-244 kubenswrapper[2565]: W0701 16:58:49.850628    2565 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 01 16:58:49.918934 ip-10-0-137-244 kubenswrapper[2565]: W0701 16:58:49.850631    2565 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 01 16:58:49.918934 ip-10-0-137-244 kubenswrapper[2565]: W0701 16:58:49.850634    2565 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 01 16:58:49.918934 ip-10-0-137-244 kubenswrapper[2565]: W0701 16:58:49.850637    2565 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 01 16:58:49.918934 ip-10-0-137-244 kubenswrapper[2565]: W0701 16:58:49.850640    2565 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 01 16:58:49.918934 ip-10-0-137-244 kubenswrapper[2565]: W0701 16:58:49.850643    2565 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 01 16:58:49.918934 ip-10-0-137-244 kubenswrapper[2565]: W0701 16:58:49.850649    2565 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 01 16:58:49.918934 ip-10-0-137-244 kubenswrapper[2565]: W0701 16:58:49.850652    2565 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 01 16:58:49.918934 ip-10-0-137-244 kubenswrapper[2565]: W0701 16:58:49.850656    2565 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 01 16:58:49.918934 ip-10-0-137-244 kubenswrapper[2565]: W0701 16:58:49.850659    2565 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 01 16:58:49.918934 ip-10-0-137-244 kubenswrapper[2565]: W0701 16:58:49.850662    2565 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 01 16:58:49.918934 ip-10-0-137-244 kubenswrapper[2565]: W0701 16:58:49.850664    2565 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 01 16:58:49.918934 ip-10-0-137-244 kubenswrapper[2565]: W0701 16:58:49.850667    2565 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 01 16:58:49.918934 ip-10-0-137-244 kubenswrapper[2565]: W0701 16:58:49.850672    2565 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 01 16:58:49.918934 ip-10-0-137-244 kubenswrapper[2565]: W0701 16:58:49.850674    2565 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 01 16:58:49.918934 ip-10-0-137-244 kubenswrapper[2565]: W0701 16:58:49.850677    2565 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 01 16:58:49.918934 ip-10-0-137-244 kubenswrapper[2565]: W0701 16:58:49.850680    2565 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 01 16:58:49.918934 ip-10-0-137-244 kubenswrapper[2565]: W0701 16:58:49.850683    2565 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 01 16:58:49.918934 ip-10-0-137-244 kubenswrapper[2565]: W0701 16:58:49.850685    2565 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 01 16:58:49.918934 ip-10-0-137-244 kubenswrapper[2565]: W0701 16:58:49.850688    2565 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 01 16:58:49.918934 ip-10-0-137-244 kubenswrapper[2565]: W0701 16:58:49.850690    2565 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 01 16:58:49.919843 ip-10-0-137-244 kubenswrapper[2565]: W0701 16:58:49.850693    2565 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 01 16:58:49.919843 ip-10-0-137-244 kubenswrapper[2565]: W0701 16:58:49.850696    2565 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 01 16:58:49.919843 ip-10-0-137-244 kubenswrapper[2565]: W0701 16:58:49.850699    2565 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 01 16:58:49.919843 ip-10-0-137-244 kubenswrapper[2565]: W0701 16:58:49.850701    2565 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 01 16:58:49.919843 ip-10-0-137-244 kubenswrapper[2565]: W0701 16:58:49.850704    2565 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 01 16:58:49.919843 ip-10-0-137-244 kubenswrapper[2565]: W0701 16:58:49.850706    2565 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 01 16:58:49.919843 ip-10-0-137-244 kubenswrapper[2565]: W0701 16:58:49.850710    2565 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 01 16:58:49.919843 ip-10-0-137-244 kubenswrapper[2565]: W0701 16:58:49.850714    2565 feature_gate.go:328] unrecognized feature gate: Example2
Jul 01 16:58:49.919843 ip-10-0-137-244 kubenswrapper[2565]: W0701 16:58:49.850717    2565 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 01 16:58:49.919843 ip-10-0-137-244 kubenswrapper[2565]: W0701 16:58:49.850719    2565 feature_gate.go:328] unrecognized feature gate: Example
Jul 01 16:58:49.919843 ip-10-0-137-244 kubenswrapper[2565]: W0701 16:58:49.850722    2565 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 01 16:58:49.919843 ip-10-0-137-244 kubenswrapper[2565]: W0701 16:58:49.850725    2565 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 01 16:58:49.919843 ip-10-0-137-244 kubenswrapper[2565]: W0701 16:58:49.850728    2565 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 01 16:58:49.919843 ip-10-0-137-244 kubenswrapper[2565]: W0701 16:58:49.850730    2565 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 01 16:58:49.919843 ip-10-0-137-244 kubenswrapper[2565]: W0701 16:58:49.850733    2565 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 01 16:58:49.919843 ip-10-0-137-244 kubenswrapper[2565]: W0701 16:58:49.850736    2565 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 01 16:58:49.919843 ip-10-0-137-244 kubenswrapper[2565]: W0701 16:58:49.850738    2565 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 01 16:58:49.919843 ip-10-0-137-244 kubenswrapper[2565]: W0701 16:58:49.850742    2565 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 01 16:58:49.919843 ip-10-0-137-244 kubenswrapper[2565]: W0701 16:58:49.850744    2565 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 01 16:58:49.921421 ip-10-0-137-244 kubenswrapper[2565]: W0701 16:58:49.850748    2565 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 01 16:58:49.921421 ip-10-0-137-244 kubenswrapper[2565]: W0701 16:58:49.850751    2565 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 01 16:58:49.921421 ip-10-0-137-244 kubenswrapper[2565]: W0701 16:58:49.850753    2565 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 01 16:58:49.921421 ip-10-0-137-244 kubenswrapper[2565]: W0701 16:58:49.850756    2565 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 01 16:58:49.921421 ip-10-0-137-244 kubenswrapper[2565]: W0701 16:58:49.850758    2565 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 01 16:58:49.921421 ip-10-0-137-244 kubenswrapper[2565]: W0701 16:58:49.850761    2565 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 01 16:58:49.921421 ip-10-0-137-244 kubenswrapper[2565]: W0701 16:58:49.850764    2565 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 01 16:58:49.921421 ip-10-0-137-244 kubenswrapper[2565]: W0701 16:58:49.850767    2565 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 01 16:58:49.921421 ip-10-0-137-244 kubenswrapper[2565]: W0701 16:58:49.850769    2565 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 01 16:58:49.921421 ip-10-0-137-244 kubenswrapper[2565]: W0701 16:58:49.850772    2565 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 01 16:58:49.921421 ip-10-0-137-244 kubenswrapper[2565]: W0701 16:58:49.850774    2565 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 01 16:58:49.921421 ip-10-0-137-244 kubenswrapper[2565]: W0701 16:58:49.850777    2565 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 01 16:58:49.921421 ip-10-0-137-244 kubenswrapper[2565]: W0701 16:58:49.850780    2565 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 01 16:58:49.921421 ip-10-0-137-244 kubenswrapper[2565]: W0701 16:58:49.850782    2565 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 01 16:58:49.921421 ip-10-0-137-244 kubenswrapper[2565]: W0701 16:58:49.850785    2565 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 01 16:58:49.921421 ip-10-0-137-244 kubenswrapper[2565]: W0701 16:58:49.850787    2565 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 01 16:58:49.921421 ip-10-0-137-244 kubenswrapper[2565]: W0701 16:58:49.850790    2565 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 01 16:58:49.921421 ip-10-0-137-244 kubenswrapper[2565]: W0701 16:58:49.850792    2565 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 01 16:58:49.921421 ip-10-0-137-244 kubenswrapper[2565]: W0701 16:58:49.850795    2565 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 01 16:58:49.922325 ip-10-0-137-244 kubenswrapper[2565]: W0701 16:58:49.850797    2565 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 01 16:58:49.922325 ip-10-0-137-244 kubenswrapper[2565]: W0701 16:58:49.850800    2565 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 01 16:58:49.922325 ip-10-0-137-244 kubenswrapper[2565]: W0701 16:58:49.850803    2565 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 01 16:58:49.922325 ip-10-0-137-244 kubenswrapper[2565]: W0701 16:58:49.850806    2565 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 01 16:58:49.922325 ip-10-0-137-244 kubenswrapper[2565]: W0701 16:58:49.850808    2565 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 01 16:58:49.922325 ip-10-0-137-244 kubenswrapper[2565]: W0701 16:58:49.850811    2565 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 01 16:58:49.922325 ip-10-0-137-244 kubenswrapper[2565]: W0701 16:58:49.850814    2565 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 01 16:58:49.922325 ip-10-0-137-244 kubenswrapper[2565]: W0701 16:58:49.850816    2565 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 01 16:58:49.922325 ip-10-0-137-244 kubenswrapper[2565]: W0701 16:58:49.850818    2565 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 01 16:58:49.922325 ip-10-0-137-244 kubenswrapper[2565]: W0701 16:58:49.850821    2565 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 01 16:58:49.922325 ip-10-0-137-244 kubenswrapper[2565]: W0701 16:58:49.850823    2565 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 01 16:58:49.922325 ip-10-0-137-244 kubenswrapper[2565]: W0701 16:58:49.850828    2565 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 01 16:58:49.922325 ip-10-0-137-244 kubenswrapper[2565]: W0701 16:58:49.850830    2565 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 01 16:58:49.922325 ip-10-0-137-244 kubenswrapper[2565]: W0701 16:58:49.850833    2565 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 01 16:58:49.922325 ip-10-0-137-244 kubenswrapper[2565]: W0701 16:58:49.850835    2565 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 01 16:58:49.922325 ip-10-0-137-244 kubenswrapper[2565]: W0701 16:58:49.850838    2565 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 01 16:58:49.922325 ip-10-0-137-244 kubenswrapper[2565]: W0701 16:58:49.850841    2565 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 01 16:58:49.922325 ip-10-0-137-244 kubenswrapper[2565]: W0701 16:58:49.850843    2565 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 01 16:58:49.922325 ip-10-0-137-244 kubenswrapper[2565]: W0701 16:58:49.850847    2565 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 01 16:58:49.923687 ip-10-0-137-244 kubenswrapper[2565]: W0701 16:58:49.850850    2565 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 01 16:58:49.923687 ip-10-0-137-244 kubenswrapper[2565]: W0701 16:58:49.850852    2565 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 01 16:58:49.923687 ip-10-0-137-244 kubenswrapper[2565]: W0701 16:58:49.850855    2565 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 01 16:58:49.923687 ip-10-0-137-244 kubenswrapper[2565]: W0701 16:58:49.850857    2565 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 01 16:58:49.923687 ip-10-0-137-244 kubenswrapper[2565]: W0701 16:58:49.850860    2565 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 01 16:58:49.923687 ip-10-0-137-244 kubenswrapper[2565]: W0701 16:58:49.850862    2565 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 01 16:58:49.923687 ip-10-0-137-244 kubenswrapper[2565]: W0701 16:58:49.850865    2565 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 01 16:58:49.923687 ip-10-0-137-244 kubenswrapper[2565]: W0701 16:58:49.850867    2565 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 01 16:58:49.923687 ip-10-0-137-244 kubenswrapper[2565]: W0701 16:58:49.850870    2565 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 01 16:58:49.923687 ip-10-0-137-244 kubenswrapper[2565]: W0701 16:58:49.850872    2565 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 01 16:58:49.923687 ip-10-0-137-244 kubenswrapper[2565]: W0701 16:58:49.850875    2565 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 01 16:58:49.923687 ip-10-0-137-244 kubenswrapper[2565]: W0701 16:58:49.850878    2565 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 01 16:58:49.923687 ip-10-0-137-244 kubenswrapper[2565]: W0701 16:58:49.850880    2565 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 01 16:58:49.923687 ip-10-0-137-244 kubenswrapper[2565]: W0701 16:58:49.850883    2565 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 01 16:58:49.923687 ip-10-0-137-244 kubenswrapper[2565]: W0701 16:58:49.850885    2565 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 01 16:58:49.923687 ip-10-0-137-244 kubenswrapper[2565]: W0701 16:58:49.850888    2565 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 01 16:58:49.923687 ip-10-0-137-244 kubenswrapper[2565]: W0701 16:58:49.850891    2565 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 01 16:58:49.923687 ip-10-0-137-244 kubenswrapper[2565]: W0701 16:58:49.850893    2565 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 01 16:58:49.923687 ip-10-0-137-244 kubenswrapper[2565]: W0701 16:58:49.850896    2565 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 01 16:58:49.924568 ip-10-0-137-244 kubenswrapper[2565]: I0701 16:58:49.851522    2565 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]}
Jul 01 16:58:49.924568 ip-10-0-137-244 kubenswrapper[2565]: I0701 16:58:49.863887    2565 server.go:529] "Kubelet version" kubeletVersion="v1.34.8"
Jul 01 16:58:49.924568 ip-10-0-137-244 kubenswrapper[2565]: I0701 16:58:49.863903    2565 server.go:531] "Golang settings" GOGC="" GOMAXPROCS="" GOTRACEBACK=""
Jul 01 16:58:49.924568 ip-10-0-137-244 kubenswrapper[2565]: I0701 16:58:49.863916    2565 watchdog_linux.go:95] "Systemd watchdog is not enabled"
Jul 01 16:58:49.924568 ip-10-0-137-244 kubenswrapper[2565]: I0701 16:58:49.863922    2565 watchdog_linux.go:137] "Systemd watchdog is not enabled or the interval is invalid, so health checking will not be started."
Jul 01 16:58:49.924568 ip-10-0-137-244 kubenswrapper[2565]: W0701 16:58:49.863986    2565 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 01 16:58:49.924568 ip-10-0-137-244 kubenswrapper[2565]: W0701 16:58:49.863990    2565 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 01 16:58:49.924568 ip-10-0-137-244 kubenswrapper[2565]: W0701 16:58:49.863994    2565 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 01 16:58:49.924568 ip-10-0-137-244 kubenswrapper[2565]: W0701 16:58:49.863997    2565 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 01 16:58:49.924568 ip-10-0-137-244 kubenswrapper[2565]: W0701 16:58:49.864000    2565 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 01 16:58:49.924568 ip-10-0-137-244 kubenswrapper[2565]: W0701 16:58:49.864003    2565 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 01 16:58:49.924568 ip-10-0-137-244 kubenswrapper[2565]: W0701 16:58:49.864006    2565 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 01 16:58:49.924568 ip-10-0-137-244 kubenswrapper[2565]: W0701 16:58:49.864009    2565 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 01 16:58:49.924568 ip-10-0-137-244 kubenswrapper[2565]: W0701 16:58:49.864012    2565 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 01 16:58:49.925679 ip-10-0-137-244 kubenswrapper[2565]: W0701 16:58:49.864015    2565 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 01 16:58:49.925679 ip-10-0-137-244 kubenswrapper[2565]: W0701 16:58:49.864035    2565 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 01 16:58:49.925679 ip-10-0-137-244 kubenswrapper[2565]: W0701 16:58:49.864040    2565 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 01 16:58:49.925679 ip-10-0-137-244 kubenswrapper[2565]: W0701 16:58:49.864042    2565 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 01 16:58:49.925679 ip-10-0-137-244 kubenswrapper[2565]: W0701 16:58:49.864045    2565 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 01 16:58:49.925679 ip-10-0-137-244 kubenswrapper[2565]: W0701 16:58:49.864048    2565 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 01 16:58:49.925679 ip-10-0-137-244 kubenswrapper[2565]: W0701 16:58:49.864051    2565 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 01 16:58:49.925679 ip-10-0-137-244 kubenswrapper[2565]: W0701 16:58:49.864054    2565 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 01 16:58:49.925679 ip-10-0-137-244 kubenswrapper[2565]: W0701 16:58:49.864056    2565 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 01 16:58:49.925679 ip-10-0-137-244 kubenswrapper[2565]: W0701 16:58:49.864059    2565 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 01 16:58:49.925679 ip-10-0-137-244 kubenswrapper[2565]: W0701 16:58:49.864061    2565 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 01 16:58:49.925679 ip-10-0-137-244 kubenswrapper[2565]: W0701 16:58:49.864064    2565 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 01 16:58:49.925679 ip-10-0-137-244 kubenswrapper[2565]: W0701 16:58:49.864067    2565 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 01 16:58:49.925679 ip-10-0-137-244 kubenswrapper[2565]: W0701 16:58:49.864069    2565 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 01 16:58:49.925679 ip-10-0-137-244 kubenswrapper[2565]: W0701 16:58:49.864072    2565 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 01 16:58:49.925679 ip-10-0-137-244 kubenswrapper[2565]: W0701 16:58:49.864074    2565 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 01 16:58:49.925679 ip-10-0-137-244 kubenswrapper[2565]: W0701 16:58:49.864077    2565 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 01 16:58:49.925679 ip-10-0-137-244 kubenswrapper[2565]: W0701 16:58:49.864080    2565 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 01 16:58:49.925679 ip-10-0-137-244 kubenswrapper[2565]: W0701 16:58:49.864082    2565 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 01 16:58:49.925679 ip-10-0-137-244 kubenswrapper[2565]: W0701 16:58:49.864085    2565 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 01 16:58:49.926559 ip-10-0-137-244 kubenswrapper[2565]: W0701 16:58:49.864089    2565 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 01 16:58:49.926559 ip-10-0-137-244 kubenswrapper[2565]: W0701 16:58:49.864092    2565 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 01 16:58:49.926559 ip-10-0-137-244 kubenswrapper[2565]: W0701 16:58:49.864094    2565 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 01 16:58:49.926559 ip-10-0-137-244 kubenswrapper[2565]: W0701 16:58:49.864097    2565 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 01 16:58:49.926559 ip-10-0-137-244 kubenswrapper[2565]: W0701 16:58:49.864100    2565 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 01 16:58:49.926559 ip-10-0-137-244 kubenswrapper[2565]: W0701 16:58:49.864102    2565 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 01 16:58:49.926559 ip-10-0-137-244 kubenswrapper[2565]: W0701 16:58:49.864106    2565 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 01 16:58:49.926559 ip-10-0-137-244 kubenswrapper[2565]: W0701 16:58:49.864108    2565 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 01 16:58:49.926559 ip-10-0-137-244 kubenswrapper[2565]: W0701 16:58:49.864111    2565 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 01 16:58:49.926559 ip-10-0-137-244 kubenswrapper[2565]: W0701 16:58:49.864114    2565 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 01 16:58:49.926559 ip-10-0-137-244 kubenswrapper[2565]: W0701 16:58:49.864117    2565 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 01 16:58:49.926559 ip-10-0-137-244 kubenswrapper[2565]: W0701 16:58:49.864119    2565 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 01 16:58:49.926559 ip-10-0-137-244 kubenswrapper[2565]: W0701 16:58:49.864122    2565 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 01 16:58:49.926559 ip-10-0-137-244 kubenswrapper[2565]: W0701 16:58:49.864124    2565 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 01 16:58:49.926559 ip-10-0-137-244 kubenswrapper[2565]: W0701 16:58:49.864127    2565 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 01 16:58:49.926559 ip-10-0-137-244 kubenswrapper[2565]: W0701 16:58:49.864134    2565 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 01 16:58:49.926559 ip-10-0-137-244 kubenswrapper[2565]: W0701 16:58:49.864137    2565 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 01 16:58:49.926559 ip-10-0-137-244 kubenswrapper[2565]: W0701 16:58:49.864139    2565 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 01 16:58:49.926559 ip-10-0-137-244 kubenswrapper[2565]: W0701 16:58:49.864142    2565 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 01 16:58:49.926559 ip-10-0-137-244 kubenswrapper[2565]: W0701 16:58:49.864145    2565 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 01 16:58:49.927891 ip-10-0-137-244 kubenswrapper[2565]: W0701 16:58:49.864149    2565 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 01 16:58:49.927891 ip-10-0-137-244 kubenswrapper[2565]: W0701 16:58:49.864152    2565 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 01 16:58:49.927891 ip-10-0-137-244 kubenswrapper[2565]: W0701 16:58:49.864154    2565 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 01 16:58:49.927891 ip-10-0-137-244 kubenswrapper[2565]: W0701 16:58:49.864157    2565 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 01 16:58:49.927891 ip-10-0-137-244 kubenswrapper[2565]: W0701 16:58:49.864159    2565 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 01 16:58:49.927891 ip-10-0-137-244 kubenswrapper[2565]: W0701 16:58:49.864162    2565 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 01 16:58:49.927891 ip-10-0-137-244 kubenswrapper[2565]: W0701 16:58:49.864165    2565 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 01 16:58:49.927891 ip-10-0-137-244 kubenswrapper[2565]: W0701 16:58:49.864167    2565 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 01 16:58:49.927891 ip-10-0-137-244 kubenswrapper[2565]: W0701 16:58:49.864170    2565 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 01 16:58:49.927891 ip-10-0-137-244 kubenswrapper[2565]: W0701 16:58:49.864172    2565 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 01 16:58:49.927891 ip-10-0-137-244 kubenswrapper[2565]: W0701 16:58:49.864175    2565 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 01 16:58:49.927891 ip-10-0-137-244 kubenswrapper[2565]: W0701 16:58:49.864177    2565 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 01 16:58:49.927891 ip-10-0-137-244 kubenswrapper[2565]: W0701 16:58:49.864180    2565 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 01 16:58:49.927891 ip-10-0-137-244 kubenswrapper[2565]: W0701 16:58:49.864183    2565 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 01 16:58:49.927891 ip-10-0-137-244 kubenswrapper[2565]: W0701 16:58:49.864186    2565 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 01 16:58:49.927891 ip-10-0-137-244 kubenswrapper[2565]: W0701 16:58:49.864188    2565 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 01 16:58:49.927891 ip-10-0-137-244 kubenswrapper[2565]: W0701 16:58:49.864191    2565 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 01 16:58:49.927891 ip-10-0-137-244 kubenswrapper[2565]: W0701 16:58:49.864193    2565 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 01 16:58:49.927891 ip-10-0-137-244 kubenswrapper[2565]: W0701 16:58:49.864196    2565 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 01 16:58:49.927891 ip-10-0-137-244 kubenswrapper[2565]: W0701 16:58:49.864198    2565 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 01 16:58:49.928783 ip-10-0-137-244 kubenswrapper[2565]: W0701 16:58:49.864201    2565 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 01 16:58:49.928783 ip-10-0-137-244 kubenswrapper[2565]: W0701 16:58:49.864203    2565 feature_gate.go:328] unrecognized feature gate: Example
Jul 01 16:58:49.928783 ip-10-0-137-244 kubenswrapper[2565]: W0701 16:58:49.864206    2565 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 01 16:58:49.928783 ip-10-0-137-244 kubenswrapper[2565]: W0701 16:58:49.864208    2565 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 01 16:58:49.928783 ip-10-0-137-244 kubenswrapper[2565]: W0701 16:58:49.864211    2565 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 01 16:58:49.928783 ip-10-0-137-244 kubenswrapper[2565]: W0701 16:58:49.864213    2565 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 01 16:58:49.928783 ip-10-0-137-244 kubenswrapper[2565]: W0701 16:58:49.864218    2565 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 01 16:58:49.928783 ip-10-0-137-244 kubenswrapper[2565]: W0701 16:58:49.864223    2565 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 01 16:58:49.928783 ip-10-0-137-244 kubenswrapper[2565]: W0701 16:58:49.864226    2565 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 01 16:58:49.928783 ip-10-0-137-244 kubenswrapper[2565]: W0701 16:58:49.864228    2565 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 01 16:58:49.928783 ip-10-0-137-244 kubenswrapper[2565]: W0701 16:58:49.864231    2565 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 01 16:58:49.928783 ip-10-0-137-244 kubenswrapper[2565]: W0701 16:58:49.864234    2565 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 01 16:58:49.928783 ip-10-0-137-244 kubenswrapper[2565]: W0701 16:58:49.864238    2565 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 01 16:58:49.928783 ip-10-0-137-244 kubenswrapper[2565]: W0701 16:58:49.864240    2565 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 01 16:58:49.928783 ip-10-0-137-244 kubenswrapper[2565]: W0701 16:58:49.864243    2565 feature_gate.go:328] unrecognized feature gate: Example2
Jul 01 16:58:49.928783 ip-10-0-137-244 kubenswrapper[2565]: W0701 16:58:49.864245    2565 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 01 16:58:49.928783 ip-10-0-137-244 kubenswrapper[2565]: W0701 16:58:49.864248    2565 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 01 16:58:49.928783 ip-10-0-137-244 kubenswrapper[2565]: W0701 16:58:49.864250    2565 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 01 16:58:49.928783 ip-10-0-137-244 kubenswrapper[2565]: W0701 16:58:49.864253    2565 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 01 16:58:49.928783 ip-10-0-137-244 kubenswrapper[2565]: W0701 16:58:49.864256    2565 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 01 16:58:49.930368 ip-10-0-137-244 kubenswrapper[2565]: W0701 16:58:49.864260    2565 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 01 16:58:49.930368 ip-10-0-137-244 kubenswrapper[2565]: W0701 16:58:49.864263    2565 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 01 16:58:49.930368 ip-10-0-137-244 kubenswrapper[2565]: W0701 16:58:49.864266    2565 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 01 16:58:49.930368 ip-10-0-137-244 kubenswrapper[2565]: W0701 16:58:49.864269    2565 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 01 16:58:49.930368 ip-10-0-137-244 kubenswrapper[2565]: W0701 16:58:49.864272    2565 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 01 16:58:49.930368 ip-10-0-137-244 kubenswrapper[2565]: W0701 16:58:49.864275    2565 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 01 16:58:49.930368 ip-10-0-137-244 kubenswrapper[2565]: W0701 16:58:49.864278    2565 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 01 16:58:49.930368 ip-10-0-137-244 kubenswrapper[2565]: W0701 16:58:49.864280    2565 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 01 16:58:49.930368 ip-10-0-137-244 kubenswrapper[2565]: W0701 16:58:49.864283    2565 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 01 16:58:49.930368 ip-10-0-137-244 kubenswrapper[2565]: W0701 16:58:49.864287    2565 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 01 16:58:49.930368 ip-10-0-137-244 kubenswrapper[2565]: I0701 16:58:49.864293    2565 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]}
Jul 01 16:58:49.930368 ip-10-0-137-244 kubenswrapper[2565]: W0701 16:58:49.864437    2565 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 01 16:58:49.930368 ip-10-0-137-244 kubenswrapper[2565]: W0701 16:58:49.864441    2565 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 01 16:58:49.930368 ip-10-0-137-244 kubenswrapper[2565]: W0701 16:58:49.864445    2565 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 01 16:58:49.982826 ip-10-0-137-244 kubenswrapper[2565]: W0701 16:58:49.864448    2565 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 01 16:58:49.982826 ip-10-0-137-244 kubenswrapper[2565]: W0701 16:58:49.864450    2565 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 01 16:58:49.982826 ip-10-0-137-244 kubenswrapper[2565]: W0701 16:58:49.864453    2565 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 01 16:58:49.982826 ip-10-0-137-244 kubenswrapper[2565]: W0701 16:58:49.864456    2565 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 01 16:58:49.982826 ip-10-0-137-244 kubenswrapper[2565]: W0701 16:58:49.864459    2565 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 01 16:58:49.982826 ip-10-0-137-244 kubenswrapper[2565]: W0701 16:58:49.864461    2565 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 01 16:58:49.982826 ip-10-0-137-244 kubenswrapper[2565]: W0701 16:58:49.864465    2565 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 01 16:58:49.982826 ip-10-0-137-244 kubenswrapper[2565]: W0701 16:58:49.864468    2565 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 01 16:58:49.982826 ip-10-0-137-244 kubenswrapper[2565]: W0701 16:58:49.864470    2565 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 01 16:58:49.982826 ip-10-0-137-244 kubenswrapper[2565]: W0701 16:58:49.864473    2565 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 01 16:58:49.982826 ip-10-0-137-244 kubenswrapper[2565]: W0701 16:58:49.864480    2565 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 01 16:58:49.982826 ip-10-0-137-244 kubenswrapper[2565]: W0701 16:58:49.864483    2565 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 01 16:58:49.982826 ip-10-0-137-244 kubenswrapper[2565]: W0701 16:58:49.864485    2565 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 01 16:58:49.982826 ip-10-0-137-244 kubenswrapper[2565]: W0701 16:58:49.864488    2565 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 01 16:58:49.982826 ip-10-0-137-244 kubenswrapper[2565]: W0701 16:58:49.864491    2565 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 01 16:58:49.982826 ip-10-0-137-244 kubenswrapper[2565]: W0701 16:58:49.864493    2565 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 01 16:58:49.982826 ip-10-0-137-244 kubenswrapper[2565]: W0701 16:58:49.864496    2565 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 01 16:58:49.982826 ip-10-0-137-244 kubenswrapper[2565]: W0701 16:58:49.864499    2565 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 01 16:58:49.982826 ip-10-0-137-244 kubenswrapper[2565]: W0701 16:58:49.864501    2565 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 01 16:58:49.982826 ip-10-0-137-244 kubenswrapper[2565]: W0701 16:58:49.864504    2565 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 01 16:58:49.983891 ip-10-0-137-244 kubenswrapper[2565]: W0701 16:58:49.864507    2565 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 01 16:58:49.983891 ip-10-0-137-244 kubenswrapper[2565]: W0701 16:58:49.864509    2565 feature_gate.go:328] unrecognized feature gate: Example2
Jul 01 16:58:49.983891 ip-10-0-137-244 kubenswrapper[2565]: W0701 16:58:49.864512    2565 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 01 16:58:49.983891 ip-10-0-137-244 kubenswrapper[2565]: W0701 16:58:49.864514    2565 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 01 16:58:49.983891 ip-10-0-137-244 kubenswrapper[2565]: W0701 16:58:49.864517    2565 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 01 16:58:49.983891 ip-10-0-137-244 kubenswrapper[2565]: W0701 16:58:49.864520    2565 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 01 16:58:49.983891 ip-10-0-137-244 kubenswrapper[2565]: W0701 16:58:49.864522    2565 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 01 16:58:49.983891 ip-10-0-137-244 kubenswrapper[2565]: W0701 16:58:49.864525    2565 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 01 16:58:49.983891 ip-10-0-137-244 kubenswrapper[2565]: W0701 16:58:49.864528    2565 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 01 16:58:49.983891 ip-10-0-137-244 kubenswrapper[2565]: W0701 16:58:49.864531    2565 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 01 16:58:49.983891 ip-10-0-137-244 kubenswrapper[2565]: W0701 16:58:49.864533    2565 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 01 16:58:49.983891 ip-10-0-137-244 kubenswrapper[2565]: W0701 16:58:49.864536    2565 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 01 16:58:49.983891 ip-10-0-137-244 kubenswrapper[2565]: W0701 16:58:49.864538    2565 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 01 16:58:49.983891 ip-10-0-137-244 kubenswrapper[2565]: W0701 16:58:49.864541    2565 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 01 16:58:49.983891 ip-10-0-137-244 kubenswrapper[2565]: W0701 16:58:49.864543    2565 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 01 16:58:49.983891 ip-10-0-137-244 kubenswrapper[2565]: W0701 16:58:49.864546    2565 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 01 16:58:49.983891 ip-10-0-137-244 kubenswrapper[2565]: W0701 16:58:49.864548    2565 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 01 16:58:49.983891 ip-10-0-137-244 kubenswrapper[2565]: W0701 16:58:49.864551    2565 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 01 16:58:49.983891 ip-10-0-137-244 kubenswrapper[2565]: W0701 16:58:49.864555    2565 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 01 16:58:49.983891 ip-10-0-137-244 kubenswrapper[2565]: W0701 16:58:49.864557    2565 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 01 16:58:49.984786 ip-10-0-137-244 kubenswrapper[2565]: W0701 16:58:49.864560    2565 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 01 16:58:49.984786 ip-10-0-137-244 kubenswrapper[2565]: W0701 16:58:49.864563    2565 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 01 16:58:49.984786 ip-10-0-137-244 kubenswrapper[2565]: W0701 16:58:49.864566    2565 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 01 16:58:49.984786 ip-10-0-137-244 kubenswrapper[2565]: W0701 16:58:49.864569    2565 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 01 16:58:49.984786 ip-10-0-137-244 kubenswrapper[2565]: W0701 16:58:49.864572    2565 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 01 16:58:49.984786 ip-10-0-137-244 kubenswrapper[2565]: W0701 16:58:49.864574    2565 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 01 16:58:49.984786 ip-10-0-137-244 kubenswrapper[2565]: W0701 16:58:49.864577    2565 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 01 16:58:49.984786 ip-10-0-137-244 kubenswrapper[2565]: W0701 16:58:49.864580    2565 feature_gate.go:328] unrecognized feature gate: Example
Jul 01 16:58:49.984786 ip-10-0-137-244 kubenswrapper[2565]: W0701 16:58:49.864582    2565 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 01 16:58:49.984786 ip-10-0-137-244 kubenswrapper[2565]: W0701 16:58:49.864585    2565 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 01 16:58:49.984786 ip-10-0-137-244 kubenswrapper[2565]: W0701 16:58:49.864589    2565 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 01 16:58:49.984786 ip-10-0-137-244 kubenswrapper[2565]: W0701 16:58:49.864592    2565 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 01 16:58:49.984786 ip-10-0-137-244 kubenswrapper[2565]: W0701 16:58:49.864596    2565 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 01 16:58:49.984786 ip-10-0-137-244 kubenswrapper[2565]: W0701 16:58:49.864599    2565 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 01 16:58:49.984786 ip-10-0-137-244 kubenswrapper[2565]: W0701 16:58:49.864601    2565 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 01 16:58:49.984786 ip-10-0-137-244 kubenswrapper[2565]: W0701 16:58:49.864604    2565 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 01 16:58:49.984786 ip-10-0-137-244 kubenswrapper[2565]: W0701 16:58:49.864607    2565 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 01 16:58:49.984786 ip-10-0-137-244 kubenswrapper[2565]: W0701 16:58:49.864610    2565 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 01 16:58:49.984786 ip-10-0-137-244 kubenswrapper[2565]: W0701 16:58:49.864613    2565 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 01 16:58:49.985688 ip-10-0-137-244 kubenswrapper[2565]: W0701 16:58:49.864616    2565 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 01 16:58:49.985688 ip-10-0-137-244 kubenswrapper[2565]: W0701 16:58:49.864619    2565 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 01 16:58:49.985688 ip-10-0-137-244 kubenswrapper[2565]: W0701 16:58:49.864621    2565 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 01 16:58:49.985688 ip-10-0-137-244 kubenswrapper[2565]: W0701 16:58:49.864624    2565 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 01 16:58:49.985688 ip-10-0-137-244 kubenswrapper[2565]: W0701 16:58:49.864626    2565 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 01 16:58:49.985688 ip-10-0-137-244 kubenswrapper[2565]: W0701 16:58:49.864629    2565 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 01 16:58:49.985688 ip-10-0-137-244 kubenswrapper[2565]: W0701 16:58:49.864632    2565 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 01 16:58:49.985688 ip-10-0-137-244 kubenswrapper[2565]: W0701 16:58:49.864634    2565 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 01 16:58:49.985688 ip-10-0-137-244 kubenswrapper[2565]: W0701 16:58:49.864637    2565 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 01 16:58:49.985688 ip-10-0-137-244 kubenswrapper[2565]: W0701 16:58:49.864640    2565 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 01 16:58:49.985688 ip-10-0-137-244 kubenswrapper[2565]: W0701 16:58:49.864644    2565 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 01 16:58:49.985688 ip-10-0-137-244 kubenswrapper[2565]: W0701 16:58:49.864648    2565 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 01 16:58:49.985688 ip-10-0-137-244 kubenswrapper[2565]: W0701 16:58:49.864651    2565 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 01 16:58:49.985688 ip-10-0-137-244 kubenswrapper[2565]: W0701 16:58:49.864654    2565 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 01 16:58:49.985688 ip-10-0-137-244 kubenswrapper[2565]: W0701 16:58:49.864656    2565 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 01 16:58:49.985688 ip-10-0-137-244 kubenswrapper[2565]: W0701 16:58:49.864660    2565 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 01 16:58:49.985688 ip-10-0-137-244 kubenswrapper[2565]: W0701 16:58:49.864663    2565 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 01 16:58:49.985688 ip-10-0-137-244 kubenswrapper[2565]: W0701 16:58:49.864666    2565 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 01 16:58:49.985688 ip-10-0-137-244 kubenswrapper[2565]: W0701 16:58:49.864669    2565 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 01 16:58:49.986576 ip-10-0-137-244 kubenswrapper[2565]: W0701 16:58:49.864672    2565 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 01 16:58:49.986576 ip-10-0-137-244 kubenswrapper[2565]: W0701 16:58:49.864675    2565 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 01 16:58:49.986576 ip-10-0-137-244 kubenswrapper[2565]: W0701 16:58:49.864678    2565 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 01 16:58:49.986576 ip-10-0-137-244 kubenswrapper[2565]: W0701 16:58:49.864682    2565 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 01 16:58:49.986576 ip-10-0-137-244 kubenswrapper[2565]: W0701 16:58:49.864685    2565 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 01 16:58:49.986576 ip-10-0-137-244 kubenswrapper[2565]: W0701 16:58:49.864688    2565 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 01 16:58:49.986576 ip-10-0-137-244 kubenswrapper[2565]: W0701 16:58:49.864690    2565 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 01 16:58:49.986576 ip-10-0-137-244 kubenswrapper[2565]: W0701 16:58:49.864693    2565 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 01 16:58:49.986576 ip-10-0-137-244 kubenswrapper[2565]: W0701 16:58:49.864696    2565 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 01 16:58:49.986576 ip-10-0-137-244 kubenswrapper[2565]: W0701 16:58:49.864698    2565 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 01 16:58:49.986576 ip-10-0-137-244 kubenswrapper[2565]: W0701 16:58:49.864701    2565 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 01 16:58:49.986576 ip-10-0-137-244 kubenswrapper[2565]: W0701 16:58:49.864704    2565 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 01 16:58:49.986576 ip-10-0-137-244 kubenswrapper[2565]: W0701 16:58:49.864706    2565 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 01 16:58:49.986576 ip-10-0-137-244 kubenswrapper[2565]: W0701 16:58:49.864709    2565 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 01 16:58:49.986576 ip-10-0-137-244 kubenswrapper[2565]: W0701 16:58:49.864712    2565 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 01 16:58:49.986576 ip-10-0-137-244 kubenswrapper[2565]: W0701 16:58:49.864714    2565 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 01 16:58:49.986576 ip-10-0-137-244 kubenswrapper[2565]: W0701 16:58:49.864717    2565 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 01 16:58:49.986576 ip-10-0-137-244 kubenswrapper[2565]: W0701 16:58:49.864719    2565 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 01 16:58:49.987460 ip-10-0-137-244 kubenswrapper[2565]: I0701 16:58:49.864723    2565 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]}
Jul 01 16:58:49.987460 ip-10-0-137-244 kubenswrapper[2565]: I0701 16:58:49.864820    2565 server.go:962] "Client rotation is on, will bootstrap in background"
Jul 01 16:58:49.987460 ip-10-0-137-244 kubenswrapper[2565]: I0701 16:58:49.867066    2565 bootstrap.go:101] "Use the bootstrap credentials to request a cert, and set kubeconfig to point to the certificate dir"
Jul 01 16:58:49.987460 ip-10-0-137-244 kubenswrapper[2565]: I0701 16:58:49.868218    2565 server.go:1019] "Starting client certificate rotation"
Jul 01 16:58:49.987460 ip-10-0-137-244 kubenswrapper[2565]: I0701 16:58:49.868316    2565 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kube-apiserver-client-kubelet"
Jul 01 16:58:49.987460 ip-10-0-137-244 kubenswrapper[2565]: I0701 16:58:49.868352    2565 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kube-apiserver-client-kubelet"
Jul 01 16:58:49.987460 ip-10-0-137-244 kubenswrapper[2565]: I0701 16:58:49.870903    2565 dynamic_cafile_content.go:123] "Loaded a new CA Bundle and Verifier" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt"
Jul 01 16:58:49.987460 ip-10-0-137-244 kubenswrapper[2565]: I0701 16:58:49.872559    2565 dynamic_cafile_content.go:161] "Starting controller" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt"
Jul 01 16:58:49.987460 ip-10-0-137-244 kubenswrapper[2565]: I0701 16:58:49.909350    2565 reflector.go:439] "Caches populated" logger="kubernetes.io/kube-apiserver-client-kubelet" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162"
Jul 01 16:58:49.987460 ip-10-0-137-244 kubenswrapper[2565]: I0701 16:58:49.985703    2565 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-xc2vn"
Jul 01 16:58:49.996554 ip-10-0-137-244 kubenswrapper[2565]: I0701 16:58:49.996290    2565 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-xc2vn"
Jul 01 16:58:50.068659 ip-10-0-137-244 kubenswrapper[2565]: I0701 16:58:50.025180    2565 log.go:25] "Validated CRI v1 runtime API"
Jul 01 16:58:50.068659 ip-10-0-137-244 kubenswrapper[2565]: I0701 16:58:50.027306    2565 log.go:25] "Validated CRI v1 image API"
Jul 01 16:58:50.068659 ip-10-0-137-244 kubenswrapper[2565]: I0701 16:58:50.043966    2565 server.go:1429] "Using cgroup driver setting received from the CRI runtime" cgroupDriver="systemd"
Jul 01 16:58:50.068659 ip-10-0-137-244 kubenswrapper[2565]: I0701 16:58:50.047584    2565 fs.go:135] Filesystem UUIDs: map[7B77-95E7:/dev/nvme0n1p2 84addb73-79d0-491c-b839-acb3601bc07b:/dev/nvme0n1p4 c4ee0a81-9989-47a1-829b-6496b54e4771:/dev/nvme0n1p3]
Jul 01 16:58:50.068659 ip-10-0-137-244 kubenswrapper[2565]: I0701 16:58:50.047604    2565 fs.go:136] Filesystem partitions: map[/dev/nvme0n1p3:{mountpoint:/boot major:259 minor:3 fsType:ext4 blockSize:0} /dev/nvme0n1p4:{mountpoint:/var major:259 minor:4 fsType:xfs blockSize:0} /dev/shm:{mountpoint:/dev/shm major:0 minor:22 fsType:tmpfs blockSize:0} /run:{mountpoint:/run major:0 minor:24 fsType:tmpfs blockSize:0} /tmp:{mountpoint:/tmp major:0 minor:31 fsType:tmpfs blockSize:0} composefs_0-33:{mountpoint:/ major:0 minor:33 fsType:overlay blockSize:0}]
Jul 01 16:58:50.069747 ip-10-0-137-244 kubenswrapper[2565]: I0701 16:58:50.053392    2565 manager.go:216] Machine: {Timestamp:2026-07-01 16:58:50.052015823 +0000 UTC m=+0.520279944 CPUVendorID:GenuineIntel NumCores:8 NumPhysicalCores:4 NumSockets:1 CpuFrequency:3097089 MemoryCapacity:33164492800 SwapCapacity:0 MemoryByType:map[] NVMInfo:{MemoryModeCapacity:0 AppDirectModeCapacity:0 AvgPowerBudget:0} HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] MachineID:ec2341d06782b374c876907d4070d005 SystemUUID:ec2341d0-6782-b374-c876-907d4070d005 BootID:1940d91c-9a9b-48c3-a13e-12aecef91f73 Filesystems:[{Device:/dev/shm DeviceMajor:0 DeviceMinor:22 Capacity:16582246400 Type:vfs Inodes:4048400 HasInodes:true} {Device:/run DeviceMajor:0 DeviceMinor:24 Capacity:6632898560 Type:vfs Inodes:819200 HasInodes:true} {Device:/dev/nvme0n1p4 DeviceMajor:259 DeviceMinor:4 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:composefs_0-33 DeviceMajor:0 DeviceMinor:33 Capacity:6107136 Type:vfs Inodes:18446744073709551615 HasInodes:true} {Device:/tmp DeviceMajor:0 DeviceMinor:31 Capacity:16582246400 Type:vfs Inodes:1048576 HasInodes:true} {Device:/dev/nvme0n1p3 DeviceMajor:259 DeviceMinor:3 Capacity:366869504 Type:vfs Inodes:98304 HasInodes:true}] DiskMap:map[259:0:{Name:nvme0n1 Major:259 Minor:0 Size:128849018880 Scheduler:none}] NetworkDevices:[{Name:br-ex MacAddress:02:3a:a4:e8:12:17 Speed:0 Mtu:9001} {Name:ens5 MacAddress:02:3a:a4:e8:12:17 Speed:0 Mtu:9001} {Name:ovs-system MacAddress:d2:86:c7:39:2c:e6 Speed:0 Mtu:1500}] Topology:[{Id:0 Memory:33164492800 HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] Cores:[{Id:0 Threads:[0 4] Caches:[{Id:0 Size:32768 Type:Data Level:1} {Id:0 Size:32768 Type:Instruction Level:1} {Id:0 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:1 Threads:[1 5] Caches:[{Id:1 Size:32768 Type:Data Level:1} {Id:1 Size:32768 Type:Instruction Level:1} {Id:1 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:2 Threads:[2 6] Caches:[{Id:2 Size:32768 Type:Data Level:1} {Id:2 Size:32768 Type:Instruction Level:1} {Id:2 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:3 Threads:[3 7] Caches:[{Id:3 Size:32768 Type:Data Level:1} {Id:3 Size:32768 Type:Instruction Level:1} {Id:3 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:}] Caches:[{Id:0 Size:37486592 Type:Unified Level:3}] Distances:[10]}] CloudProvider:Unknown InstanceType:Unknown InstanceID:None}
Jul 01 16:58:50.069747 ip-10-0-137-244 kubenswrapper[2565]: I0701 16:58:50.053487    2565 manager_no_libpfm.go:29] cAdvisor is build without cgo and/or libpfm support. Perf event counters are not available.
Jul 01 16:58:50.069747 ip-10-0-137-244 kubenswrapper[2565]: I0701 16:58:50.053580    2565 manager.go:232] Version: {KernelVersion:5.14.0-570.124.1.el9_6.x86_64 ContainerOsVersion:Red Hat Enterprise Linux CoreOS 9.6.20260623-0 (Plow) DockerVersion: DockerAPIVersion: CadvisorVersion: CadvisorRevision:}
Jul 01 16:58:50.069747 ip-10-0-137-244 kubenswrapper[2565]: I0701 16:58:50.057318    2565 container_manager_linux.go:273] "Container manager verified user specified cgroup-root exists" cgroupRoot=[]
Jul 01 16:58:50.069747 ip-10-0-137-244 kubenswrapper[2565]: I0701 16:58:50.057340    2565 container_manager_linux.go:278] "Creating Container Manager object based on Node Config" nodeConfig={"NodeName":"ip-10-0-137-244.ec2.internal","RuntimeCgroupsName":"/system.slice/crio.service","SystemCgroupsName":"/system.slice","KubeletCgroupsName":"","KubeletOOMScoreAdj":-999,"ContainerRuntime":"","CgroupsPerQOS":true,"CgroupRoot":"/","CgroupDriver":"systemd","KubeletRootDir":"/var/lib/kubelet","ProtectKernelDefaults":true,"KubeReservedCgroupName":"","SystemReservedCgroupName":"","ReservedSystemCPUs":{},"EnforceNodeAllocatable":{"pods":{}},"KubeReserved":null,"SystemReserved":{"cpu":"500m","ephemeral-storage":"1Gi","memory":"1Gi"},"HardEvictionThresholds":[{"Signal":"imagefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.15},"GracePeriod":0,"MinReclaim":null},{"Signal":"imagefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null},{"Signal":"memory.available","Operator":"LessThan","Value":{"Quantity":"100Mi","Percentage":0},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.1},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null}],"QOSReserved":{},"CPUManagerPolicy":"none","CPUManagerPolicyOptions":null,"TopologyManagerScope":"container","CPUManagerReconcilePeriod":10000000000,"MemoryManagerPolicy":"None","MemoryManagerReservedMemory":null,"PodPidsLimit":4096,"EnforceCPULimits":true,"CPUCFSQuotaPeriod":100000000,"TopologyManagerPolicy":"none","TopologyManagerPolicyOptions":null,"CgroupVersion":2}
Jul 01 16:58:50.069747 ip-10-0-137-244 kubenswrapper[2565]: I0701 16:58:50.058063    2565 topology_manager.go:138] "Creating topology manager with none policy"
Jul 01 16:58:50.069747 ip-10-0-137-244 kubenswrapper[2565]: I0701 16:58:50.058073    2565 container_manager_linux.go:309] "Creating device plugin manager"
Jul 01 16:58:50.069747 ip-10-0-137-244 kubenswrapper[2565]: I0701 16:58:50.058080    2565 manager.go:141] "Creating Device Plugin manager" path="/var/lib/kubelet/device-plugins/kubelet.sock"
Jul 01 16:58:50.069747 ip-10-0-137-244 kubenswrapper[2565]: I0701 16:58:50.058842    2565 server.go:74] "Creating device plugin registration server" version="v1beta1" socket="/var/lib/kubelet/device-plugins/kubelet.sock"
Jul 01 16:58:50.069747 ip-10-0-137-244 kubenswrapper[2565]: I0701 16:58:50.059711    2565 container_manager_linux.go:318] "Creating Dynamic Resource Allocation (DRA) manager"
Jul 01 16:58:50.069747 ip-10-0-137-244 kubenswrapper[2565]: I0701 16:58:50.062769    2565 state_mem.go:36] "Initialized new in-memory state store"
Jul 01 16:58:50.069747 ip-10-0-137-244 kubenswrapper[2565]: I0701 16:58:50.062877    2565 server.go:1240] "Using root directory" path="/var/lib/kubelet"
Jul 01 16:58:50.069747 ip-10-0-137-244 kubenswrapper[2565]: I0701 16:58:50.062904    2565 kubelet.go:486] "Attempting to sync node with API server"
Jul 01 16:58:50.069747 ip-10-0-137-244 kubenswrapper[2565]: I0701 16:58:50.062922    2565 kubelet.go:387] "Adding static pod path" path="/etc/kubernetes/manifests"
Jul 01 16:58:50.069747 ip-10-0-137-244 kubenswrapper[2565]: I0701 16:58:50.062935    2565 file.go:69] "Watching path" path="/etc/kubernetes/manifests"
Jul 01 16:58:50.069747 ip-10-0-137-244 kubenswrapper[2565]: I0701 16:58:50.062944    2565 kubelet.go:398] "Adding apiserver pod source"
Jul 01 16:58:50.069747 ip-10-0-137-244 kubenswrapper[2565]: I0701 16:58:50.062953    2565 apiserver.go:42] "Waiting for node sync before watching apiserver pods"
Jul 01 16:58:50.069747 ip-10-0-137-244 kubenswrapper[2565]: I0701 16:58:50.063286    2565 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking"
Jul 01 16:58:50.070425 ip-10-0-137-244 kubenswrapper[2565]: I0701 16:58:50.063302    2565 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking"
Jul 01 16:58:50.070425 ip-10-0-137-244 kubenswrapper[2565]: I0701 16:58:50.065925    2565 kuberuntime_manager.go:301] "Container runtime initialized" containerRuntime="cri-o" version="1.34.8-3.rhaos4.21.git91b4b0b.el9" apiVersion="v1"
Jul 01 16:58:50.070425 ip-10-0-137-244 kubenswrapper[2565]: I0701 16:58:50.067451    2565 kubelet.go:960] "Not starting ClusterTrustBundle informer because we are in static kubelet mode or the ClusterTrustBundleProjection featuregate is disabled"
Jul 01 16:58:50.070425 ip-10-0-137-244 kubenswrapper[2565]: I0701 16:58:50.067487    2565 kubelet.go:984] "Not starting PodCertificateRequest manager because we are in static kubelet mode or the PodCertificateProjection feature gate is disabled"
Jul 01 16:58:50.070425 ip-10-0-137-244 kubenswrapper[2565]: I0701 16:58:50.068171    2565 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/portworx-volume"
Jul 01 16:58:50.070425 ip-10-0-137-244 kubenswrapper[2565]: I0701 16:58:50.068188    2565 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/empty-dir"
Jul 01 16:58:50.070425 ip-10-0-137-244 kubenswrapper[2565]: I0701 16:58:50.068205    2565 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/git-repo"
Jul 01 16:58:50.070425 ip-10-0-137-244 kubenswrapper[2565]: I0701 16:58:50.068214    2565 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/host-path"
Jul 01 16:58:50.070425 ip-10-0-137-244 kubenswrapper[2565]: I0701 16:58:50.068226    2565 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/nfs"
Jul 01 16:58:50.070425 ip-10-0-137-244 kubenswrapper[2565]: I0701 16:58:50.068233    2565 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/secret"
Jul 01 16:58:50.070425 ip-10-0-137-244 kubenswrapper[2565]: I0701 16:58:50.068243    2565 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/iscsi"
Jul 01 16:58:50.070425 ip-10-0-137-244 kubenswrapper[2565]: I0701 16:58:50.068250    2565 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/downward-api"
Jul 01 16:58:50.070425 ip-10-0-137-244 kubenswrapper[2565]: I0701 16:58:50.068260    2565 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/fc"
Jul 01 16:58:50.070425 ip-10-0-137-244 kubenswrapper[2565]: I0701 16:58:50.068267    2565 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/configmap"
Jul 01 16:58:50.070425 ip-10-0-137-244 kubenswrapper[2565]: I0701 16:58:50.068284    2565 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/projected"
Jul 01 16:58:50.070425 ip-10-0-137-244 kubenswrapper[2565]: I0701 16:58:50.068291    2565 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/local-volume"
Jul 01 16:58:50.070425 ip-10-0-137-244 kubenswrapper[2565]: I0701 16:58:50.068311    2565 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/csi"
Jul 01 16:58:50.070425 ip-10-0-137-244 kubenswrapper[2565]: I0701 16:58:50.068323    2565 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/image"
Jul 01 16:58:50.073397 ip-10-0-137-244 kubenswrapper[2565]: I0701 16:58:50.071959    2565 server.go:1268] "Started kubelet"
Jul 01 16:58:50.152316 ip-10-0-137-244 kubenswrapper[2565]: I0701 16:58:50.086198    2565 server.go:180] "Starting to listen" address="0.0.0.0" port=10250
Jul 01 16:58:50.152316 ip-10-0-137-244 kubenswrapper[2565]: I0701 16:58:50.087373    2565 ratelimit.go:56] "Setting rate limiting for endpoint" service="podresources" qps=100 burstTokens=10
Jul 01 16:58:50.152316 ip-10-0-137-244 kubenswrapper[2565]: I0701 16:58:50.087527    2565 server_v1.go:49] "podresources" method="list" useActivePods=true
Jul 01 16:58:50.152316 ip-10-0-137-244 kubenswrapper[2565]: I0701 16:58:50.088722    2565 server.go:249] "Starting to serve the podresources API" endpoint="unix:/var/lib/kubelet/pod-resources/kubelet.sock"
Jul 01 16:58:50.152316 ip-10-0-137-244 kubenswrapper[2565]: I0701 16:58:50.089564    2565 server.go:310] "Adding debug handlers to kubelet server"
Jul 01 16:58:50.152316 ip-10-0-137-244 kubenswrapper[2565]: I0701 16:58:50.093888    2565 reflector.go:439] "Caches populated" type="*v1.Node" reflector="k8s.io/client-go/informers/factory.go:160"
Jul 01 16:58:50.152316 ip-10-0-137-244 kubenswrapper[2565]: E0701 16:58:50.093992    2565 kubelet.go:1638] "Image garbage collection failed once. Stats initialization may not have completed yet" err="failed to get imageFs info: unable to find data in memory cache"
Jul 01 16:58:50.152316 ip-10-0-137-244 kubenswrapper[2565]: I0701 16:58:50.094400    2565 reflector.go:439] "Caches populated" type="*v1.Service" reflector="k8s.io/client-go/informers/factory.go:160"
Jul 01 16:58:50.152316 ip-10-0-137-244 kubenswrapper[2565]: I0701 16:58:50.097593    2565 fs_resource_analyzer.go:67] "Starting FS ResourceAnalyzer"
Jul 01 16:58:50.152316 ip-10-0-137-244 kubenswrapper[2565]: I0701 16:58:50.097608    2565 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kubelet-serving"
Jul 01 16:58:50.152316 ip-10-0-137-244 kubenswrapper[2565]: E0701 16:58:50.097666    2565 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-137-244.ec2.internal\" not found"
Jul 01 16:58:50.152316 ip-10-0-137-244 kubenswrapper[2565]: I0701 16:58:50.098123    2565 desired_state_of_world_populator.go:146] "Desired state populator starts to run"
Jul 01 16:58:50.152316 ip-10-0-137-244 kubenswrapper[2565]: I0701 16:58:50.098125    2565 volume_manager.go:311] "The desired_state_of_world populator starts"
Jul 01 16:58:50.152316 ip-10-0-137-244 kubenswrapper[2565]: I0701 16:58:50.098152    2565 volume_manager.go:313] "Starting Kubelet Volume Manager"
Jul 01 16:58:50.152316 ip-10-0-137-244 kubenswrapper[2565]: I0701 16:58:50.098242    2565 reconstruct.go:97] "Volume reconstruction finished"
Jul 01 16:58:50.152316 ip-10-0-137-244 kubenswrapper[2565]: I0701 16:58:50.098249    2565 reconciler.go:29] "Reconciler: start to sync state"
Jul 01 16:58:50.152316 ip-10-0-137-244 kubenswrapper[2565]: I0701 16:58:50.098529    2565 factory.go:55] Registering systemd factory
Jul 01 16:58:50.152316 ip-10-0-137-244 kubenswrapper[2565]: I0701 16:58:50.098570    2565 factory.go:223] Registration of the systemd container factory successfully
Jul 01 16:58:50.152316 ip-10-0-137-244 kubenswrapper[2565]: I0701 16:58:50.098805    2565 factory.go:153] Registering CRI-O factory
Jul 01 16:58:50.152316 ip-10-0-137-244 kubenswrapper[2565]: I0701 16:58:50.098820    2565 factory.go:223] Registration of the crio container factory successfully
Jul 01 16:58:50.152316 ip-10-0-137-244 kubenswrapper[2565]: I0701 16:58:50.098882    2565 factory.go:221] Registration of the containerd container factory failed: unable to create containerd client: containerd: cannot unix dial containerd api service: dial unix /run/containerd/containerd.sock: connect: no such file or directory
Jul 01 16:58:50.152316 ip-10-0-137-244 kubenswrapper[2565]: I0701 16:58:50.098911    2565 factory.go:103] Registering Raw factory
Jul 01 16:58:50.152316 ip-10-0-137-244 kubenswrapper[2565]: I0701 16:58:50.098924    2565 manager.go:1196] Started watching for new ooms in manager
Jul 01 16:58:50.152316 ip-10-0-137-244 kubenswrapper[2565]: I0701 16:58:50.099430    2565 manager.go:349] Starting recovery of all containers
Jul 01 16:58:50.152316 ip-10-0-137-244 kubenswrapper[2565]: I0701 16:58:50.099745    2565 reflector.go:439] "Caches populated" type="*v1.CSIDriver" reflector="k8s.io/client-go/informers/factory.go:160"
Jul 01 16:58:50.152316 ip-10-0-137-244 kubenswrapper[2565]: I0701 16:58:50.099959    2565 nodeinfomanager.go:417] Failed to publish CSINode: nodes "ip-10-0-137-244.ec2.internal" not found
Jul 01 16:58:50.152316 ip-10-0-137-244 kubenswrapper[2565]: I0701 16:58:50.105364    2565 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv6"
Jul 01 16:58:50.152316 ip-10-0-137-244 kubenswrapper[2565]: E0701 16:58:50.105890    2565 nodelease.go:49] "Failed to get node when trying to set owner ref to the node lease" err="nodes \"ip-10-0-137-244.ec2.internal\" not found" node="ip-10-0-137-244.ec2.internal"
Jul 01 16:58:50.152316 ip-10-0-137-244 kubenswrapper[2565]: I0701 16:58:50.109970    2565 manager.go:354] Recovery completed
Jul 01 16:58:50.152316 ip-10-0-137-244 kubenswrapper[2565]: I0701 16:58:50.116287    2565 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 01 16:58:50.152316 ip-10-0-137-244 kubenswrapper[2565]: I0701 16:58:50.117933    2565 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-137-244.ec2.internal" event="NodeHasSufficientMemory"
Jul 01 16:58:50.152316 ip-10-0-137-244 kubenswrapper[2565]: I0701 16:58:50.117963    2565 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-137-244.ec2.internal" event="NodeHasNoDiskPressure"
Jul 01 16:58:50.152316 ip-10-0-137-244 kubenswrapper[2565]: I0701 16:58:50.117976    2565 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-137-244.ec2.internal" event="NodeHasSufficientPID"
Jul 01 16:58:50.152316 ip-10-0-137-244 kubenswrapper[2565]: I0701 16:58:50.118334    2565 cpu_manager.go:222] "Starting CPU manager" policy="none"
Jul 01 16:58:50.152316 ip-10-0-137-244 kubenswrapper[2565]: I0701 16:58:50.118342    2565 cpu_manager.go:223] "Reconciling" reconcilePeriod="10s"
Jul 01 16:58:50.152316 ip-10-0-137-244 kubenswrapper[2565]: I0701 16:58:50.118344    2565 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 01 16:58:50.152316 ip-10-0-137-244 kubenswrapper[2565]: I0701 16:58:50.118360    2565 state_mem.go:36] "Initialized new in-memory state store"
Jul 01 16:58:50.152316 ip-10-0-137-244 kubenswrapper[2565]: I0701 16:58:50.119006    2565 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-137-244.ec2.internal" event="NodeHasSufficientMemory"
Jul 01 16:58:50.152316 ip-10-0-137-244 kubenswrapper[2565]: I0701 16:58:50.119049    2565 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-137-244.ec2.internal" event="NodeHasNoDiskPressure"
Jul 01 16:58:50.152316 ip-10-0-137-244 kubenswrapper[2565]: I0701 16:58:50.119063    2565 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-137-244.ec2.internal" event="NodeHasSufficientPID"
Jul 01 16:58:50.152316 ip-10-0-137-244 kubenswrapper[2565]: I0701 16:58:50.119450    2565 policy_none.go:49] "None policy: Start"
Jul 01 16:58:50.152316 ip-10-0-137-244 kubenswrapper[2565]: I0701 16:58:50.119462    2565 memory_manager.go:187] "Starting memorymanager" policy="None"
Jul 01 16:58:50.152316 ip-10-0-137-244 kubenswrapper[2565]: I0701 16:58:50.119471    2565 state_mem.go:36] "Initializing new in-memory state store" logger="Memory Manager state checkpoint"
Jul 01 16:58:50.152316 ip-10-0-137-244 kubenswrapper[2565]: I0701 16:58:50.120334    2565 policy_none.go:47] "Start"
Jul 01 16:58:50.152316 ip-10-0-137-244 kubenswrapper[2565]: I0701 16:58:50.138460    2565 nodeinfomanager.go:417] Failed to publish CSINode: nodes "ip-10-0-137-244.ec2.internal" not found
Jul 01 16:58:50.086601 ip-10-0-137-244 systemd[1]: Started Kubernetes Kubelet.
Jul 01 16:58:50.241735 ip-10-0-137-244 kubenswrapper[2565]: I0701 16:58:50.186601    2565 manager.go:337] "Starting Device Plugin manager"
Jul 01 16:58:50.241735 ip-10-0-137-244 kubenswrapper[2565]: E0701 16:58:50.186743    2565 manager.go:513] "Failed to read data from checkpoint" err="checkpoint is not found" checkpoint="kubelet_internal_checkpoint"
Jul 01 16:58:50.241735 ip-10-0-137-244 kubenswrapper[2565]: I0701 16:58:50.186756    2565 server.go:87] "Starting device plugin registration server"
Jul 01 16:58:50.241735 ip-10-0-137-244 kubenswrapper[2565]: I0701 16:58:50.186965    2565 eviction_manager.go:189] "Eviction manager: starting control loop"
Jul 01 16:58:50.241735 ip-10-0-137-244 kubenswrapper[2565]: I0701 16:58:50.186973    2565 container_log_manager.go:146] "Initializing container log rotate workers" workers=1 monitorPeriod="10s"
Jul 01 16:58:50.241735 ip-10-0-137-244 kubenswrapper[2565]: I0701 16:58:50.187080    2565 plugin_watcher.go:51] "Plugin Watcher Start" path="/var/lib/kubelet/plugins_registry"
Jul 01 16:58:50.241735 ip-10-0-137-244 kubenswrapper[2565]: I0701 16:58:50.187169    2565 plugin_manager.go:116] "The desired_state_of_world populator (plugin watcher) starts"
Jul 01 16:58:50.241735 ip-10-0-137-244 kubenswrapper[2565]: I0701 16:58:50.187178    2565 plugin_manager.go:118] "Starting Kubelet Plugin Manager"
Jul 01 16:58:50.241735 ip-10-0-137-244 kubenswrapper[2565]: E0701 16:58:50.187806    2565 eviction_manager.go:267] "eviction manager: failed to check if we have separate container filesystem. Ignoring." err="non-existent label \"crio-containers\""
Jul 01 16:58:50.241735 ip-10-0-137-244 kubenswrapper[2565]: E0701 16:58:50.187843    2565 eviction_manager.go:292] "Eviction manager: failed to get summary stats" err="failed to get node info: node \"ip-10-0-137-244.ec2.internal\" not found"
Jul 01 16:58:50.241735 ip-10-0-137-244 kubenswrapper[2565]: I0701 16:58:50.199169    2565 nodeinfomanager.go:417] Failed to publish CSINode: nodes "ip-10-0-137-244.ec2.internal" not found
Jul 01 16:58:50.242941 ip-10-0-137-244 kubenswrapper[2565]: I0701 16:58:50.242777    2565 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv4"
Jul 01 16:58:50.242941 ip-10-0-137-244 kubenswrapper[2565]: I0701 16:58:50.242817    2565 status_manager.go:244] "Starting to sync pod status with apiserver"
Jul 01 16:58:50.242941 ip-10-0-137-244 kubenswrapper[2565]: I0701 16:58:50.242836    2565 kubelet.go:2448] "Starting kubelet main sync loop"
Jul 01 16:58:50.242941 ip-10-0-137-244 kubenswrapper[2565]: E0701 16:58:50.242863    2565 kubelet.go:2472] "Skipping pod synchronization" err="PLEG is not healthy: pleg has yet to be successful"
Jul 01 16:58:50.351788 ip-10-0-137-244 kubenswrapper[2565]: I0701 16:58:50.246313    2565 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160"
Jul 01 16:58:50.351788 ip-10-0-137-244 kubenswrapper[2565]: I0701 16:58:50.287613    2565 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 01 16:58:50.351788 ip-10-0-137-244 kubenswrapper[2565]: I0701 16:58:50.288419    2565 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-137-244.ec2.internal" event="NodeHasSufficientMemory"
Jul 01 16:58:50.351788 ip-10-0-137-244 kubenswrapper[2565]: I0701 16:58:50.288447    2565 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-137-244.ec2.internal" event="NodeHasNoDiskPressure"
Jul 01 16:58:50.351788 ip-10-0-137-244 kubenswrapper[2565]: I0701 16:58:50.288460    2565 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-137-244.ec2.internal" event="NodeHasSufficientPID"
Jul 01 16:58:50.351788 ip-10-0-137-244 kubenswrapper[2565]: I0701 16:58:50.288482    2565 kubelet_node_status.go:78] "Attempting to register node" node="ip-10-0-137-244.ec2.internal"
Jul 01 16:58:50.351788 ip-10-0-137-244 kubenswrapper[2565]: I0701 16:58:50.301863    2565 kubelet_node_status.go:81] "Successfully registered node" node="ip-10-0-137-244.ec2.internal"
Jul 01 16:58:50.351788 ip-10-0-137-244 kubenswrapper[2565]: E0701 16:58:50.301879    2565 kubelet_node_status.go:536] "Error updating node status, will retry" err="error getting node \"ip-10-0-137-244.ec2.internal\": node \"ip-10-0-137-244.ec2.internal\" not found"
Jul 01 16:58:50.351788 ip-10-0-137-244 kubenswrapper[2565]: I0701 16:58:50.344034    2565 kubelet.go:2535] "SyncLoop ADD" source="file" pods=["openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-137-244.ec2.internal","kube-system/kube-apiserver-proxy-ip-10-0-137-244.ec2.internal"]
Jul 01 16:58:50.351788 ip-10-0-137-244 kubenswrapper[2565]: I0701 16:58:50.345351    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-137-244.ec2.internal"
Jul 01 16:58:50.351788 ip-10-0-137-244 kubenswrapper[2565]: I0701 16:58:50.346352    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-137-244.ec2.internal"
Jul 01 16:58:50.478756 ip-10-0-137-244 kubenswrapper[2565]: I0701 16:58:50.381797    2565 kubelet.go:3248] "Creating a mirror pod for static pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-137-244.ec2.internal"
Jul 01 16:58:50.478756 ip-10-0-137-244 kubenswrapper[2565]: I0701 16:58:50.386672    2565 kubelet.go:3248] "Creating a mirror pod for static pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-137-244.ec2.internal"
Jul 01 16:58:50.478756 ip-10-0-137-244 kubenswrapper[2565]: I0701 16:58:50.398884    2565 kubelet.go:3248] "Creating a mirror pod for static pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-137-244.ec2.internal"
Jul 01 16:58:50.478756 ip-10-0-137-244 kubenswrapper[2565]: I0701 16:58:50.400103    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/8ab5de924319ee4f32e67a5f2f6ee7fe-config\") pod \"kube-apiserver-proxy-ip-10-0-137-244.ec2.internal\" (UID: \"8ab5de924319ee4f32e67a5f2f6ee7fe\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-137-244.ec2.internal"
Jul 01 16:58:50.478756 ip-10-0-137-244 kubenswrapper[2565]: I0701 16:58:50.400134    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/d96f2d2f4c7e883a098f7adf30674021-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-137-244.ec2.internal\" (UID: \"d96f2d2f4c7e883a098f7adf30674021\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-137-244.ec2.internal"
Jul 01 16:58:50.478756 ip-10-0-137-244 kubenswrapper[2565]: I0701 16:58:50.400152    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/d96f2d2f4c7e883a098f7adf30674021-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-137-244.ec2.internal\" (UID: \"d96f2d2f4c7e883a098f7adf30674021\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-137-244.ec2.internal"
Jul 01 16:58:50.478756 ip-10-0-137-244 kubenswrapper[2565]: I0701 16:58:50.400735    2565 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]"
Jul 01 16:58:50.478756 ip-10-0-137-244 kubenswrapper[2565]: I0701 16:58:50.408723    2565 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]"
Jul 01 16:58:50.478756 ip-10-0-137-244 kubenswrapper[2565]: I0701 16:58:50.431703    2565 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]"
Jul 01 16:58:50.478756 ip-10-0-137-244 kubenswrapper[2565]: E0701 16:58:50.431737    2565 kubelet.go:3250] "Failed creating a mirror pod" err="pods \"kube-rbac-proxy-crio-ip-10-0-137-244.ec2.internal\" already exists" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-137-244.ec2.internal"
Jul 01 16:58:50.478756 ip-10-0-137-244 kubenswrapper[2565]: I0701 16:58:50.431748    2565 kubelet.go:3248] "Creating a mirror pod for static pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-137-244.ec2.internal"
Jul 01 16:58:50.478756 ip-10-0-137-244 kubenswrapper[2565]: I0701 16:58:50.448135    2565 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]"
Jul 01 16:58:50.478756 ip-10-0-137-244 kubenswrapper[2565]: E0701 16:58:50.448164    2565 kubelet.go:3250] "Failed creating a mirror pod" err="pods \"kube-apiserver-proxy-ip-10-0-137-244.ec2.internal\" already exists" pod="kube-system/kube-apiserver-proxy-ip-10-0-137-244.ec2.internal"
Jul 01 16:58:50.501131 ip-10-0-137-244 kubenswrapper[2565]: I0701 16:58:50.501109    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/d96f2d2f4c7e883a098f7adf30674021-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-137-244.ec2.internal\" (UID: \"d96f2d2f4c7e883a098f7adf30674021\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-137-244.ec2.internal"
Jul 01 16:58:50.501213 ip-10-0-137-244 kubenswrapper[2565]: I0701 16:58:50.501136    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/d96f2d2f4c7e883a098f7adf30674021-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-137-244.ec2.internal\" (UID: \"d96f2d2f4c7e883a098f7adf30674021\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-137-244.ec2.internal"
Jul 01 16:58:50.501213 ip-10-0-137-244 kubenswrapper[2565]: I0701 16:58:50.501153    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/8ab5de924319ee4f32e67a5f2f6ee7fe-config\") pod \"kube-apiserver-proxy-ip-10-0-137-244.ec2.internal\" (UID: \"8ab5de924319ee4f32e67a5f2f6ee7fe\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-137-244.ec2.internal"
Jul 01 16:58:50.501213 ip-10-0-137-244 kubenswrapper[2565]: I0701 16:58:50.501205    2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/d96f2d2f4c7e883a098f7adf30674021-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-137-244.ec2.internal\" (UID: \"d96f2d2f4c7e883a098f7adf30674021\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-137-244.ec2.internal"
Jul 01 16:58:50.501331 ip-10-0-137-244 kubenswrapper[2565]: I0701 16:58:50.501218    2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/d96f2d2f4c7e883a098f7adf30674021-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-137-244.ec2.internal\" (UID: \"d96f2d2f4c7e883a098f7adf30674021\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-137-244.ec2.internal"
Jul 01 16:58:50.501331 ip-10-0-137-244 kubenswrapper[2565]: I0701 16:58:50.501205    2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/host-path/8ab5de924319ee4f32e67a5f2f6ee7fe-config\") pod \"kube-apiserver-proxy-ip-10-0-137-244.ec2.internal\" (UID: \"8ab5de924319ee4f32e67a5f2f6ee7fe\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-137-244.ec2.internal"
Jul 01 16:58:50.703874 ip-10-0-137-244 kubenswrapper[2565]: I0701 16:58:50.703845    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-137-244.ec2.internal"
Jul 01 16:58:50.716120 ip-10-0-137-244 kubenswrapper[2565]: I0701 16:58:50.716102    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-137-244.ec2.internal"
Jul 01 16:58:50.867914 ip-10-0-137-244 kubenswrapper[2565]: I0701 16:58:50.867861    2565 transport.go:147] "Certificate rotation detected, shutting down client connections to start using new credentials"
Jul 01 16:58:50.868008 ip-10-0-137-244 kubenswrapper[2565]: I0701 16:58:50.867980    2565 reflector.go:571] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.CSIDriver" err="very short watch: k8s.io/client-go/informers/factory.go:160: Unexpected watch close - watch lasted less than a second and no items received"
Jul 01 16:58:50.868092 ip-10-0-137-244 kubenswrapper[2565]: I0701 16:58:50.868000    2565 reflector.go:571] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.RuntimeClass" err="very short watch: k8s.io/client-go/informers/factory.go:160: Unexpected watch close - watch lasted less than a second and no items received"
Jul 01 16:58:50.868092 ip-10-0-137-244 kubenswrapper[2565]: I0701 16:58:50.867998    2565 reflector.go:571] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Service" err="very short watch: k8s.io/client-go/informers/factory.go:160: Unexpected watch close - watch lasted less than a second and no items received"
Jul 01 16:58:51.000593 ip-10-0-137-244 kubenswrapper[2565]: I0701 16:58:51.000529    2565 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kube-apiserver-client-kubelet" expiration="2028-06-30 16:53:50 +0000 UTC" deadline="2028-02-14 22:45:43.732330993 +0000 UTC"
Jul 01 16:58:51.000593 ip-10-0-137-244 kubenswrapper[2565]: I0701 16:58:51.000585    2565 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kube-apiserver-client-kubelet" sleep="14237h46m52.731750008s"
Jul 01 16:58:51.063439 ip-10-0-137-244 kubenswrapper[2565]: I0701 16:58:51.063411    2565 apiserver.go:52] "Watching apiserver"
Jul 01 16:58:51.072410 ip-10-0-137-244 kubenswrapper[2565]: I0701 16:58:51.072387    2565 reflector.go:439] "Caches populated" type="*v1.Pod" reflector="pkg/kubelet/config/apiserver.go:66"
Jul 01 16:58:51.073623 ip-10-0-137-244 kubenswrapper[2565]: I0701 16:58:51.073588    2565 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-multus/multus-nlxxc","openshift-network-diagnostics/network-check-target-fvqs4","openshift-network-operator/iptables-alerter-r59mr","openshift-ovn-kubernetes/ovnkube-node-zjsw4","kube-system/global-pull-secret-syncer-vfng9","kube-system/kube-apiserver-proxy-ip-10-0-137-244.ec2.internal","openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-rh87s","openshift-cluster-node-tuning-operator/tuned-8j5xx","openshift-dns/node-resolver-mth9w","openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-137-244.ec2.internal","openshift-multus/multus-additional-cni-plugins-kzwpl","openshift-multus/network-metrics-daemon-hthvc","kube-system/konnectivity-agent-w6h6d","openshift-image-registry/node-ca-g7t25"]
Jul 01 16:58:51.076070 ip-10-0-137-244 kubenswrapper[2565]: I0701 16:58:51.076043    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-nlxxc"
Jul 01 16:58:51.077504 ip-10-0-137-244 kubenswrapper[2565]: I0701 16:58:51.077484    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-fvqs4"
Jul 01 16:58:51.077606 ip-10-0-137-244 kubenswrapper[2565]: E0701 16:58:51.077565    2565 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-fvqs4" podUID="43a7325b-d5b7-492e-961d-52f0b9b32a17"
Jul 01 16:58:51.078603 ip-10-0-137-244 kubenswrapper[2565]: I0701 16:58:51.078587    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-r59mr"
Jul 01 16:58:51.079875 ip-10-0-137-244 kubenswrapper[2565]: I0701 16:58:51.079858    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-zjsw4"
Jul 01 16:58:51.081194 ip-10-0-137-244 kubenswrapper[2565]: I0701 16:58:51.081175    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-vfng9"
Jul 01 16:58:51.081303 ip-10-0-137-244 kubenswrapper[2565]: E0701 16:58:51.081254    2565 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-vfng9" podUID="b2746391-676c-4add-9406-a60838c1ac2c"
Jul 01 16:58:51.081646 ip-10-0-137-244 kubenswrapper[2565]: I0701 16:58:51.081625    2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"multus-daemon-config\""
Jul 01 16:58:51.081719 ip-10-0-137-244 kubenswrapper[2565]: I0701 16:58:51.081675    2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"cni-copy-resources\""
Jul 01 16:58:51.081878 ip-10-0-137-244 kubenswrapper[2565]: I0701 16:58:51.081861    2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"default-dockercfg-xl6r9\""
Jul 01 16:58:51.082572 ip-10-0-137-244 kubenswrapper[2565]: I0701 16:58:51.082557    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-rh87s"
Jul 01 16:58:51.083817 ip-10-0-137-244 kubenswrapper[2565]: I0701 16:58:51.083798    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-8j5xx"
Jul 01 16:58:51.085203 ip-10-0-137-244 kubenswrapper[2565]: I0701 16:58:51.085179    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-mth9w"
Jul 01 16:58:51.086570 ip-10-0-137-244 kubenswrapper[2565]: I0701 16:58:51.086546    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-kzwpl"
Jul 01 16:58:51.087923 ip-10-0-137-244 kubenswrapper[2565]: I0701 16:58:51.087903    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-hthvc"
Jul 01 16:58:51.088031 ip-10-0-137-244 kubenswrapper[2565]: E0701 16:58:51.087961    2565 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-hthvc" podUID="a6488c7c-9ea5-4ad2-91ba-b1ffda18b203"
Jul 01 16:58:51.088244 ip-10-0-137-244 kubenswrapper[2565]: I0701 16:58:51.088219    2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"tuned-dockercfg-56v98\""
Jul 01 16:58:51.088244 ip-10-0-137-244 kubenswrapper[2565]: I0701 16:58:51.088245    2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"openshift-service-ca.crt\""
Jul 01 16:58:51.088396 ip-10-0-137-244 kubenswrapper[2565]: I0701 16:58:51.088245    2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"kube-root-ca.crt\""
Jul 01 16:58:51.089363 ip-10-0-137-244 kubenswrapper[2565]: I0701 16:58:51.089347    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-w6h6d"
Jul 01 16:58:51.091005 ip-10-0-137-244 kubenswrapper[2565]: I0701 16:58:51.090985    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-g7t25"
Jul 01 16:58:51.097250 ip-10-0-137-244 kubenswrapper[2565]: I0701 16:58:51.097225    2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"openshift-service-ca.crt\""
Jul 01 16:58:51.097363 ip-10-0-137-244 kubenswrapper[2565]: I0701 16:58:51.097249    2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"kube-root-ca.crt\""
Jul 01 16:58:51.097363 ip-10-0-137-244 kubenswrapper[2565]: I0701 16:58:51.097282    2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"kube-root-ca.crt\""
Jul 01 16:58:51.097363 ip-10-0-137-244 kubenswrapper[2565]: I0701 16:58:51.097328    2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"openshift-service-ca.crt\""
Jul 01 16:58:51.097504 ip-10-0-137-244 kubenswrapper[2565]: I0701 16:58:51.097396    2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-node-metrics-cert\""
Jul 01 16:58:51.097504 ip-10-0-137-244 kubenswrapper[2565]: I0701 16:58:51.097410    2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"env-overrides\""
Jul 01 16:58:51.097686 ip-10-0-137-244 kubenswrapper[2565]: I0701 16:58:51.097669    2565 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kubelet-serving"
Jul 01 16:58:51.097786 ip-10-0-137-244 kubenswrapper[2565]: I0701 16:58:51.097768    2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"openshift-service-ca.crt\""
Jul 01 16:58:51.097841 ip-10-0-137-244 kubenswrapper[2565]: I0701 16:58:51.097810    2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"kube-root-ca.crt\""
Jul 01 16:58:51.097841 ip-10-0-137-244 kubenswrapper[2565]: I0701 16:58:51.097822    2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"node-resolver-dockercfg-dhvvb\""
Jul 01 16:58:51.097841 ip-10-0-137-244 kubenswrapper[2565]: I0701 16:58:51.097838    2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-script-lib\""
Jul 01 16:58:51.097983 ip-10-0-137-244 kubenswrapper[2565]: I0701 16:58:51.097807    2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"openshift-service-ca.crt\""
Jul 01 16:58:51.097983 ip-10-0-137-244 kubenswrapper[2565]: I0701 16:58:51.097827    2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-dockercfg-nln7q\""
Jul 01 16:58:51.097983 ip-10-0-137-244 kubenswrapper[2565]: I0701 16:58:51.097954    2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"default-dockercfg-cdpwf\""
Jul 01 16:58:51.097983 ip-10-0-137-244 kubenswrapper[2565]: I0701 16:58:51.097981    2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-config\""
Jul 01 16:58:51.098186 ip-10-0-137-244 kubenswrapper[2565]: I0701 16:58:51.098144    2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"whereabouts-flatfile-config\""
Jul 01 16:58:51.098248 ip-10-0-137-244 kubenswrapper[2565]: I0701 16:58:51.098232    2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"kube-root-ca.crt\""
Jul 01 16:58:51.098300 ip-10-0-137-244 kubenswrapper[2565]: I0701 16:58:51.098269    2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"kube-root-ca.crt\""
Jul 01 16:58:51.098300 ip-10-0-137-244 kubenswrapper[2565]: I0701 16:58:51.098271    2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"image-registry-certificates\""
Jul 01 16:58:51.098470 ip-10-0-137-244 kubenswrapper[2565]: I0701 16:58:51.098232    2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"kube-root-ca.crt\""
Jul 01 16:58:51.098470 ip-10-0-137-244 kubenswrapper[2565]: I0701 16:58:51.098401    2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-kubernetes-node-dockercfg-xxtgc\""
Jul 01 16:58:51.098593 ip-10-0-137-244 kubenswrapper[2565]: I0701 16:58:51.098480    2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"node-ca-dockercfg-wqzxw\""
Jul 01 16:58:51.098593 ip-10-0-137-244 kubenswrapper[2565]: I0701 16:58:51.098503    2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"konnectivity-agent\""
Jul 01 16:58:51.098593 ip-10-0-137-244 kubenswrapper[2565]: I0701 16:58:51.098531    2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kube-system\"/\"konnectivity-ca-bundle\""
Jul 01 16:58:51.098750 ip-10-0-137-244 kubenswrapper[2565]: I0701 16:58:51.098603    2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"openshift-service-ca.crt\""
Jul 01 16:58:51.098750 ip-10-0-137-244 kubenswrapper[2565]: I0701 16:58:51.098722    2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"multus-ancillary-tools-dockercfg-mf8h7\""
Jul 01 16:58:51.098860 ip-10-0-137-244 kubenswrapper[2565]: I0701 16:58:51.098766    2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-script\""
Jul 01 16:58:51.098860 ip-10-0-137-244 kubenswrapper[2565]: I0701 16:58:51.098830    2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"default-cni-sysctl-allowlist\""
Jul 01 16:58:51.098973 ip-10-0-137-244 kubenswrapper[2565]: I0701 16:58:51.098868    2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-metrics-serving-cert\""
Jul 01 16:58:51.099072 ip-10-0-137-244 kubenswrapper[2565]: I0701 16:58:51.099052    2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-sa-dockercfg-kp2v9\""
Jul 01 16:58:51.099146 ip-10-0-137-244 kubenswrapper[2565]: I0701 16:58:51.099096    2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"openshift-service-ca.crt\""
Jul 01 16:58:51.099763 ip-10-0-137-244 kubenswrapper[2565]: I0701 16:58:51.099747    2565 desired_state_of_world_populator.go:154] "Finished populating initial desired state of world"
Jul 01 16:58:51.105171 ip-10-0-137-244 kubenswrapper[2565]: I0701 16:58:51.105140    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/33c916f6-76ab-4a81-9955-3ffae48a52f8-var-lib-kubelet\") pod \"tuned-8j5xx\" (UID: \"33c916f6-76ab-4a81-9955-3ffae48a52f8\") " pod="openshift-cluster-node-tuning-operator/tuned-8j5xx"
Jul 01 16:58:51.105276 ip-10-0-137-244 kubenswrapper[2565]: I0701 16:58:51.105178    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/bb7ebc32-d0cd-4eb0-b998-3ce298f73809-etc-openvswitch\") pod \"ovnkube-node-zjsw4\" (UID: \"bb7ebc32-d0cd-4eb0-b998-3ce298f73809\") " pod="openshift-ovn-kubernetes/ovnkube-node-zjsw4"
Jul 01 16:58:51.105276 ip-10-0-137-244 kubenswrapper[2565]: I0701 16:58:51.105204    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/c564aa0e-7841-4b52-8262-4ee37c496c54-device-dir\") pod \"aws-ebs-csi-driver-node-rh87s\" (UID: \"c564aa0e-7841-4b52-8262-4ee37c496c54\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-rh87s"
Jul 01 16:58:51.105276 ip-10-0-137-244 kubenswrapper[2565]: I0701 16:58:51.105228    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vzm9j\" (UniqueName: \"kubernetes.io/projected/c564aa0e-7841-4b52-8262-4ee37c496c54-kube-api-access-vzm9j\") pod \"aws-ebs-csi-driver-node-rh87s\" (UID: \"c564aa0e-7841-4b52-8262-4ee37c496c54\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-rh87s"
Jul 01 16:58:51.105417 ip-10-0-137-244 kubenswrapper[2565]: I0701 16:58:51.105266    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/6040517d-fe23-442e-968b-b58733d448a0-host-run-multus-certs\") pod \"multus-nlxxc\" (UID: \"6040517d-fe23-442e-968b-b58733d448a0\") " pod="openshift-multus/multus-nlxxc"
Jul 01 16:58:51.106054 ip-10-0-137-244 kubenswrapper[2565]: I0701 16:58:51.105620    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/b878d4e6-fe3b-4c52-aaea-de5a58d48afc-host\") pod \"node-ca-g7t25\" (UID: \"b878d4e6-fe3b-4c52-aaea-de5a58d48afc\") " pod="openshift-image-registry/node-ca-g7t25"
Jul 01 16:58:51.106339 ip-10-0-137-244 kubenswrapper[2565]: I0701 16:58:51.105669    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/bb7ebc32-d0cd-4eb0-b998-3ce298f73809-log-socket\") pod \"ovnkube-node-zjsw4\" (UID: \"bb7ebc32-d0cd-4eb0-b998-3ce298f73809\") " pod="openshift-ovn-kubernetes/ovnkube-node-zjsw4"
Jul 01 16:58:51.106423 ip-10-0-137-244 kubenswrapper[2565]: I0701 16:58:51.106384    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/bb7ebc32-d0cd-4eb0-b998-3ce298f73809-ovnkube-config\") pod \"ovnkube-node-zjsw4\" (UID: \"bb7ebc32-d0cd-4eb0-b998-3ce298f73809\") " pod="openshift-ovn-kubernetes/ovnkube-node-zjsw4"
Jul 01 16:58:51.106475 ip-10-0-137-244 kubenswrapper[2565]: I0701 16:58:51.106442    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/33c916f6-76ab-4a81-9955-3ffae48a52f8-etc-sysctl-d\") pod \"tuned-8j5xx\" (UID: \"33c916f6-76ab-4a81-9955-3ffae48a52f8\") " pod="openshift-cluster-node-tuning-operator/tuned-8j5xx"
Jul 01 16:58:51.106528 ip-10-0-137-244 kubenswrapper[2565]: I0701 16:58:51.106495    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gwhf8\" (UniqueName: \"kubernetes.io/projected/16665992-76de-4ba6-b887-abaadbc31ae6-kube-api-access-gwhf8\") pod \"node-resolver-mth9w\" (UID: \"16665992-76de-4ba6-b887-abaadbc31ae6\") " pod="openshift-dns/node-resolver-mth9w"
Jul 01 16:58:51.106577 ip-10-0-137-244 kubenswrapper[2565]: I0701 16:58:51.106530    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/bffd2a49-ec4f-44ee-a47c-19180aaeeec2-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-kzwpl\" (UID: \"bffd2a49-ec4f-44ee-a47c-19180aaeeec2\") " pod="openshift-multus/multus-additional-cni-plugins-kzwpl"
Jul 01 16:58:51.106577 ip-10-0-137-244 kubenswrapper[2565]: I0701 16:58:51.106568    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-m4gcz\" (UniqueName: \"kubernetes.io/projected/33c916f6-76ab-4a81-9955-3ffae48a52f8-kube-api-access-m4gcz\") pod \"tuned-8j5xx\" (UID: \"33c916f6-76ab-4a81-9955-3ffae48a52f8\") " pod="openshift-cluster-node-tuning-operator/tuned-8j5xx"
Jul 01 16:58:51.106664 ip-10-0-137-244 kubenswrapper[2565]: I0701 16:58:51.106593    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/6040517d-fe23-442e-968b-b58733d448a0-multus-socket-dir-parent\") pod \"multus-nlxxc\" (UID: \"6040517d-fe23-442e-968b-b58733d448a0\") " pod="openshift-multus/multus-nlxxc"
Jul 01 16:58:51.106664 ip-10-0-137-244 kubenswrapper[2565]: I0701 16:58:51.106625    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/6040517d-fe23-442e-968b-b58733d448a0-etc-kubernetes\") pod \"multus-nlxxc\" (UID: \"6040517d-fe23-442e-968b-b58733d448a0\") " pod="openshift-multus/multus-nlxxc"
Jul 01 16:58:51.106755 ip-10-0-137-244 kubenswrapper[2565]: I0701 16:58:51.106664    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/bffd2a49-ec4f-44ee-a47c-19180aaeeec2-os-release\") pod \"multus-additional-cni-plugins-kzwpl\" (UID: \"bffd2a49-ec4f-44ee-a47c-19180aaeeec2\") " pod="openshift-multus/multus-additional-cni-plugins-kzwpl"
Jul 01 16:58:51.106755 ip-10-0-137-244 kubenswrapper[2565]: I0701 16:58:51.106699    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/bb7ebc32-d0cd-4eb0-b998-3ce298f73809-ovnkube-script-lib\") pod \"ovnkube-node-zjsw4\" (UID: \"bb7ebc32-d0cd-4eb0-b998-3ce298f73809\") " pod="openshift-ovn-kubernetes/ovnkube-node-zjsw4"
Jul 01 16:58:51.106755 ip-10-0-137-244 kubenswrapper[2565]: I0701 16:58:51.106730    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/33c916f6-76ab-4a81-9955-3ffae48a52f8-etc-tuned\") pod \"tuned-8j5xx\" (UID: \"33c916f6-76ab-4a81-9955-3ffae48a52f8\") " pod="openshift-cluster-node-tuning-operator/tuned-8j5xx"
Jul 01 16:58:51.106887 ip-10-0-137-244 kubenswrapper[2565]: I0701 16:58:51.106762    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/6040517d-fe23-442e-968b-b58733d448a0-host-run-k8s-cni-cncf-io\") pod \"multus-nlxxc\" (UID: \"6040517d-fe23-442e-968b-b58733d448a0\") " pod="openshift-multus/multus-nlxxc"
Jul 01 16:58:51.106887 ip-10-0-137-244 kubenswrapper[2565]: I0701 16:58:51.106809    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/6040517d-fe23-442e-968b-b58733d448a0-host-var-lib-cni-bin\") pod \"multus-nlxxc\" (UID: \"6040517d-fe23-442e-968b-b58733d448a0\") " pod="openshift-multus/multus-nlxxc"
Jul 01 16:58:51.106887 ip-10-0-137-244 kubenswrapper[2565]: I0701 16:58:51.106844    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/16665992-76de-4ba6-b887-abaadbc31ae6-tmp-dir\") pod \"node-resolver-mth9w\" (UID: \"16665992-76de-4ba6-b887-abaadbc31ae6\") " pod="openshift-dns/node-resolver-mth9w"
Jul 01 16:58:51.106887 ip-10-0-137-244 kubenswrapper[2565]: I0701 16:58:51.106874    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/bb7ebc32-d0cd-4eb0-b998-3ce298f73809-node-log\") pod \"ovnkube-node-zjsw4\" (UID: \"bb7ebc32-d0cd-4eb0-b998-3ce298f73809\") " pod="openshift-ovn-kubernetes/ovnkube-node-zjsw4"
Jul 01 16:58:51.107071 ip-10-0-137-244 kubenswrapper[2565]: I0701 16:58:51.106902    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/33c916f6-76ab-4a81-9955-3ffae48a52f8-run\") pod \"tuned-8j5xx\" (UID: \"33c916f6-76ab-4a81-9955-3ffae48a52f8\") " pod="openshift-cluster-node-tuning-operator/tuned-8j5xx"
Jul 01 16:58:51.107071 ip-10-0-137-244 kubenswrapper[2565]: I0701 16:58:51.106929    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/6040517d-fe23-442e-968b-b58733d448a0-multus-cni-dir\") pod \"multus-nlxxc\" (UID: \"6040517d-fe23-442e-968b-b58733d448a0\") " pod="openshift-multus/multus-nlxxc"
Jul 01 16:58:51.107071 ip-10-0-137-244 kubenswrapper[2565]: I0701 16:58:51.106976    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/6040517d-fe23-442e-968b-b58733d448a0-host-var-lib-cni-multus\") pod \"multus-nlxxc\" (UID: \"6040517d-fe23-442e-968b-b58733d448a0\") " pod="openshift-multus/multus-nlxxc"
Jul 01 16:58:51.107071 ip-10-0-137-244 kubenswrapper[2565]: I0701 16:58:51.107010    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/6040517d-fe23-442e-968b-b58733d448a0-multus-daemon-config\") pod \"multus-nlxxc\" (UID: \"6040517d-fe23-442e-968b-b58733d448a0\") " pod="openshift-multus/multus-nlxxc"
Jul 01 16:58:51.107071 ip-10-0-137-244 kubenswrapper[2565]: I0701 16:58:51.107040    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rsprp\" (UniqueName: \"kubernetes.io/projected/6040517d-fe23-442e-968b-b58733d448a0-kube-api-access-rsprp\") pod \"multus-nlxxc\" (UID: \"6040517d-fe23-442e-968b-b58733d448a0\") " pod="openshift-multus/multus-nlxxc"
Jul 01 16:58:51.107334 ip-10-0-137-244 kubenswrapper[2565]: I0701 16:58:51.107090    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/bffd2a49-ec4f-44ee-a47c-19180aaeeec2-cnibin\") pod \"multus-additional-cni-plugins-kzwpl\" (UID: \"bffd2a49-ec4f-44ee-a47c-19180aaeeec2\") " pod="openshift-multus/multus-additional-cni-plugins-kzwpl"
Jul 01 16:58:51.107334 ip-10-0-137-244 kubenswrapper[2565]: I0701 16:58:51.107155    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-c9p7h\" (UniqueName: \"kubernetes.io/projected/121518fd-3292-4547-bc31-921e4db77e54-kube-api-access-c9p7h\") pod \"iptables-alerter-r59mr\" (UID: \"121518fd-3292-4547-bc31-921e4db77e54\") " pod="openshift-network-operator/iptables-alerter-r59mr"
Jul 01 16:58:51.107334 ip-10-0-137-244 kubenswrapper[2565]: I0701 16:58:51.107181    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/bb7ebc32-d0cd-4eb0-b998-3ce298f73809-host-kubelet\") pod \"ovnkube-node-zjsw4\" (UID: \"bb7ebc32-d0cd-4eb0-b998-3ce298f73809\") " pod="openshift-ovn-kubernetes/ovnkube-node-zjsw4"
Jul 01 16:58:51.107334 ip-10-0-137-244 kubenswrapper[2565]: I0701 16:58:51.107209    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/bb7ebc32-d0cd-4eb0-b998-3ce298f73809-host-slash\") pod \"ovnkube-node-zjsw4\" (UID: \"bb7ebc32-d0cd-4eb0-b998-3ce298f73809\") " pod="openshift-ovn-kubernetes/ovnkube-node-zjsw4"
Jul 01 16:58:51.107334 ip-10-0-137-244 kubenswrapper[2565]: I0701 16:58:51.107241    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/c564aa0e-7841-4b52-8262-4ee37c496c54-etc-selinux\") pod \"aws-ebs-csi-driver-node-rh87s\" (UID: \"c564aa0e-7841-4b52-8262-4ee37c496c54\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-rh87s"
Jul 01 16:58:51.107334 ip-10-0-137-244 kubenswrapper[2565]: I0701 16:58:51.107270    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/6040517d-fe23-442e-968b-b58733d448a0-multus-conf-dir\") pod \"multus-nlxxc\" (UID: \"6040517d-fe23-442e-968b-b58733d448a0\") " pod="openshift-multus/multus-nlxxc"
Jul 01 16:58:51.107334 ip-10-0-137-244 kubenswrapper[2565]: I0701 16:58:51.107303    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/bffd2a49-ec4f-44ee-a47c-19180aaeeec2-tuning-conf-dir\") pod \"multus-additional-cni-plugins-kzwpl\" (UID: \"bffd2a49-ec4f-44ee-a47c-19180aaeeec2\") " pod="openshift-multus/multus-additional-cni-plugins-kzwpl"
Jul 01 16:58:51.107334 ip-10-0-137-244 kubenswrapper[2565]: I0701 16:58:51.107330    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9rt5m\" (UniqueName: \"kubernetes.io/projected/43a7325b-d5b7-492e-961d-52f0b9b32a17-kube-api-access-9rt5m\") pod \"network-check-target-fvqs4\" (UID: \"43a7325b-d5b7-492e-961d-52f0b9b32a17\") " pod="openshift-network-diagnostics/network-check-target-fvqs4"
Jul 01 16:58:51.107689 ip-10-0-137-244 kubenswrapper[2565]: I0701 16:58:51.107361    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/bb7ebc32-d0cd-4eb0-b998-3ce298f73809-var-lib-openvswitch\") pod \"ovnkube-node-zjsw4\" (UID: \"bb7ebc32-d0cd-4eb0-b998-3ce298f73809\") " pod="openshift-ovn-kubernetes/ovnkube-node-zjsw4"
Jul 01 16:58:51.107689 ip-10-0-137-244 kubenswrapper[2565]: I0701 16:58:51.107393    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-r4xhm\" (UniqueName: \"kubernetes.io/projected/bb7ebc32-d0cd-4eb0-b998-3ce298f73809-kube-api-access-r4xhm\") pod \"ovnkube-node-zjsw4\" (UID: \"bb7ebc32-d0cd-4eb0-b998-3ce298f73809\") " pod="openshift-ovn-kubernetes/ovnkube-node-zjsw4"
Jul 01 16:58:51.107689 ip-10-0-137-244 kubenswrapper[2565]: I0701 16:58:51.107424    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/b2746391-676c-4add-9406-a60838c1ac2c-dbus\") pod \"global-pull-secret-syncer-vfng9\" (UID: \"b2746391-676c-4add-9406-a60838c1ac2c\") " pod="kube-system/global-pull-secret-syncer-vfng9"
Jul 01 16:58:51.107689 ip-10-0-137-244 kubenswrapper[2565]: I0701 16:58:51.107451    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/c564aa0e-7841-4b52-8262-4ee37c496c54-kubelet-dir\") pod \"aws-ebs-csi-driver-node-rh87s\" (UID: \"c564aa0e-7841-4b52-8262-4ee37c496c54\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-rh87s"
Jul 01 16:58:51.107689 ip-10-0-137-244 kubenswrapper[2565]: I0701 16:58:51.107482    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/c564aa0e-7841-4b52-8262-4ee37c496c54-sys-fs\") pod \"aws-ebs-csi-driver-node-rh87s\" (UID: \"c564aa0e-7841-4b52-8262-4ee37c496c54\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-rh87s"
Jul 01 16:58:51.107689 ip-10-0-137-244 kubenswrapper[2565]: I0701 16:58:51.107512    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/33c916f6-76ab-4a81-9955-3ffae48a52f8-etc-modprobe-d\") pod \"tuned-8j5xx\" (UID: \"33c916f6-76ab-4a81-9955-3ffae48a52f8\") " pod="openshift-cluster-node-tuning-operator/tuned-8j5xx"
Jul 01 16:58:51.107689 ip-10-0-137-244 kubenswrapper[2565]: I0701 16:58:51.107548    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/33c916f6-76ab-4a81-9955-3ffae48a52f8-etc-systemd\") pod \"tuned-8j5xx\" (UID: \"33c916f6-76ab-4a81-9955-3ffae48a52f8\") " pod="openshift-cluster-node-tuning-operator/tuned-8j5xx"
Jul 01 16:58:51.107689 ip-10-0-137-244 kubenswrapper[2565]: I0701 16:58:51.107583    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/6040517d-fe23-442e-968b-b58733d448a0-host-var-lib-kubelet\") pod \"multus-nlxxc\" (UID: \"6040517d-fe23-442e-968b-b58733d448a0\") " pod="openshift-multus/multus-nlxxc"
Jul 01 16:58:51.107689 ip-10-0-137-244 kubenswrapper[2565]: I0701 16:58:51.107615    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/bb7ebc32-d0cd-4eb0-b998-3ce298f73809-run-openvswitch\") pod \"ovnkube-node-zjsw4\" (UID: \"bb7ebc32-d0cd-4eb0-b998-3ce298f73809\") " pod="openshift-ovn-kubernetes/ovnkube-node-zjsw4"
Jul 01 16:58:51.107689 ip-10-0-137-244 kubenswrapper[2565]: I0701 16:58:51.107646    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/bb7ebc32-d0cd-4eb0-b998-3ce298f73809-host-run-ovn-kubernetes\") pod \"ovnkube-node-zjsw4\" (UID: \"bb7ebc32-d0cd-4eb0-b998-3ce298f73809\") " pod="openshift-ovn-kubernetes/ovnkube-node-zjsw4"
Jul 01 16:58:51.107689 ip-10-0-137-244 kubenswrapper[2565]: I0701 16:58:51.107672    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/b2746391-676c-4add-9406-a60838c1ac2c-kubelet-config\") pod \"global-pull-secret-syncer-vfng9\" (UID: \"b2746391-676c-4add-9406-a60838c1ac2c\") " pod="kube-system/global-pull-secret-syncer-vfng9"
Jul 01 16:58:51.108283 ip-10-0-137-244 kubenswrapper[2565]: I0701 16:58:51.107707    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/c564aa0e-7841-4b52-8262-4ee37c496c54-registration-dir\") pod \"aws-ebs-csi-driver-node-rh87s\" (UID: \"c564aa0e-7841-4b52-8262-4ee37c496c54\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-rh87s"
Jul 01 16:58:51.108283 ip-10-0-137-244 kubenswrapper[2565]: I0701 16:58:51.107740    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/33c916f6-76ab-4a81-9955-3ffae48a52f8-tmp\") pod \"tuned-8j5xx\" (UID: \"33c916f6-76ab-4a81-9955-3ffae48a52f8\") " pod="openshift-cluster-node-tuning-operator/tuned-8j5xx"
Jul 01 16:58:51.108283 ip-10-0-137-244 kubenswrapper[2565]: I0701 16:58:51.107772    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/6040517d-fe23-442e-968b-b58733d448a0-system-cni-dir\") pod \"multus-nlxxc\" (UID: \"6040517d-fe23-442e-968b-b58733d448a0\") " pod="openshift-multus/multus-nlxxc"
Jul 01 16:58:51.108283 ip-10-0-137-244 kubenswrapper[2565]: I0701 16:58:51.107805    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/16665992-76de-4ba6-b887-abaadbc31ae6-hosts-file\") pod \"node-resolver-mth9w\" (UID: \"16665992-76de-4ba6-b887-abaadbc31ae6\") " pod="openshift-dns/node-resolver-mth9w"
Jul 01 16:58:51.108283 ip-10-0-137-244 kubenswrapper[2565]: I0701 16:58:51.107831    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-29ccc\" (UniqueName: \"kubernetes.io/projected/a6488c7c-9ea5-4ad2-91ba-b1ffda18b203-kube-api-access-29ccc\") pod \"network-metrics-daemon-hthvc\" (UID: \"a6488c7c-9ea5-4ad2-91ba-b1ffda18b203\") " pod="openshift-multus/network-metrics-daemon-hthvc"
Jul 01 16:58:51.108283 ip-10-0-137-244 kubenswrapper[2565]: I0701 16:58:51.107864    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/bffd2a49-ec4f-44ee-a47c-19180aaeeec2-cni-binary-copy\") pod \"multus-additional-cni-plugins-kzwpl\" (UID: \"bffd2a49-ec4f-44ee-a47c-19180aaeeec2\") " pod="openshift-multus/multus-additional-cni-plugins-kzwpl"
Jul 01 16:58:51.108283 ip-10-0-137-244 kubenswrapper[2565]: I0701 16:58:51.107896    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/adf23d12-6f62-40f2-9ca0-026c878bfe07-konnectivity-ca\") pod \"konnectivity-agent-w6h6d\" (UID: \"adf23d12-6f62-40f2-9ca0-026c878bfe07\") " pod="kube-system/konnectivity-agent-w6h6d"
Jul 01 16:58:51.108283 ip-10-0-137-244 kubenswrapper[2565]: I0701 16:58:51.107929    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/bb7ebc32-d0cd-4eb0-b998-3ce298f73809-run-systemd\") pod \"ovnkube-node-zjsw4\" (UID: \"bb7ebc32-d0cd-4eb0-b998-3ce298f73809\") " pod="openshift-ovn-kubernetes/ovnkube-node-zjsw4"
Jul 01 16:58:51.108283 ip-10-0-137-244 kubenswrapper[2565]: I0701 16:58:51.107954    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/bb7ebc32-d0cd-4eb0-b998-3ce298f73809-run-ovn\") pod \"ovnkube-node-zjsw4\" (UID: \"bb7ebc32-d0cd-4eb0-b998-3ce298f73809\") " pod="openshift-ovn-kubernetes/ovnkube-node-zjsw4"
Jul 01 16:58:51.108283 ip-10-0-137-244 kubenswrapper[2565]: I0701 16:58:51.108006    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/bb7ebc32-d0cd-4eb0-b998-3ce298f73809-host-cni-netd\") pod \"ovnkube-node-zjsw4\" (UID: \"bb7ebc32-d0cd-4eb0-b998-3ce298f73809\") " pod="openshift-ovn-kubernetes/ovnkube-node-zjsw4"
Jul 01 16:58:51.108283 ip-10-0-137-244 kubenswrapper[2565]: I0701 16:58:51.108035    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/33c916f6-76ab-4a81-9955-3ffae48a52f8-host\") pod \"tuned-8j5xx\" (UID: \"33c916f6-76ab-4a81-9955-3ffae48a52f8\") " pod="openshift-cluster-node-tuning-operator/tuned-8j5xx"
Jul 01 16:58:51.108283 ip-10-0-137-244 kubenswrapper[2565]: I0701 16:58:51.108066    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/6040517d-fe23-442e-968b-b58733d448a0-os-release\") pod \"multus-nlxxc\" (UID: \"6040517d-fe23-442e-968b-b58733d448a0\") " pod="openshift-multus/multus-nlxxc"
Jul 01 16:58:51.108283 ip-10-0-137-244 kubenswrapper[2565]: I0701 16:58:51.108117    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/6040517d-fe23-442e-968b-b58733d448a0-cni-binary-copy\") pod \"multus-nlxxc\" (UID: \"6040517d-fe23-442e-968b-b58733d448a0\") " pod="openshift-multus/multus-nlxxc"
Jul 01 16:58:51.108283 ip-10-0-137-244 kubenswrapper[2565]: I0701 16:58:51.108147    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/bb7ebc32-d0cd-4eb0-b998-3ce298f73809-host-cni-bin\") pod \"ovnkube-node-zjsw4\" (UID: \"bb7ebc32-d0cd-4eb0-b998-3ce298f73809\") " pod="openshift-ovn-kubernetes/ovnkube-node-zjsw4"
Jul 01 16:58:51.108283 ip-10-0-137-244 kubenswrapper[2565]: I0701 16:58:51.108179    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/bb7ebc32-d0cd-4eb0-b998-3ce298f73809-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-zjsw4\" (UID: \"bb7ebc32-d0cd-4eb0-b998-3ce298f73809\") " pod="openshift-ovn-kubernetes/ovnkube-node-zjsw4"
Jul 01 16:58:51.108283 ip-10-0-137-244 kubenswrapper[2565]: I0701 16:58:51.108214    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/33c916f6-76ab-4a81-9955-3ffae48a52f8-etc-sysctl-conf\") pod \"tuned-8j5xx\" (UID: \"33c916f6-76ab-4a81-9955-3ffae48a52f8\") " pod="openshift-cluster-node-tuning-operator/tuned-8j5xx"
Jul 01 16:58:51.108985 ip-10-0-137-244 kubenswrapper[2565]: I0701 16:58:51.108245    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/bffd2a49-ec4f-44ee-a47c-19180aaeeec2-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-kzwpl\" (UID: \"bffd2a49-ec4f-44ee-a47c-19180aaeeec2\") " pod="openshift-multus/multus-additional-cni-plugins-kzwpl"
Jul 01 16:58:51.108985 ip-10-0-137-244 kubenswrapper[2565]: I0701 16:58:51.108278    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/a6488c7c-9ea5-4ad2-91ba-b1ffda18b203-metrics-certs\") pod \"network-metrics-daemon-hthvc\" (UID: \"a6488c7c-9ea5-4ad2-91ba-b1ffda18b203\") " pod="openshift-multus/network-metrics-daemon-hthvc"
Jul 01 16:58:51.108985 ip-10-0-137-244 kubenswrapper[2565]: I0701 16:58:51.108303    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/adf23d12-6f62-40f2-9ca0-026c878bfe07-agent-certs\") pod \"konnectivity-agent-w6h6d\" (UID: \"adf23d12-6f62-40f2-9ca0-026c878bfe07\") " pod="kube-system/konnectivity-agent-w6h6d"
Jul 01 16:58:51.108985 ip-10-0-137-244 kubenswrapper[2565]: I0701 16:58:51.108334    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9db26\" (UniqueName: \"kubernetes.io/projected/b878d4e6-fe3b-4c52-aaea-de5a58d48afc-kube-api-access-9db26\") pod \"node-ca-g7t25\" (UID: \"b878d4e6-fe3b-4c52-aaea-de5a58d48afc\") " pod="openshift-image-registry/node-ca-g7t25"
Jul 01 16:58:51.108985 ip-10-0-137-244 kubenswrapper[2565]: I0701 16:58:51.108366    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/bb7ebc32-d0cd-4eb0-b998-3ce298f73809-systemd-units\") pod \"ovnkube-node-zjsw4\" (UID: \"bb7ebc32-d0cd-4eb0-b998-3ce298f73809\") " pod="openshift-ovn-kubernetes/ovnkube-node-zjsw4"
Jul 01 16:58:51.108985 ip-10-0-137-244 kubenswrapper[2565]: I0701 16:58:51.108400    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/bb7ebc32-d0cd-4eb0-b998-3ce298f73809-host-run-netns\") pod \"ovnkube-node-zjsw4\" (UID: \"bb7ebc32-d0cd-4eb0-b998-3ce298f73809\") " pod="openshift-ovn-kubernetes/ovnkube-node-zjsw4"
Jul 01 16:58:51.108985 ip-10-0-137-244 kubenswrapper[2565]: I0701 16:58:51.108452    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/6040517d-fe23-442e-968b-b58733d448a0-hostroot\") pod \"multus-nlxxc\" (UID: \"6040517d-fe23-442e-968b-b58733d448a0\") " pod="openshift-multus/multus-nlxxc"
Jul 01 16:58:51.108985 ip-10-0-137-244 kubenswrapper[2565]: I0701 16:58:51.108479    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/121518fd-3292-4547-bc31-921e4db77e54-iptables-alerter-script\") pod \"iptables-alerter-r59mr\" (UID: \"121518fd-3292-4547-bc31-921e4db77e54\") " pod="openshift-network-operator/iptables-alerter-r59mr"
Jul 01 16:58:51.108985 ip-10-0-137-244 kubenswrapper[2565]: I0701 16:58:51.108512    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/bb7ebc32-d0cd-4eb0-b998-3ce298f73809-ovn-node-metrics-cert\") pod \"ovnkube-node-zjsw4\" (UID: \"bb7ebc32-d0cd-4eb0-b998-3ce298f73809\") " pod="openshift-ovn-kubernetes/ovnkube-node-zjsw4"
Jul 01 16:58:51.108985 ip-10-0-137-244 kubenswrapper[2565]: I0701 16:58:51.108544    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/c564aa0e-7841-4b52-8262-4ee37c496c54-socket-dir\") pod \"aws-ebs-csi-driver-node-rh87s\" (UID: \"c564aa0e-7841-4b52-8262-4ee37c496c54\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-rh87s"
Jul 01 16:58:51.108985 ip-10-0-137-244 kubenswrapper[2565]: I0701 16:58:51.108574    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/33c916f6-76ab-4a81-9955-3ffae48a52f8-etc-kubernetes\") pod \"tuned-8j5xx\" (UID: \"33c916f6-76ab-4a81-9955-3ffae48a52f8\") " pod="openshift-cluster-node-tuning-operator/tuned-8j5xx"
Jul 01 16:58:51.108985 ip-10-0-137-244 kubenswrapper[2565]: I0701 16:58:51.108599    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/33c916f6-76ab-4a81-9955-3ffae48a52f8-lib-modules\") pod \"tuned-8j5xx\" (UID: \"33c916f6-76ab-4a81-9955-3ffae48a52f8\") " pod="openshift-cluster-node-tuning-operator/tuned-8j5xx"
Jul 01 16:58:51.108985 ip-10-0-137-244 kubenswrapper[2565]: I0701 16:58:51.108671    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/bffd2a49-ec4f-44ee-a47c-19180aaeeec2-system-cni-dir\") pod \"multus-additional-cni-plugins-kzwpl\" (UID: \"bffd2a49-ec4f-44ee-a47c-19180aaeeec2\") " pod="openshift-multus/multus-additional-cni-plugins-kzwpl"
Jul 01 16:58:51.108985 ip-10-0-137-244 kubenswrapper[2565]: I0701 16:58:51.108741    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zbt7k\" (UniqueName: \"kubernetes.io/projected/bffd2a49-ec4f-44ee-a47c-19180aaeeec2-kube-api-access-zbt7k\") pod \"multus-additional-cni-plugins-kzwpl\" (UID: \"bffd2a49-ec4f-44ee-a47c-19180aaeeec2\") " pod="openshift-multus/multus-additional-cni-plugins-kzwpl"
Jul 01 16:58:51.108985 ip-10-0-137-244 kubenswrapper[2565]: I0701 16:58:51.108785    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/bb7ebc32-d0cd-4eb0-b998-3ce298f73809-env-overrides\") pod \"ovnkube-node-zjsw4\" (UID: \"bb7ebc32-d0cd-4eb0-b998-3ce298f73809\") " pod="openshift-ovn-kubernetes/ovnkube-node-zjsw4"
Jul 01 16:58:51.108985 ip-10-0-137-244 kubenswrapper[2565]: I0701 16:58:51.108811    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/b2746391-676c-4add-9406-a60838c1ac2c-original-pull-secret\") pod \"global-pull-secret-syncer-vfng9\" (UID: \"b2746391-676c-4add-9406-a60838c1ac2c\") " pod="kube-system/global-pull-secret-syncer-vfng9"
Jul 01 16:58:51.109669 ip-10-0-137-244 kubenswrapper[2565]: I0701 16:58:51.108843    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/33c916f6-76ab-4a81-9955-3ffae48a52f8-etc-sysconfig\") pod \"tuned-8j5xx\" (UID: \"33c916f6-76ab-4a81-9955-3ffae48a52f8\") " pod="openshift-cluster-node-tuning-operator/tuned-8j5xx"
Jul 01 16:58:51.109669 ip-10-0-137-244 kubenswrapper[2565]: I0701 16:58:51.108902    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/33c916f6-76ab-4a81-9955-3ffae48a52f8-sys\") pod \"tuned-8j5xx\" (UID: \"33c916f6-76ab-4a81-9955-3ffae48a52f8\") " pod="openshift-cluster-node-tuning-operator/tuned-8j5xx"
Jul 01 16:58:51.109669 ip-10-0-137-244 kubenswrapper[2565]: I0701 16:58:51.109177    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/6040517d-fe23-442e-968b-b58733d448a0-cnibin\") pod \"multus-nlxxc\" (UID: \"6040517d-fe23-442e-968b-b58733d448a0\") " pod="openshift-multus/multus-nlxxc"
Jul 01 16:58:51.109669 ip-10-0-137-244 kubenswrapper[2565]: I0701 16:58:51.109386    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/6040517d-fe23-442e-968b-b58733d448a0-host-run-netns\") pod \"multus-nlxxc\" (UID: \"6040517d-fe23-442e-968b-b58733d448a0\") " pod="openshift-multus/multus-nlxxc"
Jul 01 16:58:51.109669 ip-10-0-137-244 kubenswrapper[2565]: I0701 16:58:51.109485    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/b878d4e6-fe3b-4c52-aaea-de5a58d48afc-serviceca\") pod \"node-ca-g7t25\" (UID: \"b878d4e6-fe3b-4c52-aaea-de5a58d48afc\") " pod="openshift-image-registry/node-ca-g7t25"
Jul 01 16:58:51.109984 ip-10-0-137-244 kubenswrapper[2565]: I0701 16:58:51.109949    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/121518fd-3292-4547-bc31-921e4db77e54-host-slash\") pod \"iptables-alerter-r59mr\" (UID: \"121518fd-3292-4547-bc31-921e4db77e54\") " pod="openshift-network-operator/iptables-alerter-r59mr"
Jul 01 16:58:51.120186 ip-10-0-137-244 kubenswrapper[2565]: I0701 16:58:51.120137    2565 reflector.go:439] "Caches populated" logger="kubernetes.io/kubelet-serving" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162"
Jul 01 16:58:51.162148 ip-10-0-137-244 kubenswrapper[2565]: I0701 16:58:51.162127    2565 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kubelet-serving" csr="csr-9tnvv"
Jul 01 16:58:51.174533 ip-10-0-137-244 kubenswrapper[2565]: I0701 16:58:51.174517    2565 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kubelet-serving" csr="csr-9tnvv"
Jul 01 16:58:51.210945 ip-10-0-137-244 kubenswrapper[2565]: I0701 16:58:51.210924    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/33c916f6-76ab-4a81-9955-3ffae48a52f8-etc-sysconfig\") pod \"tuned-8j5xx\" (UID: \"33c916f6-76ab-4a81-9955-3ffae48a52f8\") " pod="openshift-cluster-node-tuning-operator/tuned-8j5xx"
Jul 01 16:58:51.211022 ip-10-0-137-244 kubenswrapper[2565]: I0701 16:58:51.210948    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/33c916f6-76ab-4a81-9955-3ffae48a52f8-sys\") pod \"tuned-8j5xx\" (UID: \"33c916f6-76ab-4a81-9955-3ffae48a52f8\") " pod="openshift-cluster-node-tuning-operator/tuned-8j5xx"
Jul 01 16:58:51.211022 ip-10-0-137-244 kubenswrapper[2565]: I0701 16:58:51.210964    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/6040517d-fe23-442e-968b-b58733d448a0-cnibin\") pod \"multus-nlxxc\" (UID: \"6040517d-fe23-442e-968b-b58733d448a0\") " pod="openshift-multus/multus-nlxxc"
Jul 01 16:58:51.211022 ip-10-0-137-244 kubenswrapper[2565]: I0701 16:58:51.210978    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/6040517d-fe23-442e-968b-b58733d448a0-host-run-netns\") pod \"multus-nlxxc\" (UID: \"6040517d-fe23-442e-968b-b58733d448a0\") " pod="openshift-multus/multus-nlxxc"
Jul 01 16:58:51.211022 ip-10-0-137-244 kubenswrapper[2565]: I0701 16:58:51.210992    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/b878d4e6-fe3b-4c52-aaea-de5a58d48afc-serviceca\") pod \"node-ca-g7t25\" (UID: \"b878d4e6-fe3b-4c52-aaea-de5a58d48afc\") " pod="openshift-image-registry/node-ca-g7t25"
Jul 01 16:58:51.211303 ip-10-0-137-244 kubenswrapper[2565]: I0701 16:58:51.211056    2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/6040517d-fe23-442e-968b-b58733d448a0-host-run-netns\") pod \"multus-nlxxc\" (UID: \"6040517d-fe23-442e-968b-b58733d448a0\") " pod="openshift-multus/multus-nlxxc"
Jul 01 16:58:51.211303 ip-10-0-137-244 kubenswrapper[2565]: I0701 16:58:51.211060    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/121518fd-3292-4547-bc31-921e4db77e54-host-slash\") pod \"iptables-alerter-r59mr\" (UID: \"121518fd-3292-4547-bc31-921e4db77e54\") " pod="openshift-network-operator/iptables-alerter-r59mr"
Jul 01 16:58:51.211303 ip-10-0-137-244 kubenswrapper[2565]: I0701 16:58:51.211074    2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/33c916f6-76ab-4a81-9955-3ffae48a52f8-sys\") pod \"tuned-8j5xx\" (UID: \"33c916f6-76ab-4a81-9955-3ffae48a52f8\") " pod="openshift-cluster-node-tuning-operator/tuned-8j5xx"
Jul 01 16:58:51.211303 ip-10-0-137-244 kubenswrapper[2565]: I0701 16:58:51.211102    2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/6040517d-fe23-442e-968b-b58733d448a0-cnibin\") pod \"multus-nlxxc\" (UID: \"6040517d-fe23-442e-968b-b58733d448a0\") " pod="openshift-multus/multus-nlxxc"
Jul 01 16:58:51.211303 ip-10-0-137-244 kubenswrapper[2565]: I0701 16:58:51.211099    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/33c916f6-76ab-4a81-9955-3ffae48a52f8-var-lib-kubelet\") pod \"tuned-8j5xx\" (UID: \"33c916f6-76ab-4a81-9955-3ffae48a52f8\") " pod="openshift-cluster-node-tuning-operator/tuned-8j5xx"
Jul 01 16:58:51.211303 ip-10-0-137-244 kubenswrapper[2565]: I0701 16:58:51.211122    2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/121518fd-3292-4547-bc31-921e4db77e54-host-slash\") pod \"iptables-alerter-r59mr\" (UID: \"121518fd-3292-4547-bc31-921e4db77e54\") " pod="openshift-network-operator/iptables-alerter-r59mr"
Jul 01 16:58:51.211303 ip-10-0-137-244 kubenswrapper[2565]: I0701 16:58:51.211126    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/bb7ebc32-d0cd-4eb0-b998-3ce298f73809-etc-openvswitch\") pod \"ovnkube-node-zjsw4\" (UID: \"bb7ebc32-d0cd-4eb0-b998-3ce298f73809\") " pod="openshift-ovn-kubernetes/ovnkube-node-zjsw4"
Jul 01 16:58:51.211303 ip-10-0-137-244 kubenswrapper[2565]: I0701 16:58:51.211151    2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/33c916f6-76ab-4a81-9955-3ffae48a52f8-var-lib-kubelet\") pod \"tuned-8j5xx\" (UID: \"33c916f6-76ab-4a81-9955-3ffae48a52f8\") " pod="openshift-cluster-node-tuning-operator/tuned-8j5xx"
Jul 01 16:58:51.211303 ip-10-0-137-244 kubenswrapper[2565]: I0701 16:58:51.211159    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/c564aa0e-7841-4b52-8262-4ee37c496c54-device-dir\") pod \"aws-ebs-csi-driver-node-rh87s\" (UID: \"c564aa0e-7841-4b52-8262-4ee37c496c54\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-rh87s"
Jul 01 16:58:51.211303 ip-10-0-137-244 kubenswrapper[2565]: I0701 16:58:51.211154    2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/bb7ebc32-d0cd-4eb0-b998-3ce298f73809-etc-openvswitch\") pod \"ovnkube-node-zjsw4\" (UID: \"bb7ebc32-d0cd-4eb0-b998-3ce298f73809\") " pod="openshift-ovn-kubernetes/ovnkube-node-zjsw4"
Jul 01 16:58:51.211303 ip-10-0-137-244 kubenswrapper[2565]: I0701 16:58:51.211180    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vzm9j\" (UniqueName: \"kubernetes.io/projected/c564aa0e-7841-4b52-8262-4ee37c496c54-kube-api-access-vzm9j\") pod \"aws-ebs-csi-driver-node-rh87s\" (UID: \"c564aa0e-7841-4b52-8262-4ee37c496c54\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-rh87s"
Jul 01 16:58:51.211303 ip-10-0-137-244 kubenswrapper[2565]: I0701 16:58:51.211179    2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/33c916f6-76ab-4a81-9955-3ffae48a52f8-etc-sysconfig\") pod \"tuned-8j5xx\" (UID: \"33c916f6-76ab-4a81-9955-3ffae48a52f8\") " pod="openshift-cluster-node-tuning-operator/tuned-8j5xx"
Jul 01 16:58:51.211303 ip-10-0-137-244 kubenswrapper[2565]: I0701 16:58:51.211201    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/6040517d-fe23-442e-968b-b58733d448a0-host-run-multus-certs\") pod \"multus-nlxxc\" (UID: \"6040517d-fe23-442e-968b-b58733d448a0\") " pod="openshift-multus/multus-nlxxc"
Jul 01 16:58:51.211303 ip-10-0-137-244 kubenswrapper[2565]: I0701 16:58:51.211218    2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/c564aa0e-7841-4b52-8262-4ee37c496c54-device-dir\") pod \"aws-ebs-csi-driver-node-rh87s\" (UID: \"c564aa0e-7841-4b52-8262-4ee37c496c54\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-rh87s"
Jul 01 16:58:51.211303 ip-10-0-137-244 kubenswrapper[2565]: I0701 16:58:51.211225    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/b878d4e6-fe3b-4c52-aaea-de5a58d48afc-host\") pod \"node-ca-g7t25\" (UID: \"b878d4e6-fe3b-4c52-aaea-de5a58d48afc\") " pod="openshift-image-registry/node-ca-g7t25"
Jul 01 16:58:51.211303 ip-10-0-137-244 kubenswrapper[2565]: I0701 16:58:51.211237    2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/6040517d-fe23-442e-968b-b58733d448a0-host-run-multus-certs\") pod \"multus-nlxxc\" (UID: \"6040517d-fe23-442e-968b-b58733d448a0\") " pod="openshift-multus/multus-nlxxc"
Jul 01 16:58:51.211303 ip-10-0-137-244 kubenswrapper[2565]: I0701 16:58:51.211251    2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/b878d4e6-fe3b-4c52-aaea-de5a58d48afc-host\") pod \"node-ca-g7t25\" (UID: \"b878d4e6-fe3b-4c52-aaea-de5a58d48afc\") " pod="openshift-image-registry/node-ca-g7t25"
Jul 01 16:58:51.211303 ip-10-0-137-244 kubenswrapper[2565]: I0701 16:58:51.211254    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/bb7ebc32-d0cd-4eb0-b998-3ce298f73809-log-socket\") pod \"ovnkube-node-zjsw4\" (UID: \"bb7ebc32-d0cd-4eb0-b998-3ce298f73809\") " pod="openshift-ovn-kubernetes/ovnkube-node-zjsw4"
Jul 01 16:58:51.212260 ip-10-0-137-244 kubenswrapper[2565]: I0701 16:58:51.211280    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/bb7ebc32-d0cd-4eb0-b998-3ce298f73809-ovnkube-config\") pod \"ovnkube-node-zjsw4\" (UID: \"bb7ebc32-d0cd-4eb0-b998-3ce298f73809\") " pod="openshift-ovn-kubernetes/ovnkube-node-zjsw4"
Jul 01 16:58:51.212260 ip-10-0-137-244 kubenswrapper[2565]: I0701 16:58:51.211303    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/33c916f6-76ab-4a81-9955-3ffae48a52f8-etc-sysctl-d\") pod \"tuned-8j5xx\" (UID: \"33c916f6-76ab-4a81-9955-3ffae48a52f8\") " pod="openshift-cluster-node-tuning-operator/tuned-8j5xx"
Jul 01 16:58:51.212260 ip-10-0-137-244 kubenswrapper[2565]: I0701 16:58:51.211311    2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/bb7ebc32-d0cd-4eb0-b998-3ce298f73809-log-socket\") pod \"ovnkube-node-zjsw4\" (UID: \"bb7ebc32-d0cd-4eb0-b998-3ce298f73809\") " pod="openshift-ovn-kubernetes/ovnkube-node-zjsw4"
Jul 01 16:58:51.212260 ip-10-0-137-244 kubenswrapper[2565]: I0701 16:58:51.211330    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gwhf8\" (UniqueName: \"kubernetes.io/projected/16665992-76de-4ba6-b887-abaadbc31ae6-kube-api-access-gwhf8\") pod \"node-resolver-mth9w\" (UID: \"16665992-76de-4ba6-b887-abaadbc31ae6\") " pod="openshift-dns/node-resolver-mth9w"
Jul 01 16:58:51.212260 ip-10-0-137-244 kubenswrapper[2565]: I0701 16:58:51.211377    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/bffd2a49-ec4f-44ee-a47c-19180aaeeec2-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-kzwpl\" (UID: \"bffd2a49-ec4f-44ee-a47c-19180aaeeec2\") " pod="openshift-multus/multus-additional-cni-plugins-kzwpl"
Jul 01 16:58:51.212260 ip-10-0-137-244 kubenswrapper[2565]: I0701 16:58:51.211407    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-m4gcz\" (UniqueName: \"kubernetes.io/projected/33c916f6-76ab-4a81-9955-3ffae48a52f8-kube-api-access-m4gcz\") pod \"tuned-8j5xx\" (UID: \"33c916f6-76ab-4a81-9955-3ffae48a52f8\") " pod="openshift-cluster-node-tuning-operator/tuned-8j5xx"
Jul 01 16:58:51.212260 ip-10-0-137-244 kubenswrapper[2565]: I0701 16:58:51.211434    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/6040517d-fe23-442e-968b-b58733d448a0-multus-socket-dir-parent\") pod \"multus-nlxxc\" (UID: \"6040517d-fe23-442e-968b-b58733d448a0\") " pod="openshift-multus/multus-nlxxc"
Jul 01 16:58:51.212260 ip-10-0-137-244 kubenswrapper[2565]: I0701 16:58:51.211459    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/6040517d-fe23-442e-968b-b58733d448a0-etc-kubernetes\") pod \"multus-nlxxc\" (UID: \"6040517d-fe23-442e-968b-b58733d448a0\") " pod="openshift-multus/multus-nlxxc"
Jul 01 16:58:51.212260 ip-10-0-137-244 kubenswrapper[2565]: I0701 16:58:51.211485    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/bffd2a49-ec4f-44ee-a47c-19180aaeeec2-os-release\") pod \"multus-additional-cni-plugins-kzwpl\" (UID: \"bffd2a49-ec4f-44ee-a47c-19180aaeeec2\") " pod="openshift-multus/multus-additional-cni-plugins-kzwpl"
Jul 01 16:58:51.212260 ip-10-0-137-244 kubenswrapper[2565]: I0701 16:58:51.211510    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/bb7ebc32-d0cd-4eb0-b998-3ce298f73809-ovnkube-script-lib\") pod \"ovnkube-node-zjsw4\" (UID: \"bb7ebc32-d0cd-4eb0-b998-3ce298f73809\") " pod="openshift-ovn-kubernetes/ovnkube-node-zjsw4"
Jul 01 16:58:51.212260 ip-10-0-137-244 kubenswrapper[2565]: I0701 16:58:51.211533    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/33c916f6-76ab-4a81-9955-3ffae48a52f8-etc-tuned\") pod \"tuned-8j5xx\" (UID: \"33c916f6-76ab-4a81-9955-3ffae48a52f8\") " pod="openshift-cluster-node-tuning-operator/tuned-8j5xx"
Jul 01 16:58:51.212260 ip-10-0-137-244 kubenswrapper[2565]: I0701 16:58:51.211548    2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/6040517d-fe23-442e-968b-b58733d448a0-multus-socket-dir-parent\") pod \"multus-nlxxc\" (UID: \"6040517d-fe23-442e-968b-b58733d448a0\") " pod="openshift-multus/multus-nlxxc"
Jul 01 16:58:51.212260 ip-10-0-137-244 kubenswrapper[2565]: I0701 16:58:51.211557    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/6040517d-fe23-442e-968b-b58733d448a0-host-run-k8s-cni-cncf-io\") pod \"multus-nlxxc\" (UID: \"6040517d-fe23-442e-968b-b58733d448a0\") " pod="openshift-multus/multus-nlxxc"
Jul 01 16:58:51.212260 ip-10-0-137-244 kubenswrapper[2565]: I0701 16:58:51.211605    2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/6040517d-fe23-442e-968b-b58733d448a0-host-run-k8s-cni-cncf-io\") pod \"multus-nlxxc\" (UID: \"6040517d-fe23-442e-968b-b58733d448a0\") " pod="openshift-multus/multus-nlxxc"
Jul 01 16:58:51.212260 ip-10-0-137-244 kubenswrapper[2565]: I0701 16:58:51.211606    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/6040517d-fe23-442e-968b-b58733d448a0-host-var-lib-cni-bin\") pod \"multus-nlxxc\" (UID: \"6040517d-fe23-442e-968b-b58733d448a0\") " pod="openshift-multus/multus-nlxxc"
Jul 01 16:58:51.212260 ip-10-0-137-244 kubenswrapper[2565]: I0701 16:58:51.211637    2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/6040517d-fe23-442e-968b-b58733d448a0-host-var-lib-cni-bin\") pod \"multus-nlxxc\" (UID: \"6040517d-fe23-442e-968b-b58733d448a0\") " pod="openshift-multus/multus-nlxxc"
Jul 01 16:58:51.212260 ip-10-0-137-244 kubenswrapper[2565]: I0701 16:58:51.211621    2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/bffd2a49-ec4f-44ee-a47c-19180aaeeec2-os-release\") pod \"multus-additional-cni-plugins-kzwpl\" (UID: \"bffd2a49-ec4f-44ee-a47c-19180aaeeec2\") " pod="openshift-multus/multus-additional-cni-plugins-kzwpl"
Jul 01 16:58:51.213126 ip-10-0-137-244 kubenswrapper[2565]: I0701 16:58:51.211648    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/16665992-76de-4ba6-b887-abaadbc31ae6-tmp-dir\") pod \"node-resolver-mth9w\" (UID: \"16665992-76de-4ba6-b887-abaadbc31ae6\") " pod="openshift-dns/node-resolver-mth9w"
Jul 01 16:58:51.213126 ip-10-0-137-244 kubenswrapper[2565]: I0701 16:58:51.211673    2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/b878d4e6-fe3b-4c52-aaea-de5a58d48afc-serviceca\") pod \"node-ca-g7t25\" (UID: \"b878d4e6-fe3b-4c52-aaea-de5a58d48afc\") " pod="openshift-image-registry/node-ca-g7t25"
Jul 01 16:58:51.213126 ip-10-0-137-244 kubenswrapper[2565]: I0701 16:58:51.211684    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/bb7ebc32-d0cd-4eb0-b998-3ce298f73809-node-log\") pod \"ovnkube-node-zjsw4\" (UID: \"bb7ebc32-d0cd-4eb0-b998-3ce298f73809\") " pod="openshift-ovn-kubernetes/ovnkube-node-zjsw4"
Jul 01 16:58:51.213126 ip-10-0-137-244 kubenswrapper[2565]: I0701 16:58:51.211722    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/33c916f6-76ab-4a81-9955-3ffae48a52f8-run\") pod \"tuned-8j5xx\" (UID: \"33c916f6-76ab-4a81-9955-3ffae48a52f8\") " pod="openshift-cluster-node-tuning-operator/tuned-8j5xx"
Jul 01 16:58:51.213126 ip-10-0-137-244 kubenswrapper[2565]: I0701 16:58:51.211750    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/6040517d-fe23-442e-968b-b58733d448a0-multus-cni-dir\") pod \"multus-nlxxc\" (UID: \"6040517d-fe23-442e-968b-b58733d448a0\") " pod="openshift-multus/multus-nlxxc"
Jul 01 16:58:51.213126 ip-10-0-137-244 kubenswrapper[2565]: I0701 16:58:51.211765    2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/6040517d-fe23-442e-968b-b58733d448a0-etc-kubernetes\") pod \"multus-nlxxc\" (UID: \"6040517d-fe23-442e-968b-b58733d448a0\") " pod="openshift-multus/multus-nlxxc"
Jul 01 16:58:51.213126 ip-10-0-137-244 kubenswrapper[2565]: I0701 16:58:51.211777    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/6040517d-fe23-442e-968b-b58733d448a0-host-var-lib-cni-multus\") pod \"multus-nlxxc\" (UID: \"6040517d-fe23-442e-968b-b58733d448a0\") " pod="openshift-multus/multus-nlxxc"
Jul 01 16:58:51.213126 ip-10-0-137-244 kubenswrapper[2565]: I0701 16:58:51.211803    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/6040517d-fe23-442e-968b-b58733d448a0-multus-daemon-config\") pod \"multus-nlxxc\" (UID: \"6040517d-fe23-442e-968b-b58733d448a0\") " pod="openshift-multus/multus-nlxxc"
Jul 01 16:58:51.213126 ip-10-0-137-244 kubenswrapper[2565]: I0701 16:58:51.211815    2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run\" (UniqueName: \"kubernetes.io/host-path/33c916f6-76ab-4a81-9955-3ffae48a52f8-run\") pod \"tuned-8j5xx\" (UID: \"33c916f6-76ab-4a81-9955-3ffae48a52f8\") " pod="openshift-cluster-node-tuning-operator/tuned-8j5xx"
Jul 01 16:58:51.213126 ip-10-0-137-244 kubenswrapper[2565]: I0701 16:58:51.211829    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rsprp\" (UniqueName: \"kubernetes.io/projected/6040517d-fe23-442e-968b-b58733d448a0-kube-api-access-rsprp\") pod \"multus-nlxxc\" (UID: \"6040517d-fe23-442e-968b-b58733d448a0\") " pod="openshift-multus/multus-nlxxc"
Jul 01 16:58:51.213126 ip-10-0-137-244 kubenswrapper[2565]: I0701 16:58:51.211854    2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/bb7ebc32-d0cd-4eb0-b998-3ce298f73809-node-log\") pod \"ovnkube-node-zjsw4\" (UID: \"bb7ebc32-d0cd-4eb0-b998-3ce298f73809\") " pod="openshift-ovn-kubernetes/ovnkube-node-zjsw4"
Jul 01 16:58:51.213126 ip-10-0-137-244 kubenswrapper[2565]: I0701 16:58:51.211854    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/bffd2a49-ec4f-44ee-a47c-19180aaeeec2-cnibin\") pod \"multus-additional-cni-plugins-kzwpl\" (UID: \"bffd2a49-ec4f-44ee-a47c-19180aaeeec2\") " pod="openshift-multus/multus-additional-cni-plugins-kzwpl"
Jul 01 16:58:51.213126 ip-10-0-137-244 kubenswrapper[2565]: I0701 16:58:51.211893    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-c9p7h\" (UniqueName: \"kubernetes.io/projected/121518fd-3292-4547-bc31-921e4db77e54-kube-api-access-c9p7h\") pod \"iptables-alerter-r59mr\" (UID: \"121518fd-3292-4547-bc31-921e4db77e54\") " pod="openshift-network-operator/iptables-alerter-r59mr"
Jul 01 16:58:51.213126 ip-10-0-137-244 kubenswrapper[2565]: I0701 16:58:51.211921    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/bb7ebc32-d0cd-4eb0-b998-3ce298f73809-host-kubelet\") pod \"ovnkube-node-zjsw4\" (UID: \"bb7ebc32-d0cd-4eb0-b998-3ce298f73809\") " pod="openshift-ovn-kubernetes/ovnkube-node-zjsw4"
Jul 01 16:58:51.213126 ip-10-0-137-244 kubenswrapper[2565]: I0701 16:58:51.211930    2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/16665992-76de-4ba6-b887-abaadbc31ae6-tmp-dir\") pod \"node-resolver-mth9w\" (UID: \"16665992-76de-4ba6-b887-abaadbc31ae6\") " pod="openshift-dns/node-resolver-mth9w"
Jul 01 16:58:51.213126 ip-10-0-137-244 kubenswrapper[2565]: I0701 16:58:51.211948    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/bb7ebc32-d0cd-4eb0-b998-3ce298f73809-host-slash\") pod \"ovnkube-node-zjsw4\" (UID: \"bb7ebc32-d0cd-4eb0-b998-3ce298f73809\") " pod="openshift-ovn-kubernetes/ovnkube-node-zjsw4"
Jul 01 16:58:51.213126 ip-10-0-137-244 kubenswrapper[2565]: I0701 16:58:51.211894    2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/bffd2a49-ec4f-44ee-a47c-19180aaeeec2-cnibin\") pod \"multus-additional-cni-plugins-kzwpl\" (UID: \"bffd2a49-ec4f-44ee-a47c-19180aaeeec2\") " pod="openshift-multus/multus-additional-cni-plugins-kzwpl"
Jul 01 16:58:51.213126 ip-10-0-137-244 kubenswrapper[2565]: I0701 16:58:51.211945    2565 swap_util.go:74] "error creating dir to test if tmpfs noswap is enabled. Assuming not supported" mount path="" error="stat /var/lib/kubelet/plugins/kubernetes.io/empty-dir: no such file or directory"
Jul 01 16:58:51.214017 ip-10-0-137-244 kubenswrapper[2565]: I0701 16:58:51.211976    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/c564aa0e-7841-4b52-8262-4ee37c496c54-etc-selinux\") pod \"aws-ebs-csi-driver-node-rh87s\" (UID: \"c564aa0e-7841-4b52-8262-4ee37c496c54\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-rh87s"
Jul 01 16:58:51.214017 ip-10-0-137-244 kubenswrapper[2565]: I0701 16:58:51.212003    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/6040517d-fe23-442e-968b-b58733d448a0-multus-conf-dir\") pod \"multus-nlxxc\" (UID: \"6040517d-fe23-442e-968b-b58733d448a0\") " pod="openshift-multus/multus-nlxxc"
Jul 01 16:58:51.214017 ip-10-0-137-244 kubenswrapper[2565]: I0701 16:58:51.212019    2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/6040517d-fe23-442e-968b-b58733d448a0-multus-cni-dir\") pod \"multus-nlxxc\" (UID: \"6040517d-fe23-442e-968b-b58733d448a0\") " pod="openshift-multus/multus-nlxxc"
Jul 01 16:58:51.214017 ip-10-0-137-244 kubenswrapper[2565]: I0701 16:58:51.212021    2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/bffd2a49-ec4f-44ee-a47c-19180aaeeec2-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-kzwpl\" (UID: \"bffd2a49-ec4f-44ee-a47c-19180aaeeec2\") " pod="openshift-multus/multus-additional-cni-plugins-kzwpl"
Jul 01 16:58:51.214017 ip-10-0-137-244 kubenswrapper[2565]: I0701 16:58:51.212031    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/bffd2a49-ec4f-44ee-a47c-19180aaeeec2-tuning-conf-dir\") pod \"multus-additional-cni-plugins-kzwpl\" (UID: \"bffd2a49-ec4f-44ee-a47c-19180aaeeec2\") " pod="openshift-multus/multus-additional-cni-plugins-kzwpl"
Jul 01 16:58:51.214017 ip-10-0-137-244 kubenswrapper[2565]: I0701 16:58:51.212059    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9rt5m\" (UniqueName: \"kubernetes.io/projected/43a7325b-d5b7-492e-961d-52f0b9b32a17-kube-api-access-9rt5m\") pod \"network-check-target-fvqs4\" (UID: \"43a7325b-d5b7-492e-961d-52f0b9b32a17\") " pod="openshift-network-diagnostics/network-check-target-fvqs4"
Jul 01 16:58:51.214017 ip-10-0-137-244 kubenswrapper[2565]: I0701 16:58:51.212083    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/bb7ebc32-d0cd-4eb0-b998-3ce298f73809-var-lib-openvswitch\") pod \"ovnkube-node-zjsw4\" (UID: \"bb7ebc32-d0cd-4eb0-b998-3ce298f73809\") " pod="openshift-ovn-kubernetes/ovnkube-node-zjsw4"
Jul 01 16:58:51.214017 ip-10-0-137-244 kubenswrapper[2565]: I0701 16:58:51.212098    2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/6040517d-fe23-442e-968b-b58733d448a0-host-var-lib-cni-multus\") pod \"multus-nlxxc\" (UID: \"6040517d-fe23-442e-968b-b58733d448a0\") " pod="openshift-multus/multus-nlxxc"
Jul 01 16:58:51.214017 ip-10-0-137-244 kubenswrapper[2565]: I0701 16:58:51.212120    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-r4xhm\" (UniqueName: \"kubernetes.io/projected/bb7ebc32-d0cd-4eb0-b998-3ce298f73809-kube-api-access-r4xhm\") pod \"ovnkube-node-zjsw4\" (UID: \"bb7ebc32-d0cd-4eb0-b998-3ce298f73809\") " pod="openshift-ovn-kubernetes/ovnkube-node-zjsw4"
Jul 01 16:58:51.214017 ip-10-0-137-244 kubenswrapper[2565]: I0701 16:58:51.212133    2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/bb7ebc32-d0cd-4eb0-b998-3ce298f73809-ovnkube-script-lib\") pod \"ovnkube-node-zjsw4\" (UID: \"bb7ebc32-d0cd-4eb0-b998-3ce298f73809\") " pod="openshift-ovn-kubernetes/ovnkube-node-zjsw4"
Jul 01 16:58:51.214017 ip-10-0-137-244 kubenswrapper[2565]: I0701 16:58:51.212145    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/b2746391-676c-4add-9406-a60838c1ac2c-dbus\") pod \"global-pull-secret-syncer-vfng9\" (UID: \"b2746391-676c-4add-9406-a60838c1ac2c\") " pod="kube-system/global-pull-secret-syncer-vfng9"
Jul 01 16:58:51.214017 ip-10-0-137-244 kubenswrapper[2565]: I0701 16:58:51.212171    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/c564aa0e-7841-4b52-8262-4ee37c496c54-kubelet-dir\") pod \"aws-ebs-csi-driver-node-rh87s\" (UID: \"c564aa0e-7841-4b52-8262-4ee37c496c54\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-rh87s"
Jul 01 16:58:51.214017 ip-10-0-137-244 kubenswrapper[2565]: I0701 16:58:51.212179    2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/bb7ebc32-d0cd-4eb0-b998-3ce298f73809-host-slash\") pod \"ovnkube-node-zjsw4\" (UID: \"bb7ebc32-d0cd-4eb0-b998-3ce298f73809\") " pod="openshift-ovn-kubernetes/ovnkube-node-zjsw4"
Jul 01 16:58:51.214017 ip-10-0-137-244 kubenswrapper[2565]: I0701 16:58:51.212184    2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/bb7ebc32-d0cd-4eb0-b998-3ce298f73809-host-kubelet\") pod \"ovnkube-node-zjsw4\" (UID: \"bb7ebc32-d0cd-4eb0-b998-3ce298f73809\") " pod="openshift-ovn-kubernetes/ovnkube-node-zjsw4"
Jul 01 16:58:51.214017 ip-10-0-137-244 kubenswrapper[2565]: I0701 16:58:51.212196    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/c564aa0e-7841-4b52-8262-4ee37c496c54-sys-fs\") pod \"aws-ebs-csi-driver-node-rh87s\" (UID: \"c564aa0e-7841-4b52-8262-4ee37c496c54\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-rh87s"
Jul 01 16:58:51.214017 ip-10-0-137-244 kubenswrapper[2565]: I0701 16:58:51.212210    2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/bb7ebc32-d0cd-4eb0-b998-3ce298f73809-var-lib-openvswitch\") pod \"ovnkube-node-zjsw4\" (UID: \"bb7ebc32-d0cd-4eb0-b998-3ce298f73809\") " pod="openshift-ovn-kubernetes/ovnkube-node-zjsw4"
Jul 01 16:58:51.214017 ip-10-0-137-244 kubenswrapper[2565]: I0701 16:58:51.212221    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/33c916f6-76ab-4a81-9955-3ffae48a52f8-etc-modprobe-d\") pod \"tuned-8j5xx\" (UID: \"33c916f6-76ab-4a81-9955-3ffae48a52f8\") " pod="openshift-cluster-node-tuning-operator/tuned-8j5xx"
Jul 01 16:58:51.214935 ip-10-0-137-244 kubenswrapper[2565]: I0701 16:58:51.212247    2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/c564aa0e-7841-4b52-8262-4ee37c496c54-etc-selinux\") pod \"aws-ebs-csi-driver-node-rh87s\" (UID: \"c564aa0e-7841-4b52-8262-4ee37c496c54\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-rh87s"
Jul 01 16:58:51.214935 ip-10-0-137-244 kubenswrapper[2565]: I0701 16:58:51.212274    2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/33c916f6-76ab-4a81-9955-3ffae48a52f8-etc-sysctl-d\") pod \"tuned-8j5xx\" (UID: \"33c916f6-76ab-4a81-9955-3ffae48a52f8\") " pod="openshift-cluster-node-tuning-operator/tuned-8j5xx"
Jul 01 16:58:51.214935 ip-10-0-137-244 kubenswrapper[2565]: I0701 16:58:51.212308    2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/33c916f6-76ab-4a81-9955-3ffae48a52f8-etc-modprobe-d\") pod \"tuned-8j5xx\" (UID: \"33c916f6-76ab-4a81-9955-3ffae48a52f8\") " pod="openshift-cluster-node-tuning-operator/tuned-8j5xx"
Jul 01 16:58:51.214935 ip-10-0-137-244 kubenswrapper[2565]: I0701 16:58:51.212349    2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/bffd2a49-ec4f-44ee-a47c-19180aaeeec2-tuning-conf-dir\") pod \"multus-additional-cni-plugins-kzwpl\" (UID: \"bffd2a49-ec4f-44ee-a47c-19180aaeeec2\") " pod="openshift-multus/multus-additional-cni-plugins-kzwpl"
Jul 01 16:58:51.214935 ip-10-0-137-244 kubenswrapper[2565]: I0701 16:58:51.212146    2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/6040517d-fe23-442e-968b-b58733d448a0-multus-conf-dir\") pod \"multus-nlxxc\" (UID: \"6040517d-fe23-442e-968b-b58733d448a0\") " pod="openshift-multus/multus-nlxxc"
Jul 01 16:58:51.214935 ip-10-0-137-244 kubenswrapper[2565]: I0701 16:58:51.212382    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/33c916f6-76ab-4a81-9955-3ffae48a52f8-etc-systemd\") pod \"tuned-8j5xx\" (UID: \"33c916f6-76ab-4a81-9955-3ffae48a52f8\") " pod="openshift-cluster-node-tuning-operator/tuned-8j5xx"
Jul 01 16:58:51.214935 ip-10-0-137-244 kubenswrapper[2565]: I0701 16:58:51.212409    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/6040517d-fe23-442e-968b-b58733d448a0-host-var-lib-kubelet\") pod \"multus-nlxxc\" (UID: \"6040517d-fe23-442e-968b-b58733d448a0\") " pod="openshift-multus/multus-nlxxc"
Jul 01 16:58:51.214935 ip-10-0-137-244 kubenswrapper[2565]: I0701 16:58:51.212415    2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/b2746391-676c-4add-9406-a60838c1ac2c-dbus\") pod \"global-pull-secret-syncer-vfng9\" (UID: \"b2746391-676c-4add-9406-a60838c1ac2c\") " pod="kube-system/global-pull-secret-syncer-vfng9"
Jul 01 16:58:51.214935 ip-10-0-137-244 kubenswrapper[2565]: I0701 16:58:51.212434    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/bb7ebc32-d0cd-4eb0-b998-3ce298f73809-run-openvswitch\") pod \"ovnkube-node-zjsw4\" (UID: \"bb7ebc32-d0cd-4eb0-b998-3ce298f73809\") " pod="openshift-ovn-kubernetes/ovnkube-node-zjsw4"
Jul 01 16:58:51.214935 ip-10-0-137-244 kubenswrapper[2565]: I0701 16:58:51.212447    2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/33c916f6-76ab-4a81-9955-3ffae48a52f8-etc-systemd\") pod \"tuned-8j5xx\" (UID: \"33c916f6-76ab-4a81-9955-3ffae48a52f8\") " pod="openshift-cluster-node-tuning-operator/tuned-8j5xx"
Jul 01 16:58:51.214935 ip-10-0-137-244 kubenswrapper[2565]: I0701 16:58:51.212461    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/bb7ebc32-d0cd-4eb0-b998-3ce298f73809-host-run-ovn-kubernetes\") pod \"ovnkube-node-zjsw4\" (UID: \"bb7ebc32-d0cd-4eb0-b998-3ce298f73809\") " pod="openshift-ovn-kubernetes/ovnkube-node-zjsw4"
Jul 01 16:58:51.214935 ip-10-0-137-244 kubenswrapper[2565]: I0701 16:58:51.212478    2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/6040517d-fe23-442e-968b-b58733d448a0-host-var-lib-kubelet\") pod \"multus-nlxxc\" (UID: \"6040517d-fe23-442e-968b-b58733d448a0\") " pod="openshift-multus/multus-nlxxc"
Jul 01 16:58:51.214935 ip-10-0-137-244 kubenswrapper[2565]: I0701 16:58:51.212487    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/b2746391-676c-4add-9406-a60838c1ac2c-kubelet-config\") pod \"global-pull-secret-syncer-vfng9\" (UID: \"b2746391-676c-4add-9406-a60838c1ac2c\") " pod="kube-system/global-pull-secret-syncer-vfng9"
Jul 01 16:58:51.214935 ip-10-0-137-244 kubenswrapper[2565]: I0701 16:58:51.212497    2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/c564aa0e-7841-4b52-8262-4ee37c496c54-sys-fs\") pod \"aws-ebs-csi-driver-node-rh87s\" (UID: \"c564aa0e-7841-4b52-8262-4ee37c496c54\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-rh87s"
Jul 01 16:58:51.214935 ip-10-0-137-244 kubenswrapper[2565]: I0701 16:58:51.212504    2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/bb7ebc32-d0cd-4eb0-b998-3ce298f73809-host-run-ovn-kubernetes\") pod \"ovnkube-node-zjsw4\" (UID: \"bb7ebc32-d0cd-4eb0-b998-3ce298f73809\") " pod="openshift-ovn-kubernetes/ovnkube-node-zjsw4"
Jul 01 16:58:51.214935 ip-10-0-137-244 kubenswrapper[2565]: I0701 16:58:51.212461    2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/c564aa0e-7841-4b52-8262-4ee37c496c54-kubelet-dir\") pod \"aws-ebs-csi-driver-node-rh87s\" (UID: \"c564aa0e-7841-4b52-8262-4ee37c496c54\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-rh87s"
Jul 01 16:58:51.214935 ip-10-0-137-244 kubenswrapper[2565]: I0701 16:58:51.212515    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/c564aa0e-7841-4b52-8262-4ee37c496c54-registration-dir\") pod \"aws-ebs-csi-driver-node-rh87s\" (UID: \"c564aa0e-7841-4b52-8262-4ee37c496c54\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-rh87s"
Jul 01 16:58:51.215849 ip-10-0-137-244 kubenswrapper[2565]: I0701 16:58:51.212535    2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/bb7ebc32-d0cd-4eb0-b998-3ce298f73809-run-openvswitch\") pod \"ovnkube-node-zjsw4\" (UID: \"bb7ebc32-d0cd-4eb0-b998-3ce298f73809\") " pod="openshift-ovn-kubernetes/ovnkube-node-zjsw4"
Jul 01 16:58:51.215849 ip-10-0-137-244 kubenswrapper[2565]: I0701 16:58:51.212541    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/33c916f6-76ab-4a81-9955-3ffae48a52f8-tmp\") pod \"tuned-8j5xx\" (UID: \"33c916f6-76ab-4a81-9955-3ffae48a52f8\") " pod="openshift-cluster-node-tuning-operator/tuned-8j5xx"
Jul 01 16:58:51.215849 ip-10-0-137-244 kubenswrapper[2565]: I0701 16:58:51.212550    2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/c564aa0e-7841-4b52-8262-4ee37c496c54-registration-dir\") pod \"aws-ebs-csi-driver-node-rh87s\" (UID: \"c564aa0e-7841-4b52-8262-4ee37c496c54\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-rh87s"
Jul 01 16:58:51.215849 ip-10-0-137-244 kubenswrapper[2565]: I0701 16:58:51.212552    2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/b2746391-676c-4add-9406-a60838c1ac2c-kubelet-config\") pod \"global-pull-secret-syncer-vfng9\" (UID: \"b2746391-676c-4add-9406-a60838c1ac2c\") " pod="kube-system/global-pull-secret-syncer-vfng9"
Jul 01 16:58:51.215849 ip-10-0-137-244 kubenswrapper[2565]: I0701 16:58:51.212574    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/6040517d-fe23-442e-968b-b58733d448a0-system-cni-dir\") pod \"multus-nlxxc\" (UID: \"6040517d-fe23-442e-968b-b58733d448a0\") " pod="openshift-multus/multus-nlxxc"
Jul 01 16:58:51.215849 ip-10-0-137-244 kubenswrapper[2565]: I0701 16:58:51.212591    2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/6040517d-fe23-442e-968b-b58733d448a0-system-cni-dir\") pod \"multus-nlxxc\" (UID: \"6040517d-fe23-442e-968b-b58733d448a0\") " pod="openshift-multus/multus-nlxxc"
Jul 01 16:58:51.215849 ip-10-0-137-244 kubenswrapper[2565]: I0701 16:58:51.212604    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/16665992-76de-4ba6-b887-abaadbc31ae6-hosts-file\") pod \"node-resolver-mth9w\" (UID: \"16665992-76de-4ba6-b887-abaadbc31ae6\") " pod="openshift-dns/node-resolver-mth9w"
Jul 01 16:58:51.215849 ip-10-0-137-244 kubenswrapper[2565]: I0701 16:58:51.212627    2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/6040517d-fe23-442e-968b-b58733d448a0-multus-daemon-config\") pod \"multus-nlxxc\" (UID: \"6040517d-fe23-442e-968b-b58733d448a0\") " pod="openshift-multus/multus-nlxxc"
Jul 01 16:58:51.215849 ip-10-0-137-244 kubenswrapper[2565]: I0701 16:58:51.212629    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-29ccc\" (UniqueName: \"kubernetes.io/projected/a6488c7c-9ea5-4ad2-91ba-b1ffda18b203-kube-api-access-29ccc\") pod \"network-metrics-daemon-hthvc\" (UID: \"a6488c7c-9ea5-4ad2-91ba-b1ffda18b203\") " pod="openshift-multus/network-metrics-daemon-hthvc"
Jul 01 16:58:51.215849 ip-10-0-137-244 kubenswrapper[2565]: I0701 16:58:51.212693    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/bffd2a49-ec4f-44ee-a47c-19180aaeeec2-cni-binary-copy\") pod \"multus-additional-cni-plugins-kzwpl\" (UID: \"bffd2a49-ec4f-44ee-a47c-19180aaeeec2\") " pod="openshift-multus/multus-additional-cni-plugins-kzwpl"
Jul 01 16:58:51.215849 ip-10-0-137-244 kubenswrapper[2565]: I0701 16:58:51.212724    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/adf23d12-6f62-40f2-9ca0-026c878bfe07-konnectivity-ca\") pod \"konnectivity-agent-w6h6d\" (UID: \"adf23d12-6f62-40f2-9ca0-026c878bfe07\") " pod="kube-system/konnectivity-agent-w6h6d"
Jul 01 16:58:51.215849 ip-10-0-137-244 kubenswrapper[2565]: I0701 16:58:51.212729    2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/16665992-76de-4ba6-b887-abaadbc31ae6-hosts-file\") pod \"node-resolver-mth9w\" (UID: \"16665992-76de-4ba6-b887-abaadbc31ae6\") " pod="openshift-dns/node-resolver-mth9w"
Jul 01 16:58:51.215849 ip-10-0-137-244 kubenswrapper[2565]: I0701 16:58:51.212753    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/bb7ebc32-d0cd-4eb0-b998-3ce298f73809-run-systemd\") pod \"ovnkube-node-zjsw4\" (UID: \"bb7ebc32-d0cd-4eb0-b998-3ce298f73809\") " pod="openshift-ovn-kubernetes/ovnkube-node-zjsw4"
Jul 01 16:58:51.215849 ip-10-0-137-244 kubenswrapper[2565]: I0701 16:58:51.212780    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/bb7ebc32-d0cd-4eb0-b998-3ce298f73809-run-ovn\") pod \"ovnkube-node-zjsw4\" (UID: \"bb7ebc32-d0cd-4eb0-b998-3ce298f73809\") " pod="openshift-ovn-kubernetes/ovnkube-node-zjsw4"
Jul 01 16:58:51.215849 ip-10-0-137-244 kubenswrapper[2565]: I0701 16:58:51.212805    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/bb7ebc32-d0cd-4eb0-b998-3ce298f73809-host-cni-netd\") pod \"ovnkube-node-zjsw4\" (UID: \"bb7ebc32-d0cd-4eb0-b998-3ce298f73809\") " pod="openshift-ovn-kubernetes/ovnkube-node-zjsw4"
Jul 01 16:58:51.215849 ip-10-0-137-244 kubenswrapper[2565]: I0701 16:58:51.212862    2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/bb7ebc32-d0cd-4eb0-b998-3ce298f73809-run-systemd\") pod \"ovnkube-node-zjsw4\" (UID: \"bb7ebc32-d0cd-4eb0-b998-3ce298f73809\") " pod="openshift-ovn-kubernetes/ovnkube-node-zjsw4"
Jul 01 16:58:51.215849 ip-10-0-137-244 kubenswrapper[2565]: I0701 16:58:51.212867    2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/bb7ebc32-d0cd-4eb0-b998-3ce298f73809-host-cni-netd\") pod \"ovnkube-node-zjsw4\" (UID: \"bb7ebc32-d0cd-4eb0-b998-3ce298f73809\") " pod="openshift-ovn-kubernetes/ovnkube-node-zjsw4"
Jul 01 16:58:51.215849 ip-10-0-137-244 kubenswrapper[2565]: I0701 16:58:51.212924    2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/bb7ebc32-d0cd-4eb0-b998-3ce298f73809-run-ovn\") pod \"ovnkube-node-zjsw4\" (UID: \"bb7ebc32-d0cd-4eb0-b998-3ce298f73809\") " pod="openshift-ovn-kubernetes/ovnkube-node-zjsw4"
Jul 01 16:58:51.216536 ip-10-0-137-244 kubenswrapper[2565]: I0701 16:58:51.212983    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/33c916f6-76ab-4a81-9955-3ffae48a52f8-host\") pod \"tuned-8j5xx\" (UID: \"33c916f6-76ab-4a81-9955-3ffae48a52f8\") " pod="openshift-cluster-node-tuning-operator/tuned-8j5xx"
Jul 01 16:58:51.216536 ip-10-0-137-244 kubenswrapper[2565]: I0701 16:58:51.212999    2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/bb7ebc32-d0cd-4eb0-b998-3ce298f73809-ovnkube-config\") pod \"ovnkube-node-zjsw4\" (UID: \"bb7ebc32-d0cd-4eb0-b998-3ce298f73809\") " pod="openshift-ovn-kubernetes/ovnkube-node-zjsw4"
Jul 01 16:58:51.216536 ip-10-0-137-244 kubenswrapper[2565]: I0701 16:58:51.213008    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/6040517d-fe23-442e-968b-b58733d448a0-os-release\") pod \"multus-nlxxc\" (UID: \"6040517d-fe23-442e-968b-b58733d448a0\") " pod="openshift-multus/multus-nlxxc"
Jul 01 16:58:51.216536 ip-10-0-137-244 kubenswrapper[2565]: I0701 16:58:51.213035    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/6040517d-fe23-442e-968b-b58733d448a0-cni-binary-copy\") pod \"multus-nlxxc\" (UID: \"6040517d-fe23-442e-968b-b58733d448a0\") " pod="openshift-multus/multus-nlxxc"
Jul 01 16:58:51.216536 ip-10-0-137-244 kubenswrapper[2565]: I0701 16:58:51.213063    2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/33c916f6-76ab-4a81-9955-3ffae48a52f8-host\") pod \"tuned-8j5xx\" (UID: \"33c916f6-76ab-4a81-9955-3ffae48a52f8\") " pod="openshift-cluster-node-tuning-operator/tuned-8j5xx"
Jul 01 16:58:51.216536 ip-10-0-137-244 kubenswrapper[2565]: I0701 16:58:51.213065    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/bb7ebc32-d0cd-4eb0-b998-3ce298f73809-host-cni-bin\") pod \"ovnkube-node-zjsw4\" (UID: \"bb7ebc32-d0cd-4eb0-b998-3ce298f73809\") " pod="openshift-ovn-kubernetes/ovnkube-node-zjsw4"
Jul 01 16:58:51.216536 ip-10-0-137-244 kubenswrapper[2565]: I0701 16:58:51.213121    2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/bb7ebc32-d0cd-4eb0-b998-3ce298f73809-host-cni-bin\") pod \"ovnkube-node-zjsw4\" (UID: \"bb7ebc32-d0cd-4eb0-b998-3ce298f73809\") " pod="openshift-ovn-kubernetes/ovnkube-node-zjsw4"
Jul 01 16:58:51.216536 ip-10-0-137-244 kubenswrapper[2565]: I0701 16:58:51.213130    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/bb7ebc32-d0cd-4eb0-b998-3ce298f73809-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-zjsw4\" (UID: \"bb7ebc32-d0cd-4eb0-b998-3ce298f73809\") " pod="openshift-ovn-kubernetes/ovnkube-node-zjsw4"
Jul 01 16:58:51.216536 ip-10-0-137-244 kubenswrapper[2565]: I0701 16:58:51.213161    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/33c916f6-76ab-4a81-9955-3ffae48a52f8-etc-sysctl-conf\") pod \"tuned-8j5xx\" (UID: \"33c916f6-76ab-4a81-9955-3ffae48a52f8\") " pod="openshift-cluster-node-tuning-operator/tuned-8j5xx"
Jul 01 16:58:51.216536 ip-10-0-137-244 kubenswrapper[2565]: I0701 16:58:51.213188    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/bffd2a49-ec4f-44ee-a47c-19180aaeeec2-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-kzwpl\" (UID: \"bffd2a49-ec4f-44ee-a47c-19180aaeeec2\") " pod="openshift-multus/multus-additional-cni-plugins-kzwpl"
Jul 01 16:58:51.216536 ip-10-0-137-244 kubenswrapper[2565]: I0701 16:58:51.213213    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/a6488c7c-9ea5-4ad2-91ba-b1ffda18b203-metrics-certs\") pod \"network-metrics-daemon-hthvc\" (UID: \"a6488c7c-9ea5-4ad2-91ba-b1ffda18b203\") " pod="openshift-multus/network-metrics-daemon-hthvc"
Jul 01 16:58:51.216536 ip-10-0-137-244 kubenswrapper[2565]: I0701 16:58:51.213229    2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/bb7ebc32-d0cd-4eb0-b998-3ce298f73809-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-zjsw4\" (UID: \"bb7ebc32-d0cd-4eb0-b998-3ce298f73809\") " pod="openshift-ovn-kubernetes/ovnkube-node-zjsw4"
Jul 01 16:58:51.216536 ip-10-0-137-244 kubenswrapper[2565]: I0701 16:58:51.213237    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/adf23d12-6f62-40f2-9ca0-026c878bfe07-agent-certs\") pod \"konnectivity-agent-w6h6d\" (UID: \"adf23d12-6f62-40f2-9ca0-026c878bfe07\") " pod="kube-system/konnectivity-agent-w6h6d"
Jul 01 16:58:51.216536 ip-10-0-137-244 kubenswrapper[2565]: I0701 16:58:51.213244    2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/bffd2a49-ec4f-44ee-a47c-19180aaeeec2-cni-binary-copy\") pod \"multus-additional-cni-plugins-kzwpl\" (UID: \"bffd2a49-ec4f-44ee-a47c-19180aaeeec2\") " pod="openshift-multus/multus-additional-cni-plugins-kzwpl"
Jul 01 16:58:51.216536 ip-10-0-137-244 kubenswrapper[2565]: I0701 16:58:51.213261    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9db26\" (UniqueName: \"kubernetes.io/projected/b878d4e6-fe3b-4c52-aaea-de5a58d48afc-kube-api-access-9db26\") pod \"node-ca-g7t25\" (UID: \"b878d4e6-fe3b-4c52-aaea-de5a58d48afc\") " pod="openshift-image-registry/node-ca-g7t25"
Jul 01 16:58:51.216536 ip-10-0-137-244 kubenswrapper[2565]: I0701 16:58:51.213287    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/bb7ebc32-d0cd-4eb0-b998-3ce298f73809-systemd-units\") pod \"ovnkube-node-zjsw4\" (UID: \"bb7ebc32-d0cd-4eb0-b998-3ce298f73809\") " pod="openshift-ovn-kubernetes/ovnkube-node-zjsw4"
Jul 01 16:58:51.216536 ip-10-0-137-244 kubenswrapper[2565]: E0701 16:58:51.213527    2565 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 01 16:58:51.216536 ip-10-0-137-244 kubenswrapper[2565]: I0701 16:58:51.213545    2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/adf23d12-6f62-40f2-9ca0-026c878bfe07-konnectivity-ca\") pod \"konnectivity-agent-w6h6d\" (UID: \"adf23d12-6f62-40f2-9ca0-026c878bfe07\") " pod="kube-system/konnectivity-agent-w6h6d"
Jul 01 16:58:51.217068 ip-10-0-137-244 kubenswrapper[2565]: I0701 16:58:51.213455    2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/bb7ebc32-d0cd-4eb0-b998-3ce298f73809-systemd-units\") pod \"ovnkube-node-zjsw4\" (UID: \"bb7ebc32-d0cd-4eb0-b998-3ce298f73809\") " pod="openshift-ovn-kubernetes/ovnkube-node-zjsw4"
Jul 01 16:58:51.217068 ip-10-0-137-244 kubenswrapper[2565]: I0701 16:58:51.213569    2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/bb7ebc32-d0cd-4eb0-b998-3ce298f73809-host-run-netns\") pod \"ovnkube-node-zjsw4\" (UID: \"bb7ebc32-d0cd-4eb0-b998-3ce298f73809\") " pod="openshift-ovn-kubernetes/ovnkube-node-zjsw4"
Jul 01 16:58:51.217068 ip-10-0-137-244 kubenswrapper[2565]: I0701 16:58:51.213529    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/bb7ebc32-d0cd-4eb0-b998-3ce298f73809-host-run-netns\") pod \"ovnkube-node-zjsw4\" (UID: \"bb7ebc32-d0cd-4eb0-b998-3ce298f73809\") " pod="openshift-ovn-kubernetes/ovnkube-node-zjsw4"
Jul 01 16:58:51.217068 ip-10-0-137-244 kubenswrapper[2565]: I0701 16:58:51.213346    2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/33c916f6-76ab-4a81-9955-3ffae48a52f8-etc-sysctl-conf\") pod \"tuned-8j5xx\" (UID: \"33c916f6-76ab-4a81-9955-3ffae48a52f8\") " pod="openshift-cluster-node-tuning-operator/tuned-8j5xx"
Jul 01 16:58:51.217068 ip-10-0-137-244 kubenswrapper[2565]: I0701 16:58:51.213122    2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/6040517d-fe23-442e-968b-b58733d448a0-os-release\") pod \"multus-nlxxc\" (UID: \"6040517d-fe23-442e-968b-b58733d448a0\") " pod="openshift-multus/multus-nlxxc"
Jul 01 16:58:51.217068 ip-10-0-137-244 kubenswrapper[2565]: E0701 16:58:51.213619    2565 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a6488c7c-9ea5-4ad2-91ba-b1ffda18b203-metrics-certs podName:a6488c7c-9ea5-4ad2-91ba-b1ffda18b203 nodeName:}" failed. No retries permitted until 2026-07-01 16:58:51.71357745 +0000 UTC m=+2.181841576 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/a6488c7c-9ea5-4ad2-91ba-b1ffda18b203-metrics-certs") pod "network-metrics-daemon-hthvc" (UID: "a6488c7c-9ea5-4ad2-91ba-b1ffda18b203") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 01 16:58:51.217068 ip-10-0-137-244 kubenswrapper[2565]: I0701 16:58:51.213643    2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/6040517d-fe23-442e-968b-b58733d448a0-cni-binary-copy\") pod \"multus-nlxxc\" (UID: \"6040517d-fe23-442e-968b-b58733d448a0\") " pod="openshift-multus/multus-nlxxc"
Jul 01 16:58:51.217068 ip-10-0-137-244 kubenswrapper[2565]: I0701 16:58:51.213668    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/6040517d-fe23-442e-968b-b58733d448a0-hostroot\") pod \"multus-nlxxc\" (UID: \"6040517d-fe23-442e-968b-b58733d448a0\") " pod="openshift-multus/multus-nlxxc"
Jul 01 16:58:51.217068 ip-10-0-137-244 kubenswrapper[2565]: I0701 16:58:51.213700    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/121518fd-3292-4547-bc31-921e4db77e54-iptables-alerter-script\") pod \"iptables-alerter-r59mr\" (UID: \"121518fd-3292-4547-bc31-921e4db77e54\") " pod="openshift-network-operator/iptables-alerter-r59mr"
Jul 01 16:58:51.217068 ip-10-0-137-244 kubenswrapper[2565]: I0701 16:58:51.213717    2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/bffd2a49-ec4f-44ee-a47c-19180aaeeec2-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-kzwpl\" (UID: \"bffd2a49-ec4f-44ee-a47c-19180aaeeec2\") " pod="openshift-multus/multus-additional-cni-plugins-kzwpl"
Jul 01 16:58:51.217068 ip-10-0-137-244 kubenswrapper[2565]: I0701 16:58:51.213727    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/bb7ebc32-d0cd-4eb0-b998-3ce298f73809-ovn-node-metrics-cert\") pod \"ovnkube-node-zjsw4\" (UID: \"bb7ebc32-d0cd-4eb0-b998-3ce298f73809\") " pod="openshift-ovn-kubernetes/ovnkube-node-zjsw4"
Jul 01 16:58:51.217068 ip-10-0-137-244 kubenswrapper[2565]: I0701 16:58:51.213751    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/c564aa0e-7841-4b52-8262-4ee37c496c54-socket-dir\") pod \"aws-ebs-csi-driver-node-rh87s\" (UID: \"c564aa0e-7841-4b52-8262-4ee37c496c54\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-rh87s"
Jul 01 16:58:51.217068 ip-10-0-137-244 kubenswrapper[2565]: I0701 16:58:51.213763    2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/6040517d-fe23-442e-968b-b58733d448a0-hostroot\") pod \"multus-nlxxc\" (UID: \"6040517d-fe23-442e-968b-b58733d448a0\") " pod="openshift-multus/multus-nlxxc"
Jul 01 16:58:51.217068 ip-10-0-137-244 kubenswrapper[2565]: I0701 16:58:51.213776    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/33c916f6-76ab-4a81-9955-3ffae48a52f8-etc-kubernetes\") pod \"tuned-8j5xx\" (UID: \"33c916f6-76ab-4a81-9955-3ffae48a52f8\") " pod="openshift-cluster-node-tuning-operator/tuned-8j5xx"
Jul 01 16:58:51.217068 ip-10-0-137-244 kubenswrapper[2565]: I0701 16:58:51.213802    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/33c916f6-76ab-4a81-9955-3ffae48a52f8-lib-modules\") pod \"tuned-8j5xx\" (UID: \"33c916f6-76ab-4a81-9955-3ffae48a52f8\") " pod="openshift-cluster-node-tuning-operator/tuned-8j5xx"
Jul 01 16:58:51.217068 ip-10-0-137-244 kubenswrapper[2565]: I0701 16:58:51.213830    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/bffd2a49-ec4f-44ee-a47c-19180aaeeec2-system-cni-dir\") pod \"multus-additional-cni-plugins-kzwpl\" (UID: \"bffd2a49-ec4f-44ee-a47c-19180aaeeec2\") " pod="openshift-multus/multus-additional-cni-plugins-kzwpl"
Jul 01 16:58:51.217585 ip-10-0-137-244 kubenswrapper[2565]: I0701 16:58:51.213857    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zbt7k\" (UniqueName: \"kubernetes.io/projected/bffd2a49-ec4f-44ee-a47c-19180aaeeec2-kube-api-access-zbt7k\") pod \"multus-additional-cni-plugins-kzwpl\" (UID: \"bffd2a49-ec4f-44ee-a47c-19180aaeeec2\") " pod="openshift-multus/multus-additional-cni-plugins-kzwpl"
Jul 01 16:58:51.217585 ip-10-0-137-244 kubenswrapper[2565]: I0701 16:58:51.213860    2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/33c916f6-76ab-4a81-9955-3ffae48a52f8-etc-kubernetes\") pod \"tuned-8j5xx\" (UID: \"33c916f6-76ab-4a81-9955-3ffae48a52f8\") " pod="openshift-cluster-node-tuning-operator/tuned-8j5xx"
Jul 01 16:58:51.217585 ip-10-0-137-244 kubenswrapper[2565]: I0701 16:58:51.213881    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/bb7ebc32-d0cd-4eb0-b998-3ce298f73809-env-overrides\") pod \"ovnkube-node-zjsw4\" (UID: \"bb7ebc32-d0cd-4eb0-b998-3ce298f73809\") " pod="openshift-ovn-kubernetes/ovnkube-node-zjsw4"
Jul 01 16:58:51.217585 ip-10-0-137-244 kubenswrapper[2565]: I0701 16:58:51.213899    2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/c564aa0e-7841-4b52-8262-4ee37c496c54-socket-dir\") pod \"aws-ebs-csi-driver-node-rh87s\" (UID: \"c564aa0e-7841-4b52-8262-4ee37c496c54\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-rh87s"
Jul 01 16:58:51.217585 ip-10-0-137-244 kubenswrapper[2565]: I0701 16:58:51.213921    2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/bffd2a49-ec4f-44ee-a47c-19180aaeeec2-system-cni-dir\") pod \"multus-additional-cni-plugins-kzwpl\" (UID: \"bffd2a49-ec4f-44ee-a47c-19180aaeeec2\") " pod="openshift-multus/multus-additional-cni-plugins-kzwpl"
Jul 01 16:58:51.217585 ip-10-0-137-244 kubenswrapper[2565]: I0701 16:58:51.213956    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/b2746391-676c-4add-9406-a60838c1ac2c-original-pull-secret\") pod \"global-pull-secret-syncer-vfng9\" (UID: \"b2746391-676c-4add-9406-a60838c1ac2c\") " pod="kube-system/global-pull-secret-syncer-vfng9"
Jul 01 16:58:51.217585 ip-10-0-137-244 kubenswrapper[2565]: E0701 16:58:51.214068    2565 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 01 16:58:51.217585 ip-10-0-137-244 kubenswrapper[2565]: E0701 16:58:51.214156    2565 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b2746391-676c-4add-9406-a60838c1ac2c-original-pull-secret podName:b2746391-676c-4add-9406-a60838c1ac2c nodeName:}" failed. No retries permitted until 2026-07-01 16:58:51.71413747 +0000 UTC m=+2.182401596 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/b2746391-676c-4add-9406-a60838c1ac2c-original-pull-secret") pod "global-pull-secret-syncer-vfng9" (UID: "b2746391-676c-4add-9406-a60838c1ac2c") : object "kube-system"/"original-pull-secret" not registered
Jul 01 16:58:51.217585 ip-10-0-137-244 kubenswrapper[2565]: I0701 16:58:51.214221    2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/33c916f6-76ab-4a81-9955-3ffae48a52f8-lib-modules\") pod \"tuned-8j5xx\" (UID: \"33c916f6-76ab-4a81-9955-3ffae48a52f8\") " pod="openshift-cluster-node-tuning-operator/tuned-8j5xx"
Jul 01 16:58:51.217585 ip-10-0-137-244 kubenswrapper[2565]: I0701 16:58:51.214283    2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/121518fd-3292-4547-bc31-921e4db77e54-iptables-alerter-script\") pod \"iptables-alerter-r59mr\" (UID: \"121518fd-3292-4547-bc31-921e4db77e54\") " pod="openshift-network-operator/iptables-alerter-r59mr"
Jul 01 16:58:51.217585 ip-10-0-137-244 kubenswrapper[2565]: I0701 16:58:51.214333    2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/bb7ebc32-d0cd-4eb0-b998-3ce298f73809-env-overrides\") pod \"ovnkube-node-zjsw4\" (UID: \"bb7ebc32-d0cd-4eb0-b998-3ce298f73809\") " pod="openshift-ovn-kubernetes/ovnkube-node-zjsw4"
Jul 01 16:58:51.217585 ip-10-0-137-244 kubenswrapper[2565]: I0701 16:58:51.215666    2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/33c916f6-76ab-4a81-9955-3ffae48a52f8-etc-tuned\") pod \"tuned-8j5xx\" (UID: \"33c916f6-76ab-4a81-9955-3ffae48a52f8\") " pod="openshift-cluster-node-tuning-operator/tuned-8j5xx"
Jul 01 16:58:51.217585 ip-10-0-137-244 kubenswrapper[2565]: I0701 16:58:51.215667    2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/33c916f6-76ab-4a81-9955-3ffae48a52f8-tmp\") pod \"tuned-8j5xx\" (UID: \"33c916f6-76ab-4a81-9955-3ffae48a52f8\") " pod="openshift-cluster-node-tuning-operator/tuned-8j5xx"
Jul 01 16:58:51.217585 ip-10-0-137-244 kubenswrapper[2565]: I0701 16:58:51.216284    2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/bb7ebc32-d0cd-4eb0-b998-3ce298f73809-ovn-node-metrics-cert\") pod \"ovnkube-node-zjsw4\" (UID: \"bb7ebc32-d0cd-4eb0-b998-3ce298f73809\") " pod="openshift-ovn-kubernetes/ovnkube-node-zjsw4"
Jul 01 16:58:51.217585 ip-10-0-137-244 kubenswrapper[2565]: I0701 16:58:51.216542    2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/adf23d12-6f62-40f2-9ca0-026c878bfe07-agent-certs\") pod \"konnectivity-agent-w6h6d\" (UID: \"adf23d12-6f62-40f2-9ca0-026c878bfe07\") " pod="kube-system/konnectivity-agent-w6h6d"
Jul 01 16:58:51.230324 ip-10-0-137-244 kubenswrapper[2565]: I0701 16:58:51.230304    2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-vzm9j\" (UniqueName: \"kubernetes.io/projected/c564aa0e-7841-4b52-8262-4ee37c496c54-kube-api-access-vzm9j\") pod \"aws-ebs-csi-driver-node-rh87s\" (UID: \"c564aa0e-7841-4b52-8262-4ee37c496c54\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-rh87s"
Jul 01 16:58:51.235468 ip-10-0-137-244 kubenswrapper[2565]: E0701 16:58:51.235453    2565 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 01 16:58:51.235530 ip-10-0-137-244 kubenswrapper[2565]: E0701 16:58:51.235471    2565 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 01 16:58:51.235530 ip-10-0-137-244 kubenswrapper[2565]: E0701 16:58:51.235488    2565 projected.go:196] Error preparing data for projected volume kube-api-access-9rt5m for pod openshift-network-diagnostics/network-check-target-fvqs4: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 01 16:58:51.235604 ip-10-0-137-244 kubenswrapper[2565]: E0701 16:58:51.235531    2565 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/43a7325b-d5b7-492e-961d-52f0b9b32a17-kube-api-access-9rt5m podName:43a7325b-d5b7-492e-961d-52f0b9b32a17 nodeName:}" failed. No retries permitted until 2026-07-01 16:58:51.735519106 +0000 UTC m=+2.203783241 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-api-access-9rt5m" (UniqueName: "kubernetes.io/projected/43a7325b-d5b7-492e-961d-52f0b9b32a17-kube-api-access-9rt5m") pod "network-check-target-fvqs4" (UID: "43a7325b-d5b7-492e-961d-52f0b9b32a17") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 01 16:58:51.236252 ip-10-0-137-244 kubenswrapper[2565]: I0701 16:58:51.236235    2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-r4xhm\" (UniqueName: \"kubernetes.io/projected/bb7ebc32-d0cd-4eb0-b998-3ce298f73809-kube-api-access-r4xhm\") pod \"ovnkube-node-zjsw4\" (UID: \"bb7ebc32-d0cd-4eb0-b998-3ce298f73809\") " pod="openshift-ovn-kubernetes/ovnkube-node-zjsw4"
Jul 01 16:58:51.238447 ip-10-0-137-244 kubenswrapper[2565]: I0701 16:58:51.238427    2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-c9p7h\" (UniqueName: \"kubernetes.io/projected/121518fd-3292-4547-bc31-921e4db77e54-kube-api-access-c9p7h\") pod \"iptables-alerter-r59mr\" (UID: \"121518fd-3292-4547-bc31-921e4db77e54\") " pod="openshift-network-operator/iptables-alerter-r59mr"
Jul 01 16:58:51.244640 ip-10-0-137-244 kubenswrapper[2565]: I0701 16:58:51.244620    2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-zbt7k\" (UniqueName: \"kubernetes.io/projected/bffd2a49-ec4f-44ee-a47c-19180aaeeec2-kube-api-access-zbt7k\") pod \"multus-additional-cni-plugins-kzwpl\" (UID: \"bffd2a49-ec4f-44ee-a47c-19180aaeeec2\") " pod="openshift-multus/multus-additional-cni-plugins-kzwpl"
Jul 01 16:58:51.245622 ip-10-0-137-244 kubenswrapper[2565]: I0701 16:58:51.245600    2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-gwhf8\" (UniqueName: \"kubernetes.io/projected/16665992-76de-4ba6-b887-abaadbc31ae6-kube-api-access-gwhf8\") pod \"node-resolver-mth9w\" (UID: \"16665992-76de-4ba6-b887-abaadbc31ae6\") " pod="openshift-dns/node-resolver-mth9w"
Jul 01 16:58:51.246501 ip-10-0-137-244 kubenswrapper[2565]: I0701 16:58:51.246478    2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-9db26\" (UniqueName: \"kubernetes.io/projected/b878d4e6-fe3b-4c52-aaea-de5a58d48afc-kube-api-access-9db26\") pod \"node-ca-g7t25\" (UID: \"b878d4e6-fe3b-4c52-aaea-de5a58d48afc\") " pod="openshift-image-registry/node-ca-g7t25"
Jul 01 16:58:51.247169 ip-10-0-137-244 kubenswrapper[2565]: I0701 16:58:51.247153    2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-m4gcz\" (UniqueName: \"kubernetes.io/projected/33c916f6-76ab-4a81-9955-3ffae48a52f8-kube-api-access-m4gcz\") pod \"tuned-8j5xx\" (UID: \"33c916f6-76ab-4a81-9955-3ffae48a52f8\") " pod="openshift-cluster-node-tuning-operator/tuned-8j5xx"
Jul 01 16:58:51.247516 ip-10-0-137-244 kubenswrapper[2565]: I0701 16:58:51.247497    2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-29ccc\" (UniqueName: \"kubernetes.io/projected/a6488c7c-9ea5-4ad2-91ba-b1ffda18b203-kube-api-access-29ccc\") pod \"network-metrics-daemon-hthvc\" (UID: \"a6488c7c-9ea5-4ad2-91ba-b1ffda18b203\") " pod="openshift-multus/network-metrics-daemon-hthvc"
Jul 01 16:58:51.249539 ip-10-0-137-244 kubenswrapper[2565]: I0701 16:58:51.249522    2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-rsprp\" (UniqueName: \"kubernetes.io/projected/6040517d-fe23-442e-968b-b58733d448a0-kube-api-access-rsprp\") pod \"multus-nlxxc\" (UID: \"6040517d-fe23-442e-968b-b58733d448a0\") " pod="openshift-multus/multus-nlxxc"
Jul 01 16:58:51.411181 ip-10-0-137-244 kubenswrapper[2565]: I0701 16:58:51.411125    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-nlxxc"
Jul 01 16:58:51.432623 ip-10-0-137-244 kubenswrapper[2565]: I0701 16:58:51.432601    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-r59mr"
Jul 01 16:58:51.441019 ip-10-0-137-244 kubenswrapper[2565]: I0701 16:58:51.440999    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-zjsw4"
Jul 01 16:58:51.452804 ip-10-0-137-244 kubenswrapper[2565]: I0701 16:58:51.452784    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-rh87s"
Jul 01 16:58:51.468834 ip-10-0-137-244 kubenswrapper[2565]: I0701 16:58:51.468816    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-8j5xx"
Jul 01 16:58:51.487240 ip-10-0-137-244 kubenswrapper[2565]: I0701 16:58:51.487218    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-mth9w"
Jul 01 16:58:51.499933 ip-10-0-137-244 kubenswrapper[2565]: I0701 16:58:51.499916    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-kzwpl"
Jul 01 16:58:51.516205 ip-10-0-137-244 kubenswrapper[2565]: I0701 16:58:51.516183    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-w6h6d"
Jul 01 16:58:51.527168 ip-10-0-137-244 kubenswrapper[2565]: I0701 16:58:51.527149    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-g7t25"
Jul 01 16:58:51.531160 ip-10-0-137-244 kubenswrapper[2565]: W0701 16:58:51.531123    2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podbffd2a49_ec4f_44ee_a47c_19180aaeeec2.slice/crio-6cbcd883bda3eee5fd9962f01ec817d4aab9ed4cefa5a7fe1db926419baefbc9 WatchSource:0}: Error finding container 6cbcd883bda3eee5fd9962f01ec817d4aab9ed4cefa5a7fe1db926419baefbc9: Status 404 returned error can't find the container with id 6cbcd883bda3eee5fd9962f01ec817d4aab9ed4cefa5a7fe1db926419baefbc9
Jul 01 16:58:51.531595 ip-10-0-137-244 kubenswrapper[2565]: W0701 16:58:51.531577    2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8ab5de924319ee4f32e67a5f2f6ee7fe.slice/crio-75ccd9213df1f0db2befa587a55d8044bdd3cf1d12e8e7fcc3013e0a8f3f1ced WatchSource:0}: Error finding container 75ccd9213df1f0db2befa587a55d8044bdd3cf1d12e8e7fcc3013e0a8f3f1ced: Status 404 returned error can't find the container with id 75ccd9213df1f0db2befa587a55d8044bdd3cf1d12e8e7fcc3013e0a8f3f1ced
Jul 01 16:58:51.532359 ip-10-0-137-244 kubenswrapper[2565]: W0701 16:58:51.532333    2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podadf23d12_6f62_40f2_9ca0_026c878bfe07.slice/crio-7b97725da6a97caadd1bf139b4e879ac3537aa96a531d670c228022a001f2663 WatchSource:0}: Error finding container 7b97725da6a97caadd1bf139b4e879ac3537aa96a531d670c228022a001f2663: Status 404 returned error can't find the container with id 7b97725da6a97caadd1bf139b4e879ac3537aa96a531d670c228022a001f2663
Jul 01 16:58:51.533299 ip-10-0-137-244 kubenswrapper[2565]: W0701 16:58:51.533278    2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd96f2d2f4c7e883a098f7adf30674021.slice/crio-192d521855b734e4f27d8d11e655041ee4efdf5afb4f8378c9118aef77857ca5 WatchSource:0}: Error finding container 192d521855b734e4f27d8d11e655041ee4efdf5afb4f8378c9118aef77857ca5: Status 404 returned error can't find the container with id 192d521855b734e4f27d8d11e655041ee4efdf5afb4f8378c9118aef77857ca5
Jul 01 16:58:51.536193 ip-10-0-137-244 kubenswrapper[2565]: I0701 16:58:51.536179    2565 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Jul 01 16:58:51.537500 ip-10-0-137-244 kubenswrapper[2565]: W0701 16:58:51.537460    2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb878d4e6_fe3b_4c52_aaea_de5a58d48afc.slice/crio-2f8a54072061d05749e9cd1528703b64e773db92454d037753e99f091c0cb9c3 WatchSource:0}: Error finding container 2f8a54072061d05749e9cd1528703b64e773db92454d037753e99f091c0cb9c3: Status 404 returned error can't find the container with id 2f8a54072061d05749e9cd1528703b64e773db92454d037753e99f091c0cb9c3
Jul 01 16:58:51.715334 ip-10-0-137-244 kubenswrapper[2565]: I0701 16:58:51.715261    2565 reflector.go:439] "Caches populated" type="*v1.Service" reflector="k8s.io/client-go/informers/factory.go:160"
Jul 01 16:58:51.716221 ip-10-0-137-244 kubenswrapper[2565]: I0701 16:58:51.716204    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/a6488c7c-9ea5-4ad2-91ba-b1ffda18b203-metrics-certs\") pod \"network-metrics-daemon-hthvc\" (UID: \"a6488c7c-9ea5-4ad2-91ba-b1ffda18b203\") " pod="openshift-multus/network-metrics-daemon-hthvc"
Jul 01 16:58:51.716299 ip-10-0-137-244 kubenswrapper[2565]: I0701 16:58:51.716233    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/b2746391-676c-4add-9406-a60838c1ac2c-original-pull-secret\") pod \"global-pull-secret-syncer-vfng9\" (UID: \"b2746391-676c-4add-9406-a60838c1ac2c\") " pod="kube-system/global-pull-secret-syncer-vfng9"
Jul 01 16:58:51.716425 ip-10-0-137-244 kubenswrapper[2565]: E0701 16:58:51.716318    2565 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 01 16:58:51.716425 ip-10-0-137-244 kubenswrapper[2565]: E0701 16:58:51.716362    2565 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b2746391-676c-4add-9406-a60838c1ac2c-original-pull-secret podName:b2746391-676c-4add-9406-a60838c1ac2c nodeName:}" failed. No retries permitted until 2026-07-01 16:58:52.716348464 +0000 UTC m=+3.184612568 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/b2746391-676c-4add-9406-a60838c1ac2c-original-pull-secret") pod "global-pull-secret-syncer-vfng9" (UID: "b2746391-676c-4add-9406-a60838c1ac2c") : object "kube-system"/"original-pull-secret" not registered
Jul 01 16:58:51.716425 ip-10-0-137-244 kubenswrapper[2565]: E0701 16:58:51.716363    2565 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 01 16:58:51.716425 ip-10-0-137-244 kubenswrapper[2565]: E0701 16:58:51.716424    2565 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a6488c7c-9ea5-4ad2-91ba-b1ffda18b203-metrics-certs podName:a6488c7c-9ea5-4ad2-91ba-b1ffda18b203 nodeName:}" failed. No retries permitted until 2026-07-01 16:58:52.716402836 +0000 UTC m=+3.184666940 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/a6488c7c-9ea5-4ad2-91ba-b1ffda18b203-metrics-certs") pod "network-metrics-daemon-hthvc" (UID: "a6488c7c-9ea5-4ad2-91ba-b1ffda18b203") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 01 16:58:51.816640 ip-10-0-137-244 kubenswrapper[2565]: I0701 16:58:51.816613    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9rt5m\" (UniqueName: \"kubernetes.io/projected/43a7325b-d5b7-492e-961d-52f0b9b32a17-kube-api-access-9rt5m\") pod \"network-check-target-fvqs4\" (UID: \"43a7325b-d5b7-492e-961d-52f0b9b32a17\") " pod="openshift-network-diagnostics/network-check-target-fvqs4"
Jul 01 16:58:51.816747 ip-10-0-137-244 kubenswrapper[2565]: E0701 16:58:51.816735    2565 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 01 16:58:51.816790 ip-10-0-137-244 kubenswrapper[2565]: E0701 16:58:51.816751    2565 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 01 16:58:51.816790 ip-10-0-137-244 kubenswrapper[2565]: E0701 16:58:51.816761    2565 projected.go:196] Error preparing data for projected volume kube-api-access-9rt5m for pod openshift-network-diagnostics/network-check-target-fvqs4: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 01 16:58:51.816859 ip-10-0-137-244 kubenswrapper[2565]: E0701 16:58:51.816799    2565 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/43a7325b-d5b7-492e-961d-52f0b9b32a17-kube-api-access-9rt5m podName:43a7325b-d5b7-492e-961d-52f0b9b32a17 nodeName:}" failed. No retries permitted until 2026-07-01 16:58:52.816789144 +0000 UTC m=+3.285053248 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-api-access-9rt5m" (UniqueName: "kubernetes.io/projected/43a7325b-d5b7-492e-961d-52f0b9b32a17-kube-api-access-9rt5m") pod "network-check-target-fvqs4" (UID: "43a7325b-d5b7-492e-961d-52f0b9b32a17") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 01 16:58:51.876582 ip-10-0-137-244 kubenswrapper[2565]: W0701 16:58:51.876563    2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod121518fd_3292_4547_bc31_921e4db77e54.slice/crio-66ab260a03b265dc17429538eef63bd0c3724bc616a3e61dad69c1f61e2cd8d7 WatchSource:0}: Error finding container 66ab260a03b265dc17429538eef63bd0c3724bc616a3e61dad69c1f61e2cd8d7: Status 404 returned error can't find the container with id 66ab260a03b265dc17429538eef63bd0c3724bc616a3e61dad69c1f61e2cd8d7
Jul 01 16:58:51.886406 ip-10-0-137-244 kubenswrapper[2565]: W0701 16:58:51.886386    2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podbb7ebc32_d0cd_4eb0_b998_3ce298f73809.slice/crio-14857a68861a6aa2ea6f61d1314861bed02ff6a6cfedf9191942a2277012d433 WatchSource:0}: Error finding container 14857a68861a6aa2ea6f61d1314861bed02ff6a6cfedf9191942a2277012d433: Status 404 returned error can't find the container with id 14857a68861a6aa2ea6f61d1314861bed02ff6a6cfedf9191942a2277012d433
Jul 01 16:58:51.932848 ip-10-0-137-244 kubenswrapper[2565]: W0701 16:58:51.932820    2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod33c916f6_76ab_4a81_9955_3ffae48a52f8.slice/crio-2b859d09b0e213bb73fee0e0d6a9a85200be5d4b65f8afa35f71030efa581d75 WatchSource:0}: Error finding container 2b859d09b0e213bb73fee0e0d6a9a85200be5d4b65f8afa35f71030efa581d75: Status 404 returned error can't find the container with id 2b859d09b0e213bb73fee0e0d6a9a85200be5d4b65f8afa35f71030efa581d75
Jul 01 16:58:51.960825 ip-10-0-137-244 kubenswrapper[2565]: I0701 16:58:51.960802    2565 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160"
Jul 01 16:58:52.095497 ip-10-0-137-244 kubenswrapper[2565]: W0701 16:58:52.095465    2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc564aa0e_7841_4b52_8262_4ee37c496c54.slice/crio-867c235f38084d14640979db33f3ba412fd78d84ccf5581dd6cbd0ed15835fb7 WatchSource:0}: Error finding container 867c235f38084d14640979db33f3ba412fd78d84ccf5581dd6cbd0ed15835fb7: Status 404 returned error can't find the container with id 867c235f38084d14640979db33f3ba412fd78d84ccf5581dd6cbd0ed15835fb7
Jul 01 16:58:52.153108 ip-10-0-137-244 kubenswrapper[2565]: W0701 16:58:52.153075    2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6040517d_fe23_442e_968b_b58733d448a0.slice/crio-46e2453a594229d634a331f348180be1596dfe7a860a02585e4985190e0ec245 WatchSource:0}: Error finding container 46e2453a594229d634a331f348180be1596dfe7a860a02585e4985190e0ec245: Status 404 returned error can't find the container with id 46e2453a594229d634a331f348180be1596dfe7a860a02585e4985190e0ec245
Jul 01 16:58:52.177532 ip-10-0-137-244 kubenswrapper[2565]: I0701 16:58:52.177478    2565 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-06-30 16:53:51 +0000 UTC" deadline="2028-04-08 19:19:24.176978276 +0000 UTC"
Jul 01 16:58:52.177638 ip-10-0-137-244 kubenswrapper[2565]: I0701 16:58:52.177533    2565 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="15530h20m31.99944873s"
Jul 01 16:58:52.243671 ip-10-0-137-244 kubenswrapper[2565]: I0701 16:58:52.243646    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-fvqs4"
Jul 01 16:58:52.243800 ip-10-0-137-244 kubenswrapper[2565]: E0701 16:58:52.243769    2565 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-fvqs4" podUID="43a7325b-d5b7-492e-961d-52f0b9b32a17"
Jul 01 16:58:52.249724 ip-10-0-137-244 kubenswrapper[2565]: I0701 16:58:52.249663    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-mth9w" event={"ID":"16665992-76de-4ba6-b887-abaadbc31ae6","Type":"ContainerStarted","Data":"01bbe47eb7a693224038c871f32413fe8e3f0a1ae9f30f3c85a0697136cd8458"}
Jul 01 16:58:52.251612 ip-10-0-137-244 kubenswrapper[2565]: I0701 16:58:52.251582    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-nlxxc" event={"ID":"6040517d-fe23-442e-968b-b58733d448a0","Type":"ContainerStarted","Data":"46e2453a594229d634a331f348180be1596dfe7a860a02585e4985190e0ec245"}
Jul 01 16:58:52.255275 ip-10-0-137-244 kubenswrapper[2565]: I0701 16:58:52.255133    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-rh87s" event={"ID":"c564aa0e-7841-4b52-8262-4ee37c496c54","Type":"ContainerStarted","Data":"867c235f38084d14640979db33f3ba412fd78d84ccf5581dd6cbd0ed15835fb7"}
Jul 01 16:58:52.258259 ip-10-0-137-244 kubenswrapper[2565]: I0701 16:58:52.258142    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-8j5xx" event={"ID":"33c916f6-76ab-4a81-9955-3ffae48a52f8","Type":"ContainerStarted","Data":"2b859d09b0e213bb73fee0e0d6a9a85200be5d4b65f8afa35f71030efa581d75"}
Jul 01 16:58:52.260296 ip-10-0-137-244 kubenswrapper[2565]: I0701 16:58:52.260185    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-zjsw4" event={"ID":"bb7ebc32-d0cd-4eb0-b998-3ce298f73809","Type":"ContainerStarted","Data":"14857a68861a6aa2ea6f61d1314861bed02ff6a6cfedf9191942a2277012d433"}
Jul 01 16:58:52.262008 ip-10-0-137-244 kubenswrapper[2565]: I0701 16:58:52.261912    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-r59mr" event={"ID":"121518fd-3292-4547-bc31-921e4db77e54","Type":"ContainerStarted","Data":"66ab260a03b265dc17429538eef63bd0c3724bc616a3e61dad69c1f61e2cd8d7"}
Jul 01 16:58:52.263590 ip-10-0-137-244 kubenswrapper[2565]: I0701 16:58:52.263516    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-137-244.ec2.internal" event={"ID":"8ab5de924319ee4f32e67a5f2f6ee7fe","Type":"ContainerStarted","Data":"75ccd9213df1f0db2befa587a55d8044bdd3cf1d12e8e7fcc3013e0a8f3f1ced"}
Jul 01 16:58:52.267384 ip-10-0-137-244 kubenswrapper[2565]: I0701 16:58:52.267307    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-137-244.ec2.internal" event={"ID":"d96f2d2f4c7e883a098f7adf30674021","Type":"ContainerStarted","Data":"192d521855b734e4f27d8d11e655041ee4efdf5afb4f8378c9118aef77857ca5"}
Jul 01 16:58:52.269257 ip-10-0-137-244 kubenswrapper[2565]: I0701 16:58:52.269233    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-g7t25" event={"ID":"b878d4e6-fe3b-4c52-aaea-de5a58d48afc","Type":"ContainerStarted","Data":"2f8a54072061d05749e9cd1528703b64e773db92454d037753e99f091c0cb9c3"}
Jul 01 16:58:52.271305 ip-10-0-137-244 kubenswrapper[2565]: I0701 16:58:52.271279    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-w6h6d" event={"ID":"adf23d12-6f62-40f2-9ca0-026c878bfe07","Type":"ContainerStarted","Data":"7b97725da6a97caadd1bf139b4e879ac3537aa96a531d670c228022a001f2663"}
Jul 01 16:58:52.273494 ip-10-0-137-244 kubenswrapper[2565]: I0701 16:58:52.273471    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-kzwpl" event={"ID":"bffd2a49-ec4f-44ee-a47c-19180aaeeec2","Type":"ContainerStarted","Data":"6cbcd883bda3eee5fd9962f01ec817d4aab9ed4cefa5a7fe1db926419baefbc9"}
Jul 01 16:58:52.307469 ip-10-0-137-244 kubenswrapper[2565]: I0701 16:58:52.307437    2565 reflector.go:439] "Caches populated" type="*v1.CSIDriver" reflector="k8s.io/client-go/informers/factory.go:160"
Jul 01 16:58:52.727134 ip-10-0-137-244 kubenswrapper[2565]: I0701 16:58:52.724368    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/b2746391-676c-4add-9406-a60838c1ac2c-original-pull-secret\") pod \"global-pull-secret-syncer-vfng9\" (UID: \"b2746391-676c-4add-9406-a60838c1ac2c\") " pod="kube-system/global-pull-secret-syncer-vfng9"
Jul 01 16:58:52.727134 ip-10-0-137-244 kubenswrapper[2565]: I0701 16:58:52.724467    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/a6488c7c-9ea5-4ad2-91ba-b1ffda18b203-metrics-certs\") pod \"network-metrics-daemon-hthvc\" (UID: \"a6488c7c-9ea5-4ad2-91ba-b1ffda18b203\") " pod="openshift-multus/network-metrics-daemon-hthvc"
Jul 01 16:58:52.727134 ip-10-0-137-244 kubenswrapper[2565]: E0701 16:58:52.724590    2565 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 01 16:58:52.727134 ip-10-0-137-244 kubenswrapper[2565]: E0701 16:58:52.724651    2565 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a6488c7c-9ea5-4ad2-91ba-b1ffda18b203-metrics-certs podName:a6488c7c-9ea5-4ad2-91ba-b1ffda18b203 nodeName:}" failed. No retries permitted until 2026-07-01 16:58:54.72463171 +0000 UTC m=+5.192895829 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/a6488c7c-9ea5-4ad2-91ba-b1ffda18b203-metrics-certs") pod "network-metrics-daemon-hthvc" (UID: "a6488c7c-9ea5-4ad2-91ba-b1ffda18b203") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 01 16:58:52.727134 ip-10-0-137-244 kubenswrapper[2565]: E0701 16:58:52.725064    2565 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 01 16:58:52.727134 ip-10-0-137-244 kubenswrapper[2565]: E0701 16:58:52.725132    2565 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b2746391-676c-4add-9406-a60838c1ac2c-original-pull-secret podName:b2746391-676c-4add-9406-a60838c1ac2c nodeName:}" failed. No retries permitted until 2026-07-01 16:58:54.725115792 +0000 UTC m=+5.193379914 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/b2746391-676c-4add-9406-a60838c1ac2c-original-pull-secret") pod "global-pull-secret-syncer-vfng9" (UID: "b2746391-676c-4add-9406-a60838c1ac2c") : object "kube-system"/"original-pull-secret" not registered
Jul 01 16:58:52.825562 ip-10-0-137-244 kubenswrapper[2565]: I0701 16:58:52.825533    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9rt5m\" (UniqueName: \"kubernetes.io/projected/43a7325b-d5b7-492e-961d-52f0b9b32a17-kube-api-access-9rt5m\") pod \"network-check-target-fvqs4\" (UID: \"43a7325b-d5b7-492e-961d-52f0b9b32a17\") " pod="openshift-network-diagnostics/network-check-target-fvqs4"
Jul 01 16:58:52.825743 ip-10-0-137-244 kubenswrapper[2565]: E0701 16:58:52.825691    2565 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 01 16:58:52.825743 ip-10-0-137-244 kubenswrapper[2565]: E0701 16:58:52.825713    2565 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 01 16:58:52.825743 ip-10-0-137-244 kubenswrapper[2565]: E0701 16:58:52.825726    2565 projected.go:196] Error preparing data for projected volume kube-api-access-9rt5m for pod openshift-network-diagnostics/network-check-target-fvqs4: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 01 16:58:52.825948 ip-10-0-137-244 kubenswrapper[2565]: E0701 16:58:52.825781    2565 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/43a7325b-d5b7-492e-961d-52f0b9b32a17-kube-api-access-9rt5m podName:43a7325b-d5b7-492e-961d-52f0b9b32a17 nodeName:}" failed. No retries permitted until 2026-07-01 16:58:54.825762227 +0000 UTC m=+5.294026337 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-api-access-9rt5m" (UniqueName: "kubernetes.io/projected/43a7325b-d5b7-492e-961d-52f0b9b32a17-kube-api-access-9rt5m") pod "network-check-target-fvqs4" (UID: "43a7325b-d5b7-492e-961d-52f0b9b32a17") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 01 16:58:53.178025 ip-10-0-137-244 kubenswrapper[2565]: I0701 16:58:53.177926    2565 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-06-30 16:53:51 +0000 UTC" deadline="2028-03-08 21:40:24.24241051 +0000 UTC"
Jul 01 16:58:53.178025 ip-10-0-137-244 kubenswrapper[2565]: I0701 16:58:53.177970    2565 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="14788h41m31.064445962s"
Jul 01 16:58:53.243384 ip-10-0-137-244 kubenswrapper[2565]: I0701 16:58:53.243350    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-vfng9"
Jul 01 16:58:53.243559 ip-10-0-137-244 kubenswrapper[2565]: E0701 16:58:53.243488    2565 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-vfng9" podUID="b2746391-676c-4add-9406-a60838c1ac2c"
Jul 01 16:58:53.243926 ip-10-0-137-244 kubenswrapper[2565]: I0701 16:58:53.243905    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-hthvc"
Jul 01 16:58:53.244046 ip-10-0-137-244 kubenswrapper[2565]: E0701 16:58:53.244020    2565 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-hthvc" podUID="a6488c7c-9ea5-4ad2-91ba-b1ffda18b203"
Jul 01 16:58:54.245902 ip-10-0-137-244 kubenswrapper[2565]: I0701 16:58:54.245569    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-fvqs4"
Jul 01 16:58:54.245902 ip-10-0-137-244 kubenswrapper[2565]: E0701 16:58:54.245697    2565 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-fvqs4" podUID="43a7325b-d5b7-492e-961d-52f0b9b32a17"
Jul 01 16:58:54.739800 ip-10-0-137-244 kubenswrapper[2565]: I0701 16:58:54.739741    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/a6488c7c-9ea5-4ad2-91ba-b1ffda18b203-metrics-certs\") pod \"network-metrics-daemon-hthvc\" (UID: \"a6488c7c-9ea5-4ad2-91ba-b1ffda18b203\") " pod="openshift-multus/network-metrics-daemon-hthvc"
Jul 01 16:58:54.740002 ip-10-0-137-244 kubenswrapper[2565]: I0701 16:58:54.739841    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/b2746391-676c-4add-9406-a60838c1ac2c-original-pull-secret\") pod \"global-pull-secret-syncer-vfng9\" (UID: \"b2746391-676c-4add-9406-a60838c1ac2c\") " pod="kube-system/global-pull-secret-syncer-vfng9"
Jul 01 16:58:54.740100 ip-10-0-137-244 kubenswrapper[2565]: E0701 16:58:54.740055    2565 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 01 16:58:54.740170 ip-10-0-137-244 kubenswrapper[2565]: E0701 16:58:54.740147    2565 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b2746391-676c-4add-9406-a60838c1ac2c-original-pull-secret podName:b2746391-676c-4add-9406-a60838c1ac2c nodeName:}" failed. No retries permitted until 2026-07-01 16:58:58.740126957 +0000 UTC m=+9.208391066 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/b2746391-676c-4add-9406-a60838c1ac2c-original-pull-secret") pod "global-pull-secret-syncer-vfng9" (UID: "b2746391-676c-4add-9406-a60838c1ac2c") : object "kube-system"/"original-pull-secret" not registered
Jul 01 16:58:54.740604 ip-10-0-137-244 kubenswrapper[2565]: E0701 16:58:54.740584    2565 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 01 16:58:54.740700 ip-10-0-137-244 kubenswrapper[2565]: E0701 16:58:54.740647    2565 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a6488c7c-9ea5-4ad2-91ba-b1ffda18b203-metrics-certs podName:a6488c7c-9ea5-4ad2-91ba-b1ffda18b203 nodeName:}" failed. No retries permitted until 2026-07-01 16:58:58.740627731 +0000 UTC m=+9.208891838 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/a6488c7c-9ea5-4ad2-91ba-b1ffda18b203-metrics-certs") pod "network-metrics-daemon-hthvc" (UID: "a6488c7c-9ea5-4ad2-91ba-b1ffda18b203") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 01 16:58:54.841265 ip-10-0-137-244 kubenswrapper[2565]: I0701 16:58:54.841179    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9rt5m\" (UniqueName: \"kubernetes.io/projected/43a7325b-d5b7-492e-961d-52f0b9b32a17-kube-api-access-9rt5m\") pod \"network-check-target-fvqs4\" (UID: \"43a7325b-d5b7-492e-961d-52f0b9b32a17\") " pod="openshift-network-diagnostics/network-check-target-fvqs4"
Jul 01 16:58:54.841426 ip-10-0-137-244 kubenswrapper[2565]: E0701 16:58:54.841375    2565 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 01 16:58:54.841426 ip-10-0-137-244 kubenswrapper[2565]: E0701 16:58:54.841402    2565 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 01 16:58:54.841426 ip-10-0-137-244 kubenswrapper[2565]: E0701 16:58:54.841416    2565 projected.go:196] Error preparing data for projected volume kube-api-access-9rt5m for pod openshift-network-diagnostics/network-check-target-fvqs4: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 01 16:58:54.841592 ip-10-0-137-244 kubenswrapper[2565]: E0701 16:58:54.841480    2565 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/43a7325b-d5b7-492e-961d-52f0b9b32a17-kube-api-access-9rt5m podName:43a7325b-d5b7-492e-961d-52f0b9b32a17 nodeName:}" failed. No retries permitted until 2026-07-01 16:58:58.841460472 +0000 UTC m=+9.309724581 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-api-access-9rt5m" (UniqueName: "kubernetes.io/projected/43a7325b-d5b7-492e-961d-52f0b9b32a17-kube-api-access-9rt5m") pod "network-check-target-fvqs4" (UID: "43a7325b-d5b7-492e-961d-52f0b9b32a17") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 01 16:58:55.243163 ip-10-0-137-244 kubenswrapper[2565]: I0701 16:58:55.243064    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-vfng9"
Jul 01 16:58:55.243333 ip-10-0-137-244 kubenswrapper[2565]: E0701 16:58:55.243203    2565 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-vfng9" podUID="b2746391-676c-4add-9406-a60838c1ac2c"
Jul 01 16:58:55.243423 ip-10-0-137-244 kubenswrapper[2565]: I0701 16:58:55.243401    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-hthvc"
Jul 01 16:58:55.243561 ip-10-0-137-244 kubenswrapper[2565]: E0701 16:58:55.243538    2565 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-hthvc" podUID="a6488c7c-9ea5-4ad2-91ba-b1ffda18b203"
Jul 01 16:58:56.244244 ip-10-0-137-244 kubenswrapper[2565]: I0701 16:58:56.244212    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-fvqs4"
Jul 01 16:58:56.244696 ip-10-0-137-244 kubenswrapper[2565]: E0701 16:58:56.244339    2565 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-fvqs4" podUID="43a7325b-d5b7-492e-961d-52f0b9b32a17"
Jul 01 16:58:57.244033 ip-10-0-137-244 kubenswrapper[2565]: I0701 16:58:57.243925    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-hthvc"
Jul 01 16:58:57.244033 ip-10-0-137-244 kubenswrapper[2565]: I0701 16:58:57.243971    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-vfng9"
Jul 01 16:58:57.244266 ip-10-0-137-244 kubenswrapper[2565]: E0701 16:58:57.244079    2565 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-hthvc" podUID="a6488c7c-9ea5-4ad2-91ba-b1ffda18b203"
Jul 01 16:58:57.244266 ip-10-0-137-244 kubenswrapper[2565]: E0701 16:58:57.244202    2565 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-vfng9" podUID="b2746391-676c-4add-9406-a60838c1ac2c"
Jul 01 16:58:58.245952 ip-10-0-137-244 kubenswrapper[2565]: I0701 16:58:58.245448    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-fvqs4"
Jul 01 16:58:58.245952 ip-10-0-137-244 kubenswrapper[2565]: E0701 16:58:58.245566    2565 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-fvqs4" podUID="43a7325b-d5b7-492e-961d-52f0b9b32a17"
Jul 01 16:58:58.779082 ip-10-0-137-244 kubenswrapper[2565]: I0701 16:58:58.779045    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/a6488c7c-9ea5-4ad2-91ba-b1ffda18b203-metrics-certs\") pod \"network-metrics-daemon-hthvc\" (UID: \"a6488c7c-9ea5-4ad2-91ba-b1ffda18b203\") " pod="openshift-multus/network-metrics-daemon-hthvc"
Jul 01 16:58:58.779291 ip-10-0-137-244 kubenswrapper[2565]: I0701 16:58:58.779102    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/b2746391-676c-4add-9406-a60838c1ac2c-original-pull-secret\") pod \"global-pull-secret-syncer-vfng9\" (UID: \"b2746391-676c-4add-9406-a60838c1ac2c\") " pod="kube-system/global-pull-secret-syncer-vfng9"
Jul 01 16:58:58.779291 ip-10-0-137-244 kubenswrapper[2565]: E0701 16:58:58.779236    2565 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 01 16:58:58.779406 ip-10-0-137-244 kubenswrapper[2565]: E0701 16:58:58.779299    2565 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b2746391-676c-4add-9406-a60838c1ac2c-original-pull-secret podName:b2746391-676c-4add-9406-a60838c1ac2c nodeName:}" failed. No retries permitted until 2026-07-01 16:59:06.779281273 +0000 UTC m=+17.247545380 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/b2746391-676c-4add-9406-a60838c1ac2c-original-pull-secret") pod "global-pull-secret-syncer-vfng9" (UID: "b2746391-676c-4add-9406-a60838c1ac2c") : object "kube-system"/"original-pull-secret" not registered
Jul 01 16:58:58.779745 ip-10-0-137-244 kubenswrapper[2565]: E0701 16:58:58.779718    2565 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 01 16:58:58.779823 ip-10-0-137-244 kubenswrapper[2565]: E0701 16:58:58.779777    2565 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a6488c7c-9ea5-4ad2-91ba-b1ffda18b203-metrics-certs podName:a6488c7c-9ea5-4ad2-91ba-b1ffda18b203 nodeName:}" failed. No retries permitted until 2026-07-01 16:59:06.779760733 +0000 UTC m=+17.248024839 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/a6488c7c-9ea5-4ad2-91ba-b1ffda18b203-metrics-certs") pod "network-metrics-daemon-hthvc" (UID: "a6488c7c-9ea5-4ad2-91ba-b1ffda18b203") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 01 16:58:58.880338 ip-10-0-137-244 kubenswrapper[2565]: I0701 16:58:58.880299    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9rt5m\" (UniqueName: \"kubernetes.io/projected/43a7325b-d5b7-492e-961d-52f0b9b32a17-kube-api-access-9rt5m\") pod \"network-check-target-fvqs4\" (UID: \"43a7325b-d5b7-492e-961d-52f0b9b32a17\") " pod="openshift-network-diagnostics/network-check-target-fvqs4"
Jul 01 16:58:58.880480 ip-10-0-137-244 kubenswrapper[2565]: E0701 16:58:58.880461    2565 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 01 16:58:58.880528 ip-10-0-137-244 kubenswrapper[2565]: E0701 16:58:58.880487    2565 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 01 16:58:58.880528 ip-10-0-137-244 kubenswrapper[2565]: E0701 16:58:58.880497    2565 projected.go:196] Error preparing data for projected volume kube-api-access-9rt5m for pod openshift-network-diagnostics/network-check-target-fvqs4: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 01 16:58:58.880595 ip-10-0-137-244 kubenswrapper[2565]: E0701 16:58:58.880545    2565 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/43a7325b-d5b7-492e-961d-52f0b9b32a17-kube-api-access-9rt5m podName:43a7325b-d5b7-492e-961d-52f0b9b32a17 nodeName:}" failed. No retries permitted until 2026-07-01 16:59:06.880529979 +0000 UTC m=+17.348794081 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "kube-api-access-9rt5m" (UniqueName: "kubernetes.io/projected/43a7325b-d5b7-492e-961d-52f0b9b32a17-kube-api-access-9rt5m") pod "network-check-target-fvqs4" (UID: "43a7325b-d5b7-492e-961d-52f0b9b32a17") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 01 16:58:59.243282 ip-10-0-137-244 kubenswrapper[2565]: I0701 16:58:59.243051    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-vfng9"
Jul 01 16:58:59.243282 ip-10-0-137-244 kubenswrapper[2565]: E0701 16:58:59.243192    2565 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-vfng9" podUID="b2746391-676c-4add-9406-a60838c1ac2c"
Jul 01 16:58:59.243552 ip-10-0-137-244 kubenswrapper[2565]: I0701 16:58:59.243529    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-hthvc"
Jul 01 16:58:59.243713 ip-10-0-137-244 kubenswrapper[2565]: E0701 16:58:59.243661    2565 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-hthvc" podUID="a6488c7c-9ea5-4ad2-91ba-b1ffda18b203"
Jul 01 16:59:00.244233 ip-10-0-137-244 kubenswrapper[2565]: I0701 16:59:00.244031    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-fvqs4"
Jul 01 16:59:00.244233 ip-10-0-137-244 kubenswrapper[2565]: E0701 16:59:00.244144    2565 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-fvqs4" podUID="43a7325b-d5b7-492e-961d-52f0b9b32a17"
Jul 01 16:59:01.243102 ip-10-0-137-244 kubenswrapper[2565]: I0701 16:59:01.243066    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-hthvc"
Jul 01 16:59:01.243264 ip-10-0-137-244 kubenswrapper[2565]: I0701 16:59:01.243066    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-vfng9"
Jul 01 16:59:01.243264 ip-10-0-137-244 kubenswrapper[2565]: E0701 16:59:01.243203    2565 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-hthvc" podUID="a6488c7c-9ea5-4ad2-91ba-b1ffda18b203"
Jul 01 16:59:01.243388 ip-10-0-137-244 kubenswrapper[2565]: E0701 16:59:01.243298    2565 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-vfng9" podUID="b2746391-676c-4add-9406-a60838c1ac2c"
Jul 01 16:59:02.243187 ip-10-0-137-244 kubenswrapper[2565]: I0701 16:59:02.243165    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-fvqs4"
Jul 01 16:59:02.243446 ip-10-0-137-244 kubenswrapper[2565]: E0701 16:59:02.243260    2565 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-fvqs4" podUID="43a7325b-d5b7-492e-961d-52f0b9b32a17"
Jul 01 16:59:03.242972 ip-10-0-137-244 kubenswrapper[2565]: I0701 16:59:03.242945    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-hthvc"
Jul 01 16:59:03.243171 ip-10-0-137-244 kubenswrapper[2565]: E0701 16:59:03.243072    2565 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-hthvc" podUID="a6488c7c-9ea5-4ad2-91ba-b1ffda18b203"
Jul 01 16:59:03.243171 ip-10-0-137-244 kubenswrapper[2565]: I0701 16:59:03.243153    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-vfng9"
Jul 01 16:59:03.243547 ip-10-0-137-244 kubenswrapper[2565]: E0701 16:59:03.243234    2565 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-vfng9" podUID="b2746391-676c-4add-9406-a60838c1ac2c"
Jul 01 16:59:04.243083 ip-10-0-137-244 kubenswrapper[2565]: I0701 16:59:04.243051    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-fvqs4"
Jul 01 16:59:04.243242 ip-10-0-137-244 kubenswrapper[2565]: E0701 16:59:04.243173    2565 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-fvqs4" podUID="43a7325b-d5b7-492e-961d-52f0b9b32a17"
Jul 01 16:59:05.243008 ip-10-0-137-244 kubenswrapper[2565]: I0701 16:59:05.242974    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-vfng9"
Jul 01 16:59:05.243186 ip-10-0-137-244 kubenswrapper[2565]: I0701 16:59:05.243030    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-hthvc"
Jul 01 16:59:05.243186 ip-10-0-137-244 kubenswrapper[2565]: E0701 16:59:05.243105    2565 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-vfng9" podUID="b2746391-676c-4add-9406-a60838c1ac2c"
Jul 01 16:59:05.243310 ip-10-0-137-244 kubenswrapper[2565]: E0701 16:59:05.243193    2565 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-hthvc" podUID="a6488c7c-9ea5-4ad2-91ba-b1ffda18b203"
Jul 01 16:59:06.243788 ip-10-0-137-244 kubenswrapper[2565]: I0701 16:59:06.243760    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-fvqs4"
Jul 01 16:59:06.244272 ip-10-0-137-244 kubenswrapper[2565]: E0701 16:59:06.243879    2565 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-fvqs4" podUID="43a7325b-d5b7-492e-961d-52f0b9b32a17"
Jul 01 16:59:06.848412 ip-10-0-137-244 kubenswrapper[2565]: I0701 16:59:06.848378    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/a6488c7c-9ea5-4ad2-91ba-b1ffda18b203-metrics-certs\") pod \"network-metrics-daemon-hthvc\" (UID: \"a6488c7c-9ea5-4ad2-91ba-b1ffda18b203\") " pod="openshift-multus/network-metrics-daemon-hthvc"
Jul 01 16:59:06.848412 ip-10-0-137-244 kubenswrapper[2565]: I0701 16:59:06.848419    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/b2746391-676c-4add-9406-a60838c1ac2c-original-pull-secret\") pod \"global-pull-secret-syncer-vfng9\" (UID: \"b2746391-676c-4add-9406-a60838c1ac2c\") " pod="kube-system/global-pull-secret-syncer-vfng9"
Jul 01 16:59:06.848652 ip-10-0-137-244 kubenswrapper[2565]: E0701 16:59:06.848527    2565 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 01 16:59:06.848652 ip-10-0-137-244 kubenswrapper[2565]: E0701 16:59:06.848532    2565 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 01 16:59:06.848652 ip-10-0-137-244 kubenswrapper[2565]: E0701 16:59:06.848584    2565 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b2746391-676c-4add-9406-a60838c1ac2c-original-pull-secret podName:b2746391-676c-4add-9406-a60838c1ac2c nodeName:}" failed. No retries permitted until 2026-07-01 16:59:22.84857081 +0000 UTC m=+33.316834915 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/b2746391-676c-4add-9406-a60838c1ac2c-original-pull-secret") pod "global-pull-secret-syncer-vfng9" (UID: "b2746391-676c-4add-9406-a60838c1ac2c") : object "kube-system"/"original-pull-secret" not registered
Jul 01 16:59:06.848652 ip-10-0-137-244 kubenswrapper[2565]: E0701 16:59:06.848603    2565 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a6488c7c-9ea5-4ad2-91ba-b1ffda18b203-metrics-certs podName:a6488c7c-9ea5-4ad2-91ba-b1ffda18b203 nodeName:}" failed. No retries permitted until 2026-07-01 16:59:22.848593129 +0000 UTC m=+33.316857232 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/a6488c7c-9ea5-4ad2-91ba-b1ffda18b203-metrics-certs") pod "network-metrics-daemon-hthvc" (UID: "a6488c7c-9ea5-4ad2-91ba-b1ffda18b203") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 01 16:59:06.949197 ip-10-0-137-244 kubenswrapper[2565]: I0701 16:59:06.949157    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9rt5m\" (UniqueName: \"kubernetes.io/projected/43a7325b-d5b7-492e-961d-52f0b9b32a17-kube-api-access-9rt5m\") pod \"network-check-target-fvqs4\" (UID: \"43a7325b-d5b7-492e-961d-52f0b9b32a17\") " pod="openshift-network-diagnostics/network-check-target-fvqs4"
Jul 01 16:59:06.949371 ip-10-0-137-244 kubenswrapper[2565]: E0701 16:59:06.949320    2565 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 01 16:59:06.949371 ip-10-0-137-244 kubenswrapper[2565]: E0701 16:59:06.949340    2565 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 01 16:59:06.949371 ip-10-0-137-244 kubenswrapper[2565]: E0701 16:59:06.949349    2565 projected.go:196] Error preparing data for projected volume kube-api-access-9rt5m for pod openshift-network-diagnostics/network-check-target-fvqs4: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 01 16:59:06.949528 ip-10-0-137-244 kubenswrapper[2565]: E0701 16:59:06.949402    2565 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/43a7325b-d5b7-492e-961d-52f0b9b32a17-kube-api-access-9rt5m podName:43a7325b-d5b7-492e-961d-52f0b9b32a17 nodeName:}" failed. No retries permitted until 2026-07-01 16:59:22.949383423 +0000 UTC m=+33.417647528 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "kube-api-access-9rt5m" (UniqueName: "kubernetes.io/projected/43a7325b-d5b7-492e-961d-52f0b9b32a17-kube-api-access-9rt5m") pod "network-check-target-fvqs4" (UID: "43a7325b-d5b7-492e-961d-52f0b9b32a17") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 01 16:59:07.243481 ip-10-0-137-244 kubenswrapper[2565]: I0701 16:59:07.243450    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-hthvc"
Jul 01 16:59:07.243655 ip-10-0-137-244 kubenswrapper[2565]: I0701 16:59:07.243488    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-vfng9"
Jul 01 16:59:07.243655 ip-10-0-137-244 kubenswrapper[2565]: E0701 16:59:07.243589    2565 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-hthvc" podUID="a6488c7c-9ea5-4ad2-91ba-b1ffda18b203"
Jul 01 16:59:07.243843 ip-10-0-137-244 kubenswrapper[2565]: E0701 16:59:07.243721    2565 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-vfng9" podUID="b2746391-676c-4add-9406-a60838c1ac2c"
Jul 01 16:59:08.243594 ip-10-0-137-244 kubenswrapper[2565]: I0701 16:59:08.243558    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-fvqs4"
Jul 01 16:59:08.243766 ip-10-0-137-244 kubenswrapper[2565]: E0701 16:59:08.243683    2565 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-fvqs4" podUID="43a7325b-d5b7-492e-961d-52f0b9b32a17"
Jul 01 16:59:09.243328 ip-10-0-137-244 kubenswrapper[2565]: I0701 16:59:09.243292    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-hthvc"
Jul 01 16:59:09.243784 ip-10-0-137-244 kubenswrapper[2565]: E0701 16:59:09.243468    2565 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-hthvc" podUID="a6488c7c-9ea5-4ad2-91ba-b1ffda18b203"
Jul 01 16:59:09.243784 ip-10-0-137-244 kubenswrapper[2565]: I0701 16:59:09.243507    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-vfng9"
Jul 01 16:59:09.243784 ip-10-0-137-244 kubenswrapper[2565]: E0701 16:59:09.243611    2565 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-vfng9" podUID="b2746391-676c-4add-9406-a60838c1ac2c"
Jul 01 16:59:10.244930 ip-10-0-137-244 kubenswrapper[2565]: I0701 16:59:10.244685    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-fvqs4"
Jul 01 16:59:10.245438 ip-10-0-137-244 kubenswrapper[2565]: E0701 16:59:10.245045    2565 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-fvqs4" podUID="43a7325b-d5b7-492e-961d-52f0b9b32a17"
Jul 01 16:59:11.243838 ip-10-0-137-244 kubenswrapper[2565]: I0701 16:59:11.243806    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-hthvc"
Jul 01 16:59:11.243974 ip-10-0-137-244 kubenswrapper[2565]: I0701 16:59:11.243850    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-vfng9"
Jul 01 16:59:11.243974 ip-10-0-137-244 kubenswrapper[2565]: E0701 16:59:11.243942    2565 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-hthvc" podUID="a6488c7c-9ea5-4ad2-91ba-b1ffda18b203"
Jul 01 16:59:11.244090 ip-10-0-137-244 kubenswrapper[2565]: E0701 16:59:11.244048    2565 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-vfng9" podUID="b2746391-676c-4add-9406-a60838c1ac2c"
Jul 01 16:59:12.243948 ip-10-0-137-244 kubenswrapper[2565]: I0701 16:59:12.243920    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-fvqs4"
Jul 01 16:59:12.244363 ip-10-0-137-244 kubenswrapper[2565]: E0701 16:59:12.244070    2565 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-fvqs4" podUID="43a7325b-d5b7-492e-961d-52f0b9b32a17"
Jul 01 16:59:12.326192 ip-10-0-137-244 kubenswrapper[2565]: I0701 16:59:12.326124    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-nlxxc" event={"ID":"6040517d-fe23-442e-968b-b58733d448a0","Type":"ContainerStarted","Data":"83b7de34f1d6347ec2e99d83ffb22f451231ce17e0ac0eba5d65be821abe8a14"}
Jul 01 16:59:12.330366 ip-10-0-137-244 kubenswrapper[2565]: I0701 16:59:12.330290    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-8j5xx" event={"ID":"33c916f6-76ab-4a81-9955-3ffae48a52f8","Type":"ContainerStarted","Data":"e5656a874ecfab0d9efcc23c3ba178c1ae1ebefd25a637b5c2c42c2fa724dcca"}
Jul 01 16:59:12.335121 ip-10-0-137-244 kubenswrapper[2565]: I0701 16:59:12.335081    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-zjsw4" event={"ID":"bb7ebc32-d0cd-4eb0-b998-3ce298f73809","Type":"ContainerStarted","Data":"fe2efabb42bc6796b5bd0a865c5bf2ec5a0ae755a919cda9d82da4ec7e695a2a"}
Jul 01 16:59:12.339112 ip-10-0-137-244 kubenswrapper[2565]: I0701 16:59:12.339067    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-137-244.ec2.internal" event={"ID":"8ab5de924319ee4f32e67a5f2f6ee7fe","Type":"ContainerStarted","Data":"c0dfccf824a6b63d0f0d3c4d5c781feaf21bef98a105c732b0e1fa4191fc2c94"}
Jul 01 16:59:12.481931 ip-10-0-137-244 kubenswrapper[2565]: I0701 16:59:12.481886    2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-nlxxc" podStartSLOduration=2.674962229 podStartE2EDuration="22.481864143s" totalImagesPullingTime="19.806901914s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-01 16:58:50 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-01 16:59:12.46074824 +0000 UTC m=+22.929012364" watchObservedRunningTime="2026-07-01 16:59:12.481864143 +0000 UTC m=+22.950128266"
Jul 01 16:59:12.528397 ip-10-0-137-244 kubenswrapper[2565]: I0701 16:59:12.528346    2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/kube-apiserver-proxy-ip-10-0-137-244.ec2.internal" podStartSLOduration=22.528325612 podStartE2EDuration="22.528325612s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-01 16:58:50 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-01 16:59:12.528179164 +0000 UTC m=+22.996443289" watchObservedRunningTime="2026-07-01 16:59:12.528325612 +0000 UTC m=+22.996589739"
Jul 01 16:59:12.569492 ip-10-0-137-244 kubenswrapper[2565]: I0701 16:59:12.569335    2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-node-tuning-operator/tuned-8j5xx" podStartSLOduration=2.560661689 podStartE2EDuration="22.569319074s" totalImagesPullingTime="20.008657385s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-01 16:58:50 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-01 16:59:12.56787739 +0000 UTC m=+23.036141515" watchObservedRunningTime="2026-07-01 16:59:12.569319074 +0000 UTC m=+23.037583198"
Jul 01 16:59:13.243371 ip-10-0-137-244 kubenswrapper[2565]: I0701 16:59:13.243341    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-hthvc"
Jul 01 16:59:13.243788 ip-10-0-137-244 kubenswrapper[2565]: I0701 16:59:13.243341    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-vfng9"
Jul 01 16:59:13.243788 ip-10-0-137-244 kubenswrapper[2565]: E0701 16:59:13.243440    2565 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-hthvc" podUID="a6488c7c-9ea5-4ad2-91ba-b1ffda18b203"
Jul 01 16:59:13.243788 ip-10-0-137-244 kubenswrapper[2565]: E0701 16:59:13.243537    2565 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-vfng9" podUID="b2746391-676c-4add-9406-a60838c1ac2c"
Jul 01 16:59:13.342643 ip-10-0-137-244 kubenswrapper[2565]: I0701 16:59:13.342619    2565 generic.go:356] "Generic (PLEG): container finished" podID="d96f2d2f4c7e883a098f7adf30674021" containerID="03d0af2eae3c554822612452ff8fc711a9658ebc43d6cfdfadf90e6944e0c4dc" exitCode=0
Jul 01 16:59:13.343296 ip-10-0-137-244 kubenswrapper[2565]: I0701 16:59:13.342684    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-137-244.ec2.internal" event={"ID":"d96f2d2f4c7e883a098f7adf30674021","Type":"ContainerDied","Data":"03d0af2eae3c554822612452ff8fc711a9658ebc43d6cfdfadf90e6944e0c4dc"}
Jul 01 16:59:13.345166 ip-10-0-137-244 kubenswrapper[2565]: I0701 16:59:13.344678    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-g7t25" event={"ID":"b878d4e6-fe3b-4c52-aaea-de5a58d48afc","Type":"ContainerStarted","Data":"17a1d27deb0ec5bb1891e50e68299b53b3f8d839b5debf3cbfb0562a66bf0375"}
Jul 01 16:59:13.346924 ip-10-0-137-244 kubenswrapper[2565]: I0701 16:59:13.346904    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-w6h6d" event={"ID":"adf23d12-6f62-40f2-9ca0-026c878bfe07","Type":"ContainerStarted","Data":"ff00dfc277840da24f86473ba73b0472c6204d5a556f7e02f2b3ff59460056a5"}
Jul 01 16:59:13.348161 ip-10-0-137-244 kubenswrapper[2565]: I0701 16:59:13.348142    2565 generic.go:356] "Generic (PLEG): container finished" podID="bffd2a49-ec4f-44ee-a47c-19180aaeeec2" containerID="0cb780b971cc83a4bb7c4f18ae0bb71fb16dff8c3b5deaf561a5cd934ed56724" exitCode=0
Jul 01 16:59:13.348249 ip-10-0-137-244 kubenswrapper[2565]: I0701 16:59:13.348198    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-kzwpl" event={"ID":"bffd2a49-ec4f-44ee-a47c-19180aaeeec2","Type":"ContainerDied","Data":"0cb780b971cc83a4bb7c4f18ae0bb71fb16dff8c3b5deaf561a5cd934ed56724"}
Jul 01 16:59:13.349471 ip-10-0-137-244 kubenswrapper[2565]: I0701 16:59:13.349454    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-mth9w" event={"ID":"16665992-76de-4ba6-b887-abaadbc31ae6","Type":"ContainerStarted","Data":"716c7d726a57532feb194e4b4369cceb9b44ea3d3e51b61d7b1322aaa23e4c07"}
Jul 01 16:59:13.351076 ip-10-0-137-244 kubenswrapper[2565]: I0701 16:59:13.351052    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-rh87s" event={"ID":"c564aa0e-7841-4b52-8262-4ee37c496c54","Type":"ContainerStarted","Data":"7b980f75e4566981b9101f34a34fc8a343478a907e44a857e76e15e93201d121"}
Jul 01 16:59:13.353840 ip-10-0-137-244 kubenswrapper[2565]: I0701 16:59:13.353824    2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-zjsw4_bb7ebc32-d0cd-4eb0-b998-3ce298f73809/ovn-acl-logging/0.log"
Jul 01 16:59:13.354191 ip-10-0-137-244 kubenswrapper[2565]: I0701 16:59:13.354172    2565 generic.go:356] "Generic (PLEG): container finished" podID="bb7ebc32-d0cd-4eb0-b998-3ce298f73809" containerID="c8114152ac903f4c2f40573a62805885c3ee68258d3cd7a726379ee45231725d" exitCode=1
Jul 01 16:59:13.354265 ip-10-0-137-244 kubenswrapper[2565]: I0701 16:59:13.354209    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-zjsw4" event={"ID":"bb7ebc32-d0cd-4eb0-b998-3ce298f73809","Type":"ContainerStarted","Data":"535cffab5af76a08e2e0fba45da1350b8018b2aaa9b2f7f77d85bdc19c505d1e"}
Jul 01 16:59:13.354378 ip-10-0-137-244 kubenswrapper[2565]: I0701 16:59:13.354264    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-zjsw4" event={"ID":"bb7ebc32-d0cd-4eb0-b998-3ce298f73809","Type":"ContainerStarted","Data":"0935e673dcc4cb9403cfbd384c8f6360bcd2c19f879a94c429b0d59c5e253fbf"}
Jul 01 16:59:13.354378 ip-10-0-137-244 kubenswrapper[2565]: I0701 16:59:13.354308    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-zjsw4" event={"ID":"bb7ebc32-d0cd-4eb0-b998-3ce298f73809","Type":"ContainerStarted","Data":"a040b7f6db0a8f52f11f0fe8bf1c2fc169d9bfde1f311afe5c5dec4dc10a3e0c"}
Jul 01 16:59:13.354378 ip-10-0-137-244 kubenswrapper[2565]: I0701 16:59:13.354353    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-zjsw4" event={"ID":"bb7ebc32-d0cd-4eb0-b998-3ce298f73809","Type":"ContainerStarted","Data":"393ed2082dea57b144cf08c7a05d75e550101657be5081d02918baf7596b4626"}
Jul 01 16:59:13.354522 ip-10-0-137-244 kubenswrapper[2565]: I0701 16:59:13.354391    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-zjsw4" event={"ID":"bb7ebc32-d0cd-4eb0-b998-3ce298f73809","Type":"ContainerDied","Data":"c8114152ac903f4c2f40573a62805885c3ee68258d3cd7a726379ee45231725d"}
Jul 01 16:59:13.487070 ip-10-0-137-244 kubenswrapper[2565]: I0701 16:59:13.486858    2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/node-resolver-mth9w" podStartSLOduration=3.721230619 podStartE2EDuration="23.486839236s" totalImagesPullingTime="19.765608617s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-01 16:58:50 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-01 16:59:13.431630904 +0000 UTC m=+23.899895027" watchObservedRunningTime="2026-07-01 16:59:13.486839236 +0000 UTC m=+23.955103378"
Jul 01 16:59:13.514867 ip-10-0-137-244 kubenswrapper[2565]: I0701 16:59:13.514779    2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/konnectivity-agent-w6h6d" podStartSLOduration=7.632625583 podStartE2EDuration="23.5147651s" totalImagesPullingTime="15.882139517s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-01 16:58:50 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-01 16:59:13.512714317 +0000 UTC m=+23.980978442" watchObservedRunningTime="2026-07-01 16:59:13.5147651 +0000 UTC m=+23.983029253"
Jul 01 16:59:14.089105 ip-10-0-137-244 kubenswrapper[2565]: I0701 16:59:14.089084    2565 plugin_watcher.go:194] "Adding socket path or updating timestamp to desired state cache" path="/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock"
Jul 01 16:59:14.199702 ip-10-0-137-244 kubenswrapper[2565]: I0701 16:59:14.199524    2565 reconciler.go:161] "OperationExecutor.RegisterPlugin started" plugin={"SocketPath":"/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock","Timestamp":"2026-07-01T16:59:14.089100807Z","UUID":"5cf2ccbd-89de-4e12-b301-a17078ea0d8c","Handler":null,"Name":"","Endpoint":""}
Jul 01 16:59:14.201016 ip-10-0-137-244 kubenswrapper[2565]: I0701 16:59:14.200997    2565 csi_plugin.go:106] kubernetes.io/csi: Trying to validate a new CSI Driver with name: ebs.csi.aws.com endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock versions: 1.0.0
Jul 01 16:59:14.201143 ip-10-0-137-244 kubenswrapper[2565]: I0701 16:59:14.201054    2565 csi_plugin.go:119] kubernetes.io/csi: Register new plugin with name: ebs.csi.aws.com at endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock
Jul 01 16:59:14.244010 ip-10-0-137-244 kubenswrapper[2565]: I0701 16:59:14.243981    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-fvqs4"
Jul 01 16:59:14.244141 ip-10-0-137-244 kubenswrapper[2565]: E0701 16:59:14.244115    2565 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-fvqs4" podUID="43a7325b-d5b7-492e-961d-52f0b9b32a17"
Jul 01 16:59:14.358393 ip-10-0-137-244 kubenswrapper[2565]: I0701 16:59:14.358360    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-rh87s" event={"ID":"c564aa0e-7841-4b52-8262-4ee37c496c54","Type":"ContainerStarted","Data":"168919ce3b59a401d0b977b97ceda8da092940f249c4358e91da722d86acece3"}
Jul 01 16:59:14.359910 ip-10-0-137-244 kubenswrapper[2565]: I0701 16:59:14.359877    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-r59mr" event={"ID":"121518fd-3292-4547-bc31-921e4db77e54","Type":"ContainerStarted","Data":"3711892b0f7ac79e0cbb684c1f7afbee614f0e37dcb716bad57fd0bf9a9caa3f"}
Jul 01 16:59:14.361873 ip-10-0-137-244 kubenswrapper[2565]: I0701 16:59:14.361848    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-137-244.ec2.internal" event={"ID":"d96f2d2f4c7e883a098f7adf30674021","Type":"ContainerStarted","Data":"41074a754c60048b91dd4d8528bf5625e7402a4ae3101896487ff029fe060241"}
Jul 01 16:59:14.389840 ip-10-0-137-244 kubenswrapper[2565]: I0701 16:59:14.389786    2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/node-ca-g7t25" podStartSLOduration=4.08971608 podStartE2EDuration="24.389768945s" totalImagesPullingTime="20.300052865s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-01 16:58:50 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-01 16:59:13.553285561 +0000 UTC m=+24.021549686" watchObservedRunningTime="2026-07-01 16:59:14.389768945 +0000 UTC m=+24.858033071"
Jul 01 16:59:14.390093 ip-10-0-137-244 kubenswrapper[2565]: I0701 16:59:14.390049    2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-operator/iptables-alerter-r59mr" podStartSLOduration=4.326698229 podStartE2EDuration="24.390038005s" totalImagesPullingTime="20.063339776s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-01 16:58:50 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-01 16:59:14.389690996 +0000 UTC m=+24.857955121" watchObservedRunningTime="2026-07-01 16:59:14.390038005 +0000 UTC m=+24.858302124"
Jul 01 16:59:14.433789 ip-10-0-137-244 kubenswrapper[2565]: I0701 16:59:14.433738    2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-137-244.ec2.internal" podStartSLOduration=24.433729138 podStartE2EDuration="24.433729138s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-01 16:58:50 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-01 16:59:14.433662993 +0000 UTC m=+24.901927113" watchObservedRunningTime="2026-07-01 16:59:14.433729138 +0000 UTC m=+24.901993264"
Jul 01 16:59:14.927902 ip-10-0-137-244 kubenswrapper[2565]: I0701 16:59:14.927874    2565 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="kube-system/konnectivity-agent-w6h6d"
Jul 01 16:59:14.928581 ip-10-0-137-244 kubenswrapper[2565]: I0701 16:59:14.928549    2565 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="kube-system/konnectivity-agent-w6h6d"
Jul 01 16:59:15.243810 ip-10-0-137-244 kubenswrapper[2565]: I0701 16:59:15.243788    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-vfng9"
Jul 01 16:59:15.243933 ip-10-0-137-244 kubenswrapper[2565]: E0701 16:59:15.243910    2565 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-vfng9" podUID="b2746391-676c-4add-9406-a60838c1ac2c"
Jul 01 16:59:15.244001 ip-10-0-137-244 kubenswrapper[2565]: I0701 16:59:15.243788    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-hthvc"
Jul 01 16:59:15.244077 ip-10-0-137-244 kubenswrapper[2565]: E0701 16:59:15.244030    2565 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-hthvc" podUID="a6488c7c-9ea5-4ad2-91ba-b1ffda18b203"
Jul 01 16:59:15.366540 ip-10-0-137-244 kubenswrapper[2565]: I0701 16:59:15.366509    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-rh87s" event={"ID":"c564aa0e-7841-4b52-8262-4ee37c496c54","Type":"ContainerStarted","Data":"5acc15acd33ab51c9bd425bc1d7c02cbdf0a18a1d1548f4ab79840e28793bdb7"}
Jul 01 16:59:15.370118 ip-10-0-137-244 kubenswrapper[2565]: I0701 16:59:15.370099    2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-zjsw4_bb7ebc32-d0cd-4eb0-b998-3ce298f73809/ovn-acl-logging/0.log"
Jul 01 16:59:15.370554 ip-10-0-137-244 kubenswrapper[2565]: I0701 16:59:15.370530    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-zjsw4" event={"ID":"bb7ebc32-d0cd-4eb0-b998-3ce298f73809","Type":"ContainerStarted","Data":"99aa2a047f2d16fc5e463c7c774891254afbbbff24f7cbe8015058dc7c7618dc"}
Jul 01 16:59:15.395060 ip-10-0-137-244 kubenswrapper[2565]: I0701 16:59:15.395007    2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-rh87s" podStartSLOduration=3.195479522 podStartE2EDuration="25.39499103s" totalImagesPullingTime="22.199511508s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-01 16:58:50 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-01 16:59:15.394750717 +0000 UTC m=+25.863014842" watchObservedRunningTime="2026-07-01 16:59:15.39499103 +0000 UTC m=+25.863255157"
Jul 01 16:59:15.976966 ip-10-0-137-244 kubenswrapper[2565]: I0701 16:59:15.976927    2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kube-system/konnectivity-agent-w6h6d"
Jul 01 16:59:15.977515 ip-10-0-137-244 kubenswrapper[2565]: I0701 16:59:15.977491    2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kube-system/konnectivity-agent-w6h6d"
Jul 01 16:59:16.243468 ip-10-0-137-244 kubenswrapper[2565]: I0701 16:59:16.243392    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-fvqs4"
Jul 01 16:59:16.243625 ip-10-0-137-244 kubenswrapper[2565]: E0701 16:59:16.243519    2565 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-fvqs4" podUID="43a7325b-d5b7-492e-961d-52f0b9b32a17"
Jul 01 16:59:17.243297 ip-10-0-137-244 kubenswrapper[2565]: I0701 16:59:17.243264    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-vfng9"
Jul 01 16:59:17.243738 ip-10-0-137-244 kubenswrapper[2565]: I0701 16:59:17.243313    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-hthvc"
Jul 01 16:59:17.243738 ip-10-0-137-244 kubenswrapper[2565]: E0701 16:59:17.243387    2565 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-vfng9" podUID="b2746391-676c-4add-9406-a60838c1ac2c"
Jul 01 16:59:17.243738 ip-10-0-137-244 kubenswrapper[2565]: E0701 16:59:17.243528    2565 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-hthvc" podUID="a6488c7c-9ea5-4ad2-91ba-b1ffda18b203"
Jul 01 16:59:18.243695 ip-10-0-137-244 kubenswrapper[2565]: I0701 16:59:18.243524    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-fvqs4"
Jul 01 16:59:18.244275 ip-10-0-137-244 kubenswrapper[2565]: E0701 16:59:18.243782    2565 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-fvqs4" podUID="43a7325b-d5b7-492e-961d-52f0b9b32a17"
Jul 01 16:59:18.382444 ip-10-0-137-244 kubenswrapper[2565]: I0701 16:59:18.382423    2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-zjsw4_bb7ebc32-d0cd-4eb0-b998-3ce298f73809/ovn-acl-logging/0.log"
Jul 01 16:59:18.382855 ip-10-0-137-244 kubenswrapper[2565]: I0701 16:59:18.382832    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-zjsw4" event={"ID":"bb7ebc32-d0cd-4eb0-b998-3ce298f73809","Type":"ContainerStarted","Data":"7057e77c7417974cbccbff1fffd0aed433a2fb0e09abecb5230165e1172f6524"}
Jul 01 16:59:18.383070 ip-10-0-137-244 kubenswrapper[2565]: I0701 16:59:18.383054    2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-zjsw4"
Jul 01 16:59:18.383183 ip-10-0-137-244 kubenswrapper[2565]: I0701 16:59:18.383122    2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-zjsw4"
Jul 01 16:59:18.383245 ip-10-0-137-244 kubenswrapper[2565]: I0701 16:59:18.383229    2565 scope.go:117] "RemoveContainer" containerID="c8114152ac903f4c2f40573a62805885c3ee68258d3cd7a726379ee45231725d"
Jul 01 16:59:18.384925 ip-10-0-137-244 kubenswrapper[2565]: I0701 16:59:18.384897    2565 generic.go:356] "Generic (PLEG): container finished" podID="bffd2a49-ec4f-44ee-a47c-19180aaeeec2" containerID="4527ae4da8703ca91d61dcb1d69ade60bf2373182153c2fef8937924d6f8815f" exitCode=0
Jul 01 16:59:18.385016 ip-10-0-137-244 kubenswrapper[2565]: I0701 16:59:18.384943    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-kzwpl" event={"ID":"bffd2a49-ec4f-44ee-a47c-19180aaeeec2","Type":"ContainerDied","Data":"4527ae4da8703ca91d61dcb1d69ade60bf2373182153c2fef8937924d6f8815f"}
Jul 01 16:59:18.398782 ip-10-0-137-244 kubenswrapper[2565]: I0701 16:59:18.398760    2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-zjsw4"
Jul 01 16:59:19.243058 ip-10-0-137-244 kubenswrapper[2565]: I0701 16:59:19.243015    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-hthvc"
Jul 01 16:59:19.243284 ip-10-0-137-244 kubenswrapper[2565]: E0701 16:59:19.243148    2565 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-hthvc" podUID="a6488c7c-9ea5-4ad2-91ba-b1ffda18b203"
Jul 01 16:59:19.243284 ip-10-0-137-244 kubenswrapper[2565]: I0701 16:59:19.243198    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-vfng9"
Jul 01 16:59:19.243376 ip-10-0-137-244 kubenswrapper[2565]: E0701 16:59:19.243302    2565 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-vfng9" podUID="b2746391-676c-4add-9406-a60838c1ac2c"
Jul 01 16:59:19.389751 ip-10-0-137-244 kubenswrapper[2565]: I0701 16:59:19.389729    2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-zjsw4_bb7ebc32-d0cd-4eb0-b998-3ce298f73809/ovn-acl-logging/0.log"
Jul 01 16:59:19.390252 ip-10-0-137-244 kubenswrapper[2565]: I0701 16:59:19.390037    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-zjsw4" event={"ID":"bb7ebc32-d0cd-4eb0-b998-3ce298f73809","Type":"ContainerStarted","Data":"51148517a83ad1f65f0a68a3e77c2fcb81609056a2bdeae867290af37f9e3a7e"}
Jul 01 16:59:19.390252 ip-10-0-137-244 kubenswrapper[2565]: I0701 16:59:19.390210    2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-zjsw4"
Jul 01 16:59:19.404346 ip-10-0-137-244 kubenswrapper[2565]: I0701 16:59:19.404327    2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-zjsw4"
Jul 01 16:59:19.471437 ip-10-0-137-244 kubenswrapper[2565]: I0701 16:59:19.471391    2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ovn-kubernetes/ovnkube-node-zjsw4" podStartSLOduration=9.142544168 podStartE2EDuration="29.471377659s" totalImagesPullingTime="20.328833491s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-01 16:58:50 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-01 16:59:19.470160238 +0000 UTC m=+29.938424374" watchObservedRunningTime="2026-07-01 16:59:19.471377659 +0000 UTC m=+29.939641782"
Jul 01 16:59:20.127968 ip-10-0-137-244 kubenswrapper[2565]: I0701 16:59:20.127806    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-hthvc"]
Jul 01 16:59:20.128158 ip-10-0-137-244 kubenswrapper[2565]: I0701 16:59:20.128073    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-hthvc"
Jul 01 16:59:20.128217 ip-10-0-137-244 kubenswrapper[2565]: E0701 16:59:20.128188    2565 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-hthvc" podUID="a6488c7c-9ea5-4ad2-91ba-b1ffda18b203"
Jul 01 16:59:20.138648 ip-10-0-137-244 kubenswrapper[2565]: I0701 16:59:20.138624    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-vfng9"]
Jul 01 16:59:20.138745 ip-10-0-137-244 kubenswrapper[2565]: I0701 16:59:20.138703    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-vfng9"
Jul 01 16:59:20.138816 ip-10-0-137-244 kubenswrapper[2565]: E0701 16:59:20.138792    2565 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-vfng9" podUID="b2746391-676c-4add-9406-a60838c1ac2c"
Jul 01 16:59:20.148109 ip-10-0-137-244 kubenswrapper[2565]: I0701 16:59:20.148063    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-fvqs4"]
Jul 01 16:59:20.148183 ip-10-0-137-244 kubenswrapper[2565]: I0701 16:59:20.148171    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-fvqs4"
Jul 01 16:59:20.148326 ip-10-0-137-244 kubenswrapper[2565]: E0701 16:59:20.148259    2565 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-fvqs4" podUID="43a7325b-d5b7-492e-961d-52f0b9b32a17"
Jul 01 16:59:20.393358 ip-10-0-137-244 kubenswrapper[2565]: I0701 16:59:20.393329    2565 generic.go:356] "Generic (PLEG): container finished" podID="bffd2a49-ec4f-44ee-a47c-19180aaeeec2" containerID="603d9b7cc85b90646c685ac75dbb2506594206a4ae011c50ad9d853704dbdbd3" exitCode=0
Jul 01 16:59:20.393756 ip-10-0-137-244 kubenswrapper[2565]: I0701 16:59:20.393419    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-kzwpl" event={"ID":"bffd2a49-ec4f-44ee-a47c-19180aaeeec2","Type":"ContainerDied","Data":"603d9b7cc85b90646c685ac75dbb2506594206a4ae011c50ad9d853704dbdbd3"}
Jul 01 16:59:22.246601 ip-10-0-137-244 kubenswrapper[2565]: I0701 16:59:22.246578    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-fvqs4"
Jul 01 16:59:22.246936 ip-10-0-137-244 kubenswrapper[2565]: I0701 16:59:22.246579    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-hthvc"
Jul 01 16:59:22.246936 ip-10-0-137-244 kubenswrapper[2565]: E0701 16:59:22.246660    2565 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-fvqs4" podUID="43a7325b-d5b7-492e-961d-52f0b9b32a17"
Jul 01 16:59:22.246936 ip-10-0-137-244 kubenswrapper[2565]: E0701 16:59:22.246752    2565 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-hthvc" podUID="a6488c7c-9ea5-4ad2-91ba-b1ffda18b203"
Jul 01 16:59:22.246936 ip-10-0-137-244 kubenswrapper[2565]: I0701 16:59:22.246578    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-vfng9"
Jul 01 16:59:22.246936 ip-10-0-137-244 kubenswrapper[2565]: E0701 16:59:22.246827    2565 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-vfng9" podUID="b2746391-676c-4add-9406-a60838c1ac2c"
Jul 01 16:59:22.403217 ip-10-0-137-244 kubenswrapper[2565]: I0701 16:59:22.403193    2565 generic.go:356] "Generic (PLEG): container finished" podID="bffd2a49-ec4f-44ee-a47c-19180aaeeec2" containerID="5f876287970d320f0c38fb05d1d7e5832344b29676f981b6a01bcf2d9bad5b0b" exitCode=0
Jul 01 16:59:22.403334 ip-10-0-137-244 kubenswrapper[2565]: I0701 16:59:22.403243    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-kzwpl" event={"ID":"bffd2a49-ec4f-44ee-a47c-19180aaeeec2","Type":"ContainerDied","Data":"5f876287970d320f0c38fb05d1d7e5832344b29676f981b6a01bcf2d9bad5b0b"}
Jul 01 16:59:22.865582 ip-10-0-137-244 kubenswrapper[2565]: I0701 16:59:22.865556    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/a6488c7c-9ea5-4ad2-91ba-b1ffda18b203-metrics-certs\") pod \"network-metrics-daemon-hthvc\" (UID: \"a6488c7c-9ea5-4ad2-91ba-b1ffda18b203\") " pod="openshift-multus/network-metrics-daemon-hthvc"
Jul 01 16:59:22.865694 ip-10-0-137-244 kubenswrapper[2565]: I0701 16:59:22.865588    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/b2746391-676c-4add-9406-a60838c1ac2c-original-pull-secret\") pod \"global-pull-secret-syncer-vfng9\" (UID: \"b2746391-676c-4add-9406-a60838c1ac2c\") " pod="kube-system/global-pull-secret-syncer-vfng9"
Jul 01 16:59:22.865694 ip-10-0-137-244 kubenswrapper[2565]: E0701 16:59:22.865688    2565 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 01 16:59:22.865767 ip-10-0-137-244 kubenswrapper[2565]: E0701 16:59:22.865737    2565 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b2746391-676c-4add-9406-a60838c1ac2c-original-pull-secret podName:b2746391-676c-4add-9406-a60838c1ac2c nodeName:}" failed. No retries permitted until 2026-07-01 16:59:54.865723895 +0000 UTC m=+65.333987998 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/b2746391-676c-4add-9406-a60838c1ac2c-original-pull-secret") pod "global-pull-secret-syncer-vfng9" (UID: "b2746391-676c-4add-9406-a60838c1ac2c") : object "kube-system"/"original-pull-secret" not registered
Jul 01 16:59:22.865819 ip-10-0-137-244 kubenswrapper[2565]: E0701 16:59:22.865689    2565 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 01 16:59:22.865819 ip-10-0-137-244 kubenswrapper[2565]: E0701 16:59:22.865814    2565 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a6488c7c-9ea5-4ad2-91ba-b1ffda18b203-metrics-certs podName:a6488c7c-9ea5-4ad2-91ba-b1ffda18b203 nodeName:}" failed. No retries permitted until 2026-07-01 16:59:54.865801272 +0000 UTC m=+65.334065377 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/a6488c7c-9ea5-4ad2-91ba-b1ffda18b203-metrics-certs") pod "network-metrics-daemon-hthvc" (UID: "a6488c7c-9ea5-4ad2-91ba-b1ffda18b203") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 01 16:59:22.966769 ip-10-0-137-244 kubenswrapper[2565]: I0701 16:59:22.966745    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9rt5m\" (UniqueName: \"kubernetes.io/projected/43a7325b-d5b7-492e-961d-52f0b9b32a17-kube-api-access-9rt5m\") pod \"network-check-target-fvqs4\" (UID: \"43a7325b-d5b7-492e-961d-52f0b9b32a17\") " pod="openshift-network-diagnostics/network-check-target-fvqs4"
Jul 01 16:59:22.966874 ip-10-0-137-244 kubenswrapper[2565]: E0701 16:59:22.966856    2565 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 01 16:59:22.966874 ip-10-0-137-244 kubenswrapper[2565]: E0701 16:59:22.966868    2565 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 01 16:59:22.966941 ip-10-0-137-244 kubenswrapper[2565]: E0701 16:59:22.966876    2565 projected.go:196] Error preparing data for projected volume kube-api-access-9rt5m for pod openshift-network-diagnostics/network-check-target-fvqs4: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 01 16:59:22.966941 ip-10-0-137-244 kubenswrapper[2565]: E0701 16:59:22.966914    2565 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/43a7325b-d5b7-492e-961d-52f0b9b32a17-kube-api-access-9rt5m podName:43a7325b-d5b7-492e-961d-52f0b9b32a17 nodeName:}" failed. No retries permitted until 2026-07-01 16:59:54.966901979 +0000 UTC m=+65.435166081 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "kube-api-access-9rt5m" (UniqueName: "kubernetes.io/projected/43a7325b-d5b7-492e-961d-52f0b9b32a17-kube-api-access-9rt5m") pod "network-check-target-fvqs4" (UID: "43a7325b-d5b7-492e-961d-52f0b9b32a17") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 01 16:59:24.243627 ip-10-0-137-244 kubenswrapper[2565]: I0701 16:59:24.243595    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-vfng9"
Jul 01 16:59:24.244122 ip-10-0-137-244 kubenswrapper[2565]: I0701 16:59:24.243595    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-fvqs4"
Jul 01 16:59:24.244122 ip-10-0-137-244 kubenswrapper[2565]: E0701 16:59:24.243745    2565 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-vfng9" podUID="b2746391-676c-4add-9406-a60838c1ac2c"
Jul 01 16:59:24.244122 ip-10-0-137-244 kubenswrapper[2565]: I0701 16:59:24.243600    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-hthvc"
Jul 01 16:59:24.244122 ip-10-0-137-244 kubenswrapper[2565]: E0701 16:59:24.243807    2565 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-fvqs4" podUID="43a7325b-d5b7-492e-961d-52f0b9b32a17"
Jul 01 16:59:24.244122 ip-10-0-137-244 kubenswrapper[2565]: E0701 16:59:24.243913    2565 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-hthvc" podUID="a6488c7c-9ea5-4ad2-91ba-b1ffda18b203"
Jul 01 16:59:24.860898 ip-10-0-137-244 kubenswrapper[2565]: I0701 16:59:24.860866    2565 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-137-244.ec2.internal" event="NodeReady"
Jul 01 16:59:24.861053 ip-10-0-137-244 kubenswrapper[2565]: I0701 16:59:24.860993    2565 kubelet_node_status.go:489] "Fast updating node status as it just became ready"
Jul 01 16:59:24.944917 ip-10-0-137-244 kubenswrapper[2565]: I0701 16:59:24.944843    2565 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-dns/dns-default-544rz"]
Jul 01 16:59:24.947073 ip-10-0-137-244 kubenswrapper[2565]: I0701 16:59:24.947054    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-544rz"
Jul 01 16:59:24.950562 ip-10-0-137-244 kubenswrapper[2565]: I0701 16:59:24.950534    2565 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress-canary/ingress-canary-4dfnf"]
Jul 01 16:59:24.951546 ip-10-0-137-244 kubenswrapper[2565]: I0701 16:59:24.951531    2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-default-metrics-tls\""
Jul 01 16:59:24.951726 ip-10-0-137-244 kubenswrapper[2565]: I0701 16:59:24.951689    2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-96p59\""
Jul 01 16:59:24.951921 ip-10-0-137-244 kubenswrapper[2565]: I0701 16:59:24.951907    2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"dns-default\""
Jul 01 16:59:24.952386 ip-10-0-137-244 kubenswrapper[2565]: I0701 16:59:24.952369    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-4dfnf"
Jul 01 16:59:24.959123 ip-10-0-137-244 kubenswrapper[2565]: I0701 16:59:24.959100    2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"canary-serving-cert\""
Jul 01 16:59:24.959218 ip-10-0-137-244 kubenswrapper[2565]: I0701 16:59:24.959104    2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"kube-root-ca.crt\""
Jul 01 16:59:24.959356 ip-10-0-137-244 kubenswrapper[2565]: I0701 16:59:24.959337    2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"openshift-service-ca.crt\""
Jul 01 16:59:24.959775 ip-10-0-137-244 kubenswrapper[2565]: I0701 16:59:24.959756    2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"default-dockercfg-7lmd6\""
Jul 01 16:59:24.966075 ip-10-0-137-244 kubenswrapper[2565]: I0701 16:59:24.966055    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-4dfnf"]
Jul 01 16:59:24.969837 ip-10-0-137-244 kubenswrapper[2565]: I0701 16:59:24.969784    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-544rz"]
Jul 01 16:59:25.082985 ip-10-0-137-244 kubenswrapper[2565]: I0701 16:59:25.082957    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/c91603a3-7fb8-488e-905d-c2b2791f4f8c-metrics-tls\") pod \"dns-default-544rz\" (UID: \"c91603a3-7fb8-488e-905d-c2b2791f4f8c\") " pod="openshift-dns/dns-default-544rz"
Jul 01 16:59:25.083149 ip-10-0-137-244 kubenswrapper[2565]: I0701 16:59:25.083052    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-f49s2\" (UniqueName: \"kubernetes.io/projected/b433bedb-7f27-4ad0-a5ce-6d573c5aa004-kube-api-access-f49s2\") pod \"ingress-canary-4dfnf\" (UID: \"b433bedb-7f27-4ad0-a5ce-6d573c5aa004\") " pod="openshift-ingress-canary/ingress-canary-4dfnf"
Jul 01 16:59:25.083149 ip-10-0-137-244 kubenswrapper[2565]: I0701 16:59:25.083087    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/b433bedb-7f27-4ad0-a5ce-6d573c5aa004-cert\") pod \"ingress-canary-4dfnf\" (UID: \"b433bedb-7f27-4ad0-a5ce-6d573c5aa004\") " pod="openshift-ingress-canary/ingress-canary-4dfnf"
Jul 01 16:59:25.083149 ip-10-0-137-244 kubenswrapper[2565]: I0701 16:59:25.083145    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/c91603a3-7fb8-488e-905d-c2b2791f4f8c-config-volume\") pod \"dns-default-544rz\" (UID: \"c91603a3-7fb8-488e-905d-c2b2791f4f8c\") " pod="openshift-dns/dns-default-544rz"
Jul 01 16:59:25.083317 ip-10-0-137-244 kubenswrapper[2565]: I0701 16:59:25.083176    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-56l44\" (UniqueName: \"kubernetes.io/projected/c91603a3-7fb8-488e-905d-c2b2791f4f8c-kube-api-access-56l44\") pod \"dns-default-544rz\" (UID: \"c91603a3-7fb8-488e-905d-c2b2791f4f8c\") " pod="openshift-dns/dns-default-544rz"
Jul 01 16:59:25.083317 ip-10-0-137-244 kubenswrapper[2565]: I0701 16:59:25.083193    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/c91603a3-7fb8-488e-905d-c2b2791f4f8c-tmp-dir\") pod \"dns-default-544rz\" (UID: \"c91603a3-7fb8-488e-905d-c2b2791f4f8c\") " pod="openshift-dns/dns-default-544rz"
Jul 01 16:59:25.184305 ip-10-0-137-244 kubenswrapper[2565]: I0701 16:59:25.184272    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-56l44\" (UniqueName: \"kubernetes.io/projected/c91603a3-7fb8-488e-905d-c2b2791f4f8c-kube-api-access-56l44\") pod \"dns-default-544rz\" (UID: \"c91603a3-7fb8-488e-905d-c2b2791f4f8c\") " pod="openshift-dns/dns-default-544rz"
Jul 01 16:59:25.184469 ip-10-0-137-244 kubenswrapper[2565]: I0701 16:59:25.184317    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/c91603a3-7fb8-488e-905d-c2b2791f4f8c-tmp-dir\") pod \"dns-default-544rz\" (UID: \"c91603a3-7fb8-488e-905d-c2b2791f4f8c\") " pod="openshift-dns/dns-default-544rz"
Jul 01 16:59:25.184469 ip-10-0-137-244 kubenswrapper[2565]: I0701 16:59:25.184381    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/c91603a3-7fb8-488e-905d-c2b2791f4f8c-metrics-tls\") pod \"dns-default-544rz\" (UID: \"c91603a3-7fb8-488e-905d-c2b2791f4f8c\") " pod="openshift-dns/dns-default-544rz"
Jul 01 16:59:25.184469 ip-10-0-137-244 kubenswrapper[2565]: I0701 16:59:25.184420    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-f49s2\" (UniqueName: \"kubernetes.io/projected/b433bedb-7f27-4ad0-a5ce-6d573c5aa004-kube-api-access-f49s2\") pod \"ingress-canary-4dfnf\" (UID: \"b433bedb-7f27-4ad0-a5ce-6d573c5aa004\") " pod="openshift-ingress-canary/ingress-canary-4dfnf"
Jul 01 16:59:25.184469 ip-10-0-137-244 kubenswrapper[2565]: I0701 16:59:25.184450    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/b433bedb-7f27-4ad0-a5ce-6d573c5aa004-cert\") pod \"ingress-canary-4dfnf\" (UID: \"b433bedb-7f27-4ad0-a5ce-6d573c5aa004\") " pod="openshift-ingress-canary/ingress-canary-4dfnf"
Jul 01 16:59:25.184628 ip-10-0-137-244 kubenswrapper[2565]: I0701 16:59:25.184489    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/c91603a3-7fb8-488e-905d-c2b2791f4f8c-config-volume\") pod \"dns-default-544rz\" (UID: \"c91603a3-7fb8-488e-905d-c2b2791f4f8c\") " pod="openshift-dns/dns-default-544rz"
Jul 01 16:59:25.184628 ip-10-0-137-244 kubenswrapper[2565]: E0701 16:59:25.184581    2565 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Jul 01 16:59:25.184688 ip-10-0-137-244 kubenswrapper[2565]: E0701 16:59:25.184647    2565 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/c91603a3-7fb8-488e-905d-c2b2791f4f8c-metrics-tls podName:c91603a3-7fb8-488e-905d-c2b2791f4f8c nodeName:}" failed. No retries permitted until 2026-07-01 16:59:25.684628709 +0000 UTC m=+36.152892811 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/c91603a3-7fb8-488e-905d-c2b2791f4f8c-metrics-tls") pod "dns-default-544rz" (UID: "c91603a3-7fb8-488e-905d-c2b2791f4f8c") : secret "dns-default-metrics-tls" not found
Jul 01 16:59:25.185061 ip-10-0-137-244 kubenswrapper[2565]: I0701 16:59:25.185016    2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/c91603a3-7fb8-488e-905d-c2b2791f4f8c-tmp-dir\") pod \"dns-default-544rz\" (UID: \"c91603a3-7fb8-488e-905d-c2b2791f4f8c\") " pod="openshift-dns/dns-default-544rz"
Jul 01 16:59:25.185222 ip-10-0-137-244 kubenswrapper[2565]: I0701 16:59:25.185201    2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/c91603a3-7fb8-488e-905d-c2b2791f4f8c-config-volume\") pod \"dns-default-544rz\" (UID: \"c91603a3-7fb8-488e-905d-c2b2791f4f8c\") " pod="openshift-dns/dns-default-544rz"
Jul 01 16:59:25.189727 ip-10-0-137-244 kubenswrapper[2565]: I0701 16:59:25.189700    2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/b433bedb-7f27-4ad0-a5ce-6d573c5aa004-cert\") pod \"ingress-canary-4dfnf\" (UID: \"b433bedb-7f27-4ad0-a5ce-6d573c5aa004\") " pod="openshift-ingress-canary/ingress-canary-4dfnf"
Jul 01 16:59:25.193872 ip-10-0-137-244 kubenswrapper[2565]: I0701 16:59:25.193852    2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-56l44\" (UniqueName: \"kubernetes.io/projected/c91603a3-7fb8-488e-905d-c2b2791f4f8c-kube-api-access-56l44\") pod \"dns-default-544rz\" (UID: \"c91603a3-7fb8-488e-905d-c2b2791f4f8c\") " pod="openshift-dns/dns-default-544rz"
Jul 01 16:59:25.199664 ip-10-0-137-244 kubenswrapper[2565]: I0701 16:59:25.199507    2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-f49s2\" (UniqueName: \"kubernetes.io/projected/b433bedb-7f27-4ad0-a5ce-6d573c5aa004-kube-api-access-f49s2\") pod \"ingress-canary-4dfnf\" (UID: \"b433bedb-7f27-4ad0-a5ce-6d573c5aa004\") " pod="openshift-ingress-canary/ingress-canary-4dfnf"
Jul 01 16:59:25.266583 ip-10-0-137-244 kubenswrapper[2565]: I0701 16:59:25.266562    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-4dfnf"
Jul 01 16:59:25.688081 ip-10-0-137-244 kubenswrapper[2565]: I0701 16:59:25.688045    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/c91603a3-7fb8-488e-905d-c2b2791f4f8c-metrics-tls\") pod \"dns-default-544rz\" (UID: \"c91603a3-7fb8-488e-905d-c2b2791f4f8c\") " pod="openshift-dns/dns-default-544rz"
Jul 01 16:59:25.688268 ip-10-0-137-244 kubenswrapper[2565]: E0701 16:59:25.688185    2565 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Jul 01 16:59:25.688268 ip-10-0-137-244 kubenswrapper[2565]: E0701 16:59:25.688261    2565 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/c91603a3-7fb8-488e-905d-c2b2791f4f8c-metrics-tls podName:c91603a3-7fb8-488e-905d-c2b2791f4f8c nodeName:}" failed. No retries permitted until 2026-07-01 16:59:26.688239453 +0000 UTC m=+37.156503558 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/c91603a3-7fb8-488e-905d-c2b2791f4f8c-metrics-tls") pod "dns-default-544rz" (UID: "c91603a3-7fb8-488e-905d-c2b2791f4f8c") : secret "dns-default-metrics-tls" not found
Jul 01 16:59:26.243626 ip-10-0-137-244 kubenswrapper[2565]: I0701 16:59:26.243588    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-hthvc"
Jul 01 16:59:26.243801 ip-10-0-137-244 kubenswrapper[2565]: I0701 16:59:26.243589    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-fvqs4"
Jul 01 16:59:26.243801 ip-10-0-137-244 kubenswrapper[2565]: I0701 16:59:26.243589    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-vfng9"
Jul 01 16:59:26.247299 ip-10-0-137-244 kubenswrapper[2565]: I0701 16:59:26.247274    2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\""
Jul 01 16:59:26.247441 ip-10-0-137-244 kubenswrapper[2565]: I0701 16:59:26.247307    2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-rp97d\""
Jul 01 16:59:26.248202 ip-10-0-137-244 kubenswrapper[2565]: I0701 16:59:26.248175    2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-nt4t9\""
Jul 01 16:59:26.248202 ip-10-0-137-244 kubenswrapper[2565]: I0701 16:59:26.248183    2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\""
Jul 01 16:59:26.248202 ip-10-0-137-244 kubenswrapper[2565]: I0701 16:59:26.248206    2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\""
Jul 01 16:59:26.248402 ip-10-0-137-244 kubenswrapper[2565]: I0701 16:59:26.248268    2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\""
Jul 01 16:59:26.651215 ip-10-0-137-244 kubenswrapper[2565]: I0701 16:59:26.651189    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-4dfnf"]
Jul 01 16:59:26.654552 ip-10-0-137-244 kubenswrapper[2565]: W0701 16:59:26.654519    2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb433bedb_7f27_4ad0_a5ce_6d573c5aa004.slice/crio-e0f31a5e8aa21957ac0870bb46aafa6087df780f09f127c9153623a7f7d19e28 WatchSource:0}: Error finding container e0f31a5e8aa21957ac0870bb46aafa6087df780f09f127c9153623a7f7d19e28: Status 404 returned error can't find the container with id e0f31a5e8aa21957ac0870bb46aafa6087df780f09f127c9153623a7f7d19e28
Jul 01 16:59:26.694881 ip-10-0-137-244 kubenswrapper[2565]: I0701 16:59:26.694852    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/c91603a3-7fb8-488e-905d-c2b2791f4f8c-metrics-tls\") pod \"dns-default-544rz\" (UID: \"c91603a3-7fb8-488e-905d-c2b2791f4f8c\") " pod="openshift-dns/dns-default-544rz"
Jul 01 16:59:26.694988 ip-10-0-137-244 kubenswrapper[2565]: E0701 16:59:26.694955    2565 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Jul 01 16:59:26.695064 ip-10-0-137-244 kubenswrapper[2565]: E0701 16:59:26.695005    2565 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/c91603a3-7fb8-488e-905d-c2b2791f4f8c-metrics-tls podName:c91603a3-7fb8-488e-905d-c2b2791f4f8c nodeName:}" failed. No retries permitted until 2026-07-01 16:59:28.694990424 +0000 UTC m=+39.163254527 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/c91603a3-7fb8-488e-905d-c2b2791f4f8c-metrics-tls") pod "dns-default-544rz" (UID: "c91603a3-7fb8-488e-905d-c2b2791f4f8c") : secret "dns-default-metrics-tls" not found
Jul 01 16:59:27.419351 ip-10-0-137-244 kubenswrapper[2565]: I0701 16:59:27.419119    2565 generic.go:356] "Generic (PLEG): container finished" podID="bffd2a49-ec4f-44ee-a47c-19180aaeeec2" containerID="f843b707f4112aa887a1a3f8e6dbf9c09c72ded0bf13f6359f2e6efc12c54b98" exitCode=0
Jul 01 16:59:27.419521 ip-10-0-137-244 kubenswrapper[2565]: I0701 16:59:27.419206    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-kzwpl" event={"ID":"bffd2a49-ec4f-44ee-a47c-19180aaeeec2","Type":"ContainerDied","Data":"f843b707f4112aa887a1a3f8e6dbf9c09c72ded0bf13f6359f2e6efc12c54b98"}
Jul 01 16:59:27.420704 ip-10-0-137-244 kubenswrapper[2565]: I0701 16:59:27.420674    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-4dfnf" event={"ID":"b433bedb-7f27-4ad0-a5ce-6d573c5aa004","Type":"ContainerStarted","Data":"e0f31a5e8aa21957ac0870bb46aafa6087df780f09f127c9153623a7f7d19e28"}
Jul 01 16:59:28.427415 ip-10-0-137-244 kubenswrapper[2565]: I0701 16:59:28.427381    2565 generic.go:356] "Generic (PLEG): container finished" podID="bffd2a49-ec4f-44ee-a47c-19180aaeeec2" containerID="aeb6c8bd35d656207b87ad942653ba1448efc2b1fac0c104f6909b3d1d9db4b2" exitCode=0
Jul 01 16:59:28.427845 ip-10-0-137-244 kubenswrapper[2565]: I0701 16:59:28.427464    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-kzwpl" event={"ID":"bffd2a49-ec4f-44ee-a47c-19180aaeeec2","Type":"ContainerDied","Data":"aeb6c8bd35d656207b87ad942653ba1448efc2b1fac0c104f6909b3d1d9db4b2"}
Jul 01 16:59:28.709809 ip-10-0-137-244 kubenswrapper[2565]: I0701 16:59:28.709734    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/c91603a3-7fb8-488e-905d-c2b2791f4f8c-metrics-tls\") pod \"dns-default-544rz\" (UID: \"c91603a3-7fb8-488e-905d-c2b2791f4f8c\") " pod="openshift-dns/dns-default-544rz"
Jul 01 16:59:28.709940 ip-10-0-137-244 kubenswrapper[2565]: E0701 16:59:28.709879    2565 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Jul 01 16:59:28.709979 ip-10-0-137-244 kubenswrapper[2565]: E0701 16:59:28.709941    2565 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/c91603a3-7fb8-488e-905d-c2b2791f4f8c-metrics-tls podName:c91603a3-7fb8-488e-905d-c2b2791f4f8c nodeName:}" failed. No retries permitted until 2026-07-01 16:59:32.709925497 +0000 UTC m=+43.178189603 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/c91603a3-7fb8-488e-905d-c2b2791f4f8c-metrics-tls") pod "dns-default-544rz" (UID: "c91603a3-7fb8-488e-905d-c2b2791f4f8c") : secret "dns-default-metrics-tls" not found
Jul 01 16:59:29.433448 ip-10-0-137-244 kubenswrapper[2565]: I0701 16:59:29.433376    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-kzwpl" event={"ID":"bffd2a49-ec4f-44ee-a47c-19180aaeeec2","Type":"ContainerStarted","Data":"2b4b907d87e269d6d1f945671b1535e043495063ed338d497d246d988b5d54de"}
Jul 01 16:59:29.434682 ip-10-0-137-244 kubenswrapper[2565]: I0701 16:59:29.434654    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-4dfnf" event={"ID":"b433bedb-7f27-4ad0-a5ce-6d573c5aa004","Type":"ContainerStarted","Data":"7e5a720db73d116015d6781f7e08d14dc938e01b3a98ada082f53ba9450b4900"}
Jul 01 16:59:29.489613 ip-10-0-137-244 kubenswrapper[2565]: I0701 16:59:29.489570    2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-additional-cni-plugins-kzwpl" podStartSLOduration=12.769319762 podStartE2EDuration="39.48955715s" totalImagesPullingTime="26.720237388s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-01 16:58:50 +0000 UTC" imagePullSessionsCount=5 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-01 16:59:29.463622931 +0000 UTC m=+39.931887055" watchObservedRunningTime="2026-07-01 16:59:29.48955715 +0000 UTC m=+39.957821324"
Jul 01 16:59:29.489726 ip-10-0-137-244 kubenswrapper[2565]: I0701 16:59:29.489708    2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress-canary/ingress-canary-4dfnf" podStartSLOduration=3.00995502 podStartE2EDuration="5.48969908s" totalImagesPullingTime="2.47974406s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-01 16:59:24 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-01 16:59:29.488598501 +0000 UTC m=+39.956862624" watchObservedRunningTime="2026-07-01 16:59:29.48969908 +0000 UTC m=+39.957963218"
Jul 01 16:59:32.739159 ip-10-0-137-244 kubenswrapper[2565]: I0701 16:59:32.739114    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/c91603a3-7fb8-488e-905d-c2b2791f4f8c-metrics-tls\") pod \"dns-default-544rz\" (UID: \"c91603a3-7fb8-488e-905d-c2b2791f4f8c\") " pod="openshift-dns/dns-default-544rz"
Jul 01 16:59:32.739565 ip-10-0-137-244 kubenswrapper[2565]: E0701 16:59:32.739271    2565 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Jul 01 16:59:32.739565 ip-10-0-137-244 kubenswrapper[2565]: E0701 16:59:32.739345    2565 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/c91603a3-7fb8-488e-905d-c2b2791f4f8c-metrics-tls podName:c91603a3-7fb8-488e-905d-c2b2791f4f8c nodeName:}" failed. No retries permitted until 2026-07-01 16:59:40.739325396 +0000 UTC m=+51.207589501 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/c91603a3-7fb8-488e-905d-c2b2791f4f8c-metrics-tls") pod "dns-default-544rz" (UID: "c91603a3-7fb8-488e-905d-c2b2791f4f8c") : secret "dns-default-metrics-tls" not found
Jul 01 16:59:40.785612 ip-10-0-137-244 kubenswrapper[2565]: I0701 16:59:40.785566    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/c91603a3-7fb8-488e-905d-c2b2791f4f8c-metrics-tls\") pod \"dns-default-544rz\" (UID: \"c91603a3-7fb8-488e-905d-c2b2791f4f8c\") " pod="openshift-dns/dns-default-544rz"
Jul 01 16:59:40.786165 ip-10-0-137-244 kubenswrapper[2565]: E0701 16:59:40.785702    2565 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Jul 01 16:59:40.786165 ip-10-0-137-244 kubenswrapper[2565]: E0701 16:59:40.785775    2565 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/c91603a3-7fb8-488e-905d-c2b2791f4f8c-metrics-tls podName:c91603a3-7fb8-488e-905d-c2b2791f4f8c nodeName:}" failed. No retries permitted until 2026-07-01 16:59:56.78575663 +0000 UTC m=+67.254020737 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/c91603a3-7fb8-488e-905d-c2b2791f4f8c-metrics-tls") pod "dns-default-544rz" (UID: "c91603a3-7fb8-488e-905d-c2b2791f4f8c") : secret "dns-default-metrics-tls" not found
Jul 01 16:59:45.577959 ip-10-0-137-244 kubenswrapper[2565]: I0701 16:59:45.577924    2565 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/cluster-monitoring-operator-54cb84584-t2hmp"]
Jul 01 16:59:45.579771 ip-10-0-137-244 kubenswrapper[2565]: I0701 16:59:45.579748    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/cluster-monitoring-operator-54cb84584-t2hmp"
Jul 01 16:59:45.582347 ip-10-0-137-244 kubenswrapper[2565]: I0701 16:59:45.582325    2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"cluster-monitoring-operator-dockercfg-2bpn2\""
Jul 01 16:59:45.583670 ip-10-0-137-244 kubenswrapper[2565]: I0701 16:59:45.583639    2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"cluster-monitoring-operator-tls\""
Jul 01 16:59:45.583742 ip-10-0-137-244 kubenswrapper[2565]: I0701 16:59:45.583717    2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kube-root-ca.crt\""
Jul 01 16:59:45.585002 ip-10-0-137-244 kubenswrapper[2565]: I0701 16:59:45.584979    2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"openshift-service-ca.crt\""
Jul 01 16:59:45.585151 ip-10-0-137-244 kubenswrapper[2565]: I0701 16:59:45.585056    2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"telemetry-config\""
Jul 01 16:59:45.600069 ip-10-0-137-244 kubenswrapper[2565]: I0701 16:59:45.600049    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/cluster-monitoring-operator-54cb84584-t2hmp"]
Jul 01 16:59:45.615745 ip-10-0-137-244 kubenswrapper[2565]: I0701 16:59:45.615700    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"telemetry-config\" (UniqueName: \"kubernetes.io/configmap/cbc9a766-e08a-4666-b4e5-f3db8dcadc45-telemetry-config\") pod \"cluster-monitoring-operator-54cb84584-t2hmp\" (UID: \"cbc9a766-e08a-4666-b4e5-f3db8dcadc45\") " pod="openshift-monitoring/cluster-monitoring-operator-54cb84584-t2hmp"
Jul 01 16:59:45.615903 ip-10-0-137-244 kubenswrapper[2565]: I0701 16:59:45.615871    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vzfsl\" (UniqueName: \"kubernetes.io/projected/cbc9a766-e08a-4666-b4e5-f3db8dcadc45-kube-api-access-vzfsl\") pod \"cluster-monitoring-operator-54cb84584-t2hmp\" (UID: \"cbc9a766-e08a-4666-b4e5-f3db8dcadc45\") " pod="openshift-monitoring/cluster-monitoring-operator-54cb84584-t2hmp"
Jul 01 16:59:45.616195 ip-10-0-137-244 kubenswrapper[2565]: I0701 16:59:45.616095    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/cbc9a766-e08a-4666-b4e5-f3db8dcadc45-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-54cb84584-t2hmp\" (UID: \"cbc9a766-e08a-4666-b4e5-f3db8dcadc45\") " pod="openshift-monitoring/cluster-monitoring-operator-54cb84584-t2hmp"
Jul 01 16:59:45.716951 ip-10-0-137-244 kubenswrapper[2565]: I0701 16:59:45.716925    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/cbc9a766-e08a-4666-b4e5-f3db8dcadc45-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-54cb84584-t2hmp\" (UID: \"cbc9a766-e08a-4666-b4e5-f3db8dcadc45\") " pod="openshift-monitoring/cluster-monitoring-operator-54cb84584-t2hmp"
Jul 01 16:59:45.717080 ip-10-0-137-244 kubenswrapper[2565]: I0701 16:59:45.716981    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"telemetry-config\" (UniqueName: \"kubernetes.io/configmap/cbc9a766-e08a-4666-b4e5-f3db8dcadc45-telemetry-config\") pod \"cluster-monitoring-operator-54cb84584-t2hmp\" (UID: \"cbc9a766-e08a-4666-b4e5-f3db8dcadc45\") " pod="openshift-monitoring/cluster-monitoring-operator-54cb84584-t2hmp"
Jul 01 16:59:45.717080 ip-10-0-137-244 kubenswrapper[2565]: I0701 16:59:45.717001    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vzfsl\" (UniqueName: \"kubernetes.io/projected/cbc9a766-e08a-4666-b4e5-f3db8dcadc45-kube-api-access-vzfsl\") pod \"cluster-monitoring-operator-54cb84584-t2hmp\" (UID: \"cbc9a766-e08a-4666-b4e5-f3db8dcadc45\") " pod="openshift-monitoring/cluster-monitoring-operator-54cb84584-t2hmp"
Jul 01 16:59:45.717154 ip-10-0-137-244 kubenswrapper[2565]: E0701 16:59:45.717086    2565 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found
Jul 01 16:59:45.717154 ip-10-0-137-244 kubenswrapper[2565]: E0701 16:59:45.717145    2565 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/cbc9a766-e08a-4666-b4e5-f3db8dcadc45-cluster-monitoring-operator-tls podName:cbc9a766-e08a-4666-b4e5-f3db8dcadc45 nodeName:}" failed. No retries permitted until 2026-07-01 16:59:46.217128732 +0000 UTC m=+56.685392835 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/cbc9a766-e08a-4666-b4e5-f3db8dcadc45-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-54cb84584-t2hmp" (UID: "cbc9a766-e08a-4666-b4e5-f3db8dcadc45") : secret "cluster-monitoring-operator-tls" not found
Jul 01 16:59:45.717682 ip-10-0-137-244 kubenswrapper[2565]: I0701 16:59:45.717664    2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"telemetry-config\" (UniqueName: \"kubernetes.io/configmap/cbc9a766-e08a-4666-b4e5-f3db8dcadc45-telemetry-config\") pod \"cluster-monitoring-operator-54cb84584-t2hmp\" (UID: \"cbc9a766-e08a-4666-b4e5-f3db8dcadc45\") " pod="openshift-monitoring/cluster-monitoring-operator-54cb84584-t2hmp"
Jul 01 16:59:45.734768 ip-10-0-137-244 kubenswrapper[2565]: I0701 16:59:45.734745    2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-vzfsl\" (UniqueName: \"kubernetes.io/projected/cbc9a766-e08a-4666-b4e5-f3db8dcadc45-kube-api-access-vzfsl\") pod \"cluster-monitoring-operator-54cb84584-t2hmp\" (UID: \"cbc9a766-e08a-4666-b4e5-f3db8dcadc45\") " pod="openshift-monitoring/cluster-monitoring-operator-54cb84584-t2hmp"
Jul 01 16:59:45.855284 ip-10-0-137-244 kubenswrapper[2565]: I0701 16:59:45.855234    2565 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-image-registry/image-registry-b84855d98-h2wv8"]
Jul 01 16:59:45.856945 ip-10-0-137-244 kubenswrapper[2565]: I0701 16:59:45.856930    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-b84855d98-h2wv8"
Jul 01 16:59:45.877529 ip-10-0-137-244 kubenswrapper[2565]: E0701 16:59:45.877504    2565 status_manager.go:1018] "Failed to get status for pod" err="pods \"image-registry-b84855d98-h2wv8\" is forbidden: User \"system:node:ip-10-0-137-244.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"openshift-image-registry\": no relationship found between node 'ip-10-0-137-244.ec2.internal' and this object" podUID="1ab9ee0c-b365-44ad-8770-ac00fe1c2c98" pod="openshift-image-registry/image-registry-b84855d98-h2wv8"
Jul 01 16:59:45.897300 ip-10-0-137-244 kubenswrapper[2565]: I0701 16:59:45.897281    2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-tls\""
Jul 01 16:59:45.897300 ip-10-0-137-244 kubenswrapper[2565]: E0701 16:59:45.897287    2565 reflector.go:205] "Failed to watch" err="failed to list *v1.ConfigMap: configmaps \"trusted-ca\" is forbidden: User \"system:node:ip-10-0-137-244.ec2.internal\" cannot list resource \"configmaps\" in API group \"\" in the namespace \"openshift-image-registry\": no relationship found between node 'ip-10-0-137-244.ec2.internal' and this object" logger="UnhandledError" reflector="object-\"openshift-image-registry\"/\"trusted-ca\"" type="*v1.ConfigMap"
Jul 01 16:59:45.897602 ip-10-0-137-244 kubenswrapper[2565]: E0701 16:59:45.897573    2565 reflector.go:205] "Failed to watch" err="failed to list *v1.Secret: secrets \"image-registry-private-configuration\" is forbidden: User \"system:node:ip-10-0-137-244.ec2.internal\" cannot list resource \"secrets\" in API group \"\" in the namespace \"openshift-image-registry\": no relationship found between node 'ip-10-0-137-244.ec2.internal' and this object" logger="UnhandledError" reflector="object-\"openshift-image-registry\"/\"image-registry-private-configuration\"" type="*v1.Secret"
Jul 01 16:59:45.897695 ip-10-0-137-244 kubenswrapper[2565]: I0701 16:59:45.897636    2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"registry-dockercfg-5lvm2\""
Jul 01 16:59:45.906575 ip-10-0-137-244 kubenswrapper[2565]: I0701 16:59:45.906545    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-b84855d98-h2wv8"]
Jul 01 16:59:45.915898 ip-10-0-137-244 kubenswrapper[2565]: I0701 16:59:45.915881    2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"installation-pull-secrets\""
Jul 01 16:59:45.918109 ip-10-0-137-244 kubenswrapper[2565]: I0701 16:59:45.918092    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/1ab9ee0c-b365-44ad-8770-ac00fe1c2c98-registry-certificates\") pod \"image-registry-b84855d98-h2wv8\" (UID: \"1ab9ee0c-b365-44ad-8770-ac00fe1c2c98\") " pod="openshift-image-registry/image-registry-b84855d98-h2wv8"
Jul 01 16:59:45.918163 ip-10-0-137-244 kubenswrapper[2565]: I0701 16:59:45.918119    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/1ab9ee0c-b365-44ad-8770-ac00fe1c2c98-trusted-ca\") pod \"image-registry-b84855d98-h2wv8\" (UID: \"1ab9ee0c-b365-44ad-8770-ac00fe1c2c98\") " pod="openshift-image-registry/image-registry-b84855d98-h2wv8"
Jul 01 16:59:45.918163 ip-10-0-137-244 kubenswrapper[2565]: I0701 16:59:45.918142    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/1ab9ee0c-b365-44ad-8770-ac00fe1c2c98-registry-tls\") pod \"image-registry-b84855d98-h2wv8\" (UID: \"1ab9ee0c-b365-44ad-8770-ac00fe1c2c98\") " pod="openshift-image-registry/image-registry-b84855d98-h2wv8"
Jul 01 16:59:45.918226 ip-10-0-137-244 kubenswrapper[2565]: I0701 16:59:45.918191    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/1ab9ee0c-b365-44ad-8770-ac00fe1c2c98-ca-trust-extracted\") pod \"image-registry-b84855d98-h2wv8\" (UID: \"1ab9ee0c-b365-44ad-8770-ac00fe1c2c98\") " pod="openshift-image-registry/image-registry-b84855d98-h2wv8"
Jul 01 16:59:45.918261 ip-10-0-137-244 kubenswrapper[2565]: I0701 16:59:45.918244    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/1ab9ee0c-b365-44ad-8770-ac00fe1c2c98-installation-pull-secrets\") pod \"image-registry-b84855d98-h2wv8\" (UID: \"1ab9ee0c-b365-44ad-8770-ac00fe1c2c98\") " pod="openshift-image-registry/image-registry-b84855d98-h2wv8"
Jul 01 16:59:45.918292 ip-10-0-137-244 kubenswrapper[2565]: I0701 16:59:45.918264    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/1ab9ee0c-b365-44ad-8770-ac00fe1c2c98-bound-sa-token\") pod \"image-registry-b84855d98-h2wv8\" (UID: \"1ab9ee0c-b365-44ad-8770-ac00fe1c2c98\") " pod="openshift-image-registry/image-registry-b84855d98-h2wv8"
Jul 01 16:59:45.918292 ip-10-0-137-244 kubenswrapper[2565]: I0701 16:59:45.918278    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-fp9nj\" (UniqueName: \"kubernetes.io/projected/1ab9ee0c-b365-44ad-8770-ac00fe1c2c98-kube-api-access-fp9nj\") pod \"image-registry-b84855d98-h2wv8\" (UID: \"1ab9ee0c-b365-44ad-8770-ac00fe1c2c98\") " pod="openshift-image-registry/image-registry-b84855d98-h2wv8"
Jul 01 16:59:45.918383 ip-10-0-137-244 kubenswrapper[2565]: I0701 16:59:45.918362    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/1ab9ee0c-b365-44ad-8770-ac00fe1c2c98-image-registry-private-configuration\") pod \"image-registry-b84855d98-h2wv8\" (UID: \"1ab9ee0c-b365-44ad-8770-ac00fe1c2c98\") " pod="openshift-image-registry/image-registry-b84855d98-h2wv8"
Jul 01 16:59:46.018639 ip-10-0-137-244 kubenswrapper[2565]: I0701 16:59:46.018609    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/1ab9ee0c-b365-44ad-8770-ac00fe1c2c98-image-registry-private-configuration\") pod \"image-registry-b84855d98-h2wv8\" (UID: \"1ab9ee0c-b365-44ad-8770-ac00fe1c2c98\") " pod="openshift-image-registry/image-registry-b84855d98-h2wv8"
Jul 01 16:59:46.018733 ip-10-0-137-244 kubenswrapper[2565]: I0701 16:59:46.018674    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/1ab9ee0c-b365-44ad-8770-ac00fe1c2c98-registry-certificates\") pod \"image-registry-b84855d98-h2wv8\" (UID: \"1ab9ee0c-b365-44ad-8770-ac00fe1c2c98\") " pod="openshift-image-registry/image-registry-b84855d98-h2wv8"
Jul 01 16:59:46.018733 ip-10-0-137-244 kubenswrapper[2565]: I0701 16:59:46.018692    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/1ab9ee0c-b365-44ad-8770-ac00fe1c2c98-trusted-ca\") pod \"image-registry-b84855d98-h2wv8\" (UID: \"1ab9ee0c-b365-44ad-8770-ac00fe1c2c98\") " pod="openshift-image-registry/image-registry-b84855d98-h2wv8"
Jul 01 16:59:46.018733 ip-10-0-137-244 kubenswrapper[2565]: I0701 16:59:46.018713    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/1ab9ee0c-b365-44ad-8770-ac00fe1c2c98-registry-tls\") pod \"image-registry-b84855d98-h2wv8\" (UID: \"1ab9ee0c-b365-44ad-8770-ac00fe1c2c98\") " pod="openshift-image-registry/image-registry-b84855d98-h2wv8"
Jul 01 16:59:46.018733 ip-10-0-137-244 kubenswrapper[2565]: I0701 16:59:46.018730    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/1ab9ee0c-b365-44ad-8770-ac00fe1c2c98-ca-trust-extracted\") pod \"image-registry-b84855d98-h2wv8\" (UID: \"1ab9ee0c-b365-44ad-8770-ac00fe1c2c98\") " pod="openshift-image-registry/image-registry-b84855d98-h2wv8"
Jul 01 16:59:46.019074 ip-10-0-137-244 kubenswrapper[2565]: I0701 16:59:46.018752    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/1ab9ee0c-b365-44ad-8770-ac00fe1c2c98-installation-pull-secrets\") pod \"image-registry-b84855d98-h2wv8\" (UID: \"1ab9ee0c-b365-44ad-8770-ac00fe1c2c98\") " pod="openshift-image-registry/image-registry-b84855d98-h2wv8"
Jul 01 16:59:46.019074 ip-10-0-137-244 kubenswrapper[2565]: I0701 16:59:46.018775    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/1ab9ee0c-b365-44ad-8770-ac00fe1c2c98-bound-sa-token\") pod \"image-registry-b84855d98-h2wv8\" (UID: \"1ab9ee0c-b365-44ad-8770-ac00fe1c2c98\") " pod="openshift-image-registry/image-registry-b84855d98-h2wv8"
Jul 01 16:59:46.019074 ip-10-0-137-244 kubenswrapper[2565]: I0701 16:59:46.018799    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fp9nj\" (UniqueName: \"kubernetes.io/projected/1ab9ee0c-b365-44ad-8770-ac00fe1c2c98-kube-api-access-fp9nj\") pod \"image-registry-b84855d98-h2wv8\" (UID: \"1ab9ee0c-b365-44ad-8770-ac00fe1c2c98\") " pod="openshift-image-registry/image-registry-b84855d98-h2wv8"
Jul 01 16:59:46.019074 ip-10-0-137-244 kubenswrapper[2565]: E0701 16:59:46.018828    2565 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Jul 01 16:59:46.019074 ip-10-0-137-244 kubenswrapper[2565]: E0701 16:59:46.018847    2565 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-b84855d98-h2wv8: secret "image-registry-tls" not found
Jul 01 16:59:46.019074 ip-10-0-137-244 kubenswrapper[2565]: E0701 16:59:46.018894    2565 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/1ab9ee0c-b365-44ad-8770-ac00fe1c2c98-registry-tls podName:1ab9ee0c-b365-44ad-8770-ac00fe1c2c98 nodeName:}" failed. No retries permitted until 2026-07-01 16:59:46.518878552 +0000 UTC m=+56.987142653 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/1ab9ee0c-b365-44ad-8770-ac00fe1c2c98-registry-tls") pod "image-registry-b84855d98-h2wv8" (UID: "1ab9ee0c-b365-44ad-8770-ac00fe1c2c98") : secret "image-registry-tls" not found
Jul 01 16:59:46.019318 ip-10-0-137-244 kubenswrapper[2565]: I0701 16:59:46.019104    2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/1ab9ee0c-b365-44ad-8770-ac00fe1c2c98-ca-trust-extracted\") pod \"image-registry-b84855d98-h2wv8\" (UID: \"1ab9ee0c-b365-44ad-8770-ac00fe1c2c98\") " pod="openshift-image-registry/image-registry-b84855d98-h2wv8"
Jul 01 16:59:46.019318 ip-10-0-137-244 kubenswrapper[2565]: I0701 16:59:46.019295    2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/1ab9ee0c-b365-44ad-8770-ac00fe1c2c98-registry-certificates\") pod \"image-registry-b84855d98-h2wv8\" (UID: \"1ab9ee0c-b365-44ad-8770-ac00fe1c2c98\") " pod="openshift-image-registry/image-registry-b84855d98-h2wv8"
Jul 01 16:59:46.020991 ip-10-0-137-244 kubenswrapper[2565]: I0701 16:59:46.020974    2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/1ab9ee0c-b365-44ad-8770-ac00fe1c2c98-installation-pull-secrets\") pod \"image-registry-b84855d98-h2wv8\" (UID: \"1ab9ee0c-b365-44ad-8770-ac00fe1c2c98\") " pod="openshift-image-registry/image-registry-b84855d98-h2wv8"
Jul 01 16:59:46.073229 ip-10-0-137-244 kubenswrapper[2565]: I0701 16:59:46.073196    2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-fp9nj\" (UniqueName: \"kubernetes.io/projected/1ab9ee0c-b365-44ad-8770-ac00fe1c2c98-kube-api-access-fp9nj\") pod \"image-registry-b84855d98-h2wv8\" (UID: \"1ab9ee0c-b365-44ad-8770-ac00fe1c2c98\") " pod="openshift-image-registry/image-registry-b84855d98-h2wv8"
Jul 01 16:59:46.076785 ip-10-0-137-244 kubenswrapper[2565]: I0701 16:59:46.076765    2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/1ab9ee0c-b365-44ad-8770-ac00fe1c2c98-bound-sa-token\") pod \"image-registry-b84855d98-h2wv8\" (UID: \"1ab9ee0c-b365-44ad-8770-ac00fe1c2c98\") " pod="openshift-image-registry/image-registry-b84855d98-h2wv8"
Jul 01 16:59:46.094594 ip-10-0-137-244 kubenswrapper[2565]: I0701 16:59:46.094572    2565 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-service-ca-operator/service-ca-operator-5d7b7dc95c-6mhnm"]
Jul 01 16:59:46.096494 ip-10-0-137-244 kubenswrapper[2565]: I0701 16:59:46.096481    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-service-ca-operator/service-ca-operator-5d7b7dc95c-6mhnm"
Jul 01 16:59:46.106256 ip-10-0-137-244 kubenswrapper[2565]: I0701 16:59:46.106210    2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-service-ca-operator\"/\"serving-cert\""
Jul 01 16:59:46.106332 ip-10-0-137-244 kubenswrapper[2565]: I0701 16:59:46.106265    2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca-operator\"/\"service-ca-operator-config\""
Jul 01 16:59:46.110625 ip-10-0-137-244 kubenswrapper[2565]: I0701 16:59:46.110606    2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca-operator\"/\"openshift-service-ca.crt\""
Jul 01 16:59:46.111626 ip-10-0-137-244 kubenswrapper[2565]: I0701 16:59:46.111612    2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-service-ca-operator\"/\"service-ca-operator-dockercfg-hl782\""
Jul 01 16:59:46.114611 ip-10-0-137-244 kubenswrapper[2565]: I0701 16:59:46.114596    2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca-operator\"/\"kube-root-ca.crt\""
Jul 01 16:59:46.119185 ip-10-0-137-244 kubenswrapper[2565]: I0701 16:59:46.119163    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/4e819e79-27ae-485d-9920-00fcca464c44-serving-cert\") pod \"service-ca-operator-5d7b7dc95c-6mhnm\" (UID: \"4e819e79-27ae-485d-9920-00fcca464c44\") " pod="openshift-service-ca-operator/service-ca-operator-5d7b7dc95c-6mhnm"
Jul 01 16:59:46.119282 ip-10-0-137-244 kubenswrapper[2565]: I0701 16:59:46.119224    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/4e819e79-27ae-485d-9920-00fcca464c44-config\") pod \"service-ca-operator-5d7b7dc95c-6mhnm\" (UID: \"4e819e79-27ae-485d-9920-00fcca464c44\") " pod="openshift-service-ca-operator/service-ca-operator-5d7b7dc95c-6mhnm"
Jul 01 16:59:46.119337 ip-10-0-137-244 kubenswrapper[2565]: I0701 16:59:46.119284    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-sl4j5\" (UniqueName: \"kubernetes.io/projected/4e819e79-27ae-485d-9920-00fcca464c44-kube-api-access-sl4j5\") pod \"service-ca-operator-5d7b7dc95c-6mhnm\" (UID: \"4e819e79-27ae-485d-9920-00fcca464c44\") " pod="openshift-service-ca-operator/service-ca-operator-5d7b7dc95c-6mhnm"
Jul 01 16:59:46.148877 ip-10-0-137-244 kubenswrapper[2565]: I0701 16:59:46.148855    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-service-ca-operator/service-ca-operator-5d7b7dc95c-6mhnm"]
Jul 01 16:59:46.220197 ip-10-0-137-244 kubenswrapper[2565]: I0701 16:59:46.220172    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/4e819e79-27ae-485d-9920-00fcca464c44-serving-cert\") pod \"service-ca-operator-5d7b7dc95c-6mhnm\" (UID: \"4e819e79-27ae-485d-9920-00fcca464c44\") " pod="openshift-service-ca-operator/service-ca-operator-5d7b7dc95c-6mhnm"
Jul 01 16:59:46.220282 ip-10-0-137-244 kubenswrapper[2565]: I0701 16:59:46.220232    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/4e819e79-27ae-485d-9920-00fcca464c44-config\") pod \"service-ca-operator-5d7b7dc95c-6mhnm\" (UID: \"4e819e79-27ae-485d-9920-00fcca464c44\") " pod="openshift-service-ca-operator/service-ca-operator-5d7b7dc95c-6mhnm"
Jul 01 16:59:46.220282 ip-10-0-137-244 kubenswrapper[2565]: I0701 16:59:46.220273    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/cbc9a766-e08a-4666-b4e5-f3db8dcadc45-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-54cb84584-t2hmp\" (UID: \"cbc9a766-e08a-4666-b4e5-f3db8dcadc45\") " pod="openshift-monitoring/cluster-monitoring-operator-54cb84584-t2hmp"
Jul 01 16:59:46.220353 ip-10-0-137-244 kubenswrapper[2565]: I0701 16:59:46.220307    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-sl4j5\" (UniqueName: \"kubernetes.io/projected/4e819e79-27ae-485d-9920-00fcca464c44-kube-api-access-sl4j5\") pod \"service-ca-operator-5d7b7dc95c-6mhnm\" (UID: \"4e819e79-27ae-485d-9920-00fcca464c44\") " pod="openshift-service-ca-operator/service-ca-operator-5d7b7dc95c-6mhnm"
Jul 01 16:59:46.220414 ip-10-0-137-244 kubenswrapper[2565]: E0701 16:59:46.220393    2565 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found
Jul 01 16:59:46.220468 ip-10-0-137-244 kubenswrapper[2565]: E0701 16:59:46.220459    2565 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/cbc9a766-e08a-4666-b4e5-f3db8dcadc45-cluster-monitoring-operator-tls podName:cbc9a766-e08a-4666-b4e5-f3db8dcadc45 nodeName:}" failed. No retries permitted until 2026-07-01 16:59:47.220445122 +0000 UTC m=+57.688709224 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/cbc9a766-e08a-4666-b4e5-f3db8dcadc45-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-54cb84584-t2hmp" (UID: "cbc9a766-e08a-4666-b4e5-f3db8dcadc45") : secret "cluster-monitoring-operator-tls" not found
Jul 01 16:59:46.220789 ip-10-0-137-244 kubenswrapper[2565]: I0701 16:59:46.220770    2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/configmap/4e819e79-27ae-485d-9920-00fcca464c44-config\") pod \"service-ca-operator-5d7b7dc95c-6mhnm\" (UID: \"4e819e79-27ae-485d-9920-00fcca464c44\") " pod="openshift-service-ca-operator/service-ca-operator-5d7b7dc95c-6mhnm"
Jul 01 16:59:46.222231 ip-10-0-137-244 kubenswrapper[2565]: I0701 16:59:46.222213    2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/4e819e79-27ae-485d-9920-00fcca464c44-serving-cert\") pod \"service-ca-operator-5d7b7dc95c-6mhnm\" (UID: \"4e819e79-27ae-485d-9920-00fcca464c44\") " pod="openshift-service-ca-operator/service-ca-operator-5d7b7dc95c-6mhnm"
Jul 01 16:59:46.238098 ip-10-0-137-244 kubenswrapper[2565]: I0701 16:59:46.238077    2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-sl4j5\" (UniqueName: \"kubernetes.io/projected/4e819e79-27ae-485d-9920-00fcca464c44-kube-api-access-sl4j5\") pod \"service-ca-operator-5d7b7dc95c-6mhnm\" (UID: \"4e819e79-27ae-485d-9920-00fcca464c44\") " pod="openshift-service-ca-operator/service-ca-operator-5d7b7dc95c-6mhnm"
Jul 01 16:59:46.404851 ip-10-0-137-244 kubenswrapper[2565]: I0701 16:59:46.404828    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-service-ca-operator/service-ca-operator-5d7b7dc95c-6mhnm"
Jul 01 16:59:46.522658 ip-10-0-137-244 kubenswrapper[2565]: I0701 16:59:46.522632    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/1ab9ee0c-b365-44ad-8770-ac00fe1c2c98-registry-tls\") pod \"image-registry-b84855d98-h2wv8\" (UID: \"1ab9ee0c-b365-44ad-8770-ac00fe1c2c98\") " pod="openshift-image-registry/image-registry-b84855d98-h2wv8"
Jul 01 16:59:46.522818 ip-10-0-137-244 kubenswrapper[2565]: E0701 16:59:46.522798    2565 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Jul 01 16:59:46.522881 ip-10-0-137-244 kubenswrapper[2565]: E0701 16:59:46.522822    2565 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-b84855d98-h2wv8: secret "image-registry-tls" not found
Jul 01 16:59:46.522935 ip-10-0-137-244 kubenswrapper[2565]: E0701 16:59:46.522890    2565 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/1ab9ee0c-b365-44ad-8770-ac00fe1c2c98-registry-tls podName:1ab9ee0c-b365-44ad-8770-ac00fe1c2c98 nodeName:}" failed. No retries permitted until 2026-07-01 16:59:47.522868782 +0000 UTC m=+57.991132904 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/1ab9ee0c-b365-44ad-8770-ac00fe1c2c98-registry-tls") pod "image-registry-b84855d98-h2wv8" (UID: "1ab9ee0c-b365-44ad-8770-ac00fe1c2c98") : secret "image-registry-tls" not found
Jul 01 16:59:46.547041 ip-10-0-137-244 kubenswrapper[2565]: I0701 16:59:46.547001    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-service-ca-operator/service-ca-operator-5d7b7dc95c-6mhnm"]
Jul 01 16:59:46.549958 ip-10-0-137-244 kubenswrapper[2565]: W0701 16:59:46.549935    2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4e819e79_27ae_485d_9920_00fcca464c44.slice/crio-e8a93e8e8b199f712ef6ef4ee4e91897ed891a3c32f8e18cda19f01fad503cdb WatchSource:0}: Error finding container e8a93e8e8b199f712ef6ef4ee4e91897ed891a3c32f8e18cda19f01fad503cdb: Status 404 returned error can't find the container with id e8a93e8e8b199f712ef6ef4ee4e91897ed891a3c32f8e18cda19f01fad503cdb
Jul 01 16:59:46.744889 ip-10-0-137-244 kubenswrapper[2565]: I0701 16:59:46.744835    2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"trusted-ca\""
Jul 01 16:59:46.750424 ip-10-0-137-244 kubenswrapper[2565]: I0701 16:59:46.750403    2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/1ab9ee0c-b365-44ad-8770-ac00fe1c2c98-trusted-ca\") pod \"image-registry-b84855d98-h2wv8\" (UID: \"1ab9ee0c-b365-44ad-8770-ac00fe1c2c98\") " pod="openshift-image-registry/image-registry-b84855d98-h2wv8"
Jul 01 16:59:47.019706 ip-10-0-137-244 kubenswrapper[2565]: E0701 16:59:47.019649    2565 secret.go:189] Couldn't get secret openshift-image-registry/image-registry-private-configuration: failed to sync secret cache: timed out waiting for the condition
Jul 01 16:59:47.019794 ip-10-0-137-244 kubenswrapper[2565]: E0701 16:59:47.019715    2565 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/1ab9ee0c-b365-44ad-8770-ac00fe1c2c98-image-registry-private-configuration podName:1ab9ee0c-b365-44ad-8770-ac00fe1c2c98 nodeName:}" failed. No retries permitted until 2026-07-01 16:59:47.519699092 +0000 UTC m=+57.987963193 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "image-registry-private-configuration" (UniqueName: "kubernetes.io/secret/1ab9ee0c-b365-44ad-8770-ac00fe1c2c98-image-registry-private-configuration") pod "image-registry-b84855d98-h2wv8" (UID: "1ab9ee0c-b365-44ad-8770-ac00fe1c2c98") : failed to sync secret cache: timed out waiting for the condition
Jul 01 16:59:47.209196 ip-10-0-137-244 kubenswrapper[2565]: I0701 16:59:47.209169    2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-private-configuration\""
Jul 01 16:59:47.228799 ip-10-0-137-244 kubenswrapper[2565]: I0701 16:59:47.228748    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/cbc9a766-e08a-4666-b4e5-f3db8dcadc45-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-54cb84584-t2hmp\" (UID: \"cbc9a766-e08a-4666-b4e5-f3db8dcadc45\") " pod="openshift-monitoring/cluster-monitoring-operator-54cb84584-t2hmp"
Jul 01 16:59:47.228900 ip-10-0-137-244 kubenswrapper[2565]: E0701 16:59:47.228881    2565 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found
Jul 01 16:59:47.228947 ip-10-0-137-244 kubenswrapper[2565]: E0701 16:59:47.228937    2565 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/cbc9a766-e08a-4666-b4e5-f3db8dcadc45-cluster-monitoring-operator-tls podName:cbc9a766-e08a-4666-b4e5-f3db8dcadc45 nodeName:}" failed. No retries permitted until 2026-07-01 16:59:49.228921495 +0000 UTC m=+59.697185597 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/cbc9a766-e08a-4666-b4e5-f3db8dcadc45-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-54cb84584-t2hmp" (UID: "cbc9a766-e08a-4666-b4e5-f3db8dcadc45") : secret "cluster-monitoring-operator-tls" not found
Jul 01 16:59:47.476201 ip-10-0-137-244 kubenswrapper[2565]: I0701 16:59:47.476167    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca-operator/service-ca-operator-5d7b7dc95c-6mhnm" event={"ID":"4e819e79-27ae-485d-9920-00fcca464c44","Type":"ContainerStarted","Data":"e8a93e8e8b199f712ef6ef4ee4e91897ed891a3c32f8e18cda19f01fad503cdb"}
Jul 01 16:59:47.531689 ip-10-0-137-244 kubenswrapper[2565]: I0701 16:59:47.531662    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/1ab9ee0c-b365-44ad-8770-ac00fe1c2c98-registry-tls\") pod \"image-registry-b84855d98-h2wv8\" (UID: \"1ab9ee0c-b365-44ad-8770-ac00fe1c2c98\") " pod="openshift-image-registry/image-registry-b84855d98-h2wv8"
Jul 01 16:59:47.531832 ip-10-0-137-244 kubenswrapper[2565]: I0701 16:59:47.531729    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/1ab9ee0c-b365-44ad-8770-ac00fe1c2c98-image-registry-private-configuration\") pod \"image-registry-b84855d98-h2wv8\" (UID: \"1ab9ee0c-b365-44ad-8770-ac00fe1c2c98\") " pod="openshift-image-registry/image-registry-b84855d98-h2wv8"
Jul 01 16:59:47.531894 ip-10-0-137-244 kubenswrapper[2565]: E0701 16:59:47.531851    2565 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Jul 01 16:59:47.531941 ip-10-0-137-244 kubenswrapper[2565]: E0701 16:59:47.531907    2565 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-b84855d98-h2wv8: secret "image-registry-tls" not found
Jul 01 16:59:47.531992 ip-10-0-137-244 kubenswrapper[2565]: E0701 16:59:47.531967    2565 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/1ab9ee0c-b365-44ad-8770-ac00fe1c2c98-registry-tls podName:1ab9ee0c-b365-44ad-8770-ac00fe1c2c98 nodeName:}" failed. No retries permitted until 2026-07-01 16:59:49.531949526 +0000 UTC m=+60.000213630 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/1ab9ee0c-b365-44ad-8770-ac00fe1c2c98-registry-tls") pod "image-registry-b84855d98-h2wv8" (UID: "1ab9ee0c-b365-44ad-8770-ac00fe1c2c98") : secret "image-registry-tls" not found
Jul 01 16:59:47.534255 ip-10-0-137-244 kubenswrapper[2565]: I0701 16:59:47.534230    2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/1ab9ee0c-b365-44ad-8770-ac00fe1c2c98-image-registry-private-configuration\") pod \"image-registry-b84855d98-h2wv8\" (UID: \"1ab9ee0c-b365-44ad-8770-ac00fe1c2c98\") " pod="openshift-image-registry/image-registry-b84855d98-h2wv8"
Jul 01 16:59:48.479755 ip-10-0-137-244 kubenswrapper[2565]: I0701 16:59:48.479731    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca-operator/service-ca-operator-5d7b7dc95c-6mhnm" event={"ID":"4e819e79-27ae-485d-9920-00fcca464c44","Type":"ContainerStarted","Data":"4be2de9a7b83619645ff82d097e64a6ea9c772392219d59092ccf9221fc3e2a0"}
Jul 01 16:59:48.501510 ip-10-0-137-244 kubenswrapper[2565]: I0701 16:59:48.501464    2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-service-ca-operator/service-ca-operator-5d7b7dc95c-6mhnm" podStartSLOduration=0.625496412 podStartE2EDuration="2.501447819s" totalImagesPullingTime="1.875951407s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-01 16:59:46 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-01 16:59:48.50122381 +0000 UTC m=+58.969487936" watchObservedRunningTime="2026-07-01 16:59:48.501447819 +0000 UTC m=+58.969711943"
Jul 01 16:59:49.245807 ip-10-0-137-244 kubenswrapper[2565]: I0701 16:59:49.245764    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/cbc9a766-e08a-4666-b4e5-f3db8dcadc45-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-54cb84584-t2hmp\" (UID: \"cbc9a766-e08a-4666-b4e5-f3db8dcadc45\") " pod="openshift-monitoring/cluster-monitoring-operator-54cb84584-t2hmp"
Jul 01 16:59:49.246038 ip-10-0-137-244 kubenswrapper[2565]: E0701 16:59:49.245962    2565 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found
Jul 01 16:59:49.246111 ip-10-0-137-244 kubenswrapper[2565]: E0701 16:59:49.246061    2565 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/cbc9a766-e08a-4666-b4e5-f3db8dcadc45-cluster-monitoring-operator-tls podName:cbc9a766-e08a-4666-b4e5-f3db8dcadc45 nodeName:}" failed. No retries permitted until 2026-07-01 16:59:53.246040694 +0000 UTC m=+63.714304818 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/cbc9a766-e08a-4666-b4e5-f3db8dcadc45-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-54cb84584-t2hmp" (UID: "cbc9a766-e08a-4666-b4e5-f3db8dcadc45") : secret "cluster-monitoring-operator-tls" not found
Jul 01 16:59:49.548720 ip-10-0-137-244 kubenswrapper[2565]: I0701 16:59:49.548639    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/1ab9ee0c-b365-44ad-8770-ac00fe1c2c98-registry-tls\") pod \"image-registry-b84855d98-h2wv8\" (UID: \"1ab9ee0c-b365-44ad-8770-ac00fe1c2c98\") " pod="openshift-image-registry/image-registry-b84855d98-h2wv8"
Jul 01 16:59:49.549101 ip-10-0-137-244 kubenswrapper[2565]: E0701 16:59:49.548741    2565 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Jul 01 16:59:49.549101 ip-10-0-137-244 kubenswrapper[2565]: E0701 16:59:49.548757    2565 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-b84855d98-h2wv8: secret "image-registry-tls" not found
Jul 01 16:59:49.549101 ip-10-0-137-244 kubenswrapper[2565]: E0701 16:59:49.548810    2565 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/1ab9ee0c-b365-44ad-8770-ac00fe1c2c98-registry-tls podName:1ab9ee0c-b365-44ad-8770-ac00fe1c2c98 nodeName:}" failed. No retries permitted until 2026-07-01 16:59:53.54879433 +0000 UTC m=+64.017058442 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/1ab9ee0c-b365-44ad-8770-ac00fe1c2c98-registry-tls") pod "image-registry-b84855d98-h2wv8" (UID: "1ab9ee0c-b365-44ad-8770-ac00fe1c2c98") : secret "image-registry-tls" not found
Jul 01 16:59:49.874601 ip-10-0-137-244 kubenswrapper[2565]: I0701 16:59:49.874540    2565 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-insights/insights-runtime-extractor-v5999"]
Jul 01 16:59:49.877186 ip-10-0-137-244 kubenswrapper[2565]: I0701 16:59:49.877172    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-v5999"
Jul 01 16:59:49.890103 ip-10-0-137-244 kubenswrapper[2565]: I0701 16:59:49.890080    2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-tls\""
Jul 01 16:59:49.890363 ip-10-0-137-244 kubenswrapper[2565]: I0701 16:59:49.890341    2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-rbac-proxy\""
Jul 01 16:59:49.890934 ip-10-0-137-244 kubenswrapper[2565]: I0701 16:59:49.890916    2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-sa-dockercfg-9w45h\""
Jul 01 16:59:49.890934 ip-10-0-137-244 kubenswrapper[2565]: I0701 16:59:49.890932    2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"openshift-service-ca.crt\""
Jul 01 16:59:49.891093 ip-10-0-137-244 kubenswrapper[2565]: I0701 16:59:49.890920    2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-root-ca.crt\""
Jul 01 16:59:49.901652 ip-10-0-137-244 kubenswrapper[2565]: I0701 16:59:49.901635    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-v5999"]
Jul 01 16:59:49.951901 ip-10-0-137-244 kubenswrapper[2565]: I0701 16:59:49.951881    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/2824e47e-25e4-4940-8e30-5e4598a825ed-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-v5999\" (UID: \"2824e47e-25e4-4940-8e30-5e4598a825ed\") " pod="openshift-insights/insights-runtime-extractor-v5999"
Jul 01 16:59:49.951997 ip-10-0-137-244 kubenswrapper[2565]: I0701 16:59:49.951943    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/2824e47e-25e4-4940-8e30-5e4598a825ed-crio-socket\") pod \"insights-runtime-extractor-v5999\" (UID: \"2824e47e-25e4-4940-8e30-5e4598a825ed\") " pod="openshift-insights/insights-runtime-extractor-v5999"
Jul 01 16:59:49.952065 ip-10-0-137-244 kubenswrapper[2565]: I0701 16:59:49.952036    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mc8td\" (UniqueName: \"kubernetes.io/projected/2824e47e-25e4-4940-8e30-5e4598a825ed-kube-api-access-mc8td\") pod \"insights-runtime-extractor-v5999\" (UID: \"2824e47e-25e4-4940-8e30-5e4598a825ed\") " pod="openshift-insights/insights-runtime-extractor-v5999"
Jul 01 16:59:49.952109 ip-10-0-137-244 kubenswrapper[2565]: I0701 16:59:49.952084    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/2824e47e-25e4-4940-8e30-5e4598a825ed-data-volume\") pod \"insights-runtime-extractor-v5999\" (UID: \"2824e47e-25e4-4940-8e30-5e4598a825ed\") " pod="openshift-insights/insights-runtime-extractor-v5999"
Jul 01 16:59:49.952144 ip-10-0-137-244 kubenswrapper[2565]: I0701 16:59:49.952109    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/2824e47e-25e4-4940-8e30-5e4598a825ed-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-v5999\" (UID: \"2824e47e-25e4-4940-8e30-5e4598a825ed\") " pod="openshift-insights/insights-runtime-extractor-v5999"
Jul 01 16:59:50.052878 ip-10-0-137-244 kubenswrapper[2565]: I0701 16:59:50.052847    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mc8td\" (UniqueName: \"kubernetes.io/projected/2824e47e-25e4-4940-8e30-5e4598a825ed-kube-api-access-mc8td\") pod \"insights-runtime-extractor-v5999\" (UID: \"2824e47e-25e4-4940-8e30-5e4598a825ed\") " pod="openshift-insights/insights-runtime-extractor-v5999"
Jul 01 16:59:50.052968 ip-10-0-137-244 kubenswrapper[2565]: I0701 16:59:50.052884    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/2824e47e-25e4-4940-8e30-5e4598a825ed-data-volume\") pod \"insights-runtime-extractor-v5999\" (UID: \"2824e47e-25e4-4940-8e30-5e4598a825ed\") " pod="openshift-insights/insights-runtime-extractor-v5999"
Jul 01 16:59:50.052968 ip-10-0-137-244 kubenswrapper[2565]: I0701 16:59:50.052901    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/2824e47e-25e4-4940-8e30-5e4598a825ed-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-v5999\" (UID: \"2824e47e-25e4-4940-8e30-5e4598a825ed\") " pod="openshift-insights/insights-runtime-extractor-v5999"
Jul 01 16:59:50.052968 ip-10-0-137-244 kubenswrapper[2565]: I0701 16:59:50.052934    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/2824e47e-25e4-4940-8e30-5e4598a825ed-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-v5999\" (UID: \"2824e47e-25e4-4940-8e30-5e4598a825ed\") " pod="openshift-insights/insights-runtime-extractor-v5999"
Jul 01 16:59:50.053106 ip-10-0-137-244 kubenswrapper[2565]: E0701 16:59:50.052989    2565 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Jul 01 16:59:50.053402 ip-10-0-137-244 kubenswrapper[2565]: I0701 16:59:50.053374    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/2824e47e-25e4-4940-8e30-5e4598a825ed-crio-socket\") pod \"insights-runtime-extractor-v5999\" (UID: \"2824e47e-25e4-4940-8e30-5e4598a825ed\") " pod="openshift-insights/insights-runtime-extractor-v5999"
Jul 01 16:59:50.053402 ip-10-0-137-244 kubenswrapper[2565]: I0701 16:59:50.053397    2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/2824e47e-25e4-4940-8e30-5e4598a825ed-crio-socket\") pod \"insights-runtime-extractor-v5999\" (UID: \"2824e47e-25e4-4940-8e30-5e4598a825ed\") " pod="openshift-insights/insights-runtime-extractor-v5999"
Jul 01 16:59:50.053545 ip-10-0-137-244 kubenswrapper[2565]: E0701 16:59:50.053499    2565 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Jul 01 16:59:50.053545 ip-10-0-137-244 kubenswrapper[2565]: I0701 16:59:50.053520    2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/2824e47e-25e4-4940-8e30-5e4598a825ed-data-volume\") pod \"insights-runtime-extractor-v5999\" (UID: \"2824e47e-25e4-4940-8e30-5e4598a825ed\") " pod="openshift-insights/insights-runtime-extractor-v5999"
Jul 01 16:59:50.053638 ip-10-0-137-244 kubenswrapper[2565]: E0701 16:59:50.053574    2565 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/2824e47e-25e4-4940-8e30-5e4598a825ed-insights-runtime-extractor-tls podName:2824e47e-25e4-4940-8e30-5e4598a825ed nodeName:}" failed. No retries permitted until 2026-07-01 16:59:50.553554463 +0000 UTC m=+61.021818579 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/2824e47e-25e4-4940-8e30-5e4598a825ed-insights-runtime-extractor-tls") pod "insights-runtime-extractor-v5999" (UID: "2824e47e-25e4-4940-8e30-5e4598a825ed") : secret "insights-runtime-extractor-tls" not found
Jul 01 16:59:50.053809 ip-10-0-137-244 kubenswrapper[2565]: E0701 16:59:50.053793    2565 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/2824e47e-25e4-4940-8e30-5e4598a825ed-kube-rbac-proxy-cm podName:2824e47e-25e4-4940-8e30-5e4598a825ed nodeName:}" failed. No retries permitted until 2026-07-01 16:59:50.553762956 +0000 UTC m=+61.022027076 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/2824e47e-25e4-4940-8e30-5e4598a825ed-kube-rbac-proxy-cm") pod "insights-runtime-extractor-v5999" (UID: "2824e47e-25e4-4940-8e30-5e4598a825ed") : configmap "kube-rbac-proxy" not found
Jul 01 16:59:50.080720 ip-10-0-137-244 kubenswrapper[2565]: I0701 16:59:50.080694    2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-root-ca.crt\""
Jul 01 16:59:50.085845 ip-10-0-137-244 kubenswrapper[2565]: I0701 16:59:50.085831    2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"openshift-service-ca.crt\""
Jul 01 16:59:50.096321 ip-10-0-137-244 kubenswrapper[2565]: I0701 16:59:50.096300    2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-mc8td\" (UniqueName: \"kubernetes.io/projected/2824e47e-25e4-4940-8e30-5e4598a825ed-kube-api-access-mc8td\") pod \"insights-runtime-extractor-v5999\" (UID: \"2824e47e-25e4-4940-8e30-5e4598a825ed\") " pod="openshift-insights/insights-runtime-extractor-v5999"
Jul 01 16:59:50.558553 ip-10-0-137-244 kubenswrapper[2565]: I0701 16:59:50.558522    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/2824e47e-25e4-4940-8e30-5e4598a825ed-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-v5999\" (UID: \"2824e47e-25e4-4940-8e30-5e4598a825ed\") " pod="openshift-insights/insights-runtime-extractor-v5999"
Jul 01 16:59:50.558962 ip-10-0-137-244 kubenswrapper[2565]: I0701 16:59:50.558575    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/2824e47e-25e4-4940-8e30-5e4598a825ed-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-v5999\" (UID: \"2824e47e-25e4-4940-8e30-5e4598a825ed\") " pod="openshift-insights/insights-runtime-extractor-v5999"
Jul 01 16:59:50.558962 ip-10-0-137-244 kubenswrapper[2565]: E0701 16:59:50.558644    2565 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Jul 01 16:59:50.558962 ip-10-0-137-244 kubenswrapper[2565]: E0701 16:59:50.558693    2565 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Jul 01 16:59:50.558962 ip-10-0-137-244 kubenswrapper[2565]: E0701 16:59:50.558703    2565 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/2824e47e-25e4-4940-8e30-5e4598a825ed-kube-rbac-proxy-cm podName:2824e47e-25e4-4940-8e30-5e4598a825ed nodeName:}" failed. No retries permitted until 2026-07-01 16:59:51.558687859 +0000 UTC m=+62.026951962 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/2824e47e-25e4-4940-8e30-5e4598a825ed-kube-rbac-proxy-cm") pod "insights-runtime-extractor-v5999" (UID: "2824e47e-25e4-4940-8e30-5e4598a825ed") : configmap "kube-rbac-proxy" not found
Jul 01 16:59:50.558962 ip-10-0-137-244 kubenswrapper[2565]: E0701 16:59:50.558740    2565 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/2824e47e-25e4-4940-8e30-5e4598a825ed-insights-runtime-extractor-tls podName:2824e47e-25e4-4940-8e30-5e4598a825ed nodeName:}" failed. No retries permitted until 2026-07-01 16:59:51.558727681 +0000 UTC m=+62.026991783 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/2824e47e-25e4-4940-8e30-5e4598a825ed-insights-runtime-extractor-tls") pod "insights-runtime-extractor-v5999" (UID: "2824e47e-25e4-4940-8e30-5e4598a825ed") : secret "insights-runtime-extractor-tls" not found
Jul 01 16:59:50.817196 ip-10-0-137-244 kubenswrapper[2565]: I0701 16:59:50.817124    2565 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-kube-storage-version-migrator/migrator-786f9fbc5-lg2z5"]
Jul 01 16:59:50.843567 ip-10-0-137-244 kubenswrapper[2565]: I0701 16:59:50.843540    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-kube-storage-version-migrator/migrator-786f9fbc5-lg2z5"]
Jul 01 16:59:50.843698 ip-10-0-137-244 kubenswrapper[2565]: I0701 16:59:50.843636    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-kube-storage-version-migrator/migrator-786f9fbc5-lg2z5"
Jul 01 16:59:50.847626 ip-10-0-137-244 kubenswrapper[2565]: I0701 16:59:50.847607    2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator\"/\"openshift-service-ca.crt\""
Jul 01 16:59:50.847832 ip-10-0-137-244 kubenswrapper[2565]: I0701 16:59:50.847813    2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator\"/\"kube-root-ca.crt\""
Jul 01 16:59:50.847893 ip-10-0-137-244 kubenswrapper[2565]: I0701 16:59:50.847881    2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-kube-storage-version-migrator\"/\"kube-storage-version-migrator-sa-dockercfg-5mcn6\""
Jul 01 16:59:50.861052 ip-10-0-137-244 kubenswrapper[2565]: I0701 16:59:50.861009    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6v6n4\" (UniqueName: \"kubernetes.io/projected/d9f4a278-3bce-4ecd-957b-0fa58a4be3cb-kube-api-access-6v6n4\") pod \"migrator-786f9fbc5-lg2z5\" (UID: \"d9f4a278-3bce-4ecd-957b-0fa58a4be3cb\") " pod="openshift-kube-storage-version-migrator/migrator-786f9fbc5-lg2z5"
Jul 01 16:59:50.961695 ip-10-0-137-244 kubenswrapper[2565]: I0701 16:59:50.961668    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6v6n4\" (UniqueName: \"kubernetes.io/projected/d9f4a278-3bce-4ecd-957b-0fa58a4be3cb-kube-api-access-6v6n4\") pod \"migrator-786f9fbc5-lg2z5\" (UID: \"d9f4a278-3bce-4ecd-957b-0fa58a4be3cb\") " pod="openshift-kube-storage-version-migrator/migrator-786f9fbc5-lg2z5"
Jul 01 16:59:50.979374 ip-10-0-137-244 kubenswrapper[2565]: I0701 16:59:50.979357    2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-6v6n4\" (UniqueName: \"kubernetes.io/projected/d9f4a278-3bce-4ecd-957b-0fa58a4be3cb-kube-api-access-6v6n4\") pod \"migrator-786f9fbc5-lg2z5\" (UID: \"d9f4a278-3bce-4ecd-957b-0fa58a4be3cb\") " pod="openshift-kube-storage-version-migrator/migrator-786f9fbc5-lg2z5"
Jul 01 16:59:51.152616 ip-10-0-137-244 kubenswrapper[2565]: I0701 16:59:51.152592    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-kube-storage-version-migrator/migrator-786f9fbc5-lg2z5"
Jul 01 16:59:51.307972 ip-10-0-137-244 kubenswrapper[2565]: I0701 16:59:51.307942    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-kube-storage-version-migrator/migrator-786f9fbc5-lg2z5"]
Jul 01 16:59:51.310987 ip-10-0-137-244 kubenswrapper[2565]: W0701 16:59:51.310962    2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd9f4a278_3bce_4ecd_957b_0fa58a4be3cb.slice/crio-964441497eff12cccaa7a9b8f2821ee27785f1277fec9bba8777a94bd5504ee1 WatchSource:0}: Error finding container 964441497eff12cccaa7a9b8f2821ee27785f1277fec9bba8777a94bd5504ee1: Status 404 returned error can't find the container with id 964441497eff12cccaa7a9b8f2821ee27785f1277fec9bba8777a94bd5504ee1
Jul 01 16:59:51.409261 ip-10-0-137-244 kubenswrapper[2565]: I0701 16:59:51.409211    2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-zjsw4"
Jul 01 16:59:51.488145 ip-10-0-137-244 kubenswrapper[2565]: I0701 16:59:51.488118    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator/migrator-786f9fbc5-lg2z5" event={"ID":"d9f4a278-3bce-4ecd-957b-0fa58a4be3cb","Type":"ContainerStarted","Data":"964441497eff12cccaa7a9b8f2821ee27785f1277fec9bba8777a94bd5504ee1"}
Jul 01 16:59:51.565730 ip-10-0-137-244 kubenswrapper[2565]: I0701 16:59:51.565708    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/2824e47e-25e4-4940-8e30-5e4598a825ed-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-v5999\" (UID: \"2824e47e-25e4-4940-8e30-5e4598a825ed\") " pod="openshift-insights/insights-runtime-extractor-v5999"
Jul 01 16:59:51.566033 ip-10-0-137-244 kubenswrapper[2565]: I0701 16:59:51.565750    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/2824e47e-25e4-4940-8e30-5e4598a825ed-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-v5999\" (UID: \"2824e47e-25e4-4940-8e30-5e4598a825ed\") " pod="openshift-insights/insights-runtime-extractor-v5999"
Jul 01 16:59:51.566033 ip-10-0-137-244 kubenswrapper[2565]: E0701 16:59:51.565808    2565 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Jul 01 16:59:51.566033 ip-10-0-137-244 kubenswrapper[2565]: E0701 16:59:51.565822    2565 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Jul 01 16:59:51.566033 ip-10-0-137-244 kubenswrapper[2565]: E0701 16:59:51.565859    2565 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/2824e47e-25e4-4940-8e30-5e4598a825ed-insights-runtime-extractor-tls podName:2824e47e-25e4-4940-8e30-5e4598a825ed nodeName:}" failed. No retries permitted until 2026-07-01 16:59:53.565847319 +0000 UTC m=+64.034111421 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/2824e47e-25e4-4940-8e30-5e4598a825ed-insights-runtime-extractor-tls") pod "insights-runtime-extractor-v5999" (UID: "2824e47e-25e4-4940-8e30-5e4598a825ed") : secret "insights-runtime-extractor-tls" not found
Jul 01 16:59:51.566033 ip-10-0-137-244 kubenswrapper[2565]: E0701 16:59:51.565871    2565 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/2824e47e-25e4-4940-8e30-5e4598a825ed-kube-rbac-proxy-cm podName:2824e47e-25e4-4940-8e30-5e4598a825ed nodeName:}" failed. No retries permitted until 2026-07-01 16:59:53.565865548 +0000 UTC m=+64.034129650 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/2824e47e-25e4-4940-8e30-5e4598a825ed-kube-rbac-proxy-cm") pod "insights-runtime-extractor-v5999" (UID: "2824e47e-25e4-4940-8e30-5e4598a825ed") : configmap "kube-rbac-proxy" not found
Jul 01 16:59:51.652079 ip-10-0-137-244 kubenswrapper[2565]: I0701 16:59:51.652060    2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-mth9w_16665992-76de-4ba6-b887-abaadbc31ae6/dns-node-resolver/0.log"
Jul 01 16:59:53.091089 ip-10-0-137-244 kubenswrapper[2565]: I0701 16:59:53.091065    2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-g7t25_b878d4e6-fe3b-4c52-aaea-de5a58d48afc/node-ca/0.log"
Jul 01 16:59:53.280435 ip-10-0-137-244 kubenswrapper[2565]: I0701 16:59:53.280404    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/cbc9a766-e08a-4666-b4e5-f3db8dcadc45-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-54cb84584-t2hmp\" (UID: \"cbc9a766-e08a-4666-b4e5-f3db8dcadc45\") " pod="openshift-monitoring/cluster-monitoring-operator-54cb84584-t2hmp"
Jul 01 16:59:53.280601 ip-10-0-137-244 kubenswrapper[2565]: E0701 16:59:53.280552    2565 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found
Jul 01 16:59:53.280658 ip-10-0-137-244 kubenswrapper[2565]: E0701 16:59:53.280606    2565 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/cbc9a766-e08a-4666-b4e5-f3db8dcadc45-cluster-monitoring-operator-tls podName:cbc9a766-e08a-4666-b4e5-f3db8dcadc45 nodeName:}" failed. No retries permitted until 2026-07-01 17:00:01.280591239 +0000 UTC m=+71.748855340 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/cbc9a766-e08a-4666-b4e5-f3db8dcadc45-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-54cb84584-t2hmp" (UID: "cbc9a766-e08a-4666-b4e5-f3db8dcadc45") : secret "cluster-monitoring-operator-tls" not found
Jul 01 16:59:53.450668 ip-10-0-137-244 kubenswrapper[2565]: I0701 16:59:53.450646    2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-4dfnf_b433bedb-7f27-4ad0-a5ce-6d573c5aa004/serve-healthcheck-canary/0.log"
Jul 01 16:59:53.497002 ip-10-0-137-244 kubenswrapper[2565]: I0701 16:59:53.496976    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator/migrator-786f9fbc5-lg2z5" event={"ID":"d9f4a278-3bce-4ecd-957b-0fa58a4be3cb","Type":"ContainerStarted","Data":"e47293a301d7a56239eba8792a8ab89fe996ab2ffb7db5b94d0bf00a68d45d81"}
Jul 01 16:59:53.497102 ip-10-0-137-244 kubenswrapper[2565]: I0701 16:59:53.497052    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator/migrator-786f9fbc5-lg2z5" event={"ID":"d9f4a278-3bce-4ecd-957b-0fa58a4be3cb","Type":"ContainerStarted","Data":"fb74849b26b66457faac3a12bfdcf8f9c12ab2012dca91313d3597397cdc80fb"}
Jul 01 16:59:53.519461 ip-10-0-137-244 kubenswrapper[2565]: I0701 16:59:53.519424    2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-kube-storage-version-migrator/migrator-786f9fbc5-lg2z5" podStartSLOduration=1.600504657 podStartE2EDuration="3.519410728s" totalImagesPullingTime="1.918906071s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-01 16:59:50 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-01 16:59:53.518790532 +0000 UTC m=+63.987054655" watchObservedRunningTime="2026-07-01 16:59:53.519410728 +0000 UTC m=+63.987674852"
Jul 01 16:59:53.582007 ip-10-0-137-244 kubenswrapper[2565]: I0701 16:59:53.581981    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/1ab9ee0c-b365-44ad-8770-ac00fe1c2c98-registry-tls\") pod \"image-registry-b84855d98-h2wv8\" (UID: \"1ab9ee0c-b365-44ad-8770-ac00fe1c2c98\") " pod="openshift-image-registry/image-registry-b84855d98-h2wv8"
Jul 01 16:59:53.582106 ip-10-0-137-244 kubenswrapper[2565]: I0701 16:59:53.582016    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/2824e47e-25e4-4940-8e30-5e4598a825ed-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-v5999\" (UID: \"2824e47e-25e4-4940-8e30-5e4598a825ed\") " pod="openshift-insights/insights-runtime-extractor-v5999"
Jul 01 16:59:53.582106 ip-10-0-137-244 kubenswrapper[2565]: I0701 16:59:53.582089    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/2824e47e-25e4-4940-8e30-5e4598a825ed-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-v5999\" (UID: \"2824e47e-25e4-4940-8e30-5e4598a825ed\") " pod="openshift-insights/insights-runtime-extractor-v5999"
Jul 01 16:59:53.582208 ip-10-0-137-244 kubenswrapper[2565]: E0701 16:59:53.582112    2565 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Jul 01 16:59:53.582208 ip-10-0-137-244 kubenswrapper[2565]: E0701 16:59:53.582158    2565 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/2824e47e-25e4-4940-8e30-5e4598a825ed-kube-rbac-proxy-cm podName:2824e47e-25e4-4940-8e30-5e4598a825ed nodeName:}" failed. No retries permitted until 2026-07-01 16:59:57.582145593 +0000 UTC m=+68.050409724 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/2824e47e-25e4-4940-8e30-5e4598a825ed-kube-rbac-proxy-cm") pod "insights-runtime-extractor-v5999" (UID: "2824e47e-25e4-4940-8e30-5e4598a825ed") : configmap "kube-rbac-proxy" not found
Jul 01 16:59:53.582208 ip-10-0-137-244 kubenswrapper[2565]: E0701 16:59:53.582191    2565 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Jul 01 16:59:53.582208 ip-10-0-137-244 kubenswrapper[2565]: E0701 16:59:53.582207    2565 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Jul 01 16:59:53.582372 ip-10-0-137-244 kubenswrapper[2565]: E0701 16:59:53.582215    2565 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-b84855d98-h2wv8: secret "image-registry-tls" not found
Jul 01 16:59:53.582372 ip-10-0-137-244 kubenswrapper[2565]: E0701 16:59:53.582234    2565 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/2824e47e-25e4-4940-8e30-5e4598a825ed-insights-runtime-extractor-tls podName:2824e47e-25e4-4940-8e30-5e4598a825ed nodeName:}" failed. No retries permitted until 2026-07-01 16:59:57.582221999 +0000 UTC m=+68.050486100 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/2824e47e-25e4-4940-8e30-5e4598a825ed-insights-runtime-extractor-tls") pod "insights-runtime-extractor-v5999" (UID: "2824e47e-25e4-4940-8e30-5e4598a825ed") : secret "insights-runtime-extractor-tls" not found
Jul 01 16:59:53.582372 ip-10-0-137-244 kubenswrapper[2565]: E0701 16:59:53.582251    2565 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/1ab9ee0c-b365-44ad-8770-ac00fe1c2c98-registry-tls podName:1ab9ee0c-b365-44ad-8770-ac00fe1c2c98 nodeName:}" failed. No retries permitted until 2026-07-01 17:00:01.582242742 +0000 UTC m=+72.050506850 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/1ab9ee0c-b365-44ad-8770-ac00fe1c2c98-registry-tls") pod "image-registry-b84855d98-h2wv8" (UID: "1ab9ee0c-b365-44ad-8770-ac00fe1c2c98") : secret "image-registry-tls" not found
Jul 01 16:59:54.890917 ip-10-0-137-244 kubenswrapper[2565]: I0701 16:59:54.890881    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/a6488c7c-9ea5-4ad2-91ba-b1ffda18b203-metrics-certs\") pod \"network-metrics-daemon-hthvc\" (UID: \"a6488c7c-9ea5-4ad2-91ba-b1ffda18b203\") " pod="openshift-multus/network-metrics-daemon-hthvc"
Jul 01 16:59:54.890917 ip-10-0-137-244 kubenswrapper[2565]: I0701 16:59:54.890919    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/b2746391-676c-4add-9406-a60838c1ac2c-original-pull-secret\") pod \"global-pull-secret-syncer-vfng9\" (UID: \"b2746391-676c-4add-9406-a60838c1ac2c\") " pod="kube-system/global-pull-secret-syncer-vfng9"
Jul 01 16:59:54.893818 ip-10-0-137-244 kubenswrapper[2565]: I0701 16:59:54.893798    2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\""
Jul 01 16:59:54.894811 ip-10-0-137-244 kubenswrapper[2565]: I0701 16:59:54.894796    2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\""
Jul 01 16:59:54.901472 ip-10-0-137-244 kubenswrapper[2565]: E0701 16:59:54.901454    2565 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: secret "metrics-daemon-secret" not found
Jul 01 16:59:54.901528 ip-10-0-137-244 kubenswrapper[2565]: E0701 16:59:54.901518    2565 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a6488c7c-9ea5-4ad2-91ba-b1ffda18b203-metrics-certs podName:a6488c7c-9ea5-4ad2-91ba-b1ffda18b203 nodeName:}" failed. No retries permitted until 2026-07-01 17:00:58.901495828 +0000 UTC m=+129.369759944 (durationBeforeRetry 1m4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/a6488c7c-9ea5-4ad2-91ba-b1ffda18b203-metrics-certs") pod "network-metrics-daemon-hthvc" (UID: "a6488c7c-9ea5-4ad2-91ba-b1ffda18b203") : secret "metrics-daemon-secret" not found
Jul 01 16:59:54.903918 ip-10-0-137-244 kubenswrapper[2565]: I0701 16:59:54.903894    2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/b2746391-676c-4add-9406-a60838c1ac2c-original-pull-secret\") pod \"global-pull-secret-syncer-vfng9\" (UID: \"b2746391-676c-4add-9406-a60838c1ac2c\") " pod="kube-system/global-pull-secret-syncer-vfng9"
Jul 01 16:59:54.991741 ip-10-0-137-244 kubenswrapper[2565]: I0701 16:59:54.991713    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9rt5m\" (UniqueName: \"kubernetes.io/projected/43a7325b-d5b7-492e-961d-52f0b9b32a17-kube-api-access-9rt5m\") pod \"network-check-target-fvqs4\" (UID: \"43a7325b-d5b7-492e-961d-52f0b9b32a17\") " pod="openshift-network-diagnostics/network-check-target-fvqs4"
Jul 01 16:59:54.996512 ip-10-0-137-244 kubenswrapper[2565]: I0701 16:59:54.996493    2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\""
Jul 01 16:59:55.004794 ip-10-0-137-244 kubenswrapper[2565]: I0701 16:59:55.004774    2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\""
Jul 01 16:59:55.015125 ip-10-0-137-244 kubenswrapper[2565]: I0701 16:59:55.015100    2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-9rt5m\" (UniqueName: \"kubernetes.io/projected/43a7325b-d5b7-492e-961d-52f0b9b32a17-kube-api-access-9rt5m\") pod \"network-check-target-fvqs4\" (UID: \"43a7325b-d5b7-492e-961d-52f0b9b32a17\") " pod="openshift-network-diagnostics/network-check-target-fvqs4"
Jul 01 16:59:55.068133 ip-10-0-137-244 kubenswrapper[2565]: I0701 16:59:55.068108    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-vfng9"
Jul 01 16:59:55.068384 ip-10-0-137-244 kubenswrapper[2565]: I0701 16:59:55.068367    2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-nt4t9\""
Jul 01 16:59:55.072894 ip-10-0-137-244 kubenswrapper[2565]: I0701 16:59:55.072873    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-fvqs4"
Jul 01 16:59:55.222416 ip-10-0-137-244 kubenswrapper[2565]: I0701 16:59:55.222392    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-vfng9"]
Jul 01 16:59:55.225262 ip-10-0-137-244 kubenswrapper[2565]: W0701 16:59:55.225237    2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb2746391_676c_4add_9406_a60838c1ac2c.slice/crio-e00dc5f08c4167c03af1e8aeeb8667ca1bb987aa7847fdfc843faa09ef91af4d WatchSource:0}: Error finding container e00dc5f08c4167c03af1e8aeeb8667ca1bb987aa7847fdfc843faa09ef91af4d: Status 404 returned error can't find the container with id e00dc5f08c4167c03af1e8aeeb8667ca1bb987aa7847fdfc843faa09ef91af4d
Jul 01 16:59:55.256780 ip-10-0-137-244 kubenswrapper[2565]: I0701 16:59:55.256761    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-fvqs4"]
Jul 01 16:59:55.259175 ip-10-0-137-244 kubenswrapper[2565]: W0701 16:59:55.259155    2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod43a7325b_d5b7_492e_961d_52f0b9b32a17.slice/crio-d0b9d4e7684498a5f94f6bb921974362235c8a5c17851336adc13ad96297675e WatchSource:0}: Error finding container d0b9d4e7684498a5f94f6bb921974362235c8a5c17851336adc13ad96297675e: Status 404 returned error can't find the container with id d0b9d4e7684498a5f94f6bb921974362235c8a5c17851336adc13ad96297675e
Jul 01 16:59:55.502826 ip-10-0-137-244 kubenswrapper[2565]: I0701 16:59:55.502755    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-fvqs4" event={"ID":"43a7325b-d5b7-492e-961d-52f0b9b32a17","Type":"ContainerStarted","Data":"d0b9d4e7684498a5f94f6bb921974362235c8a5c17851336adc13ad96297675e"}
Jul 01 16:59:55.508087 ip-10-0-137-244 kubenswrapper[2565]: I0701 16:59:55.508046    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-vfng9" event={"ID":"b2746391-676c-4add-9406-a60838c1ac2c","Type":"ContainerStarted","Data":"e00dc5f08c4167c03af1e8aeeb8667ca1bb987aa7847fdfc843faa09ef91af4d"}
Jul 01 16:59:56.806056 ip-10-0-137-244 kubenswrapper[2565]: I0701 16:59:56.805999    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/c91603a3-7fb8-488e-905d-c2b2791f4f8c-metrics-tls\") pod \"dns-default-544rz\" (UID: \"c91603a3-7fb8-488e-905d-c2b2791f4f8c\") " pod="openshift-dns/dns-default-544rz"
Jul 01 16:59:56.806493 ip-10-0-137-244 kubenswrapper[2565]: E0701 16:59:56.806174    2565 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Jul 01 16:59:56.806493 ip-10-0-137-244 kubenswrapper[2565]: E0701 16:59:56.806251    2565 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/c91603a3-7fb8-488e-905d-c2b2791f4f8c-metrics-tls podName:c91603a3-7fb8-488e-905d-c2b2791f4f8c nodeName:}" failed. No retries permitted until 2026-07-01 17:00:28.80623109 +0000 UTC m=+99.274495198 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/c91603a3-7fb8-488e-905d-c2b2791f4f8c-metrics-tls") pod "dns-default-544rz" (UID: "c91603a3-7fb8-488e-905d-c2b2791f4f8c") : secret "dns-default-metrics-tls" not found
Jul 01 16:59:57.611716 ip-10-0-137-244 kubenswrapper[2565]: I0701 16:59:57.611679    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/2824e47e-25e4-4940-8e30-5e4598a825ed-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-v5999\" (UID: \"2824e47e-25e4-4940-8e30-5e4598a825ed\") " pod="openshift-insights/insights-runtime-extractor-v5999"
Jul 01 16:59:57.611901 ip-10-0-137-244 kubenswrapper[2565]: I0701 16:59:57.611753    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/2824e47e-25e4-4940-8e30-5e4598a825ed-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-v5999\" (UID: \"2824e47e-25e4-4940-8e30-5e4598a825ed\") " pod="openshift-insights/insights-runtime-extractor-v5999"
Jul 01 16:59:57.611972 ip-10-0-137-244 kubenswrapper[2565]: E0701 16:59:57.611923    2565 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Jul 01 16:59:57.611972 ip-10-0-137-244 kubenswrapper[2565]: E0701 16:59:57.611942    2565 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Jul 01 16:59:57.612131 ip-10-0-137-244 kubenswrapper[2565]: E0701 16:59:57.611999    2565 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/2824e47e-25e4-4940-8e30-5e4598a825ed-kube-rbac-proxy-cm podName:2824e47e-25e4-4940-8e30-5e4598a825ed nodeName:}" failed. No retries permitted until 2026-07-01 17:00:05.611975529 +0000 UTC m=+76.080239638 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/2824e47e-25e4-4940-8e30-5e4598a825ed-kube-rbac-proxy-cm") pod "insights-runtime-extractor-v5999" (UID: "2824e47e-25e4-4940-8e30-5e4598a825ed") : configmap "kube-rbac-proxy" not found
Jul 01 16:59:57.612131 ip-10-0-137-244 kubenswrapper[2565]: E0701 16:59:57.612035    2565 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/2824e47e-25e4-4940-8e30-5e4598a825ed-insights-runtime-extractor-tls podName:2824e47e-25e4-4940-8e30-5e4598a825ed nodeName:}" failed. No retries permitted until 2026-07-01 17:00:05.612008897 +0000 UTC m=+76.080273005 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/2824e47e-25e4-4940-8e30-5e4598a825ed-insights-runtime-extractor-tls") pod "insights-runtime-extractor-v5999" (UID: "2824e47e-25e4-4940-8e30-5e4598a825ed") : secret "insights-runtime-extractor-tls" not found
Jul 01 17:00:00.524169 ip-10-0-137-244 kubenswrapper[2565]: I0701 17:00:00.524133    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-fvqs4" event={"ID":"43a7325b-d5b7-492e-961d-52f0b9b32a17","Type":"ContainerStarted","Data":"cc772cce8075f0cabd31582b0909f5d879b7e57ee7207f4540f549854a15617b"}
Jul 01 17:00:00.524592 ip-10-0-137-244 kubenswrapper[2565]: I0701 17:00:00.524216    2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-network-diagnostics/network-check-target-fvqs4"
Jul 01 17:00:00.525515 ip-10-0-137-244 kubenswrapper[2565]: I0701 17:00:00.525487    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-vfng9" event={"ID":"b2746391-676c-4add-9406-a60838c1ac2c","Type":"ContainerStarted","Data":"2e65405f57ba3a7746aba509c67887be2d7c7f6205da70ad42cc1d791ff3629b"}
Jul 01 17:00:00.561916 ip-10-0-137-244 kubenswrapper[2565]: I0701 17:00:00.561871    2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-diagnostics/network-check-target-fvqs4" podStartSLOduration=65.795116934 podStartE2EDuration="1m10.561857039s" totalImagesPullingTime="4.766740105s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-01 16:58:50 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-01 17:00:00.559206405 +0000 UTC m=+71.027470529" watchObservedRunningTime="2026-07-01 17:00:00.561857039 +0000 UTC m=+71.030121163"
Jul 01 17:00:00.598081 ip-10-0-137-244 kubenswrapper[2565]: I0701 17:00:00.598040    2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/global-pull-secret-syncer-vfng9" podStartSLOduration=65.785351553 podStartE2EDuration="1m10.598008631s" totalImagesPullingTime="4.812657078s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-01 16:58:50 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-01 17:00:00.597480937 +0000 UTC m=+71.065745062" watchObservedRunningTime="2026-07-01 17:00:00.598008631 +0000 UTC m=+71.066272755"
Jul 01 17:00:01.340549 ip-10-0-137-244 kubenswrapper[2565]: I0701 17:00:01.340515    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/cbc9a766-e08a-4666-b4e5-f3db8dcadc45-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-54cb84584-t2hmp\" (UID: \"cbc9a766-e08a-4666-b4e5-f3db8dcadc45\") " pod="openshift-monitoring/cluster-monitoring-operator-54cb84584-t2hmp"
Jul 01 17:00:01.340672 ip-10-0-137-244 kubenswrapper[2565]: E0701 17:00:01.340659    2565 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found
Jul 01 17:00:01.340738 ip-10-0-137-244 kubenswrapper[2565]: E0701 17:00:01.340725    2565 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/cbc9a766-e08a-4666-b4e5-f3db8dcadc45-cluster-monitoring-operator-tls podName:cbc9a766-e08a-4666-b4e5-f3db8dcadc45 nodeName:}" failed. No retries permitted until 2026-07-01 17:00:17.340710102 +0000 UTC m=+87.808974203 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/cbc9a766-e08a-4666-b4e5-f3db8dcadc45-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-54cb84584-t2hmp" (UID: "cbc9a766-e08a-4666-b4e5-f3db8dcadc45") : secret "cluster-monitoring-operator-tls" not found
Jul 01 17:00:01.642720 ip-10-0-137-244 kubenswrapper[2565]: I0701 17:00:01.642691    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/1ab9ee0c-b365-44ad-8770-ac00fe1c2c98-registry-tls\") pod \"image-registry-b84855d98-h2wv8\" (UID: \"1ab9ee0c-b365-44ad-8770-ac00fe1c2c98\") " pod="openshift-image-registry/image-registry-b84855d98-h2wv8"
Jul 01 17:00:01.645389 ip-10-0-137-244 kubenswrapper[2565]: I0701 17:00:01.645364    2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/1ab9ee0c-b365-44ad-8770-ac00fe1c2c98-registry-tls\") pod \"image-registry-b84855d98-h2wv8\" (UID: \"1ab9ee0c-b365-44ad-8770-ac00fe1c2c98\") " pod="openshift-image-registry/image-registry-b84855d98-h2wv8"
Jul 01 17:00:01.772146 ip-10-0-137-244 kubenswrapper[2565]: I0701 17:00:01.772117    2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"registry-dockercfg-5lvm2\""
Jul 01 17:00:01.776188 ip-10-0-137-244 kubenswrapper[2565]: I0701 17:00:01.776170    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-b84855d98-h2wv8"
Jul 01 17:00:01.931614 ip-10-0-137-244 kubenswrapper[2565]: I0701 17:00:01.931594    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-b84855d98-h2wv8"]
Jul 01 17:00:01.933669 ip-10-0-137-244 kubenswrapper[2565]: W0701 17:00:01.933642    2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1ab9ee0c_b365_44ad_8770_ac00fe1c2c98.slice/crio-e65c713bb029eecd1304e9fa8fb0318d596d6f0377c9e14be23d5d23be9b8bb4 WatchSource:0}: Error finding container e65c713bb029eecd1304e9fa8fb0318d596d6f0377c9e14be23d5d23be9b8bb4: Status 404 returned error can't find the container with id e65c713bb029eecd1304e9fa8fb0318d596d6f0377c9e14be23d5d23be9b8bb4
Jul 01 17:00:02.534059 ip-10-0-137-244 kubenswrapper[2565]: I0701 17:00:02.534010    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-b84855d98-h2wv8" event={"ID":"1ab9ee0c-b365-44ad-8770-ac00fe1c2c98","Type":"ContainerStarted","Data":"5ecfda4b9aba391adf3ad9260d1d476a29b598f26814eeb8f692d4f530b6e416"}
Jul 01 17:00:02.534234 ip-10-0-137-244 kubenswrapper[2565]: I0701 17:00:02.534079    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-b84855d98-h2wv8" event={"ID":"1ab9ee0c-b365-44ad-8770-ac00fe1c2c98","Type":"ContainerStarted","Data":"e65c713bb029eecd1304e9fa8fb0318d596d6f0377c9e14be23d5d23be9b8bb4"}
Jul 01 17:00:02.534234 ip-10-0-137-244 kubenswrapper[2565]: I0701 17:00:02.534172    2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-image-registry/image-registry-b84855d98-h2wv8"
Jul 01 17:00:02.651733 ip-10-0-137-244 kubenswrapper[2565]: I0701 17:00:02.651687    2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/image-registry-b84855d98-h2wv8" podStartSLOduration=17.651672308 podStartE2EDuration="17.651672308s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-01 16:59:45 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-01 17:00:02.640316793 +0000 UTC m=+73.108580917" watchObservedRunningTime="2026-07-01 17:00:02.651672308 +0000 UTC m=+73.119936410"
Jul 01 17:00:05.671252 ip-10-0-137-244 kubenswrapper[2565]: I0701 17:00:05.671191    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/2824e47e-25e4-4940-8e30-5e4598a825ed-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-v5999\" (UID: \"2824e47e-25e4-4940-8e30-5e4598a825ed\") " pod="openshift-insights/insights-runtime-extractor-v5999"
Jul 01 17:00:05.671631 ip-10-0-137-244 kubenswrapper[2565]: I0701 17:00:05.671271    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/2824e47e-25e4-4940-8e30-5e4598a825ed-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-v5999\" (UID: \"2824e47e-25e4-4940-8e30-5e4598a825ed\") " pod="openshift-insights/insights-runtime-extractor-v5999"
Jul 01 17:00:05.671809 ip-10-0-137-244 kubenswrapper[2565]: I0701 17:00:05.671786    2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/2824e47e-25e4-4940-8e30-5e4598a825ed-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-v5999\" (UID: \"2824e47e-25e4-4940-8e30-5e4598a825ed\") " pod="openshift-insights/insights-runtime-extractor-v5999"
Jul 01 17:00:05.673548 ip-10-0-137-244 kubenswrapper[2565]: I0701 17:00:05.673529    2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/2824e47e-25e4-4940-8e30-5e4598a825ed-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-v5999\" (UID: \"2824e47e-25e4-4940-8e30-5e4598a825ed\") " pod="openshift-insights/insights-runtime-extractor-v5999"
Jul 01 17:00:05.787625 ip-10-0-137-244 kubenswrapper[2565]: I0701 17:00:05.787594    2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-sa-dockercfg-9w45h\""
Jul 01 17:00:05.795622 ip-10-0-137-244 kubenswrapper[2565]: I0701 17:00:05.795607    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-v5999"
Jul 01 17:00:05.940042 ip-10-0-137-244 kubenswrapper[2565]: I0701 17:00:05.939998    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-v5999"]
Jul 01 17:00:05.941133 ip-10-0-137-244 kubenswrapper[2565]: W0701 17:00:05.941105    2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2824e47e_25e4_4940_8e30_5e4598a825ed.slice/crio-f2df9f549e1a7fa490c579cd7faf13e93a00bee64a2b292c3906361129948123 WatchSource:0}: Error finding container f2df9f549e1a7fa490c579cd7faf13e93a00bee64a2b292c3906361129948123: Status 404 returned error can't find the container with id f2df9f549e1a7fa490c579cd7faf13e93a00bee64a2b292c3906361129948123
Jul 01 17:00:06.547699 ip-10-0-137-244 kubenswrapper[2565]: I0701 17:00:06.547653    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-v5999" event={"ID":"2824e47e-25e4-4940-8e30-5e4598a825ed","Type":"ContainerStarted","Data":"f2df9f549e1a7fa490c579cd7faf13e93a00bee64a2b292c3906361129948123"}
Jul 01 17:00:11.565836 ip-10-0-137-244 kubenswrapper[2565]: I0701 17:00:11.565802    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-v5999" event={"ID":"2824e47e-25e4-4940-8e30-5e4598a825ed","Type":"ContainerStarted","Data":"2fe08e9ef962ef59cb4dad778d21774c2cf6e59fe3a4f57dc014282a15157f39"}
Jul 01 17:00:12.570801 ip-10-0-137-244 kubenswrapper[2565]: I0701 17:00:12.570679    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-v5999" event={"ID":"2824e47e-25e4-4940-8e30-5e4598a825ed","Type":"ContainerStarted","Data":"d6d4a19aa4bda0d78daf966c06f1e81624713154ac33da3ed22739b5e205dcb7"}
Jul 01 17:00:14.578511 ip-10-0-137-244 kubenswrapper[2565]: I0701 17:00:14.578481    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-v5999" event={"ID":"2824e47e-25e4-4940-8e30-5e4598a825ed","Type":"ContainerStarted","Data":"cb1397836c9cbc7623cd7f0b91476dd4330e072d070cd3bea551ab08f203a1f8"}
Jul 01 17:00:17.350804 ip-10-0-137-244 kubenswrapper[2565]: I0701 17:00:17.350763    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/cbc9a766-e08a-4666-b4e5-f3db8dcadc45-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-54cb84584-t2hmp\" (UID: \"cbc9a766-e08a-4666-b4e5-f3db8dcadc45\") " pod="openshift-monitoring/cluster-monitoring-operator-54cb84584-t2hmp"
Jul 01 17:00:17.353578 ip-10-0-137-244 kubenswrapper[2565]: I0701 17:00:17.353545    2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/cbc9a766-e08a-4666-b4e5-f3db8dcadc45-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-54cb84584-t2hmp\" (UID: \"cbc9a766-e08a-4666-b4e5-f3db8dcadc45\") " pod="openshift-monitoring/cluster-monitoring-operator-54cb84584-t2hmp"
Jul 01 17:00:17.392759 ip-10-0-137-244 kubenswrapper[2565]: I0701 17:00:17.392734    2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"cluster-monitoring-operator-dockercfg-2bpn2\""
Jul 01 17:00:17.400369 ip-10-0-137-244 kubenswrapper[2565]: I0701 17:00:17.400354    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/cluster-monitoring-operator-54cb84584-t2hmp"
Jul 01 17:00:17.537743 ip-10-0-137-244 kubenswrapper[2565]: I0701 17:00:17.537699    2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-insights/insights-runtime-extractor-v5999" podStartSLOduration=20.118420918 podStartE2EDuration="28.537680801s" totalImagesPullingTime="8.419259883s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-01 16:59:49 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-01 17:00:14.607647289 +0000 UTC m=+85.075911423" watchObservedRunningTime="2026-07-01 17:00:17.537680801 +0000 UTC m=+88.005944925"
Jul 01 17:00:17.537899 ip-10-0-137-244 kubenswrapper[2565]: I0701 17:00:17.537833    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/cluster-monitoring-operator-54cb84584-t2hmp"]
Jul 01 17:00:17.539946 ip-10-0-137-244 kubenswrapper[2565]: W0701 17:00:17.539913    2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podcbc9a766_e08a_4666_b4e5_f3db8dcadc45.slice/crio-a4db237a1dd771af47c2328eafb3773e00f69d93c7a519e42d519e615fb9f9d8 WatchSource:0}: Error finding container a4db237a1dd771af47c2328eafb3773e00f69d93c7a519e42d519e615fb9f9d8: Status 404 returned error can't find the container with id a4db237a1dd771af47c2328eafb3773e00f69d93c7a519e42d519e615fb9f9d8
Jul 01 17:00:17.589534 ip-10-0-137-244 kubenswrapper[2565]: I0701 17:00:17.589499    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/cluster-monitoring-operator-54cb84584-t2hmp" event={"ID":"cbc9a766-e08a-4666-b4e5-f3db8dcadc45","Type":"ContainerStarted","Data":"a4db237a1dd771af47c2328eafb3773e00f69d93c7a519e42d519e615fb9f9d8"}
Jul 01 17:00:19.598572 ip-10-0-137-244 kubenswrapper[2565]: I0701 17:00:19.598540    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/cluster-monitoring-operator-54cb84584-t2hmp" event={"ID":"cbc9a766-e08a-4666-b4e5-f3db8dcadc45","Type":"ContainerStarted","Data":"92634e28d5a51a33ab41c287156031ed31dacff8c2259d9a7853c96ac6192469"}
Jul 01 17:00:19.619932 ip-10-0-137-244 kubenswrapper[2565]: I0701 17:00:19.619881    2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/cluster-monitoring-operator-54cb84584-t2hmp" podStartSLOduration=32.701324712 podStartE2EDuration="34.619866794s" totalImagesPullingTime="1.918542082s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-01 16:59:45 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-01 17:00:19.61921528 +0000 UTC m=+90.087479394" watchObservedRunningTime="2026-07-01 17:00:19.619866794 +0000 UTC m=+90.088130897"
Jul 01 17:00:20.334940 ip-10-0-137-244 kubenswrapper[2565]: I0701 17:00:20.334907    2565 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["open-cluster-management-agent-addon/cluster-proxy-proxy-agent-544859795f-8628h"]
Jul 01 17:00:20.338016 ip-10-0-137-244 kubenswrapper[2565]: I0701 17:00:20.338000    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-544859795f-8628h"
Jul 01 17:00:20.364331 ip-10-0-137-244 kubenswrapper[2565]: I0701 17:00:20.364306    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/cluster-proxy-proxy-agent-544859795f-8628h"]
Jul 01 17:00:20.364458 ip-10-0-137-244 kubenswrapper[2565]: E0701 17:00:20.364438    2565 reflector.go:205] "Failed to watch" err="failed to list *v1.Secret: secrets \"cluster-proxy-ca\" is forbidden: User \"system:node:ip-10-0-137-244.ec2.internal\" cannot list resource \"secrets\" in API group \"\" in the namespace \"open-cluster-management-agent-addon\": no relationship found between node 'ip-10-0-137-244.ec2.internal' and this object" logger="UnhandledError" reflector="object-\"open-cluster-management-agent-addon\"/\"cluster-proxy-ca\"" type="*v1.Secret"
Jul 01 17:00:20.364761 ip-10-0-137-244 kubenswrapper[2565]: E0701 17:00:20.364737    2565 reflector.go:205] "Failed to watch" err="failed to list *v1.Secret: secrets \"open-cluster-management-image-pull-credentials\" is forbidden: User \"system:node:ip-10-0-137-244.ec2.internal\" cannot list resource \"secrets\" in API group \"\" in the namespace \"open-cluster-management-agent-addon\": no relationship found between node 'ip-10-0-137-244.ec2.internal' and this object" logger="UnhandledError" reflector="object-\"open-cluster-management-agent-addon\"/\"open-cluster-management-image-pull-credentials\"" type="*v1.Secret"
Jul 01 17:00:20.364948 ip-10-0-137-244 kubenswrapper[2565]: E0701 17:00:20.364923    2565 reflector.go:205] "Failed to watch" err="failed to list *v1.Secret: secrets \"cluster-proxy-service-proxy-server-certificates\" is forbidden: User \"system:node:ip-10-0-137-244.ec2.internal\" cannot list resource \"secrets\" in API group \"\" in the namespace \"open-cluster-management-agent-addon\": no relationship found between node 'ip-10-0-137-244.ec2.internal' and this object" logger="UnhandledError" reflector="object-\"open-cluster-management-agent-addon\"/\"cluster-proxy-service-proxy-server-certificates\"" type="*v1.Secret"
Jul 01 17:00:20.365365 ip-10-0-137-244 kubenswrapper[2565]: E0701 17:00:20.365337    2565 status_manager.go:1018] "Failed to get status for pod" err="pods \"cluster-proxy-proxy-agent-544859795f-8628h\" is forbidden: User \"system:node:ip-10-0-137-244.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"open-cluster-management-agent-addon\": no relationship found between node 'ip-10-0-137-244.ec2.internal' and this object" podUID="ff145f39-13d0-4b93-8e4e-4d8db363b92c" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-544859795f-8628h"
Jul 01 17:00:20.365440 ip-10-0-137-244 kubenswrapper[2565]: E0701 17:00:20.365346    2565 reflector.go:205] "Failed to watch" err="failed to list *v1.Secret: secrets \"cluster-proxy-hub-kubeconfig\" is forbidden: User \"system:node:ip-10-0-137-244.ec2.internal\" cannot list resource \"secrets\" in API group \"\" in the namespace \"open-cluster-management-agent-addon\": no relationship found between node 'ip-10-0-137-244.ec2.internal' and this object" logger="UnhandledError" reflector="object-\"open-cluster-management-agent-addon\"/\"cluster-proxy-hub-kubeconfig\"" type="*v1.Secret"
Jul 01 17:00:20.365609 ip-10-0-137-244 kubenswrapper[2565]: E0701 17:00:20.365582    2565 reflector.go:205] "Failed to watch" err="failed to list *v1.Secret: secrets \"cluster-proxy-open-cluster-management.io-proxy-agent-signer-client-cert\" is forbidden: User \"system:node:ip-10-0-137-244.ec2.internal\" cannot list resource \"secrets\" in API group \"\" in the namespace \"open-cluster-management-agent-addon\": no relationship found between node 'ip-10-0-137-244.ec2.internal' and this object" logger="UnhandledError" reflector="object-\"open-cluster-management-agent-addon\"/\"cluster-proxy-open-cluster-management.io-proxy-agent-signer-client-cert\"" type="*v1.Secret"
Jul 01 17:00:20.365705 ip-10-0-137-244 kubenswrapper[2565]: E0701 17:00:20.365606    2565 reflector.go:205] "Failed to watch" err="failed to list *v1.ConfigMap: configmaps \"openshift-service-ca.crt\" is forbidden: User \"system:node:ip-10-0-137-244.ec2.internal\" cannot list resource \"configmaps\" in API group \"\" in the namespace \"open-cluster-management-agent-addon\": no relationship found between node 'ip-10-0-137-244.ec2.internal' and this object" logger="UnhandledError" reflector="object-\"open-cluster-management-agent-addon\"/\"openshift-service-ca.crt\"" type="*v1.ConfigMap"
Jul 01 17:00:20.365833 ip-10-0-137-244 kubenswrapper[2565]: E0701 17:00:20.365809    2565 reflector.go:205] "Failed to watch" err="failed to list *v1.ConfigMap: configmaps \"kube-root-ca.crt\" is forbidden: User \"system:node:ip-10-0-137-244.ec2.internal\" cannot list resource \"configmaps\" in API group \"\" in the namespace \"open-cluster-management-agent-addon\": no relationship found between node 'ip-10-0-137-244.ec2.internal' and this object" logger="UnhandledError" reflector="object-\"open-cluster-management-agent-addon\"/\"kube-root-ca.crt\"" type="*v1.ConfigMap"
Jul 01 17:00:20.471450 ip-10-0-137-244 kubenswrapper[2565]: I0701 17:00:20.471427    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/ff145f39-13d0-4b93-8e4e-4d8db363b92c-hub-kubeconfig\") pod \"cluster-proxy-proxy-agent-544859795f-8628h\" (UID: \"ff145f39-13d0-4b93-8e4e-4d8db363b92c\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-544859795f-8628h"
Jul 01 17:00:20.471555 ip-10-0-137-244 kubenswrapper[2565]: I0701 17:00:20.471458    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hub\" (UniqueName: \"kubernetes.io/secret/ff145f39-13d0-4b93-8e4e-4d8db363b92c-hub\") pod \"cluster-proxy-proxy-agent-544859795f-8628h\" (UID: \"ff145f39-13d0-4b93-8e4e-4d8db363b92c\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-544859795f-8628h"
Jul 01 17:00:20.471555 ip-10-0-137-244 kubenswrapper[2565]: I0701 17:00:20.471487    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-proxy-server-cert\" (UniqueName: \"kubernetes.io/secret/ff145f39-13d0-4b93-8e4e-4d8db363b92c-service-proxy-server-cert\") pod \"cluster-proxy-proxy-agent-544859795f-8628h\" (UID: \"ff145f39-13d0-4b93-8e4e-4d8db363b92c\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-544859795f-8628h"
Jul 01 17:00:20.471636 ip-10-0-137-244 kubenswrapper[2565]: I0701 17:00:20.471560    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rjg9t\" (UniqueName: \"kubernetes.io/projected/ff145f39-13d0-4b93-8e4e-4d8db363b92c-kube-api-access-rjg9t\") pod \"cluster-proxy-proxy-agent-544859795f-8628h\" (UID: \"ff145f39-13d0-4b93-8e4e-4d8db363b92c\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-544859795f-8628h"
Jul 01 17:00:20.471636 ip-10-0-137-244 kubenswrapper[2565]: I0701 17:00:20.471625    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ca\" (UniqueName: \"kubernetes.io/secret/ff145f39-13d0-4b93-8e4e-4d8db363b92c-ca\") pod \"cluster-proxy-proxy-agent-544859795f-8628h\" (UID: \"ff145f39-13d0-4b93-8e4e-4d8db363b92c\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-544859795f-8628h"
Jul 01 17:00:20.471719 ip-10-0-137-244 kubenswrapper[2565]: I0701 17:00:20.471652    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"additional-service-ca\" (UniqueName: \"kubernetes.io/configmap/ff145f39-13d0-4b93-8e4e-4d8db363b92c-additional-service-ca\") pod \"cluster-proxy-proxy-agent-544859795f-8628h\" (UID: \"ff145f39-13d0-4b93-8e4e-4d8db363b92c\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-544859795f-8628h"
Jul 01 17:00:20.504932 ip-10-0-137-244 kubenswrapper[2565]: I0701 17:00:20.504913    2565 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-image-registry/image-registry-b84855d98-h2wv8"]
Jul 01 17:00:20.509170 ip-10-0-137-244 kubenswrapper[2565]: I0701 17:00:20.509146    2565 patch_prober.go:28] interesting pod/image-registry-b84855d98-h2wv8 container/registry namespace/openshift-image-registry: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body={"errors":[{"code":"UNAVAILABLE","message":"service unavailable","detail":"health check failed: please see /debug/health"}]}
Jul 01 17:00:20.509274 ip-10-0-137-244 kubenswrapper[2565]: I0701 17:00:20.509188    2565 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-image-registry/image-registry-b84855d98-h2wv8" podUID="1ab9ee0c-b365-44ad-8770-ac00fe1c2c98" containerName="registry" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Jul 01 17:00:20.572269 ip-10-0-137-244 kubenswrapper[2565]: I0701 17:00:20.572244    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"additional-service-ca\" (UniqueName: \"kubernetes.io/configmap/ff145f39-13d0-4b93-8e4e-4d8db363b92c-additional-service-ca\") pod \"cluster-proxy-proxy-agent-544859795f-8628h\" (UID: \"ff145f39-13d0-4b93-8e4e-4d8db363b92c\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-544859795f-8628h"
Jul 01 17:00:20.572382 ip-10-0-137-244 kubenswrapper[2565]: I0701 17:00:20.572294    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/ff145f39-13d0-4b93-8e4e-4d8db363b92c-hub-kubeconfig\") pod \"cluster-proxy-proxy-agent-544859795f-8628h\" (UID: \"ff145f39-13d0-4b93-8e4e-4d8db363b92c\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-544859795f-8628h"
Jul 01 17:00:20.572382 ip-10-0-137-244 kubenswrapper[2565]: I0701 17:00:20.572328    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hub\" (UniqueName: \"kubernetes.io/secret/ff145f39-13d0-4b93-8e4e-4d8db363b92c-hub\") pod \"cluster-proxy-proxy-agent-544859795f-8628h\" (UID: \"ff145f39-13d0-4b93-8e4e-4d8db363b92c\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-544859795f-8628h"
Jul 01 17:00:20.572382 ip-10-0-137-244 kubenswrapper[2565]: I0701 17:00:20.572372    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-proxy-server-cert\" (UniqueName: \"kubernetes.io/secret/ff145f39-13d0-4b93-8e4e-4d8db363b92c-service-proxy-server-cert\") pod \"cluster-proxy-proxy-agent-544859795f-8628h\" (UID: \"ff145f39-13d0-4b93-8e4e-4d8db363b92c\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-544859795f-8628h"
Jul 01 17:00:20.572561 ip-10-0-137-244 kubenswrapper[2565]: I0701 17:00:20.572411    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rjg9t\" (UniqueName: \"kubernetes.io/projected/ff145f39-13d0-4b93-8e4e-4d8db363b92c-kube-api-access-rjg9t\") pod \"cluster-proxy-proxy-agent-544859795f-8628h\" (UID: \"ff145f39-13d0-4b93-8e4e-4d8db363b92c\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-544859795f-8628h"
Jul 01 17:00:20.572561 ip-10-0-137-244 kubenswrapper[2565]: I0701 17:00:20.572449    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ca\" (UniqueName: \"kubernetes.io/secret/ff145f39-13d0-4b93-8e4e-4d8db363b92c-ca\") pod \"cluster-proxy-proxy-agent-544859795f-8628h\" (UID: \"ff145f39-13d0-4b93-8e4e-4d8db363b92c\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-544859795f-8628h"
Jul 01 17:00:21.358892 ip-10-0-137-244 kubenswrapper[2565]: I0701 17:00:21.358862    2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"cluster-proxy-hub-kubeconfig\""
Jul 01 17:00:21.365309 ip-10-0-137-244 kubenswrapper[2565]: I0701 17:00:21.365280    2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/ff145f39-13d0-4b93-8e4e-4d8db363b92c-hub-kubeconfig\") pod \"cluster-proxy-proxy-agent-544859795f-8628h\" (UID: \"ff145f39-13d0-4b93-8e4e-4d8db363b92c\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-544859795f-8628h"
Jul 01 17:00:21.399264 ip-10-0-137-244 kubenswrapper[2565]: I0701 17:00:21.399242    2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"cluster-proxy-service-proxy-server-certificates\""
Jul 01 17:00:21.404958 ip-10-0-137-244 kubenswrapper[2565]: I0701 17:00:21.404939    2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"service-proxy-server-cert\" (UniqueName: \"kubernetes.io/secret/ff145f39-13d0-4b93-8e4e-4d8db363b92c-service-proxy-server-cert\") pod \"cluster-proxy-proxy-agent-544859795f-8628h\" (UID: \"ff145f39-13d0-4b93-8e4e-4d8db363b92c\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-544859795f-8628h"
Jul 01 17:00:21.461290 ip-10-0-137-244 kubenswrapper[2565]: I0701 17:00:21.461265    2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"cluster-proxy-ca\""
Jul 01 17:00:21.465209 ip-10-0-137-244 kubenswrapper[2565]: I0701 17:00:21.465188    2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ca\" (UniqueName: \"kubernetes.io/secret/ff145f39-13d0-4b93-8e4e-4d8db363b92c-ca\") pod \"cluster-proxy-proxy-agent-544859795f-8628h\" (UID: \"ff145f39-13d0-4b93-8e4e-4d8db363b92c\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-544859795f-8628h"
Jul 01 17:00:21.477815 ip-10-0-137-244 kubenswrapper[2565]: I0701 17:00:21.477789    2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"open-cluster-management-agent-addon\"/\"openshift-service-ca.crt\""
Jul 01 17:00:21.483536 ip-10-0-137-244 kubenswrapper[2565]: I0701 17:00:21.483519    2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"additional-service-ca\" (UniqueName: \"kubernetes.io/configmap/ff145f39-13d0-4b93-8e4e-4d8db363b92c-additional-service-ca\") pod \"cluster-proxy-proxy-agent-544859795f-8628h\" (UID: \"ff145f39-13d0-4b93-8e4e-4d8db363b92c\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-544859795f-8628h"
Jul 01 17:00:21.572849 ip-10-0-137-244 kubenswrapper[2565]: E0701 17:00:21.572828    2565 secret.go:189] Couldn't get secret open-cluster-management-agent-addon/cluster-proxy-open-cluster-management.io-proxy-agent-signer-client-cert: failed to sync secret cache: timed out waiting for the condition
Jul 01 17:00:21.572930 ip-10-0-137-244 kubenswrapper[2565]: E0701 17:00:21.572883    2565 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ff145f39-13d0-4b93-8e4e-4d8db363b92c-hub podName:ff145f39-13d0-4b93-8e4e-4d8db363b92c nodeName:}" failed. No retries permitted until 2026-07-01 17:00:22.07287027 +0000 UTC m=+92.541134372 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "hub" (UniqueName: "kubernetes.io/secret/ff145f39-13d0-4b93-8e4e-4d8db363b92c-hub") pod "cluster-proxy-proxy-agent-544859795f-8628h" (UID: "ff145f39-13d0-4b93-8e4e-4d8db363b92c") : failed to sync secret cache: timed out waiting for the condition
Jul 01 17:00:21.644874 ip-10-0-137-244 kubenswrapper[2565]: E0701 17:00:21.644847    2565 projected.go:291] Couldn't get configMap open-cluster-management-agent-addon/kube-root-ca.crt: failed to sync configmap cache: timed out waiting for the condition
Jul 01 17:00:21.645008 ip-10-0-137-244 kubenswrapper[2565]: E0701 17:00:21.644881    2565 projected.go:196] Error preparing data for projected volume kube-api-access-rjg9t for pod open-cluster-management-agent-addon/cluster-proxy-proxy-agent-544859795f-8628h: failed to sync configmap cache: timed out waiting for the condition
Jul 01 17:00:21.645008 ip-10-0-137-244 kubenswrapper[2565]: E0701 17:00:21.644934    2565 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/ff145f39-13d0-4b93-8e4e-4d8db363b92c-kube-api-access-rjg9t podName:ff145f39-13d0-4b93-8e4e-4d8db363b92c nodeName:}" failed. No retries permitted until 2026-07-01 17:00:22.144919258 +0000 UTC m=+92.613183365 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-api-access-rjg9t" (UniqueName: "kubernetes.io/projected/ff145f39-13d0-4b93-8e4e-4d8db363b92c-kube-api-access-rjg9t") pod "cluster-proxy-proxy-agent-544859795f-8628h" (UID: "ff145f39-13d0-4b93-8e4e-4d8db363b92c") : failed to sync configmap cache: timed out waiting for the condition
Jul 01 17:00:21.704775 ip-10-0-137-244 kubenswrapper[2565]: I0701 17:00:21.704746    2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"open-cluster-management-agent-addon\"/\"kube-root-ca.crt\""
Jul 01 17:00:21.810111 ip-10-0-137-244 kubenswrapper[2565]: I0701 17:00:21.810079    2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"cluster-proxy-open-cluster-management.io-proxy-agent-signer-client-cert\""
Jul 01 17:00:21.908558 ip-10-0-137-244 kubenswrapper[2565]: I0701 17:00:21.908472    2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"open-cluster-management-image-pull-credentials\""
Jul 01 17:00:22.082339 ip-10-0-137-244 kubenswrapper[2565]: I0701 17:00:22.082308    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hub\" (UniqueName: \"kubernetes.io/secret/ff145f39-13d0-4b93-8e4e-4d8db363b92c-hub\") pod \"cluster-proxy-proxy-agent-544859795f-8628h\" (UID: \"ff145f39-13d0-4b93-8e4e-4d8db363b92c\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-544859795f-8628h"
Jul 01 17:00:22.084751 ip-10-0-137-244 kubenswrapper[2565]: I0701 17:00:22.084721    2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"hub\" (UniqueName: \"kubernetes.io/secret/ff145f39-13d0-4b93-8e4e-4d8db363b92c-hub\") pod \"cluster-proxy-proxy-agent-544859795f-8628h\" (UID: \"ff145f39-13d0-4b93-8e4e-4d8db363b92c\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-544859795f-8628h"
Jul 01 17:00:22.183041 ip-10-0-137-244 kubenswrapper[2565]: I0701 17:00:22.182960    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rjg9t\" (UniqueName: \"kubernetes.io/projected/ff145f39-13d0-4b93-8e4e-4d8db363b92c-kube-api-access-rjg9t\") pod \"cluster-proxy-proxy-agent-544859795f-8628h\" (UID: \"ff145f39-13d0-4b93-8e4e-4d8db363b92c\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-544859795f-8628h"
Jul 01 17:00:22.185266 ip-10-0-137-244 kubenswrapper[2565]: I0701 17:00:22.185246    2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-rjg9t\" (UniqueName: \"kubernetes.io/projected/ff145f39-13d0-4b93-8e4e-4d8db363b92c-kube-api-access-rjg9t\") pod \"cluster-proxy-proxy-agent-544859795f-8628h\" (UID: \"ff145f39-13d0-4b93-8e4e-4d8db363b92c\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-544859795f-8628h"
Jul 01 17:00:22.463764 ip-10-0-137-244 kubenswrapper[2565]: I0701 17:00:22.463685    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-544859795f-8628h"
Jul 01 17:00:22.597906 ip-10-0-137-244 kubenswrapper[2565]: I0701 17:00:22.597887    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/cluster-proxy-proxy-agent-544859795f-8628h"]
Jul 01 17:00:22.599551 ip-10-0-137-244 kubenswrapper[2565]: W0701 17:00:22.599527    2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podff145f39_13d0_4b93_8e4e_4d8db363b92c.slice/crio-66e0a877ac4ba772c622398409a411ae75d09bba2b2b9641e268f77d63d97968 WatchSource:0}: Error finding container 66e0a877ac4ba772c622398409a411ae75d09bba2b2b9641e268f77d63d97968: Status 404 returned error can't find the container with id 66e0a877ac4ba772c622398409a411ae75d09bba2b2b9641e268f77d63d97968
Jul 01 17:00:22.608489 ip-10-0-137-244 kubenswrapper[2565]: I0701 17:00:22.608464    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-544859795f-8628h" event={"ID":"ff145f39-13d0-4b93-8e4e-4d8db363b92c","Type":"ContainerStarted","Data":"66e0a877ac4ba772c622398409a411ae75d09bba2b2b9641e268f77d63d97968"}
Jul 01 17:00:27.630910 ip-10-0-137-244 kubenswrapper[2565]: I0701 17:00:27.630871    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-544859795f-8628h" event={"ID":"ff145f39-13d0-4b93-8e4e-4d8db363b92c","Type":"ContainerStarted","Data":"ade6531c5df19160adb01baeebb180720cd4b0e90cdc3491197f3d8d232c88c3"}
Jul 01 17:00:27.631287 ip-10-0-137-244 kubenswrapper[2565]: I0701 17:00:27.630931    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-544859795f-8628h" event={"ID":"ff145f39-13d0-4b93-8e4e-4d8db363b92c","Type":"ContainerStarted","Data":"be6d17c8b4f32a19693384dbd4fe9c7f8a676e3203dc0311e7003097ece56eb5"}
Jul 01 17:00:27.631287 ip-10-0-137-244 kubenswrapper[2565]: I0701 17:00:27.630952    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-544859795f-8628h" event={"ID":"ff145f39-13d0-4b93-8e4e-4d8db363b92c","Type":"ContainerStarted","Data":"d34b08f4b6e420e1b15c58e5959d0b131d4a4cac5c963874898cf85a82bebff5"}
Jul 01 17:00:28.829774 ip-10-0-137-244 kubenswrapper[2565]: I0701 17:00:28.829738    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/c91603a3-7fb8-488e-905d-c2b2791f4f8c-metrics-tls\") pod \"dns-default-544rz\" (UID: \"c91603a3-7fb8-488e-905d-c2b2791f4f8c\") " pod="openshift-dns/dns-default-544rz"
Jul 01 17:00:28.832187 ip-10-0-137-244 kubenswrapper[2565]: I0701 17:00:28.832165    2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/c91603a3-7fb8-488e-905d-c2b2791f4f8c-metrics-tls\") pod \"dns-default-544rz\" (UID: \"c91603a3-7fb8-488e-905d-c2b2791f4f8c\") " pod="openshift-dns/dns-default-544rz"
Jul 01 17:00:28.861912 ip-10-0-137-244 kubenswrapper[2565]: I0701 17:00:28.861875    2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-96p59\""
Jul 01 17:00:28.869809 ip-10-0-137-244 kubenswrapper[2565]: I0701 17:00:28.869775    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-544rz"
Jul 01 17:00:28.995959 ip-10-0-137-244 kubenswrapper[2565]: I0701 17:00:28.995902    2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-544859795f-8628h" podStartSLOduration=4.531728245 podStartE2EDuration="8.995881652s" totalImagesPullingTime="4.464153407s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-01 17:00:20 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-01 17:00:27.659533396 +0000 UTC m=+98.127797521" watchObservedRunningTime="2026-07-01 17:00:28.995881652 +0000 UTC m=+99.464145780"
Jul 01 17:00:28.996842 ip-10-0-137-244 kubenswrapper[2565]: I0701 17:00:28.996824    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-544rz"]
Jul 01 17:00:28.999979 ip-10-0-137-244 kubenswrapper[2565]: W0701 17:00:28.999951    2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc91603a3_7fb8_488e_905d_c2b2791f4f8c.slice/crio-53b11051b95cb254894adc10cf377914c179ab5c89af38fd135ae6057dc42bf0 WatchSource:0}: Error finding container 53b11051b95cb254894adc10cf377914c179ab5c89af38fd135ae6057dc42bf0: Status 404 returned error can't find the container with id 53b11051b95cb254894adc10cf377914c179ab5c89af38fd135ae6057dc42bf0
Jul 01 17:00:29.638426 ip-10-0-137-244 kubenswrapper[2565]: I0701 17:00:29.638399    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-544rz" event={"ID":"c91603a3-7fb8-488e-905d-c2b2791f4f8c","Type":"ContainerStarted","Data":"53b11051b95cb254894adc10cf377914c179ab5c89af38fd135ae6057dc42bf0"}
Jul 01 17:00:29.811207 ip-10-0-137-244 kubenswrapper[2565]: I0701 17:00:29.811144    2565 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/node-exporter-wrt9l"]
Jul 01 17:00:29.815640 ip-10-0-137-244 kubenswrapper[2565]: I0701 17:00:29.815617    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-wrt9l"
Jul 01 17:00:29.818434 ip-10-0-137-244 kubenswrapper[2565]: I0701 17:00:29.818409    2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-tls\""
Jul 01 17:00:29.818522 ip-10-0-137-244 kubenswrapper[2565]: I0701 17:00:29.818479    2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-kube-rbac-proxy-config\""
Jul 01 17:00:29.818522 ip-10-0-137-244 kubenswrapper[2565]: I0701 17:00:29.818481    2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"node-exporter-accelerators-collector-config\""
Jul 01 17:00:29.819350 ip-10-0-137-244 kubenswrapper[2565]: I0701 17:00:29.819336    2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-dockercfg-djdgq\""
Jul 01 17:00:29.820181 ip-10-0-137-244 kubenswrapper[2565]: I0701 17:00:29.820165    2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"metrics-client-ca\""
Jul 01 17:00:29.938198 ip-10-0-137-244 kubenswrapper[2565]: I0701 17:00:29.938177    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/3e53c2ff-69b2-4ae3-a6a9-fa732a9ccc9e-root\") pod \"node-exporter-wrt9l\" (UID: \"3e53c2ff-69b2-4ae3-a6a9-fa732a9ccc9e\") " pod="openshift-monitoring/node-exporter-wrt9l"
Jul 01 17:00:29.938538 ip-10-0-137-244 kubenswrapper[2565]: I0701 17:00:29.938232    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/3e53c2ff-69b2-4ae3-a6a9-fa732a9ccc9e-node-exporter-accelerators-collector-config\") pod \"node-exporter-wrt9l\" (UID: \"3e53c2ff-69b2-4ae3-a6a9-fa732a9ccc9e\") " pod="openshift-monitoring/node-exporter-wrt9l"
Jul 01 17:00:29.938538 ip-10-0-137-244 kubenswrapper[2565]: I0701 17:00:29.938261    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/3e53c2ff-69b2-4ae3-a6a9-fa732a9ccc9e-node-exporter-tls\") pod \"node-exporter-wrt9l\" (UID: \"3e53c2ff-69b2-4ae3-a6a9-fa732a9ccc9e\") " pod="openshift-monitoring/node-exporter-wrt9l"
Jul 01 17:00:29.938538 ip-10-0-137-244 kubenswrapper[2565]: I0701 17:00:29.938277    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/3e53c2ff-69b2-4ae3-a6a9-fa732a9ccc9e-node-exporter-wtmp\") pod \"node-exporter-wrt9l\" (UID: \"3e53c2ff-69b2-4ae3-a6a9-fa732a9ccc9e\") " pod="openshift-monitoring/node-exporter-wrt9l"
Jul 01 17:00:29.938538 ip-10-0-137-244 kubenswrapper[2565]: I0701 17:00:29.938331    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/3e53c2ff-69b2-4ae3-a6a9-fa732a9ccc9e-sys\") pod \"node-exporter-wrt9l\" (UID: \"3e53c2ff-69b2-4ae3-a6a9-fa732a9ccc9e\") " pod="openshift-monitoring/node-exporter-wrt9l"
Jul 01 17:00:29.938538 ip-10-0-137-244 kubenswrapper[2565]: I0701 17:00:29.938371    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/3e53c2ff-69b2-4ae3-a6a9-fa732a9ccc9e-node-exporter-textfile\") pod \"node-exporter-wrt9l\" (UID: \"3e53c2ff-69b2-4ae3-a6a9-fa732a9ccc9e\") " pod="openshift-monitoring/node-exporter-wrt9l"
Jul 01 17:00:29.938538 ip-10-0-137-244 kubenswrapper[2565]: I0701 17:00:29.938409    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/3e53c2ff-69b2-4ae3-a6a9-fa732a9ccc9e-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-wrt9l\" (UID: \"3e53c2ff-69b2-4ae3-a6a9-fa732a9ccc9e\") " pod="openshift-monitoring/node-exporter-wrt9l"
Jul 01 17:00:29.938538 ip-10-0-137-244 kubenswrapper[2565]: I0701 17:00:29.938455    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/3e53c2ff-69b2-4ae3-a6a9-fa732a9ccc9e-metrics-client-ca\") pod \"node-exporter-wrt9l\" (UID: \"3e53c2ff-69b2-4ae3-a6a9-fa732a9ccc9e\") " pod="openshift-monitoring/node-exporter-wrt9l"
Jul 01 17:00:29.938538 ip-10-0-137-244 kubenswrapper[2565]: I0701 17:00:29.938489    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4g9x4\" (UniqueName: \"kubernetes.io/projected/3e53c2ff-69b2-4ae3-a6a9-fa732a9ccc9e-kube-api-access-4g9x4\") pod \"node-exporter-wrt9l\" (UID: \"3e53c2ff-69b2-4ae3-a6a9-fa732a9ccc9e\") " pod="openshift-monitoring/node-exporter-wrt9l"
Jul 01 17:00:30.039482 ip-10-0-137-244 kubenswrapper[2565]: I0701 17:00:30.039460    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/3e53c2ff-69b2-4ae3-a6a9-fa732a9ccc9e-root\") pod \"node-exporter-wrt9l\" (UID: \"3e53c2ff-69b2-4ae3-a6a9-fa732a9ccc9e\") " pod="openshift-monitoring/node-exporter-wrt9l"
Jul 01 17:00:30.039592 ip-10-0-137-244 kubenswrapper[2565]: I0701 17:00:30.039488    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/3e53c2ff-69b2-4ae3-a6a9-fa732a9ccc9e-node-exporter-accelerators-collector-config\") pod \"node-exporter-wrt9l\" (UID: \"3e53c2ff-69b2-4ae3-a6a9-fa732a9ccc9e\") " pod="openshift-monitoring/node-exporter-wrt9l"
Jul 01 17:00:30.039592 ip-10-0-137-244 kubenswrapper[2565]: I0701 17:00:30.039511    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/3e53c2ff-69b2-4ae3-a6a9-fa732a9ccc9e-node-exporter-tls\") pod \"node-exporter-wrt9l\" (UID: \"3e53c2ff-69b2-4ae3-a6a9-fa732a9ccc9e\") " pod="openshift-monitoring/node-exporter-wrt9l"
Jul 01 17:00:30.039592 ip-10-0-137-244 kubenswrapper[2565]: I0701 17:00:30.039525    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/3e53c2ff-69b2-4ae3-a6a9-fa732a9ccc9e-node-exporter-wtmp\") pod \"node-exporter-wrt9l\" (UID: \"3e53c2ff-69b2-4ae3-a6a9-fa732a9ccc9e\") " pod="openshift-monitoring/node-exporter-wrt9l"
Jul 01 17:00:30.039592 ip-10-0-137-244 kubenswrapper[2565]: I0701 17:00:30.039543    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/3e53c2ff-69b2-4ae3-a6a9-fa732a9ccc9e-sys\") pod \"node-exporter-wrt9l\" (UID: \"3e53c2ff-69b2-4ae3-a6a9-fa732a9ccc9e\") " pod="openshift-monitoring/node-exporter-wrt9l"
Jul 01 17:00:30.039592 ip-10-0-137-244 kubenswrapper[2565]: I0701 17:00:30.039560    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/3e53c2ff-69b2-4ae3-a6a9-fa732a9ccc9e-node-exporter-textfile\") pod \"node-exporter-wrt9l\" (UID: \"3e53c2ff-69b2-4ae3-a6a9-fa732a9ccc9e\") " pod="openshift-monitoring/node-exporter-wrt9l"
Jul 01 17:00:30.039849 ip-10-0-137-244 kubenswrapper[2565]: I0701 17:00:30.039599    2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"root\" (UniqueName: \"kubernetes.io/host-path/3e53c2ff-69b2-4ae3-a6a9-fa732a9ccc9e-root\") pod \"node-exporter-wrt9l\" (UID: \"3e53c2ff-69b2-4ae3-a6a9-fa732a9ccc9e\") " pod="openshift-monitoring/node-exporter-wrt9l"
Jul 01 17:00:30.039849 ip-10-0-137-244 kubenswrapper[2565]: I0701 17:00:30.039621    2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/3e53c2ff-69b2-4ae3-a6a9-fa732a9ccc9e-sys\") pod \"node-exporter-wrt9l\" (UID: \"3e53c2ff-69b2-4ae3-a6a9-fa732a9ccc9e\") " pod="openshift-monitoring/node-exporter-wrt9l"
Jul 01 17:00:30.039849 ip-10-0-137-244 kubenswrapper[2565]: E0701 17:00:30.039694    2565 secret.go:189] Couldn't get secret openshift-monitoring/node-exporter-tls: secret "node-exporter-tls" not found
Jul 01 17:00:30.039849 ip-10-0-137-244 kubenswrapper[2565]: I0701 17:00:30.039699    2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/3e53c2ff-69b2-4ae3-a6a9-fa732a9ccc9e-node-exporter-wtmp\") pod \"node-exporter-wrt9l\" (UID: \"3e53c2ff-69b2-4ae3-a6a9-fa732a9ccc9e\") " pod="openshift-monitoring/node-exporter-wrt9l"
Jul 01 17:00:30.039849 ip-10-0-137-244 kubenswrapper[2565]: I0701 17:00:30.039732    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/3e53c2ff-69b2-4ae3-a6a9-fa732a9ccc9e-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-wrt9l\" (UID: \"3e53c2ff-69b2-4ae3-a6a9-fa732a9ccc9e\") " pod="openshift-monitoring/node-exporter-wrt9l"
Jul 01 17:00:30.039849 ip-10-0-137-244 kubenswrapper[2565]: E0701 17:00:30.039752    2565 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/3e53c2ff-69b2-4ae3-a6a9-fa732a9ccc9e-node-exporter-tls podName:3e53c2ff-69b2-4ae3-a6a9-fa732a9ccc9e nodeName:}" failed. No retries permitted until 2026-07-01 17:00:30.539737345 +0000 UTC m=+101.008001461 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "node-exporter-tls" (UniqueName: "kubernetes.io/secret/3e53c2ff-69b2-4ae3-a6a9-fa732a9ccc9e-node-exporter-tls") pod "node-exporter-wrt9l" (UID: "3e53c2ff-69b2-4ae3-a6a9-fa732a9ccc9e") : secret "node-exporter-tls" not found
Jul 01 17:00:30.039849 ip-10-0-137-244 kubenswrapper[2565]: I0701 17:00:30.039779    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/3e53c2ff-69b2-4ae3-a6a9-fa732a9ccc9e-metrics-client-ca\") pod \"node-exporter-wrt9l\" (UID: \"3e53c2ff-69b2-4ae3-a6a9-fa732a9ccc9e\") " pod="openshift-monitoring/node-exporter-wrt9l"
Jul 01 17:00:30.039849 ip-10-0-137-244 kubenswrapper[2565]: I0701 17:00:30.039812    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4g9x4\" (UniqueName: \"kubernetes.io/projected/3e53c2ff-69b2-4ae3-a6a9-fa732a9ccc9e-kube-api-access-4g9x4\") pod \"node-exporter-wrt9l\" (UID: \"3e53c2ff-69b2-4ae3-a6a9-fa732a9ccc9e\") " pod="openshift-monitoring/node-exporter-wrt9l"
Jul 01 17:00:30.040294 ip-10-0-137-244 kubenswrapper[2565]: I0701 17:00:30.039884    2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/3e53c2ff-69b2-4ae3-a6a9-fa732a9ccc9e-node-exporter-textfile\") pod \"node-exporter-wrt9l\" (UID: \"3e53c2ff-69b2-4ae3-a6a9-fa732a9ccc9e\") " pod="openshift-monitoring/node-exporter-wrt9l"
Jul 01 17:00:30.040294 ip-10-0-137-244 kubenswrapper[2565]: I0701 17:00:30.040204    2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/3e53c2ff-69b2-4ae3-a6a9-fa732a9ccc9e-node-exporter-accelerators-collector-config\") pod \"node-exporter-wrt9l\" (UID: \"3e53c2ff-69b2-4ae3-a6a9-fa732a9ccc9e\") " pod="openshift-monitoring/node-exporter-wrt9l"
Jul 01 17:00:30.040404 ip-10-0-137-244 kubenswrapper[2565]: I0701 17:00:30.040304    2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/3e53c2ff-69b2-4ae3-a6a9-fa732a9ccc9e-metrics-client-ca\") pod \"node-exporter-wrt9l\" (UID: \"3e53c2ff-69b2-4ae3-a6a9-fa732a9ccc9e\") " pod="openshift-monitoring/node-exporter-wrt9l"
Jul 01 17:00:30.041944 ip-10-0-137-244 kubenswrapper[2565]: I0701 17:00:30.041922    2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/3e53c2ff-69b2-4ae3-a6a9-fa732a9ccc9e-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-wrt9l\" (UID: \"3e53c2ff-69b2-4ae3-a6a9-fa732a9ccc9e\") " pod="openshift-monitoring/node-exporter-wrt9l"
Jul 01 17:00:30.053458 ip-10-0-137-244 kubenswrapper[2565]: I0701 17:00:30.053435    2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-4g9x4\" (UniqueName: \"kubernetes.io/projected/3e53c2ff-69b2-4ae3-a6a9-fa732a9ccc9e-kube-api-access-4g9x4\") pod \"node-exporter-wrt9l\" (UID: \"3e53c2ff-69b2-4ae3-a6a9-fa732a9ccc9e\") " pod="openshift-monitoring/node-exporter-wrt9l"
Jul 01 17:00:30.509872 ip-10-0-137-244 kubenswrapper[2565]: I0701 17:00:30.509837    2565 patch_prober.go:28] interesting pod/image-registry-b84855d98-h2wv8 container/registry namespace/openshift-image-registry: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body={"errors":[{"code":"UNAVAILABLE","message":"service unavailable","detail":"health check failed: please see /debug/health"}]}
Jul 01 17:00:30.510050 ip-10-0-137-244 kubenswrapper[2565]: I0701 17:00:30.509888    2565 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-image-registry/image-registry-b84855d98-h2wv8" podUID="1ab9ee0c-b365-44ad-8770-ac00fe1c2c98" containerName="registry" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Jul 01 17:00:30.543547 ip-10-0-137-244 kubenswrapper[2565]: I0701 17:00:30.543524    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/3e53c2ff-69b2-4ae3-a6a9-fa732a9ccc9e-node-exporter-tls\") pod \"node-exporter-wrt9l\" (UID: \"3e53c2ff-69b2-4ae3-a6a9-fa732a9ccc9e\") " pod="openshift-monitoring/node-exporter-wrt9l"
Jul 01 17:00:30.545625 ip-10-0-137-244 kubenswrapper[2565]: I0701 17:00:30.545609    2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/3e53c2ff-69b2-4ae3-a6a9-fa732a9ccc9e-node-exporter-tls\") pod \"node-exporter-wrt9l\" (UID: \"3e53c2ff-69b2-4ae3-a6a9-fa732a9ccc9e\") " pod="openshift-monitoring/node-exporter-wrt9l"
Jul 01 17:00:30.724829 ip-10-0-137-244 kubenswrapper[2565]: I0701 17:00:30.724803    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-wrt9l"
Jul 01 17:00:30.736004 ip-10-0-137-244 kubenswrapper[2565]: W0701 17:00:30.735981    2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3e53c2ff_69b2_4ae3_a6a9_fa732a9ccc9e.slice/crio-571f6badfdb3eab59bd343ec6758ac5c8bb546d2ed2c283634e4d77c81c9e6c9 WatchSource:0}: Error finding container 571f6badfdb3eab59bd343ec6758ac5c8bb546d2ed2c283634e4d77c81c9e6c9: Status 404 returned error can't find the container with id 571f6badfdb3eab59bd343ec6758ac5c8bb546d2ed2c283634e4d77c81c9e6c9
Jul 01 17:00:31.532188 ip-10-0-137-244 kubenswrapper[2565]: I0701 17:00:31.532153    2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-network-diagnostics/network-check-target-fvqs4"
Jul 01 17:00:31.645752 ip-10-0-137-244 kubenswrapper[2565]: I0701 17:00:31.645715    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-wrt9l" event={"ID":"3e53c2ff-69b2-4ae3-a6a9-fa732a9ccc9e","Type":"ContainerStarted","Data":"571f6badfdb3eab59bd343ec6758ac5c8bb546d2ed2c283634e4d77c81c9e6c9"}
Jul 01 17:00:32.651836 ip-10-0-137-244 kubenswrapper[2565]: I0701 17:00:32.651775    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-544rz" event={"ID":"c91603a3-7fb8-488e-905d-c2b2791f4f8c","Type":"ContainerStarted","Data":"e1da64c7e14f2b8fb2f637ce29e791b8f432d86e6afee32baa8303f930cef30f"}
Jul 01 17:00:32.653707 ip-10-0-137-244 kubenswrapper[2565]: I0701 17:00:32.653677    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-wrt9l" event={"ID":"3e53c2ff-69b2-4ae3-a6a9-fa732a9ccc9e","Type":"ContainerStarted","Data":"7af4d6731d6229583d6d83c3fa5589b62207a58a3c8cf3ca4ca3c0930ab430c6"}
Jul 01 17:00:33.658090 ip-10-0-137-244 kubenswrapper[2565]: I0701 17:00:33.658054    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-544rz" event={"ID":"c91603a3-7fb8-488e-905d-c2b2791f4f8c","Type":"ContainerStarted","Data":"e2ecd81a367d290277b646e780416184736b54ff6761682f34817adbf4f7d63a"}
Jul 01 17:00:33.658512 ip-10-0-137-244 kubenswrapper[2565]: I0701 17:00:33.658174    2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-dns/dns-default-544rz"
Jul 01 17:00:33.659511 ip-10-0-137-244 kubenswrapper[2565]: I0701 17:00:33.659490    2565 generic.go:356] "Generic (PLEG): container finished" podID="3e53c2ff-69b2-4ae3-a6a9-fa732a9ccc9e" containerID="7af4d6731d6229583d6d83c3fa5589b62207a58a3c8cf3ca4ca3c0930ab430c6" exitCode=0
Jul 01 17:00:33.659594 ip-10-0-137-244 kubenswrapper[2565]: I0701 17:00:33.659517    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-wrt9l" event={"ID":"3e53c2ff-69b2-4ae3-a6a9-fa732a9ccc9e","Type":"ContainerDied","Data":"7af4d6731d6229583d6d83c3fa5589b62207a58a3c8cf3ca4ca3c0930ab430c6"}
Jul 01 17:00:33.714246 ip-10-0-137-244 kubenswrapper[2565]: I0701 17:00:33.714208    2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/dns-default-544rz" podStartSLOduration=66.305213827 podStartE2EDuration="1m9.714195368s" totalImagesPullingTime="3.408981541s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-01 16:59:24 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-01 17:00:33.713588752 +0000 UTC m=+104.181852900" watchObservedRunningTime="2026-07-01 17:00:33.714195368 +0000 UTC m=+104.182459545"
Jul 01 17:00:34.665109 ip-10-0-137-244 kubenswrapper[2565]: I0701 17:00:34.665064    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-wrt9l" event={"ID":"3e53c2ff-69b2-4ae3-a6a9-fa732a9ccc9e","Type":"ContainerStarted","Data":"d89b5b58dd29e4c66ae14684248b545f7598d874d23af4d01973b125becf9aef"}
Jul 01 17:00:34.665595 ip-10-0-137-244 kubenswrapper[2565]: I0701 17:00:34.665131    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-wrt9l" event={"ID":"3e53c2ff-69b2-4ae3-a6a9-fa732a9ccc9e","Type":"ContainerStarted","Data":"243fba3a5bbedcb8fcfeabdbe557b89127f14169819f5e923dcd280a9e344d4e"}
Jul 01 17:00:34.689497 ip-10-0-137-244 kubenswrapper[2565]: I0701 17:00:34.689445    2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/node-exporter-wrt9l" podStartSLOduration=3.900810722 podStartE2EDuration="5.689428793s" totalImagesPullingTime="1.668837458s" totalInitContainerRuntime="119.780613ms" isStatefulPod=true podCreationTimestamp="2026-07-01 17:00:29 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-01 17:00:34.687854947 +0000 UTC m=+105.156119071" watchObservedRunningTime="2026-07-01 17:00:34.689428793 +0000 UTC m=+105.157692992"
Jul 01 17:00:37.192208 ip-10-0-137-244 kubenswrapper[2565]: I0701 17:00:37.192166    2565 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-69f897785d-dwsw6"]
Jul 01 17:00:37.194520 ip-10-0-137-244 kubenswrapper[2565]: I0701 17:00:37.194497    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-69f897785d-dwsw6"
Jul 01 17:00:37.198011 ip-10-0-137-244 kubenswrapper[2565]: I0701 17:00:37.197983    2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-dockercfg-4nn42\""
Jul 01 17:00:37.198231 ip-10-0-137-244 kubenswrapper[2565]: I0701 17:00:37.198211    2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"oauth-serving-cert\""
Jul 01 17:00:37.198862 ip-10-0-137-244 kubenswrapper[2565]: I0701 17:00:37.198842    2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"kube-root-ca.crt\""
Jul 01 17:00:37.199200 ip-10-0-137-244 kubenswrapper[2565]: I0701 17:00:37.199184    2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-serving-cert\""
Jul 01 17:00:37.199269 ip-10-0-137-244 kubenswrapper[2565]: I0701 17:00:37.199199    2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"openshift-service-ca.crt\""
Jul 01 17:00:37.199531 ip-10-0-137-244 kubenswrapper[2565]: I0701 17:00:37.199508    2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-oauth-config\""
Jul 01 17:00:37.199901 ip-10-0-137-244 kubenswrapper[2565]: I0701 17:00:37.199882    2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"console-config\""
Jul 01 17:00:37.199998 ip-10-0-137-244 kubenswrapper[2565]: I0701 17:00:37.199937    2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"service-ca\""
Jul 01 17:00:37.204289 ip-10-0-137-244 kubenswrapper[2565]: I0701 17:00:37.204261    2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"trusted-ca-bundle\""
Jul 01 17:00:37.210166 ip-10-0-137-244 kubenswrapper[2565]: I0701 17:00:37.210140    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-69f897785d-dwsw6"]
Jul 01 17:00:37.290754 ip-10-0-137-244 kubenswrapper[2565]: I0701 17:00:37.290722    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/58a76674-cb3b-4227-827b-1fc6e68f63ae-console-serving-cert\") pod \"console-69f897785d-dwsw6\" (UID: \"58a76674-cb3b-4227-827b-1fc6e68f63ae\") " pod="openshift-console/console-69f897785d-dwsw6"
Jul 01 17:00:37.290754 ip-10-0-137-244 kubenswrapper[2565]: I0701 17:00:37.290759    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/58a76674-cb3b-4227-827b-1fc6e68f63ae-console-config\") pod \"console-69f897785d-dwsw6\" (UID: \"58a76674-cb3b-4227-827b-1fc6e68f63ae\") " pod="openshift-console/console-69f897785d-dwsw6"
Jul 01 17:00:37.290931 ip-10-0-137-244 kubenswrapper[2565]: I0701 17:00:37.290786    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/58a76674-cb3b-4227-827b-1fc6e68f63ae-console-oauth-config\") pod \"console-69f897785d-dwsw6\" (UID: \"58a76674-cb3b-4227-827b-1fc6e68f63ae\") " pod="openshift-console/console-69f897785d-dwsw6"
Jul 01 17:00:37.290931 ip-10-0-137-244 kubenswrapper[2565]: I0701 17:00:37.290832    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/58a76674-cb3b-4227-827b-1fc6e68f63ae-oauth-serving-cert\") pod \"console-69f897785d-dwsw6\" (UID: \"58a76674-cb3b-4227-827b-1fc6e68f63ae\") " pod="openshift-console/console-69f897785d-dwsw6"
Jul 01 17:00:37.290931 ip-10-0-137-244 kubenswrapper[2565]: I0701 17:00:37.290854    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-flchk\" (UniqueName: \"kubernetes.io/projected/58a76674-cb3b-4227-827b-1fc6e68f63ae-kube-api-access-flchk\") pod \"console-69f897785d-dwsw6\" (UID: \"58a76674-cb3b-4227-827b-1fc6e68f63ae\") " pod="openshift-console/console-69f897785d-dwsw6"
Jul 01 17:00:37.290931 ip-10-0-137-244 kubenswrapper[2565]: I0701 17:00:37.290869    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/58a76674-cb3b-4227-827b-1fc6e68f63ae-service-ca\") pod \"console-69f897785d-dwsw6\" (UID: \"58a76674-cb3b-4227-827b-1fc6e68f63ae\") " pod="openshift-console/console-69f897785d-dwsw6"
Jul 01 17:00:37.290931 ip-10-0-137-244 kubenswrapper[2565]: I0701 17:00:37.290903    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/58a76674-cb3b-4227-827b-1fc6e68f63ae-trusted-ca-bundle\") pod \"console-69f897785d-dwsw6\" (UID: \"58a76674-cb3b-4227-827b-1fc6e68f63ae\") " pod="openshift-console/console-69f897785d-dwsw6"
Jul 01 17:00:37.391319 ip-10-0-137-244 kubenswrapper[2565]: I0701 17:00:37.391283    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/58a76674-cb3b-4227-827b-1fc6e68f63ae-console-serving-cert\") pod \"console-69f897785d-dwsw6\" (UID: \"58a76674-cb3b-4227-827b-1fc6e68f63ae\") " pod="openshift-console/console-69f897785d-dwsw6"
Jul 01 17:00:37.391404 ip-10-0-137-244 kubenswrapper[2565]: I0701 17:00:37.391326    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/58a76674-cb3b-4227-827b-1fc6e68f63ae-console-config\") pod \"console-69f897785d-dwsw6\" (UID: \"58a76674-cb3b-4227-827b-1fc6e68f63ae\") " pod="openshift-console/console-69f897785d-dwsw6"
Jul 01 17:00:37.391404 ip-10-0-137-244 kubenswrapper[2565]: I0701 17:00:37.391356    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/58a76674-cb3b-4227-827b-1fc6e68f63ae-console-oauth-config\") pod \"console-69f897785d-dwsw6\" (UID: \"58a76674-cb3b-4227-827b-1fc6e68f63ae\") " pod="openshift-console/console-69f897785d-dwsw6"
Jul 01 17:00:37.391404 ip-10-0-137-244 kubenswrapper[2565]: I0701 17:00:37.391399    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/58a76674-cb3b-4227-827b-1fc6e68f63ae-oauth-serving-cert\") pod \"console-69f897785d-dwsw6\" (UID: \"58a76674-cb3b-4227-827b-1fc6e68f63ae\") " pod="openshift-console/console-69f897785d-dwsw6"
Jul 01 17:00:37.391512 ip-10-0-137-244 kubenswrapper[2565]: I0701 17:00:37.391426    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-flchk\" (UniqueName: \"kubernetes.io/projected/58a76674-cb3b-4227-827b-1fc6e68f63ae-kube-api-access-flchk\") pod \"console-69f897785d-dwsw6\" (UID: \"58a76674-cb3b-4227-827b-1fc6e68f63ae\") " pod="openshift-console/console-69f897785d-dwsw6"
Jul 01 17:00:37.391512 ip-10-0-137-244 kubenswrapper[2565]: I0701 17:00:37.391452    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/58a76674-cb3b-4227-827b-1fc6e68f63ae-service-ca\") pod \"console-69f897785d-dwsw6\" (UID: \"58a76674-cb3b-4227-827b-1fc6e68f63ae\") " pod="openshift-console/console-69f897785d-dwsw6"
Jul 01 17:00:37.391630 ip-10-0-137-244 kubenswrapper[2565]: I0701 17:00:37.391607    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/58a76674-cb3b-4227-827b-1fc6e68f63ae-trusted-ca-bundle\") pod \"console-69f897785d-dwsw6\" (UID: \"58a76674-cb3b-4227-827b-1fc6e68f63ae\") " pod="openshift-console/console-69f897785d-dwsw6"
Jul 01 17:00:37.392297 ip-10-0-137-244 kubenswrapper[2565]: I0701 17:00:37.392268    2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/58a76674-cb3b-4227-827b-1fc6e68f63ae-console-config\") pod \"console-69f897785d-dwsw6\" (UID: \"58a76674-cb3b-4227-827b-1fc6e68f63ae\") " pod="openshift-console/console-69f897785d-dwsw6"
Jul 01 17:00:37.392408 ip-10-0-137-244 kubenswrapper[2565]: I0701 17:00:37.392323    2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/58a76674-cb3b-4227-827b-1fc6e68f63ae-service-ca\") pod \"console-69f897785d-dwsw6\" (UID: \"58a76674-cb3b-4227-827b-1fc6e68f63ae\") " pod="openshift-console/console-69f897785d-dwsw6"
Jul 01 17:00:37.392408 ip-10-0-137-244 kubenswrapper[2565]: I0701 17:00:37.392327    2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/58a76674-cb3b-4227-827b-1fc6e68f63ae-oauth-serving-cert\") pod \"console-69f897785d-dwsw6\" (UID: \"58a76674-cb3b-4227-827b-1fc6e68f63ae\") " pod="openshift-console/console-69f897785d-dwsw6"
Jul 01 17:00:37.392725 ip-10-0-137-244 kubenswrapper[2565]: I0701 17:00:37.392696    2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/58a76674-cb3b-4227-827b-1fc6e68f63ae-trusted-ca-bundle\") pod \"console-69f897785d-dwsw6\" (UID: \"58a76674-cb3b-4227-827b-1fc6e68f63ae\") " pod="openshift-console/console-69f897785d-dwsw6"
Jul 01 17:00:37.393969 ip-10-0-137-244 kubenswrapper[2565]: I0701 17:00:37.393941    2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/58a76674-cb3b-4227-827b-1fc6e68f63ae-console-oauth-config\") pod \"console-69f897785d-dwsw6\" (UID: \"58a76674-cb3b-4227-827b-1fc6e68f63ae\") " pod="openshift-console/console-69f897785d-dwsw6"
Jul 01 17:00:37.394246 ip-10-0-137-244 kubenswrapper[2565]: I0701 17:00:37.394227    2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/58a76674-cb3b-4227-827b-1fc6e68f63ae-console-serving-cert\") pod \"console-69f897785d-dwsw6\" (UID: \"58a76674-cb3b-4227-827b-1fc6e68f63ae\") " pod="openshift-console/console-69f897785d-dwsw6"
Jul 01 17:00:37.403510 ip-10-0-137-244 kubenswrapper[2565]: I0701 17:00:37.403483    2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-flchk\" (UniqueName: \"kubernetes.io/projected/58a76674-cb3b-4227-827b-1fc6e68f63ae-kube-api-access-flchk\") pod \"console-69f897785d-dwsw6\" (UID: \"58a76674-cb3b-4227-827b-1fc6e68f63ae\") " pod="openshift-console/console-69f897785d-dwsw6"
Jul 01 17:00:37.505976 ip-10-0-137-244 kubenswrapper[2565]: I0701 17:00:37.505916    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-69f897785d-dwsw6"
Jul 01 17:00:37.644433 ip-10-0-137-244 kubenswrapper[2565]: I0701 17:00:37.644412    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-69f897785d-dwsw6"]
Jul 01 17:00:37.646229 ip-10-0-137-244 kubenswrapper[2565]: W0701 17:00:37.646208    2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod58a76674_cb3b_4227_827b_1fc6e68f63ae.slice/crio-1242119cfe2f9a2d92b72948ce2e128dc3e49c2259349a1d86a01f3a665edaf8 WatchSource:0}: Error finding container 1242119cfe2f9a2d92b72948ce2e128dc3e49c2259349a1d86a01f3a665edaf8: Status 404 returned error can't find the container with id 1242119cfe2f9a2d92b72948ce2e128dc3e49c2259349a1d86a01f3a665edaf8
Jul 01 17:00:37.676355 ip-10-0-137-244 kubenswrapper[2565]: I0701 17:00:37.676326    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-69f897785d-dwsw6" event={"ID":"58a76674-cb3b-4227-827b-1fc6e68f63ae","Type":"ContainerStarted","Data":"1242119cfe2f9a2d92b72948ce2e128dc3e49c2259349a1d86a01f3a665edaf8"}
Jul 01 17:00:40.510593 ip-10-0-137-244 kubenswrapper[2565]: I0701 17:00:40.510533    2565 patch_prober.go:28] interesting pod/image-registry-b84855d98-h2wv8 container/registry namespace/openshift-image-registry: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body={"errors":[{"code":"UNAVAILABLE","message":"service unavailable","detail":"health check failed: please see /debug/health"}]}
Jul 01 17:00:40.511056 ip-10-0-137-244 kubenswrapper[2565]: I0701 17:00:40.510620    2565 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-image-registry/image-registry-b84855d98-h2wv8" podUID="1ab9ee0c-b365-44ad-8770-ac00fe1c2c98" containerName="registry" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Jul 01 17:00:42.695051 ip-10-0-137-244 kubenswrapper[2565]: I0701 17:00:42.694997    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-69f897785d-dwsw6" event={"ID":"58a76674-cb3b-4227-827b-1fc6e68f63ae","Type":"ContainerStarted","Data":"4071ea0168352095901ae1f1a4a7477122fc47add00c70a0d2a592ef235dc2c1"}
Jul 01 17:00:42.783167 ip-10-0-137-244 kubenswrapper[2565]: I0701 17:00:42.783119    2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-69f897785d-dwsw6" podStartSLOduration=1.283737639 podStartE2EDuration="5.783103632s" totalImagesPullingTime="4.499365993s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-01 17:00:37 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-01 17:00:42.780619221 +0000 UTC m=+113.248883345" watchObservedRunningTime="2026-07-01 17:00:42.783103632 +0000 UTC m=+113.251367827"
Jul 01 17:00:43.667139 ip-10-0-137-244 kubenswrapper[2565]: I0701 17:00:43.667112    2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-dns/dns-default-544rz"
Jul 01 17:00:45.524496 ip-10-0-137-244 kubenswrapper[2565]: I0701 17:00:45.524448    2565 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-image-registry/image-registry-b84855d98-h2wv8" podUID="1ab9ee0c-b365-44ad-8770-ac00fe1c2c98" containerName="registry" containerID="cri-o://5ecfda4b9aba391adf3ad9260d1d476a29b598f26814eeb8f692d4f530b6e416" gracePeriod=30
Jul 01 17:00:45.707366 ip-10-0-137-244 kubenswrapper[2565]: I0701 17:00:45.707332    2565 generic.go:356] "Generic (PLEG): container finished" podID="1ab9ee0c-b365-44ad-8770-ac00fe1c2c98" containerID="5ecfda4b9aba391adf3ad9260d1d476a29b598f26814eeb8f692d4f530b6e416" exitCode=0
Jul 01 17:00:45.707478 ip-10-0-137-244 kubenswrapper[2565]: I0701 17:00:45.707385    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-b84855d98-h2wv8" event={"ID":"1ab9ee0c-b365-44ad-8770-ac00fe1c2c98","Type":"ContainerDied","Data":"5ecfda4b9aba391adf3ad9260d1d476a29b598f26814eeb8f692d4f530b6e416"}
Jul 01 17:00:45.750722 ip-10-0-137-244 kubenswrapper[2565]: I0701 17:00:45.750703    2565 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-b84855d98-h2wv8"
Jul 01 17:00:45.853653 ip-10-0-137-244 kubenswrapper[2565]: I0701 17:00:45.853583    2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/1ab9ee0c-b365-44ad-8770-ac00fe1c2c98-ca-trust-extracted\") pod \"1ab9ee0c-b365-44ad-8770-ac00fe1c2c98\" (UID: \"1ab9ee0c-b365-44ad-8770-ac00fe1c2c98\") "
Jul 01 17:00:45.853653 ip-10-0-137-244 kubenswrapper[2565]: I0701 17:00:45.853621    2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/1ab9ee0c-b365-44ad-8770-ac00fe1c2c98-bound-sa-token\") pod \"1ab9ee0c-b365-44ad-8770-ac00fe1c2c98\" (UID: \"1ab9ee0c-b365-44ad-8770-ac00fe1c2c98\") "
Jul 01 17:00:45.853835 ip-10-0-137-244 kubenswrapper[2565]: I0701 17:00:45.853666    2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/1ab9ee0c-b365-44ad-8770-ac00fe1c2c98-registry-certificates\") pod \"1ab9ee0c-b365-44ad-8770-ac00fe1c2c98\" (UID: \"1ab9ee0c-b365-44ad-8770-ac00fe1c2c98\") "
Jul 01 17:00:45.853835 ip-10-0-137-244 kubenswrapper[2565]: I0701 17:00:45.853701    2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/1ab9ee0c-b365-44ad-8770-ac00fe1c2c98-registry-tls\") pod \"1ab9ee0c-b365-44ad-8770-ac00fe1c2c98\" (UID: \"1ab9ee0c-b365-44ad-8770-ac00fe1c2c98\") "
Jul 01 17:00:45.853835 ip-10-0-137-244 kubenswrapper[2565]: I0701 17:00:45.853725    2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/1ab9ee0c-b365-44ad-8770-ac00fe1c2c98-image-registry-private-configuration\") pod \"1ab9ee0c-b365-44ad-8770-ac00fe1c2c98\" (UID: \"1ab9ee0c-b365-44ad-8770-ac00fe1c2c98\") "
Jul 01 17:00:45.853835 ip-10-0-137-244 kubenswrapper[2565]: I0701 17:00:45.853756    2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/1ab9ee0c-b365-44ad-8770-ac00fe1c2c98-installation-pull-secrets\") pod \"1ab9ee0c-b365-44ad-8770-ac00fe1c2c98\" (UID: \"1ab9ee0c-b365-44ad-8770-ac00fe1c2c98\") "
Jul 01 17:00:45.853835 ip-10-0-137-244 kubenswrapper[2565]: I0701 17:00:45.853796    2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/1ab9ee0c-b365-44ad-8770-ac00fe1c2c98-trusted-ca\") pod \"1ab9ee0c-b365-44ad-8770-ac00fe1c2c98\" (UID: \"1ab9ee0c-b365-44ad-8770-ac00fe1c2c98\") "
Jul 01 17:00:45.853835 ip-10-0-137-244 kubenswrapper[2565]: I0701 17:00:45.853822    2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-fp9nj\" (UniqueName: \"kubernetes.io/projected/1ab9ee0c-b365-44ad-8770-ac00fe1c2c98-kube-api-access-fp9nj\") pod \"1ab9ee0c-b365-44ad-8770-ac00fe1c2c98\" (UID: \"1ab9ee0c-b365-44ad-8770-ac00fe1c2c98\") "
Jul 01 17:00:45.854349 ip-10-0-137-244 kubenswrapper[2565]: I0701 17:00:45.854219    2565 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/1ab9ee0c-b365-44ad-8770-ac00fe1c2c98-registry-certificates" (OuterVolumeSpecName: "registry-certificates") pod "1ab9ee0c-b365-44ad-8770-ac00fe1c2c98" (UID: "1ab9ee0c-b365-44ad-8770-ac00fe1c2c98"). InnerVolumeSpecName "registry-certificates". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 01 17:00:45.854497 ip-10-0-137-244 kubenswrapper[2565]: I0701 17:00:45.854443    2565 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/1ab9ee0c-b365-44ad-8770-ac00fe1c2c98-trusted-ca" (OuterVolumeSpecName: "trusted-ca") pod "1ab9ee0c-b365-44ad-8770-ac00fe1c2c98" (UID: "1ab9ee0c-b365-44ad-8770-ac00fe1c2c98"). InnerVolumeSpecName "trusted-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 01 17:00:45.856137 ip-10-0-137-244 kubenswrapper[2565]: I0701 17:00:45.856115    2565 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/1ab9ee0c-b365-44ad-8770-ac00fe1c2c98-registry-tls" (OuterVolumeSpecName: "registry-tls") pod "1ab9ee0c-b365-44ad-8770-ac00fe1c2c98" (UID: "1ab9ee0c-b365-44ad-8770-ac00fe1c2c98"). InnerVolumeSpecName "registry-tls". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 01 17:00:45.856277 ip-10-0-137-244 kubenswrapper[2565]: I0701 17:00:45.856255    2565 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/1ab9ee0c-b365-44ad-8770-ac00fe1c2c98-bound-sa-token" (OuterVolumeSpecName: "bound-sa-token") pod "1ab9ee0c-b365-44ad-8770-ac00fe1c2c98" (UID: "1ab9ee0c-b365-44ad-8770-ac00fe1c2c98"). InnerVolumeSpecName "bound-sa-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 01 17:00:45.856391 ip-10-0-137-244 kubenswrapper[2565]: I0701 17:00:45.856373    2565 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/1ab9ee0c-b365-44ad-8770-ac00fe1c2c98-kube-api-access-fp9nj" (OuterVolumeSpecName: "kube-api-access-fp9nj") pod "1ab9ee0c-b365-44ad-8770-ac00fe1c2c98" (UID: "1ab9ee0c-b365-44ad-8770-ac00fe1c2c98"). InnerVolumeSpecName "kube-api-access-fp9nj". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 01 17:00:45.856442 ip-10-0-137-244 kubenswrapper[2565]: I0701 17:00:45.856403    2565 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/1ab9ee0c-b365-44ad-8770-ac00fe1c2c98-installation-pull-secrets" (OuterVolumeSpecName: "installation-pull-secrets") pod "1ab9ee0c-b365-44ad-8770-ac00fe1c2c98" (UID: "1ab9ee0c-b365-44ad-8770-ac00fe1c2c98"). InnerVolumeSpecName "installation-pull-secrets". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 01 17:00:45.856611 ip-10-0-137-244 kubenswrapper[2565]: I0701 17:00:45.856594    2565 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/1ab9ee0c-b365-44ad-8770-ac00fe1c2c98-image-registry-private-configuration" (OuterVolumeSpecName: "image-registry-private-configuration") pod "1ab9ee0c-b365-44ad-8770-ac00fe1c2c98" (UID: "1ab9ee0c-b365-44ad-8770-ac00fe1c2c98"). InnerVolumeSpecName "image-registry-private-configuration". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 01 17:00:45.862168 ip-10-0-137-244 kubenswrapper[2565]: I0701 17:00:45.862146    2565 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/1ab9ee0c-b365-44ad-8770-ac00fe1c2c98-ca-trust-extracted" (OuterVolumeSpecName: "ca-trust-extracted") pod "1ab9ee0c-b365-44ad-8770-ac00fe1c2c98" (UID: "1ab9ee0c-b365-44ad-8770-ac00fe1c2c98"). InnerVolumeSpecName "ca-trust-extracted". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 01 17:00:45.954580 ip-10-0-137-244 kubenswrapper[2565]: I0701 17:00:45.954548    2565 reconciler_common.go:299] "Volume detached for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/1ab9ee0c-b365-44ad-8770-ac00fe1c2c98-registry-tls\") on node \"ip-10-0-137-244.ec2.internal\" DevicePath \"\""
Jul 01 17:00:45.954580 ip-10-0-137-244 kubenswrapper[2565]: I0701 17:00:45.954578    2565 reconciler_common.go:299] "Volume detached for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/1ab9ee0c-b365-44ad-8770-ac00fe1c2c98-image-registry-private-configuration\") on node \"ip-10-0-137-244.ec2.internal\" DevicePath \"\""
Jul 01 17:00:45.954580 ip-10-0-137-244 kubenswrapper[2565]: I0701 17:00:45.954588    2565 reconciler_common.go:299] "Volume detached for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/1ab9ee0c-b365-44ad-8770-ac00fe1c2c98-installation-pull-secrets\") on node \"ip-10-0-137-244.ec2.internal\" DevicePath \"\""
Jul 01 17:00:45.954746 ip-10-0-137-244 kubenswrapper[2565]: I0701 17:00:45.954598    2565 reconciler_common.go:299] "Volume detached for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/1ab9ee0c-b365-44ad-8770-ac00fe1c2c98-trusted-ca\") on node \"ip-10-0-137-244.ec2.internal\" DevicePath \"\""
Jul 01 17:00:45.954746 ip-10-0-137-244 kubenswrapper[2565]: I0701 17:00:45.954606    2565 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-fp9nj\" (UniqueName: \"kubernetes.io/projected/1ab9ee0c-b365-44ad-8770-ac00fe1c2c98-kube-api-access-fp9nj\") on node \"ip-10-0-137-244.ec2.internal\" DevicePath \"\""
Jul 01 17:00:45.954746 ip-10-0-137-244 kubenswrapper[2565]: I0701 17:00:45.954614    2565 reconciler_common.go:299] "Volume detached for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/1ab9ee0c-b365-44ad-8770-ac00fe1c2c98-ca-trust-extracted\") on node \"ip-10-0-137-244.ec2.internal\" DevicePath \"\""
Jul 01 17:00:45.954746 ip-10-0-137-244 kubenswrapper[2565]: I0701 17:00:45.954622    2565 reconciler_common.go:299] "Volume detached for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/1ab9ee0c-b365-44ad-8770-ac00fe1c2c98-bound-sa-token\") on node \"ip-10-0-137-244.ec2.internal\" DevicePath \"\""
Jul 01 17:00:45.954746 ip-10-0-137-244 kubenswrapper[2565]: I0701 17:00:45.954630    2565 reconciler_common.go:299] "Volume detached for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/1ab9ee0c-b365-44ad-8770-ac00fe1c2c98-registry-certificates\") on node \"ip-10-0-137-244.ec2.internal\" DevicePath \"\""
Jul 01 17:00:46.716890 ip-10-0-137-244 kubenswrapper[2565]: I0701 17:00:46.716860    2565 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-b84855d98-h2wv8"
Jul 01 17:00:46.717252 ip-10-0-137-244 kubenswrapper[2565]: I0701 17:00:46.716860    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-b84855d98-h2wv8" event={"ID":"1ab9ee0c-b365-44ad-8770-ac00fe1c2c98","Type":"ContainerDied","Data":"e65c713bb029eecd1304e9fa8fb0318d596d6f0377c9e14be23d5d23be9b8bb4"}
Jul 01 17:00:46.717252 ip-10-0-137-244 kubenswrapper[2565]: I0701 17:00:46.717043    2565 scope.go:117] "RemoveContainer" containerID="5ecfda4b9aba391adf3ad9260d1d476a29b598f26814eeb8f692d4f530b6e416"
Jul 01 17:00:46.737566 ip-10-0-137-244 kubenswrapper[2565]: I0701 17:00:46.737537    2565 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-image-registry/image-registry-b84855d98-h2wv8"]
Jul 01 17:00:46.742291 ip-10-0-137-244 kubenswrapper[2565]: I0701 17:00:46.742264    2565 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-image-registry/image-registry-b84855d98-h2wv8"]
Jul 01 17:00:47.506221 ip-10-0-137-244 kubenswrapper[2565]: I0701 17:00:47.506178    2565 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-69f897785d-dwsw6"
Jul 01 17:00:47.506432 ip-10-0-137-244 kubenswrapper[2565]: I0701 17:00:47.506247    2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-69f897785d-dwsw6"
Jul 01 17:00:47.510939 ip-10-0-137-244 kubenswrapper[2565]: I0701 17:00:47.510915    2565 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-69f897785d-dwsw6"
Jul 01 17:00:47.724052 ip-10-0-137-244 kubenswrapper[2565]: I0701 17:00:47.724033    2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-69f897785d-dwsw6"
Jul 01 17:00:48.248144 ip-10-0-137-244 kubenswrapper[2565]: I0701 17:00:48.248112    2565 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="1ab9ee0c-b365-44ad-8770-ac00fe1c2c98" path="/var/lib/kubelet/pods/1ab9ee0c-b365-44ad-8770-ac00fe1c2c98/volumes"
Jul 01 17:00:58.947404 ip-10-0-137-244 kubenswrapper[2565]: I0701 17:00:58.947364    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/a6488c7c-9ea5-4ad2-91ba-b1ffda18b203-metrics-certs\") pod \"network-metrics-daemon-hthvc\" (UID: \"a6488c7c-9ea5-4ad2-91ba-b1ffda18b203\") " pod="openshift-multus/network-metrics-daemon-hthvc"
Jul 01 17:00:58.949832 ip-10-0-137-244 kubenswrapper[2565]: I0701 17:00:58.949806    2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/a6488c7c-9ea5-4ad2-91ba-b1ffda18b203-metrics-certs\") pod \"network-metrics-daemon-hthvc\" (UID: \"a6488c7c-9ea5-4ad2-91ba-b1ffda18b203\") " pod="openshift-multus/network-metrics-daemon-hthvc"
Jul 01 17:00:58.958320 ip-10-0-137-244 kubenswrapper[2565]: I0701 17:00:58.958297    2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-rp97d\""
Jul 01 17:00:58.966447 ip-10-0-137-244 kubenswrapper[2565]: I0701 17:00:58.966431    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-hthvc"
Jul 01 17:00:59.104840 ip-10-0-137-244 kubenswrapper[2565]: I0701 17:00:59.104742    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-hthvc"]
Jul 01 17:00:59.107622 ip-10-0-137-244 kubenswrapper[2565]: W0701 17:00:59.107597    2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda6488c7c_9ea5_4ad2_91ba_b1ffda18b203.slice/crio-7d1c4ac02454992fdac91fac1228324df651bdbdf7f2f07fe27c8262ab3b2544 WatchSource:0}: Error finding container 7d1c4ac02454992fdac91fac1228324df651bdbdf7f2f07fe27c8262ab3b2544: Status 404 returned error can't find the container with id 7d1c4ac02454992fdac91fac1228324df651bdbdf7f2f07fe27c8262ab3b2544
Jul 01 17:00:59.761602 ip-10-0-137-244 kubenswrapper[2565]: I0701 17:00:59.761564    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-hthvc" event={"ID":"a6488c7c-9ea5-4ad2-91ba-b1ffda18b203","Type":"ContainerStarted","Data":"7d1c4ac02454992fdac91fac1228324df651bdbdf7f2f07fe27c8262ab3b2544"}
Jul 01 17:01:00.767249 ip-10-0-137-244 kubenswrapper[2565]: I0701 17:01:00.767207    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-hthvc" event={"ID":"a6488c7c-9ea5-4ad2-91ba-b1ffda18b203","Type":"ContainerStarted","Data":"f758c1e3d5c54f4459438ccd3dbcc50d4ca9aa944206b2d8bdb0b9c0b3e34cc3"}
Jul 01 17:01:00.767733 ip-10-0-137-244 kubenswrapper[2565]: I0701 17:01:00.767280    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-hthvc" event={"ID":"a6488c7c-9ea5-4ad2-91ba-b1ffda18b203","Type":"ContainerStarted","Data":"a46a48d5154bbf993dab665a27d8f1b163d1c3c155dd6b9a035201a96c3a0346"}
Jul 01 17:01:00.787387 ip-10-0-137-244 kubenswrapper[2565]: I0701 17:01:00.787339    2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/network-metrics-daemon-hthvc" podStartSLOduration=129.745283005 podStartE2EDuration="2m10.78732255s" totalImagesPullingTime="1.042039545s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-01 16:58:50 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-01 17:01:00.78631136 +0000 UTC m=+131.254575484" watchObservedRunningTime="2026-07-01 17:01:00.78732255 +0000 UTC m=+131.255586728"
Jul 01 17:01:03.441881 ip-10-0-137-244 kubenswrapper[2565]: I0701 17:01:03.441850    2565 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-69f897785d-dwsw6"]
Jul 01 17:01:14.814982 ip-10-0-137-244 kubenswrapper[2565]: I0701 17:01:14.814944    2565 generic.go:356] "Generic (PLEG): container finished" podID="4e819e79-27ae-485d-9920-00fcca464c44" containerID="4be2de9a7b83619645ff82d097e64a6ea9c772392219d59092ccf9221fc3e2a0" exitCode=0
Jul 01 17:01:14.815478 ip-10-0-137-244 kubenswrapper[2565]: I0701 17:01:14.815038    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca-operator/service-ca-operator-5d7b7dc95c-6mhnm" event={"ID":"4e819e79-27ae-485d-9920-00fcca464c44","Type":"ContainerDied","Data":"4be2de9a7b83619645ff82d097e64a6ea9c772392219d59092ccf9221fc3e2a0"}
Jul 01 17:01:14.815478 ip-10-0-137-244 kubenswrapper[2565]: I0701 17:01:14.815393    2565 scope.go:117] "RemoveContainer" containerID="4be2de9a7b83619645ff82d097e64a6ea9c772392219d59092ccf9221fc3e2a0"
Jul 01 17:01:15.820761 ip-10-0-137-244 kubenswrapper[2565]: I0701 17:01:15.820726    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca-operator/service-ca-operator-5d7b7dc95c-6mhnm" event={"ID":"4e819e79-27ae-485d-9920-00fcca464c44","Type":"ContainerStarted","Data":"cd72c1b7c00de491a43bba745fd603bf3a26b4d53d110c5802b1eeb617928718"}
Jul 01 17:01:21.192480 ip-10-0-137-244 kubenswrapper[2565]: I0701 17:01:21.192452    2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-4dfnf_b433bedb-7f27-4ad0-a5ce-6d573c5aa004/serve-healthcheck-canary/0.log"
Jul 01 17:01:28.460737 ip-10-0-137-244 kubenswrapper[2565]: I0701 17:01:28.460679    2565 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-console/console-69f897785d-dwsw6" podUID="58a76674-cb3b-4227-827b-1fc6e68f63ae" containerName="console" containerID="cri-o://4071ea0168352095901ae1f1a4a7477122fc47add00c70a0d2a592ef235dc2c1" gracePeriod=15
Jul 01 17:01:28.690533 ip-10-0-137-244 kubenswrapper[2565]: I0701 17:01:28.690504    2565 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-69f897785d-dwsw6"
Jul 01 17:01:28.832802 ip-10-0-137-244 kubenswrapper[2565]: I0701 17:01:28.832741    2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-flchk\" (UniqueName: \"kubernetes.io/projected/58a76674-cb3b-4227-827b-1fc6e68f63ae-kube-api-access-flchk\") pod \"58a76674-cb3b-4227-827b-1fc6e68f63ae\" (UID: \"58a76674-cb3b-4227-827b-1fc6e68f63ae\") "
Jul 01 17:01:28.832802 ip-10-0-137-244 kubenswrapper[2565]: I0701 17:01:28.832777    2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/58a76674-cb3b-4227-827b-1fc6e68f63ae-console-config\") pod \"58a76674-cb3b-4227-827b-1fc6e68f63ae\" (UID: \"58a76674-cb3b-4227-827b-1fc6e68f63ae\") "
Jul 01 17:01:28.832802 ip-10-0-137-244 kubenswrapper[2565]: I0701 17:01:28.832795    2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/58a76674-cb3b-4227-827b-1fc6e68f63ae-service-ca\") pod \"58a76674-cb3b-4227-827b-1fc6e68f63ae\" (UID: \"58a76674-cb3b-4227-827b-1fc6e68f63ae\") "
Jul 01 17:01:28.832972 ip-10-0-137-244 kubenswrapper[2565]: I0701 17:01:28.832814    2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/58a76674-cb3b-4227-827b-1fc6e68f63ae-trusted-ca-bundle\") pod \"58a76674-cb3b-4227-827b-1fc6e68f63ae\" (UID: \"58a76674-cb3b-4227-827b-1fc6e68f63ae\") "
Jul 01 17:01:28.833064 ip-10-0-137-244 kubenswrapper[2565]: I0701 17:01:28.833012    2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/58a76674-cb3b-4227-827b-1fc6e68f63ae-oauth-serving-cert\") pod \"58a76674-cb3b-4227-827b-1fc6e68f63ae\" (UID: \"58a76674-cb3b-4227-827b-1fc6e68f63ae\") "
Jul 01 17:01:28.833180 ip-10-0-137-244 kubenswrapper[2565]: I0701 17:01:28.833086    2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/58a76674-cb3b-4227-827b-1fc6e68f63ae-console-oauth-config\") pod \"58a76674-cb3b-4227-827b-1fc6e68f63ae\" (UID: \"58a76674-cb3b-4227-827b-1fc6e68f63ae\") "
Jul 01 17:01:28.833180 ip-10-0-137-244 kubenswrapper[2565]: I0701 17:01:28.833115    2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/58a76674-cb3b-4227-827b-1fc6e68f63ae-console-serving-cert\") pod \"58a76674-cb3b-4227-827b-1fc6e68f63ae\" (UID: \"58a76674-cb3b-4227-827b-1fc6e68f63ae\") "
Jul 01 17:01:28.833293 ip-10-0-137-244 kubenswrapper[2565]: I0701 17:01:28.833176    2565 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/58a76674-cb3b-4227-827b-1fc6e68f63ae-console-config" (OuterVolumeSpecName: "console-config") pod "58a76674-cb3b-4227-827b-1fc6e68f63ae" (UID: "58a76674-cb3b-4227-827b-1fc6e68f63ae"). InnerVolumeSpecName "console-config". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 01 17:01:28.833362 ip-10-0-137-244 kubenswrapper[2565]: I0701 17:01:28.833331    2565 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/58a76674-cb3b-4227-827b-1fc6e68f63ae-trusted-ca-bundle" (OuterVolumeSpecName: "trusted-ca-bundle") pod "58a76674-cb3b-4227-827b-1fc6e68f63ae" (UID: "58a76674-cb3b-4227-827b-1fc6e68f63ae"). InnerVolumeSpecName "trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 01 17:01:28.833362 ip-10-0-137-244 kubenswrapper[2565]: I0701 17:01:28.833351    2565 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/58a76674-cb3b-4227-827b-1fc6e68f63ae-service-ca" (OuterVolumeSpecName: "service-ca") pod "58a76674-cb3b-4227-827b-1fc6e68f63ae" (UID: "58a76674-cb3b-4227-827b-1fc6e68f63ae"). InnerVolumeSpecName "service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 01 17:01:28.833533 ip-10-0-137-244 kubenswrapper[2565]: I0701 17:01:28.833364    2565 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/58a76674-cb3b-4227-827b-1fc6e68f63ae-oauth-serving-cert" (OuterVolumeSpecName: "oauth-serving-cert") pod "58a76674-cb3b-4227-827b-1fc6e68f63ae" (UID: "58a76674-cb3b-4227-827b-1fc6e68f63ae"). InnerVolumeSpecName "oauth-serving-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 01 17:01:28.833533 ip-10-0-137-244 kubenswrapper[2565]: I0701 17:01:28.833461    2565 reconciler_common.go:299] "Volume detached for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/58a76674-cb3b-4227-827b-1fc6e68f63ae-console-config\") on node \"ip-10-0-137-244.ec2.internal\" DevicePath \"\""
Jul 01 17:01:28.835234 ip-10-0-137-244 kubenswrapper[2565]: I0701 17:01:28.835211    2565 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/58a76674-cb3b-4227-827b-1fc6e68f63ae-console-oauth-config" (OuterVolumeSpecName: "console-oauth-config") pod "58a76674-cb3b-4227-827b-1fc6e68f63ae" (UID: "58a76674-cb3b-4227-827b-1fc6e68f63ae"). InnerVolumeSpecName "console-oauth-config". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 01 17:01:28.835321 ip-10-0-137-244 kubenswrapper[2565]: I0701 17:01:28.835295    2565 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/58a76674-cb3b-4227-827b-1fc6e68f63ae-kube-api-access-flchk" (OuterVolumeSpecName: "kube-api-access-flchk") pod "58a76674-cb3b-4227-827b-1fc6e68f63ae" (UID: "58a76674-cb3b-4227-827b-1fc6e68f63ae"). InnerVolumeSpecName "kube-api-access-flchk". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 01 17:01:28.835388 ip-10-0-137-244 kubenswrapper[2565]: I0701 17:01:28.835369    2565 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/58a76674-cb3b-4227-827b-1fc6e68f63ae-console-serving-cert" (OuterVolumeSpecName: "console-serving-cert") pod "58a76674-cb3b-4227-827b-1fc6e68f63ae" (UID: "58a76674-cb3b-4227-827b-1fc6e68f63ae"). InnerVolumeSpecName "console-serving-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 01 17:01:28.867272 ip-10-0-137-244 kubenswrapper[2565]: I0701 17:01:28.867248    2565 generic.go:356] "Generic (PLEG): container finished" podID="58a76674-cb3b-4227-827b-1fc6e68f63ae" containerID="4071ea0168352095901ae1f1a4a7477122fc47add00c70a0d2a592ef235dc2c1" exitCode=0
Jul 01 17:01:28.867362 ip-10-0-137-244 kubenswrapper[2565]: I0701 17:01:28.867285    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-69f897785d-dwsw6" event={"ID":"58a76674-cb3b-4227-827b-1fc6e68f63ae","Type":"ContainerDied","Data":"4071ea0168352095901ae1f1a4a7477122fc47add00c70a0d2a592ef235dc2c1"}
Jul 01 17:01:28.867362 ip-10-0-137-244 kubenswrapper[2565]: I0701 17:01:28.867288    2565 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-69f897785d-dwsw6"
Jul 01 17:01:28.867362 ip-10-0-137-244 kubenswrapper[2565]: I0701 17:01:28.867330    2565 scope.go:117] "RemoveContainer" containerID="4071ea0168352095901ae1f1a4a7477122fc47add00c70a0d2a592ef235dc2c1"
Jul 01 17:01:28.867516 ip-10-0-137-244 kubenswrapper[2565]: I0701 17:01:28.867320    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-69f897785d-dwsw6" event={"ID":"58a76674-cb3b-4227-827b-1fc6e68f63ae","Type":"ContainerDied","Data":"1242119cfe2f9a2d92b72948ce2e128dc3e49c2259349a1d86a01f3a665edaf8"}
Jul 01 17:01:28.876605 ip-10-0-137-244 kubenswrapper[2565]: I0701 17:01:28.876587    2565 scope.go:117] "RemoveContainer" containerID="4071ea0168352095901ae1f1a4a7477122fc47add00c70a0d2a592ef235dc2c1"
Jul 01 17:01:28.876889 ip-10-0-137-244 kubenswrapper[2565]: E0701 17:01:28.876869    2565 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"4071ea0168352095901ae1f1a4a7477122fc47add00c70a0d2a592ef235dc2c1\": container with ID starting with 4071ea0168352095901ae1f1a4a7477122fc47add00c70a0d2a592ef235dc2c1 not found: ID does not exist" containerID="4071ea0168352095901ae1f1a4a7477122fc47add00c70a0d2a592ef235dc2c1"
Jul 01 17:01:28.876938 ip-10-0-137-244 kubenswrapper[2565]: I0701 17:01:28.876896    2565 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"4071ea0168352095901ae1f1a4a7477122fc47add00c70a0d2a592ef235dc2c1"} err="failed to get container status \"4071ea0168352095901ae1f1a4a7477122fc47add00c70a0d2a592ef235dc2c1\": rpc error: code = NotFound desc = could not find container \"4071ea0168352095901ae1f1a4a7477122fc47add00c70a0d2a592ef235dc2c1\": container with ID starting with 4071ea0168352095901ae1f1a4a7477122fc47add00c70a0d2a592ef235dc2c1 not found: ID does not exist"
Jul 01 17:01:28.890362 ip-10-0-137-244 kubenswrapper[2565]: I0701 17:01:28.890343    2565 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-69f897785d-dwsw6"]
Jul 01 17:01:28.900223 ip-10-0-137-244 kubenswrapper[2565]: I0701 17:01:28.897350    2565 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-console/console-69f897785d-dwsw6"]
Jul 01 17:01:28.933768 ip-10-0-137-244 kubenswrapper[2565]: I0701 17:01:28.933744    2565 reconciler_common.go:299] "Volume detached for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/58a76674-cb3b-4227-827b-1fc6e68f63ae-oauth-serving-cert\") on node \"ip-10-0-137-244.ec2.internal\" DevicePath \"\""
Jul 01 17:01:28.933768 ip-10-0-137-244 kubenswrapper[2565]: I0701 17:01:28.933765    2565 reconciler_common.go:299] "Volume detached for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/58a76674-cb3b-4227-827b-1fc6e68f63ae-console-oauth-config\") on node \"ip-10-0-137-244.ec2.internal\" DevicePath \"\""
Jul 01 17:01:28.933881 ip-10-0-137-244 kubenswrapper[2565]: I0701 17:01:28.933774    2565 reconciler_common.go:299] "Volume detached for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/58a76674-cb3b-4227-827b-1fc6e68f63ae-console-serving-cert\") on node \"ip-10-0-137-244.ec2.internal\" DevicePath \"\""
Jul 01 17:01:28.933881 ip-10-0-137-244 kubenswrapper[2565]: I0701 17:01:28.933782    2565 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-flchk\" (UniqueName: \"kubernetes.io/projected/58a76674-cb3b-4227-827b-1fc6e68f63ae-kube-api-access-flchk\") on node \"ip-10-0-137-244.ec2.internal\" DevicePath \"\""
Jul 01 17:01:28.933881 ip-10-0-137-244 kubenswrapper[2565]: I0701 17:01:28.933791    2565 reconciler_common.go:299] "Volume detached for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/58a76674-cb3b-4227-827b-1fc6e68f63ae-service-ca\") on node \"ip-10-0-137-244.ec2.internal\" DevicePath \"\""
Jul 01 17:01:28.933881 ip-10-0-137-244 kubenswrapper[2565]: I0701 17:01:28.933799    2565 reconciler_common.go:299] "Volume detached for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/58a76674-cb3b-4227-827b-1fc6e68f63ae-trusted-ca-bundle\") on node \"ip-10-0-137-244.ec2.internal\" DevicePath \"\""
Jul 01 17:01:30.247937 ip-10-0-137-244 kubenswrapper[2565]: I0701 17:01:30.247896    2565 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="58a76674-cb3b-4227-827b-1fc6e68f63ae" path="/var/lib/kubelet/pods/58a76674-cb3b-4227-827b-1fc6e68f63ae/volumes"
Jul 01 17:01:54.291010 ip-10-0-137-244 kubenswrapper[2565]: I0701 17:01:54.290979    2565 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-5b74c58dcb-qtxpp"]
Jul 01 17:01:54.291445 ip-10-0-137-244 kubenswrapper[2565]: I0701 17:01:54.291220    2565 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="58a76674-cb3b-4227-827b-1fc6e68f63ae" containerName="console"
Jul 01 17:01:54.291445 ip-10-0-137-244 kubenswrapper[2565]: I0701 17:01:54.291231    2565 state_mem.go:107] "Deleted CPUSet assignment" podUID="58a76674-cb3b-4227-827b-1fc6e68f63ae" containerName="console"
Jul 01 17:01:54.291445 ip-10-0-137-244 kubenswrapper[2565]: I0701 17:01:54.291252    2565 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="1ab9ee0c-b365-44ad-8770-ac00fe1c2c98" containerName="registry"
Jul 01 17:01:54.291445 ip-10-0-137-244 kubenswrapper[2565]: I0701 17:01:54.291258    2565 state_mem.go:107] "Deleted CPUSet assignment" podUID="1ab9ee0c-b365-44ad-8770-ac00fe1c2c98" containerName="registry"
Jul 01 17:01:54.291445 ip-10-0-137-244 kubenswrapper[2565]: I0701 17:01:54.291300    2565 memory_manager.go:368] "RemoveStaleState removing state" podUID="58a76674-cb3b-4227-827b-1fc6e68f63ae" containerName="console"
Jul 01 17:01:54.291445 ip-10-0-137-244 kubenswrapper[2565]: I0701 17:01:54.291308    2565 memory_manager.go:368] "RemoveStaleState removing state" podUID="1ab9ee0c-b365-44ad-8770-ac00fe1c2c98" containerName="registry"
Jul 01 17:01:54.293533 ip-10-0-137-244 kubenswrapper[2565]: I0701 17:01:54.293515    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-5b74c58dcb-qtxpp"
Jul 01 17:01:54.298942 ip-10-0-137-244 kubenswrapper[2565]: I0701 17:01:54.298917    2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-serving-cert\""
Jul 01 17:01:54.299393 ip-10-0-137-244 kubenswrapper[2565]: I0701 17:01:54.299375    2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"service-ca\""
Jul 01 17:01:54.299989 ip-10-0-137-244 kubenswrapper[2565]: I0701 17:01:54.299837    2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"oauth-serving-cert\""
Jul 01 17:01:54.299989 ip-10-0-137-244 kubenswrapper[2565]: I0701 17:01:54.299854    2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-oauth-config\""
Jul 01 17:01:54.300137 ip-10-0-137-244 kubenswrapper[2565]: I0701 17:01:54.299989    2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-dockercfg-4nn42\""
Jul 01 17:01:54.300177 ip-10-0-137-244 kubenswrapper[2565]: I0701 17:01:54.300160    2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"kube-root-ca.crt\""
Jul 01 17:01:54.300301 ip-10-0-137-244 kubenswrapper[2565]: I0701 17:01:54.300281    2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"console-config\""
Jul 01 17:01:54.300420 ip-10-0-137-244 kubenswrapper[2565]: I0701 17:01:54.300402    2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"openshift-service-ca.crt\""
Jul 01 17:01:54.306722 ip-10-0-137-244 kubenswrapper[2565]: I0701 17:01:54.306700    2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"trusted-ca-bundle\""
Jul 01 17:01:54.316447 ip-10-0-137-244 kubenswrapper[2565]: I0701 17:01:54.316421    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-5b74c58dcb-qtxpp"]
Jul 01 17:01:54.397592 ip-10-0-137-244 kubenswrapper[2565]: I0701 17:01:54.397567    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-s8xd4\" (UniqueName: \"kubernetes.io/projected/75034c52-ff8a-4e43-917f-1061c4b9175a-kube-api-access-s8xd4\") pod \"console-5b74c58dcb-qtxpp\" (UID: \"75034c52-ff8a-4e43-917f-1061c4b9175a\") " pod="openshift-console/console-5b74c58dcb-qtxpp"
Jul 01 17:01:54.397696 ip-10-0-137-244 kubenswrapper[2565]: I0701 17:01:54.397619    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/75034c52-ff8a-4e43-917f-1061c4b9175a-console-oauth-config\") pod \"console-5b74c58dcb-qtxpp\" (UID: \"75034c52-ff8a-4e43-917f-1061c4b9175a\") " pod="openshift-console/console-5b74c58dcb-qtxpp"
Jul 01 17:01:54.397769 ip-10-0-137-244 kubenswrapper[2565]: I0701 17:01:54.397750    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/75034c52-ff8a-4e43-917f-1061c4b9175a-console-config\") pod \"console-5b74c58dcb-qtxpp\" (UID: \"75034c52-ff8a-4e43-917f-1061c4b9175a\") " pod="openshift-console/console-5b74c58dcb-qtxpp"
Jul 01 17:01:54.397813 ip-10-0-137-244 kubenswrapper[2565]: I0701 17:01:54.397781    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/75034c52-ff8a-4e43-917f-1061c4b9175a-console-serving-cert\") pod \"console-5b74c58dcb-qtxpp\" (UID: \"75034c52-ff8a-4e43-917f-1061c4b9175a\") " pod="openshift-console/console-5b74c58dcb-qtxpp"
Jul 01 17:01:54.397813 ip-10-0-137-244 kubenswrapper[2565]: I0701 17:01:54.397798    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/75034c52-ff8a-4e43-917f-1061c4b9175a-service-ca\") pod \"console-5b74c58dcb-qtxpp\" (UID: \"75034c52-ff8a-4e43-917f-1061c4b9175a\") " pod="openshift-console/console-5b74c58dcb-qtxpp"
Jul 01 17:01:54.397923 ip-10-0-137-244 kubenswrapper[2565]: I0701 17:01:54.397861    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/75034c52-ff8a-4e43-917f-1061c4b9175a-oauth-serving-cert\") pod \"console-5b74c58dcb-qtxpp\" (UID: \"75034c52-ff8a-4e43-917f-1061c4b9175a\") " pod="openshift-console/console-5b74c58dcb-qtxpp"
Jul 01 17:01:54.397923 ip-10-0-137-244 kubenswrapper[2565]: I0701 17:01:54.397898    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/75034c52-ff8a-4e43-917f-1061c4b9175a-trusted-ca-bundle\") pod \"console-5b74c58dcb-qtxpp\" (UID: \"75034c52-ff8a-4e43-917f-1061c4b9175a\") " pod="openshift-console/console-5b74c58dcb-qtxpp"
Jul 01 17:01:54.498436 ip-10-0-137-244 kubenswrapper[2565]: I0701 17:01:54.498410    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-s8xd4\" (UniqueName: \"kubernetes.io/projected/75034c52-ff8a-4e43-917f-1061c4b9175a-kube-api-access-s8xd4\") pod \"console-5b74c58dcb-qtxpp\" (UID: \"75034c52-ff8a-4e43-917f-1061c4b9175a\") " pod="openshift-console/console-5b74c58dcb-qtxpp"
Jul 01 17:01:54.498548 ip-10-0-137-244 kubenswrapper[2565]: I0701 17:01:54.498442    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/75034c52-ff8a-4e43-917f-1061c4b9175a-console-oauth-config\") pod \"console-5b74c58dcb-qtxpp\" (UID: \"75034c52-ff8a-4e43-917f-1061c4b9175a\") " pod="openshift-console/console-5b74c58dcb-qtxpp"
Jul 01 17:01:54.498548 ip-10-0-137-244 kubenswrapper[2565]: I0701 17:01:54.498461    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/75034c52-ff8a-4e43-917f-1061c4b9175a-console-config\") pod \"console-5b74c58dcb-qtxpp\" (UID: \"75034c52-ff8a-4e43-917f-1061c4b9175a\") " pod="openshift-console/console-5b74c58dcb-qtxpp"
Jul 01 17:01:54.498548 ip-10-0-137-244 kubenswrapper[2565]: I0701 17:01:54.498478    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/75034c52-ff8a-4e43-917f-1061c4b9175a-console-serving-cert\") pod \"console-5b74c58dcb-qtxpp\" (UID: \"75034c52-ff8a-4e43-917f-1061c4b9175a\") " pod="openshift-console/console-5b74c58dcb-qtxpp"
Jul 01 17:01:54.498548 ip-10-0-137-244 kubenswrapper[2565]: I0701 17:01:54.498493    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/75034c52-ff8a-4e43-917f-1061c4b9175a-service-ca\") pod \"console-5b74c58dcb-qtxpp\" (UID: \"75034c52-ff8a-4e43-917f-1061c4b9175a\") " pod="openshift-console/console-5b74c58dcb-qtxpp"
Jul 01 17:01:54.498548 ip-10-0-137-244 kubenswrapper[2565]: I0701 17:01:54.498515    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/75034c52-ff8a-4e43-917f-1061c4b9175a-oauth-serving-cert\") pod \"console-5b74c58dcb-qtxpp\" (UID: \"75034c52-ff8a-4e43-917f-1061c4b9175a\") " pod="openshift-console/console-5b74c58dcb-qtxpp"
Jul 01 17:01:54.498548 ip-10-0-137-244 kubenswrapper[2565]: I0701 17:01:54.498547    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/75034c52-ff8a-4e43-917f-1061c4b9175a-trusted-ca-bundle\") pod \"console-5b74c58dcb-qtxpp\" (UID: \"75034c52-ff8a-4e43-917f-1061c4b9175a\") " pod="openshift-console/console-5b74c58dcb-qtxpp"
Jul 01 17:01:54.499293 ip-10-0-137-244 kubenswrapper[2565]: I0701 17:01:54.499256    2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/75034c52-ff8a-4e43-917f-1061c4b9175a-service-ca\") pod \"console-5b74c58dcb-qtxpp\" (UID: \"75034c52-ff8a-4e43-917f-1061c4b9175a\") " pod="openshift-console/console-5b74c58dcb-qtxpp"
Jul 01 17:01:54.499408 ip-10-0-137-244 kubenswrapper[2565]: I0701 17:01:54.499325    2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/75034c52-ff8a-4e43-917f-1061c4b9175a-oauth-serving-cert\") pod \"console-5b74c58dcb-qtxpp\" (UID: \"75034c52-ff8a-4e43-917f-1061c4b9175a\") " pod="openshift-console/console-5b74c58dcb-qtxpp"
Jul 01 17:01:54.499475 ip-10-0-137-244 kubenswrapper[2565]: I0701 17:01:54.499406    2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/75034c52-ff8a-4e43-917f-1061c4b9175a-console-config\") pod \"console-5b74c58dcb-qtxpp\" (UID: \"75034c52-ff8a-4e43-917f-1061c4b9175a\") " pod="openshift-console/console-5b74c58dcb-qtxpp"
Jul 01 17:01:54.499533 ip-10-0-137-244 kubenswrapper[2565]: I0701 17:01:54.499492    2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/75034c52-ff8a-4e43-917f-1061c4b9175a-trusted-ca-bundle\") pod \"console-5b74c58dcb-qtxpp\" (UID: \"75034c52-ff8a-4e43-917f-1061c4b9175a\") " pod="openshift-console/console-5b74c58dcb-qtxpp"
Jul 01 17:01:54.501146 ip-10-0-137-244 kubenswrapper[2565]: I0701 17:01:54.501117    2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/75034c52-ff8a-4e43-917f-1061c4b9175a-console-oauth-config\") pod \"console-5b74c58dcb-qtxpp\" (UID: \"75034c52-ff8a-4e43-917f-1061c4b9175a\") " pod="openshift-console/console-5b74c58dcb-qtxpp"
Jul 01 17:01:54.501237 ip-10-0-137-244 kubenswrapper[2565]: I0701 17:01:54.501167    2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/75034c52-ff8a-4e43-917f-1061c4b9175a-console-serving-cert\") pod \"console-5b74c58dcb-qtxpp\" (UID: \"75034c52-ff8a-4e43-917f-1061c4b9175a\") " pod="openshift-console/console-5b74c58dcb-qtxpp"
Jul 01 17:01:54.506786 ip-10-0-137-244 kubenswrapper[2565]: I0701 17:01:54.506768    2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-s8xd4\" (UniqueName: \"kubernetes.io/projected/75034c52-ff8a-4e43-917f-1061c4b9175a-kube-api-access-s8xd4\") pod \"console-5b74c58dcb-qtxpp\" (UID: \"75034c52-ff8a-4e43-917f-1061c4b9175a\") " pod="openshift-console/console-5b74c58dcb-qtxpp"
Jul 01 17:01:54.602897 ip-10-0-137-244 kubenswrapper[2565]: I0701 17:01:54.602842    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-5b74c58dcb-qtxpp"
Jul 01 17:01:54.745174 ip-10-0-137-244 kubenswrapper[2565]: I0701 17:01:54.745150    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-5b74c58dcb-qtxpp"]
Jul 01 17:01:54.747568 ip-10-0-137-244 kubenswrapper[2565]: W0701 17:01:54.747533    2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod75034c52_ff8a_4e43_917f_1061c4b9175a.slice/crio-75bd41a37c02a05ea76892aba454d53f0145e66a82b6c2f8be3d5fca0eff12ce WatchSource:0}: Error finding container 75bd41a37c02a05ea76892aba454d53f0145e66a82b6c2f8be3d5fca0eff12ce: Status 404 returned error can't find the container with id 75bd41a37c02a05ea76892aba454d53f0145e66a82b6c2f8be3d5fca0eff12ce
Jul 01 17:01:54.961810 ip-10-0-137-244 kubenswrapper[2565]: I0701 17:01:54.961780    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-5b74c58dcb-qtxpp" event={"ID":"75034c52-ff8a-4e43-917f-1061c4b9175a","Type":"ContainerStarted","Data":"413411056cd1dcd8dffa34e4d5109c298f43c944c6d2f608063eeb6a289718f8"}
Jul 01 17:01:54.961956 ip-10-0-137-244 kubenswrapper[2565]: I0701 17:01:54.961829    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-5b74c58dcb-qtxpp" event={"ID":"75034c52-ff8a-4e43-917f-1061c4b9175a","Type":"ContainerStarted","Data":"75bd41a37c02a05ea76892aba454d53f0145e66a82b6c2f8be3d5fca0eff12ce"}
Jul 01 17:01:54.982525 ip-10-0-137-244 kubenswrapper[2565]: I0701 17:01:54.982481    2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-5b74c58dcb-qtxpp" podStartSLOduration=0.98246281 podStartE2EDuration="982.46281ms" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-01 17:01:54 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-01 17:01:54.981057132 +0000 UTC m=+185.449321253" watchObservedRunningTime="2026-07-01 17:01:54.98246281 +0000 UTC m=+185.450726940"
Jul 01 17:02:04.603590 ip-10-0-137-244 kubenswrapper[2565]: I0701 17:02:04.603543    2565 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-5b74c58dcb-qtxpp"
Jul 01 17:02:04.604116 ip-10-0-137-244 kubenswrapper[2565]: I0701 17:02:04.603607    2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-5b74c58dcb-qtxpp"
Jul 01 17:02:04.608330 ip-10-0-137-244 kubenswrapper[2565]: I0701 17:02:04.608310    2565 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-5b74c58dcb-qtxpp"
Jul 01 17:02:05.001882 ip-10-0-137-244 kubenswrapper[2565]: I0701 17:02:05.001854    2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-5b74c58dcb-qtxpp"
Jul 01 17:03:10.918768 ip-10-0-137-244 kubenswrapper[2565]: I0701 17:03:10.918718    2565 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-marketplace/74e94af6ec69867d65f1d90648440bdc4f0845d5e32c4818a460ebb589brc5r"]
Jul 01 17:03:10.922281 ip-10-0-137-244 kubenswrapper[2565]: I0701 17:03:10.922255    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/74e94af6ec69867d65f1d90648440bdc4f0845d5e32c4818a460ebb589brc5r"
Jul 01 17:03:10.924990 ip-10-0-137-244 kubenswrapper[2565]: I0701 17:03:10.924967    2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-marketplace\"/\"default-dockercfg-jptf6\""
Jul 01 17:03:10.925579 ip-10-0-137-244 kubenswrapper[2565]: I0701 17:03:10.925560    2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"kube-root-ca.crt\""
Jul 01 17:03:10.925922 ip-10-0-137-244 kubenswrapper[2565]: I0701 17:03:10.925898    2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"openshift-service-ca.crt\""
Jul 01 17:03:10.935801 ip-10-0-137-244 kubenswrapper[2565]: I0701 17:03:10.935777    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/74e94af6ec69867d65f1d90648440bdc4f0845d5e32c4818a460ebb589brc5r"]
Jul 01 17:03:11.006091 ip-10-0-137-244 kubenswrapper[2565]: I0701 17:03:11.006067    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/f577b235-3a97-40ea-9644-755bc65ab754-bundle\") pod \"74e94af6ec69867d65f1d90648440bdc4f0845d5e32c4818a460ebb589brc5r\" (UID: \"f577b235-3a97-40ea-9644-755bc65ab754\") " pod="openshift-marketplace/74e94af6ec69867d65f1d90648440bdc4f0845d5e32c4818a460ebb589brc5r"
Jul 01 17:03:11.006185 ip-10-0-137-244 kubenswrapper[2565]: I0701 17:03:11.006113    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dlmm2\" (UniqueName: \"kubernetes.io/projected/f577b235-3a97-40ea-9644-755bc65ab754-kube-api-access-dlmm2\") pod \"74e94af6ec69867d65f1d90648440bdc4f0845d5e32c4818a460ebb589brc5r\" (UID: \"f577b235-3a97-40ea-9644-755bc65ab754\") " pod="openshift-marketplace/74e94af6ec69867d65f1d90648440bdc4f0845d5e32c4818a460ebb589brc5r"
Jul 01 17:03:11.006185 ip-10-0-137-244 kubenswrapper[2565]: I0701 17:03:11.006166    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/f577b235-3a97-40ea-9644-755bc65ab754-util\") pod \"74e94af6ec69867d65f1d90648440bdc4f0845d5e32c4818a460ebb589brc5r\" (UID: \"f577b235-3a97-40ea-9644-755bc65ab754\") " pod="openshift-marketplace/74e94af6ec69867d65f1d90648440bdc4f0845d5e32c4818a460ebb589brc5r"
Jul 01 17:03:11.107178 ip-10-0-137-244 kubenswrapper[2565]: I0701 17:03:11.107141    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dlmm2\" (UniqueName: \"kubernetes.io/projected/f577b235-3a97-40ea-9644-755bc65ab754-kube-api-access-dlmm2\") pod \"74e94af6ec69867d65f1d90648440bdc4f0845d5e32c4818a460ebb589brc5r\" (UID: \"f577b235-3a97-40ea-9644-755bc65ab754\") " pod="openshift-marketplace/74e94af6ec69867d65f1d90648440bdc4f0845d5e32c4818a460ebb589brc5r"
Jul 01 17:03:11.107279 ip-10-0-137-244 kubenswrapper[2565]: I0701 17:03:11.107197    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/f577b235-3a97-40ea-9644-755bc65ab754-util\") pod \"74e94af6ec69867d65f1d90648440bdc4f0845d5e32c4818a460ebb589brc5r\" (UID: \"f577b235-3a97-40ea-9644-755bc65ab754\") " pod="openshift-marketplace/74e94af6ec69867d65f1d90648440bdc4f0845d5e32c4818a460ebb589brc5r"
Jul 01 17:03:11.107279 ip-10-0-137-244 kubenswrapper[2565]: I0701 17:03:11.107241    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/f577b235-3a97-40ea-9644-755bc65ab754-bundle\") pod \"74e94af6ec69867d65f1d90648440bdc4f0845d5e32c4818a460ebb589brc5r\" (UID: \"f577b235-3a97-40ea-9644-755bc65ab754\") " pod="openshift-marketplace/74e94af6ec69867d65f1d90648440bdc4f0845d5e32c4818a460ebb589brc5r"
Jul 01 17:03:11.107624 ip-10-0-137-244 kubenswrapper[2565]: I0701 17:03:11.107603    2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/f577b235-3a97-40ea-9644-755bc65ab754-util\") pod \"74e94af6ec69867d65f1d90648440bdc4f0845d5e32c4818a460ebb589brc5r\" (UID: \"f577b235-3a97-40ea-9644-755bc65ab754\") " pod="openshift-marketplace/74e94af6ec69867d65f1d90648440bdc4f0845d5e32c4818a460ebb589brc5r"
Jul 01 17:03:11.107669 ip-10-0-137-244 kubenswrapper[2565]: I0701 17:03:11.107648    2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/f577b235-3a97-40ea-9644-755bc65ab754-bundle\") pod \"74e94af6ec69867d65f1d90648440bdc4f0845d5e32c4818a460ebb589brc5r\" (UID: \"f577b235-3a97-40ea-9644-755bc65ab754\") " pod="openshift-marketplace/74e94af6ec69867d65f1d90648440bdc4f0845d5e32c4818a460ebb589brc5r"
Jul 01 17:03:11.118305 ip-10-0-137-244 kubenswrapper[2565]: I0701 17:03:11.118275    2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-dlmm2\" (UniqueName: \"kubernetes.io/projected/f577b235-3a97-40ea-9644-755bc65ab754-kube-api-access-dlmm2\") pod \"74e94af6ec69867d65f1d90648440bdc4f0845d5e32c4818a460ebb589brc5r\" (UID: \"f577b235-3a97-40ea-9644-755bc65ab754\") " pod="openshift-marketplace/74e94af6ec69867d65f1d90648440bdc4f0845d5e32c4818a460ebb589brc5r"
Jul 01 17:03:11.232927 ip-10-0-137-244 kubenswrapper[2565]: I0701 17:03:11.232861    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/74e94af6ec69867d65f1d90648440bdc4f0845d5e32c4818a460ebb589brc5r"
Jul 01 17:03:11.368132 ip-10-0-137-244 kubenswrapper[2565]: I0701 17:03:11.368106    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/74e94af6ec69867d65f1d90648440bdc4f0845d5e32c4818a460ebb589brc5r"]
Jul 01 17:03:11.370517 ip-10-0-137-244 kubenswrapper[2565]: W0701 17:03:11.370485    2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf577b235_3a97_40ea_9644_755bc65ab754.slice/crio-df86759034b2721b284f5bda17c102822e4b88bb472d0aa7d0af2dae7b5fd6cd WatchSource:0}: Error finding container df86759034b2721b284f5bda17c102822e4b88bb472d0aa7d0af2dae7b5fd6cd: Status 404 returned error can't find the container with id df86759034b2721b284f5bda17c102822e4b88bb472d0aa7d0af2dae7b5fd6cd
Jul 01 17:03:12.241772 ip-10-0-137-244 kubenswrapper[2565]: I0701 17:03:12.241687    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/74e94af6ec69867d65f1d90648440bdc4f0845d5e32c4818a460ebb589brc5r" event={"ID":"f577b235-3a97-40ea-9644-755bc65ab754","Type":"ContainerStarted","Data":"df86759034b2721b284f5bda17c102822e4b88bb472d0aa7d0af2dae7b5fd6cd"}
Jul 01 17:03:19.268066 ip-10-0-137-244 kubenswrapper[2565]: I0701 17:03:19.267999    2565 generic.go:356] "Generic (PLEG): container finished" podID="f577b235-3a97-40ea-9644-755bc65ab754" containerID="966806a278e88e8fabfb5f6dbc7bb102f6241f9226514694376370d318c72fff" exitCode=0
Jul 01 17:03:19.268487 ip-10-0-137-244 kubenswrapper[2565]: I0701 17:03:19.268099    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/74e94af6ec69867d65f1d90648440bdc4f0845d5e32c4818a460ebb589brc5r" event={"ID":"f577b235-3a97-40ea-9644-755bc65ab754","Type":"ContainerDied","Data":"966806a278e88e8fabfb5f6dbc7bb102f6241f9226514694376370d318c72fff"}
Jul 01 17:03:21.277084 ip-10-0-137-244 kubenswrapper[2565]: I0701 17:03:21.277054    2565 generic.go:356] "Generic (PLEG): container finished" podID="f577b235-3a97-40ea-9644-755bc65ab754" containerID="5b9845f3463813a979f5405c737f536c90a1e2ebf300a1908b4b2e3359fbd936" exitCode=0
Jul 01 17:03:21.277455 ip-10-0-137-244 kubenswrapper[2565]: I0701 17:03:21.277104    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/74e94af6ec69867d65f1d90648440bdc4f0845d5e32c4818a460ebb589brc5r" event={"ID":"f577b235-3a97-40ea-9644-755bc65ab754","Type":"ContainerDied","Data":"5b9845f3463813a979f5405c737f536c90a1e2ebf300a1908b4b2e3359fbd936"}
Jul 01 17:03:29.308348 ip-10-0-137-244 kubenswrapper[2565]: I0701 17:03:29.308304    2565 generic.go:356] "Generic (PLEG): container finished" podID="f577b235-3a97-40ea-9644-755bc65ab754" containerID="089055a386430e6f99e3d1bfc48103f9ef6282dd19cdebbac66cd22bbdf7ee39" exitCode=0
Jul 01 17:03:29.308680 ip-10-0-137-244 kubenswrapper[2565]: I0701 17:03:29.308365    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/74e94af6ec69867d65f1d90648440bdc4f0845d5e32c4818a460ebb589brc5r" event={"ID":"f577b235-3a97-40ea-9644-755bc65ab754","Type":"ContainerDied","Data":"089055a386430e6f99e3d1bfc48103f9ef6282dd19cdebbac66cd22bbdf7ee39"}
Jul 01 17:03:30.436051 ip-10-0-137-244 kubenswrapper[2565]: I0701 17:03:30.436008    2565 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/74e94af6ec69867d65f1d90648440bdc4f0845d5e32c4818a460ebb589brc5r"
Jul 01 17:03:30.541372 ip-10-0-137-244 kubenswrapper[2565]: I0701 17:03:30.541343    2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/f577b235-3a97-40ea-9644-755bc65ab754-bundle\") pod \"f577b235-3a97-40ea-9644-755bc65ab754\" (UID: \"f577b235-3a97-40ea-9644-755bc65ab754\") "
Jul 01 17:03:30.541478 ip-10-0-137-244 kubenswrapper[2565]: I0701 17:03:30.541398    2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-dlmm2\" (UniqueName: \"kubernetes.io/projected/f577b235-3a97-40ea-9644-755bc65ab754-kube-api-access-dlmm2\") pod \"f577b235-3a97-40ea-9644-755bc65ab754\" (UID: \"f577b235-3a97-40ea-9644-755bc65ab754\") "
Jul 01 17:03:30.541478 ip-10-0-137-244 kubenswrapper[2565]: I0701 17:03:30.541421    2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/f577b235-3a97-40ea-9644-755bc65ab754-util\") pod \"f577b235-3a97-40ea-9644-755bc65ab754\" (UID: \"f577b235-3a97-40ea-9644-755bc65ab754\") "
Jul 01 17:03:30.541974 ip-10-0-137-244 kubenswrapper[2565]: I0701 17:03:30.541955    2565 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f577b235-3a97-40ea-9644-755bc65ab754-bundle" (OuterVolumeSpecName: "bundle") pod "f577b235-3a97-40ea-9644-755bc65ab754" (UID: "f577b235-3a97-40ea-9644-755bc65ab754"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 01 17:03:30.543429 ip-10-0-137-244 kubenswrapper[2565]: I0701 17:03:30.543395    2565 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f577b235-3a97-40ea-9644-755bc65ab754-kube-api-access-dlmm2" (OuterVolumeSpecName: "kube-api-access-dlmm2") pod "f577b235-3a97-40ea-9644-755bc65ab754" (UID: "f577b235-3a97-40ea-9644-755bc65ab754"). InnerVolumeSpecName "kube-api-access-dlmm2". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 01 17:03:30.545689 ip-10-0-137-244 kubenswrapper[2565]: I0701 17:03:30.545666    2565 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f577b235-3a97-40ea-9644-755bc65ab754-util" (OuterVolumeSpecName: "util") pod "f577b235-3a97-40ea-9644-755bc65ab754" (UID: "f577b235-3a97-40ea-9644-755bc65ab754"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 01 17:03:30.641951 ip-10-0-137-244 kubenswrapper[2565]: I0701 17:03:30.641928    2565 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/f577b235-3a97-40ea-9644-755bc65ab754-bundle\") on node \"ip-10-0-137-244.ec2.internal\" DevicePath \"\""
Jul 01 17:03:30.641951 ip-10-0-137-244 kubenswrapper[2565]: I0701 17:03:30.641950    2565 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-dlmm2\" (UniqueName: \"kubernetes.io/projected/f577b235-3a97-40ea-9644-755bc65ab754-kube-api-access-dlmm2\") on node \"ip-10-0-137-244.ec2.internal\" DevicePath \"\""
Jul 01 17:03:30.642095 ip-10-0-137-244 kubenswrapper[2565]: I0701 17:03:30.641961    2565 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/f577b235-3a97-40ea-9644-755bc65ab754-util\") on node \"ip-10-0-137-244.ec2.internal\" DevicePath \"\""
Jul 01 17:03:31.321008 ip-10-0-137-244 kubenswrapper[2565]: I0701 17:03:31.320984    2565 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/74e94af6ec69867d65f1d90648440bdc4f0845d5e32c4818a460ebb589brc5r"
Jul 01 17:03:31.321008 ip-10-0-137-244 kubenswrapper[2565]: I0701 17:03:31.320993    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/74e94af6ec69867d65f1d90648440bdc4f0845d5e32c4818a460ebb589brc5r" event={"ID":"f577b235-3a97-40ea-9644-755bc65ab754","Type":"ContainerDied","Data":"df86759034b2721b284f5bda17c102822e4b88bb472d0aa7d0af2dae7b5fd6cd"}
Jul 01 17:03:31.321179 ip-10-0-137-244 kubenswrapper[2565]: I0701 17:03:31.321049    2565 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="df86759034b2721b284f5bda17c102822e4b88bb472d0aa7d0af2dae7b5fd6cd"
Jul 01 17:03:33.932823 ip-10-0-137-244 kubenswrapper[2565]: I0701 17:03:33.932784    2565 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-keda/custom-metrics-autoscaler-operator-75f7c5f8f-pttcg"]
Jul 01 17:03:33.933279 ip-10-0-137-244 kubenswrapper[2565]: I0701 17:03:33.933070    2565 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f577b235-3a97-40ea-9644-755bc65ab754" containerName="extract"
Jul 01 17:03:33.933279 ip-10-0-137-244 kubenswrapper[2565]: I0701 17:03:33.933087    2565 state_mem.go:107] "Deleted CPUSet assignment" podUID="f577b235-3a97-40ea-9644-755bc65ab754" containerName="extract"
Jul 01 17:03:33.933279 ip-10-0-137-244 kubenswrapper[2565]: I0701 17:03:33.933103    2565 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f577b235-3a97-40ea-9644-755bc65ab754" containerName="util"
Jul 01 17:03:33.933279 ip-10-0-137-244 kubenswrapper[2565]: I0701 17:03:33.933108    2565 state_mem.go:107] "Deleted CPUSet assignment" podUID="f577b235-3a97-40ea-9644-755bc65ab754" containerName="util"
Jul 01 17:03:33.933279 ip-10-0-137-244 kubenswrapper[2565]: I0701 17:03:33.933121    2565 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f577b235-3a97-40ea-9644-755bc65ab754" containerName="pull"
Jul 01 17:03:33.933279 ip-10-0-137-244 kubenswrapper[2565]: I0701 17:03:33.933127    2565 state_mem.go:107] "Deleted CPUSet assignment" podUID="f577b235-3a97-40ea-9644-755bc65ab754" containerName="pull"
Jul 01 17:03:33.933279 ip-10-0-137-244 kubenswrapper[2565]: I0701 17:03:33.933181    2565 memory_manager.go:368] "RemoveStaleState removing state" podUID="f577b235-3a97-40ea-9644-755bc65ab754" containerName="extract"
Jul 01 17:03:33.935128 ip-10-0-137-244 kubenswrapper[2565]: I0701 17:03:33.935109    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-keda/custom-metrics-autoscaler-operator-75f7c5f8f-pttcg"
Jul 01 17:03:33.938711 ip-10-0-137-244 kubenswrapper[2565]: I0701 17:03:33.938686    2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-keda\"/\"kedaorg-certs\""
Jul 01 17:03:33.938867 ip-10-0-137-244 kubenswrapper[2565]: I0701 17:03:33.938765    2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-keda\"/\"custom-metrics-autoscaler-operator-dockercfg-6vqgl\""
Jul 01 17:03:33.939375 ip-10-0-137-244 kubenswrapper[2565]: I0701 17:03:33.939358    2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-keda\"/\"openshift-service-ca.crt\""
Jul 01 17:03:33.939739 ip-10-0-137-244 kubenswrapper[2565]: I0701 17:03:33.939722    2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-keda\"/\"kube-root-ca.crt\""
Jul 01 17:03:33.951855 ip-10-0-137-244 kubenswrapper[2565]: I0701 17:03:33.951829    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-keda/custom-metrics-autoscaler-operator-75f7c5f8f-pttcg"]
Jul 01 17:03:34.066876 ip-10-0-137-244 kubenswrapper[2565]: I0701 17:03:34.066844    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5vb54\" (UniqueName: \"kubernetes.io/projected/ba379dfc-e013-4371-94ef-938934f4ca4f-kube-api-access-5vb54\") pod \"custom-metrics-autoscaler-operator-75f7c5f8f-pttcg\" (UID: \"ba379dfc-e013-4371-94ef-938934f4ca4f\") " pod="openshift-keda/custom-metrics-autoscaler-operator-75f7c5f8f-pttcg"
Jul 01 17:03:34.066972 ip-10-0-137-244 kubenswrapper[2565]: I0701 17:03:34.066899    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"certificates\" (UniqueName: \"kubernetes.io/secret/ba379dfc-e013-4371-94ef-938934f4ca4f-certificates\") pod \"custom-metrics-autoscaler-operator-75f7c5f8f-pttcg\" (UID: \"ba379dfc-e013-4371-94ef-938934f4ca4f\") " pod="openshift-keda/custom-metrics-autoscaler-operator-75f7c5f8f-pttcg"
Jul 01 17:03:34.167215 ip-10-0-137-244 kubenswrapper[2565]: I0701 17:03:34.167188    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5vb54\" (UniqueName: \"kubernetes.io/projected/ba379dfc-e013-4371-94ef-938934f4ca4f-kube-api-access-5vb54\") pod \"custom-metrics-autoscaler-operator-75f7c5f8f-pttcg\" (UID: \"ba379dfc-e013-4371-94ef-938934f4ca4f\") " pod="openshift-keda/custom-metrics-autoscaler-operator-75f7c5f8f-pttcg"
Jul 01 17:03:34.167308 ip-10-0-137-244 kubenswrapper[2565]: I0701 17:03:34.167233    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"certificates\" (UniqueName: \"kubernetes.io/secret/ba379dfc-e013-4371-94ef-938934f4ca4f-certificates\") pod \"custom-metrics-autoscaler-operator-75f7c5f8f-pttcg\" (UID: \"ba379dfc-e013-4371-94ef-938934f4ca4f\") " pod="openshift-keda/custom-metrics-autoscaler-operator-75f7c5f8f-pttcg"
Jul 01 17:03:34.169552 ip-10-0-137-244 kubenswrapper[2565]: I0701 17:03:34.169524    2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"certificates\" (UniqueName: \"kubernetes.io/secret/ba379dfc-e013-4371-94ef-938934f4ca4f-certificates\") pod \"custom-metrics-autoscaler-operator-75f7c5f8f-pttcg\" (UID: \"ba379dfc-e013-4371-94ef-938934f4ca4f\") " pod="openshift-keda/custom-metrics-autoscaler-operator-75f7c5f8f-pttcg"
Jul 01 17:03:34.185201 ip-10-0-137-244 kubenswrapper[2565]: I0701 17:03:34.185147    2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-5vb54\" (UniqueName: \"kubernetes.io/projected/ba379dfc-e013-4371-94ef-938934f4ca4f-kube-api-access-5vb54\") pod \"custom-metrics-autoscaler-operator-75f7c5f8f-pttcg\" (UID: \"ba379dfc-e013-4371-94ef-938934f4ca4f\") " pod="openshift-keda/custom-metrics-autoscaler-operator-75f7c5f8f-pttcg"
Jul 01 17:03:34.245411 ip-10-0-137-244 kubenswrapper[2565]: I0701 17:03:34.245390    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-keda/custom-metrics-autoscaler-operator-75f7c5f8f-pttcg"
Jul 01 17:03:34.389325 ip-10-0-137-244 kubenswrapper[2565]: I0701 17:03:34.389294    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-keda/custom-metrics-autoscaler-operator-75f7c5f8f-pttcg"]
Jul 01 17:03:34.393006 ip-10-0-137-244 kubenswrapper[2565]: W0701 17:03:34.392971    2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podba379dfc_e013_4371_94ef_938934f4ca4f.slice/crio-bbc1e2511d067da7062ff402700d64ef925e002cbb71052ae34f5dfc465ea6b7 WatchSource:0}: Error finding container bbc1e2511d067da7062ff402700d64ef925e002cbb71052ae34f5dfc465ea6b7: Status 404 returned error can't find the container with id bbc1e2511d067da7062ff402700d64ef925e002cbb71052ae34f5dfc465ea6b7
Jul 01 17:03:35.336453 ip-10-0-137-244 kubenswrapper[2565]: I0701 17:03:35.336418    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-keda/custom-metrics-autoscaler-operator-75f7c5f8f-pttcg" event={"ID":"ba379dfc-e013-4371-94ef-938934f4ca4f","Type":"ContainerStarted","Data":"bbc1e2511d067da7062ff402700d64ef925e002cbb71052ae34f5dfc465ea6b7"}
Jul 01 17:03:37.345684 ip-10-0-137-244 kubenswrapper[2565]: I0701 17:03:37.345603    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-keda/custom-metrics-autoscaler-operator-75f7c5f8f-pttcg" event={"ID":"ba379dfc-e013-4371-94ef-938934f4ca4f","Type":"ContainerStarted","Data":"18361f495addab3a1f87225aa00bb7d871873d5164f4c360260fadf8fb76c446"}
Jul 01 17:03:37.346009 ip-10-0-137-244 kubenswrapper[2565]: I0701 17:03:37.345735    2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-keda/custom-metrics-autoscaler-operator-75f7c5f8f-pttcg"
Jul 01 17:03:37.375462 ip-10-0-137-244 kubenswrapper[2565]: I0701 17:03:37.375409    2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-keda/custom-metrics-autoscaler-operator-75f7c5f8f-pttcg" podStartSLOduration=1.775989662 podStartE2EDuration="4.375391025s" totalImagesPullingTime="2.599401363s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-01 17:03:33 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-01 17:03:37.373513665 +0000 UTC m=+287.841777789" watchObservedRunningTime="2026-07-01 17:03:37.375391025 +0000 UTC m=+287.843655151"
Jul 01 17:03:38.907203 ip-10-0-137-244 kubenswrapper[2565]: I0701 17:03:38.907172    2565 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-keda/keda-metrics-apiserver-559f75f947-vl8z7"]
Jul 01 17:03:38.909198 ip-10-0-137-244 kubenswrapper[2565]: I0701 17:03:38.909178    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-keda/keda-metrics-apiserver-559f75f947-vl8z7"
Jul 01 17:03:38.913676 ip-10-0-137-244 kubenswrapper[2565]: I0701 17:03:38.913659    2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-keda\"/\"keda-ocp-cabundle\""
Jul 01 17:03:38.913788 ip-10-0-137-244 kubenswrapper[2565]: I0701 17:03:38.913664    2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-keda\"/\"keda-metrics-apiserver-certs\""
Jul 01 17:03:38.918017 ip-10-0-137-244 kubenswrapper[2565]: I0701 17:03:38.917990    2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-keda\"/\"keda-operator-dockercfg-cnw99\""
Jul 01 17:03:38.929873 ip-10-0-137-244 kubenswrapper[2565]: I0701 17:03:38.929852    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-keda/keda-metrics-apiserver-559f75f947-vl8z7"]
Jul 01 17:03:39.000062 ip-10-0-137-244 kubenswrapper[2565]: I0701 17:03:39.000042    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"temp-vol\" (UniqueName: \"kubernetes.io/empty-dir/e51aab2e-c112-4e48-a37b-46db0ef5e33f-temp-vol\") pod \"keda-metrics-apiserver-559f75f947-vl8z7\" (UID: \"e51aab2e-c112-4e48-a37b-46db0ef5e33f\") " pod="openshift-keda/keda-metrics-apiserver-559f75f947-vl8z7"
Jul 01 17:03:39.000161 ip-10-0-137-244 kubenswrapper[2565]: I0701 17:03:39.000072    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8tzzl\" (UniqueName: \"kubernetes.io/projected/e51aab2e-c112-4e48-a37b-46db0ef5e33f-kube-api-access-8tzzl\") pod \"keda-metrics-apiserver-559f75f947-vl8z7\" (UID: \"e51aab2e-c112-4e48-a37b-46db0ef5e33f\") " pod="openshift-keda/keda-metrics-apiserver-559f75f947-vl8z7"
Jul 01 17:03:39.000161 ip-10-0-137-244 kubenswrapper[2565]: I0701 17:03:39.000124    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"certificates\" (UniqueName: \"kubernetes.io/projected/e51aab2e-c112-4e48-a37b-46db0ef5e33f-certificates\") pod \"keda-metrics-apiserver-559f75f947-vl8z7\" (UID: \"e51aab2e-c112-4e48-a37b-46db0ef5e33f\") " pod="openshift-keda/keda-metrics-apiserver-559f75f947-vl8z7"
Jul 01 17:03:39.100970 ip-10-0-137-244 kubenswrapper[2565]: I0701 17:03:39.100943    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8tzzl\" (UniqueName: \"kubernetes.io/projected/e51aab2e-c112-4e48-a37b-46db0ef5e33f-kube-api-access-8tzzl\") pod \"keda-metrics-apiserver-559f75f947-vl8z7\" (UID: \"e51aab2e-c112-4e48-a37b-46db0ef5e33f\") " pod="openshift-keda/keda-metrics-apiserver-559f75f947-vl8z7"
Jul 01 17:03:39.101103 ip-10-0-137-244 kubenswrapper[2565]: I0701 17:03:39.100997    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"certificates\" (UniqueName: \"kubernetes.io/projected/e51aab2e-c112-4e48-a37b-46db0ef5e33f-certificates\") pod \"keda-metrics-apiserver-559f75f947-vl8z7\" (UID: \"e51aab2e-c112-4e48-a37b-46db0ef5e33f\") " pod="openshift-keda/keda-metrics-apiserver-559f75f947-vl8z7"
Jul 01 17:03:39.101103 ip-10-0-137-244 kubenswrapper[2565]: I0701 17:03:39.101075    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"temp-vol\" (UniqueName: \"kubernetes.io/empty-dir/e51aab2e-c112-4e48-a37b-46db0ef5e33f-temp-vol\") pod \"keda-metrics-apiserver-559f75f947-vl8z7\" (UID: \"e51aab2e-c112-4e48-a37b-46db0ef5e33f\") " pod="openshift-keda/keda-metrics-apiserver-559f75f947-vl8z7"
Jul 01 17:03:39.101217 ip-10-0-137-244 kubenswrapper[2565]: E0701 17:03:39.101110    2565 projected.go:266] Couldn't get secret openshift-keda/keda-metrics-apiserver-certs: secret "keda-metrics-apiserver-certs" not found
Jul 01 17:03:39.101217 ip-10-0-137-244 kubenswrapper[2565]: E0701 17:03:39.101134    2565 projected.go:196] Error preparing data for projected volume certificates for pod openshift-keda/keda-metrics-apiserver-559f75f947-vl8z7: secret "keda-metrics-apiserver-certs" not found
Jul 01 17:03:39.101217 ip-10-0-137-244 kubenswrapper[2565]: E0701 17:03:39.101193    2565 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/e51aab2e-c112-4e48-a37b-46db0ef5e33f-certificates podName:e51aab2e-c112-4e48-a37b-46db0ef5e33f nodeName:}" failed. No retries permitted until 2026-07-01 17:03:39.601174135 +0000 UTC m=+290.069438238 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "certificates" (UniqueName: "kubernetes.io/projected/e51aab2e-c112-4e48-a37b-46db0ef5e33f-certificates") pod "keda-metrics-apiserver-559f75f947-vl8z7" (UID: "e51aab2e-c112-4e48-a37b-46db0ef5e33f") : secret "keda-metrics-apiserver-certs" not found
Jul 01 17:03:39.101447 ip-10-0-137-244 kubenswrapper[2565]: I0701 17:03:39.101429    2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"temp-vol\" (UniqueName: \"kubernetes.io/empty-dir/e51aab2e-c112-4e48-a37b-46db0ef5e33f-temp-vol\") pod \"keda-metrics-apiserver-559f75f947-vl8z7\" (UID: \"e51aab2e-c112-4e48-a37b-46db0ef5e33f\") " pod="openshift-keda/keda-metrics-apiserver-559f75f947-vl8z7"
Jul 01 17:03:39.110462 ip-10-0-137-244 kubenswrapper[2565]: I0701 17:03:39.110431    2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-8tzzl\" (UniqueName: \"kubernetes.io/projected/e51aab2e-c112-4e48-a37b-46db0ef5e33f-kube-api-access-8tzzl\") pod \"keda-metrics-apiserver-559f75f947-vl8z7\" (UID: \"e51aab2e-c112-4e48-a37b-46db0ef5e33f\") " pod="openshift-keda/keda-metrics-apiserver-559f75f947-vl8z7"
Jul 01 17:03:39.604770 ip-10-0-137-244 kubenswrapper[2565]: I0701 17:03:39.604742    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"certificates\" (UniqueName: \"kubernetes.io/projected/e51aab2e-c112-4e48-a37b-46db0ef5e33f-certificates\") pod \"keda-metrics-apiserver-559f75f947-vl8z7\" (UID: \"e51aab2e-c112-4e48-a37b-46db0ef5e33f\") " pod="openshift-keda/keda-metrics-apiserver-559f75f947-vl8z7"
Jul 01 17:03:39.606938 ip-10-0-137-244 kubenswrapper[2565]: I0701 17:03:39.606915    2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"certificates\" (UniqueName: \"kubernetes.io/projected/e51aab2e-c112-4e48-a37b-46db0ef5e33f-certificates\") pod \"keda-metrics-apiserver-559f75f947-vl8z7\" (UID: \"e51aab2e-c112-4e48-a37b-46db0ef5e33f\") " pod="openshift-keda/keda-metrics-apiserver-559f75f947-vl8z7"
Jul 01 17:03:39.818684 ip-10-0-137-244 kubenswrapper[2565]: I0701 17:03:39.818656    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-keda/keda-metrics-apiserver-559f75f947-vl8z7"
Jul 01 17:03:39.942160 ip-10-0-137-244 kubenswrapper[2565]: I0701 17:03:39.942129    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-keda/keda-metrics-apiserver-559f75f947-vl8z7"]
Jul 01 17:03:39.945479 ip-10-0-137-244 kubenswrapper[2565]: W0701 17:03:39.945453    2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode51aab2e_c112_4e48_a37b_46db0ef5e33f.slice/crio-344806fcbcdfe9043588a1daa9c89a8e11a0364f1a7c76d39e5029f19e3ce63f WatchSource:0}: Error finding container 344806fcbcdfe9043588a1daa9c89a8e11a0364f1a7c76d39e5029f19e3ce63f: Status 404 returned error can't find the container with id 344806fcbcdfe9043588a1daa9c89a8e11a0364f1a7c76d39e5029f19e3ce63f
Jul 01 17:03:40.356137 ip-10-0-137-244 kubenswrapper[2565]: I0701 17:03:40.356108    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-keda/keda-metrics-apiserver-559f75f947-vl8z7" event={"ID":"e51aab2e-c112-4e48-a37b-46db0ef5e33f","Type":"ContainerStarted","Data":"344806fcbcdfe9043588a1daa9c89a8e11a0364f1a7c76d39e5029f19e3ce63f"}
Jul 01 17:03:43.369521 ip-10-0-137-244 kubenswrapper[2565]: I0701 17:03:43.369488    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-keda/keda-metrics-apiserver-559f75f947-vl8z7" event={"ID":"e51aab2e-c112-4e48-a37b-46db0ef5e33f","Type":"ContainerStarted","Data":"6fbb5c6b1e9877c7a00fd87cdff2ecf488e8a9d24831f227f4f7abb35a62fe1e"}
Jul 01 17:03:43.369891 ip-10-0-137-244 kubenswrapper[2565]: I0701 17:03:43.369635    2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-keda/keda-metrics-apiserver-559f75f947-vl8z7"
Jul 01 17:03:43.390745 ip-10-0-137-244 kubenswrapper[2565]: I0701 17:03:43.390704    2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-keda/keda-metrics-apiserver-559f75f947-vl8z7" podStartSLOduration=2.114818432 podStartE2EDuration="5.390691293s" totalImagesPullingTime="3.275872861s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-01 17:03:38 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-01 17:03:43.389588686 +0000 UTC m=+293.857852818" watchObservedRunningTime="2026-07-01 17:03:43.390691293 +0000 UTC m=+293.858955417"
Jul 01 17:03:50.113230 ip-10-0-137-244 kubenswrapper[2565]: I0701 17:03:50.113198    2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-zjsw4_bb7ebc32-d0cd-4eb0-b998-3ce298f73809/ovn-acl-logging/0.log"
Jul 01 17:03:50.115742 ip-10-0-137-244 kubenswrapper[2565]: I0701 17:03:50.115714    2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-zjsw4_bb7ebc32-d0cd-4eb0-b998-3ce298f73809/ovn-acl-logging/0.log"
Jul 01 17:03:50.119367 ip-10-0-137-244 kubenswrapper[2565]: I0701 17:03:50.119344    2565 kubelet.go:1648] "Image garbage collection succeeded"
Jul 01 17:03:54.377086 ip-10-0-137-244 kubenswrapper[2565]: I0701 17:03:54.377055    2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-keda/keda-metrics-apiserver-559f75f947-vl8z7"
Jul 01 17:03:58.351094 ip-10-0-137-244 kubenswrapper[2565]: I0701 17:03:58.351067    2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-keda/custom-metrics-autoscaler-operator-75f7c5f8f-pttcg"
Jul 01 17:05:01.265730 ip-10-0-137-244 kubenswrapper[2565]: I0701 17:05:01.265693    2565 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kserve/kserve-controller-manager-84f4454459-65cjp"]
Jul 01 17:05:01.268149 ip-10-0-137-244 kubenswrapper[2565]: I0701 17:05:01.268128    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve/kserve-controller-manager-84f4454459-65cjp"
Jul 01 17:05:01.271239 ip-10-0-137-244 kubenswrapper[2565]: I0701 17:05:01.271221    2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kserve\"/\"kserve-webhook-server-cert\""
Jul 01 17:05:01.271500 ip-10-0-137-244 kubenswrapper[2565]: I0701 17:05:01.271485    2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kserve\"/\"kserve-controller-manager-dockercfg-7l2kq\""
Jul 01 17:05:01.272546 ip-10-0-137-244 kubenswrapper[2565]: I0701 17:05:01.272515    2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kserve\"/\"openshift-service-ca.crt\""
Jul 01 17:05:01.272641 ip-10-0-137-244 kubenswrapper[2565]: I0701 17:05:01.272554    2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kserve\"/\"kube-root-ca.crt\""
Jul 01 17:05:01.289689 ip-10-0-137-244 kubenswrapper[2565]: I0701 17:05:01.289671    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve/kserve-controller-manager-84f4454459-65cjp"]
Jul 01 17:05:01.317701 ip-10-0-137-244 kubenswrapper[2565]: I0701 17:05:01.317680    2565 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kserve/seaweedfs-64568bcd49-qwq8z"]
Jul 01 17:05:01.319720 ip-10-0-137-244 kubenswrapper[2565]: I0701 17:05:01.319702    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve/seaweedfs-64568bcd49-qwq8z"
Jul 01 17:05:01.322555 ip-10-0-137-244 kubenswrapper[2565]: I0701 17:05:01.322536    2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kserve\"/\"default-dockercfg-mwfdx\""
Jul 01 17:05:01.322665 ip-10-0-137-244 kubenswrapper[2565]: I0701 17:05:01.322544    2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kserve\"/\"mlpipeline-s3-artifact\""
Jul 01 17:05:01.338802 ip-10-0-137-244 kubenswrapper[2565]: I0701 17:05:01.338776    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve/seaweedfs-64568bcd49-qwq8z"]
Jul 01 17:05:01.338886 ip-10-0-137-244 kubenswrapper[2565]: I0701 17:05:01.338862    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wqhdw\" (UniqueName: \"kubernetes.io/projected/23cd82af-260d-4994-846d-761748de2cee-kube-api-access-wqhdw\") pod \"kserve-controller-manager-84f4454459-65cjp\" (UID: \"23cd82af-260d-4994-846d-761748de2cee\") " pod="kserve/kserve-controller-manager-84f4454459-65cjp"
Jul 01 17:05:01.338930 ip-10-0-137-244 kubenswrapper[2565]: I0701 17:05:01.338898    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"data\" (UniqueName: \"kubernetes.io/empty-dir/c64ad8c3-4f0f-4451-bfc1-dd91d59ecb24-data\") pod \"seaweedfs-64568bcd49-qwq8z\" (UID: \"c64ad8c3-4f0f-4451-bfc1-dd91d59ecb24\") " pod="kserve/seaweedfs-64568bcd49-qwq8z"
Jul 01 17:05:01.338967 ip-10-0-137-244 kubenswrapper[2565]: I0701 17:05:01.338928    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wtnbz\" (UniqueName: \"kubernetes.io/projected/c64ad8c3-4f0f-4451-bfc1-dd91d59ecb24-kube-api-access-wtnbz\") pod \"seaweedfs-64568bcd49-qwq8z\" (UID: \"c64ad8c3-4f0f-4451-bfc1-dd91d59ecb24\") " pod="kserve/seaweedfs-64568bcd49-qwq8z"
Jul 01 17:05:01.339001 ip-10-0-137-244 kubenswrapper[2565]: I0701 17:05:01.338966    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/23cd82af-260d-4994-846d-761748de2cee-cert\") pod \"kserve-controller-manager-84f4454459-65cjp\" (UID: \"23cd82af-260d-4994-846d-761748de2cee\") " pod="kserve/kserve-controller-manager-84f4454459-65cjp"
Jul 01 17:05:01.439569 ip-10-0-137-244 kubenswrapper[2565]: I0701 17:05:01.439546    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"data\" (UniqueName: \"kubernetes.io/empty-dir/c64ad8c3-4f0f-4451-bfc1-dd91d59ecb24-data\") pod \"seaweedfs-64568bcd49-qwq8z\" (UID: \"c64ad8c3-4f0f-4451-bfc1-dd91d59ecb24\") " pod="kserve/seaweedfs-64568bcd49-qwq8z"
Jul 01 17:05:01.439672 ip-10-0-137-244 kubenswrapper[2565]: I0701 17:05:01.439576    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wtnbz\" (UniqueName: \"kubernetes.io/projected/c64ad8c3-4f0f-4451-bfc1-dd91d59ecb24-kube-api-access-wtnbz\") pod \"seaweedfs-64568bcd49-qwq8z\" (UID: \"c64ad8c3-4f0f-4451-bfc1-dd91d59ecb24\") " pod="kserve/seaweedfs-64568bcd49-qwq8z"
Jul 01 17:05:01.439672 ip-10-0-137-244 kubenswrapper[2565]: I0701 17:05:01.439597    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/23cd82af-260d-4994-846d-761748de2cee-cert\") pod \"kserve-controller-manager-84f4454459-65cjp\" (UID: \"23cd82af-260d-4994-846d-761748de2cee\") " pod="kserve/kserve-controller-manager-84f4454459-65cjp"
Jul 01 17:05:01.439672 ip-10-0-137-244 kubenswrapper[2565]: I0701 17:05:01.439629    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wqhdw\" (UniqueName: \"kubernetes.io/projected/23cd82af-260d-4994-846d-761748de2cee-kube-api-access-wqhdw\") pod \"kserve-controller-manager-84f4454459-65cjp\" (UID: \"23cd82af-260d-4994-846d-761748de2cee\") " pod="kserve/kserve-controller-manager-84f4454459-65cjp"
Jul 01 17:05:01.439891 ip-10-0-137-244 kubenswrapper[2565]: I0701 17:05:01.439874    2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"data\" (UniqueName: \"kubernetes.io/empty-dir/c64ad8c3-4f0f-4451-bfc1-dd91d59ecb24-data\") pod \"seaweedfs-64568bcd49-qwq8z\" (UID: \"c64ad8c3-4f0f-4451-bfc1-dd91d59ecb24\") " pod="kserve/seaweedfs-64568bcd49-qwq8z"
Jul 01 17:05:01.442005 ip-10-0-137-244 kubenswrapper[2565]: I0701 17:05:01.441977    2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/23cd82af-260d-4994-846d-761748de2cee-cert\") pod \"kserve-controller-manager-84f4454459-65cjp\" (UID: \"23cd82af-260d-4994-846d-761748de2cee\") " pod="kserve/kserve-controller-manager-84f4454459-65cjp"
Jul 01 17:05:01.452748 ip-10-0-137-244 kubenswrapper[2565]: I0701 17:05:01.452730    2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-wqhdw\" (UniqueName: \"kubernetes.io/projected/23cd82af-260d-4994-846d-761748de2cee-kube-api-access-wqhdw\") pod \"kserve-controller-manager-84f4454459-65cjp\" (UID: \"23cd82af-260d-4994-846d-761748de2cee\") " pod="kserve/kserve-controller-manager-84f4454459-65cjp"
Jul 01 17:05:01.452991 ip-10-0-137-244 kubenswrapper[2565]: I0701 17:05:01.452971    2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-wtnbz\" (UniqueName: \"kubernetes.io/projected/c64ad8c3-4f0f-4451-bfc1-dd91d59ecb24-kube-api-access-wtnbz\") pod \"seaweedfs-64568bcd49-qwq8z\" (UID: \"c64ad8c3-4f0f-4451-bfc1-dd91d59ecb24\") " pod="kserve/seaweedfs-64568bcd49-qwq8z"
Jul 01 17:05:01.578219 ip-10-0-137-244 kubenswrapper[2565]: I0701 17:05:01.578170    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve/kserve-controller-manager-84f4454459-65cjp"
Jul 01 17:05:01.627846 ip-10-0-137-244 kubenswrapper[2565]: I0701 17:05:01.627822    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve/seaweedfs-64568bcd49-qwq8z"
Jul 01 17:05:01.737295 ip-10-0-137-244 kubenswrapper[2565]: I0701 17:05:01.737261    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve/kserve-controller-manager-84f4454459-65cjp"]
Jul 01 17:05:01.739809 ip-10-0-137-244 kubenswrapper[2565]: W0701 17:05:01.739785    2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod23cd82af_260d_4994_846d_761748de2cee.slice/crio-fd00ea2ada97fb0dd47e01d0f024b81cc4bbab8b83a7445bf564a8cd90fd1c7d WatchSource:0}: Error finding container fd00ea2ada97fb0dd47e01d0f024b81cc4bbab8b83a7445bf564a8cd90fd1c7d: Status 404 returned error can't find the container with id fd00ea2ada97fb0dd47e01d0f024b81cc4bbab8b83a7445bf564a8cd90fd1c7d
Jul 01 17:05:01.741215 ip-10-0-137-244 kubenswrapper[2565]: I0701 17:05:01.741197    2565 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Jul 01 17:05:01.768246 ip-10-0-137-244 kubenswrapper[2565]: I0701 17:05:01.768223    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve/seaweedfs-64568bcd49-qwq8z"]
Jul 01 17:05:01.770738 ip-10-0-137-244 kubenswrapper[2565]: W0701 17:05:01.770720    2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc64ad8c3_4f0f_4451_bfc1_dd91d59ecb24.slice/crio-3c134e0e0f030714ac53c49d0da5b9c0eaeb4f9c8f6022059479de1cb7af45b2 WatchSource:0}: Error finding container 3c134e0e0f030714ac53c49d0da5b9c0eaeb4f9c8f6022059479de1cb7af45b2: Status 404 returned error can't find the container with id 3c134e0e0f030714ac53c49d0da5b9c0eaeb4f9c8f6022059479de1cb7af45b2
Jul 01 17:05:02.652279 ip-10-0-137-244 kubenswrapper[2565]: I0701 17:05:02.652244    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve/kserve-controller-manager-84f4454459-65cjp" event={"ID":"23cd82af-260d-4994-846d-761748de2cee","Type":"ContainerStarted","Data":"fd00ea2ada97fb0dd47e01d0f024b81cc4bbab8b83a7445bf564a8cd90fd1c7d"}
Jul 01 17:05:02.653202 ip-10-0-137-244 kubenswrapper[2565]: I0701 17:05:02.653172    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve/seaweedfs-64568bcd49-qwq8z" event={"ID":"c64ad8c3-4f0f-4451-bfc1-dd91d59ecb24","Type":"ContainerStarted","Data":"3c134e0e0f030714ac53c49d0da5b9c0eaeb4f9c8f6022059479de1cb7af45b2"}
Jul 01 17:05:07.672901 ip-10-0-137-244 kubenswrapper[2565]: I0701 17:05:07.672872    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve/seaweedfs-64568bcd49-qwq8z" event={"ID":"c64ad8c3-4f0f-4451-bfc1-dd91d59ecb24","Type":"ContainerStarted","Data":"3ba5ebd311aed3024845161f7a531b433db13bf00c67918a5616697754ef6f89"}
Jul 01 17:05:07.673342 ip-10-0-137-244 kubenswrapper[2565]: I0701 17:05:07.672972    2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve/seaweedfs-64568bcd49-qwq8z"
Jul 01 17:05:07.674309 ip-10-0-137-244 kubenswrapper[2565]: I0701 17:05:07.674286    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve/kserve-controller-manager-84f4454459-65cjp" event={"ID":"23cd82af-260d-4994-846d-761748de2cee","Type":"ContainerStarted","Data":"e70d68260ce56d24449f85c783d1c40b300dc07f8ee73f56114a4539a39f0340"}
Jul 01 17:05:07.674420 ip-10-0-137-244 kubenswrapper[2565]: I0701 17:05:07.674404    2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve/kserve-controller-manager-84f4454459-65cjp"
Jul 01 17:05:07.696957 ip-10-0-137-244 kubenswrapper[2565]: I0701 17:05:07.696917    2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kserve/seaweedfs-64568bcd49-qwq8z" podStartSLOduration=1.121629352 podStartE2EDuration="6.696903778s" totalImagesPullingTime="5.575274426s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-01 17:05:01 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-01 17:05:07.696434545 +0000 UTC m=+378.164698680" watchObservedRunningTime="2026-07-01 17:05:07.696903778 +0000 UTC m=+378.165167880"
Jul 01 17:05:07.716366 ip-10-0-137-244 kubenswrapper[2565]: I0701 17:05:07.716328    2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kserve/kserve-controller-manager-84f4454459-65cjp" podStartSLOduration=1.199414583 podStartE2EDuration="6.716315651s" totalImagesPullingTime="5.516901068s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-01 17:05:01 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-01 17:05:07.714728852 +0000 UTC m=+378.182992988" watchObservedRunningTime="2026-07-01 17:05:07.716315651 +0000 UTC m=+378.184579774"
Jul 01 17:05:13.681201 ip-10-0-137-244 kubenswrapper[2565]: I0701 17:05:13.681168    2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kserve/seaweedfs-64568bcd49-qwq8z"
Jul 01 17:05:38.683695 ip-10-0-137-244 kubenswrapper[2565]: I0701 17:05:38.683668    2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kserve/kserve-controller-manager-84f4454459-65cjp"
Jul 01 17:05:41.739088 ip-10-0-137-244 kubenswrapper[2565]: I0701 17:05:41.739042    2565 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kserve/kserve-controller-manager-84f4454459-65cjp"]
Jul 01 17:05:41.739606 ip-10-0-137-244 kubenswrapper[2565]: I0701 17:05:41.739285    2565 kuberuntime_container.go:872] "Killing container with a grace period" pod="kserve/kserve-controller-manager-84f4454459-65cjp" podUID="23cd82af-260d-4994-846d-761748de2cee" containerName="manager" containerID="cri-o://e70d68260ce56d24449f85c783d1c40b300dc07f8ee73f56114a4539a39f0340" gracePeriod=10
Jul 01 17:05:41.776144 ip-10-0-137-244 kubenswrapper[2565]: I0701 17:05:41.776116    2565 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kserve/kserve-controller-manager-84f4454459-47k94"]
Jul 01 17:05:41.778414 ip-10-0-137-244 kubenswrapper[2565]: I0701 17:05:41.778398    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve/kserve-controller-manager-84f4454459-47k94"
Jul 01 17:05:41.797169 ip-10-0-137-244 kubenswrapper[2565]: I0701 17:05:41.797141    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve/kserve-controller-manager-84f4454459-47k94"]
Jul 01 17:05:41.806501 ip-10-0-137-244 kubenswrapper[2565]: I0701 17:05:41.806479    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dcwjt\" (UniqueName: \"kubernetes.io/projected/b048d835-8a05-4aa0-9975-1a4a3e2d6e4c-kube-api-access-dcwjt\") pod \"kserve-controller-manager-84f4454459-47k94\" (UID: \"b048d835-8a05-4aa0-9975-1a4a3e2d6e4c\") " pod="kserve/kserve-controller-manager-84f4454459-47k94"
Jul 01 17:05:41.806601 ip-10-0-137-244 kubenswrapper[2565]: I0701 17:05:41.806531    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/b048d835-8a05-4aa0-9975-1a4a3e2d6e4c-cert\") pod \"kserve-controller-manager-84f4454459-47k94\" (UID: \"b048d835-8a05-4aa0-9975-1a4a3e2d6e4c\") " pod="kserve/kserve-controller-manager-84f4454459-47k94"
Jul 01 17:05:41.907444 ip-10-0-137-244 kubenswrapper[2565]: I0701 17:05:41.907420    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/b048d835-8a05-4aa0-9975-1a4a3e2d6e4c-cert\") pod \"kserve-controller-manager-84f4454459-47k94\" (UID: \"b048d835-8a05-4aa0-9975-1a4a3e2d6e4c\") " pod="kserve/kserve-controller-manager-84f4454459-47k94"
Jul 01 17:05:41.907531 ip-10-0-137-244 kubenswrapper[2565]: I0701 17:05:41.907466    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dcwjt\" (UniqueName: \"kubernetes.io/projected/b048d835-8a05-4aa0-9975-1a4a3e2d6e4c-kube-api-access-dcwjt\") pod \"kserve-controller-manager-84f4454459-47k94\" (UID: \"b048d835-8a05-4aa0-9975-1a4a3e2d6e4c\") " pod="kserve/kserve-controller-manager-84f4454459-47k94"
Jul 01 17:05:41.909640 ip-10-0-137-244 kubenswrapper[2565]: I0701 17:05:41.909617    2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/b048d835-8a05-4aa0-9975-1a4a3e2d6e4c-cert\") pod \"kserve-controller-manager-84f4454459-47k94\" (UID: \"b048d835-8a05-4aa0-9975-1a4a3e2d6e4c\") " pod="kserve/kserve-controller-manager-84f4454459-47k94"
Jul 01 17:05:41.916059 ip-10-0-137-244 kubenswrapper[2565]: I0701 17:05:41.916040    2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-dcwjt\" (UniqueName: \"kubernetes.io/projected/b048d835-8a05-4aa0-9975-1a4a3e2d6e4c-kube-api-access-dcwjt\") pod \"kserve-controller-manager-84f4454459-47k94\" (UID: \"b048d835-8a05-4aa0-9975-1a4a3e2d6e4c\") " pod="kserve/kserve-controller-manager-84f4454459-47k94"
Jul 01 17:05:41.975132 ip-10-0-137-244 kubenswrapper[2565]: I0701 17:05:41.975108    2565 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve/kserve-controller-manager-84f4454459-65cjp"
Jul 01 17:05:42.008045 ip-10-0-137-244 kubenswrapper[2565]: I0701 17:05:42.007973    2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-wqhdw\" (UniqueName: \"kubernetes.io/projected/23cd82af-260d-4994-846d-761748de2cee-kube-api-access-wqhdw\") pod \"23cd82af-260d-4994-846d-761748de2cee\" (UID: \"23cd82af-260d-4994-846d-761748de2cee\") "
Jul 01 17:05:42.008157 ip-10-0-137-244 kubenswrapper[2565]: I0701 17:05:42.008044    2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/23cd82af-260d-4994-846d-761748de2cee-cert\") pod \"23cd82af-260d-4994-846d-761748de2cee\" (UID: \"23cd82af-260d-4994-846d-761748de2cee\") "
Jul 01 17:05:42.010032 ip-10-0-137-244 kubenswrapper[2565]: I0701 17:05:42.009996    2565 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/23cd82af-260d-4994-846d-761748de2cee-kube-api-access-wqhdw" (OuterVolumeSpecName: "kube-api-access-wqhdw") pod "23cd82af-260d-4994-846d-761748de2cee" (UID: "23cd82af-260d-4994-846d-761748de2cee"). InnerVolumeSpecName "kube-api-access-wqhdw". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 01 17:05:42.010114 ip-10-0-137-244 kubenswrapper[2565]: I0701 17:05:42.010043    2565 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/23cd82af-260d-4994-846d-761748de2cee-cert" (OuterVolumeSpecName: "cert") pod "23cd82af-260d-4994-846d-761748de2cee" (UID: "23cd82af-260d-4994-846d-761748de2cee"). InnerVolumeSpecName "cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 01 17:05:42.108616 ip-10-0-137-244 kubenswrapper[2565]: I0701 17:05:42.108593    2565 reconciler_common.go:299] "Volume detached for volume \"cert\" (UniqueName: \"kubernetes.io/secret/23cd82af-260d-4994-846d-761748de2cee-cert\") on node \"ip-10-0-137-244.ec2.internal\" DevicePath \"\""
Jul 01 17:05:42.108616 ip-10-0-137-244 kubenswrapper[2565]: I0701 17:05:42.108616    2565 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-wqhdw\" (UniqueName: \"kubernetes.io/projected/23cd82af-260d-4994-846d-761748de2cee-kube-api-access-wqhdw\") on node \"ip-10-0-137-244.ec2.internal\" DevicePath \"\""
Jul 01 17:05:42.121614 ip-10-0-137-244 kubenswrapper[2565]: I0701 17:05:42.121579    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve/kserve-controller-manager-84f4454459-47k94"
Jul 01 17:05:42.240603 ip-10-0-137-244 kubenswrapper[2565]: I0701 17:05:42.240579    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve/kserve-controller-manager-84f4454459-47k94"]
Jul 01 17:05:42.242599 ip-10-0-137-244 kubenswrapper[2565]: W0701 17:05:42.242574    2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb048d835_8a05_4aa0_9975_1a4a3e2d6e4c.slice/crio-88fad42fb5867eefd259927f6cb3c5308c4a17d14277bd0d37e75bc7b6f448df WatchSource:0}: Error finding container 88fad42fb5867eefd259927f6cb3c5308c4a17d14277bd0d37e75bc7b6f448df: Status 404 returned error can't find the container with id 88fad42fb5867eefd259927f6cb3c5308c4a17d14277bd0d37e75bc7b6f448df
Jul 01 17:05:42.810087 ip-10-0-137-244 kubenswrapper[2565]: I0701 17:05:42.810052    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve/kserve-controller-manager-84f4454459-47k94" event={"ID":"b048d835-8a05-4aa0-9975-1a4a3e2d6e4c","Type":"ContainerStarted","Data":"88fad42fb5867eefd259927f6cb3c5308c4a17d14277bd0d37e75bc7b6f448df"}
Jul 01 17:05:42.811293 ip-10-0-137-244 kubenswrapper[2565]: I0701 17:05:42.811268    2565 generic.go:356] "Generic (PLEG): container finished" podID="23cd82af-260d-4994-846d-761748de2cee" containerID="e70d68260ce56d24449f85c783d1c40b300dc07f8ee73f56114a4539a39f0340" exitCode=0
Jul 01 17:05:42.811413 ip-10-0-137-244 kubenswrapper[2565]: I0701 17:05:42.811318    2565 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve/kserve-controller-manager-84f4454459-65cjp"
Jul 01 17:05:42.811413 ip-10-0-137-244 kubenswrapper[2565]: I0701 17:05:42.811318    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve/kserve-controller-manager-84f4454459-65cjp" event={"ID":"23cd82af-260d-4994-846d-761748de2cee","Type":"ContainerDied","Data":"e70d68260ce56d24449f85c783d1c40b300dc07f8ee73f56114a4539a39f0340"}
Jul 01 17:05:42.811526 ip-10-0-137-244 kubenswrapper[2565]: I0701 17:05:42.811440    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve/kserve-controller-manager-84f4454459-65cjp" event={"ID":"23cd82af-260d-4994-846d-761748de2cee","Type":"ContainerDied","Data":"fd00ea2ada97fb0dd47e01d0f024b81cc4bbab8b83a7445bf564a8cd90fd1c7d"}
Jul 01 17:05:42.811526 ip-10-0-137-244 kubenswrapper[2565]: I0701 17:05:42.811472    2565 scope.go:117] "RemoveContainer" containerID="e70d68260ce56d24449f85c783d1c40b300dc07f8ee73f56114a4539a39f0340"
Jul 01 17:05:42.822961 ip-10-0-137-244 kubenswrapper[2565]: I0701 17:05:42.822945    2565 scope.go:117] "RemoveContainer" containerID="e70d68260ce56d24449f85c783d1c40b300dc07f8ee73f56114a4539a39f0340"
Jul 01 17:05:42.823220 ip-10-0-137-244 kubenswrapper[2565]: E0701 17:05:42.823198    2565 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"e70d68260ce56d24449f85c783d1c40b300dc07f8ee73f56114a4539a39f0340\": container with ID starting with e70d68260ce56d24449f85c783d1c40b300dc07f8ee73f56114a4539a39f0340 not found: ID does not exist" containerID="e70d68260ce56d24449f85c783d1c40b300dc07f8ee73f56114a4539a39f0340"
Jul 01 17:05:42.823310 ip-10-0-137-244 kubenswrapper[2565]: I0701 17:05:42.823226    2565 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e70d68260ce56d24449f85c783d1c40b300dc07f8ee73f56114a4539a39f0340"} err="failed to get container status \"e70d68260ce56d24449f85c783d1c40b300dc07f8ee73f56114a4539a39f0340\": rpc error: code = NotFound desc = could not find container \"e70d68260ce56d24449f85c783d1c40b300dc07f8ee73f56114a4539a39f0340\": container with ID starting with e70d68260ce56d24449f85c783d1c40b300dc07f8ee73f56114a4539a39f0340 not found: ID does not exist"
Jul 01 17:05:42.845106 ip-10-0-137-244 kubenswrapper[2565]: I0701 17:05:42.845086    2565 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kserve/kserve-controller-manager-84f4454459-65cjp"]
Jul 01 17:05:42.850777 ip-10-0-137-244 kubenswrapper[2565]: I0701 17:05:42.850753    2565 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kserve/kserve-controller-manager-84f4454459-65cjp"]
Jul 01 17:05:43.820291 ip-10-0-137-244 kubenswrapper[2565]: I0701 17:05:43.820249    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve/kserve-controller-manager-84f4454459-47k94" event={"ID":"b048d835-8a05-4aa0-9975-1a4a3e2d6e4c","Type":"ContainerStarted","Data":"47f920bf678329eec7e125d053f10248fad7492f1cddf97cd93d927b7b094c9a"}
Jul 01 17:05:43.820750 ip-10-0-137-244 kubenswrapper[2565]: I0701 17:05:43.820405    2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve/kserve-controller-manager-84f4454459-47k94"
Jul 01 17:05:43.855609 ip-10-0-137-244 kubenswrapper[2565]: I0701 17:05:43.855550    2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kserve/kserve-controller-manager-84f4454459-47k94" podStartSLOduration=2.297358954 podStartE2EDuration="2.855535729s" totalImagesPullingTime="558.176775ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-01 17:05:41 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-01 17:05:43.854838388 +0000 UTC m=+414.323102514" watchObservedRunningTime="2026-07-01 17:05:43.855535729 +0000 UTC m=+414.323799852"
Jul 01 17:05:44.248486 ip-10-0-137-244 kubenswrapper[2565]: I0701 17:05:44.248455    2565 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="23cd82af-260d-4994-846d-761748de2cee" path="/var/lib/kubelet/pods/23cd82af-260d-4994-846d-761748de2cee/volumes"
Jul 01 17:06:14.828743 ip-10-0-137-244 kubenswrapper[2565]: I0701 17:06:14.828653    2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kserve/kserve-controller-manager-84f4454459-47k94"
Jul 01 17:06:15.900711 ip-10-0-137-244 kubenswrapper[2565]: I0701 17:06:15.900677    2565 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kserve/model-serving-api-fd65d7d6-p4xfs"]
Jul 01 17:06:15.901190 ip-10-0-137-244 kubenswrapper[2565]: I0701 17:06:15.901101    2565 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="23cd82af-260d-4994-846d-761748de2cee" containerName="manager"
Jul 01 17:06:15.901190 ip-10-0-137-244 kubenswrapper[2565]: I0701 17:06:15.901120    2565 state_mem.go:107] "Deleted CPUSet assignment" podUID="23cd82af-260d-4994-846d-761748de2cee" containerName="manager"
Jul 01 17:06:15.901310 ip-10-0-137-244 kubenswrapper[2565]: I0701 17:06:15.901191    2565 memory_manager.go:368] "RemoveStaleState removing state" podUID="23cd82af-260d-4994-846d-761748de2cee" containerName="manager"
Jul 01 17:06:15.903701 ip-10-0-137-244 kubenswrapper[2565]: I0701 17:06:15.903680    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve/model-serving-api-fd65d7d6-p4xfs"
Jul 01 17:06:15.906520 ip-10-0-137-244 kubenswrapper[2565]: I0701 17:06:15.906500    2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kserve\"/\"model-serving-api-tls\""
Jul 01 17:06:15.906643 ip-10-0-137-244 kubenswrapper[2565]: I0701 17:06:15.906545    2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kserve\"/\"model-serving-api-dockercfg-xsc96\""
Jul 01 17:06:15.921891 ip-10-0-137-244 kubenswrapper[2565]: I0701 17:06:15.921874    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve/model-serving-api-fd65d7d6-p4xfs"]
Jul 01 17:06:16.043183 ip-10-0-137-244 kubenswrapper[2565]: I0701 17:06:16.043155    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/3bbfca26-6773-445d-b94a-e1564ffc6cac-tls-certs\") pod \"model-serving-api-fd65d7d6-p4xfs\" (UID: \"3bbfca26-6773-445d-b94a-e1564ffc6cac\") " pod="kserve/model-serving-api-fd65d7d6-p4xfs"
Jul 01 17:06:16.043299 ip-10-0-137-244 kubenswrapper[2565]: I0701 17:06:16.043221    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jnfq8\" (UniqueName: \"kubernetes.io/projected/3bbfca26-6773-445d-b94a-e1564ffc6cac-kube-api-access-jnfq8\") pod \"model-serving-api-fd65d7d6-p4xfs\" (UID: \"3bbfca26-6773-445d-b94a-e1564ffc6cac\") " pod="kserve/model-serving-api-fd65d7d6-p4xfs"
Jul 01 17:06:16.144443 ip-10-0-137-244 kubenswrapper[2565]: I0701 17:06:16.144417    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jnfq8\" (UniqueName: \"kubernetes.io/projected/3bbfca26-6773-445d-b94a-e1564ffc6cac-kube-api-access-jnfq8\") pod \"model-serving-api-fd65d7d6-p4xfs\" (UID: \"3bbfca26-6773-445d-b94a-e1564ffc6cac\") " pod="kserve/model-serving-api-fd65d7d6-p4xfs"
Jul 01 17:06:16.144544 ip-10-0-137-244 kubenswrapper[2565]: I0701 17:06:16.144453    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/3bbfca26-6773-445d-b94a-e1564ffc6cac-tls-certs\") pod \"model-serving-api-fd65d7d6-p4xfs\" (UID: \"3bbfca26-6773-445d-b94a-e1564ffc6cac\") " pod="kserve/model-serving-api-fd65d7d6-p4xfs"
Jul 01 17:06:16.146824 ip-10-0-137-244 kubenswrapper[2565]: I0701 17:06:16.146804    2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/3bbfca26-6773-445d-b94a-e1564ffc6cac-tls-certs\") pod \"model-serving-api-fd65d7d6-p4xfs\" (UID: \"3bbfca26-6773-445d-b94a-e1564ffc6cac\") " pod="kserve/model-serving-api-fd65d7d6-p4xfs"
Jul 01 17:06:16.155353 ip-10-0-137-244 kubenswrapper[2565]: I0701 17:06:16.155301    2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-jnfq8\" (UniqueName: \"kubernetes.io/projected/3bbfca26-6773-445d-b94a-e1564ffc6cac-kube-api-access-jnfq8\") pod \"model-serving-api-fd65d7d6-p4xfs\" (UID: \"3bbfca26-6773-445d-b94a-e1564ffc6cac\") " pod="kserve/model-serving-api-fd65d7d6-p4xfs"
Jul 01 17:06:16.213668 ip-10-0-137-244 kubenswrapper[2565]: I0701 17:06:16.213646    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve/model-serving-api-fd65d7d6-p4xfs"
Jul 01 17:06:16.345718 ip-10-0-137-244 kubenswrapper[2565]: I0701 17:06:16.345686    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve/model-serving-api-fd65d7d6-p4xfs"]
Jul 01 17:06:16.349115 ip-10-0-137-244 kubenswrapper[2565]: W0701 17:06:16.349077    2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3bbfca26_6773_445d_b94a_e1564ffc6cac.slice/crio-bea2b9ddebb0b37abefd0783a2110b71067336e509afe94370d5127a55af0214 WatchSource:0}: Error finding container bea2b9ddebb0b37abefd0783a2110b71067336e509afe94370d5127a55af0214: Status 404 returned error can't find the container with id bea2b9ddebb0b37abefd0783a2110b71067336e509afe94370d5127a55af0214
Jul 01 17:06:16.947999 ip-10-0-137-244 kubenswrapper[2565]: I0701 17:06:16.947965    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve/model-serving-api-fd65d7d6-p4xfs" event={"ID":"3bbfca26-6773-445d-b94a-e1564ffc6cac","Type":"ContainerStarted","Data":"bea2b9ddebb0b37abefd0783a2110b71067336e509afe94370d5127a55af0214"}
Jul 01 17:06:19.963629 ip-10-0-137-244 kubenswrapper[2565]: I0701 17:06:19.963595    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve/model-serving-api-fd65d7d6-p4xfs" event={"ID":"3bbfca26-6773-445d-b94a-e1564ffc6cac","Type":"ContainerStarted","Data":"431d51cbfcc661d026fcae43898471f3ae89b93285ea76d12a6938d0d6629fe9"}
Jul 01 17:06:19.964132 ip-10-0-137-244 kubenswrapper[2565]: I0701 17:06:19.963701    2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve/model-serving-api-fd65d7d6-p4xfs"
Jul 01 17:06:19.985758 ip-10-0-137-244 kubenswrapper[2565]: I0701 17:06:19.985710    2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kserve/model-serving-api-fd65d7d6-p4xfs" podStartSLOduration=2.279619981 podStartE2EDuration="4.985695602s" totalImagesPullingTime="2.706075621s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-01 17:06:15 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-01 17:06:19.983350278 +0000 UTC m=+450.451614420" watchObservedRunningTime="2026-07-01 17:06:19.985695602 +0000 UTC m=+450.453959798"
Jul 01 17:06:25.700131 ip-10-0-137-244 kubenswrapper[2565]: I0701 17:06:25.700089    2565 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-5b74c58dcb-qtxpp"]
Jul 01 17:06:30.972845 ip-10-0-137-244 kubenswrapper[2565]: I0701 17:06:30.972801    2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kserve/model-serving-api-fd65d7d6-p4xfs"
Jul 01 17:06:50.723778 ip-10-0-137-244 kubenswrapper[2565]: I0701 17:06:50.723702    2565 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-console/console-5b74c58dcb-qtxpp" podUID="75034c52-ff8a-4e43-917f-1061c4b9175a" containerName="console" containerID="cri-o://413411056cd1dcd8dffa34e4d5109c298f43c944c6d2f608063eeb6a289718f8" gracePeriod=15
Jul 01 17:06:50.961721 ip-10-0-137-244 kubenswrapper[2565]: I0701 17:06:50.961701    2565 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-5b74c58dcb-qtxpp"
Jul 01 17:06:50.983165 ip-10-0-137-244 kubenswrapper[2565]: I0701 17:06:50.983102    2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/75034c52-ff8a-4e43-917f-1061c4b9175a-oauth-serving-cert\") pod \"75034c52-ff8a-4e43-917f-1061c4b9175a\" (UID: \"75034c52-ff8a-4e43-917f-1061c4b9175a\") "
Jul 01 17:06:50.983165 ip-10-0-137-244 kubenswrapper[2565]: I0701 17:06:50.983154    2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-s8xd4\" (UniqueName: \"kubernetes.io/projected/75034c52-ff8a-4e43-917f-1061c4b9175a-kube-api-access-s8xd4\") pod \"75034c52-ff8a-4e43-917f-1061c4b9175a\" (UID: \"75034c52-ff8a-4e43-917f-1061c4b9175a\") "
Jul 01 17:06:50.983315 ip-10-0-137-244 kubenswrapper[2565]: I0701 17:06:50.983203    2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/75034c52-ff8a-4e43-917f-1061c4b9175a-trusted-ca-bundle\") pod \"75034c52-ff8a-4e43-917f-1061c4b9175a\" (UID: \"75034c52-ff8a-4e43-917f-1061c4b9175a\") "
Jul 01 17:06:50.983315 ip-10-0-137-244 kubenswrapper[2565]: I0701 17:06:50.983228    2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/75034c52-ff8a-4e43-917f-1061c4b9175a-console-oauth-config\") pod \"75034c52-ff8a-4e43-917f-1061c4b9175a\" (UID: \"75034c52-ff8a-4e43-917f-1061c4b9175a\") "
Jul 01 17:06:50.983315 ip-10-0-137-244 kubenswrapper[2565]: I0701 17:06:50.983267    2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/75034c52-ff8a-4e43-917f-1061c4b9175a-console-config\") pod \"75034c52-ff8a-4e43-917f-1061c4b9175a\" (UID: \"75034c52-ff8a-4e43-917f-1061c4b9175a\") "
Jul 01 17:06:50.983315 ip-10-0-137-244 kubenswrapper[2565]: I0701 17:06:50.983291    2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/75034c52-ff8a-4e43-917f-1061c4b9175a-service-ca\") pod \"75034c52-ff8a-4e43-917f-1061c4b9175a\" (UID: \"75034c52-ff8a-4e43-917f-1061c4b9175a\") "
Jul 01 17:06:50.983499 ip-10-0-137-244 kubenswrapper[2565]: I0701 17:06:50.983318    2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/75034c52-ff8a-4e43-917f-1061c4b9175a-console-serving-cert\") pod \"75034c52-ff8a-4e43-917f-1061c4b9175a\" (UID: \"75034c52-ff8a-4e43-917f-1061c4b9175a\") "
Jul 01 17:06:50.983627 ip-10-0-137-244 kubenswrapper[2565]: I0701 17:06:50.983597    2565 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/75034c52-ff8a-4e43-917f-1061c4b9175a-oauth-serving-cert" (OuterVolumeSpecName: "oauth-serving-cert") pod "75034c52-ff8a-4e43-917f-1061c4b9175a" (UID: "75034c52-ff8a-4e43-917f-1061c4b9175a"). InnerVolumeSpecName "oauth-serving-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 01 17:06:50.983701 ip-10-0-137-244 kubenswrapper[2565]: I0701 17:06:50.983653    2565 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/75034c52-ff8a-4e43-917f-1061c4b9175a-console-config" (OuterVolumeSpecName: "console-config") pod "75034c52-ff8a-4e43-917f-1061c4b9175a" (UID: "75034c52-ff8a-4e43-917f-1061c4b9175a"). InnerVolumeSpecName "console-config". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 01 17:06:50.983701 ip-10-0-137-244 kubenswrapper[2565]: I0701 17:06:50.983661    2565 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/75034c52-ff8a-4e43-917f-1061c4b9175a-service-ca" (OuterVolumeSpecName: "service-ca") pod "75034c52-ff8a-4e43-917f-1061c4b9175a" (UID: "75034c52-ff8a-4e43-917f-1061c4b9175a"). InnerVolumeSpecName "service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 01 17:06:50.983853 ip-10-0-137-244 kubenswrapper[2565]: I0701 17:06:50.983834    2565 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/75034c52-ff8a-4e43-917f-1061c4b9175a-trusted-ca-bundle" (OuterVolumeSpecName: "trusted-ca-bundle") pod "75034c52-ff8a-4e43-917f-1061c4b9175a" (UID: "75034c52-ff8a-4e43-917f-1061c4b9175a"). InnerVolumeSpecName "trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 01 17:06:50.985615 ip-10-0-137-244 kubenswrapper[2565]: I0701 17:06:50.985585    2565 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/75034c52-ff8a-4e43-917f-1061c4b9175a-kube-api-access-s8xd4" (OuterVolumeSpecName: "kube-api-access-s8xd4") pod "75034c52-ff8a-4e43-917f-1061c4b9175a" (UID: "75034c52-ff8a-4e43-917f-1061c4b9175a"). InnerVolumeSpecName "kube-api-access-s8xd4". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 01 17:06:50.985700 ip-10-0-137-244 kubenswrapper[2565]: I0701 17:06:50.985630    2565 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/75034c52-ff8a-4e43-917f-1061c4b9175a-console-serving-cert" (OuterVolumeSpecName: "console-serving-cert") pod "75034c52-ff8a-4e43-917f-1061c4b9175a" (UID: "75034c52-ff8a-4e43-917f-1061c4b9175a"). InnerVolumeSpecName "console-serving-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 01 17:06:50.985700 ip-10-0-137-244 kubenswrapper[2565]: I0701 17:06:50.985670    2565 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/75034c52-ff8a-4e43-917f-1061c4b9175a-console-oauth-config" (OuterVolumeSpecName: "console-oauth-config") pod "75034c52-ff8a-4e43-917f-1061c4b9175a" (UID: "75034c52-ff8a-4e43-917f-1061c4b9175a"). InnerVolumeSpecName "console-oauth-config". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 01 17:06:51.083778 ip-10-0-137-244 kubenswrapper[2565]: I0701 17:06:51.083757    2565 reconciler_common.go:299] "Volume detached for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/75034c52-ff8a-4e43-917f-1061c4b9175a-console-config\") on node \"ip-10-0-137-244.ec2.internal\" DevicePath \"\""
Jul 01 17:06:51.083778 ip-10-0-137-244 kubenswrapper[2565]: I0701 17:06:51.083777    2565 reconciler_common.go:299] "Volume detached for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/75034c52-ff8a-4e43-917f-1061c4b9175a-service-ca\") on node \"ip-10-0-137-244.ec2.internal\" DevicePath \"\""
Jul 01 17:06:51.083914 ip-10-0-137-244 kubenswrapper[2565]: I0701 17:06:51.083787    2565 reconciler_common.go:299] "Volume detached for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/75034c52-ff8a-4e43-917f-1061c4b9175a-console-serving-cert\") on node \"ip-10-0-137-244.ec2.internal\" DevicePath \"\""
Jul 01 17:06:51.083914 ip-10-0-137-244 kubenswrapper[2565]: I0701 17:06:51.083796    2565 reconciler_common.go:299] "Volume detached for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/75034c52-ff8a-4e43-917f-1061c4b9175a-oauth-serving-cert\") on node \"ip-10-0-137-244.ec2.internal\" DevicePath \"\""
Jul 01 17:06:51.083914 ip-10-0-137-244 kubenswrapper[2565]: I0701 17:06:51.083804    2565 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-s8xd4\" (UniqueName: \"kubernetes.io/projected/75034c52-ff8a-4e43-917f-1061c4b9175a-kube-api-access-s8xd4\") on node \"ip-10-0-137-244.ec2.internal\" DevicePath \"\""
Jul 01 17:06:51.083914 ip-10-0-137-244 kubenswrapper[2565]: I0701 17:06:51.083814    2565 reconciler_common.go:299] "Volume detached for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/75034c52-ff8a-4e43-917f-1061c4b9175a-trusted-ca-bundle\") on node \"ip-10-0-137-244.ec2.internal\" DevicePath \"\""
Jul 01 17:06:51.083914 ip-10-0-137-244 kubenswrapper[2565]: I0701 17:06:51.083822    2565 reconciler_common.go:299] "Volume detached for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/75034c52-ff8a-4e43-917f-1061c4b9175a-console-oauth-config\") on node \"ip-10-0-137-244.ec2.internal\" DevicePath \"\""
Jul 01 17:06:51.083914 ip-10-0-137-244 kubenswrapper[2565]: I0701 17:06:51.083846    2565 generic.go:356] "Generic (PLEG): container finished" podID="75034c52-ff8a-4e43-917f-1061c4b9175a" containerID="413411056cd1dcd8dffa34e4d5109c298f43c944c6d2f608063eeb6a289718f8" exitCode=0
Jul 01 17:06:51.083914 ip-10-0-137-244 kubenswrapper[2565]: I0701 17:06:51.083875    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-5b74c58dcb-qtxpp" event={"ID":"75034c52-ff8a-4e43-917f-1061c4b9175a","Type":"ContainerDied","Data":"413411056cd1dcd8dffa34e4d5109c298f43c944c6d2f608063eeb6a289718f8"}
Jul 01 17:06:51.083914 ip-10-0-137-244 kubenswrapper[2565]: I0701 17:06:51.083899    2565 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-5b74c58dcb-qtxpp"
Jul 01 17:06:51.084205 ip-10-0-137-244 kubenswrapper[2565]: I0701 17:06:51.083930    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-5b74c58dcb-qtxpp" event={"ID":"75034c52-ff8a-4e43-917f-1061c4b9175a","Type":"ContainerDied","Data":"75bd41a37c02a05ea76892aba454d53f0145e66a82b6c2f8be3d5fca0eff12ce"}
Jul 01 17:06:51.084205 ip-10-0-137-244 kubenswrapper[2565]: I0701 17:06:51.083958    2565 scope.go:117] "RemoveContainer" containerID="413411056cd1dcd8dffa34e4d5109c298f43c944c6d2f608063eeb6a289718f8"
Jul 01 17:06:51.091811 ip-10-0-137-244 kubenswrapper[2565]: I0701 17:06:51.091793    2565 scope.go:117] "RemoveContainer" containerID="413411056cd1dcd8dffa34e4d5109c298f43c944c6d2f608063eeb6a289718f8"
Jul 01 17:06:51.092055 ip-10-0-137-244 kubenswrapper[2565]: E0701 17:06:51.092032    2565 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"413411056cd1dcd8dffa34e4d5109c298f43c944c6d2f608063eeb6a289718f8\": container with ID starting with 413411056cd1dcd8dffa34e4d5109c298f43c944c6d2f608063eeb6a289718f8 not found: ID does not exist" containerID="413411056cd1dcd8dffa34e4d5109c298f43c944c6d2f608063eeb6a289718f8"
Jul 01 17:06:51.092129 ip-10-0-137-244 kubenswrapper[2565]: I0701 17:06:51.092064    2565 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"413411056cd1dcd8dffa34e4d5109c298f43c944c6d2f608063eeb6a289718f8"} err="failed to get container status \"413411056cd1dcd8dffa34e4d5109c298f43c944c6d2f608063eeb6a289718f8\": rpc error: code = NotFound desc = could not find container \"413411056cd1dcd8dffa34e4d5109c298f43c944c6d2f608063eeb6a289718f8\": container with ID starting with 413411056cd1dcd8dffa34e4d5109c298f43c944c6d2f608063eeb6a289718f8 not found: ID does not exist"
Jul 01 17:06:51.107009 ip-10-0-137-244 kubenswrapper[2565]: I0701 17:06:51.106984    2565 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-5b74c58dcb-qtxpp"]
Jul 01 17:06:51.111521 ip-10-0-137-244 kubenswrapper[2565]: I0701 17:06:51.111503    2565 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-console/console-5b74c58dcb-qtxpp"]
Jul 01 17:06:52.248346 ip-10-0-137-244 kubenswrapper[2565]: I0701 17:06:52.248311    2565 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="75034c52-ff8a-4e43-917f-1061c4b9175a" path="/var/lib/kubelet/pods/75034c52-ff8a-4e43-917f-1061c4b9175a/volumes"
Jul 01 17:07:52.742826 ip-10-0-137-244 kubenswrapper[2565]: I0701 17:07:52.742797    2565 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kserve/seaweedfs-tls-custom-6644bf5d6c-x7bxh"]
Jul 01 17:07:52.743231 ip-10-0-137-244 kubenswrapper[2565]: I0701 17:07:52.743099    2565 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="75034c52-ff8a-4e43-917f-1061c4b9175a" containerName="console"
Jul 01 17:07:52.743231 ip-10-0-137-244 kubenswrapper[2565]: I0701 17:07:52.743110    2565 state_mem.go:107] "Deleted CPUSet assignment" podUID="75034c52-ff8a-4e43-917f-1061c4b9175a" containerName="console"
Jul 01 17:07:52.743231 ip-10-0-137-244 kubenswrapper[2565]: I0701 17:07:52.743173    2565 memory_manager.go:368] "RemoveStaleState removing state" podUID="75034c52-ff8a-4e43-917f-1061c4b9175a" containerName="console"
Jul 01 17:07:52.746952 ip-10-0-137-244 kubenswrapper[2565]: I0701 17:07:52.746936    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve/seaweedfs-tls-custom-6644bf5d6c-x7bxh"
Jul 01 17:07:52.750241 ip-10-0-137-244 kubenswrapper[2565]: I0701 17:07:52.750221    2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kserve\"/\"seaweedfs-tls-custom-artifact\""
Jul 01 17:07:52.760778 ip-10-0-137-244 kubenswrapper[2565]: I0701 17:07:52.760757    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve/seaweedfs-tls-custom-6644bf5d6c-x7bxh"]
Jul 01 17:07:52.810424 ip-10-0-137-244 kubenswrapper[2565]: I0701 17:07:52.810402    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-q6k59\" (UniqueName: \"kubernetes.io/projected/fb6b57d2-b315-4b90-963e-6bd640a1cc7c-kube-api-access-q6k59\") pod \"seaweedfs-tls-custom-6644bf5d6c-x7bxh\" (UID: \"fb6b57d2-b315-4b90-963e-6bd640a1cc7c\") " pod="kserve/seaweedfs-tls-custom-6644bf5d6c-x7bxh"
Jul 01 17:07:52.810524 ip-10-0-137-244 kubenswrapper[2565]: I0701 17:07:52.810437    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"data\" (UniqueName: \"kubernetes.io/empty-dir/fb6b57d2-b315-4b90-963e-6bd640a1cc7c-data\") pod \"seaweedfs-tls-custom-6644bf5d6c-x7bxh\" (UID: \"fb6b57d2-b315-4b90-963e-6bd640a1cc7c\") " pod="kserve/seaweedfs-tls-custom-6644bf5d6c-x7bxh"
Jul 01 17:07:52.911433 ip-10-0-137-244 kubenswrapper[2565]: I0701 17:07:52.911406    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-q6k59\" (UniqueName: \"kubernetes.io/projected/fb6b57d2-b315-4b90-963e-6bd640a1cc7c-kube-api-access-q6k59\") pod \"seaweedfs-tls-custom-6644bf5d6c-x7bxh\" (UID: \"fb6b57d2-b315-4b90-963e-6bd640a1cc7c\") " pod="kserve/seaweedfs-tls-custom-6644bf5d6c-x7bxh"
Jul 01 17:07:52.911535 ip-10-0-137-244 kubenswrapper[2565]: I0701 17:07:52.911441    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"data\" (UniqueName: \"kubernetes.io/empty-dir/fb6b57d2-b315-4b90-963e-6bd640a1cc7c-data\") pod \"seaweedfs-tls-custom-6644bf5d6c-x7bxh\" (UID: \"fb6b57d2-b315-4b90-963e-6bd640a1cc7c\") " pod="kserve/seaweedfs-tls-custom-6644bf5d6c-x7bxh"
Jul 01 17:07:52.911787 ip-10-0-137-244 kubenswrapper[2565]: I0701 17:07:52.911766    2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"data\" (UniqueName: \"kubernetes.io/empty-dir/fb6b57d2-b315-4b90-963e-6bd640a1cc7c-data\") pod \"seaweedfs-tls-custom-6644bf5d6c-x7bxh\" (UID: \"fb6b57d2-b315-4b90-963e-6bd640a1cc7c\") " pod="kserve/seaweedfs-tls-custom-6644bf5d6c-x7bxh"
Jul 01 17:07:52.920550 ip-10-0-137-244 kubenswrapper[2565]: I0701 17:07:52.920532    2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-q6k59\" (UniqueName: \"kubernetes.io/projected/fb6b57d2-b315-4b90-963e-6bd640a1cc7c-kube-api-access-q6k59\") pod \"seaweedfs-tls-custom-6644bf5d6c-x7bxh\" (UID: \"fb6b57d2-b315-4b90-963e-6bd640a1cc7c\") " pod="kserve/seaweedfs-tls-custom-6644bf5d6c-x7bxh"
Jul 01 17:07:53.055765 ip-10-0-137-244 kubenswrapper[2565]: I0701 17:07:53.055692    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve/seaweedfs-tls-custom-6644bf5d6c-x7bxh"
Jul 01 17:07:53.187005 ip-10-0-137-244 kubenswrapper[2565]: I0701 17:07:53.186983    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve/seaweedfs-tls-custom-6644bf5d6c-x7bxh"]
Jul 01 17:07:53.189527 ip-10-0-137-244 kubenswrapper[2565]: W0701 17:07:53.189494    2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podfb6b57d2_b315_4b90_963e_6bd640a1cc7c.slice/crio-ca15ac375b0d38c706aac69c7ee6d40deff2a6584737c0f25c7d60fb72560499 WatchSource:0}: Error finding container ca15ac375b0d38c706aac69c7ee6d40deff2a6584737c0f25c7d60fb72560499: Status 404 returned error can't find the container with id ca15ac375b0d38c706aac69c7ee6d40deff2a6584737c0f25c7d60fb72560499
Jul 01 17:07:53.321109 ip-10-0-137-244 kubenswrapper[2565]: I0701 17:07:53.321049    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve/seaweedfs-tls-custom-6644bf5d6c-x7bxh" event={"ID":"fb6b57d2-b315-4b90-963e-6bd640a1cc7c","Type":"ContainerStarted","Data":"ca15ac375b0d38c706aac69c7ee6d40deff2a6584737c0f25c7d60fb72560499"}
Jul 01 17:07:54.330847 ip-10-0-137-244 kubenswrapper[2565]: I0701 17:07:54.330812    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve/seaweedfs-tls-custom-6644bf5d6c-x7bxh" event={"ID":"fb6b57d2-b315-4b90-963e-6bd640a1cc7c","Type":"ContainerStarted","Data":"179f6e5673b4fd4e9ef23f322d73c62bf34589323500e41ddd5cdb256dd6d0e4"}
Jul 01 17:07:54.358934 ip-10-0-137-244 kubenswrapper[2565]: I0701 17:07:54.358885    2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kserve/seaweedfs-tls-custom-6644bf5d6c-x7bxh" podStartSLOduration=2.024270936 podStartE2EDuration="2.358870508s" totalImagesPullingTime="334.599572ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-01 17:07:52 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-01 17:07:54.358071057 +0000 UTC m=+544.826335182" watchObservedRunningTime="2026-07-01 17:07:54.358870508 +0000 UTC m=+544.827134640"
Jul 01 17:07:55.390492 ip-10-0-137-244 kubenswrapper[2565]: I0701 17:07:55.390451    2565 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kserve/seaweedfs-tls-custom-6644bf5d6c-x7bxh"]
Jul 01 17:07:56.338289 ip-10-0-137-244 kubenswrapper[2565]: I0701 17:07:56.338248    2565 kuberuntime_container.go:872] "Killing container with a grace period" pod="kserve/seaweedfs-tls-custom-6644bf5d6c-x7bxh" podUID="fb6b57d2-b315-4b90-963e-6bd640a1cc7c" containerName="seaweedfs-tls-custom" containerID="cri-o://179f6e5673b4fd4e9ef23f322d73c62bf34589323500e41ddd5cdb256dd6d0e4" gracePeriod=30
Jul 01 17:08:24.180843 ip-10-0-137-244 kubenswrapper[2565]: I0701 17:08:24.180818    2565 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve/seaweedfs-tls-custom-6644bf5d6c-x7bxh"
Jul 01 17:08:24.319452 ip-10-0-137-244 kubenswrapper[2565]: I0701 17:08:24.319394    2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"data\" (UniqueName: \"kubernetes.io/empty-dir/fb6b57d2-b315-4b90-963e-6bd640a1cc7c-data\") pod \"fb6b57d2-b315-4b90-963e-6bd640a1cc7c\" (UID: \"fb6b57d2-b315-4b90-963e-6bd640a1cc7c\") "
Jul 01 17:08:24.319452 ip-10-0-137-244 kubenswrapper[2565]: I0701 17:08:24.319439    2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-q6k59\" (UniqueName: \"kubernetes.io/projected/fb6b57d2-b315-4b90-963e-6bd640a1cc7c-kube-api-access-q6k59\") pod \"fb6b57d2-b315-4b90-963e-6bd640a1cc7c\" (UID: \"fb6b57d2-b315-4b90-963e-6bd640a1cc7c\") "
Jul 01 17:08:24.320578 ip-10-0-137-244 kubenswrapper[2565]: I0701 17:08:24.320549    2565 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/fb6b57d2-b315-4b90-963e-6bd640a1cc7c-data" (OuterVolumeSpecName: "data") pod "fb6b57d2-b315-4b90-963e-6bd640a1cc7c" (UID: "fb6b57d2-b315-4b90-963e-6bd640a1cc7c"). InnerVolumeSpecName "data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 01 17:08:24.321461 ip-10-0-137-244 kubenswrapper[2565]: I0701 17:08:24.321428    2565 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/fb6b57d2-b315-4b90-963e-6bd640a1cc7c-kube-api-access-q6k59" (OuterVolumeSpecName: "kube-api-access-q6k59") pod "fb6b57d2-b315-4b90-963e-6bd640a1cc7c" (UID: "fb6b57d2-b315-4b90-963e-6bd640a1cc7c"). InnerVolumeSpecName "kube-api-access-q6k59". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 01 17:08:24.420826 ip-10-0-137-244 kubenswrapper[2565]: I0701 17:08:24.420808    2565 reconciler_common.go:299] "Volume detached for volume \"data\" (UniqueName: \"kubernetes.io/empty-dir/fb6b57d2-b315-4b90-963e-6bd640a1cc7c-data\") on node \"ip-10-0-137-244.ec2.internal\" DevicePath \"\""
Jul 01 17:08:24.420826 ip-10-0-137-244 kubenswrapper[2565]: I0701 17:08:24.420826    2565 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-q6k59\" (UniqueName: \"kubernetes.io/projected/fb6b57d2-b315-4b90-963e-6bd640a1cc7c-kube-api-access-q6k59\") on node \"ip-10-0-137-244.ec2.internal\" DevicePath \"\""
Jul 01 17:08:24.439599 ip-10-0-137-244 kubenswrapper[2565]: I0701 17:08:24.439566    2565 generic.go:356] "Generic (PLEG): container finished" podID="fb6b57d2-b315-4b90-963e-6bd640a1cc7c" containerID="179f6e5673b4fd4e9ef23f322d73c62bf34589323500e41ddd5cdb256dd6d0e4" exitCode=0
Jul 01 17:08:24.439695 ip-10-0-137-244 kubenswrapper[2565]: I0701 17:08:24.439609    2565 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve/seaweedfs-tls-custom-6644bf5d6c-x7bxh"
Jul 01 17:08:24.439695 ip-10-0-137-244 kubenswrapper[2565]: I0701 17:08:24.439656    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve/seaweedfs-tls-custom-6644bf5d6c-x7bxh" event={"ID":"fb6b57d2-b315-4b90-963e-6bd640a1cc7c","Type":"ContainerDied","Data":"179f6e5673b4fd4e9ef23f322d73c62bf34589323500e41ddd5cdb256dd6d0e4"}
Jul 01 17:08:24.439815 ip-10-0-137-244 kubenswrapper[2565]: I0701 17:08:24.439721    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve/seaweedfs-tls-custom-6644bf5d6c-x7bxh" event={"ID":"fb6b57d2-b315-4b90-963e-6bd640a1cc7c","Type":"ContainerDied","Data":"ca15ac375b0d38c706aac69c7ee6d40deff2a6584737c0f25c7d60fb72560499"}
Jul 01 17:08:24.439815 ip-10-0-137-244 kubenswrapper[2565]: I0701 17:08:24.439750    2565 scope.go:117] "RemoveContainer" containerID="179f6e5673b4fd4e9ef23f322d73c62bf34589323500e41ddd5cdb256dd6d0e4"
Jul 01 17:08:24.448445 ip-10-0-137-244 kubenswrapper[2565]: I0701 17:08:24.448428    2565 scope.go:117] "RemoveContainer" containerID="179f6e5673b4fd4e9ef23f322d73c62bf34589323500e41ddd5cdb256dd6d0e4"
Jul 01 17:08:24.448673 ip-10-0-137-244 kubenswrapper[2565]: E0701 17:08:24.448658    2565 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"179f6e5673b4fd4e9ef23f322d73c62bf34589323500e41ddd5cdb256dd6d0e4\": container with ID starting with 179f6e5673b4fd4e9ef23f322d73c62bf34589323500e41ddd5cdb256dd6d0e4 not found: ID does not exist" containerID="179f6e5673b4fd4e9ef23f322d73c62bf34589323500e41ddd5cdb256dd6d0e4"
Jul 01 17:08:24.448709 ip-10-0-137-244 kubenswrapper[2565]: I0701 17:08:24.448682    2565 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"179f6e5673b4fd4e9ef23f322d73c62bf34589323500e41ddd5cdb256dd6d0e4"} err="failed to get container status \"179f6e5673b4fd4e9ef23f322d73c62bf34589323500e41ddd5cdb256dd6d0e4\": rpc error: code = NotFound desc = could not find container \"179f6e5673b4fd4e9ef23f322d73c62bf34589323500e41ddd5cdb256dd6d0e4\": container with ID starting with 179f6e5673b4fd4e9ef23f322d73c62bf34589323500e41ddd5cdb256dd6d0e4 not found: ID does not exist"
Jul 01 17:08:24.459950 ip-10-0-137-244 kubenswrapper[2565]: I0701 17:08:24.459927    2565 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kserve/seaweedfs-tls-custom-6644bf5d6c-x7bxh"]
Jul 01 17:08:24.463643 ip-10-0-137-244 kubenswrapper[2565]: I0701 17:08:24.463624    2565 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kserve/seaweedfs-tls-custom-6644bf5d6c-x7bxh"]
Jul 01 17:08:24.501574 ip-10-0-137-244 kubenswrapper[2565]: I0701 17:08:24.501555    2565 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kserve/seaweedfs-tls-custom-8b44d767d-tbbdm"]
Jul 01 17:08:24.501809 ip-10-0-137-244 kubenswrapper[2565]: I0701 17:08:24.501797    2565 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="fb6b57d2-b315-4b90-963e-6bd640a1cc7c" containerName="seaweedfs-tls-custom"
Jul 01 17:08:24.501851 ip-10-0-137-244 kubenswrapper[2565]: I0701 17:08:24.501810    2565 state_mem.go:107] "Deleted CPUSet assignment" podUID="fb6b57d2-b315-4b90-963e-6bd640a1cc7c" containerName="seaweedfs-tls-custom"
Jul 01 17:08:24.501883 ip-10-0-137-244 kubenswrapper[2565]: I0701 17:08:24.501869    2565 memory_manager.go:368] "RemoveStaleState removing state" podUID="fb6b57d2-b315-4b90-963e-6bd640a1cc7c" containerName="seaweedfs-tls-custom"
Jul 01 17:08:24.505955 ip-10-0-137-244 kubenswrapper[2565]: I0701 17:08:24.505941    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve/seaweedfs-tls-custom-8b44d767d-tbbdm"
Jul 01 17:08:24.508475 ip-10-0-137-244 kubenswrapper[2565]: I0701 17:08:24.508458    2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kserve\"/\"seaweedfs-tls-custom\""
Jul 01 17:08:24.508568 ip-10-0-137-244 kubenswrapper[2565]: I0701 17:08:24.508548    2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kserve\"/\"seaweedfs-tls-custom-artifact\""
Jul 01 17:08:24.515896 ip-10-0-137-244 kubenswrapper[2565]: I0701 17:08:24.515878    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve/seaweedfs-tls-custom-8b44d767d-tbbdm"]
Jul 01 17:08:24.622861 ip-10-0-137-244 kubenswrapper[2565]: I0701 17:08:24.622814    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"data\" (UniqueName: \"kubernetes.io/empty-dir/4bc26672-32e0-457f-b5b7-656bf8693a6c-data\") pod \"seaweedfs-tls-custom-8b44d767d-tbbdm\" (UID: \"4bc26672-32e0-457f-b5b7-656bf8693a6c\") " pod="kserve/seaweedfs-tls-custom-8b44d767d-tbbdm"
Jul 01 17:08:24.622861 ip-10-0-137-244 kubenswrapper[2565]: I0701 17:08:24.622845    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-g2z8r\" (UniqueName: \"kubernetes.io/projected/4bc26672-32e0-457f-b5b7-656bf8693a6c-kube-api-access-g2z8r\") pod \"seaweedfs-tls-custom-8b44d767d-tbbdm\" (UID: \"4bc26672-32e0-457f-b5b7-656bf8693a6c\") " pod="kserve/seaweedfs-tls-custom-8b44d767d-tbbdm"
Jul 01 17:08:24.622983 ip-10-0-137-244 kubenswrapper[2565]: I0701 17:08:24.622878    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"seaweedfs-tls-custom\" (UniqueName: \"kubernetes.io/projected/4bc26672-32e0-457f-b5b7-656bf8693a6c-seaweedfs-tls-custom\") pod \"seaweedfs-tls-custom-8b44d767d-tbbdm\" (UID: \"4bc26672-32e0-457f-b5b7-656bf8693a6c\") " pod="kserve/seaweedfs-tls-custom-8b44d767d-tbbdm"
Jul 01 17:08:24.723792 ip-10-0-137-244 kubenswrapper[2565]: I0701 17:08:24.723770    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"data\" (UniqueName: \"kubernetes.io/empty-dir/4bc26672-32e0-457f-b5b7-656bf8693a6c-data\") pod \"seaweedfs-tls-custom-8b44d767d-tbbdm\" (UID: \"4bc26672-32e0-457f-b5b7-656bf8693a6c\") " pod="kserve/seaweedfs-tls-custom-8b44d767d-tbbdm"
Jul 01 17:08:24.723878 ip-10-0-137-244 kubenswrapper[2565]: I0701 17:08:24.723799    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-g2z8r\" (UniqueName: \"kubernetes.io/projected/4bc26672-32e0-457f-b5b7-656bf8693a6c-kube-api-access-g2z8r\") pod \"seaweedfs-tls-custom-8b44d767d-tbbdm\" (UID: \"4bc26672-32e0-457f-b5b7-656bf8693a6c\") " pod="kserve/seaweedfs-tls-custom-8b44d767d-tbbdm"
Jul 01 17:08:24.723878 ip-10-0-137-244 kubenswrapper[2565]: I0701 17:08:24.723828    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"seaweedfs-tls-custom\" (UniqueName: \"kubernetes.io/projected/4bc26672-32e0-457f-b5b7-656bf8693a6c-seaweedfs-tls-custom\") pod \"seaweedfs-tls-custom-8b44d767d-tbbdm\" (UID: \"4bc26672-32e0-457f-b5b7-656bf8693a6c\") " pod="kserve/seaweedfs-tls-custom-8b44d767d-tbbdm"
Jul 01 17:08:24.724144 ip-10-0-137-244 kubenswrapper[2565]: I0701 17:08:24.724128    2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"data\" (UniqueName: \"kubernetes.io/empty-dir/4bc26672-32e0-457f-b5b7-656bf8693a6c-data\") pod \"seaweedfs-tls-custom-8b44d767d-tbbdm\" (UID: \"4bc26672-32e0-457f-b5b7-656bf8693a6c\") " pod="kserve/seaweedfs-tls-custom-8b44d767d-tbbdm"
Jul 01 17:08:24.726238 ip-10-0-137-244 kubenswrapper[2565]: I0701 17:08:24.726219    2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"seaweedfs-tls-custom\" (UniqueName: \"kubernetes.io/projected/4bc26672-32e0-457f-b5b7-656bf8693a6c-seaweedfs-tls-custom\") pod \"seaweedfs-tls-custom-8b44d767d-tbbdm\" (UID: \"4bc26672-32e0-457f-b5b7-656bf8693a6c\") " pod="kserve/seaweedfs-tls-custom-8b44d767d-tbbdm"
Jul 01 17:08:24.733307 ip-10-0-137-244 kubenswrapper[2565]: I0701 17:08:24.733284    2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-g2z8r\" (UniqueName: \"kubernetes.io/projected/4bc26672-32e0-457f-b5b7-656bf8693a6c-kube-api-access-g2z8r\") pod \"seaweedfs-tls-custom-8b44d767d-tbbdm\" (UID: \"4bc26672-32e0-457f-b5b7-656bf8693a6c\") " pod="kserve/seaweedfs-tls-custom-8b44d767d-tbbdm"
Jul 01 17:08:24.813801 ip-10-0-137-244 kubenswrapper[2565]: I0701 17:08:24.813781    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve/seaweedfs-tls-custom-8b44d767d-tbbdm"
Jul 01 17:08:24.940867 ip-10-0-137-244 kubenswrapper[2565]: I0701 17:08:24.940842    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve/seaweedfs-tls-custom-8b44d767d-tbbdm"]
Jul 01 17:08:24.943237 ip-10-0-137-244 kubenswrapper[2565]: W0701 17:08:24.943215    2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4bc26672_32e0_457f_b5b7_656bf8693a6c.slice/crio-6483253f7b6edfd21a35eb60c50886e06d2aa7d8f1ddef0818526937a6076c91 WatchSource:0}: Error finding container 6483253f7b6edfd21a35eb60c50886e06d2aa7d8f1ddef0818526937a6076c91: Status 404 returned error can't find the container with id 6483253f7b6edfd21a35eb60c50886e06d2aa7d8f1ddef0818526937a6076c91
Jul 01 17:08:25.446342 ip-10-0-137-244 kubenswrapper[2565]: I0701 17:08:25.446307    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve/seaweedfs-tls-custom-8b44d767d-tbbdm" event={"ID":"4bc26672-32e0-457f-b5b7-656bf8693a6c","Type":"ContainerStarted","Data":"d078edbfc7d8ab2b864fbdb93bb72674d2e2dbb5e7a8dedb49652d1ed1759335"}
Jul 01 17:08:25.446756 ip-10-0-137-244 kubenswrapper[2565]: I0701 17:08:25.446366    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve/seaweedfs-tls-custom-8b44d767d-tbbdm" event={"ID":"4bc26672-32e0-457f-b5b7-656bf8693a6c","Type":"ContainerStarted","Data":"6483253f7b6edfd21a35eb60c50886e06d2aa7d8f1ddef0818526937a6076c91"}
Jul 01 17:08:25.475660 ip-10-0-137-244 kubenswrapper[2565]: I0701 17:08:25.475620    2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kserve/seaweedfs-tls-custom-8b44d767d-tbbdm" podStartSLOduration=1.162870967 podStartE2EDuration="1.475605873s" totalImagesPullingTime="312.734906ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-01 17:08:24 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-01 17:08:25.475204472 +0000 UTC m=+575.943468619" watchObservedRunningTime="2026-07-01 17:08:25.475605873 +0000 UTC m=+575.943869998"
Jul 01 17:08:26.248571 ip-10-0-137-244 kubenswrapper[2565]: I0701 17:08:26.248532    2565 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="fb6b57d2-b315-4b90-963e-6bd640a1cc7c" path="/var/lib/kubelet/pods/fb6b57d2-b315-4b90-963e-6bd640a1cc7c/volumes"
Jul 01 17:08:50.143602 ip-10-0-137-244 kubenswrapper[2565]: I0701 17:08:50.143562    2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-zjsw4_bb7ebc32-d0cd-4eb0-b998-3ce298f73809/ovn-acl-logging/0.log"
Jul 01 17:08:50.144113 ip-10-0-137-244 kubenswrapper[2565]: I0701 17:08:50.143820    2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-zjsw4_bb7ebc32-d0cd-4eb0-b998-3ce298f73809/ovn-acl-logging/0.log"
Jul 01 17:09:07.061371 ip-10-0-137-244 kubenswrapper[2565]: I0701 17:09:07.061341    2565 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kserve-ci-e2e-test/isvc-sklearn-batcher-predictor-869f8b8c7d-mn6kq"]
Jul 01 17:09:07.064540 ip-10-0-137-244 kubenswrapper[2565]: I0701 17:09:07.064525    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/isvc-sklearn-batcher-predictor-869f8b8c7d-mn6kq"
Jul 01 17:09:07.067232 ip-10-0-137-244 kubenswrapper[2565]: I0701 17:09:07.067211    2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kserve-ci-e2e-test\"/\"openshift-service-ca.crt\""
Jul 01 17:09:07.067336 ip-10-0-137-244 kubenswrapper[2565]: I0701 17:09:07.067241    2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kserve-ci-e2e-test\"/\"isvc-sklearn-batcher-predictor-serving-cert\""
Jul 01 17:09:07.067674 ip-10-0-137-244 kubenswrapper[2565]: I0701 17:09:07.067660    2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kserve-ci-e2e-test\"/\"default-dockercfg-bl7zk\""
Jul 01 17:09:07.067725 ip-10-0-137-244 kubenswrapper[2565]: I0701 17:09:07.067710    2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kserve-ci-e2e-test\"/\"seaweedfs-s3-creds\""
Jul 01 17:09:07.068387 ip-10-0-137-244 kubenswrapper[2565]: I0701 17:09:07.068369    2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kserve-ci-e2e-test\"/\"isvc-sklearn-batcher-kube-rbac-proxy-sar-config\""
Jul 01 17:09:07.068470 ip-10-0-137-244 kubenswrapper[2565]: I0701 17:09:07.068377    2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kserve-ci-e2e-test\"/\"kube-root-ca.crt\""
Jul 01 17:09:07.068628 ip-10-0-137-244 kubenswrapper[2565]: I0701 17:09:07.068607    2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kserve-ci-e2e-test\"/\"odh-kserve-custom-ca-bundle\""
Jul 01 17:09:07.085086 ip-10-0-137-244 kubenswrapper[2565]: I0701 17:09:07.085065    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/isvc-sklearn-batcher-predictor-869f8b8c7d-mn6kq"]
Jul 01 17:09:07.108153 ip-10-0-137-244 kubenswrapper[2565]: I0701 17:09:07.108128    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-n452k\" (UniqueName: \"kubernetes.io/projected/98aa0b13-a938-471c-92d2-e0a9e84651b8-kube-api-access-n452k\") pod \"isvc-sklearn-batcher-predictor-869f8b8c7d-mn6kq\" (UID: \"98aa0b13-a938-471c-92d2-e0a9e84651b8\") " pod="kserve-ci-e2e-test/isvc-sklearn-batcher-predictor-869f8b8c7d-mn6kq"
Jul 01 17:09:07.108257 ip-10-0-137-244 kubenswrapper[2565]: I0701 17:09:07.108162    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/98aa0b13-a938-471c-92d2-e0a9e84651b8-kube-rbac-proxy-sar-config\") pod \"isvc-sklearn-batcher-predictor-869f8b8c7d-mn6kq\" (UID: \"98aa0b13-a938-471c-92d2-e0a9e84651b8\") " pod="kserve-ci-e2e-test/isvc-sklearn-batcher-predictor-869f8b8c7d-mn6kq"
Jul 01 17:09:07.108257 ip-10-0-137-244 kubenswrapper[2565]: I0701 17:09:07.108187    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/98aa0b13-a938-471c-92d2-e0a9e84651b8-kserve-provision-location\") pod \"isvc-sklearn-batcher-predictor-869f8b8c7d-mn6kq\" (UID: \"98aa0b13-a938-471c-92d2-e0a9e84651b8\") " pod="kserve-ci-e2e-test/isvc-sklearn-batcher-predictor-869f8b8c7d-mn6kq"
Jul 01 17:09:07.108257 ip-10-0-137-244 kubenswrapper[2565]: I0701 17:09:07.108238    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cabundle-cert\" (UniqueName: \"kubernetes.io/configmap/98aa0b13-a938-471c-92d2-e0a9e84651b8-cabundle-cert\") pod \"isvc-sklearn-batcher-predictor-869f8b8c7d-mn6kq\" (UID: \"98aa0b13-a938-471c-92d2-e0a9e84651b8\") " pod="kserve-ci-e2e-test/isvc-sklearn-batcher-predictor-869f8b8c7d-mn6kq"
Jul 01 17:09:07.108257 ip-10-0-137-244 kubenswrapper[2565]: I0701 17:09:07.108257    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/98aa0b13-a938-471c-92d2-e0a9e84651b8-proxy-tls\") pod \"isvc-sklearn-batcher-predictor-869f8b8c7d-mn6kq\" (UID: \"98aa0b13-a938-471c-92d2-e0a9e84651b8\") " pod="kserve-ci-e2e-test/isvc-sklearn-batcher-predictor-869f8b8c7d-mn6kq"
Jul 01 17:09:07.208540 ip-10-0-137-244 kubenswrapper[2565]: I0701 17:09:07.208518    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/98aa0b13-a938-471c-92d2-e0a9e84651b8-kube-rbac-proxy-sar-config\") pod \"isvc-sklearn-batcher-predictor-869f8b8c7d-mn6kq\" (UID: \"98aa0b13-a938-471c-92d2-e0a9e84651b8\") " pod="kserve-ci-e2e-test/isvc-sklearn-batcher-predictor-869f8b8c7d-mn6kq"
Jul 01 17:09:07.208636 ip-10-0-137-244 kubenswrapper[2565]: I0701 17:09:07.208550    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/98aa0b13-a938-471c-92d2-e0a9e84651b8-kserve-provision-location\") pod \"isvc-sklearn-batcher-predictor-869f8b8c7d-mn6kq\" (UID: \"98aa0b13-a938-471c-92d2-e0a9e84651b8\") " pod="kserve-ci-e2e-test/isvc-sklearn-batcher-predictor-869f8b8c7d-mn6kq"
Jul 01 17:09:07.208636 ip-10-0-137-244 kubenswrapper[2565]: I0701 17:09:07.208605    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cabundle-cert\" (UniqueName: \"kubernetes.io/configmap/98aa0b13-a938-471c-92d2-e0a9e84651b8-cabundle-cert\") pod \"isvc-sklearn-batcher-predictor-869f8b8c7d-mn6kq\" (UID: \"98aa0b13-a938-471c-92d2-e0a9e84651b8\") " pod="kserve-ci-e2e-test/isvc-sklearn-batcher-predictor-869f8b8c7d-mn6kq"
Jul 01 17:09:07.208707 ip-10-0-137-244 kubenswrapper[2565]: I0701 17:09:07.208636    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/98aa0b13-a938-471c-92d2-e0a9e84651b8-proxy-tls\") pod \"isvc-sklearn-batcher-predictor-869f8b8c7d-mn6kq\" (UID: \"98aa0b13-a938-471c-92d2-e0a9e84651b8\") " pod="kserve-ci-e2e-test/isvc-sklearn-batcher-predictor-869f8b8c7d-mn6kq"
Jul 01 17:09:07.208707 ip-10-0-137-244 kubenswrapper[2565]: I0701 17:09:07.208683    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-n452k\" (UniqueName: \"kubernetes.io/projected/98aa0b13-a938-471c-92d2-e0a9e84651b8-kube-api-access-n452k\") pod \"isvc-sklearn-batcher-predictor-869f8b8c7d-mn6kq\" (UID: \"98aa0b13-a938-471c-92d2-e0a9e84651b8\") " pod="kserve-ci-e2e-test/isvc-sklearn-batcher-predictor-869f8b8c7d-mn6kq"
Jul 01 17:09:07.208947 ip-10-0-137-244 kubenswrapper[2565]: I0701 17:09:07.208927    2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/98aa0b13-a938-471c-92d2-e0a9e84651b8-kserve-provision-location\") pod \"isvc-sklearn-batcher-predictor-869f8b8c7d-mn6kq\" (UID: \"98aa0b13-a938-471c-92d2-e0a9e84651b8\") " pod="kserve-ci-e2e-test/isvc-sklearn-batcher-predictor-869f8b8c7d-mn6kq"
Jul 01 17:09:07.209342 ip-10-0-137-244 kubenswrapper[2565]: I0701 17:09:07.209320    2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cabundle-cert\" (UniqueName: \"kubernetes.io/configmap/98aa0b13-a938-471c-92d2-e0a9e84651b8-cabundle-cert\") pod \"isvc-sklearn-batcher-predictor-869f8b8c7d-mn6kq\" (UID: \"98aa0b13-a938-471c-92d2-e0a9e84651b8\") " pod="kserve-ci-e2e-test/isvc-sklearn-batcher-predictor-869f8b8c7d-mn6kq"
Jul 01 17:09:07.209435 ip-10-0-137-244 kubenswrapper[2565]: I0701 17:09:07.209420    2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/98aa0b13-a938-471c-92d2-e0a9e84651b8-kube-rbac-proxy-sar-config\") pod \"isvc-sklearn-batcher-predictor-869f8b8c7d-mn6kq\" (UID: \"98aa0b13-a938-471c-92d2-e0a9e84651b8\") " pod="kserve-ci-e2e-test/isvc-sklearn-batcher-predictor-869f8b8c7d-mn6kq"
Jul 01 17:09:07.211261 ip-10-0-137-244 kubenswrapper[2565]: I0701 17:09:07.211241    2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/98aa0b13-a938-471c-92d2-e0a9e84651b8-proxy-tls\") pod \"isvc-sklearn-batcher-predictor-869f8b8c7d-mn6kq\" (UID: \"98aa0b13-a938-471c-92d2-e0a9e84651b8\") " pod="kserve-ci-e2e-test/isvc-sklearn-batcher-predictor-869f8b8c7d-mn6kq"
Jul 01 17:09:07.217215 ip-10-0-137-244 kubenswrapper[2565]: I0701 17:09:07.217191    2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-n452k\" (UniqueName: \"kubernetes.io/projected/98aa0b13-a938-471c-92d2-e0a9e84651b8-kube-api-access-n452k\") pod \"isvc-sklearn-batcher-predictor-869f8b8c7d-mn6kq\" (UID: \"98aa0b13-a938-471c-92d2-e0a9e84651b8\") " pod="kserve-ci-e2e-test/isvc-sklearn-batcher-predictor-869f8b8c7d-mn6kq"
Jul 01 17:09:07.374856 ip-10-0-137-244 kubenswrapper[2565]: I0701 17:09:07.374791    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/isvc-sklearn-batcher-predictor-869f8b8c7d-mn6kq"
Jul 01 17:09:07.506982 ip-10-0-137-244 kubenswrapper[2565]: I0701 17:09:07.506853    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/isvc-sklearn-batcher-predictor-869f8b8c7d-mn6kq"]
Jul 01 17:09:07.509793 ip-10-0-137-244 kubenswrapper[2565]: W0701 17:09:07.509759    2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod98aa0b13_a938_471c_92d2_e0a9e84651b8.slice/crio-a5f82679783cbf708484665136729845fecfddebbb69f1494579db87bfe35aa7 WatchSource:0}: Error finding container a5f82679783cbf708484665136729845fecfddebbb69f1494579db87bfe35aa7: Status 404 returned error can't find the container with id a5f82679783cbf708484665136729845fecfddebbb69f1494579db87bfe35aa7
Jul 01 17:09:07.602358 ip-10-0-137-244 kubenswrapper[2565]: I0701 17:09:07.602331    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/isvc-sklearn-batcher-predictor-869f8b8c7d-mn6kq" event={"ID":"98aa0b13-a938-471c-92d2-e0a9e84651b8","Type":"ContainerStarted","Data":"a5f82679783cbf708484665136729845fecfddebbb69f1494579db87bfe35aa7"}
Jul 01 17:09:11.622245 ip-10-0-137-244 kubenswrapper[2565]: I0701 17:09:11.622207    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/isvc-sklearn-batcher-predictor-869f8b8c7d-mn6kq" event={"ID":"98aa0b13-a938-471c-92d2-e0a9e84651b8","Type":"ContainerStarted","Data":"b172891e59532a372ceb1ce766ce0d29ca741a7cf3187e2181b5b98a650d446d"}
Jul 01 17:09:15.641163 ip-10-0-137-244 kubenswrapper[2565]: I0701 17:09:15.641130    2565 generic.go:356] "Generic (PLEG): container finished" podID="98aa0b13-a938-471c-92d2-e0a9e84651b8" containerID="b172891e59532a372ceb1ce766ce0d29ca741a7cf3187e2181b5b98a650d446d" exitCode=0
Jul 01 17:09:15.641579 ip-10-0-137-244 kubenswrapper[2565]: I0701 17:09:15.641209    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/isvc-sklearn-batcher-predictor-869f8b8c7d-mn6kq" event={"ID":"98aa0b13-a938-471c-92d2-e0a9e84651b8","Type":"ContainerDied","Data":"b172891e59532a372ceb1ce766ce0d29ca741a7cf3187e2181b5b98a650d446d"}
Jul 01 17:09:29.709221 ip-10-0-137-244 kubenswrapper[2565]: I0701 17:09:29.709181    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/isvc-sklearn-batcher-predictor-869f8b8c7d-mn6kq" event={"ID":"98aa0b13-a938-471c-92d2-e0a9e84651b8","Type":"ContainerStarted","Data":"c7418a6c985834a77d12d17732fed7482a6a874be36679cd2c831a11ce861c16"}
Jul 01 17:09:31.720491 ip-10-0-137-244 kubenswrapper[2565]: I0701 17:09:31.720463    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/isvc-sklearn-batcher-predictor-869f8b8c7d-mn6kq" event={"ID":"98aa0b13-a938-471c-92d2-e0a9e84651b8","Type":"ContainerStarted","Data":"c5dbc0cce34af13c24a4c085ccb758a6d33fe5ab63508cae5049235235dff64e"}
Jul 01 17:09:34.734619 ip-10-0-137-244 kubenswrapper[2565]: I0701 17:09:34.734579    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/isvc-sklearn-batcher-predictor-869f8b8c7d-mn6kq" event={"ID":"98aa0b13-a938-471c-92d2-e0a9e84651b8","Type":"ContainerStarted","Data":"3ae071519da1e893dbced83b5dcc049ed9a9f8fcdfabe21102ef59f28500b4ef"}
Jul 01 17:09:34.735045 ip-10-0-137-244 kubenswrapper[2565]: I0701 17:09:34.734827    2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve-ci-e2e-test/isvc-sklearn-batcher-predictor-869f8b8c7d-mn6kq"
Jul 01 17:09:34.735045 ip-10-0-137-244 kubenswrapper[2565]: I0701 17:09:34.734876    2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve-ci-e2e-test/isvc-sklearn-batcher-predictor-869f8b8c7d-mn6kq"
Jul 01 17:09:34.736070 ip-10-0-137-244 kubenswrapper[2565]: I0701 17:09:34.736040    2565 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-sklearn-batcher-predictor-869f8b8c7d-mn6kq" podUID="98aa0b13-a938-471c-92d2-e0a9e84651b8" containerName="kserve-container" probeResult="failure" output="dial tcp 10.133.0.25:8080: connect: connection refused"
Jul 01 17:09:34.758860 ip-10-0-137-244 kubenswrapper[2565]: I0701 17:09:34.758820    2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kserve-ci-e2e-test/isvc-sklearn-batcher-predictor-869f8b8c7d-mn6kq" podStartSLOduration=2.292050136 podStartE2EDuration="27.758807959s" totalImagesPullingTime="21.905738041s" totalInitContainerRuntime="3.561019782s" isStatefulPod=false podCreationTimestamp="2026-07-01 17:09:07 +0000 UTC" imagePullSessionsCount=4 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-01 17:09:34.756695016 +0000 UTC m=+645.224959139" watchObservedRunningTime="2026-07-01 17:09:34.758807959 +0000 UTC m=+645.227072083"
Jul 01 17:09:35.739482 ip-10-0-137-244 kubenswrapper[2565]: I0701 17:09:35.739452    2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve-ci-e2e-test/isvc-sklearn-batcher-predictor-869f8b8c7d-mn6kq"
Jul 01 17:09:35.739919 ip-10-0-137-244 kubenswrapper[2565]: I0701 17:09:35.739548    2565 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-sklearn-batcher-predictor-869f8b8c7d-mn6kq" podUID="98aa0b13-a938-471c-92d2-e0a9e84651b8" containerName="kserve-container" probeResult="failure" output="dial tcp 10.133.0.25:8080: connect: connection refused"
Jul 01 17:09:35.740529 ip-10-0-137-244 kubenswrapper[2565]: I0701 17:09:35.740505    2565 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-sklearn-batcher-predictor-869f8b8c7d-mn6kq" podUID="98aa0b13-a938-471c-92d2-e0a9e84651b8" containerName="agent" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Jul 01 17:09:36.743460 ip-10-0-137-244 kubenswrapper[2565]: I0701 17:09:36.743410    2565 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-sklearn-batcher-predictor-869f8b8c7d-mn6kq" podUID="98aa0b13-a938-471c-92d2-e0a9e84651b8" containerName="kserve-container" probeResult="failure" output="dial tcp 10.133.0.25:8080: connect: connection refused"
Jul 01 17:09:36.743946 ip-10-0-137-244 kubenswrapper[2565]: I0701 17:09:36.743920    2565 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-sklearn-batcher-predictor-869f8b8c7d-mn6kq" podUID="98aa0b13-a938-471c-92d2-e0a9e84651b8" containerName="agent" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Jul 01 17:09:36.747443 ip-10-0-137-244 kubenswrapper[2565]: I0701 17:09:36.747422    2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kserve-ci-e2e-test/isvc-sklearn-batcher-predictor-869f8b8c7d-mn6kq"
Jul 01 17:09:37.748073 ip-10-0-137-244 kubenswrapper[2565]: I0701 17:09:37.748030    2565 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-sklearn-batcher-predictor-869f8b8c7d-mn6kq" podUID="98aa0b13-a938-471c-92d2-e0a9e84651b8" containerName="kserve-container" probeResult="failure" output="dial tcp 10.133.0.25:8080: connect: connection refused"
Jul 01 17:09:37.748525 ip-10-0-137-244 kubenswrapper[2565]: I0701 17:09:37.748417    2565 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-sklearn-batcher-predictor-869f8b8c7d-mn6kq" podUID="98aa0b13-a938-471c-92d2-e0a9e84651b8" containerName="agent" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Jul 01 17:09:47.748083 ip-10-0-137-244 kubenswrapper[2565]: I0701 17:09:47.748014    2565 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-sklearn-batcher-predictor-869f8b8c7d-mn6kq" podUID="98aa0b13-a938-471c-92d2-e0a9e84651b8" containerName="kserve-container" probeResult="failure" output="dial tcp 10.133.0.25:8080: connect: connection refused"
Jul 01 17:09:47.748536 ip-10-0-137-244 kubenswrapper[2565]: I0701 17:09:47.748429    2565 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-sklearn-batcher-predictor-869f8b8c7d-mn6kq" podUID="98aa0b13-a938-471c-92d2-e0a9e84651b8" containerName="agent" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Jul 01 17:09:57.748433 ip-10-0-137-244 kubenswrapper[2565]: I0701 17:09:57.748389    2565 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-sklearn-batcher-predictor-869f8b8c7d-mn6kq" podUID="98aa0b13-a938-471c-92d2-e0a9e84651b8" containerName="kserve-container" probeResult="failure" output="dial tcp 10.133.0.25:8080: connect: connection refused"
Jul 01 17:09:57.748825 ip-10-0-137-244 kubenswrapper[2565]: I0701 17:09:57.748799    2565 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-sklearn-batcher-predictor-869f8b8c7d-mn6kq" podUID="98aa0b13-a938-471c-92d2-e0a9e84651b8" containerName="agent" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Jul 01 17:10:07.747962 ip-10-0-137-244 kubenswrapper[2565]: I0701 17:10:07.747914    2565 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-sklearn-batcher-predictor-869f8b8c7d-mn6kq" podUID="98aa0b13-a938-471c-92d2-e0a9e84651b8" containerName="kserve-container" probeResult="failure" output="dial tcp 10.133.0.25:8080: connect: connection refused"
Jul 01 17:10:07.748462 ip-10-0-137-244 kubenswrapper[2565]: I0701 17:10:07.748402    2565 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-sklearn-batcher-predictor-869f8b8c7d-mn6kq" podUID="98aa0b13-a938-471c-92d2-e0a9e84651b8" containerName="agent" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Jul 01 17:10:17.748871 ip-10-0-137-244 kubenswrapper[2565]: I0701 17:10:17.748832    2565 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-sklearn-batcher-predictor-869f8b8c7d-mn6kq" podUID="98aa0b13-a938-471c-92d2-e0a9e84651b8" containerName="kserve-container" probeResult="failure" output="dial tcp 10.133.0.25:8080: connect: connection refused"
Jul 01 17:10:17.749274 ip-10-0-137-244 kubenswrapper[2565]: I0701 17:10:17.749252    2565 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-sklearn-batcher-predictor-869f8b8c7d-mn6kq" podUID="98aa0b13-a938-471c-92d2-e0a9e84651b8" containerName="agent" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Jul 01 17:10:27.748582 ip-10-0-137-244 kubenswrapper[2565]: I0701 17:10:27.748528    2565 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-sklearn-batcher-predictor-869f8b8c7d-mn6kq" podUID="98aa0b13-a938-471c-92d2-e0a9e84651b8" containerName="kserve-container" probeResult="failure" output="dial tcp 10.133.0.25:8080: connect: connection refused"
Jul 01 17:10:27.749211 ip-10-0-137-244 kubenswrapper[2565]: I0701 17:10:27.749014    2565 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-sklearn-batcher-predictor-869f8b8c7d-mn6kq" podUID="98aa0b13-a938-471c-92d2-e0a9e84651b8" containerName="agent" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Jul 01 17:10:37.748449 ip-10-0-137-244 kubenswrapper[2565]: I0701 17:10:37.748396    2565 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-sklearn-batcher-predictor-869f8b8c7d-mn6kq" podUID="98aa0b13-a938-471c-92d2-e0a9e84651b8" containerName="kserve-container" probeResult="failure" output="dial tcp 10.133.0.25:8080: connect: connection refused"
Jul 01 17:10:37.748863 ip-10-0-137-244 kubenswrapper[2565]: I0701 17:10:37.748804    2565 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-sklearn-batcher-predictor-869f8b8c7d-mn6kq" podUID="98aa0b13-a938-471c-92d2-e0a9e84651b8" containerName="agent" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Jul 01 17:10:47.748991 ip-10-0-137-244 kubenswrapper[2565]: I0701 17:10:47.748903    2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kserve-ci-e2e-test/isvc-sklearn-batcher-predictor-869f8b8c7d-mn6kq"
Jul 01 17:10:47.749460 ip-10-0-137-244 kubenswrapper[2565]: I0701 17:10:47.749337    2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kserve-ci-e2e-test/isvc-sklearn-batcher-predictor-869f8b8c7d-mn6kq"
Jul 01 17:11:02.220667 ip-10-0-137-244 kubenswrapper[2565]: I0701 17:11:02.220632    2565 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kserve-ci-e2e-test/isvc-sklearn-batcher-predictor-869f8b8c7d-mn6kq"]
Jul 01 17:11:02.221126 ip-10-0-137-244 kubenswrapper[2565]: I0701 17:11:02.220903    2565 kuberuntime_container.go:872] "Killing container with a grace period" pod="kserve-ci-e2e-test/isvc-sklearn-batcher-predictor-869f8b8c7d-mn6kq" podUID="98aa0b13-a938-471c-92d2-e0a9e84651b8" containerName="kserve-container" containerID="cri-o://c7418a6c985834a77d12d17732fed7482a6a874be36679cd2c831a11ce861c16" gracePeriod=30
Jul 01 17:11:02.221126 ip-10-0-137-244 kubenswrapper[2565]: I0701 17:11:02.220916    2565 kuberuntime_container.go:872] "Killing container with a grace period" pod="kserve-ci-e2e-test/isvc-sklearn-batcher-predictor-869f8b8c7d-mn6kq" podUID="98aa0b13-a938-471c-92d2-e0a9e84651b8" containerName="agent" containerID="cri-o://3ae071519da1e893dbced83b5dcc049ed9a9f8fcdfabe21102ef59f28500b4ef" gracePeriod=30
Jul 01 17:11:02.221126 ip-10-0-137-244 kubenswrapper[2565]: I0701 17:11:02.220936    2565 kuberuntime_container.go:872] "Killing container with a grace period" pod="kserve-ci-e2e-test/isvc-sklearn-batcher-predictor-869f8b8c7d-mn6kq" podUID="98aa0b13-a938-471c-92d2-e0a9e84651b8" containerName="kube-rbac-proxy" containerID="cri-o://c5dbc0cce34af13c24a4c085ccb758a6d33fe5ab63508cae5049235235dff64e" gracePeriod=30
Jul 01 17:11:02.237000 ip-10-0-137-244 kubenswrapper[2565]: I0701 17:11:02.236979    2565 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kserve-ci-e2e-test/isvc-sklearn-batcher-custom-predictor-5fccccdbd5-vc7st"]
Jul 01 17:11:02.242115 ip-10-0-137-244 kubenswrapper[2565]: I0701 17:11:02.241411    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/isvc-sklearn-batcher-custom-predictor-5fccccdbd5-vc7st"
Jul 01 17:11:02.245129 ip-10-0-137-244 kubenswrapper[2565]: I0701 17:11:02.244695    2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kserve-ci-e2e-test\"/\"isvc-sklearn-batcher-custom-kube-rbac-proxy-sar-config\""
Jul 01 17:11:02.245129 ip-10-0-137-244 kubenswrapper[2565]: I0701 17:11:02.244905    2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kserve-ci-e2e-test\"/\"isvc-sklearn-batcher-custom-predictor-serving-cert\""
Jul 01 17:11:02.253317 ip-10-0-137-244 kubenswrapper[2565]: I0701 17:11:02.253293    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/isvc-sklearn-batcher-custom-predictor-5fccccdbd5-vc7st"]
Jul 01 17:11:02.313688 ip-10-0-137-244 kubenswrapper[2565]: I0701 17:11:02.313668    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/f7c7e394-4643-49ef-b2ca-dff32f41ff0b-kube-rbac-proxy-sar-config\") pod \"isvc-sklearn-batcher-custom-predictor-5fccccdbd5-vc7st\" (UID: \"f7c7e394-4643-49ef-b2ca-dff32f41ff0b\") " pod="kserve-ci-e2e-test/isvc-sklearn-batcher-custom-predictor-5fccccdbd5-vc7st"
Jul 01 17:11:02.313804 ip-10-0-137-244 kubenswrapper[2565]: I0701 17:11:02.313705    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/f7c7e394-4643-49ef-b2ca-dff32f41ff0b-proxy-tls\") pod \"isvc-sklearn-batcher-custom-predictor-5fccccdbd5-vc7st\" (UID: \"f7c7e394-4643-49ef-b2ca-dff32f41ff0b\") " pod="kserve-ci-e2e-test/isvc-sklearn-batcher-custom-predictor-5fccccdbd5-vc7st"
Jul 01 17:11:02.313804 ip-10-0-137-244 kubenswrapper[2565]: I0701 17:11:02.313728    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/f7c7e394-4643-49ef-b2ca-dff32f41ff0b-kserve-provision-location\") pod \"isvc-sklearn-batcher-custom-predictor-5fccccdbd5-vc7st\" (UID: \"f7c7e394-4643-49ef-b2ca-dff32f41ff0b\") " pod="kserve-ci-e2e-test/isvc-sklearn-batcher-custom-predictor-5fccccdbd5-vc7st"
Jul 01 17:11:02.313804 ip-10-0-137-244 kubenswrapper[2565]: I0701 17:11:02.313797    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-f8km5\" (UniqueName: \"kubernetes.io/projected/f7c7e394-4643-49ef-b2ca-dff32f41ff0b-kube-api-access-f8km5\") pod \"isvc-sklearn-batcher-custom-predictor-5fccccdbd5-vc7st\" (UID: \"f7c7e394-4643-49ef-b2ca-dff32f41ff0b\") " pod="kserve-ci-e2e-test/isvc-sklearn-batcher-custom-predictor-5fccccdbd5-vc7st"
Jul 01 17:11:02.313913 ip-10-0-137-244 kubenswrapper[2565]: I0701 17:11:02.313850    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cabundle-cert\" (UniqueName: \"kubernetes.io/configmap/f7c7e394-4643-49ef-b2ca-dff32f41ff0b-cabundle-cert\") pod \"isvc-sklearn-batcher-custom-predictor-5fccccdbd5-vc7st\" (UID: \"f7c7e394-4643-49ef-b2ca-dff32f41ff0b\") " pod="kserve-ci-e2e-test/isvc-sklearn-batcher-custom-predictor-5fccccdbd5-vc7st"
Jul 01 17:11:02.415149 ip-10-0-137-244 kubenswrapper[2565]: I0701 17:11:02.415128    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/f7c7e394-4643-49ef-b2ca-dff32f41ff0b-proxy-tls\") pod \"isvc-sklearn-batcher-custom-predictor-5fccccdbd5-vc7st\" (UID: \"f7c7e394-4643-49ef-b2ca-dff32f41ff0b\") " pod="kserve-ci-e2e-test/isvc-sklearn-batcher-custom-predictor-5fccccdbd5-vc7st"
Jul 01 17:11:02.415269 ip-10-0-137-244 kubenswrapper[2565]: I0701 17:11:02.415160    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/f7c7e394-4643-49ef-b2ca-dff32f41ff0b-kserve-provision-location\") pod \"isvc-sklearn-batcher-custom-predictor-5fccccdbd5-vc7st\" (UID: \"f7c7e394-4643-49ef-b2ca-dff32f41ff0b\") " pod="kserve-ci-e2e-test/isvc-sklearn-batcher-custom-predictor-5fccccdbd5-vc7st"
Jul 01 17:11:02.415269 ip-10-0-137-244 kubenswrapper[2565]: I0701 17:11:02.415186    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-f8km5\" (UniqueName: \"kubernetes.io/projected/f7c7e394-4643-49ef-b2ca-dff32f41ff0b-kube-api-access-f8km5\") pod \"isvc-sklearn-batcher-custom-predictor-5fccccdbd5-vc7st\" (UID: \"f7c7e394-4643-49ef-b2ca-dff32f41ff0b\") " pod="kserve-ci-e2e-test/isvc-sklearn-batcher-custom-predictor-5fccccdbd5-vc7st"
Jul 01 17:11:02.415269 ip-10-0-137-244 kubenswrapper[2565]: I0701 17:11:02.415226    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cabundle-cert\" (UniqueName: \"kubernetes.io/configmap/f7c7e394-4643-49ef-b2ca-dff32f41ff0b-cabundle-cert\") pod \"isvc-sklearn-batcher-custom-predictor-5fccccdbd5-vc7st\" (UID: \"f7c7e394-4643-49ef-b2ca-dff32f41ff0b\") " pod="kserve-ci-e2e-test/isvc-sklearn-batcher-custom-predictor-5fccccdbd5-vc7st"
Jul 01 17:11:02.415269 ip-10-0-137-244 kubenswrapper[2565]: I0701 17:11:02.415253    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/f7c7e394-4643-49ef-b2ca-dff32f41ff0b-kube-rbac-proxy-sar-config\") pod \"isvc-sklearn-batcher-custom-predictor-5fccccdbd5-vc7st\" (UID: \"f7c7e394-4643-49ef-b2ca-dff32f41ff0b\") " pod="kserve-ci-e2e-test/isvc-sklearn-batcher-custom-predictor-5fccccdbd5-vc7st"
Jul 01 17:11:02.415606 ip-10-0-137-244 kubenswrapper[2565]: I0701 17:11:02.415584    2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/f7c7e394-4643-49ef-b2ca-dff32f41ff0b-kserve-provision-location\") pod \"isvc-sklearn-batcher-custom-predictor-5fccccdbd5-vc7st\" (UID: \"f7c7e394-4643-49ef-b2ca-dff32f41ff0b\") " pod="kserve-ci-e2e-test/isvc-sklearn-batcher-custom-predictor-5fccccdbd5-vc7st"
Jul 01 17:11:02.415876 ip-10-0-137-244 kubenswrapper[2565]: I0701 17:11:02.415850    2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/f7c7e394-4643-49ef-b2ca-dff32f41ff0b-kube-rbac-proxy-sar-config\") pod \"isvc-sklearn-batcher-custom-predictor-5fccccdbd5-vc7st\" (UID: \"f7c7e394-4643-49ef-b2ca-dff32f41ff0b\") " pod="kserve-ci-e2e-test/isvc-sklearn-batcher-custom-predictor-5fccccdbd5-vc7st"
Jul 01 17:11:02.415956 ip-10-0-137-244 kubenswrapper[2565]: I0701 17:11:02.415924    2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cabundle-cert\" (UniqueName: \"kubernetes.io/configmap/f7c7e394-4643-49ef-b2ca-dff32f41ff0b-cabundle-cert\") pod \"isvc-sklearn-batcher-custom-predictor-5fccccdbd5-vc7st\" (UID: \"f7c7e394-4643-49ef-b2ca-dff32f41ff0b\") " pod="kserve-ci-e2e-test/isvc-sklearn-batcher-custom-predictor-5fccccdbd5-vc7st"
Jul 01 17:11:02.417376 ip-10-0-137-244 kubenswrapper[2565]: I0701 17:11:02.417357    2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/f7c7e394-4643-49ef-b2ca-dff32f41ff0b-proxy-tls\") pod \"isvc-sklearn-batcher-custom-predictor-5fccccdbd5-vc7st\" (UID: \"f7c7e394-4643-49ef-b2ca-dff32f41ff0b\") " pod="kserve-ci-e2e-test/isvc-sklearn-batcher-custom-predictor-5fccccdbd5-vc7st"
Jul 01 17:11:02.423222 ip-10-0-137-244 kubenswrapper[2565]: I0701 17:11:02.423205    2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-f8km5\" (UniqueName: \"kubernetes.io/projected/f7c7e394-4643-49ef-b2ca-dff32f41ff0b-kube-api-access-f8km5\") pod \"isvc-sklearn-batcher-custom-predictor-5fccccdbd5-vc7st\" (UID: \"f7c7e394-4643-49ef-b2ca-dff32f41ff0b\") " pod="kserve-ci-e2e-test/isvc-sklearn-batcher-custom-predictor-5fccccdbd5-vc7st"
Jul 01 17:11:02.555417 ip-10-0-137-244 kubenswrapper[2565]: I0701 17:11:02.555371    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/isvc-sklearn-batcher-custom-predictor-5fccccdbd5-vc7st"
Jul 01 17:11:02.690260 ip-10-0-137-244 kubenswrapper[2565]: I0701 17:11:02.690185    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/isvc-sklearn-batcher-custom-predictor-5fccccdbd5-vc7st"]
Jul 01 17:11:02.692414 ip-10-0-137-244 kubenswrapper[2565]: W0701 17:11:02.692382    2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf7c7e394_4643_49ef_b2ca_dff32f41ff0b.slice/crio-f86d273b9759c680b1f5272aa51c4ce4c055c7e8f28776d90dfa367ee5155298 WatchSource:0}: Error finding container f86d273b9759c680b1f5272aa51c4ce4c055c7e8f28776d90dfa367ee5155298: Status 404 returned error can't find the container with id f86d273b9759c680b1f5272aa51c4ce4c055c7e8f28776d90dfa367ee5155298
Jul 01 17:11:03.088563 ip-10-0-137-244 kubenswrapper[2565]: I0701 17:11:03.088529    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/isvc-sklearn-batcher-custom-predictor-5fccccdbd5-vc7st" event={"ID":"f7c7e394-4643-49ef-b2ca-dff32f41ff0b","Type":"ContainerStarted","Data":"3970272bd5f909591a101fa75c4f1770aaa5c343b5de21974506755a8e5c0bf3"}
Jul 01 17:11:03.088752 ip-10-0-137-244 kubenswrapper[2565]: I0701 17:11:03.088607    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/isvc-sklearn-batcher-custom-predictor-5fccccdbd5-vc7st" event={"ID":"f7c7e394-4643-49ef-b2ca-dff32f41ff0b","Type":"ContainerStarted","Data":"f86d273b9759c680b1f5272aa51c4ce4c055c7e8f28776d90dfa367ee5155298"}
Jul 01 17:11:03.090944 ip-10-0-137-244 kubenswrapper[2565]: I0701 17:11:03.090915    2565 generic.go:356] "Generic (PLEG): container finished" podID="98aa0b13-a938-471c-92d2-e0a9e84651b8" containerID="c5dbc0cce34af13c24a4c085ccb758a6d33fe5ab63508cae5049235235dff64e" exitCode=2
Jul 01 17:11:03.091066 ip-10-0-137-244 kubenswrapper[2565]: I0701 17:11:03.090972    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/isvc-sklearn-batcher-predictor-869f8b8c7d-mn6kq" event={"ID":"98aa0b13-a938-471c-92d2-e0a9e84651b8","Type":"ContainerDied","Data":"c5dbc0cce34af13c24a4c085ccb758a6d33fe5ab63508cae5049235235dff64e"}
Jul 01 17:11:06.743757 ip-10-0-137-244 kubenswrapper[2565]: I0701 17:11:06.743720    2565 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-sklearn-batcher-predictor-869f8b8c7d-mn6kq" podUID="98aa0b13-a938-471c-92d2-e0a9e84651b8" containerName="kube-rbac-proxy" probeResult="failure" output="Get \"https://10.133.0.25:8643/healthz\": dial tcp 10.133.0.25:8643: connect: connection refused"
Jul 01 17:11:07.110888 ip-10-0-137-244 kubenswrapper[2565]: I0701 17:11:07.110818    2565 generic.go:356] "Generic (PLEG): container finished" podID="98aa0b13-a938-471c-92d2-e0a9e84651b8" containerID="c7418a6c985834a77d12d17732fed7482a6a874be36679cd2c831a11ce861c16" exitCode=0
Jul 01 17:11:07.111014 ip-10-0-137-244 kubenswrapper[2565]: I0701 17:11:07.110895    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/isvc-sklearn-batcher-predictor-869f8b8c7d-mn6kq" event={"ID":"98aa0b13-a938-471c-92d2-e0a9e84651b8","Type":"ContainerDied","Data":"c7418a6c985834a77d12d17732fed7482a6a874be36679cd2c831a11ce861c16"}
Jul 01 17:11:07.112651 ip-10-0-137-244 kubenswrapper[2565]: I0701 17:11:07.112627    2565 generic.go:356] "Generic (PLEG): container finished" podID="f7c7e394-4643-49ef-b2ca-dff32f41ff0b" containerID="3970272bd5f909591a101fa75c4f1770aaa5c343b5de21974506755a8e5c0bf3" exitCode=0
Jul 01 17:11:07.112768 ip-10-0-137-244 kubenswrapper[2565]: I0701 17:11:07.112658    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/isvc-sklearn-batcher-custom-predictor-5fccccdbd5-vc7st" event={"ID":"f7c7e394-4643-49ef-b2ca-dff32f41ff0b","Type":"ContainerDied","Data":"3970272bd5f909591a101fa75c4f1770aaa5c343b5de21974506755a8e5c0bf3"}
Jul 01 17:11:07.748506 ip-10-0-137-244 kubenswrapper[2565]: I0701 17:11:07.748466    2565 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-sklearn-batcher-predictor-869f8b8c7d-mn6kq" podUID="98aa0b13-a938-471c-92d2-e0a9e84651b8" containerName="agent" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Jul 01 17:11:07.749702 ip-10-0-137-244 kubenswrapper[2565]: I0701 17:11:07.749675    2565 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-sklearn-batcher-predictor-869f8b8c7d-mn6kq" podUID="98aa0b13-a938-471c-92d2-e0a9e84651b8" containerName="kserve-container" probeResult="failure" output="dial tcp 10.133.0.25:8080: connect: connection refused"
Jul 01 17:11:08.125179 ip-10-0-137-244 kubenswrapper[2565]: I0701 17:11:08.125104    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/isvc-sklearn-batcher-custom-predictor-5fccccdbd5-vc7st" event={"ID":"f7c7e394-4643-49ef-b2ca-dff32f41ff0b","Type":"ContainerStarted","Data":"01fb7bcd7b746c3c88b2f2a8ce77b95f6412545f7c254e8e49b215621be9eb1c"}
Jul 01 17:11:08.125309 ip-10-0-137-244 kubenswrapper[2565]: I0701 17:11:08.125189    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/isvc-sklearn-batcher-custom-predictor-5fccccdbd5-vc7st" event={"ID":"f7c7e394-4643-49ef-b2ca-dff32f41ff0b","Type":"ContainerStarted","Data":"53061e4c83b2ba6d9281180383047f139956cfc553f8d527e47f57d8a5140448"}
Jul 01 17:11:08.125309 ip-10-0-137-244 kubenswrapper[2565]: I0701 17:11:08.125215    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/isvc-sklearn-batcher-custom-predictor-5fccccdbd5-vc7st" event={"ID":"f7c7e394-4643-49ef-b2ca-dff32f41ff0b","Type":"ContainerStarted","Data":"88fc28d775f07e47fdd4979091800fbc114c1aee0e1b5739d568a1d759187b17"}
Jul 01 17:11:08.125536 ip-10-0-137-244 kubenswrapper[2565]: I0701 17:11:08.125508    2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve-ci-e2e-test/isvc-sklearn-batcher-custom-predictor-5fccccdbd5-vc7st"
Jul 01 17:11:08.125675 ip-10-0-137-244 kubenswrapper[2565]: I0701 17:11:08.125656    2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve-ci-e2e-test/isvc-sklearn-batcher-custom-predictor-5fccccdbd5-vc7st"
Jul 01 17:11:08.127007 ip-10-0-137-244 kubenswrapper[2565]: I0701 17:11:08.126982    2565 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-sklearn-batcher-custom-predictor-5fccccdbd5-vc7st" podUID="f7c7e394-4643-49ef-b2ca-dff32f41ff0b" containerName="kserve-container" probeResult="failure" output="dial tcp 10.133.0.26:5000: connect: connection refused"
Jul 01 17:11:08.148877 ip-10-0-137-244 kubenswrapper[2565]: I0701 17:11:08.148831    2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kserve-ci-e2e-test/isvc-sklearn-batcher-custom-predictor-5fccccdbd5-vc7st" podStartSLOduration=2.548903773 podStartE2EDuration="6.148815823s" totalImagesPullingTime="0s" totalInitContainerRuntime="3.59991205s" isStatefulPod=false podCreationTimestamp="2026-07-01 17:11:02 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-01 17:11:08.147248971 +0000 UTC m=+738.615513095" watchObservedRunningTime="2026-07-01 17:11:08.148815823 +0000 UTC m=+738.617079947"
Jul 01 17:11:09.130175 ip-10-0-137-244 kubenswrapper[2565]: I0701 17:11:09.130138    2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve-ci-e2e-test/isvc-sklearn-batcher-custom-predictor-5fccccdbd5-vc7st"
Jul 01 17:11:09.130556 ip-10-0-137-244 kubenswrapper[2565]: I0701 17:11:09.130208    2565 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-sklearn-batcher-custom-predictor-5fccccdbd5-vc7st" podUID="f7c7e394-4643-49ef-b2ca-dff32f41ff0b" containerName="kserve-container" probeResult="failure" output="dial tcp 10.133.0.26:5000: connect: connection refused"
Jul 01 17:11:09.131347 ip-10-0-137-244 kubenswrapper[2565]: I0701 17:11:09.131311    2565 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-sklearn-batcher-custom-predictor-5fccccdbd5-vc7st" podUID="f7c7e394-4643-49ef-b2ca-dff32f41ff0b" containerName="agent" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Jul 01 17:11:10.134893 ip-10-0-137-244 kubenswrapper[2565]: I0701 17:11:10.134855    2565 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-sklearn-batcher-custom-predictor-5fccccdbd5-vc7st" podUID="f7c7e394-4643-49ef-b2ca-dff32f41ff0b" containerName="kserve-container" probeResult="failure" output="dial tcp 10.133.0.26:5000: connect: connection refused"
Jul 01 17:11:10.135362 ip-10-0-137-244 kubenswrapper[2565]: I0701 17:11:10.135236    2565 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-sklearn-batcher-custom-predictor-5fccccdbd5-vc7st" podUID="f7c7e394-4643-49ef-b2ca-dff32f41ff0b" containerName="agent" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Jul 01 17:11:11.743715 ip-10-0-137-244 kubenswrapper[2565]: I0701 17:11:11.743668    2565 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-sklearn-batcher-predictor-869f8b8c7d-mn6kq" podUID="98aa0b13-a938-471c-92d2-e0a9e84651b8" containerName="kube-rbac-proxy" probeResult="failure" output="Get \"https://10.133.0.25:8643/healthz\": dial tcp 10.133.0.25:8643: connect: connection refused"
Jul 01 17:11:15.139436 ip-10-0-137-244 kubenswrapper[2565]: I0701 17:11:15.139408    2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kserve-ci-e2e-test/isvc-sklearn-batcher-custom-predictor-5fccccdbd5-vc7st"
Jul 01 17:11:15.140070 ip-10-0-137-244 kubenswrapper[2565]: I0701 17:11:15.140014    2565 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-sklearn-batcher-custom-predictor-5fccccdbd5-vc7st" podUID="f7c7e394-4643-49ef-b2ca-dff32f41ff0b" containerName="kserve-container" probeResult="failure" output="dial tcp 10.133.0.26:5000: connect: connection refused"
Jul 01 17:11:15.140226 ip-10-0-137-244 kubenswrapper[2565]: I0701 17:11:15.140198    2565 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-sklearn-batcher-custom-predictor-5fccccdbd5-vc7st" podUID="f7c7e394-4643-49ef-b2ca-dff32f41ff0b" containerName="agent" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Jul 01 17:11:16.744584 ip-10-0-137-244 kubenswrapper[2565]: I0701 17:11:16.744548    2565 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-sklearn-batcher-predictor-869f8b8c7d-mn6kq" podUID="98aa0b13-a938-471c-92d2-e0a9e84651b8" containerName="kube-rbac-proxy" probeResult="failure" output="Get \"https://10.133.0.25:8643/healthz\": dial tcp 10.133.0.25:8643: connect: connection refused"
Jul 01 17:11:16.744984 ip-10-0-137-244 kubenswrapper[2565]: I0701 17:11:16.744667    2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve-ci-e2e-test/isvc-sklearn-batcher-predictor-869f8b8c7d-mn6kq"
Jul 01 17:11:17.748295 ip-10-0-137-244 kubenswrapper[2565]: I0701 17:11:17.748259    2565 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-sklearn-batcher-predictor-869f8b8c7d-mn6kq" podUID="98aa0b13-a938-471c-92d2-e0a9e84651b8" containerName="agent" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Jul 01 17:11:17.749498 ip-10-0-137-244 kubenswrapper[2565]: I0701 17:11:17.749473    2565 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-sklearn-batcher-predictor-869f8b8c7d-mn6kq" podUID="98aa0b13-a938-471c-92d2-e0a9e84651b8" containerName="kserve-container" probeResult="failure" output="dial tcp 10.133.0.25:8080: connect: connection refused"
Jul 01 17:11:21.744430 ip-10-0-137-244 kubenswrapper[2565]: I0701 17:11:21.744387    2565 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-sklearn-batcher-predictor-869f8b8c7d-mn6kq" podUID="98aa0b13-a938-471c-92d2-e0a9e84651b8" containerName="kube-rbac-proxy" probeResult="failure" output="Get \"https://10.133.0.25:8643/healthz\": dial tcp 10.133.0.25:8643: connect: connection refused"
Jul 01 17:11:25.139989 ip-10-0-137-244 kubenswrapper[2565]: I0701 17:11:25.139949    2565 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-sklearn-batcher-custom-predictor-5fccccdbd5-vc7st" podUID="f7c7e394-4643-49ef-b2ca-dff32f41ff0b" containerName="kserve-container" probeResult="failure" output="dial tcp 10.133.0.26:5000: connect: connection refused"
Jul 01 17:11:25.140521 ip-10-0-137-244 kubenswrapper[2565]: I0701 17:11:25.140489    2565 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-sklearn-batcher-custom-predictor-5fccccdbd5-vc7st" podUID="f7c7e394-4643-49ef-b2ca-dff32f41ff0b" containerName="agent" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Jul 01 17:11:26.743835 ip-10-0-137-244 kubenswrapper[2565]: I0701 17:11:26.743799    2565 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-sklearn-batcher-predictor-869f8b8c7d-mn6kq" podUID="98aa0b13-a938-471c-92d2-e0a9e84651b8" containerName="kube-rbac-proxy" probeResult="failure" output="Get \"https://10.133.0.25:8643/healthz\": dial tcp 10.133.0.25:8643: connect: connection refused"
Jul 01 17:11:27.748883 ip-10-0-137-244 kubenswrapper[2565]: I0701 17:11:27.748844    2565 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-sklearn-batcher-predictor-869f8b8c7d-mn6kq" podUID="98aa0b13-a938-471c-92d2-e0a9e84651b8" containerName="agent" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Jul 01 17:11:27.749308 ip-10-0-137-244 kubenswrapper[2565]: I0701 17:11:27.748970    2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve-ci-e2e-test/isvc-sklearn-batcher-predictor-869f8b8c7d-mn6kq"
Jul 01 17:11:27.749308 ip-10-0-137-244 kubenswrapper[2565]: I0701 17:11:27.749296    2565 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-sklearn-batcher-predictor-869f8b8c7d-mn6kq" podUID="98aa0b13-a938-471c-92d2-e0a9e84651b8" containerName="kserve-container" probeResult="failure" output="dial tcp 10.133.0.25:8080: connect: connection refused"
Jul 01 17:11:27.749405 ip-10-0-137-244 kubenswrapper[2565]: I0701 17:11:27.749394    2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve-ci-e2e-test/isvc-sklearn-batcher-predictor-869f8b8c7d-mn6kq"
Jul 01 17:11:31.744249 ip-10-0-137-244 kubenswrapper[2565]: I0701 17:11:31.744204    2565 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-sklearn-batcher-predictor-869f8b8c7d-mn6kq" podUID="98aa0b13-a938-471c-92d2-e0a9e84651b8" containerName="kube-rbac-proxy" probeResult="failure" output="Get \"https://10.133.0.25:8643/healthz\": dial tcp 10.133.0.25:8643: connect: connection refused"
Jul 01 17:11:32.363643 ip-10-0-137-244 kubenswrapper[2565]: I0701 17:11:32.363619    2565 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/isvc-sklearn-batcher-predictor-869f8b8c7d-mn6kq"
Jul 01 17:11:32.425627 ip-10-0-137-244 kubenswrapper[2565]: I0701 17:11:32.425599    2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/98aa0b13-a938-471c-92d2-e0a9e84651b8-proxy-tls\") pod \"98aa0b13-a938-471c-92d2-e0a9e84651b8\" (UID: \"98aa0b13-a938-471c-92d2-e0a9e84651b8\") "
Jul 01 17:11:32.425755 ip-10-0-137-244 kubenswrapper[2565]: I0701 17:11:32.425645    2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"cabundle-cert\" (UniqueName: \"kubernetes.io/configmap/98aa0b13-a938-471c-92d2-e0a9e84651b8-cabundle-cert\") pod \"98aa0b13-a938-471c-92d2-e0a9e84651b8\" (UID: \"98aa0b13-a938-471c-92d2-e0a9e84651b8\") "
Jul 01 17:11:32.425755 ip-10-0-137-244 kubenswrapper[2565]: I0701 17:11:32.425685    2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-n452k\" (UniqueName: \"kubernetes.io/projected/98aa0b13-a938-471c-92d2-e0a9e84651b8-kube-api-access-n452k\") pod \"98aa0b13-a938-471c-92d2-e0a9e84651b8\" (UID: \"98aa0b13-a938-471c-92d2-e0a9e84651b8\") "
Jul 01 17:11:32.425755 ip-10-0-137-244 kubenswrapper[2565]: I0701 17:11:32.425718    2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/98aa0b13-a938-471c-92d2-e0a9e84651b8-kube-rbac-proxy-sar-config\") pod \"98aa0b13-a938-471c-92d2-e0a9e84651b8\" (UID: \"98aa0b13-a938-471c-92d2-e0a9e84651b8\") "
Jul 01 17:11:32.425912 ip-10-0-137-244 kubenswrapper[2565]: I0701 17:11:32.425798    2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/98aa0b13-a938-471c-92d2-e0a9e84651b8-kserve-provision-location\") pod \"98aa0b13-a938-471c-92d2-e0a9e84651b8\" (UID: \"98aa0b13-a938-471c-92d2-e0a9e84651b8\") "
Jul 01 17:11:32.426149 ip-10-0-137-244 kubenswrapper[2565]: I0701 17:11:32.426128    2565 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/98aa0b13-a938-471c-92d2-e0a9e84651b8-cabundle-cert" (OuterVolumeSpecName: "cabundle-cert") pod "98aa0b13-a938-471c-92d2-e0a9e84651b8" (UID: "98aa0b13-a938-471c-92d2-e0a9e84651b8"). InnerVolumeSpecName "cabundle-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 01 17:11:32.426247 ip-10-0-137-244 kubenswrapper[2565]: I0701 17:11:32.426148    2565 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/98aa0b13-a938-471c-92d2-e0a9e84651b8-kserve-provision-location" (OuterVolumeSpecName: "kserve-provision-location") pod "98aa0b13-a938-471c-92d2-e0a9e84651b8" (UID: "98aa0b13-a938-471c-92d2-e0a9e84651b8"). InnerVolumeSpecName "kserve-provision-location". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 01 17:11:32.426247 ip-10-0-137-244 kubenswrapper[2565]: I0701 17:11:32.426148    2565 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/98aa0b13-a938-471c-92d2-e0a9e84651b8-kube-rbac-proxy-sar-config" (OuterVolumeSpecName: "kube-rbac-proxy-sar-config") pod "98aa0b13-a938-471c-92d2-e0a9e84651b8" (UID: "98aa0b13-a938-471c-92d2-e0a9e84651b8"). InnerVolumeSpecName "kube-rbac-proxy-sar-config". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 01 17:11:32.427798 ip-10-0-137-244 kubenswrapper[2565]: I0701 17:11:32.427777    2565 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/98aa0b13-a938-471c-92d2-e0a9e84651b8-kube-api-access-n452k" (OuterVolumeSpecName: "kube-api-access-n452k") pod "98aa0b13-a938-471c-92d2-e0a9e84651b8" (UID: "98aa0b13-a938-471c-92d2-e0a9e84651b8"). InnerVolumeSpecName "kube-api-access-n452k". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 01 17:11:32.427918 ip-10-0-137-244 kubenswrapper[2565]: I0701 17:11:32.427897    2565 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/98aa0b13-a938-471c-92d2-e0a9e84651b8-proxy-tls" (OuterVolumeSpecName: "proxy-tls") pod "98aa0b13-a938-471c-92d2-e0a9e84651b8" (UID: "98aa0b13-a938-471c-92d2-e0a9e84651b8"). InnerVolumeSpecName "proxy-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 01 17:11:32.526266 ip-10-0-137-244 kubenswrapper[2565]: I0701 17:11:32.526211    2565 reconciler_common.go:299] "Volume detached for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/98aa0b13-a938-471c-92d2-e0a9e84651b8-kserve-provision-location\") on node \"ip-10-0-137-244.ec2.internal\" DevicePath \"\""
Jul 01 17:11:32.526266 ip-10-0-137-244 kubenswrapper[2565]: I0701 17:11:32.526236    2565 reconciler_common.go:299] "Volume detached for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/98aa0b13-a938-471c-92d2-e0a9e84651b8-proxy-tls\") on node \"ip-10-0-137-244.ec2.internal\" DevicePath \"\""
Jul 01 17:11:32.526266 ip-10-0-137-244 kubenswrapper[2565]: I0701 17:11:32.526248    2565 reconciler_common.go:299] "Volume detached for volume \"cabundle-cert\" (UniqueName: \"kubernetes.io/configmap/98aa0b13-a938-471c-92d2-e0a9e84651b8-cabundle-cert\") on node \"ip-10-0-137-244.ec2.internal\" DevicePath \"\""
Jul 01 17:11:32.526266 ip-10-0-137-244 kubenswrapper[2565]: I0701 17:11:32.526260    2565 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-n452k\" (UniqueName: \"kubernetes.io/projected/98aa0b13-a938-471c-92d2-e0a9e84651b8-kube-api-access-n452k\") on node \"ip-10-0-137-244.ec2.internal\" DevicePath \"\""
Jul 01 17:11:32.526436 ip-10-0-137-244 kubenswrapper[2565]: I0701 17:11:32.526274    2565 reconciler_common.go:299] "Volume detached for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/98aa0b13-a938-471c-92d2-e0a9e84651b8-kube-rbac-proxy-sar-config\") on node \"ip-10-0-137-244.ec2.internal\" DevicePath \"\""
Jul 01 17:11:33.236637 ip-10-0-137-244 kubenswrapper[2565]: I0701 17:11:33.236604    2565 generic.go:356] "Generic (PLEG): container finished" podID="98aa0b13-a938-471c-92d2-e0a9e84651b8" containerID="3ae071519da1e893dbced83b5dcc049ed9a9f8fcdfabe21102ef59f28500b4ef" exitCode=0
Jul 01 17:11:33.236996 ip-10-0-137-244 kubenswrapper[2565]: I0701 17:11:33.236686    2565 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/isvc-sklearn-batcher-predictor-869f8b8c7d-mn6kq"
Jul 01 17:11:33.236996 ip-10-0-137-244 kubenswrapper[2565]: I0701 17:11:33.236691    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/isvc-sklearn-batcher-predictor-869f8b8c7d-mn6kq" event={"ID":"98aa0b13-a938-471c-92d2-e0a9e84651b8","Type":"ContainerDied","Data":"3ae071519da1e893dbced83b5dcc049ed9a9f8fcdfabe21102ef59f28500b4ef"}
Jul 01 17:11:33.236996 ip-10-0-137-244 kubenswrapper[2565]: I0701 17:11:33.236760    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/isvc-sklearn-batcher-predictor-869f8b8c7d-mn6kq" event={"ID":"98aa0b13-a938-471c-92d2-e0a9e84651b8","Type":"ContainerDied","Data":"a5f82679783cbf708484665136729845fecfddebbb69f1494579db87bfe35aa7"}
Jul 01 17:11:33.236996 ip-10-0-137-244 kubenswrapper[2565]: I0701 17:11:33.236794    2565 scope.go:117] "RemoveContainer" containerID="3ae071519da1e893dbced83b5dcc049ed9a9f8fcdfabe21102ef59f28500b4ef"
Jul 01 17:11:33.245855 ip-10-0-137-244 kubenswrapper[2565]: I0701 17:11:33.245840    2565 scope.go:117] "RemoveContainer" containerID="c5dbc0cce34af13c24a4c085ccb758a6d33fe5ab63508cae5049235235dff64e"
Jul 01 17:11:33.255365 ip-10-0-137-244 kubenswrapper[2565]: I0701 17:11:33.255350    2565 scope.go:117] "RemoveContainer" containerID="c7418a6c985834a77d12d17732fed7482a6a874be36679cd2c831a11ce861c16"
Jul 01 17:11:33.261177 ip-10-0-137-244 kubenswrapper[2565]: I0701 17:11:33.261156    2565 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kserve-ci-e2e-test/isvc-sklearn-batcher-predictor-869f8b8c7d-mn6kq"]
Jul 01 17:11:33.263238 ip-10-0-137-244 kubenswrapper[2565]: I0701 17:11:33.263221    2565 scope.go:117] "RemoveContainer" containerID="b172891e59532a372ceb1ce766ce0d29ca741a7cf3187e2181b5b98a650d446d"
Jul 01 17:11:33.266378 ip-10-0-137-244 kubenswrapper[2565]: I0701 17:11:33.266358    2565 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kserve-ci-e2e-test/isvc-sklearn-batcher-predictor-869f8b8c7d-mn6kq"]
Jul 01 17:11:33.270397 ip-10-0-137-244 kubenswrapper[2565]: I0701 17:11:33.270381    2565 scope.go:117] "RemoveContainer" containerID="3ae071519da1e893dbced83b5dcc049ed9a9f8fcdfabe21102ef59f28500b4ef"
Jul 01 17:11:33.270660 ip-10-0-137-244 kubenswrapper[2565]: E0701 17:11:33.270641    2565 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"3ae071519da1e893dbced83b5dcc049ed9a9f8fcdfabe21102ef59f28500b4ef\": container with ID starting with 3ae071519da1e893dbced83b5dcc049ed9a9f8fcdfabe21102ef59f28500b4ef not found: ID does not exist" containerID="3ae071519da1e893dbced83b5dcc049ed9a9f8fcdfabe21102ef59f28500b4ef"
Jul 01 17:11:33.270710 ip-10-0-137-244 kubenswrapper[2565]: I0701 17:11:33.270667    2565 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"3ae071519da1e893dbced83b5dcc049ed9a9f8fcdfabe21102ef59f28500b4ef"} err="failed to get container status \"3ae071519da1e893dbced83b5dcc049ed9a9f8fcdfabe21102ef59f28500b4ef\": rpc error: code = NotFound desc = could not find container \"3ae071519da1e893dbced83b5dcc049ed9a9f8fcdfabe21102ef59f28500b4ef\": container with ID starting with 3ae071519da1e893dbced83b5dcc049ed9a9f8fcdfabe21102ef59f28500b4ef not found: ID does not exist"
Jul 01 17:11:33.270710 ip-10-0-137-244 kubenswrapper[2565]: I0701 17:11:33.270685    2565 scope.go:117] "RemoveContainer" containerID="c5dbc0cce34af13c24a4c085ccb758a6d33fe5ab63508cae5049235235dff64e"
Jul 01 17:11:33.270898 ip-10-0-137-244 kubenswrapper[2565]: E0701 17:11:33.270883    2565 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"c5dbc0cce34af13c24a4c085ccb758a6d33fe5ab63508cae5049235235dff64e\": container with ID starting with c5dbc0cce34af13c24a4c085ccb758a6d33fe5ab63508cae5049235235dff64e not found: ID does not exist" containerID="c5dbc0cce34af13c24a4c085ccb758a6d33fe5ab63508cae5049235235dff64e"
Jul 01 17:11:33.270939 ip-10-0-137-244 kubenswrapper[2565]: I0701 17:11:33.270902    2565 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"c5dbc0cce34af13c24a4c085ccb758a6d33fe5ab63508cae5049235235dff64e"} err="failed to get container status \"c5dbc0cce34af13c24a4c085ccb758a6d33fe5ab63508cae5049235235dff64e\": rpc error: code = NotFound desc = could not find container \"c5dbc0cce34af13c24a4c085ccb758a6d33fe5ab63508cae5049235235dff64e\": container with ID starting with c5dbc0cce34af13c24a4c085ccb758a6d33fe5ab63508cae5049235235dff64e not found: ID does not exist"
Jul 01 17:11:33.270939 ip-10-0-137-244 kubenswrapper[2565]: I0701 17:11:33.270916    2565 scope.go:117] "RemoveContainer" containerID="c7418a6c985834a77d12d17732fed7482a6a874be36679cd2c831a11ce861c16"
Jul 01 17:11:33.271136 ip-10-0-137-244 kubenswrapper[2565]: E0701 17:11:33.271116    2565 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"c7418a6c985834a77d12d17732fed7482a6a874be36679cd2c831a11ce861c16\": container with ID starting with c7418a6c985834a77d12d17732fed7482a6a874be36679cd2c831a11ce861c16 not found: ID does not exist" containerID="c7418a6c985834a77d12d17732fed7482a6a874be36679cd2c831a11ce861c16"
Jul 01 17:11:33.271185 ip-10-0-137-244 kubenswrapper[2565]: I0701 17:11:33.271140    2565 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"c7418a6c985834a77d12d17732fed7482a6a874be36679cd2c831a11ce861c16"} err="failed to get container status \"c7418a6c985834a77d12d17732fed7482a6a874be36679cd2c831a11ce861c16\": rpc error: code = NotFound desc = could not find container \"c7418a6c985834a77d12d17732fed7482a6a874be36679cd2c831a11ce861c16\": container with ID starting with c7418a6c985834a77d12d17732fed7482a6a874be36679cd2c831a11ce861c16 not found: ID does not exist"
Jul 01 17:11:33.271185 ip-10-0-137-244 kubenswrapper[2565]: I0701 17:11:33.271152    2565 scope.go:117] "RemoveContainer" containerID="b172891e59532a372ceb1ce766ce0d29ca741a7cf3187e2181b5b98a650d446d"
Jul 01 17:11:33.271392 ip-10-0-137-244 kubenswrapper[2565]: E0701 17:11:33.271376    2565 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"b172891e59532a372ceb1ce766ce0d29ca741a7cf3187e2181b5b98a650d446d\": container with ID starting with b172891e59532a372ceb1ce766ce0d29ca741a7cf3187e2181b5b98a650d446d not found: ID does not exist" containerID="b172891e59532a372ceb1ce766ce0d29ca741a7cf3187e2181b5b98a650d446d"
Jul 01 17:11:33.271448 ip-10-0-137-244 kubenswrapper[2565]: I0701 17:11:33.271394    2565 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"b172891e59532a372ceb1ce766ce0d29ca741a7cf3187e2181b5b98a650d446d"} err="failed to get container status \"b172891e59532a372ceb1ce766ce0d29ca741a7cf3187e2181b5b98a650d446d\": rpc error: code = NotFound desc = could not find container \"b172891e59532a372ceb1ce766ce0d29ca741a7cf3187e2181b5b98a650d446d\": container with ID starting with b172891e59532a372ceb1ce766ce0d29ca741a7cf3187e2181b5b98a650d446d not found: ID does not exist"
Jul 01 17:11:34.247272 ip-10-0-137-244 kubenswrapper[2565]: I0701 17:11:34.247244    2565 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="98aa0b13-a938-471c-92d2-e0a9e84651b8" path="/var/lib/kubelet/pods/98aa0b13-a938-471c-92d2-e0a9e84651b8/volumes"
Jul 01 17:11:35.139862 ip-10-0-137-244 kubenswrapper[2565]: I0701 17:11:35.139828    2565 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-sklearn-batcher-custom-predictor-5fccccdbd5-vc7st" podUID="f7c7e394-4643-49ef-b2ca-dff32f41ff0b" containerName="kserve-container" probeResult="failure" output="dial tcp 10.133.0.26:5000: connect: connection refused"
Jul 01 17:11:35.140378 ip-10-0-137-244 kubenswrapper[2565]: I0701 17:11:35.140353    2565 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-sklearn-batcher-custom-predictor-5fccccdbd5-vc7st" podUID="f7c7e394-4643-49ef-b2ca-dff32f41ff0b" containerName="agent" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Jul 01 17:11:45.140066 ip-10-0-137-244 kubenswrapper[2565]: I0701 17:11:45.140003    2565 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-sklearn-batcher-custom-predictor-5fccccdbd5-vc7st" podUID="f7c7e394-4643-49ef-b2ca-dff32f41ff0b" containerName="kserve-container" probeResult="failure" output="dial tcp 10.133.0.26:5000: connect: connection refused"
Jul 01 17:11:45.140549 ip-10-0-137-244 kubenswrapper[2565]: I0701 17:11:45.140452    2565 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-sklearn-batcher-custom-predictor-5fccccdbd5-vc7st" podUID="f7c7e394-4643-49ef-b2ca-dff32f41ff0b" containerName="agent" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Jul 01 17:11:55.139895 ip-10-0-137-244 kubenswrapper[2565]: I0701 17:11:55.139848    2565 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-sklearn-batcher-custom-predictor-5fccccdbd5-vc7st" podUID="f7c7e394-4643-49ef-b2ca-dff32f41ff0b" containerName="kserve-container" probeResult="failure" output="dial tcp 10.133.0.26:5000: connect: connection refused"
Jul 01 17:11:55.140381 ip-10-0-137-244 kubenswrapper[2565]: I0701 17:11:55.140356    2565 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-sklearn-batcher-custom-predictor-5fccccdbd5-vc7st" podUID="f7c7e394-4643-49ef-b2ca-dff32f41ff0b" containerName="agent" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Jul 01 17:12:05.139875 ip-10-0-137-244 kubenswrapper[2565]: I0701 17:12:05.139836    2565 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-sklearn-batcher-custom-predictor-5fccccdbd5-vc7st" podUID="f7c7e394-4643-49ef-b2ca-dff32f41ff0b" containerName="kserve-container" probeResult="failure" output="dial tcp 10.133.0.26:5000: connect: connection refused"
Jul 01 17:12:05.140420 ip-10-0-137-244 kubenswrapper[2565]: I0701 17:12:05.140381    2565 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-sklearn-batcher-custom-predictor-5fccccdbd5-vc7st" podUID="f7c7e394-4643-49ef-b2ca-dff32f41ff0b" containerName="agent" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Jul 01 17:12:15.140624 ip-10-0-137-244 kubenswrapper[2565]: I0701 17:12:15.140510    2565 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-sklearn-batcher-custom-predictor-5fccccdbd5-vc7st" podUID="f7c7e394-4643-49ef-b2ca-dff32f41ff0b" containerName="kserve-container" probeResult="failure" output="dial tcp 10.133.0.26:5000: connect: connection refused"
Jul 01 17:12:15.141080 ip-10-0-137-244 kubenswrapper[2565]: I0701 17:12:15.141055    2565 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-sklearn-batcher-custom-predictor-5fccccdbd5-vc7st" podUID="f7c7e394-4643-49ef-b2ca-dff32f41ff0b" containerName="agent" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Jul 01 17:12:16.248161 ip-10-0-137-244 kubenswrapper[2565]: I0701 17:12:16.248134    2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kserve-ci-e2e-test/isvc-sklearn-batcher-custom-predictor-5fccccdbd5-vc7st"
Jul 01 17:12:16.248499 ip-10-0-137-244 kubenswrapper[2565]: I0701 17:12:16.248231    2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kserve-ci-e2e-test/isvc-sklearn-batcher-custom-predictor-5fccccdbd5-vc7st"
Jul 01 17:12:27.299121 ip-10-0-137-244 kubenswrapper[2565]: I0701 17:12:27.299077    2565 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kserve-ci-e2e-test/isvc-sklearn-batcher-custom-predictor-5fccccdbd5-vc7st"]
Jul 01 17:12:27.299581 ip-10-0-137-244 kubenswrapper[2565]: I0701 17:12:27.299374    2565 kuberuntime_container.go:872] "Killing container with a grace period" pod="kserve-ci-e2e-test/isvc-sklearn-batcher-custom-predictor-5fccccdbd5-vc7st" podUID="f7c7e394-4643-49ef-b2ca-dff32f41ff0b" containerName="kserve-container" containerID="cri-o://88fc28d775f07e47fdd4979091800fbc114c1aee0e1b5739d568a1d759187b17" gracePeriod=30
Jul 01 17:12:27.299581 ip-10-0-137-244 kubenswrapper[2565]: I0701 17:12:27.299407    2565 kuberuntime_container.go:872] "Killing container with a grace period" pod="kserve-ci-e2e-test/isvc-sklearn-batcher-custom-predictor-5fccccdbd5-vc7st" podUID="f7c7e394-4643-49ef-b2ca-dff32f41ff0b" containerName="agent" containerID="cri-o://01fb7bcd7b746c3c88b2f2a8ce77b95f6412545f7c254e8e49b215621be9eb1c" gracePeriod=30
Jul 01 17:12:27.299581 ip-10-0-137-244 kubenswrapper[2565]: I0701 17:12:27.299429    2565 kuberuntime_container.go:872] "Killing container with a grace period" pod="kserve-ci-e2e-test/isvc-sklearn-batcher-custom-predictor-5fccccdbd5-vc7st" podUID="f7c7e394-4643-49ef-b2ca-dff32f41ff0b" containerName="kube-rbac-proxy" containerID="cri-o://53061e4c83b2ba6d9281180383047f139956cfc553f8d527e47f57d8a5140448" gracePeriod=30
Jul 01 17:12:27.343876 ip-10-0-137-244 kubenswrapper[2565]: I0701 17:12:27.343852    2565 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kserve-ci-e2e-test/message-dumper-predictor-6575bcfdfd-mn29k"]
Jul 01 17:12:27.344163 ip-10-0-137-244 kubenswrapper[2565]: I0701 17:12:27.344150    2565 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="98aa0b13-a938-471c-92d2-e0a9e84651b8" containerName="storage-initializer"
Jul 01 17:12:27.344217 ip-10-0-137-244 kubenswrapper[2565]: I0701 17:12:27.344164    2565 state_mem.go:107] "Deleted CPUSet assignment" podUID="98aa0b13-a938-471c-92d2-e0a9e84651b8" containerName="storage-initializer"
Jul 01 17:12:27.344217 ip-10-0-137-244 kubenswrapper[2565]: I0701 17:12:27.344176    2565 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="98aa0b13-a938-471c-92d2-e0a9e84651b8" containerName="agent"
Jul 01 17:12:27.344217 ip-10-0-137-244 kubenswrapper[2565]: I0701 17:12:27.344182    2565 state_mem.go:107] "Deleted CPUSet assignment" podUID="98aa0b13-a938-471c-92d2-e0a9e84651b8" containerName="agent"
Jul 01 17:12:27.344217 ip-10-0-137-244 kubenswrapper[2565]: I0701 17:12:27.344189    2565 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="98aa0b13-a938-471c-92d2-e0a9e84651b8" containerName="kserve-container"
Jul 01 17:12:27.344217 ip-10-0-137-244 kubenswrapper[2565]: I0701 17:12:27.344196    2565 state_mem.go:107] "Deleted CPUSet assignment" podUID="98aa0b13-a938-471c-92d2-e0a9e84651b8" containerName="kserve-container"
Jul 01 17:12:27.344217 ip-10-0-137-244 kubenswrapper[2565]: I0701 17:12:27.344202    2565 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="98aa0b13-a938-471c-92d2-e0a9e84651b8" containerName="kube-rbac-proxy"
Jul 01 17:12:27.344217 ip-10-0-137-244 kubenswrapper[2565]: I0701 17:12:27.344207    2565 state_mem.go:107] "Deleted CPUSet assignment" podUID="98aa0b13-a938-471c-92d2-e0a9e84651b8" containerName="kube-rbac-proxy"
Jul 01 17:12:27.344433 ip-10-0-137-244 kubenswrapper[2565]: I0701 17:12:27.344257    2565 memory_manager.go:368] "RemoveStaleState removing state" podUID="98aa0b13-a938-471c-92d2-e0a9e84651b8" containerName="kube-rbac-proxy"
Jul 01 17:12:27.344433 ip-10-0-137-244 kubenswrapper[2565]: I0701 17:12:27.344263    2565 memory_manager.go:368] "RemoveStaleState removing state" podUID="98aa0b13-a938-471c-92d2-e0a9e84651b8" containerName="agent"
Jul 01 17:12:27.344433 ip-10-0-137-244 kubenswrapper[2565]: I0701 17:12:27.344270    2565 memory_manager.go:368] "RemoveStaleState removing state" podUID="98aa0b13-a938-471c-92d2-e0a9e84651b8" containerName="kserve-container"
Jul 01 17:12:27.346908 ip-10-0-137-244 kubenswrapper[2565]: I0701 17:12:27.346886    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/message-dumper-predictor-6575bcfdfd-mn29k"
Jul 01 17:12:27.349182 ip-10-0-137-244 kubenswrapper[2565]: I0701 17:12:27.349163    2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kserve-ci-e2e-test\"/\"message-dumper-kube-rbac-proxy-sar-config\""
Jul 01 17:12:27.349287 ip-10-0-137-244 kubenswrapper[2565]: I0701 17:12:27.349165    2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kserve-ci-e2e-test\"/\"message-dumper-predictor-serving-cert\""
Jul 01 17:12:27.360540 ip-10-0-137-244 kubenswrapper[2565]: I0701 17:12:27.360521    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/message-dumper-predictor-6575bcfdfd-mn29k"]
Jul 01 17:12:27.452432 ip-10-0-137-244 kubenswrapper[2565]: I0701 17:12:27.452403    2565 generic.go:356] "Generic (PLEG): container finished" podID="f7c7e394-4643-49ef-b2ca-dff32f41ff0b" containerID="53061e4c83b2ba6d9281180383047f139956cfc553f8d527e47f57d8a5140448" exitCode=2
Jul 01 17:12:27.452536 ip-10-0-137-244 kubenswrapper[2565]: I0701 17:12:27.452438    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/isvc-sklearn-batcher-custom-predictor-5fccccdbd5-vc7st" event={"ID":"f7c7e394-4643-49ef-b2ca-dff32f41ff0b","Type":"ContainerDied","Data":"53061e4c83b2ba6d9281180383047f139956cfc553f8d527e47f57d8a5140448"}
Jul 01 17:12:27.490867 ip-10-0-137-244 kubenswrapper[2565]: I0701 17:12:27.490841    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/3c3bda14-25ec-4abd-b396-4e783e462da4-kube-rbac-proxy-sar-config\") pod \"message-dumper-predictor-6575bcfdfd-mn29k\" (UID: \"3c3bda14-25ec-4abd-b396-4e783e462da4\") " pod="kserve-ci-e2e-test/message-dumper-predictor-6575bcfdfd-mn29k"
Jul 01 17:12:27.491003 ip-10-0-137-244 kubenswrapper[2565]: I0701 17:12:27.490913    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/3c3bda14-25ec-4abd-b396-4e783e462da4-proxy-tls\") pod \"message-dumper-predictor-6575bcfdfd-mn29k\" (UID: \"3c3bda14-25ec-4abd-b396-4e783e462da4\") " pod="kserve-ci-e2e-test/message-dumper-predictor-6575bcfdfd-mn29k"
Jul 01 17:12:27.491003 ip-10-0-137-244 kubenswrapper[2565]: I0701 17:12:27.490952    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ncpxn\" (UniqueName: \"kubernetes.io/projected/3c3bda14-25ec-4abd-b396-4e783e462da4-kube-api-access-ncpxn\") pod \"message-dumper-predictor-6575bcfdfd-mn29k\" (UID: \"3c3bda14-25ec-4abd-b396-4e783e462da4\") " pod="kserve-ci-e2e-test/message-dumper-predictor-6575bcfdfd-mn29k"
Jul 01 17:12:27.591856 ip-10-0-137-244 kubenswrapper[2565]: I0701 17:12:27.591802    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/3c3bda14-25ec-4abd-b396-4e783e462da4-proxy-tls\") pod \"message-dumper-predictor-6575bcfdfd-mn29k\" (UID: \"3c3bda14-25ec-4abd-b396-4e783e462da4\") " pod="kserve-ci-e2e-test/message-dumper-predictor-6575bcfdfd-mn29k"
Jul 01 17:12:27.591856 ip-10-0-137-244 kubenswrapper[2565]: I0701 17:12:27.591834    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ncpxn\" (UniqueName: \"kubernetes.io/projected/3c3bda14-25ec-4abd-b396-4e783e462da4-kube-api-access-ncpxn\") pod \"message-dumper-predictor-6575bcfdfd-mn29k\" (UID: \"3c3bda14-25ec-4abd-b396-4e783e462da4\") " pod="kserve-ci-e2e-test/message-dumper-predictor-6575bcfdfd-mn29k"
Jul 01 17:12:27.591999 ip-10-0-137-244 kubenswrapper[2565]: I0701 17:12:27.591862    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/3c3bda14-25ec-4abd-b396-4e783e462da4-kube-rbac-proxy-sar-config\") pod \"message-dumper-predictor-6575bcfdfd-mn29k\" (UID: \"3c3bda14-25ec-4abd-b396-4e783e462da4\") " pod="kserve-ci-e2e-test/message-dumper-predictor-6575bcfdfd-mn29k"
Jul 01 17:12:27.591999 ip-10-0-137-244 kubenswrapper[2565]: E0701 17:12:27.591947    2565 secret.go:189] Couldn't get secret kserve-ci-e2e-test/message-dumper-predictor-serving-cert: secret "message-dumper-predictor-serving-cert" not found
Jul 01 17:12:27.592105 ip-10-0-137-244 kubenswrapper[2565]: E0701 17:12:27.592011    2565 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/3c3bda14-25ec-4abd-b396-4e783e462da4-proxy-tls podName:3c3bda14-25ec-4abd-b396-4e783e462da4 nodeName:}" failed. No retries permitted until 2026-07-01 17:12:28.091993843 +0000 UTC m=+818.560257945 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "proxy-tls" (UniqueName: "kubernetes.io/secret/3c3bda14-25ec-4abd-b396-4e783e462da4-proxy-tls") pod "message-dumper-predictor-6575bcfdfd-mn29k" (UID: "3c3bda14-25ec-4abd-b396-4e783e462da4") : secret "message-dumper-predictor-serving-cert" not found
Jul 01 17:12:27.592498 ip-10-0-137-244 kubenswrapper[2565]: I0701 17:12:27.592482    2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/3c3bda14-25ec-4abd-b396-4e783e462da4-kube-rbac-proxy-sar-config\") pod \"message-dumper-predictor-6575bcfdfd-mn29k\" (UID: \"3c3bda14-25ec-4abd-b396-4e783e462da4\") " pod="kserve-ci-e2e-test/message-dumper-predictor-6575bcfdfd-mn29k"
Jul 01 17:12:27.600982 ip-10-0-137-244 kubenswrapper[2565]: I0701 17:12:27.600963    2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-ncpxn\" (UniqueName: \"kubernetes.io/projected/3c3bda14-25ec-4abd-b396-4e783e462da4-kube-api-access-ncpxn\") pod \"message-dumper-predictor-6575bcfdfd-mn29k\" (UID: \"3c3bda14-25ec-4abd-b396-4e783e462da4\") " pod="kserve-ci-e2e-test/message-dumper-predictor-6575bcfdfd-mn29k"
Jul 01 17:12:28.095796 ip-10-0-137-244 kubenswrapper[2565]: I0701 17:12:28.095763    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/3c3bda14-25ec-4abd-b396-4e783e462da4-proxy-tls\") pod \"message-dumper-predictor-6575bcfdfd-mn29k\" (UID: \"3c3bda14-25ec-4abd-b396-4e783e462da4\") " pod="kserve-ci-e2e-test/message-dumper-predictor-6575bcfdfd-mn29k"
Jul 01 17:12:28.098167 ip-10-0-137-244 kubenswrapper[2565]: I0701 17:12:28.098144    2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/3c3bda14-25ec-4abd-b396-4e783e462da4-proxy-tls\") pod \"message-dumper-predictor-6575bcfdfd-mn29k\" (UID: \"3c3bda14-25ec-4abd-b396-4e783e462da4\") " pod="kserve-ci-e2e-test/message-dumper-predictor-6575bcfdfd-mn29k"
Jul 01 17:12:28.257013 ip-10-0-137-244 kubenswrapper[2565]: I0701 17:12:28.256989    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/message-dumper-predictor-6575bcfdfd-mn29k"
Jul 01 17:12:28.387467 ip-10-0-137-244 kubenswrapper[2565]: I0701 17:12:28.387445    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/message-dumper-predictor-6575bcfdfd-mn29k"]
Jul 01 17:12:28.390522 ip-10-0-137-244 kubenswrapper[2565]: W0701 17:12:28.390494    2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3c3bda14_25ec_4abd_b396_4e783e462da4.slice/crio-423d733845660b75159fa7469292a7e45353d89ceaa643b6d69a783b8f3d57d4 WatchSource:0}: Error finding container 423d733845660b75159fa7469292a7e45353d89ceaa643b6d69a783b8f3d57d4: Status 404 returned error can't find the container with id 423d733845660b75159fa7469292a7e45353d89ceaa643b6d69a783b8f3d57d4
Jul 01 17:12:28.392570 ip-10-0-137-244 kubenswrapper[2565]: I0701 17:12:28.392552    2565 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Jul 01 17:12:28.457997 ip-10-0-137-244 kubenswrapper[2565]: I0701 17:12:28.457972    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/message-dumper-predictor-6575bcfdfd-mn29k" event={"ID":"3c3bda14-25ec-4abd-b396-4e783e462da4","Type":"ContainerStarted","Data":"423d733845660b75159fa7469292a7e45353d89ceaa643b6d69a783b8f3d57d4"}
Jul 01 17:12:30.135389 ip-10-0-137-244 kubenswrapper[2565]: I0701 17:12:30.135348    2565 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-sklearn-batcher-custom-predictor-5fccccdbd5-vc7st" podUID="f7c7e394-4643-49ef-b2ca-dff32f41ff0b" containerName="kube-rbac-proxy" probeResult="failure" output="Get \"https://10.133.0.26:8643/healthz\": dial tcp 10.133.0.26:8643: connect: connection refused"
Jul 01 17:12:30.468375 ip-10-0-137-244 kubenswrapper[2565]: I0701 17:12:30.468281    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/message-dumper-predictor-6575bcfdfd-mn29k" event={"ID":"3c3bda14-25ec-4abd-b396-4e783e462da4","Type":"ContainerStarted","Data":"d55a29db64b82f2a8c900b2fe4d88b33943d7ea7ea0cff203a6582c6cee23546"}
Jul 01 17:12:30.468375 ip-10-0-137-244 kubenswrapper[2565]: I0701 17:12:30.468346    2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve-ci-e2e-test/message-dumper-predictor-6575bcfdfd-mn29k"
Jul 01 17:12:30.468375 ip-10-0-137-244 kubenswrapper[2565]: I0701 17:12:30.468370    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/message-dumper-predictor-6575bcfdfd-mn29k" event={"ID":"3c3bda14-25ec-4abd-b396-4e783e462da4","Type":"ContainerStarted","Data":"03acf446bfc0051f0c558339de9f6b35ee1507314036de97697e10a5bca1480e"}
Jul 01 17:12:30.468607 ip-10-0-137-244 kubenswrapper[2565]: I0701 17:12:30.468484    2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve-ci-e2e-test/message-dumper-predictor-6575bcfdfd-mn29k"
Jul 01 17:12:30.470123 ip-10-0-137-244 kubenswrapper[2565]: I0701 17:12:30.470103    2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kserve-ci-e2e-test/message-dumper-predictor-6575bcfdfd-mn29k"
Jul 01 17:12:30.491556 ip-10-0-137-244 kubenswrapper[2565]: I0701 17:12:30.491513    2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kserve-ci-e2e-test/message-dumper-predictor-6575bcfdfd-mn29k" podStartSLOduration=2.25551341 podStartE2EDuration="3.491499628s" totalImagesPullingTime="1.235986218s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-01 17:12:27 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-01 17:12:30.491189184 +0000 UTC m=+820.959453309" watchObservedRunningTime="2026-07-01 17:12:30.491499628 +0000 UTC m=+820.959763751"
Jul 01 17:12:31.474443 ip-10-0-137-244 kubenswrapper[2565]: I0701 17:12:31.474417    2565 generic.go:356] "Generic (PLEG): container finished" podID="f7c7e394-4643-49ef-b2ca-dff32f41ff0b" containerID="88fc28d775f07e47fdd4979091800fbc114c1aee0e1b5739d568a1d759187b17" exitCode=0
Jul 01 17:12:31.474711 ip-10-0-137-244 kubenswrapper[2565]: I0701 17:12:31.474490    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/isvc-sklearn-batcher-custom-predictor-5fccccdbd5-vc7st" event={"ID":"f7c7e394-4643-49ef-b2ca-dff32f41ff0b","Type":"ContainerDied","Data":"88fc28d775f07e47fdd4979091800fbc114c1aee0e1b5739d568a1d759187b17"}
Jul 01 17:12:35.134966 ip-10-0-137-244 kubenswrapper[2565]: I0701 17:12:35.134918    2565 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-sklearn-batcher-custom-predictor-5fccccdbd5-vc7st" podUID="f7c7e394-4643-49ef-b2ca-dff32f41ff0b" containerName="kube-rbac-proxy" probeResult="failure" output="Get \"https://10.133.0.26:8643/healthz\": dial tcp 10.133.0.26:8643: connect: connection refused"
Jul 01 17:12:36.244831 ip-10-0-137-244 kubenswrapper[2565]: I0701 17:12:36.244774    2565 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-sklearn-batcher-custom-predictor-5fccccdbd5-vc7st" podUID="f7c7e394-4643-49ef-b2ca-dff32f41ff0b" containerName="kserve-container" probeResult="failure" output="dial tcp 10.133.0.26:5000: connect: connection refused"
Jul 01 17:12:36.245318 ip-10-0-137-244 kubenswrapper[2565]: I0701 17:12:36.245102    2565 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-sklearn-batcher-custom-predictor-5fccccdbd5-vc7st" podUID="f7c7e394-4643-49ef-b2ca-dff32f41ff0b" containerName="agent" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Jul 01 17:12:37.483720 ip-10-0-137-244 kubenswrapper[2565]: I0701 17:12:37.483666    2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kserve-ci-e2e-test/message-dumper-predictor-6575bcfdfd-mn29k"
Jul 01 17:12:40.135239 ip-10-0-137-244 kubenswrapper[2565]: I0701 17:12:40.135190    2565 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-sklearn-batcher-custom-predictor-5fccccdbd5-vc7st" podUID="f7c7e394-4643-49ef-b2ca-dff32f41ff0b" containerName="kube-rbac-proxy" probeResult="failure" output="Get \"https://10.133.0.26:8643/healthz\": dial tcp 10.133.0.26:8643: connect: connection refused"
Jul 01 17:12:40.135708 ip-10-0-137-244 kubenswrapper[2565]: I0701 17:12:40.135369    2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve-ci-e2e-test/isvc-sklearn-batcher-custom-predictor-5fccccdbd5-vc7st"
Jul 01 17:12:45.135215 ip-10-0-137-244 kubenswrapper[2565]: I0701 17:12:45.135172    2565 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-sklearn-batcher-custom-predictor-5fccccdbd5-vc7st" podUID="f7c7e394-4643-49ef-b2ca-dff32f41ff0b" containerName="kube-rbac-proxy" probeResult="failure" output="Get \"https://10.133.0.26:8643/healthz\": dial tcp 10.133.0.26:8643: connect: connection refused"
Jul 01 17:12:46.244158 ip-10-0-137-244 kubenswrapper[2565]: I0701 17:12:46.244111    2565 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-sklearn-batcher-custom-predictor-5fccccdbd5-vc7st" podUID="f7c7e394-4643-49ef-b2ca-dff32f41ff0b" containerName="kserve-container" probeResult="failure" output="dial tcp 10.133.0.26:5000: connect: connection refused"
Jul 01 17:12:46.244600 ip-10-0-137-244 kubenswrapper[2565]: I0701 17:12:46.244574    2565 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-sklearn-batcher-custom-predictor-5fccccdbd5-vc7st" podUID="f7c7e394-4643-49ef-b2ca-dff32f41ff0b" containerName="agent" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Jul 01 17:12:47.429321 ip-10-0-137-244 kubenswrapper[2565]: I0701 17:12:47.429289    2565 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kserve-ci-e2e-test/isvc-logger-predictor-599b89d7c-p5d9m"]
Jul 01 17:12:47.432784 ip-10-0-137-244 kubenswrapper[2565]: I0701 17:12:47.432764    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/isvc-logger-predictor-599b89d7c-p5d9m"
Jul 01 17:12:47.435263 ip-10-0-137-244 kubenswrapper[2565]: I0701 17:12:47.435246    2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kserve-ci-e2e-test\"/\"isvc-logger-predictor-serving-cert\""
Jul 01 17:12:47.435263 ip-10-0-137-244 kubenswrapper[2565]: I0701 17:12:47.435252    2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kserve-ci-e2e-test\"/\"isvc-logger-kube-rbac-proxy-sar-config\""
Jul 01 17:12:47.443053 ip-10-0-137-244 kubenswrapper[2565]: I0701 17:12:47.443032    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/isvc-logger-predictor-599b89d7c-p5d9m"]
Jul 01 17:12:47.529457 ip-10-0-137-244 kubenswrapper[2565]: I0701 17:12:47.529434    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/ae0049f6-86e1-43c6-b675-21b3e853802c-kube-rbac-proxy-sar-config\") pod \"isvc-logger-predictor-599b89d7c-p5d9m\" (UID: \"ae0049f6-86e1-43c6-b675-21b3e853802c\") " pod="kserve-ci-e2e-test/isvc-logger-predictor-599b89d7c-p5d9m"
Jul 01 17:12:47.529562 ip-10-0-137-244 kubenswrapper[2565]: I0701 17:12:47.529460    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cabundle-cert\" (UniqueName: \"kubernetes.io/configmap/ae0049f6-86e1-43c6-b675-21b3e853802c-cabundle-cert\") pod \"isvc-logger-predictor-599b89d7c-p5d9m\" (UID: \"ae0049f6-86e1-43c6-b675-21b3e853802c\") " pod="kserve-ci-e2e-test/isvc-logger-predictor-599b89d7c-p5d9m"
Jul 01 17:12:47.529562 ip-10-0-137-244 kubenswrapper[2565]: I0701 17:12:47.529486    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/ae0049f6-86e1-43c6-b675-21b3e853802c-proxy-tls\") pod \"isvc-logger-predictor-599b89d7c-p5d9m\" (UID: \"ae0049f6-86e1-43c6-b675-21b3e853802c\") " pod="kserve-ci-e2e-test/isvc-logger-predictor-599b89d7c-p5d9m"
Jul 01 17:12:47.529651 ip-10-0-137-244 kubenswrapper[2565]: I0701 17:12:47.529591    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/ae0049f6-86e1-43c6-b675-21b3e853802c-kserve-provision-location\") pod \"isvc-logger-predictor-599b89d7c-p5d9m\" (UID: \"ae0049f6-86e1-43c6-b675-21b3e853802c\") " pod="kserve-ci-e2e-test/isvc-logger-predictor-599b89d7c-p5d9m"
Jul 01 17:12:47.529651 ip-10-0-137-244 kubenswrapper[2565]: I0701 17:12:47.529630    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jnc2f\" (UniqueName: \"kubernetes.io/projected/ae0049f6-86e1-43c6-b675-21b3e853802c-kube-api-access-jnc2f\") pod \"isvc-logger-predictor-599b89d7c-p5d9m\" (UID: \"ae0049f6-86e1-43c6-b675-21b3e853802c\") " pod="kserve-ci-e2e-test/isvc-logger-predictor-599b89d7c-p5d9m"
Jul 01 17:12:47.630340 ip-10-0-137-244 kubenswrapper[2565]: I0701 17:12:47.630318    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/ae0049f6-86e1-43c6-b675-21b3e853802c-kserve-provision-location\") pod \"isvc-logger-predictor-599b89d7c-p5d9m\" (UID: \"ae0049f6-86e1-43c6-b675-21b3e853802c\") " pod="kserve-ci-e2e-test/isvc-logger-predictor-599b89d7c-p5d9m"
Jul 01 17:12:47.630438 ip-10-0-137-244 kubenswrapper[2565]: I0701 17:12:47.630354    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jnc2f\" (UniqueName: \"kubernetes.io/projected/ae0049f6-86e1-43c6-b675-21b3e853802c-kube-api-access-jnc2f\") pod \"isvc-logger-predictor-599b89d7c-p5d9m\" (UID: \"ae0049f6-86e1-43c6-b675-21b3e853802c\") " pod="kserve-ci-e2e-test/isvc-logger-predictor-599b89d7c-p5d9m"
Jul 01 17:12:47.630438 ip-10-0-137-244 kubenswrapper[2565]: I0701 17:12:47.630378    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/ae0049f6-86e1-43c6-b675-21b3e853802c-kube-rbac-proxy-sar-config\") pod \"isvc-logger-predictor-599b89d7c-p5d9m\" (UID: \"ae0049f6-86e1-43c6-b675-21b3e853802c\") " pod="kserve-ci-e2e-test/isvc-logger-predictor-599b89d7c-p5d9m"
Jul 01 17:12:47.630438 ip-10-0-137-244 kubenswrapper[2565]: I0701 17:12:47.630401    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cabundle-cert\" (UniqueName: \"kubernetes.io/configmap/ae0049f6-86e1-43c6-b675-21b3e853802c-cabundle-cert\") pod \"isvc-logger-predictor-599b89d7c-p5d9m\" (UID: \"ae0049f6-86e1-43c6-b675-21b3e853802c\") " pod="kserve-ci-e2e-test/isvc-logger-predictor-599b89d7c-p5d9m"
Jul 01 17:12:47.630559 ip-10-0-137-244 kubenswrapper[2565]: I0701 17:12:47.630437    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/ae0049f6-86e1-43c6-b675-21b3e853802c-proxy-tls\") pod \"isvc-logger-predictor-599b89d7c-p5d9m\" (UID: \"ae0049f6-86e1-43c6-b675-21b3e853802c\") " pod="kserve-ci-e2e-test/isvc-logger-predictor-599b89d7c-p5d9m"
Jul 01 17:12:47.630755 ip-10-0-137-244 kubenswrapper[2565]: I0701 17:12:47.630736    2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/ae0049f6-86e1-43c6-b675-21b3e853802c-kserve-provision-location\") pod \"isvc-logger-predictor-599b89d7c-p5d9m\" (UID: \"ae0049f6-86e1-43c6-b675-21b3e853802c\") " pod="kserve-ci-e2e-test/isvc-logger-predictor-599b89d7c-p5d9m"
Jul 01 17:12:47.631116 ip-10-0-137-244 kubenswrapper[2565]: I0701 17:12:47.631095    2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/ae0049f6-86e1-43c6-b675-21b3e853802c-kube-rbac-proxy-sar-config\") pod \"isvc-logger-predictor-599b89d7c-p5d9m\" (UID: \"ae0049f6-86e1-43c6-b675-21b3e853802c\") " pod="kserve-ci-e2e-test/isvc-logger-predictor-599b89d7c-p5d9m"
Jul 01 17:12:47.631209 ip-10-0-137-244 kubenswrapper[2565]: I0701 17:12:47.631133    2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cabundle-cert\" (UniqueName: \"kubernetes.io/configmap/ae0049f6-86e1-43c6-b675-21b3e853802c-cabundle-cert\") pod \"isvc-logger-predictor-599b89d7c-p5d9m\" (UID: \"ae0049f6-86e1-43c6-b675-21b3e853802c\") " pod="kserve-ci-e2e-test/isvc-logger-predictor-599b89d7c-p5d9m"
Jul 01 17:12:47.632747 ip-10-0-137-244 kubenswrapper[2565]: I0701 17:12:47.632728    2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/ae0049f6-86e1-43c6-b675-21b3e853802c-proxy-tls\") pod \"isvc-logger-predictor-599b89d7c-p5d9m\" (UID: \"ae0049f6-86e1-43c6-b675-21b3e853802c\") " pod="kserve-ci-e2e-test/isvc-logger-predictor-599b89d7c-p5d9m"
Jul 01 17:12:47.639436 ip-10-0-137-244 kubenswrapper[2565]: I0701 17:12:47.639415    2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-jnc2f\" (UniqueName: \"kubernetes.io/projected/ae0049f6-86e1-43c6-b675-21b3e853802c-kube-api-access-jnc2f\") pod \"isvc-logger-predictor-599b89d7c-p5d9m\" (UID: \"ae0049f6-86e1-43c6-b675-21b3e853802c\") " pod="kserve-ci-e2e-test/isvc-logger-predictor-599b89d7c-p5d9m"
Jul 01 17:12:47.743360 ip-10-0-137-244 kubenswrapper[2565]: I0701 17:12:47.743308    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/isvc-logger-predictor-599b89d7c-p5d9m"
Jul 01 17:12:47.871706 ip-10-0-137-244 kubenswrapper[2565]: I0701 17:12:47.871681    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/isvc-logger-predictor-599b89d7c-p5d9m"]
Jul 01 17:12:47.874071 ip-10-0-137-244 kubenswrapper[2565]: W0701 17:12:47.874044    2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podae0049f6_86e1_43c6_b675_21b3e853802c.slice/crio-187806f25885dd073167ae7418cd1ebabf88999a3f29dd6645915cd7cd20a871 WatchSource:0}: Error finding container 187806f25885dd073167ae7418cd1ebabf88999a3f29dd6645915cd7cd20a871: Status 404 returned error can't find the container with id 187806f25885dd073167ae7418cd1ebabf88999a3f29dd6645915cd7cd20a871
Jul 01 17:12:48.546930 ip-10-0-137-244 kubenswrapper[2565]: I0701 17:12:48.546892    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/isvc-logger-predictor-599b89d7c-p5d9m" event={"ID":"ae0049f6-86e1-43c6-b675-21b3e853802c","Type":"ContainerStarted","Data":"e4a9754848d7593fc5ec226fec4d38e5cb46c3c475e8fcd3653aed2b48365e24"}
Jul 01 17:12:48.547373 ip-10-0-137-244 kubenswrapper[2565]: I0701 17:12:48.546962    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/isvc-logger-predictor-599b89d7c-p5d9m" event={"ID":"ae0049f6-86e1-43c6-b675-21b3e853802c","Type":"ContainerStarted","Data":"187806f25885dd073167ae7418cd1ebabf88999a3f29dd6645915cd7cd20a871"}
Jul 01 17:12:50.135782 ip-10-0-137-244 kubenswrapper[2565]: I0701 17:12:50.135743    2565 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-sklearn-batcher-custom-predictor-5fccccdbd5-vc7st" podUID="f7c7e394-4643-49ef-b2ca-dff32f41ff0b" containerName="kube-rbac-proxy" probeResult="failure" output="Get \"https://10.133.0.26:8643/healthz\": dial tcp 10.133.0.26:8643: connect: connection refused"
Jul 01 17:12:51.563114 ip-10-0-137-244 kubenswrapper[2565]: I0701 17:12:51.563085    2565 generic.go:356] "Generic (PLEG): container finished" podID="ae0049f6-86e1-43c6-b675-21b3e853802c" containerID="e4a9754848d7593fc5ec226fec4d38e5cb46c3c475e8fcd3653aed2b48365e24" exitCode=0
Jul 01 17:12:51.563423 ip-10-0-137-244 kubenswrapper[2565]: I0701 17:12:51.563129    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/isvc-logger-predictor-599b89d7c-p5d9m" event={"ID":"ae0049f6-86e1-43c6-b675-21b3e853802c","Type":"ContainerDied","Data":"e4a9754848d7593fc5ec226fec4d38e5cb46c3c475e8fcd3653aed2b48365e24"}
Jul 01 17:12:52.569584 ip-10-0-137-244 kubenswrapper[2565]: I0701 17:12:52.569551    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/isvc-logger-predictor-599b89d7c-p5d9m" event={"ID":"ae0049f6-86e1-43c6-b675-21b3e853802c","Type":"ContainerStarted","Data":"f178e5776aaa499faaa8f5b985045b554847cd9d3a6478a65f1a8cbc3c40f660"}
Jul 01 17:12:52.569982 ip-10-0-137-244 kubenswrapper[2565]: I0701 17:12:52.569613    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/isvc-logger-predictor-599b89d7c-p5d9m" event={"ID":"ae0049f6-86e1-43c6-b675-21b3e853802c","Type":"ContainerStarted","Data":"620764bf68c6753d91d25903ca3f4f5c11e2626888da17d6732945e624cc20fb"}
Jul 01 17:12:52.569982 ip-10-0-137-244 kubenswrapper[2565]: I0701 17:12:52.569636    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/isvc-logger-predictor-599b89d7c-p5d9m" event={"ID":"ae0049f6-86e1-43c6-b675-21b3e853802c","Type":"ContainerStarted","Data":"2c01b56be107417636fb20d1bc799b2f0018f36d25831eae3d8d33718d40d60e"}
Jul 01 17:12:52.569982 ip-10-0-137-244 kubenswrapper[2565]: I0701 17:12:52.569924    2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve-ci-e2e-test/isvc-logger-predictor-599b89d7c-p5d9m"
Jul 01 17:12:52.570142 ip-10-0-137-244 kubenswrapper[2565]: I0701 17:12:52.569989    2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve-ci-e2e-test/isvc-logger-predictor-599b89d7c-p5d9m"
Jul 01 17:12:52.571148 ip-10-0-137-244 kubenswrapper[2565]: I0701 17:12:52.571123    2565 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-logger-predictor-599b89d7c-p5d9m" podUID="ae0049f6-86e1-43c6-b675-21b3e853802c" containerName="kserve-container" probeResult="failure" output="dial tcp 10.133.0.28:8080: connect: connection refused"
Jul 01 17:12:52.592770 ip-10-0-137-244 kubenswrapper[2565]: I0701 17:12:52.592728    2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kserve-ci-e2e-test/isvc-logger-predictor-599b89d7c-p5d9m" podStartSLOduration=2.114582212 podStartE2EDuration="5.592714424s" totalImagesPullingTime="0s" totalInitContainerRuntime="3.478132212s" isStatefulPod=false podCreationTimestamp="2026-07-01 17:12:47 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-01 17:12:52.591442636 +0000 UTC m=+843.059706761" watchObservedRunningTime="2026-07-01 17:12:52.592714424 +0000 UTC m=+843.060978547"
Jul 01 17:12:53.574733 ip-10-0-137-244 kubenswrapper[2565]: I0701 17:12:53.574700    2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve-ci-e2e-test/isvc-logger-predictor-599b89d7c-p5d9m"
Jul 01 17:12:53.575156 ip-10-0-137-244 kubenswrapper[2565]: I0701 17:12:53.574853    2565 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-logger-predictor-599b89d7c-p5d9m" podUID="ae0049f6-86e1-43c6-b675-21b3e853802c" containerName="kserve-container" probeResult="failure" output="dial tcp 10.133.0.28:8080: connect: connection refused"
Jul 01 17:12:53.576013 ip-10-0-137-244 kubenswrapper[2565]: I0701 17:12:53.575983    2565 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-logger-predictor-599b89d7c-p5d9m" podUID="ae0049f6-86e1-43c6-b675-21b3e853802c" containerName="agent" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Jul 01 17:12:54.578450 ip-10-0-137-244 kubenswrapper[2565]: I0701 17:12:54.578405    2565 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-logger-predictor-599b89d7c-p5d9m" podUID="ae0049f6-86e1-43c6-b675-21b3e853802c" containerName="kserve-container" probeResult="failure" output="dial tcp 10.133.0.28:8080: connect: connection refused"
Jul 01 17:12:54.578858 ip-10-0-137-244 kubenswrapper[2565]: I0701 17:12:54.578785    2565 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-logger-predictor-599b89d7c-p5d9m" podUID="ae0049f6-86e1-43c6-b675-21b3e853802c" containerName="agent" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Jul 01 17:12:55.135665 ip-10-0-137-244 kubenswrapper[2565]: I0701 17:12:55.135627    2565 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-sklearn-batcher-custom-predictor-5fccccdbd5-vc7st" podUID="f7c7e394-4643-49ef-b2ca-dff32f41ff0b" containerName="kube-rbac-proxy" probeResult="failure" output="Get \"https://10.133.0.26:8643/healthz\": dial tcp 10.133.0.26:8643: connect: connection refused"
Jul 01 17:12:56.244765 ip-10-0-137-244 kubenswrapper[2565]: I0701 17:12:56.244718    2565 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-sklearn-batcher-custom-predictor-5fccccdbd5-vc7st" podUID="f7c7e394-4643-49ef-b2ca-dff32f41ff0b" containerName="kserve-container" probeResult="failure" output="dial tcp 10.133.0.26:5000: connect: connection refused"
Jul 01 17:12:56.245233 ip-10-0-137-244 kubenswrapper[2565]: I0701 17:12:56.245047    2565 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-sklearn-batcher-custom-predictor-5fccccdbd5-vc7st" podUID="f7c7e394-4643-49ef-b2ca-dff32f41ff0b" containerName="agent" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Jul 01 17:12:56.249542 ip-10-0-137-244 kubenswrapper[2565]: I0701 17:12:56.249521    2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve-ci-e2e-test/isvc-sklearn-batcher-custom-predictor-5fccccdbd5-vc7st"
Jul 01 17:12:56.249658 ip-10-0-137-244 kubenswrapper[2565]: I0701 17:12:56.249603    2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve-ci-e2e-test/isvc-sklearn-batcher-custom-predictor-5fccccdbd5-vc7st"
Jul 01 17:12:57.461929 ip-10-0-137-244 kubenswrapper[2565]: I0701 17:12:57.461908    2565 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/isvc-sklearn-batcher-custom-predictor-5fccccdbd5-vc7st"
Jul 01 17:12:57.495994 ip-10-0-137-244 kubenswrapper[2565]: I0701 17:12:57.495975    2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/f7c7e394-4643-49ef-b2ca-dff32f41ff0b-kserve-provision-location\") pod \"f7c7e394-4643-49ef-b2ca-dff32f41ff0b\" (UID: \"f7c7e394-4643-49ef-b2ca-dff32f41ff0b\") "
Jul 01 17:12:57.496130 ip-10-0-137-244 kubenswrapper[2565]: I0701 17:12:57.496001    2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-f8km5\" (UniqueName: \"kubernetes.io/projected/f7c7e394-4643-49ef-b2ca-dff32f41ff0b-kube-api-access-f8km5\") pod \"f7c7e394-4643-49ef-b2ca-dff32f41ff0b\" (UID: \"f7c7e394-4643-49ef-b2ca-dff32f41ff0b\") "
Jul 01 17:12:57.496130 ip-10-0-137-244 kubenswrapper[2565]: I0701 17:12:57.496062    2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/f7c7e394-4643-49ef-b2ca-dff32f41ff0b-kube-rbac-proxy-sar-config\") pod \"f7c7e394-4643-49ef-b2ca-dff32f41ff0b\" (UID: \"f7c7e394-4643-49ef-b2ca-dff32f41ff0b\") "
Jul 01 17:12:57.496130 ip-10-0-137-244 kubenswrapper[2565]: I0701 17:12:57.496080    2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/f7c7e394-4643-49ef-b2ca-dff32f41ff0b-proxy-tls\") pod \"f7c7e394-4643-49ef-b2ca-dff32f41ff0b\" (UID: \"f7c7e394-4643-49ef-b2ca-dff32f41ff0b\") "
Jul 01 17:12:57.496130 ip-10-0-137-244 kubenswrapper[2565]: I0701 17:12:57.496095    2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"cabundle-cert\" (UniqueName: \"kubernetes.io/configmap/f7c7e394-4643-49ef-b2ca-dff32f41ff0b-cabundle-cert\") pod \"f7c7e394-4643-49ef-b2ca-dff32f41ff0b\" (UID: \"f7c7e394-4643-49ef-b2ca-dff32f41ff0b\") "
Jul 01 17:12:57.496329 ip-10-0-137-244 kubenswrapper[2565]: I0701 17:12:57.496247    2565 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f7c7e394-4643-49ef-b2ca-dff32f41ff0b-kserve-provision-location" (OuterVolumeSpecName: "kserve-provision-location") pod "f7c7e394-4643-49ef-b2ca-dff32f41ff0b" (UID: "f7c7e394-4643-49ef-b2ca-dff32f41ff0b"). InnerVolumeSpecName "kserve-provision-location". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 01 17:12:57.496404 ip-10-0-137-244 kubenswrapper[2565]: I0701 17:12:57.496383    2565 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/f7c7e394-4643-49ef-b2ca-dff32f41ff0b-kube-rbac-proxy-sar-config" (OuterVolumeSpecName: "kube-rbac-proxy-sar-config") pod "f7c7e394-4643-49ef-b2ca-dff32f41ff0b" (UID: "f7c7e394-4643-49ef-b2ca-dff32f41ff0b"). InnerVolumeSpecName "kube-rbac-proxy-sar-config". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 01 17:12:57.496542 ip-10-0-137-244 kubenswrapper[2565]: I0701 17:12:57.496519    2565 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/f7c7e394-4643-49ef-b2ca-dff32f41ff0b-cabundle-cert" (OuterVolumeSpecName: "cabundle-cert") pod "f7c7e394-4643-49ef-b2ca-dff32f41ff0b" (UID: "f7c7e394-4643-49ef-b2ca-dff32f41ff0b"). InnerVolumeSpecName "cabundle-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 01 17:12:57.498123 ip-10-0-137-244 kubenswrapper[2565]: I0701 17:12:57.498095    2565 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f7c7e394-4643-49ef-b2ca-dff32f41ff0b-kube-api-access-f8km5" (OuterVolumeSpecName: "kube-api-access-f8km5") pod "f7c7e394-4643-49ef-b2ca-dff32f41ff0b" (UID: "f7c7e394-4643-49ef-b2ca-dff32f41ff0b"). InnerVolumeSpecName "kube-api-access-f8km5". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 01 17:12:57.498123 ip-10-0-137-244 kubenswrapper[2565]: I0701 17:12:57.498104    2565 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/f7c7e394-4643-49ef-b2ca-dff32f41ff0b-proxy-tls" (OuterVolumeSpecName: "proxy-tls") pod "f7c7e394-4643-49ef-b2ca-dff32f41ff0b" (UID: "f7c7e394-4643-49ef-b2ca-dff32f41ff0b"). InnerVolumeSpecName "proxy-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 01 17:12:57.593069 ip-10-0-137-244 kubenswrapper[2565]: I0701 17:12:57.592994    2565 generic.go:356] "Generic (PLEG): container finished" podID="f7c7e394-4643-49ef-b2ca-dff32f41ff0b" containerID="01fb7bcd7b746c3c88b2f2a8ce77b95f6412545f7c254e8e49b215621be9eb1c" exitCode=0
Jul 01 17:12:57.593145 ip-10-0-137-244 kubenswrapper[2565]: I0701 17:12:57.593076    2565 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/isvc-sklearn-batcher-custom-predictor-5fccccdbd5-vc7st"
Jul 01 17:12:57.593145 ip-10-0-137-244 kubenswrapper[2565]: I0701 17:12:57.593076    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/isvc-sklearn-batcher-custom-predictor-5fccccdbd5-vc7st" event={"ID":"f7c7e394-4643-49ef-b2ca-dff32f41ff0b","Type":"ContainerDied","Data":"01fb7bcd7b746c3c88b2f2a8ce77b95f6412545f7c254e8e49b215621be9eb1c"}
Jul 01 17:12:57.593145 ip-10-0-137-244 kubenswrapper[2565]: I0701 17:12:57.593140    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/isvc-sklearn-batcher-custom-predictor-5fccccdbd5-vc7st" event={"ID":"f7c7e394-4643-49ef-b2ca-dff32f41ff0b","Type":"ContainerDied","Data":"f86d273b9759c680b1f5272aa51c4ce4c055c7e8f28776d90dfa367ee5155298"}
Jul 01 17:12:57.593261 ip-10-0-137-244 kubenswrapper[2565]: I0701 17:12:57.593170    2565 scope.go:117] "RemoveContainer" containerID="01fb7bcd7b746c3c88b2f2a8ce77b95f6412545f7c254e8e49b215621be9eb1c"
Jul 01 17:12:57.596737 ip-10-0-137-244 kubenswrapper[2565]: I0701 17:12:57.596718    2565 reconciler_common.go:299] "Volume detached for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/f7c7e394-4643-49ef-b2ca-dff32f41ff0b-kserve-provision-location\") on node \"ip-10-0-137-244.ec2.internal\" DevicePath \"\""
Jul 01 17:12:57.596806 ip-10-0-137-244 kubenswrapper[2565]: I0701 17:12:57.596739    2565 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-f8km5\" (UniqueName: \"kubernetes.io/projected/f7c7e394-4643-49ef-b2ca-dff32f41ff0b-kube-api-access-f8km5\") on node \"ip-10-0-137-244.ec2.internal\" DevicePath \"\""
Jul 01 17:12:57.596806 ip-10-0-137-244 kubenswrapper[2565]: I0701 17:12:57.596749    2565 reconciler_common.go:299] "Volume detached for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/f7c7e394-4643-49ef-b2ca-dff32f41ff0b-kube-rbac-proxy-sar-config\") on node \"ip-10-0-137-244.ec2.internal\" DevicePath \"\""
Jul 01 17:12:57.596806 ip-10-0-137-244 kubenswrapper[2565]: I0701 17:12:57.596759    2565 reconciler_common.go:299] "Volume detached for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/f7c7e394-4643-49ef-b2ca-dff32f41ff0b-proxy-tls\") on node \"ip-10-0-137-244.ec2.internal\" DevicePath \"\""
Jul 01 17:12:57.596806 ip-10-0-137-244 kubenswrapper[2565]: I0701 17:12:57.596767    2565 reconciler_common.go:299] "Volume detached for volume \"cabundle-cert\" (UniqueName: \"kubernetes.io/configmap/f7c7e394-4643-49ef-b2ca-dff32f41ff0b-cabundle-cert\") on node \"ip-10-0-137-244.ec2.internal\" DevicePath \"\""
Jul 01 17:12:57.601228 ip-10-0-137-244 kubenswrapper[2565]: I0701 17:12:57.601209    2565 scope.go:117] "RemoveContainer" containerID="53061e4c83b2ba6d9281180383047f139956cfc553f8d527e47f57d8a5140448"
Jul 01 17:12:57.608293 ip-10-0-137-244 kubenswrapper[2565]: I0701 17:12:57.608277    2565 scope.go:117] "RemoveContainer" containerID="88fc28d775f07e47fdd4979091800fbc114c1aee0e1b5739d568a1d759187b17"
Jul 01 17:12:57.615256 ip-10-0-137-244 kubenswrapper[2565]: I0701 17:12:57.615243    2565 scope.go:117] "RemoveContainer" containerID="3970272bd5f909591a101fa75c4f1770aaa5c343b5de21974506755a8e5c0bf3"
Jul 01 17:12:57.615333 ip-10-0-137-244 kubenswrapper[2565]: I0701 17:12:57.615315    2565 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kserve-ci-e2e-test/isvc-sklearn-batcher-custom-predictor-5fccccdbd5-vc7st"]
Jul 01 17:12:57.621508 ip-10-0-137-244 kubenswrapper[2565]: I0701 17:12:57.621485    2565 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kserve-ci-e2e-test/isvc-sklearn-batcher-custom-predictor-5fccccdbd5-vc7st"]
Jul 01 17:12:57.622236 ip-10-0-137-244 kubenswrapper[2565]: I0701 17:12:57.622223    2565 scope.go:117] "RemoveContainer" containerID="01fb7bcd7b746c3c88b2f2a8ce77b95f6412545f7c254e8e49b215621be9eb1c"
Jul 01 17:12:57.622457 ip-10-0-137-244 kubenswrapper[2565]: E0701 17:12:57.622442    2565 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"01fb7bcd7b746c3c88b2f2a8ce77b95f6412545f7c254e8e49b215621be9eb1c\": container with ID starting with 01fb7bcd7b746c3c88b2f2a8ce77b95f6412545f7c254e8e49b215621be9eb1c not found: ID does not exist" containerID="01fb7bcd7b746c3c88b2f2a8ce77b95f6412545f7c254e8e49b215621be9eb1c"
Jul 01 17:12:57.622525 ip-10-0-137-244 kubenswrapper[2565]: I0701 17:12:57.622462    2565 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"01fb7bcd7b746c3c88b2f2a8ce77b95f6412545f7c254e8e49b215621be9eb1c"} err="failed to get container status \"01fb7bcd7b746c3c88b2f2a8ce77b95f6412545f7c254e8e49b215621be9eb1c\": rpc error: code = NotFound desc = could not find container \"01fb7bcd7b746c3c88b2f2a8ce77b95f6412545f7c254e8e49b215621be9eb1c\": container with ID starting with 01fb7bcd7b746c3c88b2f2a8ce77b95f6412545f7c254e8e49b215621be9eb1c not found: ID does not exist"
Jul 01 17:12:57.622525 ip-10-0-137-244 kubenswrapper[2565]: I0701 17:12:57.622476    2565 scope.go:117] "RemoveContainer" containerID="53061e4c83b2ba6d9281180383047f139956cfc553f8d527e47f57d8a5140448"
Jul 01 17:12:57.622700 ip-10-0-137-244 kubenswrapper[2565]: E0701 17:12:57.622684    2565 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"53061e4c83b2ba6d9281180383047f139956cfc553f8d527e47f57d8a5140448\": container with ID starting with 53061e4c83b2ba6d9281180383047f139956cfc553f8d527e47f57d8a5140448 not found: ID does not exist" containerID="53061e4c83b2ba6d9281180383047f139956cfc553f8d527e47f57d8a5140448"
Jul 01 17:12:57.622745 ip-10-0-137-244 kubenswrapper[2565]: I0701 17:12:57.622708    2565 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"53061e4c83b2ba6d9281180383047f139956cfc553f8d527e47f57d8a5140448"} err="failed to get container status \"53061e4c83b2ba6d9281180383047f139956cfc553f8d527e47f57d8a5140448\": rpc error: code = NotFound desc = could not find container \"53061e4c83b2ba6d9281180383047f139956cfc553f8d527e47f57d8a5140448\": container with ID starting with 53061e4c83b2ba6d9281180383047f139956cfc553f8d527e47f57d8a5140448 not found: ID does not exist"
Jul 01 17:12:57.622745 ip-10-0-137-244 kubenswrapper[2565]: I0701 17:12:57.622724    2565 scope.go:117] "RemoveContainer" containerID="88fc28d775f07e47fdd4979091800fbc114c1aee0e1b5739d568a1d759187b17"
Jul 01 17:12:57.622952 ip-10-0-137-244 kubenswrapper[2565]: E0701 17:12:57.622937    2565 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"88fc28d775f07e47fdd4979091800fbc114c1aee0e1b5739d568a1d759187b17\": container with ID starting with 88fc28d775f07e47fdd4979091800fbc114c1aee0e1b5739d568a1d759187b17 not found: ID does not exist" containerID="88fc28d775f07e47fdd4979091800fbc114c1aee0e1b5739d568a1d759187b17"
Jul 01 17:12:57.623004 ip-10-0-137-244 kubenswrapper[2565]: I0701 17:12:57.622954    2565 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"88fc28d775f07e47fdd4979091800fbc114c1aee0e1b5739d568a1d759187b17"} err="failed to get container status \"88fc28d775f07e47fdd4979091800fbc114c1aee0e1b5739d568a1d759187b17\": rpc error: code = NotFound desc = could not find container \"88fc28d775f07e47fdd4979091800fbc114c1aee0e1b5739d568a1d759187b17\": container with ID starting with 88fc28d775f07e47fdd4979091800fbc114c1aee0e1b5739d568a1d759187b17 not found: ID does not exist"
Jul 01 17:12:57.623004 ip-10-0-137-244 kubenswrapper[2565]: I0701 17:12:57.622965    2565 scope.go:117] "RemoveContainer" containerID="3970272bd5f909591a101fa75c4f1770aaa5c343b5de21974506755a8e5c0bf3"
Jul 01 17:12:57.623174 ip-10-0-137-244 kubenswrapper[2565]: E0701 17:12:57.623160    2565 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"3970272bd5f909591a101fa75c4f1770aaa5c343b5de21974506755a8e5c0bf3\": container with ID starting with 3970272bd5f909591a101fa75c4f1770aaa5c343b5de21974506755a8e5c0bf3 not found: ID does not exist" containerID="3970272bd5f909591a101fa75c4f1770aaa5c343b5de21974506755a8e5c0bf3"
Jul 01 17:12:57.623216 ip-10-0-137-244 kubenswrapper[2565]: I0701 17:12:57.623176    2565 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"3970272bd5f909591a101fa75c4f1770aaa5c343b5de21974506755a8e5c0bf3"} err="failed to get container status \"3970272bd5f909591a101fa75c4f1770aaa5c343b5de21974506755a8e5c0bf3\": rpc error: code = NotFound desc = could not find container \"3970272bd5f909591a101fa75c4f1770aaa5c343b5de21974506755a8e5c0bf3\": container with ID starting with 3970272bd5f909591a101fa75c4f1770aaa5c343b5de21974506755a8e5c0bf3 not found: ID does not exist"
Jul 01 17:12:58.248476 ip-10-0-137-244 kubenswrapper[2565]: I0701 17:12:58.248446    2565 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="f7c7e394-4643-49ef-b2ca-dff32f41ff0b" path="/var/lib/kubelet/pods/f7c7e394-4643-49ef-b2ca-dff32f41ff0b/volumes"
Jul 01 17:12:59.582933 ip-10-0-137-244 kubenswrapper[2565]: I0701 17:12:59.582906    2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kserve-ci-e2e-test/isvc-logger-predictor-599b89d7c-p5d9m"
Jul 01 17:12:59.583505 ip-10-0-137-244 kubenswrapper[2565]: I0701 17:12:59.583478    2565 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-logger-predictor-599b89d7c-p5d9m" podUID="ae0049f6-86e1-43c6-b675-21b3e853802c" containerName="kserve-container" probeResult="failure" output="dial tcp 10.133.0.28:8080: connect: connection refused"
Jul 01 17:12:59.583732 ip-10-0-137-244 kubenswrapper[2565]: I0701 17:12:59.583709    2565 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-logger-predictor-599b89d7c-p5d9m" podUID="ae0049f6-86e1-43c6-b675-21b3e853802c" containerName="agent" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Jul 01 17:13:09.584056 ip-10-0-137-244 kubenswrapper[2565]: I0701 17:13:09.584007    2565 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-logger-predictor-599b89d7c-p5d9m" podUID="ae0049f6-86e1-43c6-b675-21b3e853802c" containerName="kserve-container" probeResult="failure" output="dial tcp 10.133.0.28:8080: connect: connection refused"
Jul 01 17:13:09.584555 ip-10-0-137-244 kubenswrapper[2565]: I0701 17:13:09.584532    2565 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-logger-predictor-599b89d7c-p5d9m" podUID="ae0049f6-86e1-43c6-b675-21b3e853802c" containerName="agent" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Jul 01 17:13:19.584253 ip-10-0-137-244 kubenswrapper[2565]: I0701 17:13:19.584211    2565 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-logger-predictor-599b89d7c-p5d9m" podUID="ae0049f6-86e1-43c6-b675-21b3e853802c" containerName="kserve-container" probeResult="failure" output="dial tcp 10.133.0.28:8080: connect: connection refused"
Jul 01 17:13:19.584763 ip-10-0-137-244 kubenswrapper[2565]: I0701 17:13:19.584732    2565 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-logger-predictor-599b89d7c-p5d9m" podUID="ae0049f6-86e1-43c6-b675-21b3e853802c" containerName="agent" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Jul 01 17:13:29.584032 ip-10-0-137-244 kubenswrapper[2565]: I0701 17:13:29.583974    2565 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-logger-predictor-599b89d7c-p5d9m" podUID="ae0049f6-86e1-43c6-b675-21b3e853802c" containerName="kserve-container" probeResult="failure" output="dial tcp 10.133.0.28:8080: connect: connection refused"
Jul 01 17:13:29.584483 ip-10-0-137-244 kubenswrapper[2565]: I0701 17:13:29.584460    2565 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-logger-predictor-599b89d7c-p5d9m" podUID="ae0049f6-86e1-43c6-b675-21b3e853802c" containerName="agent" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Jul 01 17:13:39.584015 ip-10-0-137-244 kubenswrapper[2565]: I0701 17:13:39.583971    2565 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-logger-predictor-599b89d7c-p5d9m" podUID="ae0049f6-86e1-43c6-b675-21b3e853802c" containerName="kserve-container" probeResult="failure" output="dial tcp 10.133.0.28:8080: connect: connection refused"
Jul 01 17:13:39.584486 ip-10-0-137-244 kubenswrapper[2565]: I0701 17:13:39.584457    2565 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-logger-predictor-599b89d7c-p5d9m" podUID="ae0049f6-86e1-43c6-b675-21b3e853802c" containerName="agent" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Jul 01 17:13:49.583730 ip-10-0-137-244 kubenswrapper[2565]: I0701 17:13:49.583625    2565 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-logger-predictor-599b89d7c-p5d9m" podUID="ae0049f6-86e1-43c6-b675-21b3e853802c" containerName="kserve-container" probeResult="failure" output="dial tcp 10.133.0.28:8080: connect: connection refused"
Jul 01 17:13:49.584206 ip-10-0-137-244 kubenswrapper[2565]: I0701 17:13:49.584071    2565 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-logger-predictor-599b89d7c-p5d9m" podUID="ae0049f6-86e1-43c6-b675-21b3e853802c" containerName="agent" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Jul 01 17:13:50.167592 ip-10-0-137-244 kubenswrapper[2565]: I0701 17:13:50.167566    2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-zjsw4_bb7ebc32-d0cd-4eb0-b998-3ce298f73809/ovn-acl-logging/0.log"
Jul 01 17:13:50.168147 ip-10-0-137-244 kubenswrapper[2565]: I0701 17:13:50.168130    2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-zjsw4_bb7ebc32-d0cd-4eb0-b998-3ce298f73809/ovn-acl-logging/0.log"
Jul 01 17:13:59.583807 ip-10-0-137-244 kubenswrapper[2565]: I0701 17:13:59.583762    2565 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-logger-predictor-599b89d7c-p5d9m" podUID="ae0049f6-86e1-43c6-b675-21b3e853802c" containerName="kserve-container" probeResult="failure" output="dial tcp 10.133.0.28:8080: connect: connection refused"
Jul 01 17:13:59.584332 ip-10-0-137-244 kubenswrapper[2565]: I0701 17:13:59.584307    2565 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-logger-predictor-599b89d7c-p5d9m" podUID="ae0049f6-86e1-43c6-b675-21b3e853802c" containerName="agent" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Jul 01 17:14:05.244284 ip-10-0-137-244 kubenswrapper[2565]: I0701 17:14:05.244241    2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kserve-ci-e2e-test/isvc-logger-predictor-599b89d7c-p5d9m"
Jul 01 17:14:05.244757 ip-10-0-137-244 kubenswrapper[2565]: I0701 17:14:05.244469    2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kserve-ci-e2e-test/isvc-logger-predictor-599b89d7c-p5d9m"
Jul 01 17:14:12.425387 ip-10-0-137-244 kubenswrapper[2565]: I0701 17:14:12.425362    2565 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_message-dumper-predictor-6575bcfdfd-mn29k_3c3bda14-25ec-4abd-b396-4e783e462da4/kserve-container/0.log"
Jul 01 17:14:12.623790 ip-10-0-137-244 kubenswrapper[2565]: I0701 17:14:12.623754    2565 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kserve-ci-e2e-test/isvc-logger-predictor-599b89d7c-p5d9m"]
Jul 01 17:14:12.624090 ip-10-0-137-244 kubenswrapper[2565]: I0701 17:14:12.624059    2565 kuberuntime_container.go:872] "Killing container with a grace period" pod="kserve-ci-e2e-test/isvc-logger-predictor-599b89d7c-p5d9m" podUID="ae0049f6-86e1-43c6-b675-21b3e853802c" containerName="kserve-container" containerID="cri-o://2c01b56be107417636fb20d1bc799b2f0018f36d25831eae3d8d33718d40d60e" gracePeriod=30
Jul 01 17:14:12.624211 ip-10-0-137-244 kubenswrapper[2565]: I0701 17:14:12.624086    2565 kuberuntime_container.go:872] "Killing container with a grace period" pod="kserve-ci-e2e-test/isvc-logger-predictor-599b89d7c-p5d9m" podUID="ae0049f6-86e1-43c6-b675-21b3e853802c" containerName="agent" containerID="cri-o://f178e5776aaa499faaa8f5b985045b554847cd9d3a6478a65f1a8cbc3c40f660" gracePeriod=30
Jul 01 17:14:12.624293 ip-10-0-137-244 kubenswrapper[2565]: I0701 17:14:12.624228    2565 kuberuntime_container.go:872] "Killing container with a grace period" pod="kserve-ci-e2e-test/isvc-logger-predictor-599b89d7c-p5d9m" podUID="ae0049f6-86e1-43c6-b675-21b3e853802c" containerName="kube-rbac-proxy" containerID="cri-o://620764bf68c6753d91d25903ca3f4f5c11e2626888da17d6732945e624cc20fb" gracePeriod=30
Jul 01 17:14:12.650899 ip-10-0-137-244 kubenswrapper[2565]: I0701 17:14:12.650874    2565 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kserve-ci-e2e-test/isvc-autogluon-v1-predictor-7bbbbd79cd-4gb6q"]
Jul 01 17:14:12.651185 ip-10-0-137-244 kubenswrapper[2565]: I0701 17:14:12.651173    2565 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f7c7e394-4643-49ef-b2ca-dff32f41ff0b" containerName="kserve-container"
Jul 01 17:14:12.651276 ip-10-0-137-244 kubenswrapper[2565]: I0701 17:14:12.651187    2565 state_mem.go:107] "Deleted CPUSet assignment" podUID="f7c7e394-4643-49ef-b2ca-dff32f41ff0b" containerName="kserve-container"
Jul 01 17:14:12.651276 ip-10-0-137-244 kubenswrapper[2565]: I0701 17:14:12.651199    2565 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f7c7e394-4643-49ef-b2ca-dff32f41ff0b" containerName="storage-initializer"
Jul 01 17:14:12.651276 ip-10-0-137-244 kubenswrapper[2565]: I0701 17:14:12.651205    2565 state_mem.go:107] "Deleted CPUSet assignment" podUID="f7c7e394-4643-49ef-b2ca-dff32f41ff0b" containerName="storage-initializer"
Jul 01 17:14:12.651276 ip-10-0-137-244 kubenswrapper[2565]: I0701 17:14:12.651211    2565 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f7c7e394-4643-49ef-b2ca-dff32f41ff0b" containerName="kube-rbac-proxy"
Jul 01 17:14:12.651276 ip-10-0-137-244 kubenswrapper[2565]: I0701 17:14:12.651217    2565 state_mem.go:107] "Deleted CPUSet assignment" podUID="f7c7e394-4643-49ef-b2ca-dff32f41ff0b" containerName="kube-rbac-proxy"
Jul 01 17:14:12.651276 ip-10-0-137-244 kubenswrapper[2565]: I0701 17:14:12.651223    2565 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f7c7e394-4643-49ef-b2ca-dff32f41ff0b" containerName="agent"
Jul 01 17:14:12.651276 ip-10-0-137-244 kubenswrapper[2565]: I0701 17:14:12.651228    2565 state_mem.go:107] "Deleted CPUSet assignment" podUID="f7c7e394-4643-49ef-b2ca-dff32f41ff0b" containerName="agent"
Jul 01 17:14:12.651473 ip-10-0-137-244 kubenswrapper[2565]: I0701 17:14:12.651280    2565 memory_manager.go:368] "RemoveStaleState removing state" podUID="f7c7e394-4643-49ef-b2ca-dff32f41ff0b" containerName="kube-rbac-proxy"
Jul 01 17:14:12.651473 ip-10-0-137-244 kubenswrapper[2565]: I0701 17:14:12.651288    2565 memory_manager.go:368] "RemoveStaleState removing state" podUID="f7c7e394-4643-49ef-b2ca-dff32f41ff0b" containerName="kserve-container"
Jul 01 17:14:12.651473 ip-10-0-137-244 kubenswrapper[2565]: I0701 17:14:12.651294    2565 memory_manager.go:368] "RemoveStaleState removing state" podUID="f7c7e394-4643-49ef-b2ca-dff32f41ff0b" containerName="agent"
Jul 01 17:14:12.654376 ip-10-0-137-244 kubenswrapper[2565]: I0701 17:14:12.654358    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/isvc-autogluon-v1-predictor-7bbbbd79cd-4gb6q"
Jul 01 17:14:12.656735 ip-10-0-137-244 kubenswrapper[2565]: I0701 17:14:12.656718    2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kserve-ci-e2e-test\"/\"isvc-autogluon-v1-predictor-serving-cert\""
Jul 01 17:14:12.656824 ip-10-0-137-244 kubenswrapper[2565]: I0701 17:14:12.656772    2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kserve-ci-e2e-test\"/\"isvc-autogluon-v1-kube-rbac-proxy-sar-config\""
Jul 01 17:14:12.667199 ip-10-0-137-244 kubenswrapper[2565]: I0701 17:14:12.667177    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/isvc-autogluon-v1-predictor-7bbbbd79cd-4gb6q"]
Jul 01 17:14:12.722588 ip-10-0-137-244 kubenswrapper[2565]: I0701 17:14:12.722530    2565 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kserve-ci-e2e-test/message-dumper-predictor-6575bcfdfd-mn29k"]
Jul 01 17:14:12.722729 ip-10-0-137-244 kubenswrapper[2565]: I0701 17:14:12.722709    2565 kuberuntime_container.go:872] "Killing container with a grace period" pod="kserve-ci-e2e-test/message-dumper-predictor-6575bcfdfd-mn29k" podUID="3c3bda14-25ec-4abd-b396-4e783e462da4" containerName="kserve-container" containerID="cri-o://03acf446bfc0051f0c558339de9f6b35ee1507314036de97697e10a5bca1480e" gracePeriod=30
Jul 01 17:14:12.722783 ip-10-0-137-244 kubenswrapper[2565]: I0701 17:14:12.722744    2565 kuberuntime_container.go:872] "Killing container with a grace period" pod="kserve-ci-e2e-test/message-dumper-predictor-6575bcfdfd-mn29k" podUID="3c3bda14-25ec-4abd-b396-4e783e462da4" containerName="kube-rbac-proxy" containerID="cri-o://d55a29db64b82f2a8c900b2fe4d88b33943d7ea7ea0cff203a6582c6cee23546" gracePeriod=30
Jul 01 17:14:12.796421 ip-10-0-137-244 kubenswrapper[2565]: I0701 17:14:12.796401    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-47mzx\" (UniqueName: \"kubernetes.io/projected/c52be57c-46ef-4e73-9607-611146f0b731-kube-api-access-47mzx\") pod \"isvc-autogluon-v1-predictor-7bbbbd79cd-4gb6q\" (UID: \"c52be57c-46ef-4e73-9607-611146f0b731\") " pod="kserve-ci-e2e-test/isvc-autogluon-v1-predictor-7bbbbd79cd-4gb6q"
Jul 01 17:14:12.796526 ip-10-0-137-244 kubenswrapper[2565]: I0701 17:14:12.796428    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cabundle-cert\" (UniqueName: \"kubernetes.io/configmap/c52be57c-46ef-4e73-9607-611146f0b731-cabundle-cert\") pod \"isvc-autogluon-v1-predictor-7bbbbd79cd-4gb6q\" (UID: \"c52be57c-46ef-4e73-9607-611146f0b731\") " pod="kserve-ci-e2e-test/isvc-autogluon-v1-predictor-7bbbbd79cd-4gb6q"
Jul 01 17:14:12.796526 ip-10-0-137-244 kubenswrapper[2565]: I0701 17:14:12.796464    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/c52be57c-46ef-4e73-9607-611146f0b731-kserve-provision-location\") pod \"isvc-autogluon-v1-predictor-7bbbbd79cd-4gb6q\" (UID: \"c52be57c-46ef-4e73-9607-611146f0b731\") " pod="kserve-ci-e2e-test/isvc-autogluon-v1-predictor-7bbbbd79cd-4gb6q"
Jul 01 17:14:12.796601 ip-10-0-137-244 kubenswrapper[2565]: I0701 17:14:12.796528    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/c52be57c-46ef-4e73-9607-611146f0b731-kube-rbac-proxy-sar-config\") pod \"isvc-autogluon-v1-predictor-7bbbbd79cd-4gb6q\" (UID: \"c52be57c-46ef-4e73-9607-611146f0b731\") " pod="kserve-ci-e2e-test/isvc-autogluon-v1-predictor-7bbbbd79cd-4gb6q"
Jul 01 17:14:12.796641 ip-10-0-137-244 kubenswrapper[2565]: I0701 17:14:12.796589    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/c52be57c-46ef-4e73-9607-611146f0b731-proxy-tls\") pod \"isvc-autogluon-v1-predictor-7bbbbd79cd-4gb6q\" (UID: \"c52be57c-46ef-4e73-9607-611146f0b731\") " pod="kserve-ci-e2e-test/isvc-autogluon-v1-predictor-7bbbbd79cd-4gb6q"
Jul 01 17:14:12.897481 ip-10-0-137-244 kubenswrapper[2565]: I0701 17:14:12.897454    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/c52be57c-46ef-4e73-9607-611146f0b731-kube-rbac-proxy-sar-config\") pod \"isvc-autogluon-v1-predictor-7bbbbd79cd-4gb6q\" (UID: \"c52be57c-46ef-4e73-9607-611146f0b731\") " pod="kserve-ci-e2e-test/isvc-autogluon-v1-predictor-7bbbbd79cd-4gb6q"
Jul 01 17:14:12.897613 ip-10-0-137-244 kubenswrapper[2565]: I0701 17:14:12.897492    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/c52be57c-46ef-4e73-9607-611146f0b731-proxy-tls\") pod \"isvc-autogluon-v1-predictor-7bbbbd79cd-4gb6q\" (UID: \"c52be57c-46ef-4e73-9607-611146f0b731\") " pod="kserve-ci-e2e-test/isvc-autogluon-v1-predictor-7bbbbd79cd-4gb6q"
Jul 01 17:14:12.897613 ip-10-0-137-244 kubenswrapper[2565]: I0701 17:14:12.897518    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-47mzx\" (UniqueName: \"kubernetes.io/projected/c52be57c-46ef-4e73-9607-611146f0b731-kube-api-access-47mzx\") pod \"isvc-autogluon-v1-predictor-7bbbbd79cd-4gb6q\" (UID: \"c52be57c-46ef-4e73-9607-611146f0b731\") " pod="kserve-ci-e2e-test/isvc-autogluon-v1-predictor-7bbbbd79cd-4gb6q"
Jul 01 17:14:12.897613 ip-10-0-137-244 kubenswrapper[2565]: I0701 17:14:12.897605    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cabundle-cert\" (UniqueName: \"kubernetes.io/configmap/c52be57c-46ef-4e73-9607-611146f0b731-cabundle-cert\") pod \"isvc-autogluon-v1-predictor-7bbbbd79cd-4gb6q\" (UID: \"c52be57c-46ef-4e73-9607-611146f0b731\") " pod="kserve-ci-e2e-test/isvc-autogluon-v1-predictor-7bbbbd79cd-4gb6q"
Jul 01 17:14:12.897780 ip-10-0-137-244 kubenswrapper[2565]: I0701 17:14:12.897683    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/c52be57c-46ef-4e73-9607-611146f0b731-kserve-provision-location\") pod \"isvc-autogluon-v1-predictor-7bbbbd79cd-4gb6q\" (UID: \"c52be57c-46ef-4e73-9607-611146f0b731\") " pod="kserve-ci-e2e-test/isvc-autogluon-v1-predictor-7bbbbd79cd-4gb6q"
Jul 01 17:14:12.898124 ip-10-0-137-244 kubenswrapper[2565]: I0701 17:14:12.898095    2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/c52be57c-46ef-4e73-9607-611146f0b731-kserve-provision-location\") pod \"isvc-autogluon-v1-predictor-7bbbbd79cd-4gb6q\" (UID: \"c52be57c-46ef-4e73-9607-611146f0b731\") " pod="kserve-ci-e2e-test/isvc-autogluon-v1-predictor-7bbbbd79cd-4gb6q"
Jul 01 17:14:12.898332 ip-10-0-137-244 kubenswrapper[2565]: I0701 17:14:12.898316    2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cabundle-cert\" (UniqueName: \"kubernetes.io/configmap/c52be57c-46ef-4e73-9607-611146f0b731-cabundle-cert\") pod \"isvc-autogluon-v1-predictor-7bbbbd79cd-4gb6q\" (UID: \"c52be57c-46ef-4e73-9607-611146f0b731\") " pod="kserve-ci-e2e-test/isvc-autogluon-v1-predictor-7bbbbd79cd-4gb6q"
Jul 01 17:14:12.898406 ip-10-0-137-244 kubenswrapper[2565]: I0701 17:14:12.898328    2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/c52be57c-46ef-4e73-9607-611146f0b731-kube-rbac-proxy-sar-config\") pod \"isvc-autogluon-v1-predictor-7bbbbd79cd-4gb6q\" (UID: \"c52be57c-46ef-4e73-9607-611146f0b731\") " pod="kserve-ci-e2e-test/isvc-autogluon-v1-predictor-7bbbbd79cd-4gb6q"
Jul 01 17:14:12.900286 ip-10-0-137-244 kubenswrapper[2565]: I0701 17:14:12.900261    2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/c52be57c-46ef-4e73-9607-611146f0b731-proxy-tls\") pod \"isvc-autogluon-v1-predictor-7bbbbd79cd-4gb6q\" (UID: \"c52be57c-46ef-4e73-9607-611146f0b731\") " pod="kserve-ci-e2e-test/isvc-autogluon-v1-predictor-7bbbbd79cd-4gb6q"
Jul 01 17:14:12.905677 ip-10-0-137-244 kubenswrapper[2565]: I0701 17:14:12.905648    2565 generic.go:356] "Generic (PLEG): container finished" podID="ae0049f6-86e1-43c6-b675-21b3e853802c" containerID="620764bf68c6753d91d25903ca3f4f5c11e2626888da17d6732945e624cc20fb" exitCode=2
Jul 01 17:14:12.905829 ip-10-0-137-244 kubenswrapper[2565]: I0701 17:14:12.905789    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/isvc-logger-predictor-599b89d7c-p5d9m" event={"ID":"ae0049f6-86e1-43c6-b675-21b3e853802c","Type":"ContainerDied","Data":"620764bf68c6753d91d25903ca3f4f5c11e2626888da17d6732945e624cc20fb"}
Jul 01 17:14:12.907716 ip-10-0-137-244 kubenswrapper[2565]: I0701 17:14:12.907660    2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-47mzx\" (UniqueName: \"kubernetes.io/projected/c52be57c-46ef-4e73-9607-611146f0b731-kube-api-access-47mzx\") pod \"isvc-autogluon-v1-predictor-7bbbbd79cd-4gb6q\" (UID: \"c52be57c-46ef-4e73-9607-611146f0b731\") " pod="kserve-ci-e2e-test/isvc-autogluon-v1-predictor-7bbbbd79cd-4gb6q"
Jul 01 17:14:12.909626 ip-10-0-137-244 kubenswrapper[2565]: I0701 17:14:12.909601    2565 generic.go:356] "Generic (PLEG): container finished" podID="3c3bda14-25ec-4abd-b396-4e783e462da4" containerID="d55a29db64b82f2a8c900b2fe4d88b33943d7ea7ea0cff203a6582c6cee23546" exitCode=2
Jul 01 17:14:12.909731 ip-10-0-137-244 kubenswrapper[2565]: I0701 17:14:12.909630    2565 generic.go:356] "Generic (PLEG): container finished" podID="3c3bda14-25ec-4abd-b396-4e783e462da4" containerID="03acf446bfc0051f0c558339de9f6b35ee1507314036de97697e10a5bca1480e" exitCode=2
Jul 01 17:14:12.909731 ip-10-0-137-244 kubenswrapper[2565]: I0701 17:14:12.909660    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/message-dumper-predictor-6575bcfdfd-mn29k" event={"ID":"3c3bda14-25ec-4abd-b396-4e783e462da4","Type":"ContainerDied","Data":"d55a29db64b82f2a8c900b2fe4d88b33943d7ea7ea0cff203a6582c6cee23546"}
Jul 01 17:14:12.909731 ip-10-0-137-244 kubenswrapper[2565]: I0701 17:14:12.909709    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/message-dumper-predictor-6575bcfdfd-mn29k" event={"ID":"3c3bda14-25ec-4abd-b396-4e783e462da4","Type":"ContainerDied","Data":"03acf446bfc0051f0c558339de9f6b35ee1507314036de97697e10a5bca1480e"}
Jul 01 17:14:12.959502 ip-10-0-137-244 kubenswrapper[2565]: I0701 17:14:12.959483    2565 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/message-dumper-predictor-6575bcfdfd-mn29k"
Jul 01 17:14:12.963870 ip-10-0-137-244 kubenswrapper[2565]: I0701 17:14:12.963853    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/isvc-autogluon-v1-predictor-7bbbbd79cd-4gb6q"
Jul 01 17:14:13.090691 ip-10-0-137-244 kubenswrapper[2565]: I0701 17:14:13.090525    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/isvc-autogluon-v1-predictor-7bbbbd79cd-4gb6q"]
Jul 01 17:14:13.092751 ip-10-0-137-244 kubenswrapper[2565]: W0701 17:14:13.092722    2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc52be57c_46ef_4e73_9607_611146f0b731.slice/crio-722adb72bc6fe15d602f21ab9fb5edd51ce6d8c366decd36e0a73e162f3dcc96 WatchSource:0}: Error finding container 722adb72bc6fe15d602f21ab9fb5edd51ce6d8c366decd36e0a73e162f3dcc96: Status 404 returned error can't find the container with id 722adb72bc6fe15d602f21ab9fb5edd51ce6d8c366decd36e0a73e162f3dcc96
Jul 01 17:14:13.099288 ip-10-0-137-244 kubenswrapper[2565]: I0701 17:14:13.099194    2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/3c3bda14-25ec-4abd-b396-4e783e462da4-proxy-tls\") pod \"3c3bda14-25ec-4abd-b396-4e783e462da4\" (UID: \"3c3bda14-25ec-4abd-b396-4e783e462da4\") "
Jul 01 17:14:13.099288 ip-10-0-137-244 kubenswrapper[2565]: I0701 17:14:13.099232    2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-ncpxn\" (UniqueName: \"kubernetes.io/projected/3c3bda14-25ec-4abd-b396-4e783e462da4-kube-api-access-ncpxn\") pod \"3c3bda14-25ec-4abd-b396-4e783e462da4\" (UID: \"3c3bda14-25ec-4abd-b396-4e783e462da4\") "
Jul 01 17:14:13.099288 ip-10-0-137-244 kubenswrapper[2565]: I0701 17:14:13.099268    2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/3c3bda14-25ec-4abd-b396-4e783e462da4-kube-rbac-proxy-sar-config\") pod \"3c3bda14-25ec-4abd-b396-4e783e462da4\" (UID: \"3c3bda14-25ec-4abd-b396-4e783e462da4\") "
Jul 01 17:14:13.099709 ip-10-0-137-244 kubenswrapper[2565]: I0701 17:14:13.099683    2565 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/3c3bda14-25ec-4abd-b396-4e783e462da4-kube-rbac-proxy-sar-config" (OuterVolumeSpecName: "kube-rbac-proxy-sar-config") pod "3c3bda14-25ec-4abd-b396-4e783e462da4" (UID: "3c3bda14-25ec-4abd-b396-4e783e462da4"). InnerVolumeSpecName "kube-rbac-proxy-sar-config". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 01 17:14:13.101211 ip-10-0-137-244 kubenswrapper[2565]: I0701 17:14:13.101188    2565 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/3c3bda14-25ec-4abd-b396-4e783e462da4-proxy-tls" (OuterVolumeSpecName: "proxy-tls") pod "3c3bda14-25ec-4abd-b396-4e783e462da4" (UID: "3c3bda14-25ec-4abd-b396-4e783e462da4"). InnerVolumeSpecName "proxy-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 01 17:14:13.101316 ip-10-0-137-244 kubenswrapper[2565]: I0701 17:14:13.101227    2565 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/3c3bda14-25ec-4abd-b396-4e783e462da4-kube-api-access-ncpxn" (OuterVolumeSpecName: "kube-api-access-ncpxn") pod "3c3bda14-25ec-4abd-b396-4e783e462da4" (UID: "3c3bda14-25ec-4abd-b396-4e783e462da4"). InnerVolumeSpecName "kube-api-access-ncpxn". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 01 17:14:13.199932 ip-10-0-137-244 kubenswrapper[2565]: I0701 17:14:13.199908    2565 reconciler_common.go:299] "Volume detached for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/3c3bda14-25ec-4abd-b396-4e783e462da4-proxy-tls\") on node \"ip-10-0-137-244.ec2.internal\" DevicePath \"\""
Jul 01 17:14:13.199932 ip-10-0-137-244 kubenswrapper[2565]: I0701 17:14:13.199930    2565 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-ncpxn\" (UniqueName: \"kubernetes.io/projected/3c3bda14-25ec-4abd-b396-4e783e462da4-kube-api-access-ncpxn\") on node \"ip-10-0-137-244.ec2.internal\" DevicePath \"\""
Jul 01 17:14:13.200080 ip-10-0-137-244 kubenswrapper[2565]: I0701 17:14:13.199940    2565 reconciler_common.go:299] "Volume detached for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/3c3bda14-25ec-4abd-b396-4e783e462da4-kube-rbac-proxy-sar-config\") on node \"ip-10-0-137-244.ec2.internal\" DevicePath \"\""
Jul 01 17:14:13.915424 ip-10-0-137-244 kubenswrapper[2565]: I0701 17:14:13.915358    2565 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/message-dumper-predictor-6575bcfdfd-mn29k"
Jul 01 17:14:13.915424 ip-10-0-137-244 kubenswrapper[2565]: I0701 17:14:13.915393    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/message-dumper-predictor-6575bcfdfd-mn29k" event={"ID":"3c3bda14-25ec-4abd-b396-4e783e462da4","Type":"ContainerDied","Data":"423d733845660b75159fa7469292a7e45353d89ceaa643b6d69a783b8f3d57d4"}
Jul 01 17:14:13.915864 ip-10-0-137-244 kubenswrapper[2565]: I0701 17:14:13.915475    2565 scope.go:117] "RemoveContainer" containerID="d55a29db64b82f2a8c900b2fe4d88b33943d7ea7ea0cff203a6582c6cee23546"
Jul 01 17:14:13.917236 ip-10-0-137-244 kubenswrapper[2565]: I0701 17:14:13.917212    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/isvc-autogluon-v1-predictor-7bbbbd79cd-4gb6q" event={"ID":"c52be57c-46ef-4e73-9607-611146f0b731","Type":"ContainerStarted","Data":"28207887372e9d6cd231faf048bad9b335850151ebab20fa0f558c96043166c0"}
Jul 01 17:14:13.917359 ip-10-0-137-244 kubenswrapper[2565]: I0701 17:14:13.917266    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/isvc-autogluon-v1-predictor-7bbbbd79cd-4gb6q" event={"ID":"c52be57c-46ef-4e73-9607-611146f0b731","Type":"ContainerStarted","Data":"722adb72bc6fe15d602f21ab9fb5edd51ce6d8c366decd36e0a73e162f3dcc96"}
Jul 01 17:14:13.924341 ip-10-0-137-244 kubenswrapper[2565]: I0701 17:14:13.924260    2565 scope.go:117] "RemoveContainer" containerID="03acf446bfc0051f0c558339de9f6b35ee1507314036de97697e10a5bca1480e"
Jul 01 17:14:13.951876 ip-10-0-137-244 kubenswrapper[2565]: I0701 17:14:13.951852    2565 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kserve-ci-e2e-test/message-dumper-predictor-6575bcfdfd-mn29k"]
Jul 01 17:14:13.955471 ip-10-0-137-244 kubenswrapper[2565]: I0701 17:14:13.955450    2565 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kserve-ci-e2e-test/message-dumper-predictor-6575bcfdfd-mn29k"]
Jul 01 17:14:14.248792 ip-10-0-137-244 kubenswrapper[2565]: I0701 17:14:14.248726    2565 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="3c3bda14-25ec-4abd-b396-4e783e462da4" path="/var/lib/kubelet/pods/3c3bda14-25ec-4abd-b396-4e783e462da4/volumes"
Jul 01 17:14:14.579081 ip-10-0-137-244 kubenswrapper[2565]: I0701 17:14:14.578974    2565 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-logger-predictor-599b89d7c-p5d9m" podUID="ae0049f6-86e1-43c6-b675-21b3e853802c" containerName="kube-rbac-proxy" probeResult="failure" output="Get \"https://10.133.0.28:8643/healthz\": dial tcp 10.133.0.28:8643: connect: connection refused"
Jul 01 17:14:15.244540 ip-10-0-137-244 kubenswrapper[2565]: I0701 17:14:15.244498    2565 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-logger-predictor-599b89d7c-p5d9m" podUID="ae0049f6-86e1-43c6-b675-21b3e853802c" containerName="kserve-container" probeResult="failure" output="dial tcp 10.133.0.28:8080: connect: connection refused"
Jul 01 17:14:15.244952 ip-10-0-137-244 kubenswrapper[2565]: I0701 17:14:15.244837    2565 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-logger-predictor-599b89d7c-p5d9m" podUID="ae0049f6-86e1-43c6-b675-21b3e853802c" containerName="agent" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Jul 01 17:14:16.933514 ip-10-0-137-244 kubenswrapper[2565]: I0701 17:14:16.933480    2565 generic.go:356] "Generic (PLEG): container finished" podID="ae0049f6-86e1-43c6-b675-21b3e853802c" containerID="2c01b56be107417636fb20d1bc799b2f0018f36d25831eae3d8d33718d40d60e" exitCode=0
Jul 01 17:14:16.933845 ip-10-0-137-244 kubenswrapper[2565]: I0701 17:14:16.933564    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/isvc-logger-predictor-599b89d7c-p5d9m" event={"ID":"ae0049f6-86e1-43c6-b675-21b3e853802c","Type":"ContainerDied","Data":"2c01b56be107417636fb20d1bc799b2f0018f36d25831eae3d8d33718d40d60e"}
Jul 01 17:14:19.578850 ip-10-0-137-244 kubenswrapper[2565]: I0701 17:14:19.578809    2565 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-logger-predictor-599b89d7c-p5d9m" podUID="ae0049f6-86e1-43c6-b675-21b3e853802c" containerName="kube-rbac-proxy" probeResult="failure" output="Get \"https://10.133.0.28:8643/healthz\": dial tcp 10.133.0.28:8643: connect: connection refused"
Jul 01 17:14:24.578814 ip-10-0-137-244 kubenswrapper[2565]: I0701 17:14:24.578762    2565 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-logger-predictor-599b89d7c-p5d9m" podUID="ae0049f6-86e1-43c6-b675-21b3e853802c" containerName="kube-rbac-proxy" probeResult="failure" output="Get \"https://10.133.0.28:8643/healthz\": dial tcp 10.133.0.28:8643: connect: connection refused"
Jul 01 17:14:24.579344 ip-10-0-137-244 kubenswrapper[2565]: I0701 17:14:24.578903    2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve-ci-e2e-test/isvc-logger-predictor-599b89d7c-p5d9m"
Jul 01 17:14:25.243939 ip-10-0-137-244 kubenswrapper[2565]: I0701 17:14:25.243886    2565 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-logger-predictor-599b89d7c-p5d9m" podUID="ae0049f6-86e1-43c6-b675-21b3e853802c" containerName="kserve-container" probeResult="failure" output="dial tcp 10.133.0.28:8080: connect: connection refused"
Jul 01 17:14:25.244349 ip-10-0-137-244 kubenswrapper[2565]: I0701 17:14:25.244316    2565 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-logger-predictor-599b89d7c-p5d9m" podUID="ae0049f6-86e1-43c6-b675-21b3e853802c" containerName="agent" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Jul 01 17:14:29.579297 ip-10-0-137-244 kubenswrapper[2565]: I0701 17:14:29.579254    2565 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-logger-predictor-599b89d7c-p5d9m" podUID="ae0049f6-86e1-43c6-b675-21b3e853802c" containerName="kube-rbac-proxy" probeResult="failure" output="Get \"https://10.133.0.28:8643/healthz\": dial tcp 10.133.0.28:8643: connect: connection refused"
Jul 01 17:14:34.578979 ip-10-0-137-244 kubenswrapper[2565]: I0701 17:14:34.578927    2565 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-logger-predictor-599b89d7c-p5d9m" podUID="ae0049f6-86e1-43c6-b675-21b3e853802c" containerName="kube-rbac-proxy" probeResult="failure" output="Get \"https://10.133.0.28:8643/healthz\": dial tcp 10.133.0.28:8643: connect: connection refused"
Jul 01 17:14:35.244148 ip-10-0-137-244 kubenswrapper[2565]: I0701 17:14:35.244098    2565 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-logger-predictor-599b89d7c-p5d9m" podUID="ae0049f6-86e1-43c6-b675-21b3e853802c" containerName="kserve-container" probeResult="failure" output="dial tcp 10.133.0.28:8080: connect: connection refused"
Jul 01 17:14:35.244451 ip-10-0-137-244 kubenswrapper[2565]: I0701 17:14:35.244279    2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve-ci-e2e-test/isvc-logger-predictor-599b89d7c-p5d9m"
Jul 01 17:14:35.244530 ip-10-0-137-244 kubenswrapper[2565]: I0701 17:14:35.244500    2565 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-logger-predictor-599b89d7c-p5d9m" podUID="ae0049f6-86e1-43c6-b675-21b3e853802c" containerName="agent" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Jul 01 17:14:35.244667 ip-10-0-137-244 kubenswrapper[2565]: I0701 17:14:35.244651    2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve-ci-e2e-test/isvc-logger-predictor-599b89d7c-p5d9m"
Jul 01 17:14:39.578824 ip-10-0-137-244 kubenswrapper[2565]: I0701 17:14:39.578766    2565 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-logger-predictor-599b89d7c-p5d9m" podUID="ae0049f6-86e1-43c6-b675-21b3e853802c" containerName="kube-rbac-proxy" probeResult="failure" output="Get \"https://10.133.0.28:8643/healthz\": dial tcp 10.133.0.28:8643: connect: connection refused"
Jul 01 17:14:42.776487 ip-10-0-137-244 kubenswrapper[2565]: I0701 17:14:42.776467    2565 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/isvc-logger-predictor-599b89d7c-p5d9m"
Jul 01 17:14:42.903889 ip-10-0-137-244 kubenswrapper[2565]: I0701 17:14:42.903830    2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/ae0049f6-86e1-43c6-b675-21b3e853802c-kserve-provision-location\") pod \"ae0049f6-86e1-43c6-b675-21b3e853802c\" (UID: \"ae0049f6-86e1-43c6-b675-21b3e853802c\") "
Jul 01 17:14:42.903889 ip-10-0-137-244 kubenswrapper[2565]: I0701 17:14:42.903887    2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/ae0049f6-86e1-43c6-b675-21b3e853802c-kube-rbac-proxy-sar-config\") pod \"ae0049f6-86e1-43c6-b675-21b3e853802c\" (UID: \"ae0049f6-86e1-43c6-b675-21b3e853802c\") "
Jul 01 17:14:42.904070 ip-10-0-137-244 kubenswrapper[2565]: I0701 17:14:42.903920    2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"cabundle-cert\" (UniqueName: \"kubernetes.io/configmap/ae0049f6-86e1-43c6-b675-21b3e853802c-cabundle-cert\") pod \"ae0049f6-86e1-43c6-b675-21b3e853802c\" (UID: \"ae0049f6-86e1-43c6-b675-21b3e853802c\") "
Jul 01 17:14:42.904070 ip-10-0-137-244 kubenswrapper[2565]: I0701 17:14:42.903945    2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/ae0049f6-86e1-43c6-b675-21b3e853802c-proxy-tls\") pod \"ae0049f6-86e1-43c6-b675-21b3e853802c\" (UID: \"ae0049f6-86e1-43c6-b675-21b3e853802c\") "
Jul 01 17:14:42.904070 ip-10-0-137-244 kubenswrapper[2565]: I0701 17:14:42.903973    2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-jnc2f\" (UniqueName: \"kubernetes.io/projected/ae0049f6-86e1-43c6-b675-21b3e853802c-kube-api-access-jnc2f\") pod \"ae0049f6-86e1-43c6-b675-21b3e853802c\" (UID: \"ae0049f6-86e1-43c6-b675-21b3e853802c\") "
Jul 01 17:14:42.904225 ip-10-0-137-244 kubenswrapper[2565]: I0701 17:14:42.904146    2565 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ae0049f6-86e1-43c6-b675-21b3e853802c-kserve-provision-location" (OuterVolumeSpecName: "kserve-provision-location") pod "ae0049f6-86e1-43c6-b675-21b3e853802c" (UID: "ae0049f6-86e1-43c6-b675-21b3e853802c"). InnerVolumeSpecName "kserve-provision-location". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 01 17:14:42.904279 ip-10-0-137-244 kubenswrapper[2565]: I0701 17:14:42.904255    2565 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/ae0049f6-86e1-43c6-b675-21b3e853802c-kube-rbac-proxy-sar-config" (OuterVolumeSpecName: "kube-rbac-proxy-sar-config") pod "ae0049f6-86e1-43c6-b675-21b3e853802c" (UID: "ae0049f6-86e1-43c6-b675-21b3e853802c"). InnerVolumeSpecName "kube-rbac-proxy-sar-config". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 01 17:14:42.904337 ip-10-0-137-244 kubenswrapper[2565]: I0701 17:14:42.904291    2565 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/ae0049f6-86e1-43c6-b675-21b3e853802c-cabundle-cert" (OuterVolumeSpecName: "cabundle-cert") pod "ae0049f6-86e1-43c6-b675-21b3e853802c" (UID: "ae0049f6-86e1-43c6-b675-21b3e853802c"). InnerVolumeSpecName "cabundle-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 01 17:14:42.906069 ip-10-0-137-244 kubenswrapper[2565]: I0701 17:14:42.906046    2565 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/ae0049f6-86e1-43c6-b675-21b3e853802c-kube-api-access-jnc2f" (OuterVolumeSpecName: "kube-api-access-jnc2f") pod "ae0049f6-86e1-43c6-b675-21b3e853802c" (UID: "ae0049f6-86e1-43c6-b675-21b3e853802c"). InnerVolumeSpecName "kube-api-access-jnc2f". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 01 17:14:42.906168 ip-10-0-137-244 kubenswrapper[2565]: I0701 17:14:42.906067    2565 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/ae0049f6-86e1-43c6-b675-21b3e853802c-proxy-tls" (OuterVolumeSpecName: "proxy-tls") pod "ae0049f6-86e1-43c6-b675-21b3e853802c" (UID: "ae0049f6-86e1-43c6-b675-21b3e853802c"). InnerVolumeSpecName "proxy-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 01 17:14:43.004625 ip-10-0-137-244 kubenswrapper[2565]: I0701 17:14:43.004603    2565 reconciler_common.go:299] "Volume detached for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/ae0049f6-86e1-43c6-b675-21b3e853802c-kube-rbac-proxy-sar-config\") on node \"ip-10-0-137-244.ec2.internal\" DevicePath \"\""
Jul 01 17:14:43.004625 ip-10-0-137-244 kubenswrapper[2565]: I0701 17:14:43.004625    2565 reconciler_common.go:299] "Volume detached for volume \"cabundle-cert\" (UniqueName: \"kubernetes.io/configmap/ae0049f6-86e1-43c6-b675-21b3e853802c-cabundle-cert\") on node \"ip-10-0-137-244.ec2.internal\" DevicePath \"\""
Jul 01 17:14:43.004752 ip-10-0-137-244 kubenswrapper[2565]: I0701 17:14:43.004635    2565 reconciler_common.go:299] "Volume detached for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/ae0049f6-86e1-43c6-b675-21b3e853802c-proxy-tls\") on node \"ip-10-0-137-244.ec2.internal\" DevicePath \"\""
Jul 01 17:14:43.004752 ip-10-0-137-244 kubenswrapper[2565]: I0701 17:14:43.004643    2565 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-jnc2f\" (UniqueName: \"kubernetes.io/projected/ae0049f6-86e1-43c6-b675-21b3e853802c-kube-api-access-jnc2f\") on node \"ip-10-0-137-244.ec2.internal\" DevicePath \"\""
Jul 01 17:14:43.004752 ip-10-0-137-244 kubenswrapper[2565]: I0701 17:14:43.004651    2565 reconciler_common.go:299] "Volume detached for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/ae0049f6-86e1-43c6-b675-21b3e853802c-kserve-provision-location\") on node \"ip-10-0-137-244.ec2.internal\" DevicePath \"\""
Jul 01 17:14:43.051527 ip-10-0-137-244 kubenswrapper[2565]: I0701 17:14:43.051498    2565 generic.go:356] "Generic (PLEG): container finished" podID="ae0049f6-86e1-43c6-b675-21b3e853802c" containerID="f178e5776aaa499faaa8f5b985045b554847cd9d3a6478a65f1a8cbc3c40f660" exitCode=0
Jul 01 17:14:43.051638 ip-10-0-137-244 kubenswrapper[2565]: I0701 17:14:43.051551    2565 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/isvc-logger-predictor-599b89d7c-p5d9m"
Jul 01 17:14:43.051638 ip-10-0-137-244 kubenswrapper[2565]: I0701 17:14:43.051601    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/isvc-logger-predictor-599b89d7c-p5d9m" event={"ID":"ae0049f6-86e1-43c6-b675-21b3e853802c","Type":"ContainerDied","Data":"f178e5776aaa499faaa8f5b985045b554847cd9d3a6478a65f1a8cbc3c40f660"}
Jul 01 17:14:43.051763 ip-10-0-137-244 kubenswrapper[2565]: I0701 17:14:43.051679    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/isvc-logger-predictor-599b89d7c-p5d9m" event={"ID":"ae0049f6-86e1-43c6-b675-21b3e853802c","Type":"ContainerDied","Data":"187806f25885dd073167ae7418cd1ebabf88999a3f29dd6645915cd7cd20a871"}
Jul 01 17:14:43.051763 ip-10-0-137-244 kubenswrapper[2565]: I0701 17:14:43.051738    2565 scope.go:117] "RemoveContainer" containerID="f178e5776aaa499faaa8f5b985045b554847cd9d3a6478a65f1a8cbc3c40f660"
Jul 01 17:14:43.053334 ip-10-0-137-244 kubenswrapper[2565]: I0701 17:14:43.053311    2565 generic.go:356] "Generic (PLEG): container finished" podID="c52be57c-46ef-4e73-9607-611146f0b731" containerID="28207887372e9d6cd231faf048bad9b335850151ebab20fa0f558c96043166c0" exitCode=0
Jul 01 17:14:43.053440 ip-10-0-137-244 kubenswrapper[2565]: I0701 17:14:43.053349    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/isvc-autogluon-v1-predictor-7bbbbd79cd-4gb6q" event={"ID":"c52be57c-46ef-4e73-9607-611146f0b731","Type":"ContainerDied","Data":"28207887372e9d6cd231faf048bad9b335850151ebab20fa0f558c96043166c0"}
Jul 01 17:14:43.061465 ip-10-0-137-244 kubenswrapper[2565]: I0701 17:14:43.061445    2565 scope.go:117] "RemoveContainer" containerID="620764bf68c6753d91d25903ca3f4f5c11e2626888da17d6732945e624cc20fb"
Jul 01 17:14:43.068917 ip-10-0-137-244 kubenswrapper[2565]: I0701 17:14:43.068903    2565 scope.go:117] "RemoveContainer" containerID="2c01b56be107417636fb20d1bc799b2f0018f36d25831eae3d8d33718d40d60e"
Jul 01 17:14:43.076523 ip-10-0-137-244 kubenswrapper[2565]: I0701 17:14:43.076502    2565 scope.go:117] "RemoveContainer" containerID="e4a9754848d7593fc5ec226fec4d38e5cb46c3c475e8fcd3653aed2b48365e24"
Jul 01 17:14:43.083691 ip-10-0-137-244 kubenswrapper[2565]: I0701 17:14:43.083652    2565 scope.go:117] "RemoveContainer" containerID="f178e5776aaa499faaa8f5b985045b554847cd9d3a6478a65f1a8cbc3c40f660"
Jul 01 17:14:43.083986 ip-10-0-137-244 kubenswrapper[2565]: E0701 17:14:43.083966    2565 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"f178e5776aaa499faaa8f5b985045b554847cd9d3a6478a65f1a8cbc3c40f660\": container with ID starting with f178e5776aaa499faaa8f5b985045b554847cd9d3a6478a65f1a8cbc3c40f660 not found: ID does not exist" containerID="f178e5776aaa499faaa8f5b985045b554847cd9d3a6478a65f1a8cbc3c40f660"
Jul 01 17:14:43.084063 ip-10-0-137-244 kubenswrapper[2565]: I0701 17:14:43.083993    2565 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"f178e5776aaa499faaa8f5b985045b554847cd9d3a6478a65f1a8cbc3c40f660"} err="failed to get container status \"f178e5776aaa499faaa8f5b985045b554847cd9d3a6478a65f1a8cbc3c40f660\": rpc error: code = NotFound desc = could not find container \"f178e5776aaa499faaa8f5b985045b554847cd9d3a6478a65f1a8cbc3c40f660\": container with ID starting with f178e5776aaa499faaa8f5b985045b554847cd9d3a6478a65f1a8cbc3c40f660 not found: ID does not exist"
Jul 01 17:14:43.084063 ip-10-0-137-244 kubenswrapper[2565]: I0701 17:14:43.084010    2565 scope.go:117] "RemoveContainer" containerID="620764bf68c6753d91d25903ca3f4f5c11e2626888da17d6732945e624cc20fb"
Jul 01 17:14:43.084305 ip-10-0-137-244 kubenswrapper[2565]: E0701 17:14:43.084289    2565 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"620764bf68c6753d91d25903ca3f4f5c11e2626888da17d6732945e624cc20fb\": container with ID starting with 620764bf68c6753d91d25903ca3f4f5c11e2626888da17d6732945e624cc20fb not found: ID does not exist" containerID="620764bf68c6753d91d25903ca3f4f5c11e2626888da17d6732945e624cc20fb"
Jul 01 17:14:43.084361 ip-10-0-137-244 kubenswrapper[2565]: I0701 17:14:43.084312    2565 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"620764bf68c6753d91d25903ca3f4f5c11e2626888da17d6732945e624cc20fb"} err="failed to get container status \"620764bf68c6753d91d25903ca3f4f5c11e2626888da17d6732945e624cc20fb\": rpc error: code = NotFound desc = could not find container \"620764bf68c6753d91d25903ca3f4f5c11e2626888da17d6732945e624cc20fb\": container with ID starting with 620764bf68c6753d91d25903ca3f4f5c11e2626888da17d6732945e624cc20fb not found: ID does not exist"
Jul 01 17:14:43.084361 ip-10-0-137-244 kubenswrapper[2565]: I0701 17:14:43.084331    2565 scope.go:117] "RemoveContainer" containerID="2c01b56be107417636fb20d1bc799b2f0018f36d25831eae3d8d33718d40d60e"
Jul 01 17:14:43.084549 ip-10-0-137-244 kubenswrapper[2565]: E0701 17:14:43.084533    2565 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"2c01b56be107417636fb20d1bc799b2f0018f36d25831eae3d8d33718d40d60e\": container with ID starting with 2c01b56be107417636fb20d1bc799b2f0018f36d25831eae3d8d33718d40d60e not found: ID does not exist" containerID="2c01b56be107417636fb20d1bc799b2f0018f36d25831eae3d8d33718d40d60e"
Jul 01 17:14:43.084594 ip-10-0-137-244 kubenswrapper[2565]: I0701 17:14:43.084552    2565 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"2c01b56be107417636fb20d1bc799b2f0018f36d25831eae3d8d33718d40d60e"} err="failed to get container status \"2c01b56be107417636fb20d1bc799b2f0018f36d25831eae3d8d33718d40d60e\": rpc error: code = NotFound desc = could not find container \"2c01b56be107417636fb20d1bc799b2f0018f36d25831eae3d8d33718d40d60e\": container with ID starting with 2c01b56be107417636fb20d1bc799b2f0018f36d25831eae3d8d33718d40d60e not found: ID does not exist"
Jul 01 17:14:43.084594 ip-10-0-137-244 kubenswrapper[2565]: I0701 17:14:43.084565    2565 scope.go:117] "RemoveContainer" containerID="e4a9754848d7593fc5ec226fec4d38e5cb46c3c475e8fcd3653aed2b48365e24"
Jul 01 17:14:43.084762 ip-10-0-137-244 kubenswrapper[2565]: E0701 17:14:43.084747    2565 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"e4a9754848d7593fc5ec226fec4d38e5cb46c3c475e8fcd3653aed2b48365e24\": container with ID starting with e4a9754848d7593fc5ec226fec4d38e5cb46c3c475e8fcd3653aed2b48365e24 not found: ID does not exist" containerID="e4a9754848d7593fc5ec226fec4d38e5cb46c3c475e8fcd3653aed2b48365e24"
Jul 01 17:14:43.084804 ip-10-0-137-244 kubenswrapper[2565]: I0701 17:14:43.084765    2565 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e4a9754848d7593fc5ec226fec4d38e5cb46c3c475e8fcd3653aed2b48365e24"} err="failed to get container status \"e4a9754848d7593fc5ec226fec4d38e5cb46c3c475e8fcd3653aed2b48365e24\": rpc error: code = NotFound desc = could not find container \"e4a9754848d7593fc5ec226fec4d38e5cb46c3c475e8fcd3653aed2b48365e24\": container with ID starting with e4a9754848d7593fc5ec226fec4d38e5cb46c3c475e8fcd3653aed2b48365e24 not found: ID does not exist"
Jul 01 17:14:43.089282 ip-10-0-137-244 kubenswrapper[2565]: I0701 17:14:43.089262    2565 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kserve-ci-e2e-test/isvc-logger-predictor-599b89d7c-p5d9m"]
Jul 01 17:14:43.093672 ip-10-0-137-244 kubenswrapper[2565]: I0701 17:14:43.093653    2565 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kserve-ci-e2e-test/isvc-logger-predictor-599b89d7c-p5d9m"]
Jul 01 17:14:44.249956 ip-10-0-137-244 kubenswrapper[2565]: I0701 17:14:44.249926    2565 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="ae0049f6-86e1-43c6-b675-21b3e853802c" path="/var/lib/kubelet/pods/ae0049f6-86e1-43c6-b675-21b3e853802c/volumes"
Jul 01 17:17:09.670987 ip-10-0-137-244 kubenswrapper[2565]: I0701 17:17:09.670956    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/isvc-autogluon-v1-predictor-7bbbbd79cd-4gb6q" event={"ID":"c52be57c-46ef-4e73-9607-611146f0b731","Type":"ContainerStarted","Data":"8bae2e620ec59d13bb53fad0c2f5ae8d8f67bac24c91a3914fd958938bcf9f69"}
Jul 01 17:17:09.671497 ip-10-0-137-244 kubenswrapper[2565]: I0701 17:17:09.671009    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/isvc-autogluon-v1-predictor-7bbbbd79cd-4gb6q" event={"ID":"c52be57c-46ef-4e73-9607-611146f0b731","Type":"ContainerStarted","Data":"4f22d776f09fbc1af1742f556aa6f7ed47634e3acd511b5f7dc4b384f4fd80ae"}
Jul 01 17:17:09.671497 ip-10-0-137-244 kubenswrapper[2565]: I0701 17:17:09.671127    2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve-ci-e2e-test/isvc-autogluon-v1-predictor-7bbbbd79cd-4gb6q"
Jul 01 17:17:09.702888 ip-10-0-137-244 kubenswrapper[2565]: I0701 17:17:09.702831    2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kserve-ci-e2e-test/isvc-autogluon-v1-predictor-7bbbbd79cd-4gb6q" podStartSLOduration=2.557821145 podStartE2EDuration="2m57.702811692s" totalImagesPullingTime="2m25.588887526s" totalInitContainerRuntime="29.556103021s" isStatefulPod=false podCreationTimestamp="2026-07-01 17:14:12 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-01 17:17:09.701337261 +0000 UTC m=+1100.169601385" watchObservedRunningTime="2026-07-01 17:17:09.702811692 +0000 UTC m=+1100.171075816"
Jul 01 17:17:10.679048 ip-10-0-137-244 kubenswrapper[2565]: I0701 17:17:10.677415    2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve-ci-e2e-test/isvc-autogluon-v1-predictor-7bbbbd79cd-4gb6q"
Jul 01 17:17:10.681588 ip-10-0-137-244 kubenswrapper[2565]: I0701 17:17:10.681544    2565 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-autogluon-v1-predictor-7bbbbd79cd-4gb6q" podUID="c52be57c-46ef-4e73-9607-611146f0b731" containerName="kserve-container" probeResult="failure" output="dial tcp 10.133.0.29:8080: connect: connection refused"
Jul 01 17:17:11.681086 ip-10-0-137-244 kubenswrapper[2565]: I0701 17:17:11.681048    2565 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-autogluon-v1-predictor-7bbbbd79cd-4gb6q" podUID="c52be57c-46ef-4e73-9607-611146f0b731" containerName="kserve-container" probeResult="failure" output="dial tcp 10.133.0.29:8080: connect: connection refused"
Jul 01 17:17:16.685613 ip-10-0-137-244 kubenswrapper[2565]: I0701 17:17:16.685585    2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kserve-ci-e2e-test/isvc-autogluon-v1-predictor-7bbbbd79cd-4gb6q"
Jul 01 17:17:16.686182 ip-10-0-137-244 kubenswrapper[2565]: I0701 17:17:16.686039    2565 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-autogluon-v1-predictor-7bbbbd79cd-4gb6q" podUID="c52be57c-46ef-4e73-9607-611146f0b731" containerName="kserve-container" probeResult="failure" output="dial tcp 10.133.0.29:8080: connect: connection refused"
Jul 01 17:17:26.687158 ip-10-0-137-244 kubenswrapper[2565]: I0701 17:17:26.687124    2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kserve-ci-e2e-test/isvc-autogluon-v1-predictor-7bbbbd79cd-4gb6q"
Jul 01 17:17:32.931380 ip-10-0-137-244 kubenswrapper[2565]: I0701 17:17:32.931348    2565 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kserve-ci-e2e-test/isvc-autogluon-v1-predictor-7bbbbd79cd-4gb6q"]
Jul 01 17:17:32.931758 ip-10-0-137-244 kubenswrapper[2565]: I0701 17:17:32.931631    2565 kuberuntime_container.go:872] "Killing container with a grace period" pod="kserve-ci-e2e-test/isvc-autogluon-v1-predictor-7bbbbd79cd-4gb6q" podUID="c52be57c-46ef-4e73-9607-611146f0b731" containerName="kube-rbac-proxy" containerID="cri-o://8bae2e620ec59d13bb53fad0c2f5ae8d8f67bac24c91a3914fd958938bcf9f69" gracePeriod=30
Jul 01 17:17:32.931758 ip-10-0-137-244 kubenswrapper[2565]: I0701 17:17:32.931631    2565 kuberuntime_container.go:872] "Killing container with a grace period" pod="kserve-ci-e2e-test/isvc-autogluon-v1-predictor-7bbbbd79cd-4gb6q" podUID="c52be57c-46ef-4e73-9607-611146f0b731" containerName="kserve-container" containerID="cri-o://4f22d776f09fbc1af1742f556aa6f7ed47634e3acd511b5f7dc4b384f4fd80ae" gracePeriod=30
Jul 01 17:17:33.704639 ip-10-0-137-244 kubenswrapper[2565]: I0701 17:17:33.704618    2565 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/isvc-autogluon-v1-predictor-7bbbbd79cd-4gb6q"
Jul 01 17:17:33.767209 ip-10-0-137-244 kubenswrapper[2565]: I0701 17:17:33.767146    2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/c52be57c-46ef-4e73-9607-611146f0b731-proxy-tls\") pod \"c52be57c-46ef-4e73-9607-611146f0b731\" (UID: \"c52be57c-46ef-4e73-9607-611146f0b731\") "
Jul 01 17:17:33.767323 ip-10-0-137-244 kubenswrapper[2565]: I0701 17:17:33.767208    2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"cabundle-cert\" (UniqueName: \"kubernetes.io/configmap/c52be57c-46ef-4e73-9607-611146f0b731-cabundle-cert\") pod \"c52be57c-46ef-4e73-9607-611146f0b731\" (UID: \"c52be57c-46ef-4e73-9607-611146f0b731\") "
Jul 01 17:17:33.767323 ip-10-0-137-244 kubenswrapper[2565]: I0701 17:17:33.767246    2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/c52be57c-46ef-4e73-9607-611146f0b731-kserve-provision-location\") pod \"c52be57c-46ef-4e73-9607-611146f0b731\" (UID: \"c52be57c-46ef-4e73-9607-611146f0b731\") "
Jul 01 17:17:33.767323 ip-10-0-137-244 kubenswrapper[2565]: I0701 17:17:33.767269    2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/c52be57c-46ef-4e73-9607-611146f0b731-kube-rbac-proxy-sar-config\") pod \"c52be57c-46ef-4e73-9607-611146f0b731\" (UID: \"c52be57c-46ef-4e73-9607-611146f0b731\") "
Jul 01 17:17:33.767323 ip-10-0-137-244 kubenswrapper[2565]: I0701 17:17:33.767289    2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-47mzx\" (UniqueName: \"kubernetes.io/projected/c52be57c-46ef-4e73-9607-611146f0b731-kube-api-access-47mzx\") pod \"c52be57c-46ef-4e73-9607-611146f0b731\" (UID: \"c52be57c-46ef-4e73-9607-611146f0b731\") "
Jul 01 17:17:33.767623 ip-10-0-137-244 kubenswrapper[2565]: I0701 17:17:33.767600    2565 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/c52be57c-46ef-4e73-9607-611146f0b731-cabundle-cert" (OuterVolumeSpecName: "cabundle-cert") pod "c52be57c-46ef-4e73-9607-611146f0b731" (UID: "c52be57c-46ef-4e73-9607-611146f0b731"). InnerVolumeSpecName "cabundle-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 01 17:17:33.767760 ip-10-0-137-244 kubenswrapper[2565]: I0701 17:17:33.767720    2565 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/c52be57c-46ef-4e73-9607-611146f0b731-kube-rbac-proxy-sar-config" (OuterVolumeSpecName: "kube-rbac-proxy-sar-config") pod "c52be57c-46ef-4e73-9607-611146f0b731" (UID: "c52be57c-46ef-4e73-9607-611146f0b731"). InnerVolumeSpecName "kube-rbac-proxy-sar-config". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 01 17:17:33.767915 ip-10-0-137-244 kubenswrapper[2565]: I0701 17:17:33.767888    2565 generic.go:356] "Generic (PLEG): container finished" podID="c52be57c-46ef-4e73-9607-611146f0b731" containerID="8bae2e620ec59d13bb53fad0c2f5ae8d8f67bac24c91a3914fd958938bcf9f69" exitCode=2
Jul 01 17:17:33.767978 ip-10-0-137-244 kubenswrapper[2565]: I0701 17:17:33.767918    2565 generic.go:356] "Generic (PLEG): container finished" podID="c52be57c-46ef-4e73-9607-611146f0b731" containerID="4f22d776f09fbc1af1742f556aa6f7ed47634e3acd511b5f7dc4b384f4fd80ae" exitCode=0
Jul 01 17:17:33.767978 ip-10-0-137-244 kubenswrapper[2565]: I0701 17:17:33.767950    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/isvc-autogluon-v1-predictor-7bbbbd79cd-4gb6q" event={"ID":"c52be57c-46ef-4e73-9607-611146f0b731","Type":"ContainerDied","Data":"8bae2e620ec59d13bb53fad0c2f5ae8d8f67bac24c91a3914fd958938bcf9f69"}
Jul 01 17:17:33.767978 ip-10-0-137-244 kubenswrapper[2565]: I0701 17:17:33.767958    2565 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/isvc-autogluon-v1-predictor-7bbbbd79cd-4gb6q"
Jul 01 17:17:33.768142 ip-10-0-137-244 kubenswrapper[2565]: I0701 17:17:33.768036    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/isvc-autogluon-v1-predictor-7bbbbd79cd-4gb6q" event={"ID":"c52be57c-46ef-4e73-9607-611146f0b731","Type":"ContainerDied","Data":"4f22d776f09fbc1af1742f556aa6f7ed47634e3acd511b5f7dc4b384f4fd80ae"}
Jul 01 17:17:33.768142 ip-10-0-137-244 kubenswrapper[2565]: I0701 17:17:33.768050    2565 scope.go:117] "RemoveContainer" containerID="8bae2e620ec59d13bb53fad0c2f5ae8d8f67bac24c91a3914fd958938bcf9f69"
Jul 01 17:17:33.768142 ip-10-0-137-244 kubenswrapper[2565]: I0701 17:17:33.768075    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/isvc-autogluon-v1-predictor-7bbbbd79cd-4gb6q" event={"ID":"c52be57c-46ef-4e73-9607-611146f0b731","Type":"ContainerDied","Data":"722adb72bc6fe15d602f21ab9fb5edd51ce6d8c366decd36e0a73e162f3dcc96"}
Jul 01 17:17:33.769580 ip-10-0-137-244 kubenswrapper[2565]: I0701 17:17:33.769548    2565 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/c52be57c-46ef-4e73-9607-611146f0b731-proxy-tls" (OuterVolumeSpecName: "proxy-tls") pod "c52be57c-46ef-4e73-9607-611146f0b731" (UID: "c52be57c-46ef-4e73-9607-611146f0b731"). InnerVolumeSpecName "proxy-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 01 17:17:33.769667 ip-10-0-137-244 kubenswrapper[2565]: I0701 17:17:33.769614    2565 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/c52be57c-46ef-4e73-9607-611146f0b731-kube-api-access-47mzx" (OuterVolumeSpecName: "kube-api-access-47mzx") pod "c52be57c-46ef-4e73-9607-611146f0b731" (UID: "c52be57c-46ef-4e73-9607-611146f0b731"). InnerVolumeSpecName "kube-api-access-47mzx". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 01 17:17:33.771946 ip-10-0-137-244 kubenswrapper[2565]: I0701 17:17:33.771922    2565 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c52be57c-46ef-4e73-9607-611146f0b731-kserve-provision-location" (OuterVolumeSpecName: "kserve-provision-location") pod "c52be57c-46ef-4e73-9607-611146f0b731" (UID: "c52be57c-46ef-4e73-9607-611146f0b731"). InnerVolumeSpecName "kserve-provision-location". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 01 17:17:33.778480 ip-10-0-137-244 kubenswrapper[2565]: I0701 17:17:33.778465    2565 scope.go:117] "RemoveContainer" containerID="4f22d776f09fbc1af1742f556aa6f7ed47634e3acd511b5f7dc4b384f4fd80ae"
Jul 01 17:17:33.785055 ip-10-0-137-244 kubenswrapper[2565]: I0701 17:17:33.785038    2565 scope.go:117] "RemoveContainer" containerID="28207887372e9d6cd231faf048bad9b335850151ebab20fa0f558c96043166c0"
Jul 01 17:17:33.791546 ip-10-0-137-244 kubenswrapper[2565]: I0701 17:17:33.791533    2565 scope.go:117] "RemoveContainer" containerID="8bae2e620ec59d13bb53fad0c2f5ae8d8f67bac24c91a3914fd958938bcf9f69"
Jul 01 17:17:33.791799 ip-10-0-137-244 kubenswrapper[2565]: E0701 17:17:33.791782    2565 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"8bae2e620ec59d13bb53fad0c2f5ae8d8f67bac24c91a3914fd958938bcf9f69\": container with ID starting with 8bae2e620ec59d13bb53fad0c2f5ae8d8f67bac24c91a3914fd958938bcf9f69 not found: ID does not exist" containerID="8bae2e620ec59d13bb53fad0c2f5ae8d8f67bac24c91a3914fd958938bcf9f69"
Jul 01 17:17:33.791868 ip-10-0-137-244 kubenswrapper[2565]: I0701 17:17:33.791810    2565 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"8bae2e620ec59d13bb53fad0c2f5ae8d8f67bac24c91a3914fd958938bcf9f69"} err="failed to get container status \"8bae2e620ec59d13bb53fad0c2f5ae8d8f67bac24c91a3914fd958938bcf9f69\": rpc error: code = NotFound desc = could not find container \"8bae2e620ec59d13bb53fad0c2f5ae8d8f67bac24c91a3914fd958938bcf9f69\": container with ID starting with 8bae2e620ec59d13bb53fad0c2f5ae8d8f67bac24c91a3914fd958938bcf9f69 not found: ID does not exist"
Jul 01 17:17:33.791868 ip-10-0-137-244 kubenswrapper[2565]: I0701 17:17:33.791834    2565 scope.go:117] "RemoveContainer" containerID="4f22d776f09fbc1af1742f556aa6f7ed47634e3acd511b5f7dc4b384f4fd80ae"
Jul 01 17:17:33.792094 ip-10-0-137-244 kubenswrapper[2565]: E0701 17:17:33.792075    2565 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"4f22d776f09fbc1af1742f556aa6f7ed47634e3acd511b5f7dc4b384f4fd80ae\": container with ID starting with 4f22d776f09fbc1af1742f556aa6f7ed47634e3acd511b5f7dc4b384f4fd80ae not found: ID does not exist" containerID="4f22d776f09fbc1af1742f556aa6f7ed47634e3acd511b5f7dc4b384f4fd80ae"
Jul 01 17:17:33.792179 ip-10-0-137-244 kubenswrapper[2565]: I0701 17:17:33.792097    2565 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"4f22d776f09fbc1af1742f556aa6f7ed47634e3acd511b5f7dc4b384f4fd80ae"} err="failed to get container status \"4f22d776f09fbc1af1742f556aa6f7ed47634e3acd511b5f7dc4b384f4fd80ae\": rpc error: code = NotFound desc = could not find container \"4f22d776f09fbc1af1742f556aa6f7ed47634e3acd511b5f7dc4b384f4fd80ae\": container with ID starting with 4f22d776f09fbc1af1742f556aa6f7ed47634e3acd511b5f7dc4b384f4fd80ae not found: ID does not exist"
Jul 01 17:17:33.792179 ip-10-0-137-244 kubenswrapper[2565]: I0701 17:17:33.792113    2565 scope.go:117] "RemoveContainer" containerID="28207887372e9d6cd231faf048bad9b335850151ebab20fa0f558c96043166c0"
Jul 01 17:17:33.792338 ip-10-0-137-244 kubenswrapper[2565]: E0701 17:17:33.792322    2565 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"28207887372e9d6cd231faf048bad9b335850151ebab20fa0f558c96043166c0\": container with ID starting with 28207887372e9d6cd231faf048bad9b335850151ebab20fa0f558c96043166c0 not found: ID does not exist" containerID="28207887372e9d6cd231faf048bad9b335850151ebab20fa0f558c96043166c0"
Jul 01 17:17:33.792374 ip-10-0-137-244 kubenswrapper[2565]: I0701 17:17:33.792344    2565 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"28207887372e9d6cd231faf048bad9b335850151ebab20fa0f558c96043166c0"} err="failed to get container status \"28207887372e9d6cd231faf048bad9b335850151ebab20fa0f558c96043166c0\": rpc error: code = NotFound desc = could not find container \"28207887372e9d6cd231faf048bad9b335850151ebab20fa0f558c96043166c0\": container with ID starting with 28207887372e9d6cd231faf048bad9b335850151ebab20fa0f558c96043166c0 not found: ID does not exist"
Jul 01 17:17:33.792374 ip-10-0-137-244 kubenswrapper[2565]: I0701 17:17:33.792362    2565 scope.go:117] "RemoveContainer" containerID="8bae2e620ec59d13bb53fad0c2f5ae8d8f67bac24c91a3914fd958938bcf9f69"
Jul 01 17:17:33.792606 ip-10-0-137-244 kubenswrapper[2565]: I0701 17:17:33.792581    2565 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"8bae2e620ec59d13bb53fad0c2f5ae8d8f67bac24c91a3914fd958938bcf9f69"} err="failed to get container status \"8bae2e620ec59d13bb53fad0c2f5ae8d8f67bac24c91a3914fd958938bcf9f69\": rpc error: code = NotFound desc = could not find container \"8bae2e620ec59d13bb53fad0c2f5ae8d8f67bac24c91a3914fd958938bcf9f69\": container with ID starting with 8bae2e620ec59d13bb53fad0c2f5ae8d8f67bac24c91a3914fd958938bcf9f69 not found: ID does not exist"
Jul 01 17:17:33.792650 ip-10-0-137-244 kubenswrapper[2565]: I0701 17:17:33.792607    2565 scope.go:117] "RemoveContainer" containerID="4f22d776f09fbc1af1742f556aa6f7ed47634e3acd511b5f7dc4b384f4fd80ae"
Jul 01 17:17:33.792820 ip-10-0-137-244 kubenswrapper[2565]: I0701 17:17:33.792802    2565 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"4f22d776f09fbc1af1742f556aa6f7ed47634e3acd511b5f7dc4b384f4fd80ae"} err="failed to get container status \"4f22d776f09fbc1af1742f556aa6f7ed47634e3acd511b5f7dc4b384f4fd80ae\": rpc error: code = NotFound desc = could not find container \"4f22d776f09fbc1af1742f556aa6f7ed47634e3acd511b5f7dc4b384f4fd80ae\": container with ID starting with 4f22d776f09fbc1af1742f556aa6f7ed47634e3acd511b5f7dc4b384f4fd80ae not found: ID does not exist"
Jul 01 17:17:33.792862 ip-10-0-137-244 kubenswrapper[2565]: I0701 17:17:33.792821    2565 scope.go:117] "RemoveContainer" containerID="28207887372e9d6cd231faf048bad9b335850151ebab20fa0f558c96043166c0"
Jul 01 17:17:33.793016 ip-10-0-137-244 kubenswrapper[2565]: I0701 17:17:33.792998    2565 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"28207887372e9d6cd231faf048bad9b335850151ebab20fa0f558c96043166c0"} err="failed to get container status \"28207887372e9d6cd231faf048bad9b335850151ebab20fa0f558c96043166c0\": rpc error: code = NotFound desc = could not find container \"28207887372e9d6cd231faf048bad9b335850151ebab20fa0f558c96043166c0\": container with ID starting with 28207887372e9d6cd231faf048bad9b335850151ebab20fa0f558c96043166c0 not found: ID does not exist"
Jul 01 17:17:33.868354 ip-10-0-137-244 kubenswrapper[2565]: I0701 17:17:33.868335    2565 reconciler_common.go:299] "Volume detached for volume \"cabundle-cert\" (UniqueName: \"kubernetes.io/configmap/c52be57c-46ef-4e73-9607-611146f0b731-cabundle-cert\") on node \"ip-10-0-137-244.ec2.internal\" DevicePath \"\""
Jul 01 17:17:33.868354 ip-10-0-137-244 kubenswrapper[2565]: I0701 17:17:33.868354    2565 reconciler_common.go:299] "Volume detached for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/c52be57c-46ef-4e73-9607-611146f0b731-kserve-provision-location\") on node \"ip-10-0-137-244.ec2.internal\" DevicePath \"\""
Jul 01 17:17:33.868498 ip-10-0-137-244 kubenswrapper[2565]: I0701 17:17:33.868363    2565 reconciler_common.go:299] "Volume detached for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/c52be57c-46ef-4e73-9607-611146f0b731-kube-rbac-proxy-sar-config\") on node \"ip-10-0-137-244.ec2.internal\" DevicePath \"\""
Jul 01 17:17:33.868498 ip-10-0-137-244 kubenswrapper[2565]: I0701 17:17:33.868372    2565 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-47mzx\" (UniqueName: \"kubernetes.io/projected/c52be57c-46ef-4e73-9607-611146f0b731-kube-api-access-47mzx\") on node \"ip-10-0-137-244.ec2.internal\" DevicePath \"\""
Jul 01 17:17:33.868498 ip-10-0-137-244 kubenswrapper[2565]: I0701 17:17:33.868380    2565 reconciler_common.go:299] "Volume detached for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/c52be57c-46ef-4e73-9607-611146f0b731-proxy-tls\") on node \"ip-10-0-137-244.ec2.internal\" DevicePath \"\""
Jul 01 17:17:34.089499 ip-10-0-137-244 kubenswrapper[2565]: I0701 17:17:34.089474    2565 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kserve-ci-e2e-test/isvc-autogluon-v1-predictor-7bbbbd79cd-4gb6q"]
Jul 01 17:17:34.094062 ip-10-0-137-244 kubenswrapper[2565]: I0701 17:17:34.094041    2565 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kserve-ci-e2e-test/isvc-autogluon-v1-predictor-7bbbbd79cd-4gb6q"]
Jul 01 17:17:34.247790 ip-10-0-137-244 kubenswrapper[2565]: I0701 17:17:34.247766    2565 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="c52be57c-46ef-4e73-9607-611146f0b731" path="/var/lib/kubelet/pods/c52be57c-46ef-4e73-9607-611146f0b731/volumes"
Jul 01 17:17:38.231848 ip-10-0-137-244 kubenswrapper[2565]: I0701 17:17:38.231815    2565 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kserve-ci-e2e-test/isvc-autogluon-v2-predictor-67b849d456-bmr8g"]
Jul 01 17:17:38.232301 ip-10-0-137-244 kubenswrapper[2565]: I0701 17:17:38.232124    2565 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c52be57c-46ef-4e73-9607-611146f0b731" containerName="kserve-container"
Jul 01 17:17:38.232301 ip-10-0-137-244 kubenswrapper[2565]: I0701 17:17:38.232136    2565 state_mem.go:107] "Deleted CPUSet assignment" podUID="c52be57c-46ef-4e73-9607-611146f0b731" containerName="kserve-container"
Jul 01 17:17:38.232301 ip-10-0-137-244 kubenswrapper[2565]: I0701 17:17:38.232151    2565 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="3c3bda14-25ec-4abd-b396-4e783e462da4" containerName="kserve-container"
Jul 01 17:17:38.232301 ip-10-0-137-244 kubenswrapper[2565]: I0701 17:17:38.232156    2565 state_mem.go:107] "Deleted CPUSet assignment" podUID="3c3bda14-25ec-4abd-b396-4e783e462da4" containerName="kserve-container"
Jul 01 17:17:38.232301 ip-10-0-137-244 kubenswrapper[2565]: I0701 17:17:38.232163    2565 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c52be57c-46ef-4e73-9607-611146f0b731" containerName="storage-initializer"
Jul 01 17:17:38.232301 ip-10-0-137-244 kubenswrapper[2565]: I0701 17:17:38.232168    2565 state_mem.go:107] "Deleted CPUSet assignment" podUID="c52be57c-46ef-4e73-9607-611146f0b731" containerName="storage-initializer"
Jul 01 17:17:38.232301 ip-10-0-137-244 kubenswrapper[2565]: I0701 17:17:38.232174    2565 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c52be57c-46ef-4e73-9607-611146f0b731" containerName="kube-rbac-proxy"
Jul 01 17:17:38.232301 ip-10-0-137-244 kubenswrapper[2565]: I0701 17:17:38.232179    2565 state_mem.go:107] "Deleted CPUSet assignment" podUID="c52be57c-46ef-4e73-9607-611146f0b731" containerName="kube-rbac-proxy"
Jul 01 17:17:38.232301 ip-10-0-137-244 kubenswrapper[2565]: I0701 17:17:38.232190    2565 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ae0049f6-86e1-43c6-b675-21b3e853802c" containerName="kube-rbac-proxy"
Jul 01 17:17:38.232301 ip-10-0-137-244 kubenswrapper[2565]: I0701 17:17:38.232195    2565 state_mem.go:107] "Deleted CPUSet assignment" podUID="ae0049f6-86e1-43c6-b675-21b3e853802c" containerName="kube-rbac-proxy"
Jul 01 17:17:38.232301 ip-10-0-137-244 kubenswrapper[2565]: I0701 17:17:38.232201    2565 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="3c3bda14-25ec-4abd-b396-4e783e462da4" containerName="kube-rbac-proxy"
Jul 01 17:17:38.232301 ip-10-0-137-244 kubenswrapper[2565]: I0701 17:17:38.232206    2565 state_mem.go:107] "Deleted CPUSet assignment" podUID="3c3bda14-25ec-4abd-b396-4e783e462da4" containerName="kube-rbac-proxy"
Jul 01 17:17:38.232301 ip-10-0-137-244 kubenswrapper[2565]: I0701 17:17:38.232211    2565 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ae0049f6-86e1-43c6-b675-21b3e853802c" containerName="storage-initializer"
Jul 01 17:17:38.232301 ip-10-0-137-244 kubenswrapper[2565]: I0701 17:17:38.232216    2565 state_mem.go:107] "Deleted CPUSet assignment" podUID="ae0049f6-86e1-43c6-b675-21b3e853802c" containerName="storage-initializer"
Jul 01 17:17:38.232301 ip-10-0-137-244 kubenswrapper[2565]: I0701 17:17:38.232225    2565 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ae0049f6-86e1-43c6-b675-21b3e853802c" containerName="kserve-container"
Jul 01 17:17:38.232301 ip-10-0-137-244 kubenswrapper[2565]: I0701 17:17:38.232230    2565 state_mem.go:107] "Deleted CPUSet assignment" podUID="ae0049f6-86e1-43c6-b675-21b3e853802c" containerName="kserve-container"
Jul 01 17:17:38.232301 ip-10-0-137-244 kubenswrapper[2565]: I0701 17:17:38.232238    2565 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ae0049f6-86e1-43c6-b675-21b3e853802c" containerName="agent"
Jul 01 17:17:38.232301 ip-10-0-137-244 kubenswrapper[2565]: I0701 17:17:38.232243    2565 state_mem.go:107] "Deleted CPUSet assignment" podUID="ae0049f6-86e1-43c6-b675-21b3e853802c" containerName="agent"
Jul 01 17:17:38.232301 ip-10-0-137-244 kubenswrapper[2565]: I0701 17:17:38.232284    2565 memory_manager.go:368] "RemoveStaleState removing state" podUID="3c3bda14-25ec-4abd-b396-4e783e462da4" containerName="kube-rbac-proxy"
Jul 01 17:17:38.232301 ip-10-0-137-244 kubenswrapper[2565]: I0701 17:17:38.232292    2565 memory_manager.go:368] "RemoveStaleState removing state" podUID="ae0049f6-86e1-43c6-b675-21b3e853802c" containerName="kube-rbac-proxy"
Jul 01 17:17:38.232301 ip-10-0-137-244 kubenswrapper[2565]: I0701 17:17:38.232299    2565 memory_manager.go:368] "RemoveStaleState removing state" podUID="ae0049f6-86e1-43c6-b675-21b3e853802c" containerName="agent"
Jul 01 17:17:38.232301 ip-10-0-137-244 kubenswrapper[2565]: I0701 17:17:38.232304    2565 memory_manager.go:368] "RemoveStaleState removing state" podUID="c52be57c-46ef-4e73-9607-611146f0b731" containerName="kube-rbac-proxy"
Jul 01 17:17:38.232301 ip-10-0-137-244 kubenswrapper[2565]: I0701 17:17:38.232311    2565 memory_manager.go:368] "RemoveStaleState removing state" podUID="c52be57c-46ef-4e73-9607-611146f0b731" containerName="kserve-container"
Jul 01 17:17:38.232950 ip-10-0-137-244 kubenswrapper[2565]: I0701 17:17:38.232316    2565 memory_manager.go:368] "RemoveStaleState removing state" podUID="3c3bda14-25ec-4abd-b396-4e783e462da4" containerName="kserve-container"
Jul 01 17:17:38.232950 ip-10-0-137-244 kubenswrapper[2565]: I0701 17:17:38.232323    2565 memory_manager.go:368] "RemoveStaleState removing state" podUID="ae0049f6-86e1-43c6-b675-21b3e853802c" containerName="kserve-container"
Jul 01 17:17:38.262923 ip-10-0-137-244 kubenswrapper[2565]: I0701 17:17:38.262895    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/isvc-autogluon-v2-predictor-67b849d456-bmr8g"
Jul 01 17:17:38.265748 ip-10-0-137-244 kubenswrapper[2565]: I0701 17:17:38.265728    2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kserve-ci-e2e-test\"/\"isvc-autogluon-v2-predictor-serving-cert\""
Jul 01 17:17:38.268396 ip-10-0-137-244 kubenswrapper[2565]: I0701 17:17:38.268367    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/isvc-autogluon-v2-predictor-67b849d456-bmr8g"]
Jul 01 17:17:38.269462 ip-10-0-137-244 kubenswrapper[2565]: I0701 17:17:38.269438    2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kserve-ci-e2e-test\"/\"seaweedfs-s3-creds\""
Jul 01 17:17:38.269462 ip-10-0-137-244 kubenswrapper[2565]: I0701 17:17:38.269458    2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kserve-ci-e2e-test\"/\"openshift-service-ca.crt\""
Jul 01 17:17:38.269628 ip-10-0-137-244 kubenswrapper[2565]: I0701 17:17:38.269438    2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kserve-ci-e2e-test\"/\"isvc-autogluon-v2-kube-rbac-proxy-sar-config\""
Jul 01 17:17:38.269628 ip-10-0-137-244 kubenswrapper[2565]: I0701 17:17:38.269440    2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kserve-ci-e2e-test\"/\"kube-root-ca.crt\""
Jul 01 17:17:38.270327 ip-10-0-137-244 kubenswrapper[2565]: I0701 17:17:38.270309    2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kserve-ci-e2e-test\"/\"odh-kserve-custom-ca-bundle\""
Jul 01 17:17:38.270434 ip-10-0-137-244 kubenswrapper[2565]: I0701 17:17:38.270384    2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kserve-ci-e2e-test\"/\"default-dockercfg-bl7zk\""
Jul 01 17:17:38.401609 ip-10-0-137-244 kubenswrapper[2565]: I0701 17:17:38.401586    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/b242f7cd-ff9f-4f8c-a9df-2b6951290757-kserve-provision-location\") pod \"isvc-autogluon-v2-predictor-67b849d456-bmr8g\" (UID: \"b242f7cd-ff9f-4f8c-a9df-2b6951290757\") " pod="kserve-ci-e2e-test/isvc-autogluon-v2-predictor-67b849d456-bmr8g"
Jul 01 17:17:38.401749 ip-10-0-137-244 kubenswrapper[2565]: I0701 17:17:38.401621    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9mmdv\" (UniqueName: \"kubernetes.io/projected/b242f7cd-ff9f-4f8c-a9df-2b6951290757-kube-api-access-9mmdv\") pod \"isvc-autogluon-v2-predictor-67b849d456-bmr8g\" (UID: \"b242f7cd-ff9f-4f8c-a9df-2b6951290757\") " pod="kserve-ci-e2e-test/isvc-autogluon-v2-predictor-67b849d456-bmr8g"
Jul 01 17:17:38.401749 ip-10-0-137-244 kubenswrapper[2565]: I0701 17:17:38.401647    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/b242f7cd-ff9f-4f8c-a9df-2b6951290757-proxy-tls\") pod \"isvc-autogluon-v2-predictor-67b849d456-bmr8g\" (UID: \"b242f7cd-ff9f-4f8c-a9df-2b6951290757\") " pod="kserve-ci-e2e-test/isvc-autogluon-v2-predictor-67b849d456-bmr8g"
Jul 01 17:17:38.401749 ip-10-0-137-244 kubenswrapper[2565]: I0701 17:17:38.401673    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/b242f7cd-ff9f-4f8c-a9df-2b6951290757-kube-rbac-proxy-sar-config\") pod \"isvc-autogluon-v2-predictor-67b849d456-bmr8g\" (UID: \"b242f7cd-ff9f-4f8c-a9df-2b6951290757\") " pod="kserve-ci-e2e-test/isvc-autogluon-v2-predictor-67b849d456-bmr8g"
Jul 01 17:17:38.401749 ip-10-0-137-244 kubenswrapper[2565]: I0701 17:17:38.401701    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cabundle-cert\" (UniqueName: \"kubernetes.io/configmap/b242f7cd-ff9f-4f8c-a9df-2b6951290757-cabundle-cert\") pod \"isvc-autogluon-v2-predictor-67b849d456-bmr8g\" (UID: \"b242f7cd-ff9f-4f8c-a9df-2b6951290757\") " pod="kserve-ci-e2e-test/isvc-autogluon-v2-predictor-67b849d456-bmr8g"
Jul 01 17:17:38.502994 ip-10-0-137-244 kubenswrapper[2565]: I0701 17:17:38.502942    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/b242f7cd-ff9f-4f8c-a9df-2b6951290757-kserve-provision-location\") pod \"isvc-autogluon-v2-predictor-67b849d456-bmr8g\" (UID: \"b242f7cd-ff9f-4f8c-a9df-2b6951290757\") " pod="kserve-ci-e2e-test/isvc-autogluon-v2-predictor-67b849d456-bmr8g"
Jul 01 17:17:38.502994 ip-10-0-137-244 kubenswrapper[2565]: I0701 17:17:38.502977    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9mmdv\" (UniqueName: \"kubernetes.io/projected/b242f7cd-ff9f-4f8c-a9df-2b6951290757-kube-api-access-9mmdv\") pod \"isvc-autogluon-v2-predictor-67b849d456-bmr8g\" (UID: \"b242f7cd-ff9f-4f8c-a9df-2b6951290757\") " pod="kserve-ci-e2e-test/isvc-autogluon-v2-predictor-67b849d456-bmr8g"
Jul 01 17:17:38.503173 ip-10-0-137-244 kubenswrapper[2565]: I0701 17:17:38.503000    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/b242f7cd-ff9f-4f8c-a9df-2b6951290757-proxy-tls\") pod \"isvc-autogluon-v2-predictor-67b849d456-bmr8g\" (UID: \"b242f7cd-ff9f-4f8c-a9df-2b6951290757\") " pod="kserve-ci-e2e-test/isvc-autogluon-v2-predictor-67b849d456-bmr8g"
Jul 01 17:17:38.503173 ip-10-0-137-244 kubenswrapper[2565]: I0701 17:17:38.503017    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/b242f7cd-ff9f-4f8c-a9df-2b6951290757-kube-rbac-proxy-sar-config\") pod \"isvc-autogluon-v2-predictor-67b849d456-bmr8g\" (UID: \"b242f7cd-ff9f-4f8c-a9df-2b6951290757\") " pod="kserve-ci-e2e-test/isvc-autogluon-v2-predictor-67b849d456-bmr8g"
Jul 01 17:17:38.503173 ip-10-0-137-244 kubenswrapper[2565]: I0701 17:17:38.503057    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cabundle-cert\" (UniqueName: \"kubernetes.io/configmap/b242f7cd-ff9f-4f8c-a9df-2b6951290757-cabundle-cert\") pod \"isvc-autogluon-v2-predictor-67b849d456-bmr8g\" (UID: \"b242f7cd-ff9f-4f8c-a9df-2b6951290757\") " pod="kserve-ci-e2e-test/isvc-autogluon-v2-predictor-67b849d456-bmr8g"
Jul 01 17:17:38.503173 ip-10-0-137-244 kubenswrapper[2565]: E0701 17:17:38.503134    2565 secret.go:189] Couldn't get secret kserve-ci-e2e-test/isvc-autogluon-v2-predictor-serving-cert: secret "isvc-autogluon-v2-predictor-serving-cert" not found
Jul 01 17:17:38.503355 ip-10-0-137-244 kubenswrapper[2565]: E0701 17:17:38.503204    2565 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b242f7cd-ff9f-4f8c-a9df-2b6951290757-proxy-tls podName:b242f7cd-ff9f-4f8c-a9df-2b6951290757 nodeName:}" failed. No retries permitted until 2026-07-01 17:17:39.003182945 +0000 UTC m=+1129.471447054 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "proxy-tls" (UniqueName: "kubernetes.io/secret/b242f7cd-ff9f-4f8c-a9df-2b6951290757-proxy-tls") pod "isvc-autogluon-v2-predictor-67b849d456-bmr8g" (UID: "b242f7cd-ff9f-4f8c-a9df-2b6951290757") : secret "isvc-autogluon-v2-predictor-serving-cert" not found
Jul 01 17:17:38.503415 ip-10-0-137-244 kubenswrapper[2565]: I0701 17:17:38.503392    2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/b242f7cd-ff9f-4f8c-a9df-2b6951290757-kserve-provision-location\") pod \"isvc-autogluon-v2-predictor-67b849d456-bmr8g\" (UID: \"b242f7cd-ff9f-4f8c-a9df-2b6951290757\") " pod="kserve-ci-e2e-test/isvc-autogluon-v2-predictor-67b849d456-bmr8g"
Jul 01 17:17:38.503629 ip-10-0-137-244 kubenswrapper[2565]: I0701 17:17:38.503613    2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cabundle-cert\" (UniqueName: \"kubernetes.io/configmap/b242f7cd-ff9f-4f8c-a9df-2b6951290757-cabundle-cert\") pod \"isvc-autogluon-v2-predictor-67b849d456-bmr8g\" (UID: \"b242f7cd-ff9f-4f8c-a9df-2b6951290757\") " pod="kserve-ci-e2e-test/isvc-autogluon-v2-predictor-67b849d456-bmr8g"
Jul 01 17:17:38.503702 ip-10-0-137-244 kubenswrapper[2565]: I0701 17:17:38.503686    2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/b242f7cd-ff9f-4f8c-a9df-2b6951290757-kube-rbac-proxy-sar-config\") pod \"isvc-autogluon-v2-predictor-67b849d456-bmr8g\" (UID: \"b242f7cd-ff9f-4f8c-a9df-2b6951290757\") " pod="kserve-ci-e2e-test/isvc-autogluon-v2-predictor-67b849d456-bmr8g"
Jul 01 17:17:38.512614 ip-10-0-137-244 kubenswrapper[2565]: I0701 17:17:38.512589    2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-9mmdv\" (UniqueName: \"kubernetes.io/projected/b242f7cd-ff9f-4f8c-a9df-2b6951290757-kube-api-access-9mmdv\") pod \"isvc-autogluon-v2-predictor-67b849d456-bmr8g\" (UID: \"b242f7cd-ff9f-4f8c-a9df-2b6951290757\") " pod="kserve-ci-e2e-test/isvc-autogluon-v2-predictor-67b849d456-bmr8g"
Jul 01 17:17:39.007498 ip-10-0-137-244 kubenswrapper[2565]: I0701 17:17:39.007466    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/b242f7cd-ff9f-4f8c-a9df-2b6951290757-proxy-tls\") pod \"isvc-autogluon-v2-predictor-67b849d456-bmr8g\" (UID: \"b242f7cd-ff9f-4f8c-a9df-2b6951290757\") " pod="kserve-ci-e2e-test/isvc-autogluon-v2-predictor-67b849d456-bmr8g"
Jul 01 17:17:39.009987 ip-10-0-137-244 kubenswrapper[2565]: I0701 17:17:39.009962    2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/b242f7cd-ff9f-4f8c-a9df-2b6951290757-proxy-tls\") pod \"isvc-autogluon-v2-predictor-67b849d456-bmr8g\" (UID: \"b242f7cd-ff9f-4f8c-a9df-2b6951290757\") " pod="kserve-ci-e2e-test/isvc-autogluon-v2-predictor-67b849d456-bmr8g"
Jul 01 17:17:39.174378 ip-10-0-137-244 kubenswrapper[2565]: I0701 17:17:39.174353    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/isvc-autogluon-v2-predictor-67b849d456-bmr8g"
Jul 01 17:17:39.341405 ip-10-0-137-244 kubenswrapper[2565]: I0701 17:17:39.341373    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/isvc-autogluon-v2-predictor-67b849d456-bmr8g"]
Jul 01 17:17:39.345251 ip-10-0-137-244 kubenswrapper[2565]: W0701 17:17:39.345224    2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb242f7cd_ff9f_4f8c_a9df_2b6951290757.slice/crio-f7d20545e56be8d57967ce164ac136c2edad676dbcedd778322708e789ee2d53 WatchSource:0}: Error finding container f7d20545e56be8d57967ce164ac136c2edad676dbcedd778322708e789ee2d53: Status 404 returned error can't find the container with id f7d20545e56be8d57967ce164ac136c2edad676dbcedd778322708e789ee2d53
Jul 01 17:17:39.794618 ip-10-0-137-244 kubenswrapper[2565]: I0701 17:17:39.794587    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/isvc-autogluon-v2-predictor-67b849d456-bmr8g" event={"ID":"b242f7cd-ff9f-4f8c-a9df-2b6951290757","Type":"ContainerStarted","Data":"7a3741349fb9006e9a2e32ff50a84e8e46dabae8ca31b08d3b4816f5fceb52a7"}
Jul 01 17:17:39.794786 ip-10-0-137-244 kubenswrapper[2565]: I0701 17:17:39.794649    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/isvc-autogluon-v2-predictor-67b849d456-bmr8g" event={"ID":"b242f7cd-ff9f-4f8c-a9df-2b6951290757","Type":"ContainerStarted","Data":"f7d20545e56be8d57967ce164ac136c2edad676dbcedd778322708e789ee2d53"}
Jul 01 17:18:10.917956 ip-10-0-137-244 kubenswrapper[2565]: I0701 17:18:10.917923    2565 generic.go:356] "Generic (PLEG): container finished" podID="b242f7cd-ff9f-4f8c-a9df-2b6951290757" containerID="7a3741349fb9006e9a2e32ff50a84e8e46dabae8ca31b08d3b4816f5fceb52a7" exitCode=0
Jul 01 17:18:10.918308 ip-10-0-137-244 kubenswrapper[2565]: I0701 17:18:10.917999    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/isvc-autogluon-v2-predictor-67b849d456-bmr8g" event={"ID":"b242f7cd-ff9f-4f8c-a9df-2b6951290757","Type":"ContainerDied","Data":"7a3741349fb9006e9a2e32ff50a84e8e46dabae8ca31b08d3b4816f5fceb52a7"}
Jul 01 17:18:11.924198 ip-10-0-137-244 kubenswrapper[2565]: I0701 17:18:11.924168    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/isvc-autogluon-v2-predictor-67b849d456-bmr8g" event={"ID":"b242f7cd-ff9f-4f8c-a9df-2b6951290757","Type":"ContainerStarted","Data":"d014eb36f5a5d1f6eecb02d2bd88eb7807141d914809cfe64ee7422ab61bc3ef"}
Jul 01 17:18:11.924597 ip-10-0-137-244 kubenswrapper[2565]: I0701 17:18:11.924228    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/isvc-autogluon-v2-predictor-67b849d456-bmr8g" event={"ID":"b242f7cd-ff9f-4f8c-a9df-2b6951290757","Type":"ContainerStarted","Data":"98419f3e5dbba38a322b94d409d3bf1cf8c0eaf856b2948ac6f0284427577291"}
Jul 01 17:18:11.924597 ip-10-0-137-244 kubenswrapper[2565]: I0701 17:18:11.924468    2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve-ci-e2e-test/isvc-autogluon-v2-predictor-67b849d456-bmr8g"
Jul 01 17:18:11.946377 ip-10-0-137-244 kubenswrapper[2565]: I0701 17:18:11.946329    2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kserve-ci-e2e-test/isvc-autogluon-v2-predictor-67b849d456-bmr8g" podStartSLOduration=2.65524459 podStartE2EDuration="33.946315912s" totalImagesPullingTime="0s" totalInitContainerRuntime="31.291071322s" isStatefulPod=false podCreationTimestamp="2026-07-01 17:17:38 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-01 17:18:11.945754655 +0000 UTC m=+1162.414018778" watchObservedRunningTime="2026-07-01 17:18:11.946315912 +0000 UTC m=+1162.414580036"
Jul 01 17:18:12.928879 ip-10-0-137-244 kubenswrapper[2565]: I0701 17:18:12.928851    2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve-ci-e2e-test/isvc-autogluon-v2-predictor-67b849d456-bmr8g"
Jul 01 17:18:18.936914 ip-10-0-137-244 kubenswrapper[2565]: I0701 17:18:18.936885    2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kserve-ci-e2e-test/isvc-autogluon-v2-predictor-67b849d456-bmr8g"
Jul 01 17:18:50.189963 ip-10-0-137-244 kubenswrapper[2565]: I0701 17:18:50.189936    2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-zjsw4_bb7ebc32-d0cd-4eb0-b998-3ce298f73809/ovn-acl-logging/0.log"
Jul 01 17:18:50.192288 ip-10-0-137-244 kubenswrapper[2565]: I0701 17:18:50.192267    2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-zjsw4_bb7ebc32-d0cd-4eb0-b998-3ce298f73809/ovn-acl-logging/0.log"
Jul 01 17:19:42.248677 ip-10-0-137-244 kubenswrapper[2565]: I0701 17:19:42.248605    2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kserve-ci-e2e-test/isvc-autogluon-v2-predictor-67b849d456-bmr8g"
Jul 01 17:19:48.397250 ip-10-0-137-244 kubenswrapper[2565]: I0701 17:19:48.397222    2565 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kserve-ci-e2e-test/isvc-autogluon-v2-predictor-67b849d456-bmr8g"]
Jul 01 17:19:48.397597 ip-10-0-137-244 kubenswrapper[2565]: I0701 17:19:48.397464    2565 kuberuntime_container.go:872] "Killing container with a grace period" pod="kserve-ci-e2e-test/isvc-autogluon-v2-predictor-67b849d456-bmr8g" podUID="b242f7cd-ff9f-4f8c-a9df-2b6951290757" containerName="kube-rbac-proxy" containerID="cri-o://d014eb36f5a5d1f6eecb02d2bd88eb7807141d914809cfe64ee7422ab61bc3ef" gracePeriod=30
Jul 01 17:19:48.397597 ip-10-0-137-244 kubenswrapper[2565]: I0701 17:19:48.397456    2565 kuberuntime_container.go:872] "Killing container with a grace period" pod="kserve-ci-e2e-test/isvc-autogluon-v2-predictor-67b849d456-bmr8g" podUID="b242f7cd-ff9f-4f8c-a9df-2b6951290757" containerName="kserve-container" containerID="cri-o://98419f3e5dbba38a322b94d409d3bf1cf8c0eaf856b2948ac6f0284427577291" gracePeriod=30
Jul 01 17:19:48.932914 ip-10-0-137-244 kubenswrapper[2565]: I0701 17:19:48.932867    2565 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-autogluon-v2-predictor-67b849d456-bmr8g" podUID="b242f7cd-ff9f-4f8c-a9df-2b6951290757" containerName="kube-rbac-proxy" probeResult="failure" output="Get \"https://10.133.0.30:8643/healthz\": dial tcp 10.133.0.30:8643: connect: connection refused"
Jul 01 17:19:49.137027 ip-10-0-137-244 kubenswrapper[2565]: I0701 17:19:49.137002    2565 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/isvc-autogluon-v2-predictor-67b849d456-bmr8g"
Jul 01 17:19:49.183303 ip-10-0-137-244 kubenswrapper[2565]: I0701 17:19:49.183245    2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/b242f7cd-ff9f-4f8c-a9df-2b6951290757-proxy-tls\") pod \"b242f7cd-ff9f-4f8c-a9df-2b6951290757\" (UID: \"b242f7cd-ff9f-4f8c-a9df-2b6951290757\") "
Jul 01 17:19:49.183303 ip-10-0-137-244 kubenswrapper[2565]: I0701 17:19:49.183296    2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/b242f7cd-ff9f-4f8c-a9df-2b6951290757-kserve-provision-location\") pod \"b242f7cd-ff9f-4f8c-a9df-2b6951290757\" (UID: \"b242f7cd-ff9f-4f8c-a9df-2b6951290757\") "
Jul 01 17:19:49.183469 ip-10-0-137-244 kubenswrapper[2565]: I0701 17:19:49.183326    2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-9mmdv\" (UniqueName: \"kubernetes.io/projected/b242f7cd-ff9f-4f8c-a9df-2b6951290757-kube-api-access-9mmdv\") pod \"b242f7cd-ff9f-4f8c-a9df-2b6951290757\" (UID: \"b242f7cd-ff9f-4f8c-a9df-2b6951290757\") "
Jul 01 17:19:49.183469 ip-10-0-137-244 kubenswrapper[2565]: I0701 17:19:49.183352    2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/b242f7cd-ff9f-4f8c-a9df-2b6951290757-kube-rbac-proxy-sar-config\") pod \"b242f7cd-ff9f-4f8c-a9df-2b6951290757\" (UID: \"b242f7cd-ff9f-4f8c-a9df-2b6951290757\") "
Jul 01 17:19:49.183469 ip-10-0-137-244 kubenswrapper[2565]: I0701 17:19:49.183385    2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"cabundle-cert\" (UniqueName: \"kubernetes.io/configmap/b242f7cd-ff9f-4f8c-a9df-2b6951290757-cabundle-cert\") pod \"b242f7cd-ff9f-4f8c-a9df-2b6951290757\" (UID: \"b242f7cd-ff9f-4f8c-a9df-2b6951290757\") "
Jul 01 17:19:49.183736 ip-10-0-137-244 kubenswrapper[2565]: I0701 17:19:49.183696    2565 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/b242f7cd-ff9f-4f8c-a9df-2b6951290757-kube-rbac-proxy-sar-config" (OuterVolumeSpecName: "kube-rbac-proxy-sar-config") pod "b242f7cd-ff9f-4f8c-a9df-2b6951290757" (UID: "b242f7cd-ff9f-4f8c-a9df-2b6951290757"). InnerVolumeSpecName "kube-rbac-proxy-sar-config". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 01 17:19:49.183832 ip-10-0-137-244 kubenswrapper[2565]: I0701 17:19:49.183727    2565 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/b242f7cd-ff9f-4f8c-a9df-2b6951290757-cabundle-cert" (OuterVolumeSpecName: "cabundle-cert") pod "b242f7cd-ff9f-4f8c-a9df-2b6951290757" (UID: "b242f7cd-ff9f-4f8c-a9df-2b6951290757"). InnerVolumeSpecName "cabundle-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 01 17:19:49.185316 ip-10-0-137-244 kubenswrapper[2565]: I0701 17:19:49.185293    2565 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/b242f7cd-ff9f-4f8c-a9df-2b6951290757-proxy-tls" (OuterVolumeSpecName: "proxy-tls") pod "b242f7cd-ff9f-4f8c-a9df-2b6951290757" (UID: "b242f7cd-ff9f-4f8c-a9df-2b6951290757"). InnerVolumeSpecName "proxy-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 01 17:19:49.185430 ip-10-0-137-244 kubenswrapper[2565]: I0701 17:19:49.185412    2565 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/b242f7cd-ff9f-4f8c-a9df-2b6951290757-kube-api-access-9mmdv" (OuterVolumeSpecName: "kube-api-access-9mmdv") pod "b242f7cd-ff9f-4f8c-a9df-2b6951290757" (UID: "b242f7cd-ff9f-4f8c-a9df-2b6951290757"). InnerVolumeSpecName "kube-api-access-9mmdv". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 01 17:19:49.188036 ip-10-0-137-244 kubenswrapper[2565]: I0701 17:19:49.188004    2565 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b242f7cd-ff9f-4f8c-a9df-2b6951290757-kserve-provision-location" (OuterVolumeSpecName: "kserve-provision-location") pod "b242f7cd-ff9f-4f8c-a9df-2b6951290757" (UID: "b242f7cd-ff9f-4f8c-a9df-2b6951290757"). InnerVolumeSpecName "kserve-provision-location". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 01 17:19:49.284634 ip-10-0-137-244 kubenswrapper[2565]: I0701 17:19:49.284611    2565 reconciler_common.go:299] "Volume detached for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/b242f7cd-ff9f-4f8c-a9df-2b6951290757-proxy-tls\") on node \"ip-10-0-137-244.ec2.internal\" DevicePath \"\""
Jul 01 17:19:49.284634 ip-10-0-137-244 kubenswrapper[2565]: I0701 17:19:49.284635    2565 reconciler_common.go:299] "Volume detached for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/b242f7cd-ff9f-4f8c-a9df-2b6951290757-kserve-provision-location\") on node \"ip-10-0-137-244.ec2.internal\" DevicePath \"\""
Jul 01 17:19:49.284765 ip-10-0-137-244 kubenswrapper[2565]: I0701 17:19:49.284646    2565 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-9mmdv\" (UniqueName: \"kubernetes.io/projected/b242f7cd-ff9f-4f8c-a9df-2b6951290757-kube-api-access-9mmdv\") on node \"ip-10-0-137-244.ec2.internal\" DevicePath \"\""
Jul 01 17:19:49.284765 ip-10-0-137-244 kubenswrapper[2565]: I0701 17:19:49.284655    2565 reconciler_common.go:299] "Volume detached for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/b242f7cd-ff9f-4f8c-a9df-2b6951290757-kube-rbac-proxy-sar-config\") on node \"ip-10-0-137-244.ec2.internal\" DevicePath \"\""
Jul 01 17:19:49.284765 ip-10-0-137-244 kubenswrapper[2565]: I0701 17:19:49.284664    2565 reconciler_common.go:299] "Volume detached for volume \"cabundle-cert\" (UniqueName: \"kubernetes.io/configmap/b242f7cd-ff9f-4f8c-a9df-2b6951290757-cabundle-cert\") on node \"ip-10-0-137-244.ec2.internal\" DevicePath \"\""
Jul 01 17:19:49.318487 ip-10-0-137-244 kubenswrapper[2565]: I0701 17:19:49.318464    2565 generic.go:356] "Generic (PLEG): container finished" podID="b242f7cd-ff9f-4f8c-a9df-2b6951290757" containerID="d014eb36f5a5d1f6eecb02d2bd88eb7807141d914809cfe64ee7422ab61bc3ef" exitCode=2
Jul 01 17:19:49.318487 ip-10-0-137-244 kubenswrapper[2565]: I0701 17:19:49.318486    2565 generic.go:356] "Generic (PLEG): container finished" podID="b242f7cd-ff9f-4f8c-a9df-2b6951290757" containerID="98419f3e5dbba38a322b94d409d3bf1cf8c0eaf856b2948ac6f0284427577291" exitCode=0
Jul 01 17:19:49.318622 ip-10-0-137-244 kubenswrapper[2565]: I0701 17:19:49.318517    2565 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/isvc-autogluon-v2-predictor-67b849d456-bmr8g"
Jul 01 17:19:49.318622 ip-10-0-137-244 kubenswrapper[2565]: I0701 17:19:49.318551    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/isvc-autogluon-v2-predictor-67b849d456-bmr8g" event={"ID":"b242f7cd-ff9f-4f8c-a9df-2b6951290757","Type":"ContainerDied","Data":"d014eb36f5a5d1f6eecb02d2bd88eb7807141d914809cfe64ee7422ab61bc3ef"}
Jul 01 17:19:49.318622 ip-10-0-137-244 kubenswrapper[2565]: I0701 17:19:49.318605    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/isvc-autogluon-v2-predictor-67b849d456-bmr8g" event={"ID":"b242f7cd-ff9f-4f8c-a9df-2b6951290757","Type":"ContainerDied","Data":"98419f3e5dbba38a322b94d409d3bf1cf8c0eaf856b2948ac6f0284427577291"}
Jul 01 17:19:49.318715 ip-10-0-137-244 kubenswrapper[2565]: I0701 17:19:49.318626    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/isvc-autogluon-v2-predictor-67b849d456-bmr8g" event={"ID":"b242f7cd-ff9f-4f8c-a9df-2b6951290757","Type":"ContainerDied","Data":"f7d20545e56be8d57967ce164ac136c2edad676dbcedd778322708e789ee2d53"}
Jul 01 17:19:49.318715 ip-10-0-137-244 kubenswrapper[2565]: I0701 17:19:49.318650    2565 scope.go:117] "RemoveContainer" containerID="d014eb36f5a5d1f6eecb02d2bd88eb7807141d914809cfe64ee7422ab61bc3ef"
Jul 01 17:19:49.327320 ip-10-0-137-244 kubenswrapper[2565]: I0701 17:19:49.327297    2565 scope.go:117] "RemoveContainer" containerID="98419f3e5dbba38a322b94d409d3bf1cf8c0eaf856b2948ac6f0284427577291"
Jul 01 17:19:49.334636 ip-10-0-137-244 kubenswrapper[2565]: I0701 17:19:49.334620    2565 scope.go:117] "RemoveContainer" containerID="7a3741349fb9006e9a2e32ff50a84e8e46dabae8ca31b08d3b4816f5fceb52a7"
Jul 01 17:19:49.340844 ip-10-0-137-244 kubenswrapper[2565]: I0701 17:19:49.340824    2565 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kserve-ci-e2e-test/isvc-autogluon-v2-predictor-67b849d456-bmr8g"]
Jul 01 17:19:49.342585 ip-10-0-137-244 kubenswrapper[2565]: I0701 17:19:49.342568    2565 scope.go:117] "RemoveContainer" containerID="d014eb36f5a5d1f6eecb02d2bd88eb7807141d914809cfe64ee7422ab61bc3ef"
Jul 01 17:19:49.342833 ip-10-0-137-244 kubenswrapper[2565]: E0701 17:19:49.342811    2565 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"d014eb36f5a5d1f6eecb02d2bd88eb7807141d914809cfe64ee7422ab61bc3ef\": container with ID starting with d014eb36f5a5d1f6eecb02d2bd88eb7807141d914809cfe64ee7422ab61bc3ef not found: ID does not exist" containerID="d014eb36f5a5d1f6eecb02d2bd88eb7807141d914809cfe64ee7422ab61bc3ef"
Jul 01 17:19:49.342934 ip-10-0-137-244 kubenswrapper[2565]: I0701 17:19:49.342844    2565 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d014eb36f5a5d1f6eecb02d2bd88eb7807141d914809cfe64ee7422ab61bc3ef"} err="failed to get container status \"d014eb36f5a5d1f6eecb02d2bd88eb7807141d914809cfe64ee7422ab61bc3ef\": rpc error: code = NotFound desc = could not find container \"d014eb36f5a5d1f6eecb02d2bd88eb7807141d914809cfe64ee7422ab61bc3ef\": container with ID starting with d014eb36f5a5d1f6eecb02d2bd88eb7807141d914809cfe64ee7422ab61bc3ef not found: ID does not exist"
Jul 01 17:19:49.342934 ip-10-0-137-244 kubenswrapper[2565]: I0701 17:19:49.342868    2565 scope.go:117] "RemoveContainer" containerID="98419f3e5dbba38a322b94d409d3bf1cf8c0eaf856b2948ac6f0284427577291"
Jul 01 17:19:49.343044 ip-10-0-137-244 kubenswrapper[2565]: I0701 17:19:49.342930    2565 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kserve-ci-e2e-test/isvc-autogluon-v2-predictor-67b849d456-bmr8g"]
Jul 01 17:19:49.343159 ip-10-0-137-244 kubenswrapper[2565]: E0701 17:19:49.343140    2565 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"98419f3e5dbba38a322b94d409d3bf1cf8c0eaf856b2948ac6f0284427577291\": container with ID starting with 98419f3e5dbba38a322b94d409d3bf1cf8c0eaf856b2948ac6f0284427577291 not found: ID does not exist" containerID="98419f3e5dbba38a322b94d409d3bf1cf8c0eaf856b2948ac6f0284427577291"
Jul 01 17:19:49.343205 ip-10-0-137-244 kubenswrapper[2565]: I0701 17:19:49.343167    2565 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"98419f3e5dbba38a322b94d409d3bf1cf8c0eaf856b2948ac6f0284427577291"} err="failed to get container status \"98419f3e5dbba38a322b94d409d3bf1cf8c0eaf856b2948ac6f0284427577291\": rpc error: code = NotFound desc = could not find container \"98419f3e5dbba38a322b94d409d3bf1cf8c0eaf856b2948ac6f0284427577291\": container with ID starting with 98419f3e5dbba38a322b94d409d3bf1cf8c0eaf856b2948ac6f0284427577291 not found: ID does not exist"
Jul 01 17:19:49.343205 ip-10-0-137-244 kubenswrapper[2565]: I0701 17:19:49.343183    2565 scope.go:117] "RemoveContainer" containerID="7a3741349fb9006e9a2e32ff50a84e8e46dabae8ca31b08d3b4816f5fceb52a7"
Jul 01 17:19:49.343434 ip-10-0-137-244 kubenswrapper[2565]: E0701 17:19:49.343416    2565 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"7a3741349fb9006e9a2e32ff50a84e8e46dabae8ca31b08d3b4816f5fceb52a7\": container with ID starting with 7a3741349fb9006e9a2e32ff50a84e8e46dabae8ca31b08d3b4816f5fceb52a7 not found: ID does not exist" containerID="7a3741349fb9006e9a2e32ff50a84e8e46dabae8ca31b08d3b4816f5fceb52a7"
Jul 01 17:19:49.343473 ip-10-0-137-244 kubenswrapper[2565]: I0701 17:19:49.343438    2565 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"7a3741349fb9006e9a2e32ff50a84e8e46dabae8ca31b08d3b4816f5fceb52a7"} err="failed to get container status \"7a3741349fb9006e9a2e32ff50a84e8e46dabae8ca31b08d3b4816f5fceb52a7\": rpc error: code = NotFound desc = could not find container \"7a3741349fb9006e9a2e32ff50a84e8e46dabae8ca31b08d3b4816f5fceb52a7\": container with ID starting with 7a3741349fb9006e9a2e32ff50a84e8e46dabae8ca31b08d3b4816f5fceb52a7 not found: ID does not exist"
Jul 01 17:19:49.343473 ip-10-0-137-244 kubenswrapper[2565]: I0701 17:19:49.343453    2565 scope.go:117] "RemoveContainer" containerID="d014eb36f5a5d1f6eecb02d2bd88eb7807141d914809cfe64ee7422ab61bc3ef"
Jul 01 17:19:49.348727 ip-10-0-137-244 kubenswrapper[2565]: I0701 17:19:49.348699    2565 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d014eb36f5a5d1f6eecb02d2bd88eb7807141d914809cfe64ee7422ab61bc3ef"} err="failed to get container status \"d014eb36f5a5d1f6eecb02d2bd88eb7807141d914809cfe64ee7422ab61bc3ef\": rpc error: code = NotFound desc = could not find container \"d014eb36f5a5d1f6eecb02d2bd88eb7807141d914809cfe64ee7422ab61bc3ef\": container with ID starting with d014eb36f5a5d1f6eecb02d2bd88eb7807141d914809cfe64ee7422ab61bc3ef not found: ID does not exist"
Jul 01 17:19:49.348876 ip-10-0-137-244 kubenswrapper[2565]: I0701 17:19:49.348729    2565 scope.go:117] "RemoveContainer" containerID="98419f3e5dbba38a322b94d409d3bf1cf8c0eaf856b2948ac6f0284427577291"
Jul 01 17:19:49.349136 ip-10-0-137-244 kubenswrapper[2565]: I0701 17:19:49.349108    2565 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"98419f3e5dbba38a322b94d409d3bf1cf8c0eaf856b2948ac6f0284427577291"} err="failed to get container status \"98419f3e5dbba38a322b94d409d3bf1cf8c0eaf856b2948ac6f0284427577291\": rpc error: code = NotFound desc = could not find container \"98419f3e5dbba38a322b94d409d3bf1cf8c0eaf856b2948ac6f0284427577291\": container with ID starting with 98419f3e5dbba38a322b94d409d3bf1cf8c0eaf856b2948ac6f0284427577291 not found: ID does not exist"
Jul 01 17:19:49.349209 ip-10-0-137-244 kubenswrapper[2565]: I0701 17:19:49.349136    2565 scope.go:117] "RemoveContainer" containerID="7a3741349fb9006e9a2e32ff50a84e8e46dabae8ca31b08d3b4816f5fceb52a7"
Jul 01 17:19:49.349381 ip-10-0-137-244 kubenswrapper[2565]: I0701 17:19:49.349364    2565 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"7a3741349fb9006e9a2e32ff50a84e8e46dabae8ca31b08d3b4816f5fceb52a7"} err="failed to get container status \"7a3741349fb9006e9a2e32ff50a84e8e46dabae8ca31b08d3b4816f5fceb52a7\": rpc error: code = NotFound desc = could not find container \"7a3741349fb9006e9a2e32ff50a84e8e46dabae8ca31b08d3b4816f5fceb52a7\": container with ID starting with 7a3741349fb9006e9a2e32ff50a84e8e46dabae8ca31b08d3b4816f5fceb52a7 not found: ID does not exist"
Jul 01 17:19:50.247471 ip-10-0-137-244 kubenswrapper[2565]: I0701 17:19:50.247440    2565 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="b242f7cd-ff9f-4f8c-a9df-2b6951290757" path="/var/lib/kubelet/pods/b242f7cd-ff9f-4f8c-a9df-2b6951290757/volumes"
Jul 01 17:19:53.499545 ip-10-0-137-244 kubenswrapper[2565]: I0701 17:19:53.499514    2565 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kserve-ci-e2e-test/isvc-autogluon-v2-variants-predictor-c6767c94c-ff48j"]
Jul 01 17:19:53.499932 ip-10-0-137-244 kubenswrapper[2565]: I0701 17:19:53.499787    2565 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b242f7cd-ff9f-4f8c-a9df-2b6951290757" containerName="storage-initializer"
Jul 01 17:19:53.499932 ip-10-0-137-244 kubenswrapper[2565]: I0701 17:19:53.499798    2565 state_mem.go:107] "Deleted CPUSet assignment" podUID="b242f7cd-ff9f-4f8c-a9df-2b6951290757" containerName="storage-initializer"
Jul 01 17:19:53.499932 ip-10-0-137-244 kubenswrapper[2565]: I0701 17:19:53.499805    2565 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b242f7cd-ff9f-4f8c-a9df-2b6951290757" containerName="kube-rbac-proxy"
Jul 01 17:19:53.499932 ip-10-0-137-244 kubenswrapper[2565]: I0701 17:19:53.499811    2565 state_mem.go:107] "Deleted CPUSet assignment" podUID="b242f7cd-ff9f-4f8c-a9df-2b6951290757" containerName="kube-rbac-proxy"
Jul 01 17:19:53.499932 ip-10-0-137-244 kubenswrapper[2565]: I0701 17:19:53.499817    2565 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b242f7cd-ff9f-4f8c-a9df-2b6951290757" containerName="kserve-container"
Jul 01 17:19:53.499932 ip-10-0-137-244 kubenswrapper[2565]: I0701 17:19:53.499823    2565 state_mem.go:107] "Deleted CPUSet assignment" podUID="b242f7cd-ff9f-4f8c-a9df-2b6951290757" containerName="kserve-container"
Jul 01 17:19:53.499932 ip-10-0-137-244 kubenswrapper[2565]: I0701 17:19:53.499870    2565 memory_manager.go:368] "RemoveStaleState removing state" podUID="b242f7cd-ff9f-4f8c-a9df-2b6951290757" containerName="kserve-container"
Jul 01 17:19:53.499932 ip-10-0-137-244 kubenswrapper[2565]: I0701 17:19:53.499879    2565 memory_manager.go:368] "RemoveStaleState removing state" podUID="b242f7cd-ff9f-4f8c-a9df-2b6951290757" containerName="kube-rbac-proxy"
Jul 01 17:19:53.503230 ip-10-0-137-244 kubenswrapper[2565]: I0701 17:19:53.503212    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/isvc-autogluon-v2-variants-predictor-c6767c94c-ff48j"
Jul 01 17:19:53.505688 ip-10-0-137-244 kubenswrapper[2565]: I0701 17:19:53.505663    2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kserve-ci-e2e-test\"/\"default-dockercfg-bl7zk\""
Jul 01 17:19:53.505688 ip-10-0-137-244 kubenswrapper[2565]: I0701 17:19:53.505685    2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kserve-ci-e2e-test\"/\"kube-root-ca.crt\""
Jul 01 17:19:53.505892 ip-10-0-137-244 kubenswrapper[2565]: I0701 17:19:53.505666    2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kserve-ci-e2e-test\"/\"isvc-autogluon-v2-variants-predictor-serving-cert\""
Jul 01 17:19:53.505892 ip-10-0-137-244 kubenswrapper[2565]: I0701 17:19:53.505718    2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kserve-ci-e2e-test\"/\"openshift-service-ca.crt\""
Jul 01 17:19:53.506077 ip-10-0-137-244 kubenswrapper[2565]: I0701 17:19:53.506062    2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kserve-ci-e2e-test\"/\"odh-kserve-custom-ca-bundle\""
Jul 01 17:19:53.506133 ip-10-0-137-244 kubenswrapper[2565]: I0701 17:19:53.506106    2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kserve-ci-e2e-test\"/\"seaweedfs-s3-creds\""
Jul 01 17:19:53.506201 ip-10-0-137-244 kubenswrapper[2565]: I0701 17:19:53.506182    2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kserve-ci-e2e-test\"/\"isvc-autogluon-v2-variants-kube-rbac-proxy-sar-config\""
Jul 01 17:19:53.514529 ip-10-0-137-244 kubenswrapper[2565]: I0701 17:19:53.514506    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/isvc-autogluon-v2-variants-predictor-c6767c94c-ff48j"]
Jul 01 17:19:53.617489 ip-10-0-137-244 kubenswrapper[2565]: I0701 17:19:53.617450    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/9e61a7b3-fb58-49db-a04d-bbbfcde73fe4-proxy-tls\") pod \"isvc-autogluon-v2-variants-predictor-c6767c94c-ff48j\" (UID: \"9e61a7b3-fb58-49db-a04d-bbbfcde73fe4\") " pod="kserve-ci-e2e-test/isvc-autogluon-v2-variants-predictor-c6767c94c-ff48j"
Jul 01 17:19:53.617667 ip-10-0-137-244 kubenswrapper[2565]: I0701 17:19:53.617510    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-cpjpw\" (UniqueName: \"kubernetes.io/projected/9e61a7b3-fb58-49db-a04d-bbbfcde73fe4-kube-api-access-cpjpw\") pod \"isvc-autogluon-v2-variants-predictor-c6767c94c-ff48j\" (UID: \"9e61a7b3-fb58-49db-a04d-bbbfcde73fe4\") " pod="kserve-ci-e2e-test/isvc-autogluon-v2-variants-predictor-c6767c94c-ff48j"
Jul 01 17:19:53.617667 ip-10-0-137-244 kubenswrapper[2565]: I0701 17:19:53.617533    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/9e61a7b3-fb58-49db-a04d-bbbfcde73fe4-kserve-provision-location\") pod \"isvc-autogluon-v2-variants-predictor-c6767c94c-ff48j\" (UID: \"9e61a7b3-fb58-49db-a04d-bbbfcde73fe4\") " pod="kserve-ci-e2e-test/isvc-autogluon-v2-variants-predictor-c6767c94c-ff48j"
Jul 01 17:19:53.617667 ip-10-0-137-244 kubenswrapper[2565]: I0701 17:19:53.617590    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/9e61a7b3-fb58-49db-a04d-bbbfcde73fe4-kube-rbac-proxy-sar-config\") pod \"isvc-autogluon-v2-variants-predictor-c6767c94c-ff48j\" (UID: \"9e61a7b3-fb58-49db-a04d-bbbfcde73fe4\") " pod="kserve-ci-e2e-test/isvc-autogluon-v2-variants-predictor-c6767c94c-ff48j"
Jul 01 17:19:53.617667 ip-10-0-137-244 kubenswrapper[2565]: I0701 17:19:53.617652    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cabundle-cert\" (UniqueName: \"kubernetes.io/configmap/9e61a7b3-fb58-49db-a04d-bbbfcde73fe4-cabundle-cert\") pod \"isvc-autogluon-v2-variants-predictor-c6767c94c-ff48j\" (UID: \"9e61a7b3-fb58-49db-a04d-bbbfcde73fe4\") " pod="kserve-ci-e2e-test/isvc-autogluon-v2-variants-predictor-c6767c94c-ff48j"
Jul 01 17:19:53.719008 ip-10-0-137-244 kubenswrapper[2565]: I0701 17:19:53.718973    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-cpjpw\" (UniqueName: \"kubernetes.io/projected/9e61a7b3-fb58-49db-a04d-bbbfcde73fe4-kube-api-access-cpjpw\") pod \"isvc-autogluon-v2-variants-predictor-c6767c94c-ff48j\" (UID: \"9e61a7b3-fb58-49db-a04d-bbbfcde73fe4\") " pod="kserve-ci-e2e-test/isvc-autogluon-v2-variants-predictor-c6767c94c-ff48j"
Jul 01 17:19:53.719008 ip-10-0-137-244 kubenswrapper[2565]: I0701 17:19:53.719009    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/9e61a7b3-fb58-49db-a04d-bbbfcde73fe4-kserve-provision-location\") pod \"isvc-autogluon-v2-variants-predictor-c6767c94c-ff48j\" (UID: \"9e61a7b3-fb58-49db-a04d-bbbfcde73fe4\") " pod="kserve-ci-e2e-test/isvc-autogluon-v2-variants-predictor-c6767c94c-ff48j"
Jul 01 17:19:53.719254 ip-10-0-137-244 kubenswrapper[2565]: I0701 17:19:53.719049    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/9e61a7b3-fb58-49db-a04d-bbbfcde73fe4-kube-rbac-proxy-sar-config\") pod \"isvc-autogluon-v2-variants-predictor-c6767c94c-ff48j\" (UID: \"9e61a7b3-fb58-49db-a04d-bbbfcde73fe4\") " pod="kserve-ci-e2e-test/isvc-autogluon-v2-variants-predictor-c6767c94c-ff48j"
Jul 01 17:19:53.719254 ip-10-0-137-244 kubenswrapper[2565]: I0701 17:19:53.719073    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cabundle-cert\" (UniqueName: \"kubernetes.io/configmap/9e61a7b3-fb58-49db-a04d-bbbfcde73fe4-cabundle-cert\") pod \"isvc-autogluon-v2-variants-predictor-c6767c94c-ff48j\" (UID: \"9e61a7b3-fb58-49db-a04d-bbbfcde73fe4\") " pod="kserve-ci-e2e-test/isvc-autogluon-v2-variants-predictor-c6767c94c-ff48j"
Jul 01 17:19:53.719254 ip-10-0-137-244 kubenswrapper[2565]: I0701 17:19:53.719102    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/9e61a7b3-fb58-49db-a04d-bbbfcde73fe4-proxy-tls\") pod \"isvc-autogluon-v2-variants-predictor-c6767c94c-ff48j\" (UID: \"9e61a7b3-fb58-49db-a04d-bbbfcde73fe4\") " pod="kserve-ci-e2e-test/isvc-autogluon-v2-variants-predictor-c6767c94c-ff48j"
Jul 01 17:19:53.719527 ip-10-0-137-244 kubenswrapper[2565]: I0701 17:19:53.719503    2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/9e61a7b3-fb58-49db-a04d-bbbfcde73fe4-kserve-provision-location\") pod \"isvc-autogluon-v2-variants-predictor-c6767c94c-ff48j\" (UID: \"9e61a7b3-fb58-49db-a04d-bbbfcde73fe4\") " pod="kserve-ci-e2e-test/isvc-autogluon-v2-variants-predictor-c6767c94c-ff48j"
Jul 01 17:19:53.719792 ip-10-0-137-244 kubenswrapper[2565]: I0701 17:19:53.719774    2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/9e61a7b3-fb58-49db-a04d-bbbfcde73fe4-kube-rbac-proxy-sar-config\") pod \"isvc-autogluon-v2-variants-predictor-c6767c94c-ff48j\" (UID: \"9e61a7b3-fb58-49db-a04d-bbbfcde73fe4\") " pod="kserve-ci-e2e-test/isvc-autogluon-v2-variants-predictor-c6767c94c-ff48j"
Jul 01 17:19:53.719834 ip-10-0-137-244 kubenswrapper[2565]: I0701 17:19:53.719811    2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cabundle-cert\" (UniqueName: \"kubernetes.io/configmap/9e61a7b3-fb58-49db-a04d-bbbfcde73fe4-cabundle-cert\") pod \"isvc-autogluon-v2-variants-predictor-c6767c94c-ff48j\" (UID: \"9e61a7b3-fb58-49db-a04d-bbbfcde73fe4\") " pod="kserve-ci-e2e-test/isvc-autogluon-v2-variants-predictor-c6767c94c-ff48j"
Jul 01 17:19:53.721475 ip-10-0-137-244 kubenswrapper[2565]: I0701 17:19:53.721454    2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/9e61a7b3-fb58-49db-a04d-bbbfcde73fe4-proxy-tls\") pod \"isvc-autogluon-v2-variants-predictor-c6767c94c-ff48j\" (UID: \"9e61a7b3-fb58-49db-a04d-bbbfcde73fe4\") " pod="kserve-ci-e2e-test/isvc-autogluon-v2-variants-predictor-c6767c94c-ff48j"
Jul 01 17:19:53.727453 ip-10-0-137-244 kubenswrapper[2565]: I0701 17:19:53.727429    2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-cpjpw\" (UniqueName: \"kubernetes.io/projected/9e61a7b3-fb58-49db-a04d-bbbfcde73fe4-kube-api-access-cpjpw\") pod \"isvc-autogluon-v2-variants-predictor-c6767c94c-ff48j\" (UID: \"9e61a7b3-fb58-49db-a04d-bbbfcde73fe4\") " pod="kserve-ci-e2e-test/isvc-autogluon-v2-variants-predictor-c6767c94c-ff48j"
Jul 01 17:19:53.814062 ip-10-0-137-244 kubenswrapper[2565]: I0701 17:19:53.813991    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/isvc-autogluon-v2-variants-predictor-c6767c94c-ff48j"
Jul 01 17:19:53.941591 ip-10-0-137-244 kubenswrapper[2565]: I0701 17:19:53.941562    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/isvc-autogluon-v2-variants-predictor-c6767c94c-ff48j"]
Jul 01 17:19:53.944789 ip-10-0-137-244 kubenswrapper[2565]: W0701 17:19:53.944768    2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9e61a7b3_fb58_49db_a04d_bbbfcde73fe4.slice/crio-7e9c4bcbd5c89ccfb78eacc149a23b9bc1c012b2e20c14c3fc8959561d8066e0 WatchSource:0}: Error finding container 7e9c4bcbd5c89ccfb78eacc149a23b9bc1c012b2e20c14c3fc8959561d8066e0: Status 404 returned error can't find the container with id 7e9c4bcbd5c89ccfb78eacc149a23b9bc1c012b2e20c14c3fc8959561d8066e0
Jul 01 17:19:54.338162 ip-10-0-137-244 kubenswrapper[2565]: I0701 17:19:54.338127    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/isvc-autogluon-v2-variants-predictor-c6767c94c-ff48j" event={"ID":"9e61a7b3-fb58-49db-a04d-bbbfcde73fe4","Type":"ContainerStarted","Data":"8bf6197e05afc7828e3df6da554aaa58f50661fde758a062ba1db4c795baffd8"}
Jul 01 17:19:54.338337 ip-10-0-137-244 kubenswrapper[2565]: I0701 17:19:54.338184    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/isvc-autogluon-v2-variants-predictor-c6767c94c-ff48j" event={"ID":"9e61a7b3-fb58-49db-a04d-bbbfcde73fe4","Type":"ContainerStarted","Data":"7e9c4bcbd5c89ccfb78eacc149a23b9bc1c012b2e20c14c3fc8959561d8066e0"}
Jul 01 17:20:11.408564 ip-10-0-137-244 kubenswrapper[2565]: I0701 17:20:11.408528    2565 generic.go:356] "Generic (PLEG): container finished" podID="9e61a7b3-fb58-49db-a04d-bbbfcde73fe4" containerID="8bf6197e05afc7828e3df6da554aaa58f50661fde758a062ba1db4c795baffd8" exitCode=0
Jul 01 17:20:11.409050 ip-10-0-137-244 kubenswrapper[2565]: I0701 17:20:11.408608    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/isvc-autogluon-v2-variants-predictor-c6767c94c-ff48j" event={"ID":"9e61a7b3-fb58-49db-a04d-bbbfcde73fe4","Type":"ContainerDied","Data":"8bf6197e05afc7828e3df6da554aaa58f50661fde758a062ba1db4c795baffd8"}
Jul 01 17:20:12.414339 ip-10-0-137-244 kubenswrapper[2565]: I0701 17:20:12.414309    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/isvc-autogluon-v2-variants-predictor-c6767c94c-ff48j" event={"ID":"9e61a7b3-fb58-49db-a04d-bbbfcde73fe4","Type":"ContainerStarted","Data":"e038d278d4044d0844513cd1c8c37a64208714015600238290946fa126129035"}
Jul 01 17:20:12.414742 ip-10-0-137-244 kubenswrapper[2565]: I0701 17:20:12.414363    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/isvc-autogluon-v2-variants-predictor-c6767c94c-ff48j" event={"ID":"9e61a7b3-fb58-49db-a04d-bbbfcde73fe4","Type":"ContainerStarted","Data":"2f00001cbc657948e745cc82f278f79d14f9625bdc416a2d1f1c90ef0eaa915e"}
Jul 01 17:20:12.414742 ip-10-0-137-244 kubenswrapper[2565]: I0701 17:20:12.414527    2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve-ci-e2e-test/isvc-autogluon-v2-variants-predictor-c6767c94c-ff48j"
Jul 01 17:20:12.437320 ip-10-0-137-244 kubenswrapper[2565]: I0701 17:20:12.437279    2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kserve-ci-e2e-test/isvc-autogluon-v2-variants-predictor-c6767c94c-ff48j" podStartSLOduration=2.336264353 podStartE2EDuration="19.437263395s" totalImagesPullingTime="0s" totalInitContainerRuntime="17.100999042s" isStatefulPod=false podCreationTimestamp="2026-07-01 17:19:53 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-01 17:20:12.436894463 +0000 UTC m=+1282.905158587" watchObservedRunningTime="2026-07-01 17:20:12.437263395 +0000 UTC m=+1282.905527520"
Jul 01 17:20:13.418581 ip-10-0-137-244 kubenswrapper[2565]: I0701 17:20:13.418552    2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve-ci-e2e-test/isvc-autogluon-v2-variants-predictor-c6767c94c-ff48j"
Jul 01 17:20:19.428287 ip-10-0-137-244 kubenswrapper[2565]: I0701 17:20:19.428263    2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kserve-ci-e2e-test/isvc-autogluon-v2-variants-predictor-c6767c94c-ff48j"
Jul 01 17:21:44.249456 ip-10-0-137-244 kubenswrapper[2565]: I0701 17:21:44.249424    2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kserve-ci-e2e-test/isvc-autogluon-v2-variants-predictor-c6767c94c-ff48j"
Jul 01 17:21:53.642477 ip-10-0-137-244 kubenswrapper[2565]: I0701 17:21:53.642447    2565 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kserve-ci-e2e-test/isvc-autogluon-v2-variants-predictor-c6767c94c-ff48j"]
Jul 01 17:21:53.642838 ip-10-0-137-244 kubenswrapper[2565]: I0701 17:21:53.642715    2565 kuberuntime_container.go:872] "Killing container with a grace period" pod="kserve-ci-e2e-test/isvc-autogluon-v2-variants-predictor-c6767c94c-ff48j" podUID="9e61a7b3-fb58-49db-a04d-bbbfcde73fe4" containerName="kserve-container" containerID="cri-o://2f00001cbc657948e745cc82f278f79d14f9625bdc416a2d1f1c90ef0eaa915e" gracePeriod=30
Jul 01 17:21:53.642838 ip-10-0-137-244 kubenswrapper[2565]: I0701 17:21:53.642736    2565 kuberuntime_container.go:872] "Killing container with a grace period" pod="kserve-ci-e2e-test/isvc-autogluon-v2-variants-predictor-c6767c94c-ff48j" podUID="9e61a7b3-fb58-49db-a04d-bbbfcde73fe4" containerName="kube-rbac-proxy" containerID="cri-o://e038d278d4044d0844513cd1c8c37a64208714015600238290946fa126129035" gracePeriod=30
Jul 01 17:21:53.814542 ip-10-0-137-244 kubenswrapper[2565]: I0701 17:21:53.814510    2565 generic.go:356] "Generic (PLEG): container finished" podID="9e61a7b3-fb58-49db-a04d-bbbfcde73fe4" containerID="e038d278d4044d0844513cd1c8c37a64208714015600238290946fa126129035" exitCode=2
Jul 01 17:21:53.814662 ip-10-0-137-244 kubenswrapper[2565]: I0701 17:21:53.814559    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/isvc-autogluon-v2-variants-predictor-c6767c94c-ff48j" event={"ID":"9e61a7b3-fb58-49db-a04d-bbbfcde73fe4","Type":"ContainerDied","Data":"e038d278d4044d0844513cd1c8c37a64208714015600238290946fa126129035"}
Jul 01 17:21:54.244951 ip-10-0-137-244 kubenswrapper[2565]: I0701 17:21:54.244910    2565 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-autogluon-v2-variants-predictor-c6767c94c-ff48j" podUID="9e61a7b3-fb58-49db-a04d-bbbfcde73fe4" containerName="kserve-container" probeResult="failure" output="Get \"http://10.133.0.31:8080/v2/models/isvc-autogluon-v2-variants/ready\": dial tcp 10.133.0.31:8080: connect: connection refused"
Jul 01 17:21:54.390469 ip-10-0-137-244 kubenswrapper[2565]: I0701 17:21:54.390447    2565 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/isvc-autogluon-v2-variants-predictor-c6767c94c-ff48j"
Jul 01 17:21:54.431687 ip-10-0-137-244 kubenswrapper[2565]: I0701 17:21:54.431595    2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"cabundle-cert\" (UniqueName: \"kubernetes.io/configmap/9e61a7b3-fb58-49db-a04d-bbbfcde73fe4-cabundle-cert\") pod \"9e61a7b3-fb58-49db-a04d-bbbfcde73fe4\" (UID: \"9e61a7b3-fb58-49db-a04d-bbbfcde73fe4\") "
Jul 01 17:21:54.431791 ip-10-0-137-244 kubenswrapper[2565]: I0701 17:21:54.431723    2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-cpjpw\" (UniqueName: \"kubernetes.io/projected/9e61a7b3-fb58-49db-a04d-bbbfcde73fe4-kube-api-access-cpjpw\") pod \"9e61a7b3-fb58-49db-a04d-bbbfcde73fe4\" (UID: \"9e61a7b3-fb58-49db-a04d-bbbfcde73fe4\") "
Jul 01 17:21:54.431791 ip-10-0-137-244 kubenswrapper[2565]: I0701 17:21:54.431771    2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/9e61a7b3-fb58-49db-a04d-bbbfcde73fe4-proxy-tls\") pod \"9e61a7b3-fb58-49db-a04d-bbbfcde73fe4\" (UID: \"9e61a7b3-fb58-49db-a04d-bbbfcde73fe4\") "
Jul 01 17:21:54.431861 ip-10-0-137-244 kubenswrapper[2565]: I0701 17:21:54.431805    2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/9e61a7b3-fb58-49db-a04d-bbbfcde73fe4-kserve-provision-location\") pod \"9e61a7b3-fb58-49db-a04d-bbbfcde73fe4\" (UID: \"9e61a7b3-fb58-49db-a04d-bbbfcde73fe4\") "
Jul 01 17:21:54.431861 ip-10-0-137-244 kubenswrapper[2565]: I0701 17:21:54.431832    2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/9e61a7b3-fb58-49db-a04d-bbbfcde73fe4-kube-rbac-proxy-sar-config\") pod \"9e61a7b3-fb58-49db-a04d-bbbfcde73fe4\" (UID: \"9e61a7b3-fb58-49db-a04d-bbbfcde73fe4\") "
Jul 01 17:21:54.431861 ip-10-0-137-244 kubenswrapper[2565]: I0701 17:21:54.431848    2565 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/9e61a7b3-fb58-49db-a04d-bbbfcde73fe4-cabundle-cert" (OuterVolumeSpecName: "cabundle-cert") pod "9e61a7b3-fb58-49db-a04d-bbbfcde73fe4" (UID: "9e61a7b3-fb58-49db-a04d-bbbfcde73fe4"). InnerVolumeSpecName "cabundle-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 01 17:21:54.432055 ip-10-0-137-244 kubenswrapper[2565]: I0701 17:21:54.432039    2565 reconciler_common.go:299] "Volume detached for volume \"cabundle-cert\" (UniqueName: \"kubernetes.io/configmap/9e61a7b3-fb58-49db-a04d-bbbfcde73fe4-cabundle-cert\") on node \"ip-10-0-137-244.ec2.internal\" DevicePath \"\""
Jul 01 17:21:54.432265 ip-10-0-137-244 kubenswrapper[2565]: I0701 17:21:54.432235    2565 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/9e61a7b3-fb58-49db-a04d-bbbfcde73fe4-kube-rbac-proxy-sar-config" (OuterVolumeSpecName: "kube-rbac-proxy-sar-config") pod "9e61a7b3-fb58-49db-a04d-bbbfcde73fe4" (UID: "9e61a7b3-fb58-49db-a04d-bbbfcde73fe4"). InnerVolumeSpecName "kube-rbac-proxy-sar-config". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 01 17:21:54.433763 ip-10-0-137-244 kubenswrapper[2565]: I0701 17:21:54.433741    2565 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/9e61a7b3-fb58-49db-a04d-bbbfcde73fe4-kube-api-access-cpjpw" (OuterVolumeSpecName: "kube-api-access-cpjpw") pod "9e61a7b3-fb58-49db-a04d-bbbfcde73fe4" (UID: "9e61a7b3-fb58-49db-a04d-bbbfcde73fe4"). InnerVolumeSpecName "kube-api-access-cpjpw". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 01 17:21:54.433864 ip-10-0-137-244 kubenswrapper[2565]: I0701 17:21:54.433759    2565 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/9e61a7b3-fb58-49db-a04d-bbbfcde73fe4-proxy-tls" (OuterVolumeSpecName: "proxy-tls") pod "9e61a7b3-fb58-49db-a04d-bbbfcde73fe4" (UID: "9e61a7b3-fb58-49db-a04d-bbbfcde73fe4"). InnerVolumeSpecName "proxy-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 01 17:21:54.436615 ip-10-0-137-244 kubenswrapper[2565]: I0701 17:21:54.436587    2565 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/9e61a7b3-fb58-49db-a04d-bbbfcde73fe4-kserve-provision-location" (OuterVolumeSpecName: "kserve-provision-location") pod "9e61a7b3-fb58-49db-a04d-bbbfcde73fe4" (UID: "9e61a7b3-fb58-49db-a04d-bbbfcde73fe4"). InnerVolumeSpecName "kserve-provision-location". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 01 17:21:54.533264 ip-10-0-137-244 kubenswrapper[2565]: I0701 17:21:54.533211    2565 reconciler_common.go:299] "Volume detached for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/9e61a7b3-fb58-49db-a04d-bbbfcde73fe4-kube-rbac-proxy-sar-config\") on node \"ip-10-0-137-244.ec2.internal\" DevicePath \"\""
Jul 01 17:21:54.533264 ip-10-0-137-244 kubenswrapper[2565]: I0701 17:21:54.533231    2565 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-cpjpw\" (UniqueName: \"kubernetes.io/projected/9e61a7b3-fb58-49db-a04d-bbbfcde73fe4-kube-api-access-cpjpw\") on node \"ip-10-0-137-244.ec2.internal\" DevicePath \"\""
Jul 01 17:21:54.533264 ip-10-0-137-244 kubenswrapper[2565]: I0701 17:21:54.533241    2565 reconciler_common.go:299] "Volume detached for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/9e61a7b3-fb58-49db-a04d-bbbfcde73fe4-proxy-tls\") on node \"ip-10-0-137-244.ec2.internal\" DevicePath \"\""
Jul 01 17:21:54.533264 ip-10-0-137-244 kubenswrapper[2565]: I0701 17:21:54.533250    2565 reconciler_common.go:299] "Volume detached for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/9e61a7b3-fb58-49db-a04d-bbbfcde73fe4-kserve-provision-location\") on node \"ip-10-0-137-244.ec2.internal\" DevicePath \"\""
Jul 01 17:21:54.820765 ip-10-0-137-244 kubenswrapper[2565]: I0701 17:21:54.820695    2565 generic.go:356] "Generic (PLEG): container finished" podID="9e61a7b3-fb58-49db-a04d-bbbfcde73fe4" containerID="2f00001cbc657948e745cc82f278f79d14f9625bdc416a2d1f1c90ef0eaa915e" exitCode=0
Jul 01 17:21:54.821163 ip-10-0-137-244 kubenswrapper[2565]: I0701 17:21:54.820773    2565 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/isvc-autogluon-v2-variants-predictor-c6767c94c-ff48j"
Jul 01 17:21:54.821163 ip-10-0-137-244 kubenswrapper[2565]: I0701 17:21:54.820794    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/isvc-autogluon-v2-variants-predictor-c6767c94c-ff48j" event={"ID":"9e61a7b3-fb58-49db-a04d-bbbfcde73fe4","Type":"ContainerDied","Data":"2f00001cbc657948e745cc82f278f79d14f9625bdc416a2d1f1c90ef0eaa915e"}
Jul 01 17:21:54.821163 ip-10-0-137-244 kubenswrapper[2565]: I0701 17:21:54.820867    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/isvc-autogluon-v2-variants-predictor-c6767c94c-ff48j" event={"ID":"9e61a7b3-fb58-49db-a04d-bbbfcde73fe4","Type":"ContainerDied","Data":"7e9c4bcbd5c89ccfb78eacc149a23b9bc1c012b2e20c14c3fc8959561d8066e0"}
Jul 01 17:21:54.821163 ip-10-0-137-244 kubenswrapper[2565]: I0701 17:21:54.820903    2565 scope.go:117] "RemoveContainer" containerID="e038d278d4044d0844513cd1c8c37a64208714015600238290946fa126129035"
Jul 01 17:21:54.833840 ip-10-0-137-244 kubenswrapper[2565]: I0701 17:21:54.833819    2565 scope.go:117] "RemoveContainer" containerID="2f00001cbc657948e745cc82f278f79d14f9625bdc416a2d1f1c90ef0eaa915e"
Jul 01 17:21:54.841914 ip-10-0-137-244 kubenswrapper[2565]: I0701 17:21:54.841892    2565 scope.go:117] "RemoveContainer" containerID="8bf6197e05afc7828e3df6da554aaa58f50661fde758a062ba1db4c795baffd8"
Jul 01 17:21:54.845464 ip-10-0-137-244 kubenswrapper[2565]: I0701 17:21:54.845438    2565 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kserve-ci-e2e-test/isvc-autogluon-v2-variants-predictor-c6767c94c-ff48j"]
Jul 01 17:21:54.849529 ip-10-0-137-244 kubenswrapper[2565]: I0701 17:21:54.849501    2565 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kserve-ci-e2e-test/isvc-autogluon-v2-variants-predictor-c6767c94c-ff48j"]
Jul 01 17:21:54.850537 ip-10-0-137-244 kubenswrapper[2565]: I0701 17:21:54.850519    2565 scope.go:117] "RemoveContainer" containerID="e038d278d4044d0844513cd1c8c37a64208714015600238290946fa126129035"
Jul 01 17:21:54.850832 ip-10-0-137-244 kubenswrapper[2565]: E0701 17:21:54.850811    2565 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"e038d278d4044d0844513cd1c8c37a64208714015600238290946fa126129035\": container with ID starting with e038d278d4044d0844513cd1c8c37a64208714015600238290946fa126129035 not found: ID does not exist" containerID="e038d278d4044d0844513cd1c8c37a64208714015600238290946fa126129035"
Jul 01 17:21:54.850882 ip-10-0-137-244 kubenswrapper[2565]: I0701 17:21:54.850843    2565 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e038d278d4044d0844513cd1c8c37a64208714015600238290946fa126129035"} err="failed to get container status \"e038d278d4044d0844513cd1c8c37a64208714015600238290946fa126129035\": rpc error: code = NotFound desc = could not find container \"e038d278d4044d0844513cd1c8c37a64208714015600238290946fa126129035\": container with ID starting with e038d278d4044d0844513cd1c8c37a64208714015600238290946fa126129035 not found: ID does not exist"
Jul 01 17:21:54.850882 ip-10-0-137-244 kubenswrapper[2565]: I0701 17:21:54.850864    2565 scope.go:117] "RemoveContainer" containerID="2f00001cbc657948e745cc82f278f79d14f9625bdc416a2d1f1c90ef0eaa915e"
Jul 01 17:21:54.851215 ip-10-0-137-244 kubenswrapper[2565]: E0701 17:21:54.851198    2565 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"2f00001cbc657948e745cc82f278f79d14f9625bdc416a2d1f1c90ef0eaa915e\": container with ID starting with 2f00001cbc657948e745cc82f278f79d14f9625bdc416a2d1f1c90ef0eaa915e not found: ID does not exist" containerID="2f00001cbc657948e745cc82f278f79d14f9625bdc416a2d1f1c90ef0eaa915e"
Jul 01 17:21:54.851288 ip-10-0-137-244 kubenswrapper[2565]: I0701 17:21:54.851219    2565 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"2f00001cbc657948e745cc82f278f79d14f9625bdc416a2d1f1c90ef0eaa915e"} err="failed to get container status \"2f00001cbc657948e745cc82f278f79d14f9625bdc416a2d1f1c90ef0eaa915e\": rpc error: code = NotFound desc = could not find container \"2f00001cbc657948e745cc82f278f79d14f9625bdc416a2d1f1c90ef0eaa915e\": container with ID starting with 2f00001cbc657948e745cc82f278f79d14f9625bdc416a2d1f1c90ef0eaa915e not found: ID does not exist"
Jul 01 17:21:54.851288 ip-10-0-137-244 kubenswrapper[2565]: I0701 17:21:54.851237    2565 scope.go:117] "RemoveContainer" containerID="8bf6197e05afc7828e3df6da554aaa58f50661fde758a062ba1db4c795baffd8"
Jul 01 17:21:54.851510 ip-10-0-137-244 kubenswrapper[2565]: E0701 17:21:54.851490    2565 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"8bf6197e05afc7828e3df6da554aaa58f50661fde758a062ba1db4c795baffd8\": container with ID starting with 8bf6197e05afc7828e3df6da554aaa58f50661fde758a062ba1db4c795baffd8 not found: ID does not exist" containerID="8bf6197e05afc7828e3df6da554aaa58f50661fde758a062ba1db4c795baffd8"
Jul 01 17:21:54.851549 ip-10-0-137-244 kubenswrapper[2565]: I0701 17:21:54.851516    2565 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"8bf6197e05afc7828e3df6da554aaa58f50661fde758a062ba1db4c795baffd8"} err="failed to get container status \"8bf6197e05afc7828e3df6da554aaa58f50661fde758a062ba1db4c795baffd8\": rpc error: code = NotFound desc = could not find container \"8bf6197e05afc7828e3df6da554aaa58f50661fde758a062ba1db4c795baffd8\": container with ID starting with 8bf6197e05afc7828e3df6da554aaa58f50661fde758a062ba1db4c795baffd8 not found: ID does not exist"
Jul 01 17:21:56.248777 ip-10-0-137-244 kubenswrapper[2565]: I0701 17:21:56.248738    2565 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="9e61a7b3-fb58-49db-a04d-bbbfcde73fe4" path="/var/lib/kubelet/pods/9e61a7b3-fb58-49db-a04d-bbbfcde73fe4/volumes"
Jul 01 17:21:58.796395 ip-10-0-137-244 kubenswrapper[2565]: I0701 17:21:58.796364    2565 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kserve-ci-e2e-test/isvc-autogluon-v2-noslash-predictor-9dcfbbb84-fcdzj"]
Jul 01 17:21:58.796742 ip-10-0-137-244 kubenswrapper[2565]: I0701 17:21:58.796620    2565 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="9e61a7b3-fb58-49db-a04d-bbbfcde73fe4" containerName="kserve-container"
Jul 01 17:21:58.796742 ip-10-0-137-244 kubenswrapper[2565]: I0701 17:21:58.796630    2565 state_mem.go:107] "Deleted CPUSet assignment" podUID="9e61a7b3-fb58-49db-a04d-bbbfcde73fe4" containerName="kserve-container"
Jul 01 17:21:58.796742 ip-10-0-137-244 kubenswrapper[2565]: I0701 17:21:58.796652    2565 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="9e61a7b3-fb58-49db-a04d-bbbfcde73fe4" containerName="kube-rbac-proxy"
Jul 01 17:21:58.796742 ip-10-0-137-244 kubenswrapper[2565]: I0701 17:21:58.796683    2565 state_mem.go:107] "Deleted CPUSet assignment" podUID="9e61a7b3-fb58-49db-a04d-bbbfcde73fe4" containerName="kube-rbac-proxy"
Jul 01 17:21:58.796742 ip-10-0-137-244 kubenswrapper[2565]: I0701 17:21:58.796692    2565 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="9e61a7b3-fb58-49db-a04d-bbbfcde73fe4" containerName="storage-initializer"
Jul 01 17:21:58.796742 ip-10-0-137-244 kubenswrapper[2565]: I0701 17:21:58.796698    2565 state_mem.go:107] "Deleted CPUSet assignment" podUID="9e61a7b3-fb58-49db-a04d-bbbfcde73fe4" containerName="storage-initializer"
Jul 01 17:21:58.796742 ip-10-0-137-244 kubenswrapper[2565]: I0701 17:21:58.796742    2565 memory_manager.go:368] "RemoveStaleState removing state" podUID="9e61a7b3-fb58-49db-a04d-bbbfcde73fe4" containerName="kserve-container"
Jul 01 17:21:58.796954 ip-10-0-137-244 kubenswrapper[2565]: I0701 17:21:58.796752    2565 memory_manager.go:368] "RemoveStaleState removing state" podUID="9e61a7b3-fb58-49db-a04d-bbbfcde73fe4" containerName="kube-rbac-proxy"
Jul 01 17:21:58.801580 ip-10-0-137-244 kubenswrapper[2565]: I0701 17:21:58.801564    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/isvc-autogluon-v2-noslash-predictor-9dcfbbb84-fcdzj"
Jul 01 17:21:58.803799 ip-10-0-137-244 kubenswrapper[2565]: I0701 17:21:58.803775    2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kserve-ci-e2e-test\"/\"isvc-autogluon-v2-noslash-predictor-serving-cert\""
Jul 01 17:21:58.803928 ip-10-0-137-244 kubenswrapper[2565]: I0701 17:21:58.803848    2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kserve-ci-e2e-test\"/\"default-dockercfg-bl7zk\""
Jul 01 17:21:58.804119 ip-10-0-137-244 kubenswrapper[2565]: I0701 17:21:58.804104    2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kserve-ci-e2e-test\"/\"kube-root-ca.crt\""
Jul 01 17:21:58.804202 ip-10-0-137-244 kubenswrapper[2565]: I0701 17:21:58.804134    2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kserve-ci-e2e-test\"/\"seaweedfs-s3-creds\""
Jul 01 17:21:58.806014 ip-10-0-137-244 kubenswrapper[2565]: I0701 17:21:58.805161    2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kserve-ci-e2e-test\"/\"openshift-service-ca.crt\""
Jul 01 17:21:58.806014 ip-10-0-137-244 kubenswrapper[2565]: I0701 17:21:58.805180    2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kserve-ci-e2e-test\"/\"isvc-autogluon-v2-noslash-kube-rbac-proxy-sar-config\""
Jul 01 17:21:58.806014 ip-10-0-137-244 kubenswrapper[2565]: I0701 17:21:58.805189    2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kserve-ci-e2e-test\"/\"odh-kserve-custom-ca-bundle\""
Jul 01 17:21:58.812151 ip-10-0-137-244 kubenswrapper[2565]: I0701 17:21:58.812132    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/isvc-autogluon-v2-noslash-predictor-9dcfbbb84-fcdzj"]
Jul 01 17:21:58.863739 ip-10-0-137-244 kubenswrapper[2565]: I0701 17:21:58.863715    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-lgvf4\" (UniqueName: \"kubernetes.io/projected/4fbc0146-d07c-4375-b326-13db3405ecbb-kube-api-access-lgvf4\") pod \"isvc-autogluon-v2-noslash-predictor-9dcfbbb84-fcdzj\" (UID: \"4fbc0146-d07c-4375-b326-13db3405ecbb\") " pod="kserve-ci-e2e-test/isvc-autogluon-v2-noslash-predictor-9dcfbbb84-fcdzj"
Jul 01 17:21:58.863848 ip-10-0-137-244 kubenswrapper[2565]: I0701 17:21:58.863742    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/4fbc0146-d07c-4375-b326-13db3405ecbb-kserve-provision-location\") pod \"isvc-autogluon-v2-noslash-predictor-9dcfbbb84-fcdzj\" (UID: \"4fbc0146-d07c-4375-b326-13db3405ecbb\") " pod="kserve-ci-e2e-test/isvc-autogluon-v2-noslash-predictor-9dcfbbb84-fcdzj"
Jul 01 17:21:58.863848 ip-10-0-137-244 kubenswrapper[2565]: I0701 17:21:58.863760    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cabundle-cert\" (UniqueName: \"kubernetes.io/configmap/4fbc0146-d07c-4375-b326-13db3405ecbb-cabundle-cert\") pod \"isvc-autogluon-v2-noslash-predictor-9dcfbbb84-fcdzj\" (UID: \"4fbc0146-d07c-4375-b326-13db3405ecbb\") " pod="kserve-ci-e2e-test/isvc-autogluon-v2-noslash-predictor-9dcfbbb84-fcdzj"
Jul 01 17:21:58.863848 ip-10-0-137-244 kubenswrapper[2565]: I0701 17:21:58.863807    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/4fbc0146-d07c-4375-b326-13db3405ecbb-proxy-tls\") pod \"isvc-autogluon-v2-noslash-predictor-9dcfbbb84-fcdzj\" (UID: \"4fbc0146-d07c-4375-b326-13db3405ecbb\") " pod="kserve-ci-e2e-test/isvc-autogluon-v2-noslash-predictor-9dcfbbb84-fcdzj"
Jul 01 17:21:58.863848 ip-10-0-137-244 kubenswrapper[2565]: I0701 17:21:58.863824    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/4fbc0146-d07c-4375-b326-13db3405ecbb-kube-rbac-proxy-sar-config\") pod \"isvc-autogluon-v2-noslash-predictor-9dcfbbb84-fcdzj\" (UID: \"4fbc0146-d07c-4375-b326-13db3405ecbb\") " pod="kserve-ci-e2e-test/isvc-autogluon-v2-noslash-predictor-9dcfbbb84-fcdzj"
Jul 01 17:21:58.964810 ip-10-0-137-244 kubenswrapper[2565]: I0701 17:21:58.964785    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/4fbc0146-d07c-4375-b326-13db3405ecbb-proxy-tls\") pod \"isvc-autogluon-v2-noslash-predictor-9dcfbbb84-fcdzj\" (UID: \"4fbc0146-d07c-4375-b326-13db3405ecbb\") " pod="kserve-ci-e2e-test/isvc-autogluon-v2-noslash-predictor-9dcfbbb84-fcdzj"
Jul 01 17:21:58.964916 ip-10-0-137-244 kubenswrapper[2565]: I0701 17:21:58.964812    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/4fbc0146-d07c-4375-b326-13db3405ecbb-kube-rbac-proxy-sar-config\") pod \"isvc-autogluon-v2-noslash-predictor-9dcfbbb84-fcdzj\" (UID: \"4fbc0146-d07c-4375-b326-13db3405ecbb\") " pod="kserve-ci-e2e-test/isvc-autogluon-v2-noslash-predictor-9dcfbbb84-fcdzj"
Jul 01 17:21:58.964916 ip-10-0-137-244 kubenswrapper[2565]: I0701 17:21:58.964855    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lgvf4\" (UniqueName: \"kubernetes.io/projected/4fbc0146-d07c-4375-b326-13db3405ecbb-kube-api-access-lgvf4\") pod \"isvc-autogluon-v2-noslash-predictor-9dcfbbb84-fcdzj\" (UID: \"4fbc0146-d07c-4375-b326-13db3405ecbb\") " pod="kserve-ci-e2e-test/isvc-autogluon-v2-noslash-predictor-9dcfbbb84-fcdzj"
Jul 01 17:21:58.965017 ip-10-0-137-244 kubenswrapper[2565]: I0701 17:21:58.964971    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/4fbc0146-d07c-4375-b326-13db3405ecbb-kserve-provision-location\") pod \"isvc-autogluon-v2-noslash-predictor-9dcfbbb84-fcdzj\" (UID: \"4fbc0146-d07c-4375-b326-13db3405ecbb\") " pod="kserve-ci-e2e-test/isvc-autogluon-v2-noslash-predictor-9dcfbbb84-fcdzj"
Jul 01 17:21:58.965170 ip-10-0-137-244 kubenswrapper[2565]: I0701 17:21:58.965143    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cabundle-cert\" (UniqueName: \"kubernetes.io/configmap/4fbc0146-d07c-4375-b326-13db3405ecbb-cabundle-cert\") pod \"isvc-autogluon-v2-noslash-predictor-9dcfbbb84-fcdzj\" (UID: \"4fbc0146-d07c-4375-b326-13db3405ecbb\") " pod="kserve-ci-e2e-test/isvc-autogluon-v2-noslash-predictor-9dcfbbb84-fcdzj"
Jul 01 17:21:58.965360 ip-10-0-137-244 kubenswrapper[2565]: I0701 17:21:58.965331    2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/4fbc0146-d07c-4375-b326-13db3405ecbb-kserve-provision-location\") pod \"isvc-autogluon-v2-noslash-predictor-9dcfbbb84-fcdzj\" (UID: \"4fbc0146-d07c-4375-b326-13db3405ecbb\") " pod="kserve-ci-e2e-test/isvc-autogluon-v2-noslash-predictor-9dcfbbb84-fcdzj"
Jul 01 17:21:58.965497 ip-10-0-137-244 kubenswrapper[2565]: I0701 17:21:58.965476    2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/4fbc0146-d07c-4375-b326-13db3405ecbb-kube-rbac-proxy-sar-config\") pod \"isvc-autogluon-v2-noslash-predictor-9dcfbbb84-fcdzj\" (UID: \"4fbc0146-d07c-4375-b326-13db3405ecbb\") " pod="kserve-ci-e2e-test/isvc-autogluon-v2-noslash-predictor-9dcfbbb84-fcdzj"
Jul 01 17:21:58.965696 ip-10-0-137-244 kubenswrapper[2565]: I0701 17:21:58.965676    2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cabundle-cert\" (UniqueName: \"kubernetes.io/configmap/4fbc0146-d07c-4375-b326-13db3405ecbb-cabundle-cert\") pod \"isvc-autogluon-v2-noslash-predictor-9dcfbbb84-fcdzj\" (UID: \"4fbc0146-d07c-4375-b326-13db3405ecbb\") " pod="kserve-ci-e2e-test/isvc-autogluon-v2-noslash-predictor-9dcfbbb84-fcdzj"
Jul 01 17:21:58.967308 ip-10-0-137-244 kubenswrapper[2565]: I0701 17:21:58.967283    2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/4fbc0146-d07c-4375-b326-13db3405ecbb-proxy-tls\") pod \"isvc-autogluon-v2-noslash-predictor-9dcfbbb84-fcdzj\" (UID: \"4fbc0146-d07c-4375-b326-13db3405ecbb\") " pod="kserve-ci-e2e-test/isvc-autogluon-v2-noslash-predictor-9dcfbbb84-fcdzj"
Jul 01 17:21:58.973886 ip-10-0-137-244 kubenswrapper[2565]: I0701 17:21:58.973865    2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-lgvf4\" (UniqueName: \"kubernetes.io/projected/4fbc0146-d07c-4375-b326-13db3405ecbb-kube-api-access-lgvf4\") pod \"isvc-autogluon-v2-noslash-predictor-9dcfbbb84-fcdzj\" (UID: \"4fbc0146-d07c-4375-b326-13db3405ecbb\") " pod="kserve-ci-e2e-test/isvc-autogluon-v2-noslash-predictor-9dcfbbb84-fcdzj"
Jul 01 17:21:59.117846 ip-10-0-137-244 kubenswrapper[2565]: I0701 17:21:59.117775    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/isvc-autogluon-v2-noslash-predictor-9dcfbbb84-fcdzj"
Jul 01 17:21:59.244740 ip-10-0-137-244 kubenswrapper[2565]: I0701 17:21:59.244678    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/isvc-autogluon-v2-noslash-predictor-9dcfbbb84-fcdzj"]
Jul 01 17:21:59.246846 ip-10-0-137-244 kubenswrapper[2565]: W0701 17:21:59.246822    2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4fbc0146_d07c_4375_b326_13db3405ecbb.slice/crio-dde6cd8fecd90c0bee5c2489e12674959a48522b9324ae5642287e405088ffea WatchSource:0}: Error finding container dde6cd8fecd90c0bee5c2489e12674959a48522b9324ae5642287e405088ffea: Status 404 returned error can't find the container with id dde6cd8fecd90c0bee5c2489e12674959a48522b9324ae5642287e405088ffea
Jul 01 17:21:59.844613 ip-10-0-137-244 kubenswrapper[2565]: I0701 17:21:59.844580    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/isvc-autogluon-v2-noslash-predictor-9dcfbbb84-fcdzj" event={"ID":"4fbc0146-d07c-4375-b326-13db3405ecbb","Type":"ContainerStarted","Data":"9083bba56df82f47abb667cc261f869648a7765cfd8290b807ed48603eaccddb"}
Jul 01 17:21:59.845082 ip-10-0-137-244 kubenswrapper[2565]: I0701 17:21:59.844640    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/isvc-autogluon-v2-noslash-predictor-9dcfbbb84-fcdzj" event={"ID":"4fbc0146-d07c-4375-b326-13db3405ecbb","Type":"ContainerStarted","Data":"dde6cd8fecd90c0bee5c2489e12674959a48522b9324ae5642287e405088ffea"}
Jul 01 17:22:28.968759 ip-10-0-137-244 kubenswrapper[2565]: I0701 17:22:28.968721    2565 generic.go:356] "Generic (PLEG): container finished" podID="4fbc0146-d07c-4375-b326-13db3405ecbb" containerID="9083bba56df82f47abb667cc261f869648a7765cfd8290b807ed48603eaccddb" exitCode=0
Jul 01 17:22:28.969262 ip-10-0-137-244 kubenswrapper[2565]: I0701 17:22:28.968800    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/isvc-autogluon-v2-noslash-predictor-9dcfbbb84-fcdzj" event={"ID":"4fbc0146-d07c-4375-b326-13db3405ecbb","Type":"ContainerDied","Data":"9083bba56df82f47abb667cc261f869648a7765cfd8290b807ed48603eaccddb"}
Jul 01 17:22:29.976975 ip-10-0-137-244 kubenswrapper[2565]: I0701 17:22:29.976942    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/isvc-autogluon-v2-noslash-predictor-9dcfbbb84-fcdzj" event={"ID":"4fbc0146-d07c-4375-b326-13db3405ecbb","Type":"ContainerStarted","Data":"11a83657ac68e231c3b3fb88233cec29863d8ecc8f84cbe9e483d4049c7a3bea"}
Jul 01 17:22:29.977356 ip-10-0-137-244 kubenswrapper[2565]: I0701 17:22:29.977005    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/isvc-autogluon-v2-noslash-predictor-9dcfbbb84-fcdzj" event={"ID":"4fbc0146-d07c-4375-b326-13db3405ecbb","Type":"ContainerStarted","Data":"5434b15e7bc732bc262596dc091865cbe31265c6780f7971fc72f43766b32045"}
Jul 01 17:22:29.977356 ip-10-0-137-244 kubenswrapper[2565]: I0701 17:22:29.977214    2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve-ci-e2e-test/isvc-autogluon-v2-noslash-predictor-9dcfbbb84-fcdzj"
Jul 01 17:22:29.998249 ip-10-0-137-244 kubenswrapper[2565]: I0701 17:22:29.998207    2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kserve-ci-e2e-test/isvc-autogluon-v2-noslash-predictor-9dcfbbb84-fcdzj" podStartSLOduration=2.804076651 podStartE2EDuration="31.998194587s" totalImagesPullingTime="0s" totalInitContainerRuntime="29.194117936s" isStatefulPod=false podCreationTimestamp="2026-07-01 17:21:58 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-01 17:22:29.996595124 +0000 UTC m=+1420.464859252" watchObservedRunningTime="2026-07-01 17:22:29.998194587 +0000 UTC m=+1420.466458710"
Jul 01 17:22:30.980978 ip-10-0-137-244 kubenswrapper[2565]: I0701 17:22:30.980948    2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve-ci-e2e-test/isvc-autogluon-v2-noslash-predictor-9dcfbbb84-fcdzj"
Jul 01 17:22:36.990718 ip-10-0-137-244 kubenswrapper[2565]: I0701 17:22:36.990640    2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kserve-ci-e2e-test/isvc-autogluon-v2-noslash-predictor-9dcfbbb84-fcdzj"
Jul 01 17:23:50.215047 ip-10-0-137-244 kubenswrapper[2565]: I0701 17:23:50.214998    2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-zjsw4_bb7ebc32-d0cd-4eb0-b998-3ce298f73809/ovn-acl-logging/0.log"
Jul 01 17:23:50.216810 ip-10-0-137-244 kubenswrapper[2565]: I0701 17:23:50.216787    2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-zjsw4_bb7ebc32-d0cd-4eb0-b998-3ce298f73809/ovn-acl-logging/0.log"
Jul 01 17:24:05.247456 ip-10-0-137-244 kubenswrapper[2565]: I0701 17:24:05.247414    2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kserve-ci-e2e-test/isvc-autogluon-v2-noslash-predictor-9dcfbbb84-fcdzj"
Jul 01 17:24:08.968265 ip-10-0-137-244 kubenswrapper[2565]: I0701 17:24:08.968232    2565 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kserve-ci-e2e-test/isvc-autogluon-v2-noslash-predictor-9dcfbbb84-fcdzj"]
Jul 01 17:24:08.968673 ip-10-0-137-244 kubenswrapper[2565]: I0701 17:24:08.968460    2565 kuberuntime_container.go:872] "Killing container with a grace period" pod="kserve-ci-e2e-test/isvc-autogluon-v2-noslash-predictor-9dcfbbb84-fcdzj" podUID="4fbc0146-d07c-4375-b326-13db3405ecbb" containerName="kserve-container" containerID="cri-o://5434b15e7bc732bc262596dc091865cbe31265c6780f7971fc72f43766b32045" gracePeriod=30
Jul 01 17:24:08.968673 ip-10-0-137-244 kubenswrapper[2565]: I0701 17:24:08.968490    2565 kuberuntime_container.go:872] "Killing container with a grace period" pod="kserve-ci-e2e-test/isvc-autogluon-v2-noslash-predictor-9dcfbbb84-fcdzj" podUID="4fbc0146-d07c-4375-b326-13db3405ecbb" containerName="kube-rbac-proxy" containerID="cri-o://11a83657ac68e231c3b3fb88233cec29863d8ecc8f84cbe9e483d4049c7a3bea" gracePeriod=30
Jul 01 17:24:09.383414 ip-10-0-137-244 kubenswrapper[2565]: I0701 17:24:09.383381    2565 generic.go:356] "Generic (PLEG): container finished" podID="4fbc0146-d07c-4375-b326-13db3405ecbb" containerID="11a83657ac68e231c3b3fb88233cec29863d8ecc8f84cbe9e483d4049c7a3bea" exitCode=2
Jul 01 17:24:09.383575 ip-10-0-137-244 kubenswrapper[2565]: I0701 17:24:09.383453    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/isvc-autogluon-v2-noslash-predictor-9dcfbbb84-fcdzj" event={"ID":"4fbc0146-d07c-4375-b326-13db3405ecbb","Type":"ContainerDied","Data":"11a83657ac68e231c3b3fb88233cec29863d8ecc8f84cbe9e483d4049c7a3bea"}
Jul 01 17:24:09.707872 ip-10-0-137-244 kubenswrapper[2565]: I0701 17:24:09.707850    2565 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/isvc-autogluon-v2-noslash-predictor-9dcfbbb84-fcdzj"
Jul 01 17:24:09.765270 ip-10-0-137-244 kubenswrapper[2565]: I0701 17:24:09.765248    2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/4fbc0146-d07c-4375-b326-13db3405ecbb-kserve-provision-location\") pod \"4fbc0146-d07c-4375-b326-13db3405ecbb\" (UID: \"4fbc0146-d07c-4375-b326-13db3405ecbb\") "
Jul 01 17:24:09.765385 ip-10-0-137-244 kubenswrapper[2565]: I0701 17:24:09.765285    2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/4fbc0146-d07c-4375-b326-13db3405ecbb-proxy-tls\") pod \"4fbc0146-d07c-4375-b326-13db3405ecbb\" (UID: \"4fbc0146-d07c-4375-b326-13db3405ecbb\") "
Jul 01 17:24:09.765385 ip-10-0-137-244 kubenswrapper[2565]: I0701 17:24:09.765323    2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/4fbc0146-d07c-4375-b326-13db3405ecbb-kube-rbac-proxy-sar-config\") pod \"4fbc0146-d07c-4375-b326-13db3405ecbb\" (UID: \"4fbc0146-d07c-4375-b326-13db3405ecbb\") "
Jul 01 17:24:09.765385 ip-10-0-137-244 kubenswrapper[2565]: I0701 17:24:09.765351    2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"cabundle-cert\" (UniqueName: \"kubernetes.io/configmap/4fbc0146-d07c-4375-b326-13db3405ecbb-cabundle-cert\") pod \"4fbc0146-d07c-4375-b326-13db3405ecbb\" (UID: \"4fbc0146-d07c-4375-b326-13db3405ecbb\") "
Jul 01 17:24:09.765385 ip-10-0-137-244 kubenswrapper[2565]: I0701 17:24:09.765378    2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-lgvf4\" (UniqueName: \"kubernetes.io/projected/4fbc0146-d07c-4375-b326-13db3405ecbb-kube-api-access-lgvf4\") pod \"4fbc0146-d07c-4375-b326-13db3405ecbb\" (UID: \"4fbc0146-d07c-4375-b326-13db3405ecbb\") "
Jul 01 17:24:09.765728 ip-10-0-137-244 kubenswrapper[2565]: I0701 17:24:09.765705    2565 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/4fbc0146-d07c-4375-b326-13db3405ecbb-kube-rbac-proxy-sar-config" (OuterVolumeSpecName: "kube-rbac-proxy-sar-config") pod "4fbc0146-d07c-4375-b326-13db3405ecbb" (UID: "4fbc0146-d07c-4375-b326-13db3405ecbb"). InnerVolumeSpecName "kube-rbac-proxy-sar-config". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 01 17:24:09.765728 ip-10-0-137-244 kubenswrapper[2565]: I0701 17:24:09.765716    2565 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/4fbc0146-d07c-4375-b326-13db3405ecbb-cabundle-cert" (OuterVolumeSpecName: "cabundle-cert") pod "4fbc0146-d07c-4375-b326-13db3405ecbb" (UID: "4fbc0146-d07c-4375-b326-13db3405ecbb"). InnerVolumeSpecName "cabundle-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 01 17:24:09.767291 ip-10-0-137-244 kubenswrapper[2565]: I0701 17:24:09.767272    2565 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/4fbc0146-d07c-4375-b326-13db3405ecbb-proxy-tls" (OuterVolumeSpecName: "proxy-tls") pod "4fbc0146-d07c-4375-b326-13db3405ecbb" (UID: "4fbc0146-d07c-4375-b326-13db3405ecbb"). InnerVolumeSpecName "proxy-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 01 17:24:09.767369 ip-10-0-137-244 kubenswrapper[2565]: I0701 17:24:09.767351    2565 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/4fbc0146-d07c-4375-b326-13db3405ecbb-kube-api-access-lgvf4" (OuterVolumeSpecName: "kube-api-access-lgvf4") pod "4fbc0146-d07c-4375-b326-13db3405ecbb" (UID: "4fbc0146-d07c-4375-b326-13db3405ecbb"). InnerVolumeSpecName "kube-api-access-lgvf4". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 01 17:24:09.771381 ip-10-0-137-244 kubenswrapper[2565]: I0701 17:24:09.771362    2565 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/4fbc0146-d07c-4375-b326-13db3405ecbb-kserve-provision-location" (OuterVolumeSpecName: "kserve-provision-location") pod "4fbc0146-d07c-4375-b326-13db3405ecbb" (UID: "4fbc0146-d07c-4375-b326-13db3405ecbb"). InnerVolumeSpecName "kserve-provision-location". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 01 17:24:09.866203 ip-10-0-137-244 kubenswrapper[2565]: I0701 17:24:09.866179    2565 reconciler_common.go:299] "Volume detached for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/4fbc0146-d07c-4375-b326-13db3405ecbb-kserve-provision-location\") on node \"ip-10-0-137-244.ec2.internal\" DevicePath \"\""
Jul 01 17:24:09.866203 ip-10-0-137-244 kubenswrapper[2565]: I0701 17:24:09.866202    2565 reconciler_common.go:299] "Volume detached for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/4fbc0146-d07c-4375-b326-13db3405ecbb-proxy-tls\") on node \"ip-10-0-137-244.ec2.internal\" DevicePath \"\""
Jul 01 17:24:09.866341 ip-10-0-137-244 kubenswrapper[2565]: I0701 17:24:09.866216    2565 reconciler_common.go:299] "Volume detached for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/4fbc0146-d07c-4375-b326-13db3405ecbb-kube-rbac-proxy-sar-config\") on node \"ip-10-0-137-244.ec2.internal\" DevicePath \"\""
Jul 01 17:24:09.866341 ip-10-0-137-244 kubenswrapper[2565]: I0701 17:24:09.866254    2565 reconciler_common.go:299] "Volume detached for volume \"cabundle-cert\" (UniqueName: \"kubernetes.io/configmap/4fbc0146-d07c-4375-b326-13db3405ecbb-cabundle-cert\") on node \"ip-10-0-137-244.ec2.internal\" DevicePath \"\""
Jul 01 17:24:09.866341 ip-10-0-137-244 kubenswrapper[2565]: I0701 17:24:09.866270    2565 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-lgvf4\" (UniqueName: \"kubernetes.io/projected/4fbc0146-d07c-4375-b326-13db3405ecbb-kube-api-access-lgvf4\") on node \"ip-10-0-137-244.ec2.internal\" DevicePath \"\""
Jul 01 17:24:10.389567 ip-10-0-137-244 kubenswrapper[2565]: I0701 17:24:10.389533    2565 generic.go:356] "Generic (PLEG): container finished" podID="4fbc0146-d07c-4375-b326-13db3405ecbb" containerID="5434b15e7bc732bc262596dc091865cbe31265c6780f7971fc72f43766b32045" exitCode=0
Jul 01 17:24:10.389977 ip-10-0-137-244 kubenswrapper[2565]: I0701 17:24:10.389595    2565 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/isvc-autogluon-v2-noslash-predictor-9dcfbbb84-fcdzj"
Jul 01 17:24:10.389977 ip-10-0-137-244 kubenswrapper[2565]: I0701 17:24:10.389617    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/isvc-autogluon-v2-noslash-predictor-9dcfbbb84-fcdzj" event={"ID":"4fbc0146-d07c-4375-b326-13db3405ecbb","Type":"ContainerDied","Data":"5434b15e7bc732bc262596dc091865cbe31265c6780f7971fc72f43766b32045"}
Jul 01 17:24:10.389977 ip-10-0-137-244 kubenswrapper[2565]: I0701 17:24:10.389676    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/isvc-autogluon-v2-noslash-predictor-9dcfbbb84-fcdzj" event={"ID":"4fbc0146-d07c-4375-b326-13db3405ecbb","Type":"ContainerDied","Data":"dde6cd8fecd90c0bee5c2489e12674959a48522b9324ae5642287e405088ffea"}
Jul 01 17:24:10.389977 ip-10-0-137-244 kubenswrapper[2565]: I0701 17:24:10.389706    2565 scope.go:117] "RemoveContainer" containerID="11a83657ac68e231c3b3fb88233cec29863d8ecc8f84cbe9e483d4049c7a3bea"
Jul 01 17:24:10.398109 ip-10-0-137-244 kubenswrapper[2565]: I0701 17:24:10.398079    2565 scope.go:117] "RemoveContainer" containerID="5434b15e7bc732bc262596dc091865cbe31265c6780f7971fc72f43766b32045"
Jul 01 17:24:10.405117 ip-10-0-137-244 kubenswrapper[2565]: I0701 17:24:10.405099    2565 scope.go:117] "RemoveContainer" containerID="9083bba56df82f47abb667cc261f869648a7765cfd8290b807ed48603eaccddb"
Jul 01 17:24:10.408072 ip-10-0-137-244 kubenswrapper[2565]: I0701 17:24:10.408050    2565 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kserve-ci-e2e-test/isvc-autogluon-v2-noslash-predictor-9dcfbbb84-fcdzj"]
Jul 01 17:24:10.412442 ip-10-0-137-244 kubenswrapper[2565]: I0701 17:24:10.412422    2565 scope.go:117] "RemoveContainer" containerID="11a83657ac68e231c3b3fb88233cec29863d8ecc8f84cbe9e483d4049c7a3bea"
Jul 01 17:24:10.412793 ip-10-0-137-244 kubenswrapper[2565]: E0701 17:24:10.412757    2565 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"11a83657ac68e231c3b3fb88233cec29863d8ecc8f84cbe9e483d4049c7a3bea\": container with ID starting with 11a83657ac68e231c3b3fb88233cec29863d8ecc8f84cbe9e483d4049c7a3bea not found: ID does not exist" containerID="11a83657ac68e231c3b3fb88233cec29863d8ecc8f84cbe9e483d4049c7a3bea"
Jul 01 17:24:10.412894 ip-10-0-137-244 kubenswrapper[2565]: I0701 17:24:10.412793    2565 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"11a83657ac68e231c3b3fb88233cec29863d8ecc8f84cbe9e483d4049c7a3bea"} err="failed to get container status \"11a83657ac68e231c3b3fb88233cec29863d8ecc8f84cbe9e483d4049c7a3bea\": rpc error: code = NotFound desc = could not find container \"11a83657ac68e231c3b3fb88233cec29863d8ecc8f84cbe9e483d4049c7a3bea\": container with ID starting with 11a83657ac68e231c3b3fb88233cec29863d8ecc8f84cbe9e483d4049c7a3bea not found: ID does not exist"
Jul 01 17:24:10.412894 ip-10-0-137-244 kubenswrapper[2565]: I0701 17:24:10.412811    2565 scope.go:117] "RemoveContainer" containerID="5434b15e7bc732bc262596dc091865cbe31265c6780f7971fc72f43766b32045"
Jul 01 17:24:10.413212 ip-10-0-137-244 kubenswrapper[2565]: E0701 17:24:10.413187    2565 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"5434b15e7bc732bc262596dc091865cbe31265c6780f7971fc72f43766b32045\": container with ID starting with 5434b15e7bc732bc262596dc091865cbe31265c6780f7971fc72f43766b32045 not found: ID does not exist" containerID="5434b15e7bc732bc262596dc091865cbe31265c6780f7971fc72f43766b32045"
Jul 01 17:24:10.413335 ip-10-0-137-244 kubenswrapper[2565]: I0701 17:24:10.413280    2565 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"5434b15e7bc732bc262596dc091865cbe31265c6780f7971fc72f43766b32045"} err="failed to get container status \"5434b15e7bc732bc262596dc091865cbe31265c6780f7971fc72f43766b32045\": rpc error: code = NotFound desc = could not find container \"5434b15e7bc732bc262596dc091865cbe31265c6780f7971fc72f43766b32045\": container with ID starting with 5434b15e7bc732bc262596dc091865cbe31265c6780f7971fc72f43766b32045 not found: ID does not exist"
Jul 01 17:24:10.413335 ip-10-0-137-244 kubenswrapper[2565]: I0701 17:24:10.413305    2565 scope.go:117] "RemoveContainer" containerID="9083bba56df82f47abb667cc261f869648a7765cfd8290b807ed48603eaccddb"
Jul 01 17:24:10.413647 ip-10-0-137-244 kubenswrapper[2565]: E0701 17:24:10.413629    2565 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"9083bba56df82f47abb667cc261f869648a7765cfd8290b807ed48603eaccddb\": container with ID starting with 9083bba56df82f47abb667cc261f869648a7765cfd8290b807ed48603eaccddb not found: ID does not exist" containerID="9083bba56df82f47abb667cc261f869648a7765cfd8290b807ed48603eaccddb"
Jul 01 17:24:10.413694 ip-10-0-137-244 kubenswrapper[2565]: I0701 17:24:10.413653    2565 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"9083bba56df82f47abb667cc261f869648a7765cfd8290b807ed48603eaccddb"} err="failed to get container status \"9083bba56df82f47abb667cc261f869648a7765cfd8290b807ed48603eaccddb\": rpc error: code = NotFound desc = could not find container \"9083bba56df82f47abb667cc261f869648a7765cfd8290b807ed48603eaccddb\": container with ID starting with 9083bba56df82f47abb667cc261f869648a7765cfd8290b807ed48603eaccddb not found: ID does not exist"
Jul 01 17:24:10.414407 ip-10-0-137-244 kubenswrapper[2565]: I0701 17:24:10.414389    2565 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kserve-ci-e2e-test/isvc-autogluon-v2-noslash-predictor-9dcfbbb84-fcdzj"]
Jul 01 17:24:12.248060 ip-10-0-137-244 kubenswrapper[2565]: I0701 17:24:12.247949    2565 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="4fbc0146-d07c-4375-b326-13db3405ecbb" path="/var/lib/kubelet/pods/4fbc0146-d07c-4375-b326-13db3405ecbb/volumes"
Jul 01 17:24:14.101458 ip-10-0-137-244 kubenswrapper[2565]: I0701 17:24:14.101426    2565 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kserve-ci-e2e-test/isvc-autogluon-ts-v1-predictor-89d67dbf7-5fqpl"]
Jul 01 17:24:14.101807 ip-10-0-137-244 kubenswrapper[2565]: I0701 17:24:14.101713    2565 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="4fbc0146-d07c-4375-b326-13db3405ecbb" containerName="kserve-container"
Jul 01 17:24:14.101807 ip-10-0-137-244 kubenswrapper[2565]: I0701 17:24:14.101724    2565 state_mem.go:107] "Deleted CPUSet assignment" podUID="4fbc0146-d07c-4375-b326-13db3405ecbb" containerName="kserve-container"
Jul 01 17:24:14.101807 ip-10-0-137-244 kubenswrapper[2565]: I0701 17:24:14.101736    2565 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="4fbc0146-d07c-4375-b326-13db3405ecbb" containerName="storage-initializer"
Jul 01 17:24:14.101807 ip-10-0-137-244 kubenswrapper[2565]: I0701 17:24:14.101742    2565 state_mem.go:107] "Deleted CPUSet assignment" podUID="4fbc0146-d07c-4375-b326-13db3405ecbb" containerName="storage-initializer"
Jul 01 17:24:14.101807 ip-10-0-137-244 kubenswrapper[2565]: I0701 17:24:14.101750    2565 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="4fbc0146-d07c-4375-b326-13db3405ecbb" containerName="kube-rbac-proxy"
Jul 01 17:24:14.101807 ip-10-0-137-244 kubenswrapper[2565]: I0701 17:24:14.101755    2565 state_mem.go:107] "Deleted CPUSet assignment" podUID="4fbc0146-d07c-4375-b326-13db3405ecbb" containerName="kube-rbac-proxy"
Jul 01 17:24:14.101807 ip-10-0-137-244 kubenswrapper[2565]: I0701 17:24:14.101797    2565 memory_manager.go:368] "RemoveStaleState removing state" podUID="4fbc0146-d07c-4375-b326-13db3405ecbb" containerName="kserve-container"
Jul 01 17:24:14.101807 ip-10-0-137-244 kubenswrapper[2565]: I0701 17:24:14.101805    2565 memory_manager.go:368] "RemoveStaleState removing state" podUID="4fbc0146-d07c-4375-b326-13db3405ecbb" containerName="kube-rbac-proxy"
Jul 01 17:24:14.106328 ip-10-0-137-244 kubenswrapper[2565]: I0701 17:24:14.106312    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/isvc-autogluon-ts-v1-predictor-89d67dbf7-5fqpl"
Jul 01 17:24:14.108930 ip-10-0-137-244 kubenswrapper[2565]: I0701 17:24:14.108904    2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kserve-ci-e2e-test\"/\"seaweedfs-s3-creds\""
Jul 01 17:24:14.108930 ip-10-0-137-244 kubenswrapper[2565]: I0701 17:24:14.108921    2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kserve-ci-e2e-test\"/\"kube-root-ca.crt\""
Jul 01 17:24:14.109180 ip-10-0-137-244 kubenswrapper[2565]: I0701 17:24:14.108921    2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kserve-ci-e2e-test\"/\"isvc-autogluon-ts-v1-predictor-serving-cert\""
Jul 01 17:24:14.109180 ip-10-0-137-244 kubenswrapper[2565]: I0701 17:24:14.108958    2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kserve-ci-e2e-test\"/\"openshift-service-ca.crt\""
Jul 01 17:24:14.109180 ip-10-0-137-244 kubenswrapper[2565]: I0701 17:24:14.108924    2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kserve-ci-e2e-test\"/\"default-dockercfg-bl7zk\""
Jul 01 17:24:14.109180 ip-10-0-137-244 kubenswrapper[2565]: I0701 17:24:14.108921    2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kserve-ci-e2e-test\"/\"isvc-autogluon-ts-v1-kube-rbac-proxy-sar-config\""
Jul 01 17:24:14.109565 ip-10-0-137-244 kubenswrapper[2565]: I0701 17:24:14.109550    2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kserve-ci-e2e-test\"/\"odh-kserve-custom-ca-bundle\""
Jul 01 17:24:14.116157 ip-10-0-137-244 kubenswrapper[2565]: I0701 17:24:14.116134    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/isvc-autogluon-ts-v1-predictor-89d67dbf7-5fqpl"]
Jul 01 17:24:14.194733 ip-10-0-137-244 kubenswrapper[2565]: I0701 17:24:14.194709    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cabundle-cert\" (UniqueName: \"kubernetes.io/configmap/328e7fd3-77b8-475a-88d0-d456b0ac7b44-cabundle-cert\") pod \"isvc-autogluon-ts-v1-predictor-89d67dbf7-5fqpl\" (UID: \"328e7fd3-77b8-475a-88d0-d456b0ac7b44\") " pod="kserve-ci-e2e-test/isvc-autogluon-ts-v1-predictor-89d67dbf7-5fqpl"
Jul 01 17:24:14.194836 ip-10-0-137-244 kubenswrapper[2565]: I0701 17:24:14.194747    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zsftp\" (UniqueName: \"kubernetes.io/projected/328e7fd3-77b8-475a-88d0-d456b0ac7b44-kube-api-access-zsftp\") pod \"isvc-autogluon-ts-v1-predictor-89d67dbf7-5fqpl\" (UID: \"328e7fd3-77b8-475a-88d0-d456b0ac7b44\") " pod="kserve-ci-e2e-test/isvc-autogluon-ts-v1-predictor-89d67dbf7-5fqpl"
Jul 01 17:24:14.194836 ip-10-0-137-244 kubenswrapper[2565]: I0701 17:24:14.194773    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/328e7fd3-77b8-475a-88d0-d456b0ac7b44-proxy-tls\") pod \"isvc-autogluon-ts-v1-predictor-89d67dbf7-5fqpl\" (UID: \"328e7fd3-77b8-475a-88d0-d456b0ac7b44\") " pod="kserve-ci-e2e-test/isvc-autogluon-ts-v1-predictor-89d67dbf7-5fqpl"
Jul 01 17:24:14.194916 ip-10-0-137-244 kubenswrapper[2565]: I0701 17:24:14.194854    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/328e7fd3-77b8-475a-88d0-d456b0ac7b44-kube-rbac-proxy-sar-config\") pod \"isvc-autogluon-ts-v1-predictor-89d67dbf7-5fqpl\" (UID: \"328e7fd3-77b8-475a-88d0-d456b0ac7b44\") " pod="kserve-ci-e2e-test/isvc-autogluon-ts-v1-predictor-89d67dbf7-5fqpl"
Jul 01 17:24:14.194916 ip-10-0-137-244 kubenswrapper[2565]: I0701 17:24:14.194889    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/328e7fd3-77b8-475a-88d0-d456b0ac7b44-kserve-provision-location\") pod \"isvc-autogluon-ts-v1-predictor-89d67dbf7-5fqpl\" (UID: \"328e7fd3-77b8-475a-88d0-d456b0ac7b44\") " pod="kserve-ci-e2e-test/isvc-autogluon-ts-v1-predictor-89d67dbf7-5fqpl"
Jul 01 17:24:14.295573 ip-10-0-137-244 kubenswrapper[2565]: I0701 17:24:14.295540    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cabundle-cert\" (UniqueName: \"kubernetes.io/configmap/328e7fd3-77b8-475a-88d0-d456b0ac7b44-cabundle-cert\") pod \"isvc-autogluon-ts-v1-predictor-89d67dbf7-5fqpl\" (UID: \"328e7fd3-77b8-475a-88d0-d456b0ac7b44\") " pod="kserve-ci-e2e-test/isvc-autogluon-ts-v1-predictor-89d67dbf7-5fqpl"
Jul 01 17:24:14.295700 ip-10-0-137-244 kubenswrapper[2565]: I0701 17:24:14.295588    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zsftp\" (UniqueName: \"kubernetes.io/projected/328e7fd3-77b8-475a-88d0-d456b0ac7b44-kube-api-access-zsftp\") pod \"isvc-autogluon-ts-v1-predictor-89d67dbf7-5fqpl\" (UID: \"328e7fd3-77b8-475a-88d0-d456b0ac7b44\") " pod="kserve-ci-e2e-test/isvc-autogluon-ts-v1-predictor-89d67dbf7-5fqpl"
Jul 01 17:24:14.295700 ip-10-0-137-244 kubenswrapper[2565]: I0701 17:24:14.295607    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/328e7fd3-77b8-475a-88d0-d456b0ac7b44-proxy-tls\") pod \"isvc-autogluon-ts-v1-predictor-89d67dbf7-5fqpl\" (UID: \"328e7fd3-77b8-475a-88d0-d456b0ac7b44\") " pod="kserve-ci-e2e-test/isvc-autogluon-ts-v1-predictor-89d67dbf7-5fqpl"
Jul 01 17:24:14.295796 ip-10-0-137-244 kubenswrapper[2565]: E0701 17:24:14.295718    2565 secret.go:189] Couldn't get secret kserve-ci-e2e-test/isvc-autogluon-ts-v1-predictor-serving-cert: secret "isvc-autogluon-ts-v1-predictor-serving-cert" not found
Jul 01 17:24:14.295796 ip-10-0-137-244 kubenswrapper[2565]: E0701 17:24:14.295783    2565 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/328e7fd3-77b8-475a-88d0-d456b0ac7b44-proxy-tls podName:328e7fd3-77b8-475a-88d0-d456b0ac7b44 nodeName:}" failed. No retries permitted until 2026-07-01 17:24:14.795767874 +0000 UTC m=+1525.264031977 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "proxy-tls" (UniqueName: "kubernetes.io/secret/328e7fd3-77b8-475a-88d0-d456b0ac7b44-proxy-tls") pod "isvc-autogluon-ts-v1-predictor-89d67dbf7-5fqpl" (UID: "328e7fd3-77b8-475a-88d0-d456b0ac7b44") : secret "isvc-autogluon-ts-v1-predictor-serving-cert" not found
Jul 01 17:24:14.295946 ip-10-0-137-244 kubenswrapper[2565]: I0701 17:24:14.295926    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/328e7fd3-77b8-475a-88d0-d456b0ac7b44-kube-rbac-proxy-sar-config\") pod \"isvc-autogluon-ts-v1-predictor-89d67dbf7-5fqpl\" (UID: \"328e7fd3-77b8-475a-88d0-d456b0ac7b44\") " pod="kserve-ci-e2e-test/isvc-autogluon-ts-v1-predictor-89d67dbf7-5fqpl"
Jul 01 17:24:14.296045 ip-10-0-137-244 kubenswrapper[2565]: I0701 17:24:14.296000    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/328e7fd3-77b8-475a-88d0-d456b0ac7b44-kserve-provision-location\") pod \"isvc-autogluon-ts-v1-predictor-89d67dbf7-5fqpl\" (UID: \"328e7fd3-77b8-475a-88d0-d456b0ac7b44\") " pod="kserve-ci-e2e-test/isvc-autogluon-ts-v1-predictor-89d67dbf7-5fqpl"
Jul 01 17:24:14.296254 ip-10-0-137-244 kubenswrapper[2565]: I0701 17:24:14.296231    2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cabundle-cert\" (UniqueName: \"kubernetes.io/configmap/328e7fd3-77b8-475a-88d0-d456b0ac7b44-cabundle-cert\") pod \"isvc-autogluon-ts-v1-predictor-89d67dbf7-5fqpl\" (UID: \"328e7fd3-77b8-475a-88d0-d456b0ac7b44\") " pod="kserve-ci-e2e-test/isvc-autogluon-ts-v1-predictor-89d67dbf7-5fqpl"
Jul 01 17:24:14.296347 ip-10-0-137-244 kubenswrapper[2565]: I0701 17:24:14.296331    2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/328e7fd3-77b8-475a-88d0-d456b0ac7b44-kserve-provision-location\") pod \"isvc-autogluon-ts-v1-predictor-89d67dbf7-5fqpl\" (UID: \"328e7fd3-77b8-475a-88d0-d456b0ac7b44\") " pod="kserve-ci-e2e-test/isvc-autogluon-ts-v1-predictor-89d67dbf7-5fqpl"
Jul 01 17:24:14.296474 ip-10-0-137-244 kubenswrapper[2565]: I0701 17:24:14.296458    2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/328e7fd3-77b8-475a-88d0-d456b0ac7b44-kube-rbac-proxy-sar-config\") pod \"isvc-autogluon-ts-v1-predictor-89d67dbf7-5fqpl\" (UID: \"328e7fd3-77b8-475a-88d0-d456b0ac7b44\") " pod="kserve-ci-e2e-test/isvc-autogluon-ts-v1-predictor-89d67dbf7-5fqpl"
Jul 01 17:24:14.305577 ip-10-0-137-244 kubenswrapper[2565]: I0701 17:24:14.305556    2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-zsftp\" (UniqueName: \"kubernetes.io/projected/328e7fd3-77b8-475a-88d0-d456b0ac7b44-kube-api-access-zsftp\") pod \"isvc-autogluon-ts-v1-predictor-89d67dbf7-5fqpl\" (UID: \"328e7fd3-77b8-475a-88d0-d456b0ac7b44\") " pod="kserve-ci-e2e-test/isvc-autogluon-ts-v1-predictor-89d67dbf7-5fqpl"
Jul 01 17:24:14.799986 ip-10-0-137-244 kubenswrapper[2565]: I0701 17:24:14.799957    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/328e7fd3-77b8-475a-88d0-d456b0ac7b44-proxy-tls\") pod \"isvc-autogluon-ts-v1-predictor-89d67dbf7-5fqpl\" (UID: \"328e7fd3-77b8-475a-88d0-d456b0ac7b44\") " pod="kserve-ci-e2e-test/isvc-autogluon-ts-v1-predictor-89d67dbf7-5fqpl"
Jul 01 17:24:14.802495 ip-10-0-137-244 kubenswrapper[2565]: I0701 17:24:14.802472    2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/328e7fd3-77b8-475a-88d0-d456b0ac7b44-proxy-tls\") pod \"isvc-autogluon-ts-v1-predictor-89d67dbf7-5fqpl\" (UID: \"328e7fd3-77b8-475a-88d0-d456b0ac7b44\") " pod="kserve-ci-e2e-test/isvc-autogluon-ts-v1-predictor-89d67dbf7-5fqpl"
Jul 01 17:24:15.017174 ip-10-0-137-244 kubenswrapper[2565]: I0701 17:24:15.017147    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/isvc-autogluon-ts-v1-predictor-89d67dbf7-5fqpl"
Jul 01 17:24:15.151532 ip-10-0-137-244 kubenswrapper[2565]: I0701 17:24:15.151503    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/isvc-autogluon-ts-v1-predictor-89d67dbf7-5fqpl"]
Jul 01 17:24:15.154787 ip-10-0-137-244 kubenswrapper[2565]: W0701 17:24:15.154763    2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod328e7fd3_77b8_475a_88d0_d456b0ac7b44.slice/crio-7a5f162da12f6555e0f5863fa76952c04daa1de9d999f3d61c7831736c709141 WatchSource:0}: Error finding container 7a5f162da12f6555e0f5863fa76952c04daa1de9d999f3d61c7831736c709141: Status 404 returned error can't find the container with id 7a5f162da12f6555e0f5863fa76952c04daa1de9d999f3d61c7831736c709141
Jul 01 17:24:15.410724 ip-10-0-137-244 kubenswrapper[2565]: I0701 17:24:15.410698    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/isvc-autogluon-ts-v1-predictor-89d67dbf7-5fqpl" event={"ID":"328e7fd3-77b8-475a-88d0-d456b0ac7b44","Type":"ContainerStarted","Data":"9f80690dc5b89198c0833e8b6f2bb0802f021e1c915ce10fb47190991f055322"}
Jul 01 17:24:15.410859 ip-10-0-137-244 kubenswrapper[2565]: I0701 17:24:15.410756    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/isvc-autogluon-ts-v1-predictor-89d67dbf7-5fqpl" event={"ID":"328e7fd3-77b8-475a-88d0-d456b0ac7b44","Type":"ContainerStarted","Data":"7a5f162da12f6555e0f5863fa76952c04daa1de9d999f3d61c7831736c709141"}
Jul 01 17:24:24.446534 ip-10-0-137-244 kubenswrapper[2565]: I0701 17:24:24.446506    2565 generic.go:356] "Generic (PLEG): container finished" podID="328e7fd3-77b8-475a-88d0-d456b0ac7b44" containerID="9f80690dc5b89198c0833e8b6f2bb0802f021e1c915ce10fb47190991f055322" exitCode=0
Jul 01 17:24:24.446867 ip-10-0-137-244 kubenswrapper[2565]: I0701 17:24:24.446581    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/isvc-autogluon-ts-v1-predictor-89d67dbf7-5fqpl" event={"ID":"328e7fd3-77b8-475a-88d0-d456b0ac7b44","Type":"ContainerDied","Data":"9f80690dc5b89198c0833e8b6f2bb0802f021e1c915ce10fb47190991f055322"}
Jul 01 17:24:25.452931 ip-10-0-137-244 kubenswrapper[2565]: I0701 17:24:25.452896    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/isvc-autogluon-ts-v1-predictor-89d67dbf7-5fqpl" event={"ID":"328e7fd3-77b8-475a-88d0-d456b0ac7b44","Type":"ContainerStarted","Data":"96e5d1eb74ac875fd841e01e3ee113fc32ad62ad9283de75234c11ac2a9c5ed8"}
Jul 01 17:24:25.453331 ip-10-0-137-244 kubenswrapper[2565]: I0701 17:24:25.452965    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/isvc-autogluon-ts-v1-predictor-89d67dbf7-5fqpl" event={"ID":"328e7fd3-77b8-475a-88d0-d456b0ac7b44","Type":"ContainerStarted","Data":"30cbb5243d597c63f91fa4f731a18f7e0cb6becf005b20696cb93a91fc155f15"}
Jul 01 17:24:25.453331 ip-10-0-137-244 kubenswrapper[2565]: I0701 17:24:25.453158    2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve-ci-e2e-test/isvc-autogluon-ts-v1-predictor-89d67dbf7-5fqpl"
Jul 01 17:24:25.496626 ip-10-0-137-244 kubenswrapper[2565]: I0701 17:24:25.496581    2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kserve-ci-e2e-test/isvc-autogluon-ts-v1-predictor-89d67dbf7-5fqpl" podStartSLOduration=2.278406733 podStartE2EDuration="11.496564536s" totalImagesPullingTime="0s" totalInitContainerRuntime="9.218157803s" isStatefulPod=false podCreationTimestamp="2026-07-01 17:24:14 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-01 17:24:25.494439771 +0000 UTC m=+1535.962703894" watchObservedRunningTime="2026-07-01 17:24:25.496564536 +0000 UTC m=+1535.964828658"
Jul 01 17:24:26.456991 ip-10-0-137-244 kubenswrapper[2565]: I0701 17:24:26.456967    2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve-ci-e2e-test/isvc-autogluon-ts-v1-predictor-89d67dbf7-5fqpl"
Jul 01 17:24:26.458254 ip-10-0-137-244 kubenswrapper[2565]: I0701 17:24:26.458228    2565 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-autogluon-ts-v1-predictor-89d67dbf7-5fqpl" podUID="328e7fd3-77b8-475a-88d0-d456b0ac7b44" containerName="kserve-container" probeResult="failure" output="dial tcp 10.133.0.33:8080: connect: connection refused"
Jul 01 17:24:27.460658 ip-10-0-137-244 kubenswrapper[2565]: I0701 17:24:27.460619    2565 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-autogluon-ts-v1-predictor-89d67dbf7-5fqpl" podUID="328e7fd3-77b8-475a-88d0-d456b0ac7b44" containerName="kserve-container" probeResult="failure" output="dial tcp 10.133.0.33:8080: connect: connection refused"
Jul 01 17:24:32.464717 ip-10-0-137-244 kubenswrapper[2565]: I0701 17:24:32.464687    2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kserve-ci-e2e-test/isvc-autogluon-ts-v1-predictor-89d67dbf7-5fqpl"
Jul 01 17:24:32.465230 ip-10-0-137-244 kubenswrapper[2565]: I0701 17:24:32.465195    2565 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-autogluon-ts-v1-predictor-89d67dbf7-5fqpl" podUID="328e7fd3-77b8-475a-88d0-d456b0ac7b44" containerName="kserve-container" probeResult="failure" output="dial tcp 10.133.0.33:8080: connect: connection refused"
Jul 01 17:24:42.465749 ip-10-0-137-244 kubenswrapper[2565]: I0701 17:24:42.465721    2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kserve-ci-e2e-test/isvc-autogluon-ts-v1-predictor-89d67dbf7-5fqpl"
Jul 01 17:24:44.147221 ip-10-0-137-244 kubenswrapper[2565]: I0701 17:24:44.147189    2565 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kserve-ci-e2e-test/isvc-autogluon-ts-v1-predictor-89d67dbf7-5fqpl"]
Jul 01 17:24:44.147634 ip-10-0-137-244 kubenswrapper[2565]: I0701 17:24:44.147402    2565 kuberuntime_container.go:872] "Killing container with a grace period" pod="kserve-ci-e2e-test/isvc-autogluon-ts-v1-predictor-89d67dbf7-5fqpl" podUID="328e7fd3-77b8-475a-88d0-d456b0ac7b44" containerName="kserve-container" containerID="cri-o://30cbb5243d597c63f91fa4f731a18f7e0cb6becf005b20696cb93a91fc155f15" gracePeriod=30
Jul 01 17:24:44.147634 ip-10-0-137-244 kubenswrapper[2565]: I0701 17:24:44.147428    2565 kuberuntime_container.go:872] "Killing container with a grace period" pod="kserve-ci-e2e-test/isvc-autogluon-ts-v1-predictor-89d67dbf7-5fqpl" podUID="328e7fd3-77b8-475a-88d0-d456b0ac7b44" containerName="kube-rbac-proxy" containerID="cri-o://96e5d1eb74ac875fd841e01e3ee113fc32ad62ad9283de75234c11ac2a9c5ed8" gracePeriod=30
Jul 01 17:24:44.532558 ip-10-0-137-244 kubenswrapper[2565]: I0701 17:24:44.532458    2565 generic.go:356] "Generic (PLEG): container finished" podID="328e7fd3-77b8-475a-88d0-d456b0ac7b44" containerID="96e5d1eb74ac875fd841e01e3ee113fc32ad62ad9283de75234c11ac2a9c5ed8" exitCode=2
Jul 01 17:24:44.532558 ip-10-0-137-244 kubenswrapper[2565]: I0701 17:24:44.532536    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/isvc-autogluon-ts-v1-predictor-89d67dbf7-5fqpl" event={"ID":"328e7fd3-77b8-475a-88d0-d456b0ac7b44","Type":"ContainerDied","Data":"96e5d1eb74ac875fd841e01e3ee113fc32ad62ad9283de75234c11ac2a9c5ed8"}
Jul 01 17:24:44.895882 ip-10-0-137-244 kubenswrapper[2565]: I0701 17:24:44.895858    2565 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/isvc-autogluon-ts-v1-predictor-89d67dbf7-5fqpl"
Jul 01 17:24:45.016204 ip-10-0-137-244 kubenswrapper[2565]: I0701 17:24:45.016179    2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/328e7fd3-77b8-475a-88d0-d456b0ac7b44-proxy-tls\") pod \"328e7fd3-77b8-475a-88d0-d456b0ac7b44\" (UID: \"328e7fd3-77b8-475a-88d0-d456b0ac7b44\") "
Jul 01 17:24:45.016345 ip-10-0-137-244 kubenswrapper[2565]: I0701 17:24:45.016222    2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/328e7fd3-77b8-475a-88d0-d456b0ac7b44-kube-rbac-proxy-sar-config\") pod \"328e7fd3-77b8-475a-88d0-d456b0ac7b44\" (UID: \"328e7fd3-77b8-475a-88d0-d456b0ac7b44\") "
Jul 01 17:24:45.016345 ip-10-0-137-244 kubenswrapper[2565]: I0701 17:24:45.016261    2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/328e7fd3-77b8-475a-88d0-d456b0ac7b44-kserve-provision-location\") pod \"328e7fd3-77b8-475a-88d0-d456b0ac7b44\" (UID: \"328e7fd3-77b8-475a-88d0-d456b0ac7b44\") "
Jul 01 17:24:45.016345 ip-10-0-137-244 kubenswrapper[2565]: I0701 17:24:45.016291    2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"cabundle-cert\" (UniqueName: \"kubernetes.io/configmap/328e7fd3-77b8-475a-88d0-d456b0ac7b44-cabundle-cert\") pod \"328e7fd3-77b8-475a-88d0-d456b0ac7b44\" (UID: \"328e7fd3-77b8-475a-88d0-d456b0ac7b44\") "
Jul 01 17:24:45.016345 ip-10-0-137-244 kubenswrapper[2565]: I0701 17:24:45.016330    2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-zsftp\" (UniqueName: \"kubernetes.io/projected/328e7fd3-77b8-475a-88d0-d456b0ac7b44-kube-api-access-zsftp\") pod \"328e7fd3-77b8-475a-88d0-d456b0ac7b44\" (UID: \"328e7fd3-77b8-475a-88d0-d456b0ac7b44\") "
Jul 01 17:24:45.016648 ip-10-0-137-244 kubenswrapper[2565]: I0701 17:24:45.016621    2565 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/328e7fd3-77b8-475a-88d0-d456b0ac7b44-cabundle-cert" (OuterVolumeSpecName: "cabundle-cert") pod "328e7fd3-77b8-475a-88d0-d456b0ac7b44" (UID: "328e7fd3-77b8-475a-88d0-d456b0ac7b44"). InnerVolumeSpecName "cabundle-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 01 17:24:45.016713 ip-10-0-137-244 kubenswrapper[2565]: I0701 17:24:45.016627    2565 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/328e7fd3-77b8-475a-88d0-d456b0ac7b44-kube-rbac-proxy-sar-config" (OuterVolumeSpecName: "kube-rbac-proxy-sar-config") pod "328e7fd3-77b8-475a-88d0-d456b0ac7b44" (UID: "328e7fd3-77b8-475a-88d0-d456b0ac7b44"). InnerVolumeSpecName "kube-rbac-proxy-sar-config". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 01 17:24:45.017195 ip-10-0-137-244 kubenswrapper[2565]: I0701 17:24:45.017171    2565 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/328e7fd3-77b8-475a-88d0-d456b0ac7b44-kserve-provision-location" (OuterVolumeSpecName: "kserve-provision-location") pod "328e7fd3-77b8-475a-88d0-d456b0ac7b44" (UID: "328e7fd3-77b8-475a-88d0-d456b0ac7b44"). InnerVolumeSpecName "kserve-provision-location". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 01 17:24:45.018238 ip-10-0-137-244 kubenswrapper[2565]: I0701 17:24:45.018214    2565 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/328e7fd3-77b8-475a-88d0-d456b0ac7b44-proxy-tls" (OuterVolumeSpecName: "proxy-tls") pod "328e7fd3-77b8-475a-88d0-d456b0ac7b44" (UID: "328e7fd3-77b8-475a-88d0-d456b0ac7b44"). InnerVolumeSpecName "proxy-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 01 17:24:45.018315 ip-10-0-137-244 kubenswrapper[2565]: I0701 17:24:45.018226    2565 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/328e7fd3-77b8-475a-88d0-d456b0ac7b44-kube-api-access-zsftp" (OuterVolumeSpecName: "kube-api-access-zsftp") pod "328e7fd3-77b8-475a-88d0-d456b0ac7b44" (UID: "328e7fd3-77b8-475a-88d0-d456b0ac7b44"). InnerVolumeSpecName "kube-api-access-zsftp". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 01 17:24:45.117514 ip-10-0-137-244 kubenswrapper[2565]: I0701 17:24:45.117464    2565 reconciler_common.go:299] "Volume detached for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/328e7fd3-77b8-475a-88d0-d456b0ac7b44-kserve-provision-location\") on node \"ip-10-0-137-244.ec2.internal\" DevicePath \"\""
Jul 01 17:24:45.117514 ip-10-0-137-244 kubenswrapper[2565]: I0701 17:24:45.117484    2565 reconciler_common.go:299] "Volume detached for volume \"cabundle-cert\" (UniqueName: \"kubernetes.io/configmap/328e7fd3-77b8-475a-88d0-d456b0ac7b44-cabundle-cert\") on node \"ip-10-0-137-244.ec2.internal\" DevicePath \"\""
Jul 01 17:24:45.117514 ip-10-0-137-244 kubenswrapper[2565]: I0701 17:24:45.117495    2565 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-zsftp\" (UniqueName: \"kubernetes.io/projected/328e7fd3-77b8-475a-88d0-d456b0ac7b44-kube-api-access-zsftp\") on node \"ip-10-0-137-244.ec2.internal\" DevicePath \"\""
Jul 01 17:24:45.117514 ip-10-0-137-244 kubenswrapper[2565]: I0701 17:24:45.117504    2565 reconciler_common.go:299] "Volume detached for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/328e7fd3-77b8-475a-88d0-d456b0ac7b44-proxy-tls\") on node \"ip-10-0-137-244.ec2.internal\" DevicePath \"\""
Jul 01 17:24:45.117514 ip-10-0-137-244 kubenswrapper[2565]: I0701 17:24:45.117512    2565 reconciler_common.go:299] "Volume detached for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/328e7fd3-77b8-475a-88d0-d456b0ac7b44-kube-rbac-proxy-sar-config\") on node \"ip-10-0-137-244.ec2.internal\" DevicePath \"\""
Jul 01 17:24:45.537971 ip-10-0-137-244 kubenswrapper[2565]: I0701 17:24:45.537938    2565 generic.go:356] "Generic (PLEG): container finished" podID="328e7fd3-77b8-475a-88d0-d456b0ac7b44" containerID="30cbb5243d597c63f91fa4f731a18f7e0cb6becf005b20696cb93a91fc155f15" exitCode=0
Jul 01 17:24:45.538406 ip-10-0-137-244 kubenswrapper[2565]: I0701 17:24:45.538007    2565 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/isvc-autogluon-ts-v1-predictor-89d67dbf7-5fqpl"
Jul 01 17:24:45.538406 ip-10-0-137-244 kubenswrapper[2565]: I0701 17:24:45.538014    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/isvc-autogluon-ts-v1-predictor-89d67dbf7-5fqpl" event={"ID":"328e7fd3-77b8-475a-88d0-d456b0ac7b44","Type":"ContainerDied","Data":"30cbb5243d597c63f91fa4f731a18f7e0cb6becf005b20696cb93a91fc155f15"}
Jul 01 17:24:45.538406 ip-10-0-137-244 kubenswrapper[2565]: I0701 17:24:45.538089    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/isvc-autogluon-ts-v1-predictor-89d67dbf7-5fqpl" event={"ID":"328e7fd3-77b8-475a-88d0-d456b0ac7b44","Type":"ContainerDied","Data":"7a5f162da12f6555e0f5863fa76952c04daa1de9d999f3d61c7831736c709141"}
Jul 01 17:24:45.538406 ip-10-0-137-244 kubenswrapper[2565]: I0701 17:24:45.538117    2565 scope.go:117] "RemoveContainer" containerID="96e5d1eb74ac875fd841e01e3ee113fc32ad62ad9283de75234c11ac2a9c5ed8"
Jul 01 17:24:45.546532 ip-10-0-137-244 kubenswrapper[2565]: I0701 17:24:45.546514    2565 scope.go:117] "RemoveContainer" containerID="30cbb5243d597c63f91fa4f731a18f7e0cb6becf005b20696cb93a91fc155f15"
Jul 01 17:24:45.553869 ip-10-0-137-244 kubenswrapper[2565]: I0701 17:24:45.553854    2565 scope.go:117] "RemoveContainer" containerID="9f80690dc5b89198c0833e8b6f2bb0802f021e1c915ce10fb47190991f055322"
Jul 01 17:24:45.560981 ip-10-0-137-244 kubenswrapper[2565]: I0701 17:24:45.560965    2565 scope.go:117] "RemoveContainer" containerID="96e5d1eb74ac875fd841e01e3ee113fc32ad62ad9283de75234c11ac2a9c5ed8"
Jul 01 17:24:45.561277 ip-10-0-137-244 kubenswrapper[2565]: E0701 17:24:45.561257    2565 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"96e5d1eb74ac875fd841e01e3ee113fc32ad62ad9283de75234c11ac2a9c5ed8\": container with ID starting with 96e5d1eb74ac875fd841e01e3ee113fc32ad62ad9283de75234c11ac2a9c5ed8 not found: ID does not exist" containerID="96e5d1eb74ac875fd841e01e3ee113fc32ad62ad9283de75234c11ac2a9c5ed8"
Jul 01 17:24:45.561358 ip-10-0-137-244 kubenswrapper[2565]: I0701 17:24:45.561287    2565 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"96e5d1eb74ac875fd841e01e3ee113fc32ad62ad9283de75234c11ac2a9c5ed8"} err="failed to get container status \"96e5d1eb74ac875fd841e01e3ee113fc32ad62ad9283de75234c11ac2a9c5ed8\": rpc error: code = NotFound desc = could not find container \"96e5d1eb74ac875fd841e01e3ee113fc32ad62ad9283de75234c11ac2a9c5ed8\": container with ID starting with 96e5d1eb74ac875fd841e01e3ee113fc32ad62ad9283de75234c11ac2a9c5ed8 not found: ID does not exist"
Jul 01 17:24:45.561358 ip-10-0-137-244 kubenswrapper[2565]: I0701 17:24:45.561311    2565 scope.go:117] "RemoveContainer" containerID="30cbb5243d597c63f91fa4f731a18f7e0cb6becf005b20696cb93a91fc155f15"
Jul 01 17:24:45.561577 ip-10-0-137-244 kubenswrapper[2565]: E0701 17:24:45.561545    2565 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"30cbb5243d597c63f91fa4f731a18f7e0cb6becf005b20696cb93a91fc155f15\": container with ID starting with 30cbb5243d597c63f91fa4f731a18f7e0cb6becf005b20696cb93a91fc155f15 not found: ID does not exist" containerID="30cbb5243d597c63f91fa4f731a18f7e0cb6becf005b20696cb93a91fc155f15"
Jul 01 17:24:45.561686 ip-10-0-137-244 kubenswrapper[2565]: I0701 17:24:45.561585    2565 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"30cbb5243d597c63f91fa4f731a18f7e0cb6becf005b20696cb93a91fc155f15"} err="failed to get container status \"30cbb5243d597c63f91fa4f731a18f7e0cb6becf005b20696cb93a91fc155f15\": rpc error: code = NotFound desc = could not find container \"30cbb5243d597c63f91fa4f731a18f7e0cb6becf005b20696cb93a91fc155f15\": container with ID starting with 30cbb5243d597c63f91fa4f731a18f7e0cb6becf005b20696cb93a91fc155f15 not found: ID does not exist"
Jul 01 17:24:45.561686 ip-10-0-137-244 kubenswrapper[2565]: I0701 17:24:45.561610    2565 scope.go:117] "RemoveContainer" containerID="9f80690dc5b89198c0833e8b6f2bb0802f021e1c915ce10fb47190991f055322"
Jul 01 17:24:45.561884 ip-10-0-137-244 kubenswrapper[2565]: E0701 17:24:45.561867    2565 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"9f80690dc5b89198c0833e8b6f2bb0802f021e1c915ce10fb47190991f055322\": container with ID starting with 9f80690dc5b89198c0833e8b6f2bb0802f021e1c915ce10fb47190991f055322 not found: ID does not exist" containerID="9f80690dc5b89198c0833e8b6f2bb0802f021e1c915ce10fb47190991f055322"
Jul 01 17:24:45.561927 ip-10-0-137-244 kubenswrapper[2565]: I0701 17:24:45.561888    2565 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"9f80690dc5b89198c0833e8b6f2bb0802f021e1c915ce10fb47190991f055322"} err="failed to get container status \"9f80690dc5b89198c0833e8b6f2bb0802f021e1c915ce10fb47190991f055322\": rpc error: code = NotFound desc = could not find container \"9f80690dc5b89198c0833e8b6f2bb0802f021e1c915ce10fb47190991f055322\": container with ID starting with 9f80690dc5b89198c0833e8b6f2bb0802f021e1c915ce10fb47190991f055322 not found: ID does not exist"
Jul 01 17:24:45.561995 ip-10-0-137-244 kubenswrapper[2565]: I0701 17:24:45.561981    2565 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kserve-ci-e2e-test/isvc-autogluon-ts-v1-predictor-89d67dbf7-5fqpl"]
Jul 01 17:24:45.567955 ip-10-0-137-244 kubenswrapper[2565]: I0701 17:24:45.567934    2565 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kserve-ci-e2e-test/isvc-autogluon-ts-v1-predictor-89d67dbf7-5fqpl"]
Jul 01 17:24:46.248642 ip-10-0-137-244 kubenswrapper[2565]: I0701 17:24:46.248605    2565 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="328e7fd3-77b8-475a-88d0-d456b0ac7b44" path="/var/lib/kubelet/pods/328e7fd3-77b8-475a-88d0-d456b0ac7b44/volumes"
Jul 01 17:25:22.791792 ip-10-0-137-244 kubenswrapper[2565]: I0701 17:25:22.791758    2565 ???:1] "http: TLS handshake error from 10.0.138.194:38778: EOF"
Jul 01 17:25:22.796939 ip-10-0-137-244 kubenswrapper[2565]: I0701 17:25:22.796912    2565 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_global-pull-secret-syncer-vfng9_b2746391-676c-4add-9406-a60838c1ac2c/global-pull-secret-syncer/0.log"
Jul 01 17:25:22.945275 ip-10-0-137-244 kubenswrapper[2565]: I0701 17:25:22.945248    2565 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_konnectivity-agent-w6h6d_adf23d12-6f62-40f2-9ca0-026c878bfe07/konnectivity-agent/0.log"
Jul 01 17:25:22.964683 ip-10-0-137-244 kubenswrapper[2565]: I0701 17:25:22.964658    2565 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_kube-apiserver-proxy-ip-10-0-137-244.ec2.internal_8ab5de924319ee4f32e67a5f2f6ee7fe/haproxy/0.log"
Jul 01 17:25:27.470534 ip-10-0-137-244 kubenswrapper[2565]: I0701 17:25:27.470500    2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_cluster-monitoring-operator-54cb84584-t2hmp_cbc9a766-e08a-4666-b4e5-f3db8dcadc45/cluster-monitoring-operator/0.log"
Jul 01 17:25:27.885407 ip-10-0-137-244 kubenswrapper[2565]: I0701 17:25:27.885382    2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-wrt9l_3e53c2ff-69b2-4ae3-a6a9-fa732a9ccc9e/node-exporter/0.log"
Jul 01 17:25:27.927356 ip-10-0-137-244 kubenswrapper[2565]: I0701 17:25:27.927334    2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-wrt9l_3e53c2ff-69b2-4ae3-a6a9-fa732a9ccc9e/kube-rbac-proxy/0.log"
Jul 01 17:25:27.957117 ip-10-0-137-244 kubenswrapper[2565]: I0701 17:25:27.957095    2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-wrt9l_3e53c2ff-69b2-4ae3-a6a9-fa732a9ccc9e/init-textfile/0.log"
Jul 01 17:25:32.459852 ip-10-0-137-244 kubenswrapper[2565]: I0701 17:25:32.459820    2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-544rz_c91603a3-7fb8-488e-905d-c2b2791f4f8c/dns/0.log"
Jul 01 17:25:32.489373 ip-10-0-137-244 kubenswrapper[2565]: I0701 17:25:32.489347    2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-544rz_c91603a3-7fb8-488e-905d-c2b2791f4f8c/kube-rbac-proxy/0.log"
Jul 01 17:25:32.601111 ip-10-0-137-244 kubenswrapper[2565]: I0701 17:25:32.601083    2565 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-must-gather-4lxfx/perf-node-gather-daemonset-sq98r"]
Jul 01 17:25:32.601378 ip-10-0-137-244 kubenswrapper[2565]: I0701 17:25:32.601364    2565 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="328e7fd3-77b8-475a-88d0-d456b0ac7b44" containerName="storage-initializer"
Jul 01 17:25:32.601431 ip-10-0-137-244 kubenswrapper[2565]: I0701 17:25:32.601379    2565 state_mem.go:107] "Deleted CPUSet assignment" podUID="328e7fd3-77b8-475a-88d0-d456b0ac7b44" containerName="storage-initializer"
Jul 01 17:25:32.601431 ip-10-0-137-244 kubenswrapper[2565]: I0701 17:25:32.601388    2565 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="328e7fd3-77b8-475a-88d0-d456b0ac7b44" containerName="kserve-container"
Jul 01 17:25:32.601431 ip-10-0-137-244 kubenswrapper[2565]: I0701 17:25:32.601394    2565 state_mem.go:107] "Deleted CPUSet assignment" podUID="328e7fd3-77b8-475a-88d0-d456b0ac7b44" containerName="kserve-container"
Jul 01 17:25:32.601431 ip-10-0-137-244 kubenswrapper[2565]: I0701 17:25:32.601405    2565 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="328e7fd3-77b8-475a-88d0-d456b0ac7b44" containerName="kube-rbac-proxy"
Jul 01 17:25:32.601431 ip-10-0-137-244 kubenswrapper[2565]: I0701 17:25:32.601411    2565 state_mem.go:107] "Deleted CPUSet assignment" podUID="328e7fd3-77b8-475a-88d0-d456b0ac7b44" containerName="kube-rbac-proxy"
Jul 01 17:25:32.601583 ip-10-0-137-244 kubenswrapper[2565]: I0701 17:25:32.601456    2565 memory_manager.go:368] "RemoveStaleState removing state" podUID="328e7fd3-77b8-475a-88d0-d456b0ac7b44" containerName="kube-rbac-proxy"
Jul 01 17:25:32.601583 ip-10-0-137-244 kubenswrapper[2565]: I0701 17:25:32.601463    2565 memory_manager.go:368] "RemoveStaleState removing state" podUID="328e7fd3-77b8-475a-88d0-d456b0ac7b44" containerName="kserve-container"
Jul 01 17:25:32.604381 ip-10-0-137-244 kubenswrapper[2565]: I0701 17:25:32.604359    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-4lxfx/perf-node-gather-daemonset-sq98r"
Jul 01 17:25:32.607002 ip-10-0-137-244 kubenswrapper[2565]: I0701 17:25:32.606982    2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-4lxfx\"/\"kube-root-ca.crt\""
Jul 01 17:25:32.607425 ip-10-0-137-244 kubenswrapper[2565]: I0701 17:25:32.607399    2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-4lxfx\"/\"openshift-service-ca.crt\""
Jul 01 17:25:32.607909 ip-10-0-137-244 kubenswrapper[2565]: I0701 17:25:32.607888    2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-must-gather-4lxfx\"/\"default-dockercfg-wjjmk\""
Jul 01 17:25:32.616651 ip-10-0-137-244 kubenswrapper[2565]: I0701 17:25:32.616624    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-4lxfx/perf-node-gather-daemonset-sq98r"]
Jul 01 17:25:32.645047 ip-10-0-137-244 kubenswrapper[2565]: I0701 17:25:32.645002    2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-mth9w_16665992-76de-4ba6-b887-abaadbc31ae6/dns-node-resolver/0.log"
Jul 01 17:25:32.750422 ip-10-0-137-244 kubenswrapper[2565]: I0701 17:25:32.750371    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/77f5f062-d1e5-4cfe-a514-5f3bc04196f3-sys\") pod \"perf-node-gather-daemonset-sq98r\" (UID: \"77f5f062-d1e5-4cfe-a514-5f3bc04196f3\") " pod="openshift-must-gather-4lxfx/perf-node-gather-daemonset-sq98r"
Jul 01 17:25:32.750422 ip-10-0-137-244 kubenswrapper[2565]: I0701 17:25:32.750400    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/77f5f062-d1e5-4cfe-a514-5f3bc04196f3-podres\") pod \"perf-node-gather-daemonset-sq98r\" (UID: \"77f5f062-d1e5-4cfe-a514-5f3bc04196f3\") " pod="openshift-must-gather-4lxfx/perf-node-gather-daemonset-sq98r"
Jul 01 17:25:32.750537 ip-10-0-137-244 kubenswrapper[2565]: I0701 17:25:32.750429    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-95shl\" (UniqueName: \"kubernetes.io/projected/77f5f062-d1e5-4cfe-a514-5f3bc04196f3-kube-api-access-95shl\") pod \"perf-node-gather-daemonset-sq98r\" (UID: \"77f5f062-d1e5-4cfe-a514-5f3bc04196f3\") " pod="openshift-must-gather-4lxfx/perf-node-gather-daemonset-sq98r"
Jul 01 17:25:32.750537 ip-10-0-137-244 kubenswrapper[2565]: I0701 17:25:32.750477    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/77f5f062-d1e5-4cfe-a514-5f3bc04196f3-lib-modules\") pod \"perf-node-gather-daemonset-sq98r\" (UID: \"77f5f062-d1e5-4cfe-a514-5f3bc04196f3\") " pod="openshift-must-gather-4lxfx/perf-node-gather-daemonset-sq98r"
Jul 01 17:25:32.750537 ip-10-0-137-244 kubenswrapper[2565]: I0701 17:25:32.750516    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/77f5f062-d1e5-4cfe-a514-5f3bc04196f3-proc\") pod \"perf-node-gather-daemonset-sq98r\" (UID: \"77f5f062-d1e5-4cfe-a514-5f3bc04196f3\") " pod="openshift-must-gather-4lxfx/perf-node-gather-daemonset-sq98r"
Jul 01 17:25:32.851542 ip-10-0-137-244 kubenswrapper[2565]: I0701 17:25:32.851520    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-95shl\" (UniqueName: \"kubernetes.io/projected/77f5f062-d1e5-4cfe-a514-5f3bc04196f3-kube-api-access-95shl\") pod \"perf-node-gather-daemonset-sq98r\" (UID: \"77f5f062-d1e5-4cfe-a514-5f3bc04196f3\") " pod="openshift-must-gather-4lxfx/perf-node-gather-daemonset-sq98r"
Jul 01 17:25:32.851628 ip-10-0-137-244 kubenswrapper[2565]: I0701 17:25:32.851549    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/77f5f062-d1e5-4cfe-a514-5f3bc04196f3-lib-modules\") pod \"perf-node-gather-daemonset-sq98r\" (UID: \"77f5f062-d1e5-4cfe-a514-5f3bc04196f3\") " pod="openshift-must-gather-4lxfx/perf-node-gather-daemonset-sq98r"
Jul 01 17:25:32.851628 ip-10-0-137-244 kubenswrapper[2565]: I0701 17:25:32.851574    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/77f5f062-d1e5-4cfe-a514-5f3bc04196f3-proc\") pod \"perf-node-gather-daemonset-sq98r\" (UID: \"77f5f062-d1e5-4cfe-a514-5f3bc04196f3\") " pod="openshift-must-gather-4lxfx/perf-node-gather-daemonset-sq98r"
Jul 01 17:25:32.851709 ip-10-0-137-244 kubenswrapper[2565]: I0701 17:25:32.851629    2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/77f5f062-d1e5-4cfe-a514-5f3bc04196f3-proc\") pod \"perf-node-gather-daemonset-sq98r\" (UID: \"77f5f062-d1e5-4cfe-a514-5f3bc04196f3\") " pod="openshift-must-gather-4lxfx/perf-node-gather-daemonset-sq98r"
Jul 01 17:25:32.851709 ip-10-0-137-244 kubenswrapper[2565]: I0701 17:25:32.851693    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/77f5f062-d1e5-4cfe-a514-5f3bc04196f3-sys\") pod \"perf-node-gather-daemonset-sq98r\" (UID: \"77f5f062-d1e5-4cfe-a514-5f3bc04196f3\") " pod="openshift-must-gather-4lxfx/perf-node-gather-daemonset-sq98r"
Jul 01 17:25:32.851774 ip-10-0-137-244 kubenswrapper[2565]: I0701 17:25:32.851708    2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/77f5f062-d1e5-4cfe-a514-5f3bc04196f3-lib-modules\") pod \"perf-node-gather-daemonset-sq98r\" (UID: \"77f5f062-d1e5-4cfe-a514-5f3bc04196f3\") " pod="openshift-must-gather-4lxfx/perf-node-gather-daemonset-sq98r"
Jul 01 17:25:32.851774 ip-10-0-137-244 kubenswrapper[2565]: I0701 17:25:32.851745    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/77f5f062-d1e5-4cfe-a514-5f3bc04196f3-podres\") pod \"perf-node-gather-daemonset-sq98r\" (UID: \"77f5f062-d1e5-4cfe-a514-5f3bc04196f3\") " pod="openshift-must-gather-4lxfx/perf-node-gather-daemonset-sq98r"
Jul 01 17:25:32.851774 ip-10-0-137-244 kubenswrapper[2565]: I0701 17:25:32.851766    2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/77f5f062-d1e5-4cfe-a514-5f3bc04196f3-sys\") pod \"perf-node-gather-daemonset-sq98r\" (UID: \"77f5f062-d1e5-4cfe-a514-5f3bc04196f3\") " pod="openshift-must-gather-4lxfx/perf-node-gather-daemonset-sq98r"
Jul 01 17:25:32.851870 ip-10-0-137-244 kubenswrapper[2565]: I0701 17:25:32.851838    2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/77f5f062-d1e5-4cfe-a514-5f3bc04196f3-podres\") pod \"perf-node-gather-daemonset-sq98r\" (UID: \"77f5f062-d1e5-4cfe-a514-5f3bc04196f3\") " pod="openshift-must-gather-4lxfx/perf-node-gather-daemonset-sq98r"
Jul 01 17:25:32.859621 ip-10-0-137-244 kubenswrapper[2565]: I0701 17:25:32.859595    2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-95shl\" (UniqueName: \"kubernetes.io/projected/77f5f062-d1e5-4cfe-a514-5f3bc04196f3-kube-api-access-95shl\") pod \"perf-node-gather-daemonset-sq98r\" (UID: \"77f5f062-d1e5-4cfe-a514-5f3bc04196f3\") " pod="openshift-must-gather-4lxfx/perf-node-gather-daemonset-sq98r"
Jul 01 17:25:32.915652 ip-10-0-137-244 kubenswrapper[2565]: I0701 17:25:32.915622    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-4lxfx/perf-node-gather-daemonset-sq98r"
Jul 01 17:25:33.041773 ip-10-0-137-244 kubenswrapper[2565]: I0701 17:25:33.041711    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-4lxfx/perf-node-gather-daemonset-sq98r"]
Jul 01 17:25:33.045180 ip-10-0-137-244 kubenswrapper[2565]: W0701 17:25:33.045155    2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-pod77f5f062_d1e5_4cfe_a514_5f3bc04196f3.slice/crio-a98287d41a79fe62b79432130b408c72661615da26746463ce34b7170e16fc04 WatchSource:0}: Error finding container a98287d41a79fe62b79432130b408c72661615da26746463ce34b7170e16fc04: Status 404 returned error can't find the container with id a98287d41a79fe62b79432130b408c72661615da26746463ce34b7170e16fc04
Jul 01 17:25:33.304323 ip-10-0-137-244 kubenswrapper[2565]: I0701 17:25:33.304273    2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-g7t25_b878d4e6-fe3b-4c52-aaea-de5a58d48afc/node-ca/0.log"
Jul 01 17:25:33.727061 ip-10-0-137-244 kubenswrapper[2565]: I0701 17:25:33.727016    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-4lxfx/perf-node-gather-daemonset-sq98r" event={"ID":"77f5f062-d1e5-4cfe-a514-5f3bc04196f3","Type":"ContainerStarted","Data":"2851d4c381ed922c0d3a50a1981e72536a5d0a831eafcbe621e341ce8c7d4342"}
Jul 01 17:25:33.727380 ip-10-0-137-244 kubenswrapper[2565]: I0701 17:25:33.727089    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-4lxfx/perf-node-gather-daemonset-sq98r" event={"ID":"77f5f062-d1e5-4cfe-a514-5f3bc04196f3","Type":"ContainerStarted","Data":"a98287d41a79fe62b79432130b408c72661615da26746463ce34b7170e16fc04"}
Jul 01 17:25:33.727380 ip-10-0-137-244 kubenswrapper[2565]: I0701 17:25:33.727126    2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-must-gather-4lxfx/perf-node-gather-daemonset-sq98r"
Jul 01 17:25:33.747517 ip-10-0-137-244 kubenswrapper[2565]: I0701 17:25:33.747470    2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-must-gather-4lxfx/perf-node-gather-daemonset-sq98r" podStartSLOduration=1.7474519800000001 podStartE2EDuration="1.74745198s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-01 17:25:32 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-01 17:25:33.74620982 +0000 UTC m=+1604.214473945" watchObservedRunningTime="2026-07-01 17:25:33.74745198 +0000 UTC m=+1604.215716104"
Jul 01 17:25:34.487035 ip-10-0-137-244 kubenswrapper[2565]: I0701 17:25:34.486985    2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-4dfnf_b433bedb-7f27-4ad0-a5ce-6d573c5aa004/serve-healthcheck-canary/0.log"
Jul 01 17:25:35.151370 ip-10-0-137-244 kubenswrapper[2565]: I0701 17:25:35.151340    2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-v5999_2824e47e-25e4-4940-8e30-5e4598a825ed/kube-rbac-proxy/0.log"
Jul 01 17:25:35.180326 ip-10-0-137-244 kubenswrapper[2565]: I0701 17:25:35.180306    2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-v5999_2824e47e-25e4-4940-8e30-5e4598a825ed/exporter/0.log"
Jul 01 17:25:35.210934 ip-10-0-137-244 kubenswrapper[2565]: I0701 17:25:35.210892    2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-v5999_2824e47e-25e4-4940-8e30-5e4598a825ed/extractor/0.log"
Jul 01 17:25:37.384692 ip-10-0-137-244 kubenswrapper[2565]: I0701 17:25:37.384658    2565 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve_kserve-controller-manager-84f4454459-47k94_b048d835-8a05-4aa0-9975-1a4a3e2d6e4c/manager/0.log"
Jul 01 17:25:37.457130 ip-10-0-137-244 kubenswrapper[2565]: I0701 17:25:37.457104    2565 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve_model-serving-api-fd65d7d6-p4xfs_3bbfca26-6773-445d-b94a-e1564ffc6cac/server/0.log"
Jul 01 17:25:38.004855 ip-10-0-137-244 kubenswrapper[2565]: I0701 17:25:38.004826    2565 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve_seaweedfs-64568bcd49-qwq8z_c64ad8c3-4f0f-4451-bfc1-dd91d59ecb24/seaweedfs/0.log"
Jul 01 17:25:38.045235 ip-10-0-137-244 kubenswrapper[2565]: I0701 17:25:38.045204    2565 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve_seaweedfs-tls-custom-8b44d767d-tbbdm_4bc26672-32e0-457f-b5b7-656bf8693a6c/seaweedfs-tls-custom/0.log"
Jul 01 17:25:39.741035 ip-10-0-137-244 kubenswrapper[2565]: I0701 17:25:39.740989    2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-must-gather-4lxfx/perf-node-gather-daemonset-sq98r"
Jul 01 17:25:42.401753 ip-10-0-137-244 kubenswrapper[2565]: I0701 17:25:42.401725    2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-kube-storage-version-migrator_migrator-786f9fbc5-lg2z5_d9f4a278-3bce-4ecd-957b-0fa58a4be3cb/migrator/0.log"
Jul 01 17:25:42.426752 ip-10-0-137-244 kubenswrapper[2565]: I0701 17:25:42.426726    2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-kube-storage-version-migrator_migrator-786f9fbc5-lg2z5_d9f4a278-3bce-4ecd-957b-0fa58a4be3cb/graceful-termination/0.log"
Jul 01 17:25:43.743613 ip-10-0-137-244 kubenswrapper[2565]: I0701 17:25:43.743586    2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-kzwpl_bffd2a49-ec4f-44ee-a47c-19180aaeeec2/kube-multus-additional-cni-plugins/0.log"
Jul 01 17:25:43.790173 ip-10-0-137-244 kubenswrapper[2565]: I0701 17:25:43.790150    2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-kzwpl_bffd2a49-ec4f-44ee-a47c-19180aaeeec2/egress-router-binary-copy/0.log"
Jul 01 17:25:43.825688 ip-10-0-137-244 kubenswrapper[2565]: I0701 17:25:43.825665    2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-kzwpl_bffd2a49-ec4f-44ee-a47c-19180aaeeec2/cni-plugins/0.log"
Jul 01 17:25:43.867888 ip-10-0-137-244 kubenswrapper[2565]: I0701 17:25:43.867835    2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-kzwpl_bffd2a49-ec4f-44ee-a47c-19180aaeeec2/bond-cni-plugin/0.log"
Jul 01 17:25:43.897194 ip-10-0-137-244 kubenswrapper[2565]: I0701 17:25:43.897174    2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-kzwpl_bffd2a49-ec4f-44ee-a47c-19180aaeeec2/routeoverride-cni/0.log"
Jul 01 17:25:43.936253 ip-10-0-137-244 kubenswrapper[2565]: I0701 17:25:43.936233    2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-kzwpl_bffd2a49-ec4f-44ee-a47c-19180aaeeec2/whereabouts-cni-bincopy/0.log"
Jul 01 17:25:43.966244 ip-10-0-137-244 kubenswrapper[2565]: I0701 17:25:43.966225    2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-kzwpl_bffd2a49-ec4f-44ee-a47c-19180aaeeec2/whereabouts-cni/0.log"
Jul 01 17:25:44.526827 ip-10-0-137-244 kubenswrapper[2565]: I0701 17:25:44.526794    2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-nlxxc_6040517d-fe23-442e-968b-b58733d448a0/kube-multus/0.log"
Jul 01 17:25:44.706182 ip-10-0-137-244 kubenswrapper[2565]: I0701 17:25:44.706155    2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_network-metrics-daemon-hthvc_a6488c7c-9ea5-4ad2-91ba-b1ffda18b203/network-metrics-daemon/0.log"
Jul 01 17:25:44.749655 ip-10-0-137-244 kubenswrapper[2565]: I0701 17:25:44.749638    2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_network-metrics-daemon-hthvc_a6488c7c-9ea5-4ad2-91ba-b1ffda18b203/kube-rbac-proxy/0.log"
Jul 01 17:25:46.233510 ip-10-0-137-244 kubenswrapper[2565]: I0701 17:25:46.233471    2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-zjsw4_bb7ebc32-d0cd-4eb0-b998-3ce298f73809/ovn-controller/0.log"
Jul 01 17:25:46.261558 ip-10-0-137-244 kubenswrapper[2565]: I0701 17:25:46.261540    2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-zjsw4_bb7ebc32-d0cd-4eb0-b998-3ce298f73809/ovn-acl-logging/1.log"
Jul 01 17:25:46.261658 ip-10-0-137-244 kubenswrapper[2565]: I0701 17:25:46.261615    2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-zjsw4_bb7ebc32-d0cd-4eb0-b998-3ce298f73809/ovn-acl-logging/0.log"
Jul 01 17:25:46.288035 ip-10-0-137-244 kubenswrapper[2565]: I0701 17:25:46.288002    2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-zjsw4_bb7ebc32-d0cd-4eb0-b998-3ce298f73809/kube-rbac-proxy-node/0.log"
Jul 01 17:25:46.318891 ip-10-0-137-244 kubenswrapper[2565]: I0701 17:25:46.318868    2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-zjsw4_bb7ebc32-d0cd-4eb0-b998-3ce298f73809/kube-rbac-proxy-ovn-metrics/0.log"
Jul 01 17:25:46.348916 ip-10-0-137-244 kubenswrapper[2565]: I0701 17:25:46.348898    2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-zjsw4_bb7ebc32-d0cd-4eb0-b998-3ce298f73809/northd/0.log"
Jul 01 17:25:46.379731 ip-10-0-137-244 kubenswrapper[2565]: I0701 17:25:46.379690    2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-zjsw4_bb7ebc32-d0cd-4eb0-b998-3ce298f73809/nbdb/0.log"
Jul 01 17:25:46.412937 ip-10-0-137-244 kubenswrapper[2565]: I0701 17:25:46.412919    2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-zjsw4_bb7ebc32-d0cd-4eb0-b998-3ce298f73809/sbdb/0.log"
Jul 01 17:25:46.475093 ip-10-0-137-244 kubenswrapper[2565]: I0701 17:25:46.475076    2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-zjsw4_bb7ebc32-d0cd-4eb0-b998-3ce298f73809/ovnkube-controller/0.log"
Jul 01 17:25:47.622474 ip-10-0-137-244 kubenswrapper[2565]: I0701 17:25:47.622447    2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-network-diagnostics_network-check-target-fvqs4_43a7325b-d5b7-492e-961d-52f0b9b32a17/network-check-target-container/0.log"
Jul 01 17:25:48.924314 ip-10-0-137-244 kubenswrapper[2565]: I0701 17:25:48.924280    2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-network-operator_iptables-alerter-r59mr_121518fd-3292-4547-bc31-921e4db77e54/iptables-alerter/0.log"